Open Table-and-Text Question Answering on OTT-QA 1.0 (dev) benchmark leaderboard · SOTA2 Research

Open Table-and-Text Question Answering on OTT-QA 1.0 (dev)

56.9EMCOS

Evaluation Results

Method
COS2023.05	56.9	63.2
CORE2023.05	51.4	57.8
CORE2023.05	49	55.7
Oracle Table + Link (w/o Retrieval) + HYBRIDER2020.10	44.1	50.8
Oracle Table + Link (w/o Retrieval) + HYBRIDER2020.10	44.1	50.8
OTTer2023.05	37.1	42.8
Oracle Link + Fusion-Retrieval + Cross-Block Reader2020.10	35.8	40.1
Oracle Link + Fusion-Retrieval + Cross-Block Reader2020.10	35.2	39.9
CARP2023.05	33.2	38.6
Fusion-Retrieval + Cross-Block Reader2020.10	28.1	32.5
FR+CBR2023.05	28.1	32.5
Fusion-Retrieval + Cross-Block Reader2020.10	27.7	31.8
Iterative-Retrieval + Cross-Block Reader2020.10	17.1	20.7
Iterative-Retrieval + Cross-Block Reader2020.10	14.4	18.5
Fusion-Retrieval + Single-Block Reader2020.10	14.3	17.8
Fusion-Retrieval + Single-Block Reader2020.10	13.8	17.2
HYBRIDER (best Top-K)2020.10	10.3	13
HYBRIDER2023.05	10.3	13
HYBRIDER (best Top-K)2020.10	9.9	12.2
Iterative-Retrieval + Single-Block Reader2020.10	9.8	13.3
HYBRIDER (Top-1)2020.10	8.9	11.3
Text-only Retrieval + Cross-Block Reader2020.10	8.9	12.8
HYBRIDER (Top-1)2020.10	8.7	10.9
Text-only Retrieval + Cross-Block Reader2020.10	8.2	12.4
Iterative-Retrieval + Single-Block Reader2020.10	7.9	11.1
Table-only Retrieval + Cross-Block Reader2020.10	4.9	7.2
Table-only Retrieval + Cross-Block Reader2020.10	4.6	6.9