Open Table-and-Text Question Answering on OTT-QA 1.0 (dev)
56.9EMCOS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| COSReader=Fusion-in-Encoder (FiE)2023.05 | 56.9 | 63.2 | |
| COREReader=Fusion-in-Encoder (FiE)2023.05 | 51.4 | 57.8 | |
| CORE2023.05 | 49 | 55.7 | |
| Oracle Table + Link (w/o Retrieval) + HYBRIDERRetriever=Sparse, Ablation=Oracle Table + Link2020.10 | 44.1 | 50.8 | |
| Oracle Table + Link (w/o Retrieval) + HYBRIDERRetriever=Dense, Ablation=Oracle Table + Link2020.10 | 44.1 | 50.8 | |
| OTTer2023.05 | 37.1 | 42.8 | |
| Oracle Link + Fusion-Retrieval + Cross-Block ReaderRetriever=Sparse, Ablation=Oracle Link2020.10 | 35.8 | 40.1 | |
| Oracle Link + Fusion-Retrieval + Cross-Block ReaderRetriever=Dense, Ablation=Oracle Link2020.10 | 35.2 | 39.9 | |
| CARP2023.05 | 33.2 | 38.6 | |
| Fusion-Retrieval + Cross-Block ReaderRetriever=Dense2020.10 | 28.1 | 32.5 | |
| FR+CBR2023.05 | 28.1 | 32.5 | |
| Fusion-Retrieval + Cross-Block ReaderRetriever=Sparse2020.10 | 27.7 | 31.8 | |
| Iterative-Retrieval + Cross-Block ReaderRetriever=Sparse2020.10 | 17.1 | 20.7 | |
| Iterative-Retrieval + Cross-Block ReaderRetriever=Dense2020.10 | 14.4 | 18.5 | |
| Fusion-Retrieval + Single-Block ReaderRetriever=Sparse2020.10 | 14.3 | 17.8 | |
| Fusion-Retrieval + Single-Block ReaderRetriever=Dense2020.10 | 13.8 | 17.2 | |
| HYBRIDER (best Top-K)Retriever=Dense2020.10 | 10.3 | 13 | |
| HYBRIDER2023.05 | 10.3 | 13 | |
| HYBRIDER (best Top-K)Retriever=Sparse2020.10 | 9.9 | 12.2 | |
| Iterative-Retrieval + Single-Block ReaderRetriever=Sparse2020.10 | 9.8 | 13.3 | |
| HYBRIDER (Top-1)Retriever=Dense2020.10 | 8.9 | 11.3 | |
| Text-only Retrieval + Cross-Block ReaderRetriever=Dense, Ablation=Text-only2020.10 | 8.9 | 12.8 | |
| HYBRIDER (Top-1)Retriever=Sparse2020.10 | 8.7 | 10.9 | |
| Text-only Retrieval + Cross-Block ReaderRetriever=Sparse, Ablation=Text-only2020.10 | 8.2 | 12.4 | |
| Iterative-Retrieval + Single-Block ReaderRetriever=Dense2020.10 | 7.9 | 11.1 | |
| Table-only Retrieval + Cross-Block ReaderRetriever=Dense, Ablation=Table-only2020.10 | 4.9 | 7.2 | |
| Table-only Retrieval + Cross-Block ReaderRetriever=Sparse, Ablation=Table-only2020.10 | 4.6 | 6.9 |