Table Question Answering on WikiTableQuestions (dev)
64.8AccuracyCodex w/ DATER
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Codex w/ DATEREvaluation Protocol=LLM based2023.01 | 64.8 | — | |
| BinderEvaluation Protocol=LLM based2023.01 | 62.6 | — | |
| OmniTab w/ DATEREvaluation Protocol=Fine-tuning2023.01 | 62.5 | — | |
| OmniTabEvaluation Protocol=Fine-tuning2023.01 | 61.3 | — | |
| TaCubeEvaluation Protocol=Fine-tuning2023.01 | 60.9 | — | |
| ReasTAPEvaluation Protocol=Fine-tuning2023.01 | 58.3 | — | |
| TAPEXEvaluation Protocol=Fine-tuning2023.01 | 58 | — | |
| Yin et al. (2020b)2020.09 | 52.2 | — | |
| Wang et al. (2019) + GRAPPA (MLM+SSP)Encoder=GRAPPA, Pre-training objective=MLM+SSP2020.09 | 51.9 | — | |
| Wang et al. (2019) + GRAPPA (MLM)Encoder=GRAPPA, Pre-training objective=MLM2020.09 | 51.5 | — | |
| TableFormerEvaluation Protocol=Fine-tuning2023.01 | 51.3 | — | |
| Wang et al. (2019) + GRAPPA (SSP)Encoder=GRAPPA, Pre-training objective=SSP2020.09 | 51.2 | — | |
| Wang et al. (2019) + RoBERTa-largeEncoder=RoBERTa-large2020.09 | 50.7 | — | |
| TaPasEvaluation Protocol=Fine-tuning2023.01 | 49.9 | — | |
| CodexEvaluation Protocol=LLM based2023.01 | 49.3 | — | |
| Wang et al. (2019)2020.09 | 43.7 | — | |
| LatentAlignmentEvaluation Protocol=Fine-tuning2023.01 | 43.7 | — | |
| Agarwal et al. (2019)2020.09 | 43.2 | — | |
| MeRLEvaluation Protocol=Fine-tuning2023.01 | 43.2 | — | |
| IterativeSearchEvaluation Protocol=Fine-tuning2023.01 | 43.1 | — | |
| MAPOEvaluation Protocol=Fine-tuning2023.01 | 42.7 | — | |
| Liang et al. (2018)2020.09 | 42.3 | — | |
| Dasigi et al. (2019)2020.09 | 42.1 | — | |
| Wang et al. (2019) + GRAPPA (MLM+SSP) (10% data)Encoder=GRAPPA, Pre-training objective=MLM+SSP, Training data percentage=10%2020.09 | 40.4 | — | |
| Wang et al. (2019) + RoBERTa-large (10% data)Encoder=RoBERTa-large, Training data percentage=10%2020.09 | 37.3 | — | |
| Agarwal et al. (2019)Category=Previous Systems2021.07 | — | 43.2 | |
| BARTCategory=Pre-trained Language Models2021.07 | — | 37.2 | |
| Dasigi et al. (2019)Category=Previous Systems2021.07 | — | 43.1 | |
| Liang et al. (2018)Category=Previous Systems2021.07 | — | 42.7 | |
| Neelakantan et al. (2016)Category=Previous Systems2021.07 | — | 34.1 | |
| Pasupat & Liang (2015)Category=Previous Systems2021.07 | — | 37 | |
| TABLEFORMERModel Size=BASE2022.03 | — | 34.4 | |
| TABLEFORMERModel Size=LARGE2022.03 | — | 42.5 | |
| TABLEFORMERModel Size=BASE, Intermediate training=inter-sqa2022.03 | — | 46.7 | |
| TABLEFORMERModel Size=LARGE, Intermediate training=inter-sqa2022.03 | — | 51.3 | |
| TAPASModel Size=BASE2022.03 | — | 23.6 | |
| TAPASModel Size=LARGE2022.03 | — | 40.8 | |
| TAPASModel Size=BASE, Intermediate training=inter-sqa2022.03 | — | 44.8 | |
| TAPASModel Size=LARGE, Intermediate training=inter-sqa2022.03 | — | 49.9 | |
| TAPEXCategory=Pre-trained Language Models2021.07 | — | 57 | |
| Wang et al. (2019b)Category=Previous Systems2021.07 | — | 43.7 | |
| Yin et al. (2020)Category=Pre-trained Language Models2021.07 | — | 53 | |
| Yu et al. (2021a)Category=Pre-trained Language Models2021.07 | — | 51.9 | |
| Zhang et al. (2017)Category=Previous Systems2021.07 | — | 40.6 |