Table Question Answering on WikiSQL (test)
95.1AccuracySYNTQA (Oracle)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SYNTQA (Oracle)Model Category=Ensemble, Selector=Oracle2024.09 | 95.1 | — | |
| SYNTQA (RF)Model Category=Ensemble, Selector=Random Forest2024.09 | 93.2 | — | |
| SYNTQA (GPT)Model Category=Ensemble, Selector=GPT-3.52024.09 | 93 | — | |
| TaPExSelector=PieTa2024.12 | 91.83 | — | |
| OmniTabSelector=PieTa2024.12 | 91.17 | — | |
| RCI2024.04 | 89.8 | — | |
| OmniTabSelector=ITR2024.12 | 89.71 | — | |
| T5 + TC + PModel Category=Text-to-SQL, Table Content=true, Post-processing=true2024.09 | 89.6 | — | |
| CABINET# params=560 M2024.02 | 89.5 | — | |
| TAPEXModel Category=E2E TQA2024.09 | 89.5 | — | |
| TaPExSelector=Holistic2024.12 | 89.31 | — | |
| CABINETSelector=Holistic2024.12 | 89.17 | — | |
| TaPExSelector=ITR2024.12 | 89.09 | — | |
| OmniTabModel Category=E2E TQA2024.09 | 89 | — | |
| ReasTAP# params=406 M, Evaluation Protocol=Fine-tuning Table-specific LLMs2024.02 | 88.8 | — | |
| OmniTabf-shot=full2022.07 | 88.7 | — | |
| OmniTabSelector=Holistic2024.12 | 88.42 | — | |
| TAPEX*f-shot=full2022.07 | 88.1 | — | |
| UTP# params=345 M, Evaluation Protocol=Fine-tuning Table-specific LLMs2024.02 | 88.1 | — | |
| OmniTab# params=406 M, Evaluation Protocol=Fine-tuning Table-specific LLMs2024.02 | 87.9 | — | |
| FlanT5-xl# params=2.9 B, Evaluation Protocol=Fine-tuning text-based LLMs2024.02 | 87.8 | — | |
| TAPAS# params=345 M, Evaluation Protocol=Fine-tuning Table-specific LLMs2024.02 | 86.4 | — | |
| TAPEX# params=406 M, Evaluation Protocol=Fine-tuning Table-specific LLMs2024.02 | 86.4 | — | |
| T5-3b# params=2.9 B, Evaluation Protocol=Fine-tuning text-based LLMs2024.02 | 85.9 | — | |
| DoT# params=299 M, Evaluation Protocol=Fine-tuning Table-specific LLMs2024.02 | 85.5 | — | |
| GraPPa# params=355 M, Evaluation Protocol=Fine-tuning Table-specific LLMs2024.02 | 84.7 | — | |
| OmniTabf-shot=10242022.07 | 82.9 | — | |
| GPT + TC + PModel Category=Text-to-SQL, Table Content=true, Post-processing=true2024.09 | 82.2 | — | |
| Structured-Attention ParserEnsemble size=52019.09 | 81.7 | — | |
| DIN-SQLModel Category=Text-to-SQL2024.09 | 81.7 | — | |
| OmniTabSelector=TabSQLify2024.12 | 81.05 | — | |
| TaPExSelector=TabSQLify2024.12 | 80.68 | — | |
| Agarwal et al. (2019)Ensemble size=52019.09 | 76.9 | — | |
| TabSQLifymode=col+row2024.04 | 76.7 | — | |
| TAPEX*f-shot=10242022.07 | 75.6 | — | |
| OmniTabf-shot=1282022.07 | 75.6 | — | |
| GPT-3.5Selector=PieTa2024.12 | 75.49 | — | |
| Liang et al. (2018)Ensemble size=52019.09 | 74.2 | — | |
| GPT-3.5^TabSelector=TabSQLify2024.12 | 70.25 | — | |
| GPT-3.5Selector=TabSQLify2024.12 | 69.58 | — | |
| GPT-3.5Selector=Holistic2024.12 | 69.5 | — | |
| TAMAbackbone=LLaMA 3.1 8B Instruct2025.01 | 68.31 | — | |
| GPT-3.5Selector=ITR2024.12 | 67.6 | — | |
| StructGPT2024.04 | 65.6 | — | |
| StructGPTSelector=Holistic2024.12 | 65.6 | — | |
| TAPEX*f-shot=1282022.07 | 63.6 | — | |
| OmniTabf-shot=162022.07 | 63.6 | — | |
| GPTModel Category=E2E TQA2024.09 | 62.6 | — | |
| SEQ2SQL2024.04 | 59.4 | — | |
| StructGPT# params=175 B, Evaluation Protocol=Few/zero shot Prompting of LLMs2024.02 | 54.4 | — | |
| ChatGPT# params=175 B, Evaluation Protocol=Few/zero shot Prompting of LLMs2024.02 | 51.6 | — | |
| GPT-42025.01 | 47.6 | — | |
| TAPEX*f-shot=162022.07 | 43.4 | — | |
| GPT-3.52025.01 | 41.91 | — | |
| LLaMA 3.1 8B Instructstatus=base model2025.01 | 20.43 | — | |
| CABINETSelector=Holistic2024.12 | — | 89.17 | |
| FRESBase Model=Pixtral 12B2025.05 | — | 61.4 | |
| FRESBase Model=TableLlaVA 7B2025.05 | — | 48.4 | |
| GPT-3.5Selector=Holistic2024.12 | — | 69.5 | |
| GPT-3.5Selector=ITR2024.12 | — | 67.6 | |
| GPT-3.5Selector=TabSQLify2024.12 | — | 69.58 | |
| GPT-3.5Selector=PieTa2024.12 | — | 75.49 | |
| GPT-3.5TabSelector=TabSQLify2024.12 | — | 70.25 | |
| OmniTabSelector=Holistic2024.12 | — | 88.42 | |
| OmniTabSelector=ITR2024.12 | — | 89.71 | |
| OmniTabSelector=TabSQLify2024.12 | — | 81.05 | |
| OmniTabSelector=PieTa2024.12 | — | 91.17 | |
| PixtralModel Size=12B, Representation=text2025.05 | — | 58.8 | |
| PixtralModel Size=12B, Representation=image2025.05 | — | 51.7 | |
| PixtralModel Size=12B, Representation=text and image2025.05 | — | 60.1 | |
| StructGPTSelector=Holistic2024.12 | — | 65.6 | |
| TableLlaVAModel Size=7B, Representation=text2025.05 | — | 47.8 | |
| TableLlaVAModel Size=7B, Representation=image2025.05 | — | 31.6 | |
| TableLlaVAModel Size=7B, Representation=text and image2025.05 | — | 29.5 | |
| TaPExSelector=Holistic2024.12 | — | 89.31 | |
| TaPExSelector=ITR2024.12 | — | 89.09 | |
| TaPExSelector=TabSQLify2024.12 | — | 80.68 | |
| TaPExSelector=PieTa2024.12 | — | 91.83 |