Table Question Answering on WikiTQ (dev)
87.8Denotation AccCRAFT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CRAFTBackbone LLM=GPT-5-mini2026.06 | 87.8 | — | — | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=GPT-5-mini2026.06 | 87 | — | — | |
| Table-CriticBackbone LLM=GPT-5-mini2026.06 | 86.7 | — | — | |
| Chain-of-TableBackbone LLM=GPT-5-mini2026.06 | 86.2 | — | — | |
| Critic-CoTBackbone LLM=GPT-5-mini2026.06 | 86.1 | — | — | |
| Few-ShotBackbone LLM=GPT-5-mini2026.06 | 85.7 | — | — | |
| End-to-EndBackbone LLM=GPT-5-mini2026.06 | 85.2 | — | — | |
| DaterBackbone LLM=GPT-5-mini2026.06 | 84.9 | — | — | |
| BinderBackbone LLM=GPT-5-mini2026.06 | 84.2 | — | — | |
| CRAFTBackbone LLM=Average2026.06 | 82.4 | — | — | |
| CRAFTBackbone LLM=Llama3.3-70B2026.06 | 81.4 | — | — | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Average2026.06 | 80.7 | — | — | |
| CRAFTBackbone LLM=Qwen2.5-72B2026.06 | 80.4 | — | — | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Llama3.3-70B2026.06 | 79.9 | — | — | |
| CRAFTBackbone LLM=Deepseek-R1-14B2026.06 | 79.9 | — | — | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Qwen2.5-72B2026.06 | 78.7 | — | — | |
| Table-CriticBackbone LLM=Qwen2.5-72B2026.06 | 77.7 | — | — | |
| Table-CriticBackbone LLM=Average2026.06 | 77.7 | — | — | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Deepseek-R1-14B2026.06 | 77.2 | — | — | |
| Table-CriticBackbone LLM=Deepseek-R1-14B2026.06 | 76.3 | — | — | |
| Critic-CoTBackbone LLM=Deepseek-R1-14B2026.06 | 75.2 | — | — | |
| Chain-of-TableBackbone LLM=Deepseek-R1-14B2026.06 | 75.1 | — | — | |
| Critic-CoTBackbone LLM=Average2026.06 | 74.3 | — | — | |
| Chain-of-TableBackbone LLM=Average2026.06 | 73 | — | — | |
| DaterBackbone LLM=Deepseek-R1-14B2026.06 | 71.6 | — | — | |
| DaterBackbone LLM=Average2026.06 | 70.9 | — | — | |
| AlterBackbone LLM=Qwen2.5-72B2026.06 | 70.9 | — | — | |
| Table-CriticBackbone LLM=Llama3.3-70B2026.06 | 70.1 | — | — | |
| Few-ShotBackbone LLM=Deepseek-R1-14B2026.06 | 70 | — | — | |
| Few-ShotBackbone LLM=Average2026.06 | 69.6 | — | — | |
| BinderBackbone LLM=Average2026.06 | 69 | — | — | |
| Critic-CoTBackbone LLM=Qwen2.5-72B2026.06 | 69 | — | — | |
| AlterBackbone LLM=Average2026.06 | 68.7 | — | — | |
| BinderBackbone LLM=Deepseek-R1-14B2026.06 | 68.5 | — | — | |
| Chain-of-TableBackbone LLM=Qwen2.5-72B2026.06 | 68.3 | — | — | |
| AlterBackbone LLM=Llama3.3-70B2026.06 | 67.8 | — | — | |
| AlterBackbone LLM=Deepseek-R1-14B2026.06 | 67.4 | — | — | |
| Critic-CoTBackbone LLM=Llama3.3-70B2026.06 | 66.8 | — | — | |
| BinderBackbone LLM=Qwen2.5-72B2026.06 | 65.5 | — | — | |
| DaterBackbone LLM=Qwen2.5-72B2026.06 | 63.8 | — | — | |
| DaterBackbone LLM=Llama3.3-70B2026.06 | 63.3 | — | — | |
| Chain-of-TableBackbone LLM=Llama3.3-70B2026.06 | 62.1 | — | — | |
| End-to-EndBackbone LLM=Average2026.06 | 62 | — | — | |
| Few-ShotBackbone LLM=Llama3.3-70B2026.06 | 62 | — | — | |
| Few-ShotBackbone LLM=Qwen2.5-72B2026.06 | 60.6 | — | — | |
| REASTAPBackbone=BART-Large2022.10 | 58.3 | — | — | |
| TAPEX2022.10 | 58 | — | — | |
| BinderBackbone LLM=Llama3.3-70B2026.06 | 57.6 | — | — | |
| End-to-EndBackbone LLM=Qwen2.5-72B2026.06 | 57.3 | — | — | |
| End-to-EndBackbone LLM=Deepseek-R1-14B2026.06 | 54.3 | — | — | |
| TaBERT2022.10 | 53 | — | — | |
| GraPPa2022.10 | 51.9 | — | — | |
| TableFormer2022.10 | 51.3 | — | — | |
| End-to-EndBackbone LLM=Llama3.3-70B2026.06 | 51.1 | — | — | |
| T5-3BMode=finetune2022.01 | 50.65 | — | — | |
| TaPas2022.10 | 49.9 | — | — | |
| LatentAlignment2022.10 | 43.7 | — | — | |
| MeRL2022.10 | 43.2 | — | — | |
| IterativeSearch2022.10 | 43.1 | — | — | |
| MAPO2022.10 | 42.7 | — | — | |
| MacroGrammer2022.10 | 40.6 | — | — | |
| BARTBackbone=BART-Large2022.10 | 37.1 | — | — | |
| GPT-3 175BFew-shot selection strategy=select2022.01 | 32 | — | — | |
| GPT-3 175BFew-shot selection strategy=random2022.01 | 29.33 | — | — | |
| Codex 175BFew-shot selection strategy=select2022.01 | 26.21 | — | — | |
| Codex 175BFew-shot selection strategy=random2022.01 | 20.46 | — | — | |
| T0 3BMode=zero-shot2022.01 | 12.68 | — | — | |
| Codex BinderFinetuning status=No2023.02 | — | — | 65 | |
| Codex QA*Finetuning status=No, Modeling approach=end-to-end QA2023.02 | — | — | 50.5 | |
| Codex SQLFinetuning status=No2023.02 | — | — | 60.2 | |
| DaterBase LLM=GPT-3.52025.06 | — | 58.4 | — | |
| Dater + CoREBase LLM=GPT-3.52025.06 | — | 63.5 | — | |
| EP + MLBase Model=code-davinci-0022023.02 | — | — | 52.7 | |
| GreedyBase Model=code-davinci-0022023.02 | — | — | 49.6 | |
| LEVERBase Model=code-davinci-0022023.02 | — | — | 64.6 | |
| StructGPTBase LLM=GPT-3.52025.06 | — | 64.4 | — | |
| StructGPT + CoREBase LLM=GPT-3.52025.06 | — | 66.1 | — | |
| TaCubeFinetuning status=Yes2023.02 | — | — | 61.1 | |
| TaPEX*Finetuning status=Yes, Modeling approach=end-to-end QA2023.02 | — | — | 60.4 |