Complex Tabular Reasoning on TableBench
68.8TB-DACRAFTQA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CRAFTQAInference Engine=GPT-4o, Model Source Type=Closed-source Models2026.06 | 68.8 | — | — | — | |
| CRAFTQAInference Engine=GPT-4o-mini, Model Source Type=Closed-source Models2026.06 | 64.6 | — | — | — | |
| StructGPTInference Engine=GPT-4o2026.06 | 63.5 | — | — | — | |
| ReadiInference Engine=GPT-4o2026.06 | 62.5 | — | — | — | |
| TrustUQAInference Engine=GPT-4o2026.06 | 62.5 | — | — | — | |
| CRAFTQAInference Engine=GPT-3.5-Turbo, Model Source Type=Closed-source Models2026.06 | 61.5 | — | — | — | |
| StructGPTInference Engine=GPT-4o-mini2026.06 | 58.3 | — | — | — | |
| CRAFTQAInference Engine=Qwen2.5-7B, Model Source Type=Open-source Models2026.06 | 57.9 | — | — | — | |
| CRAFTQAInference Engine=LLaMA-3.1-8B, Model Source Type=Open-source Models2026.06 | 57.3 | — | — | — | |
| StructGPTInference Engine=GPT-3.5-Turbo2026.06 | 56.3 | — | — | — | |
| ReadiInference Engine=GPT-4o-mini2026.06 | 55.2 | — | — | — | |
| TrustUQAInference Engine=GPT-4o-mini2026.06 | 55.2 | — | — | — | |
| PoTInference Engine=GPT-3.5-Turbo2026.06 | 54.2 | — | — | — | |
| CRAFTQAInference Engine=GPT-4o, Model Source Type=Closed-source Models2026.06 | 51.3 | — | — | — | |
| PoTInference Engine=GPT-4o2026.06 | 51 | — | — | — | |
| ReadiInference Engine=GPT-3.5-Turbo2026.06 | 51 | — | — | — | |
| TrustUQAInference Engine=GPT-3.5-Turbo2026.06 | 50 | — | — | — | |
| ReadiInference Engine=GPT-4o2026.06 | 49.5 | — | — | — | |
| PoTInference Engine=GPT-4o2026.06 | 43.7 | — | — | — | |
| StructGPTInference Engine=GPT-4o2026.06 | 42.2 | — | — | — | |
| PoTInference Engine=GPT-4o-mini2026.06 | 41.7 | — | — | — | |
| CRAFTQAInference Engine=GPT-4o-mini, Model Source Type=Closed-source Models2026.06 | 40.7 | — | — | — | |
| CRAFTQAInference Engine=GPT-3.5-Turbo, Model Source Type=Closed-source Models2026.06 | 38.6 | — | — | — | |
| ReadiInference Engine=GPT-4o-mini2026.06 | 38.4 | — | — | — | |
| PoTInference Engine=GPT-4o-mini2026.06 | 36.9 | — | — | — | |
| PoTInference Engine=GPT-3.5-Turbo2026.06 | 36.4 | — | — | — | |
| CRAFTQAInference Engine=Qwen2.5-7B, Model Source Type=Open-source Models2026.06 | 35.4 | — | — | — | |
| ReadiInference Engine=GPT-3.5-Turbo2026.06 | 33.6 | — | — | — | |
| CRAFTQAInference Engine=LLaMA-3.1-8B, Model Source Type=Open-source Models2026.06 | 31.1 | — | — | — | |
| TrustUQAInference Engine=GPT-4o2026.06 | 29.6 | — | — | — | |
| StructGPTInference Engine=GPT-4o-mini2026.06 | 24.2 | — | — | — | |
| StructGPTInference Engine=GPT-3.5-Turbo2026.06 | 23 | — | — | — | |
| TrustUQAInference Engine=GPT-4o-mini2026.06 | 21.7 | — | — | — | |
| TrustUQAInference Engine=GPT-3.5-Turbo2026.06 | 20.2 | — | — | — | |
| TabTrim-8BMethod Category=Ours, Backbone/Model=TabTrim-8B, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.347 | 0.788 | 0.833 | 0.735 | |
| Table-CriticMethod Category=Critique methods, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.338 | 0.73 | 0.813 | 0.703 | |
| TabTrim-4BMethod Category=Ours, Backbone/Model=TabTrim-4B, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.321 | 0.763 | 0.792 | 0.708 | |
| Chain-of-TableMethod Category=LLM-based methods, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.303 | 0.685 | 0.781 | 0.667 | |
| TALONMethod Category=Critique methods, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.289 | 0.673 | 0.771 | 0.663 | |
| DaterMethod Category=LLM-based methods, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.286 | 0.65 | 0.698 | 0.626 | |
| TabSQLifyMethod Category=Program-based methods, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.28 | 0.652 | 0.76 | 0.632 | |
| BinderMethod Category=Program-based methods, Pipeline Reasoner=GPT-4o-mini2026.01 | 0.268 | 0.668 | 0.677 | 0.599 | |
| GPT-4o-miniMethod Category=Direct QA, Backbone/Model=GPT-4o-mini2026.01 | 0.251 | 0.655 | 0.76 | 0.598 | |
| Qwen3-8BMethod Category=Direct QA, Backbone/Model=Qwen3-8B2026.01 | 0.23 | 0.645 | 0.729 | 0.579 | |
| Qwen3-4BMethod Category=Direct QA, Backbone/Model=Qwen3-4B2026.01 | 0.192 | 0.617 | 0.708 | 0.553 | |
| CRAFTQAInference Engine=Qwen2.5-7B, Model Source Type=Open-source Models2026.06 | — | — | — | 58.3 | |
| CRAFTQAInference Engine=LLaMA-3.1-8B, Model Source Type=Open-source Models2026.06 | — | — | — | 64.4 | |
| CRAFTQAInference Engine=GPT-3.5-Turbo, Model Source Type=Closed-source Models2026.06 | — | — | — | 60.6 | |
| CRAFTQAInference Engine=GPT-4o-mini, Model Source Type=Closed-source Models2026.06 | — | — | — | 69.2 | |
| CRAFTQAInference Engine=GPT-4o, Model Source Type=Closed-source Models2026.06 | — | — | — | 76.6 | |
| PoTInference Engine=GPT-3.5-Turbo2026.06 | — | — | — | 29.4 | |
| PoTInference Engine=GPT-4o-mini2026.06 | — | — | — | 28.8 | |
| PoTInference Engine=GPT-4o2026.06 | — | — | — | 32.6 | |
| ReadiInference Engine=GPT-3.5-Turbo2026.06 | — | — | — | 42.7 | |
| ReadiInference Engine=GPT-4o-mini2026.06 | — | — | — | 38.7 | |
| ReadiInference Engine=GPT-4o2026.06 | — | — | — | 51.5 | |
| StructGPTInference Engine=GPT-3.5-Turbo2026.06 | — | — | — | 39.7 | |
| StructGPTInference Engine=GPT-4o-mini2026.06 | — | — | — | 36.5 | |
| StructGPTInference Engine=GPT-4o2026.06 | — | — | — | 44.3 | |
| TrustUQAInference Engine=GPT-3.5-Turbo2026.06 | — | — | — | 57.1 | |
| TrustUQAInference Engine=GPT-4o-mini2026.06 | — | — | — | 61 | |
| TrustUQAInference Engine=GPT-4o2026.06 | — | — | — | 67.2 |