Table-based Fact Verification on TabFact
96.7AccuracyCRAFT w/ Chain-of-Table Extractor
Evaluation results
| Method | Links | |
|---|---|---|
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=GPT-5-mini2026.06 | 96.7 | |
| CRAFTBackbone LLM=GPT-5-mini2026.06 | 96.2 | |
| Chain-of-TableBackbone LLM=GPT-5-mini2026.06 | 96 | |
| DaterBackbone LLM=GPT-5-mini2026.06 | 95.9 | |
| Table-CriticBackbone LLM=GPT-5-mini2026.06 | 95.5 | |
| Few-ShotBackbone LLM=GPT-5-mini2026.06 | 95.1 | |
| Critic-CoTBackbone LLM=GPT-5-mini2026.06 | 94.9 | |
| End-to-EndBackbone LLM=GPT-5-mini2026.06 | 94.7 | |
| AlterBackbone LLM=GPT-5-mini2026.06 | 94.6 | |
| CRAFTBackbone LLM=Average2026.06 | 94.6 | |
| BinderBackbone LLM=GPT-5-mini2026.06 | 94.5 | |
| CRAFTBackbone LLM=Deepseek-R1-14B2026.06 | 94.2 | |
| CRAFTBackbone LLM=Llama3.3-70B2026.06 | 94.1 | |
| CRAFTBackbone LLM=Qwen2.5-72B2026.06 | 93.9 | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Deepseek-R1-14B2026.06 | 93.6 | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Average2026.06 | 93.6 | |
| Table-CriticBackbone LLM=Average2026.06 | 93.5 | |
| Table-CriticBackbone LLM=Deepseek-R1-14B2026.06 | 93.3 | |
| Table-CriticBackbone LLM=Qwen2.5-72B2026.06 | 92.9 | |
| Chain-of-TableBackbone LLM=Deepseek-R1-14B2026.06 | 92.7 | |
| Critic-CoTBackbone LLM=Deepseek-R1-14B2026.06 | 92.5 | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Qwen2.5-72B2026.06 | 92.2 | |
| DaterBackbone LLM=Deepseek-R1-14B2026.06 | 92.1 | |
| Chain-of-TableBackbone LLM=Average2026.06 | 92.1 | |
| Table-CriticBackbone LLM=Llama3.3-70B2026.06 | 92.1 | |
| TabClawCategory=Ours, Backbone=DeepSeek-v3.22026.06 | 92.1 | |
| CRAFT w/ Chain-of-Table ExtractorBackbone LLM=Llama3.3-70B2026.06 | 92 | |
| DaterBackbone LLM=Average2026.06 | 91.9 | |
| End-to-EndBackbone LLM=Deepseek-R1-14B2026.06 | 91.8 | |
| Few-ShotBackbone LLM=Deepseek-R1-14B2026.06 | 91.3 | |
| Critic-CoTBackbone LLM=Average2026.06 | 91.3 | |
| AlterBackbone LLM=Average2026.06 | 90.6 | |
| AlterBackbone LLM=Deepseek-R1-14B2026.06 | 90.5 | |
| DaterBackbone LLM=Llama3.3-70B2026.06 | 89.9 | |
| Chain-of-TableBackbone LLM=Llama3.3-70B2026.06 | 89.9 | |
| Critic-CoTBackbone LLM=Qwen2.5-72B2026.06 | 89.8 | |
| AlterBackbone LLM=Llama3.3-70B2026.06 | 89.8 | |
| DaterBackbone LLM=Qwen2.5-72B2026.06 | 89.7 | |
| Chain-of-TableBackbone LLM=Qwen2.5-72B2026.06 | 89.7 | |
| BinderBackbone LLM=Deepseek-R1-14B2026.06 | 89.5 | |
| End-to-EndBackbone LLM=Average2026.06 | 88.9 | |
| Few-ShotBackbone LLM=Average2026.06 | 88.9 | |
| Critic-CoTBackbone LLM=Llama3.3-70B2026.06 | 88 | |
| ReActCategory=Autonomous Agent, Backbone=DeepSeek-v3.22026.06 | 87.9 | |
| AlterBackbone LLM=Qwen2.5-72B2026.06 | 87.4 | |
| BinderBackbone LLM=Average2026.06 | 87 | |
| SheetAgentCategory=Autonomous Agent, Backbone=DeepSeek-v3.22026.06 | 86.8 | |
| Few-ShotBackbone LLM=Qwen2.5-72B2026.06 | 86 | |
| End-to-EndBackbone LLM=Qwen2.5-72B2026.06 | 85.6 | |
| BinderCategory=Context Augmentation, Backbone=DeepSeek-v3.22026.06 | 85.4 | |
| BinderBackbone LLM=Qwen2.5-72B2026.06 | 84 | |
| End-to-EndBackbone LLM=Llama3.3-70B2026.06 | 83.4 | |
| Few-ShotBackbone LLM=Llama3.3-70B2026.06 | 83.1 | |
| Chain-of-TableCategory=Context Augmentation, Backbone=DeepSeek-v3.22026.06 | 82.8 | |
| CFMSBackbone=GPT 3.52026.04 | 81.37 | |
| Chain-of-Table2024.06 | 80.2 | |
| Chain-of-TableBackbone=GPT 3.52026.04 | 80.2 | |
| BinderBackbone LLM=Llama3.3-70B2026.06 | 79.9 | |
| SpreadsheetLLMCategory=Context Augmentation, Backbone=DeepSeek-v3.22026.06 | 79.2 | |
| BINDER2024.06 | 79.17 | |
| BinderBackbone=GPT 3.52026.04 | 79.17 | |
| DATER2024.06 | 78.01 | |
| DaterBackbone=GPT 3.52026.04 | 78.01 | |
| NormTabCategory=Context Augmentation, Backbone=DeepSeek-v3.22026.06 | 75.7 | |
| VanillaCategory=Direct Inference, Backbone=DeepSeek-v3.22026.06 | 73.4 | |
| TableCoT-chatgptModel=ChatGPT2024.06 | 73.1 | |
| ReAcTable2024.06 | 73.1 | |
| Few-Shot QABackbone=GPT 3.52026.04 | 71.54 | |
| End-to-End QABackbone=GPT 3.52026.04 | 70.45 | |
| NormTabTargeted + SQLnormalization=targeted2024.06 | 68.9 | |
| CFMSBackbone=LLaMA 22026.04 | 68.53 | |
| Chain-of-TableBackbone=LLaMA 22026.04 | 67.24 | |
| NormTabBasic + SQLnormalization=basic2024.06 | 67.1 | |
| Chain-of-ThoughtBackbone=GPT 3.52026.04 | 65.37 | |
| DaterBackbone=LLaMA 22026.04 | 65.12 | |
| Text-to-SQLsource=Rajkumar et al., 20222024.06 | 64.71 | |
| Text-to-SQLBackbone=GPT 3.52026.04 | 64.71 | |
| Text-to-SQLBackbone=LLaMA 22026.04 | 64.03 | |
| BinderBackbone=LLaMA 22026.04 | 62.76 | |
| Text-to-SQLbackbone=gpt-3.5-turbo2024.06 | 62.32 | |
| Few-Shot QABackbone=LLaMA 22026.04 | 62.01 | |
| Chain-of-ThoughtBackbone=LLaMA 22026.04 | 60.52 | |
| TableDreamerIFT Data=27K, Backbone=GPT-4o2025.06 | 57.65 | |
| TableGPT2IFT Data=2.36M, Backbone=Qwen2.5-7B2025.06 | 54.87 | |
| TableDreamerIFT Data=27K, Backbone=Llama3.1-70B-Instruct2025.06 | 49.98 | |
| Llama3.1-8B-Instruct2025.06 | 48.94 | |
| InternLM2.5-7B-Chat2025.06 | 47.33 | |
| Yi-1.5-9B-Chat2025.06 | 46.22 | |
| Evol-InstructIFT Data=100K2025.06 | 45.61 | |
| Self-InstructIFT Data=100K2025.06 | 45.07 | |
| End-to-End QABackbone=LLaMA 22026.04 | 44.86 | |
| MagpieIFT Data=100K2025.06 | 43.83 | |
| Phi3.5-mini-3.8B2025.06 | 43.37 | |
| Mistral-7B-Instruct-v0.32025.06 | 42.68 | |
| MiniCPM3-4B2025.06 | 42.33 | |
| DynasourIFT Data=132K2025.06 | 41.01 | |
| GenQAIFT Data=100K2025.06 | 39.38 | |
| DeepSeek-V2-Lite-16B-Chat2025.06 | 37.11 | |
| TableLLM-syn-dataIFT Data=80K2025.06 | 29.24 | |
| TableLLMIFT Data=80K, Backbone=CodeLlama-7B2025.06 | 27.55 |