Table Question Answering on MultiHiertt original (dev)
71.17Exact-match AccuracyMOCA-AGENT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MOCA-AGENTBackbone=Qwen3.6-27B2026.06 | 71.17 | 71.17 | |
| RL w/ CS (Formula)Backbone=Qwen2.5-Coder-7B, Approach=Reinforcement-learning-based2026.06 | 56.78 | 56.78 | |
| RL w/ CS (Formula)Backbone=Llama-3.1-8B, Approach=Reinforcement-learning-based2026.06 | 54.55 | 54.55 | |
| RL w/ CS (Text)Backbone=Qwen2.5-Coder-7B, Approach=Reinforcement-learning-based2026.06 | 54.25 | 54.25 | |
| Fortune++Backbone=Qwen2.5-Coder-7B, Approach=Reinforcement-learning-based2026.06 | 51.73 | 51.73 | |
| RL w/ CS (Text)Backbone=Llama-3.1-8B, Approach=Reinforcement-learning-based2026.06 | 49.34 | 49.34 | |
| o1Backbone=o1, Evaluation Protocol=Prompting / zero-shot LLMs (symbolic / formula reasoning)2026.06 | 48.95 | 48.95 | |
| NAPGBackbone=RoBERTa-large, Approach=Fine-tuning2026.06 | 44.19 | 44.19 | |
| FortuneBackbone=Qwen2.5-Coder-7B, Approach=Reinforcement-learning-based2026.06 | 40.85 | 40.85 | |
| GPT-4oBackbone=GPT-4o, Evaluation Protocol=Prompting / zero-shot LLMs (symbolic / formula reasoning)2026.06 | 38.41 | 38.41 | |
| o1Backbone=o1, Evaluation Protocol=Prompting / zero-shot LLMs (textual reasoning)2026.06 | 38.31 | 38.31 | |
| GPT-4oBackbone=GPT-4o, Evaluation Protocol=Prompting / zero-shot LLMs (textual reasoning)2026.06 | 36.11 | 36.11 | |
| GPT-4o-miniBackbone=GPT-4o-mini, Evaluation Protocol=Prompting / zero-shot LLMs (symbolic / formula reasoning)2026.06 | 28.26 | 28.26 | |
| GPT-4o-miniBackbone=GPT-4o-mini, Evaluation Protocol=Prompting / zero-shot LLMs (textual reasoning)2026.06 | 22.41 | 22.41 |