Mathematical Reasoning on GSM8K (test) (Accuracy, Tokens, Calls, Agents)
92.4AccuracySelf-Consistency
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Self-ConsistencyBackbone=Qwen2.5-7B-Instruct2026.05 | 92.4 | 13,551.4 | 10 | 1 | |
| CoTBackbone=Qwen2.5-7B-Instruct2026.05 | 91.6 | 1,355.1 | 1 | 1 | |
| Self-ConsistencyBackbone=Phi-3 Mini 4K Instruct2026.05 | 90.37 | 14,610.8 | 10 | 1 | |
| TRACERBackbone=Qwen2.5-7B-Instruct2026.05 | 89.01 | 1,014.1 | 3.02 | 2 | |
| Sparse MADBackbone=Qwen2.5-7B-Instruct2026.05 | 88.78 | 5,488.1 | 6 | 3 | |
| MoABackbone=Qwen2.5-7B-Instruct2026.05 | 88.7 | 8,408.5 | 12 | 1 | |
| MADBackbone=Qwen2.5-7B-Instruct2026.05 | 88.62 | 5,909.1 | 6 | 3 | |
| Single-Agent GRPOBackbone=Qwen2.5-7B-Instruct2026.05 | 88.25 | 252.3 | 1 | 1 | |
| TRACERBackbone=Phi-3 Mini 4K Instruct2026.05 | 88.02 | 959.7 | 2.94 | 2 | |
| Single-Agent GSPOBackbone=Phi-3 Mini 4K Instruct2026.05 | 87.95 | 276.2 | 1 | 1 | |
| Single-Agent GSPOBackbone=Qwen2.5-7B-Instruct2026.05 | 87.41 | 246.2 | 1 | 1 | |
| Single-Agent GRPOBackbone=Phi-3 Mini 4K Instruct2026.05 | 86.81 | 282.1 | 1 | 1 | |
| Sparse MADBackbone=Phi-3 Mini 4K Instruct2026.05 | 84.69 | 4,957 | 6 | 3 | |
| MAPoRLBackbone=Phi-3 Mini 4K Instruct2026.05 | 84.15 | 1,770.7 | 4 | 2 | |
| MADBackbone=Phi-3 Mini 4K Instruct2026.05 | 84.08 | 5,345 | 6 | 3 | |
| MAPoRLBackbone=Qwen2.5-7B-Instruct2026.05 | 83.85 | 2,098.9 | 4 | 2 | |
| CoTBackbone=Phi-3 Mini 4K Instruct2026.05 | 82.5 | 1,461.1 | 1 | 1 | |
| MoABackbone=Phi-3 Mini 4K Instruct2026.05 | 80.44 | 7,995.4 | 12 | 1 | |
| MAGRPOBackbone=Phi-3 Mini 4K Instruct2026.05 | 72.52 | 1,267.1 | 4 | 2 | |
| MAGRPOBackbone=Qwen2.5-7B-Instruct2026.05 | 69.6 | 1,297 | 4 | 2 |