Multi-step Reasoning on SVAMP (Accuracy)
94AccuracyeMoT
Evaluation results
| Method | Links | |
|---|---|---|
| eMoTModels=Qwen-32B2026.06 | 94 | |
| BoTModels=GPT-42026.06 | 91.3 | |
| Qwen-32B (Direct)Models=Qwen-32B2026.06 | 83 | |
| PaLModels=Codex (175B)2026.06 | 79.4 | |
| ToTModels=GPT-42026.06 | 60 | |
| TILRBackbone=GPT-2, Average generation tokens per inference=42026.06 | 33.9 | |
| RefinementBackbone=GPT-2, Average generation tokens per inference=42026.06 | 31.5 | |
| CoconutBackbone=GPT-2, Average generation tokens per inference=42026.06 | 30 | |
| CoTBackbone=GPT-2, Average generation tokens per inference=242026.06 | 28.2 | |
| AdaAnchorBackbone=GPT-2, Average generation tokens per inference=42026.06 | 18.4 | |
| No-CoTBackbone=GPT-2, Average generation tokens per inference=22026.06 | 14.1 |