Arithmetic on Calc.
83.3AccuracyATLAS (cluster)
Evaluation results
| Method | Links | |
|---|---|---|
| ATLAS (cluster)Evaluation Protocol=In-Distribution2026.01 | 83.3 | |
| ATLAS (cluster)Evaluation Protocol=Out-of-Distribution2026.01 | 83.3 | |
| GPT-5Evaluation Protocol=Closed-Source2026.01 | 82.9 | |
| ATLAS (RL)Evaluation Protocol=Out-of-Distribution2026.01 | 81.6 | |
| BertRouterEvaluation Protocol=In-Distribution2026.01 | 77.1 | |
| RouterDCEvaluation Protocol=In-Distribution2026.01 | 74.9 | |
| RouterDCEvaluation Protocol=Out-of-Distribution2026.01 | 70.8 | |
| BertRouterEvaluation Protocol=Out-of-Distribution2026.01 | 67 | |
| Gemini2.5-ProEvaluation Protocol=Closed-Source2026.01 | 64.7 | |
| GPT-4.1Evaluation Protocol=Closed-Source2026.01 | 62 | |
| GPT-4oEvaluation Protocol=Closed-Source2026.01 | 58.1 | |
| ZS RouterEvaluation Protocol=Training-free, Prompting Strategy=Zero-shot2026.01 | 55.7 | |
| MLPRouterEvaluation Protocol=Out-of-Distribution2026.01 | 54.6 | |
| MLPRouterEvaluation Protocol=In-Distribution2026.01 | 48.2 | |
| FS RouterEvaluation Protocol=Training-free, Prompting Strategy=Few-shot2026.01 | 47.2 | |
| Random RouterEvaluation Protocol=Training-free2026.01 | 40.2 |