Mathematical Word Problems on GSM8K (Nec, Suf, Shuf)
88.7NecNemotron-Ultra
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Nemotron-UltraN=306–498, Pattern=Ctx Dep†2026.03 | 88.7 | 11.1 | 90.8 | |
| Small models (0.8–8B avg)N=100 ea., Pattern=Task-split2026.03 | 55 | 14 | — | |
| MiniMax-M2.5N=427–496, Pattern=Genuine2026.03 | 28.4 | 70.5 | 26.8 | |
| GPT-OSS-120BN=425–496, Pattern=Decorative2026.03 | 10.9 | 88.9 | 9.4 | |
| GPT-5.4N=376–500, Pattern=Decorative2026.03 | 8.8 | 80.3 | 2.5 | |
| Claude Opus 4.6-RN=486–499, Pattern=Decorative2026.03 | 4.8 | 88.7 | 2.4 | |
| DeepSeek-V3.2N=500, Pattern=Decorative2026.03 | 3.6 | 93.4 | 2.7 | |
| Kimi-K2.5N=457–500, Pattern=Decorative2026.03 | 1.4 | 90.9 | 1.5 |