Mathematical Reasoning on AIME English 2024 (Pass@1 Accuracy)
80Pass@1 AccuracyQwen3 235B A22B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3 235B A22BContext Window=32k2026.06 | 80 | |
| LuckyStar 111BContext Window=32k2026.06 | 73.7 | |
| LuckyStar 111B 4-bitQuantization=4-bit, Context Window=32k2026.06 | 72.3 | |
| Claude 3.7 SonnetContext Window=32k2026.06 | 50 | |
| Command AContext Window=32k2026.06 | 13.3 | |
| GPT-4o (11/20)Context Window=32k2026.06 | 13.3 |