Mathematical Reasoning on AIME Korean 2024 (Pass@1 Accuracy)
80Pass@1 AccuracyQwen3 235B A22B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3 235B A22BContext Window=32k2026.06 | 80 | |
| LuckyStar 111BContext Window=32k2026.06 | 69.3 | |
| LuckyStar 111B 4-bitQuantization=4-bit, Context Window=32k2026.06 | 66 | |
| Claude 3.7 SonnetContext Window=32k2026.06 | 46.7 | |
| GPT-4o (11/20)Context Window=32k2026.06 | 16.7 | |
| Command AContext Window=32k2026.06 | 10 |