Mathematical Reasoning on AIME 24 (Acc, Tok, CR)
78.2AccuracyVanilla
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VanillaBackbone=Gemini2.5-Flash2026.04 | 78.2 | 8,476 | 100 | |
| NoThinkBackbone=Gemini2.5-Flash2026.04 | 76.3 | 8,214 | 96.9 | |
| TRACEBackbone=Gemini2.5-Flash2026.04 | 74.9 | 6,897 | 81.4 | |
| DynasorBackbone=Gemini2.5-Flash2026.04 | 74.7 | 6,976 | 82.3 | |
| VanillaBackbone=Qwen3-8B2026.04 | 70.3 | 14,477 | 100 | |
| TALEBackbone=Gemini2.5-Flash2026.04 | 69.1 | 7,361 | 86.8 | |
| TRACEBackbone=Qwen3-8B2026.04 | 68.8 | 10,532 | 72.8 | |
| TALEBackbone=Qwen3-8B2026.04 | 68.1 | 10,166 | 70.2 | |
| DynasorBackbone=Qwen3-8B2026.04 | 66.7 | 12,638 | 87.3 | |
| DEERBackbone=Gemini2.5-Flash2026.04 | 66.7 | 6,326 | 74.6 | |
| DEERBackbone=Qwen3-8B2026.04 | 66.3 | 9,645 | 66.6 | |
| VanillaBackbone=Qwen3-4B2026.04 | 64.7 | 14,318 | 100 | |
| DynasorBackbone=Qwen3-4B2026.04 | 62.3 | 12,755 | 89.1 | |
| TRACEBackbone=Qwen3-4B2026.04 | 61.5 | 9,204 | 64.3 | |
| TALEBackbone=Qwen3-4B2026.04 | 57.3 | 9,373 | 65.5 | |
| DEERBackbone=Qwen3-4B2026.04 | 52.7 | 8,720 | 60.9 | |
| VanillaBackbone=R1-Distilled-Llama-8B2026.04 | 43.3 | 12,648 | 100 | |
| DynasorBackbone=R1-Distilled-Llama-8B2026.04 | 42.2 | 10,912 | 86.3 | |
| TRACEBackbone=R1-Distilled-Llama-8B2026.04 | 41.9 | 9,858 | 77.9 | |
| NoThinkBackbone=R1-Distilled-Llama-8B2026.04 | 41.3 | 12,803 | 101 | |
| NoThinkBackbone=Qwen3-8B2026.04 | 39.7 | 3,812 | 26.3 | |
| DEERBackbone=R1-Distilled-Llama-8B2026.04 | 34.5 | 9,070 | 71.7 | |
| TALEBackbone=R1-Distilled-Llama-8B2026.04 | 32.1 | 8,724 | 69 | |
| NoThinkBackbone=Qwen3-4B2026.04 | 23.3 | 5,459 | 38.1 |