Mathematical Reasoning on AIME 2024 (Accuracy, Latency, and Operational Metrics)
66.67Accuracy (%)Original
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OriginalBackbone=Qwen3-8B2025.10 | 66.67 | 6,357.13 | 418.07 | — | |
| TokenskipBackbone=Qwen3-8B, Threshold=0.92025.10 | 66.67 | 5,912.13 | 393.51 | 93 | |
| UCoTBackbone=Qwen3-8B, Threshold=0.92025.10 | 66.67 | 5,662.74 | 382.23 | 89 | |
| MILESBase Model=GPT-4.1-mini2026.07 | 66.67 | — | — | — | |
| UCoTBackbone=Qwen3-8B, Threshold=0.72025.10 | 63.33 | 4,513.79 | 298.97 | 71 | |
| AoTMethod type=Prompt-based methods, Base Model=GPT-4.1-mini2026.07 | 63.33 | — | — | — | |
| DC (w/ coding)Method type=Memory-based methods, Base Model=GPT-4.1-mini, Coding=with2026.07 | 63.33 | — | — | — | |
| TokenskipBackbone=Qwen3-8B, Threshold=0.72025.10 | 60 | 4,765.88 | 320.55 | 75 | |
| TokenskipBackbone=Qwen3-8B, Threshold=0.52025.10 | 56.67 | 4,499.99 | 307.49 | 70 | |
| UCoTBackbone=Qwen3-8B, Threshold=0.52025.10 | 56.67 | 4,132.18 | 281.55 | 65 | |
| CoT-SCMethod type=Test-time scaling, Base Model=GPT-4.1-mini2026.07 | 56.67 | — | — | — | |
| rStarMethod type=Test-time scaling, Base Model=GPT-4.1-mini2026.07 | 56.67 | — | — | — | |
| OriginalBackbone=Deepseek-R1-Distill-Qwen-7B2025.10 | 53.33 | 1,286.16 | 417.46 | — | |
| UCoTBackbone=Deepseek-R1-Distill-Qwen-7B, Threshold=0.92025.10 | 50 | 1,093.34 | 342.96 | 85 | |
| BoTMethod type=Memory-based methods, Base Model=GPT-4.1-mini2026.07 | 50 | — | — | — | |
| ToTMethod type=Test-time scaling, Base Model=GPT-4.1-mini2026.07 | 50 | — | — | — | |
| DORAMethod type=Test-time scaling, Base Model=GPT-4.1-mini2026.07 | 50 | — | — | — | |
| TokenskipBackbone=Deepseek-R1-Distill-Qwen-7B, Threshold=0.92025.10 | 46.67 | 1,208.24 | 402.41 | 94 | |
| UCoTBackbone=Deepseek-R1-Distill-Qwen-7B, Threshold=0.72025.10 | 46.67 | 887.3 | 307.05 | 69 | |
| Zero-shot CoTMethod type=Prompt-based methods, Base Model=GPT-4.1-mini2026.07 | 46.67 | — | — | — | |
| DC (w/o coding)Method type=Memory-based methods, Base Model=GPT-4.1-mini, Coding=without2026.07 | 46.67 | — | — | — | |
| TokenskipBackbone=Deepseek-R1-Distill-Qwen-7B, Threshold=0.72025.10 | 43.33 | 977.88 | 320.44 | 76 | |
| TokenskipBackbone=Deepseek-R1-Distill-Qwen-7B, Threshold=0.52025.10 | 43.33 | 926.35 | 286.88 | 72 | |
| UCoTBackbone=Deepseek-R1-Distill-Qwen-7B, Threshold=0.52025.10 | 43.33 | 823.43 | 274.18 | 64 |