Mathematics on AIME 2024 (Accuracy and AMSR)
80AccuracyReasonCache
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReasonCacheModel=Phi 4 reasoning plus2025.07 | 80 | 16.4 | |
| DenseModel=Phi 4 reasoning plus2025.07 | 80 | 0 | |
| DenseModel=QwQ 32B2025.07 | 76.6 | 0 | |
| DenseModel=DeepSeek R1 Distill Qwen 32B2025.07 | 73.3 | 0 | |
| ReasonCacheModel=QwQ 32B2025.07 | 73.3 | 23.41 | |
| ReasonCacheModel=DeepSeek R1 Distill Qwen 32B2025.07 | 70 | 21.18 | |
| QuestModel=QwQ 32B2025.07 | 66.6 | 25.1 | |
| QuestModel=DeepSeek R1 Distill Qwen 32B2025.07 | 60 | 20.98 | |
| StreamingLLMModel=DeepSeek R1 Distill Qwen 32B2025.07 | 56.7 | 22.34 | |
| StreamingLLMModel=QwQ 32B2025.07 | 56.7 | 20.3 | |
| StreamingLLMModel=Phi 4 reasoning plus2025.07 | 56.7 | 16.6 | |
| QuestModel=Phi 4 reasoning plus2025.07 | 56.7 | 18.5 | |
| SnapKVModel=DeepSeek R1 Distill Qwen 32B2025.07 | 50 | 1.79 | |
| SnapKVModel=Phi 4 reasoning plus2025.07 | 20 | 2.56 | |
| SnapKVModel=QwQ 32B2025.07 | 10 | 1 |