Mathematical Reasoning on Average (GSM8K, MATH-500, AMC23, AIME24, AIME25)
69.9AccuracyMixed Attention Module
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Mixed Attention ModuleSeries=Qwen2.5-7B, beta=0.4, alpha=0.32026.02 | 69.9 | 262.9 | |
| OursBackbone=Qwen2.5-7B2026.02 | 68.3 | 263.3 | |
| Qwen2.5-7B (Base)Series=Qwen2.5-7B, beta=0.4, alpha=0.32026.02 | 65.9 | 314.2 | |
| Ours (Base)Backbone=Qwen2.5-7B2026.02 | 65.9 | 314.2 | |
| INFTYTHINKBackbone=Qwen2.5-7B2026.02 | 64.7 | 319 | |
| LightThinkerBackbone=Qwen2.5-7B2026.02 | 62.1 | 268.5 | |
| SepLLMBackbone=Qwen2.5-7B2026.02 | 61.1 | 275.5 | |
| H2OBackbone=Qwen2.5-7B2026.02 | 59.4 | 268.3 | |
| OursBackbone=Qwen2.5-1.5B2026.02 | 56.2 | 253 | |
| Ours (Base)Backbone=Qwen2.5-1.5B2026.02 | 51.5 | 300 | |
| INFTYTHINKBackbone=Qwen2.5-1.5B2026.02 | 50.7 | 318 | |
| LightThinkerBackbone=Qwen2.5-1.5B2026.02 | 50.1 | 263 | |
| SepLLMBackbone=Qwen2.5-1.5B2026.02 | 47.5 | 260.6 | |
| H2OBackbone=Qwen2.5-1.5B2026.02 | 46 | 260.1 |