Mathematical Reasoning on MATH 500 (Accuracy, Token Count, ReL)
90.4AccuracyMixed Attention Module
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mixed Attention ModuleSeries=Qwen2.5-7B, beta=0.4, alpha=0.32026.02 | 90.4 | 4,244 | 142.2 | |
| Qwen2.5-7B (Base)Series=Qwen2.5-7B, beta=0.4, alpha=0.32026.02 | 87.4 | 4,253 | 161.6 |