Multi-discipline Evaluation on MMLU-Pro
32.1Math AccuracyPOLAR
Evaluation Results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| POLARBackbone=Qwen2.5-3B-Instruct, Evaluation Protocol=pass@12026.06 | 32.1 | 35.6 | 33 | 27.6 | 41.5 | 38.9 | 53 | 46.5 | 57.4 | 33.4 | 66 | 38.9 | 35.4 | |
| Base (τ=0)Backbone=Qwen2.5-3B-Instruct, Evaluation Protocol=pass@12026.06 | 26.3 | 28.7 | 27.7 | 24.4 | 37 | 33.9 | 47.5 | 40.3 | 52.6 | 31.3 | 62.2 | 33.7 | 34.6 | |
| FlexiDepthBackbone=Qwen2.5-3B-Instruct, Evaluation Protocol=pass@12026.06 | 9.8 | 9.3 | 10.5 | 10.4 | 11.4 | 11.5 | 10.8 | 11.5 | 9.9 | 8.2 | 10.6 | 11.4 | 11 | |
| ShortGPTBackbone=Qwen2.5-3B-Instruct, Evaluation Protocol=pass@12026.06 | 8.9 | 9.3 | 10 | 12.1 | 12.1 | 12.8 | 12.6 | 13.4 | 8.9 | 11.5 | 10.3 | 9.8 | 12.1 | |
| MindSkipBackbone=Qwen2.5-3B-Instruct, Evaluation Protocol=pass@12026.06 | 7.6 | 7.5 | 8.3 | 6 | 9.5 | 7.4 | 8.6 | 9.8 | 7.5 | 8.5 | 7.5 | 9.4 | 6.8 | |
| DR.LLMBackbone=Qwen2.5-3B-Instruct, Evaluation Protocol=pass@12026.06 | 4 | 4.8 | 2.8 | 3.2 | 3 | 3.2 | 2.8 | 2.6 | 1.4 | 4.6 | 5.8 | 3 | 4.2 |