Subject-Specific Scores on MMLU-Pro
40.4Math ScoreMindSkip
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MindSkipBackbone=LLaMA-3.2-3B-Instruct, Evaluation Protocol=pass@12026.06 | 40.4 | 47.7 | 43.3 | 31.8 | 43.7 | 38.5 | 47.6 | 47.2 | 51.5 | 31.7 | 52.3 | 49.7 | 55.6 | |
| POLARBackbone=LLaMA-3.2-3B-Instruct, Evaluation Protocol=pass@12026.06 | 40.1 | 39 | 41 | 34.3 | 43.2 | 42.9 | 50.1 | 52.4 | 52.4 | 32.7 | 56.6 | 44.5 | 48.8 | |
| ShortGPTBackbone=LLaMA-3.2-3B-Instruct, Evaluation Protocol=pass@12026.06 | 29.9 | 26.9 | 26.9 | 23.3 | 20.1 | 33 | 36.7 | 35.3 | 38.2 | 28.6 | 36 | 41.7 | 40.9 | |
| Base (τ=0)Backbone=LLaMA-3.2-3B-Instruct, Temperature=0, Evaluation Protocol=pass@12026.06 | 19.5 | 19 | 18.6 | 17.1 | 19.8 | 22.4 | 36.1 | 28.5 | 33 | 22.2 | 44.9 | 23.8 | 28.3 | |
| DR.LLMBackbone=LLaMA-3.2-3B-Instruct, Evaluation Protocol=pass@12026.06 | 13.6 | 14.4 | 16.2 | 10 | 16.4 | 9 | 13.6 | 10.6 | 15.2 | 13 | 19.8 | 8.2 | 23.1 | |
| FlexiDepthBackbone=LLaMA-3.2-3B-Instruct, Evaluation Protocol=pass@12026.06 | 7.8 | 5.9 | 4.8 | 6.4 | 6.3 | 4.7 | 5.9 | 5.9 | 3.6 | 6 | 4.9 | 7.4 | 4.7 |