Multilingual Language Understanding on Qwen Multi-task Evaluation Suite 2.5 (test)
59.5MC ScoreTriMix (PPL)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| TriMix (PPL)Small Model Size=1.5B, Large Model Size=14B, #Param Train=1.5B, #Param Test=17B2026.04 | 59.5 | 20.5 | 16.8 | 33.6 | 46.3 | 38.6 | 26 | 36.1 | |
| TriMix (ENT)Small Model Size=1.5B, Large Model Size=14B, #Param Train=1.5B, #Param Test=17B2026.04 | 57.8 | 15.6 | 17 | 33.3 | 41.3 | 35.8 | 25.8 | 34.1 | |
| Proxy TuningSmall Model Size=1.5B, Large Model Size=14B, #Param Train=1.5B, #Param Test=17B2026.04 | 57.7 | 15.4 | 16.8 | 33.1 | 40.3 | 37.3 | 25 | 33.9 | |
| Qwen2.5-14B-ins†Small Model Size=1.5B, Large Model Size=14B, #Param Train=0B, #Param Test=14B2026.04 | 57.1 | 21 | 13.8 | 35.7 | 42.9 | 34.6 | 24.6 | 34.4 | |
| TriMix (PPL)Small Model Size=1.5B, Large Model Size=7B, #Param Train=1.5B, #Param Test=10B2026.04 | 53.4 | 19.8 | 15.7 | 27.7 | 42.7 | 38.1 | 23.6 | 33 | |
| TriMix (Upper Bound)Small Model Size=1.5B, Large Model Size=3B, #Param Train=1.5B, #Param Test=6B2026.04 | 52.4 | 21.3 | 17.6 | 27.9 | 43.8 | 37.4 | 25.3 | 33.6 | |
| TriMix (ENT)Small Model Size=1.5B, Large Model Size=7B, #Param Train=1.5B, #Param Test=10B2026.04 | 51.2 | 16.1 | 16.4 | 26.7 | 40.5 | 34.8 | 22.9 | 31.2 | |
| Proxy TuningSmall Model Size=1.5B, Large Model Size=7B, #Param Train=1.5B, #Param Test=10B2026.04 | 50.5 | 16.3 | 13.3 | 26.3 | 38.6 | 33.7 | 21.6 | 30 | |
| Qwen2.5-7B-ins†Small Model Size=1.5B, Large Model Size=7B, #Param Train=0B, #Param Test=7B2026.04 | 49.7 | 20 | 12.5 | 28.2 | 40.6 | 30.5 | 23 | 30.6 | |
| TriMix (PPL)Small Model Size=1.5B, Large Model Size=3B, #Param Train=1.5B, #Param Test=6B2026.04 | 48.7 | 19.5 | 16.3 | 25.6 | 41.4 | 36.4 | 21.1 | 31.1 | |
| Qwen2.5-1.5B-cptSmall Model Size=1.5B, Large Model Size=1.5B, #Param Train=1.5B, #Param Test=1.5B2026.04 | 48.3 | 17.6 | 17.3 | 25.7 | 41.7 | 33.8 | 21.5 | 30.7 | |
| Model MergingSmall Model Size=1.5B, Large Model Size=1.5B, #Param Train=1.5B, #Param Test=1.5B2026.04 | 48 | 18.6 | 17.5 | 27 | 41.8 | 33.8 | 21 | 30.9 | |
| Proxy TuningSmall Model Size=1.5B, Large Model Size=3B, #Param Train=1.5B, #Param Test=6B2026.04 | 45.4 | 14.1 | 14.1 | 23.5 | 40 | 29.4 | 21.2 | 28.5 | |
| Contrastive DecodingSmall Model Size=1.5B, Large Model Size=1.5B, #Param Train=1.5B, #Param Test=3B2026.04 | 45.3 | 15.7 | 15.2 | 25 | 37.7 | 32.1 | 18.1 | 28.2 | |
| TriMix (ENT)Small Model Size=1.5B, Large Model Size=3B, #Param Train=1.5B, #Param Test=6B2026.04 | 45 | 13.7 | 14.8 | 24.1 | 33.8 | 29.3 | 22.5 | 27.4 | |
| Qwen2.5-3B-ins†Small Model Size=1.5B, Large Model Size=3B, #Param Train=0B, #Param Test=3B2026.04 | 42.4 | 12.2 | 10.8 | 24.2 | 30.4 | 23.2 | 21.2 | 24.8 | |
| Qwen2.5-1.5B-base†Small Model Size=1.5B, Large Model Size=1.5B, #Param Train=0B, #Param Test=1.5B2026.04 | 40.2 | 10.8 | 11.5 | 22.3 | 26.6 | 25.5 | 20 | 23.6 | |
| Qwen2.5-1.5B-ins†Small Model Size=1.5B, Large Model Size=1.5B, #Param Train=0B, #Param Test=1.5B2026.04 | 36.4 | 10.4 | 10.5 | 19.8 | 24.3 | 23.4 | 18.9 | 21.6 |