Multilingual Language Understanding on MMLU-ProX (test)
43.6AccuracyCOMPASS
Evaluation Results
| Method | Links | |
|---|---|---|
| COMPASSBackbone=Qwen2.5-7B-Instruct2026.04 | 43.6 | |
| COMPASS-FFTBackbone=Qwen2.5-7B-Instruct, Fine-tuning=Full2026.04 | 42.5 | |
| LangRankBackbone=Qwen2.5-7B-Instruct2026.04 | 39.4 | |
| LangSimBackbone=Qwen2.5-7B-Instruct2026.04 | 39 | |
| TargetBackbone=Qwen2.5-7B-Instruct2026.04 | 38.7 | |
| RandomBackbone=Qwen2.5-7B-Instruct2026.04 | 38.4 | |
| Pretrained (ZS)Backbone=Qwen2.5-7B-Instruct, Evaluation Protocol=Zero-shot2026.04 | 37.5 | |
| AllBackbone=Qwen2.5-7B-Instruct2026.04 | 33.7 | |
| COMPASSBackbone=Phi-4-Mini-Instruct-3.8B2026.04 | 28.7 | |
| COMPASS-FFTBackbone=Phi-4-Mini-Instruct-3.8B, Fine-tuning=Full2026.04 | 27.7 | |
| COMPASSBackbone=Llama-3.1-Instruct-8B2026.04 | 26.9 | |
| COMPASS-FFTBackbone=Llama-3.1-Instruct-8B, Fine-tuning=Full2026.04 | 26.1 | |
| LangRankBackbone=Phi-4-Mini-Instruct-3.8B2026.04 | 24.5 | |
| LangRankBackbone=Llama-3.1-Instruct-8B2026.04 | 24.4 | |
| LangSimBackbone=Phi-4-Mini-Instruct-3.8B2026.04 | 24.2 | |
| LangSimBackbone=Llama-3.1-Instruct-8B2026.04 | 24 | |
| TargetBackbone=Phi-4-Mini-Instruct-3.8B2026.04 | 23.8 | |
| RandomBackbone=Llama-3.1-Instruct-8B2026.04 | 23.7 | |
| TargetBackbone=Llama-3.1-Instruct-8B2026.04 | 23.2 | |
| RandomBackbone=Phi-4-Mini-Instruct-3.8B2026.04 | 22.9 | |
| Pretrained (ZS)Backbone=Llama-3.1-Instruct-8B, Evaluation Protocol=Zero-shot2026.04 | 22.8 | |
| Pretrained (ZS)Backbone=Phi-4-Mini-Instruct-3.8B, Evaluation Protocol=Zero-shot2026.04 | 21.7 | |
| AllBackbone=Llama-3.1-Instruct-8B2026.04 | 19.6 | |
| AllBackbone=Phi-4-Mini-Instruct-3.8B2026.04 | 18.5 |