Multitask Language Understanding on MMLU (Accuracy, AVG, and Delta)
77.5MMLU AccuracyPhi-4 14B (w/ LoopUS)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Phi-4 14B (w/ LoopUS)Model=Phi-4 14B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 77.5 | 68.6 | 1.7 | |
| Phi-4 14B (w/o LoopUS)Model=Phi-4 14B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 76.9 | 67 | — | |
| Qwen 8B (w/o LoopUS)Model=Qwen 8B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 72.8 | 63.2 | — | |
| Qwen 8B (w/ LoopUS)Model=Qwen 8B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 71.5 | 65.4 | 2.2 | |
| Qwen 4B (w/o LoopUS)Model=Qwen 4B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 68.3 | 60.3 | — | |
| Qwen 4B (w/ LoopUS)Model=Qwen 4B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 67.7 | 62.1 | 1.8 | |
| Qwen 1.7B (w/ LoopUS)Model=Qwen 1.7B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 56.6 | 55.3 | 1.6 | |
| Qwen 1.7B (w/o LoopUS)Model=Qwen 1.7B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 55.4 | 53.7 | — |