Multitask Language Understanding on MMLU (MA, MI, Error Rate)
73Mean Accuracy (MA)PROBELLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PROBELLMTarget Model=GPT-oss-20b2026.02 | 73 | 27 | 36 | |
| PROBELLMTarget Model=phi-42026.02 | 64 | 36 | 70 | |
| PROBELLMTarget Model=Gemini-2.5-flash2026.02 | 63 | 37 | 38 | |
| PROBELLMTarget Model=olmo-3-7b-ins.2026.02 | 62 | 38 | 73 | |
| PROBELLMTarget Model=Llama-3.1-8b-ins.2026.02 | 59 | 41 | 86 | |
| PROBELLMTarget Model=GPT-4o-mini2026.02 | 54 | 46 | 65 | |
| PROBELLMTarget Model=Deepseek-v3.22026.02 | 51 | 49 | 36 | |
| PROBELLMTarget Model=granite-4.02026.02 | 51 | 49 | 72 | |
| PROBELLMTarget Model=devstral2026.02 | 40 | 60 | 66 | |
| PROBELLMTarget Model=ministral-14b2026.02 | 37 | 63 | 70 | |
| PROBELLMTarget Model=Claude-3.5-sonnet2026.02 | 36 | 64 | 68 | |
| PROBELLMTarget Model=Grok-4.1-fast2026.02 | 34 | 66 | 47 |