Language Understanding on SuperBLUE
0.61MA.PROBELLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PROBELLMTarget Model=granite-4.02026.02 | 0.61 | 0.39 | 0.75 | |
| PROBELLMTarget Model=devstral2026.02 | 0.59 | 0.41 | 0.44 | |
| PROBELLMTarget Model=phi-42026.02 | 0.58 | 0.42 | 0.59 | |
| PROBELLMTarget Model=olmo-3-7b-ins.2026.02 | 0.58 | 0.42 | 0.69 | |
| PROBELLMTarget Model=Claude-3.5-sonnet2026.02 | 0.53 | 0.47 | 0.5 | |
| PROBELLMTarget Model=GPT-4o-mini2026.02 | 0.53 | 0.47 | 0.65 | |
| PROBELLMTarget Model=Grok-4.1-fast2026.02 | 0.5 | 0.5 | 0.39 | |
| PROBELLMTarget Model=ministral-14b2026.02 | 0.48 | 0.52 | 0.65 | |
| PROBELLMTarget Model=GPT-oss-20b2026.02 | 0.48 | 0.52 | 0.31 | |
| PROBELLMTarget Model=Llama-3.1-8b-ins.2026.02 | 0.46 | 0.54 | 0.71 | |
| PROBELLMTarget Model=Gemini-2.5-flash2026.02 | 0.45 | 0.55 | 0.46 | |
| PROBELLMTarget Model=Deepseek-v3.22026.02 | 0.34 | 0.66 | 0.43 |