Failure Diagnosis on MMLU
34Macro Similarity TypePROBELLM
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PROBELLMTarget Model=GPT-4.1, Model Family=OpenAI Family, Release Date=Apr 20252026.02 | 34 | 66 | 31 | 36 | 6 | |
| PROBELLMTarget Model=GPT-5.2, Model Family=OpenAI Family, Release Date=Dec 20252026.02 | 32 | 68 | 27 | 29 | 5 | |
| PROBELLMTarget Model=Ministral-8B, Model Family=Mistral Family, Release Date=Oct 20242026.02 | 27 | 73 | 81 | 74 | 19 | |
| PROBELLMTarget Model=GPT-3.5-turbo, Model Family=OpenAI Family, Release Date=Mar 20232026.02 | 26 | 74 | 76 | 75 | 16 | |
| PROBELLMTarget Model=Mistral-7B-Ins.-v0.1, Model Family=Mistral Family, Release Date=Sep 20232026.02 | 23 | 77 | 86 | 81 | 16 | |
| PROBELLMTarget Model=Mistral-7B-Ins., Model Family=Mistral Family, Release Date=May 20242026.02 | 19 | 81 | 81 | 78 | 20 | |
| PROBELLMTarget Model=GPT-4o-mini, Model Family=OpenAI Family, Release Date=Jul 20242026.02 | 16 | 84 | 71 | 63 | 10 | |
| PROBELLMTarget Model=Ministral-3-8B-2512, Model Family=Mistral Family, Release Date=Dec 20252026.02 | 14 | 86 | 59 | 57 | 16 |