Failure Diagnosis on SuperGLUE
36Macro Similarity ScorePROBELLM
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PROBELLMTarget Model=GPT-5.2, Model Family=OpenAI Family, Release Date=Dec 20252026.02 | 36 | 64 | 21 | 29 | 5 | |
| PROBELLMTarget Model=Mistral-7B-Ins., Model Family=Mistral Family, Release Date=May 20242026.02 | 27 | 73 | 65 | 78 | 20 | |
| PROBELLMTarget Model=GPT-4.1, Model Family=OpenAI Family, Release Date=Apr 20252026.02 | 22 | 78 | 42 | 36 | 6 | |
| PROBELLMTarget Model=Ministral-3-8B-2512, Model Family=Mistral Family, Release Date=Dec 20252026.02 | 21 | 79 | 60 | 57 | 16 | |
| PROBELLMTarget Model=Mistral-7B-Ins.-v0.1, Model Family=Mistral Family, Release Date=Sep 20232026.02 | 19 | 81 | 72 | 81 | 16 | |
| PROBELLMTarget Model=GPT-4o-mini, Model Family=OpenAI Family, Release Date=Jul 20242026.02 | 18 | 83 | 57 | 63 | 10 | |
| PROBELLMTarget Model=GPT-3.5-turbo, Model Family=OpenAI Family, Release Date=Mar 20232026.02 | 16 | 84 | 78 | 75 | 16 | |
| PROBELLMTarget Model=Ministral-8B, Model Family=Mistral Family, Release Date=Oct 20242026.02 | 14 | 86 | 67 | 74 | 19 |