General Language Understanding on General Ability Suite (C-QA, T-QA, LAM, MMLU, L-Code)
48.1Average ScoreBase
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BaseBase Model=Llama3-8B-Instruct2025.05 | 48.1 | 11.9 | 74.5 | 63.9 | 31 | 66.9 | |
| PMETBase Model=Llama3-8B-Instruct2025.05 | 47.5 | 63.9 | 71.7 | 57.9 | 34.8 | 64.8 | |
| PMETBase Model=Mistral-7B-Instruct2025.05 | 42.7 | 62.8 | 71.1 | 60.7 | 52.4 | 58.6 | |
| JEEPBase Model=Llama3-8B-Instruct2025.05 | 41 | 60.6 | 65.3 | 53.6 | 34.3 | 63.4 | |
| BaseBase Model=Mistral-7B-Instruct2025.05 | 40.3 | 10 | 57.9 | 71.5 | 66.3 | 62.5 | |
| FT-MBase Model=Mistral-7B-Instruct2025.05 | 39.8 | 41.3 | 22.1 | 32.7 | 52.9 | 53.4 | |
| JEEPBase Model=Mistral-7B-Instruct2025.05 | 38 | 46.8 | 71.2 | 61.4 | 55.3 | 60.4 | |
| FT-MBase Model=Llama3-8B-Instruct2025.05 | 36 | 63.4 | 32.5 | 6.6 | 0 | 44 | |
| CoRE-pBase Model=Llama3-8B-Instruct2025.05 | 35.3 | 77.6 | 72.1 | 57.1 | 32.6 | 63.4 | |
| CoRE-p+rBase Model=Llama3-8B-Instruct2025.05 | 34.8 | 79.1 | 72.2 | 58.1 | 30.7 | 63.8 | |
| MEMITBase Model=Llama3-8B-Instruct2025.05 | 34.7 | 74.2 | 73.5 | 57.1 | 28.7 | 63.4 | |
| MEMITBase Model=Mistral-7B-Instruct2025.05 | 25.9 | 70 | 52.7 | 48.5 | 55.6 | 60.8 | |
| CoRE-p+rBase Model=Mistral-7B-Instruct2025.05 | 25.6 | 75.2 | 65.1 | 53.3 | 46.7 | 54.7 | |
| CoRE-pBase Model=Mistral-7B-Instruct2025.05 | 24.2 | 72.2 | 65.5 | 52.7 | 46.5 | 54.4 | |
| EMMETBase Model=Llama3-8B-Instruct2025.05 | 14.7 | 81.4 | 0.9 | 21 | 0 | 15.1 | |
| EMMETBase Model=Mistral-7B-Instruct2025.05 | 14.7 | 69.4 | 19.6 | 2.5 | 32.1 | 0 |