Multitask Language Understanding on MMLU (vbal)
65.7vbal ScoreBase
Evaluation Results
| Method | Links | |
|---|---|---|
| BaseModel=Tulu3.1-8B2026.04 | 65.7 | |
| BaseModel=OLMo2-7B2026.04 | 64 | |
| BaseModel=Llama3.1-8B2026.04 | 62.5 | |
| Oracle-rephraseModel=Llama3.1-8B2026.04 | 8.6 | |
| Oracle-rephraseModel=Tulu3.1-8B2026.04 | 6.6 | |
| Oracle-rephraseModel=OLMo2-7B2026.04 | 4.5 | |
| SIMPL.Model=Llama3.1-8B2026.04 | 4.3 | |
| STRUCT.Model=Llama3.1-8B2026.04 | 3.7 | |
| Reflect-and-rephraseModel=Llama3.1-8B2026.04 | 3.6 | |
| PROF.Model=Llama3.1-8B2026.04 | 2.9 | |
| PROF.Model=OLMo2-7B2026.04 | 1.9 | |
| STRUCT.Model=OLMo2-7B2026.04 | 1.3 | |
| PROF.Model=Tulu3.1-8B2026.04 | 0.9 | |
| Reflect-and-rephraseModel=OLMo2-7B2026.04 | 0.6 | |
| STRUCT.Model=Tulu3.1-8B2026.04 | 0.2 | |
| SIMPL.Model=OLMo2-7B2026.04 | 0.2 | |
| SIMPL.Model=Tulu3.1-8B2026.04 | 0 | |
| Reflect-and-rephraseModel=Tulu3.1-8B2026.04 | 0 |