Multiple Choice Question Answering on MMLU Basque
79.63Accuracy (MMLU Basque)Claude 3.5 Sonnet
Evaluation results
| Method | Links | |
|---|---|---|
| Claude 3.5 SonnetModel Scale=Proprietary2025.06 | 79.63 | |
| GPT-4oModel Scale=Proprietary2025.06 | 76.66 | |
| 70B + CEU IENModel Scale=70B, Training Variant=+ CEU IEN2025.06 | 68.89 | |
| 70B INSTRUCT ENModel Scale=70B, Training Variant=INSTRUCT EN2025.06 | 68.52 | |
| 8B + CEU IENModel Scale=8B, Training Variant=+ CEU IEN2025.06 | 57.04 | |
| 8B + CEU IEN+EUModel Scale=8B, Training Variant=+ CEU IEN+EU2025.06 | 56.3 | |
| 8B + CEU IEUModel Scale=8B, Training Variant=+ CEU IEU2025.06 | 52.96 | |
| 8B INSTRUCT ENModel Scale=8B, Training Variant=INSTRUCT EN2025.06 | 50.37 |