Multiple Choice Question Answering on MMLU Spanish (test)
86AccuracyOpenAI-o3-mini
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OpenAI-o3-miniScenario=base, Setup=Zero-shot2025.02 | 86 | — | |
| Claude-3.5-SonnetScenario=base, Setup=Zero-shot2025.02 | 85 | — | |
| GPT-4oScenario=base, Setup=Zero-shot2025.02 | 82 | — | |
| GPT-4-TurboScenario=base, Setup=Zero-shot2025.02 | 78 | — | |
| Llama-3-70B-InstructScenario=base, Setup=Zero-shot2025.02 | 73 | — | |
| DeepSeek-R1-70BScenario=base, Setup=Zero-shot2025.02 | 72 | — | |
| Gemma-2-27B-InstructScenario=base, Setup=Zero-shot2025.02 | 72 | — | |
| Mixtral-8x22B-InstructScenario=base, Setup=Zero-shot2025.02 | 67 | — | |
| Mixtral-8x7B-InstructScenario=base, Setup=Zero-shot2025.02 | 64 | — | |
| OpenAI-o3-miniScenario=NOTO, Setup=Zero-shot2025.02 | 60 | 30.23 | |
| GPT-3.5-TurboScenario=base, Setup=Zero-shot2025.02 | 60 | — | |
| GPT-4oScenario=NOTO, Setup=Zero-shot2025.02 | 56 | 31.71 | |
| Llama-3-8B-InstructScenario=base, Setup=Zero-shot2025.02 | 56 | — | |
| GPT-4-TurboScenario=NOTO, Setup=Zero-shot2025.02 | 53 | 32.05 | |
| Mistral-7B-InstructScenario=base, Setup=Zero-shot2025.02 | 51 | — | |
| DeepSeek-R1-70BScenario=NOTO, Setup=Zero-shot2025.02 | 44 | 38.89 | |
| Gemma-7B-InstructScenario=base, Setup=Zero-shot2025.02 | 44 | — | |
| Claude-3.5-SonnetScenario=NOTO, Setup=Zero-shot2025.02 | 43 | 49.41 | |
| Llama-2-7B-ChatScenario=base, Setup=Zero-shot2025.02 | 38 | — | |
| Salamandra-7B-InstructScenario=base, Setup=Zero-shot2025.02 | 36 | — | |
| Leniachat-Gemma-2BScenario=base, Setup=Zero-shot2025.02 | 30 | — | |
| Gemma-2-27B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 28 | 61.11 | |
| Llama-3-70B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 25 | 65.75 | |
| Llama-3-8B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 23 | 58.93 | |
| Mixtral-8x7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 22 | 65.63 | |
| Llama-2-7B-ChatScenario=NOTO, Setup=Zero-shot2025.02 | 21 | 44.74 | |
| Mistral-7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 20 | 60.78 | |
| Mixtral-8x22B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 16 | 76.12 | |
| Salamandra-7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 14 | 61.11 | |
| Gemma-7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 8 | 81.82 | |
| GPT-3.5-TurboScenario=NOTO, Setup=Zero-shot2025.02 | 8 | 86.67 | |
| Leniachat-Gemma-2BScenario=NOTO, Setup=Zero-shot2025.02 | 3 | 90 |