Multiple Choice Question Answering on MMLU English (test)
89AccuracyOpenAI-o3-mini
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OpenAI-o3-miniScenario=base, Setup=Zero-shot2025.02 | 89 | — | |
| Claude-3.5-SonnetScenario=base, Setup=Zero-shot2025.02 | 86 | — | |
| GPT-4oScenario=base, Setup=Zero-shot2025.02 | 84 | — | |
| GPT-4-TurboScenario=base, Setup=Zero-shot2025.02 | 81 | — | |
| Llama-3-70B-InstructScenario=base, Setup=Zero-shot2025.02 | 78 | — | |
| Gemma-2-27B-InstructScenario=base, Setup=Zero-shot2025.02 | 75 | — | |
| Mixtral-8x22B-InstructScenario=base, Setup=Zero-shot2025.02 | 73 | — | |
| DeepSeek-R1-70BScenario=base, Setup=Zero-shot2025.02 | 72 | — | |
| Mixtral-8x7B-InstructScenario=base, Setup=Zero-shot2025.02 | 68 | — | |
| GPT-3.5-TurboScenario=base, Setup=Zero-shot2025.02 | 66 | — | |
| OpenAI-o3-miniScenario=NOTO, Setup=Zero-shot2025.02 | 65 | 26.97 | |
| Llama-3-8B-InstructScenario=base, Setup=Zero-shot2025.02 | 64 | — | |
| DeepSeek-R1-70BScenario=NOTO, Setup=Zero-shot2025.02 | 60 | 16.67 | |
| Mistral-7B-InstructScenario=base, Setup=Zero-shot2025.02 | 59 | — | |
| GPT-4oScenario=NOTO, Setup=Zero-shot2025.02 | 58 | 30.95 | |
| GPT-4-TurboScenario=NOTO, Setup=Zero-shot2025.02 | 51 | 37.04 | |
| Gemma-7B-InstructScenario=base, Setup=Zero-shot2025.02 | 49 | — | |
| Llama-2-7B-ChatScenario=base, Setup=Zero-shot2025.02 | 40 | — | |
| Claude-3.5-SonnetScenario=NOTO, Setup=Zero-shot2025.02 | 40 | 53.49 | |
| Salamandra-7B-InstructScenario=base, Setup=Zero-shot2025.02 | 38 | — | |
| Gemma-2-27B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 32 | 57.33 | |
| Leniachat-Gemma-2BScenario=base, Setup=Zero-shot2025.02 | 32 | — | |
| Llama-3-70B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 30 | 61.54 | |
| Llama-2-7B-ChatScenario=NOTO, Setup=Zero-shot2025.02 | 29 | 27.5 | |
| Mixtral-8x7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 29 | 57.35 | |
| Llama-3-8B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 26 | 59.38 | |
| Mistral-7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 18 | 69.49 | |
| Mixtral-8x22B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 16 | 78.08 | |
| Salamandra-7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 11 | 71.05 | |
| GPT-3.5-TurboScenario=NOTO, Setup=Zero-shot2025.02 | 9 | 86.36 | |
| Gemma-7B-InstructScenario=NOTO, Setup=Zero-shot2025.02 | 7 | 85.71 | |
| Leniachat-Gemma-2BScenario=NOTO, Setup=Zero-shot2025.02 | 3 | 90.63 |