Multitask Language Understanding on MMLU Medical Genetics FR (test)
86AccuracyMedGemma-27B-it
Evaluation results
| Method | Links | |
|---|---|---|
| MedGemma-27B-itType=BioMed2026.04 | 86 | |
| Qwen3-32BType=Instruct2026.04 | 85 | |
| Qwen3-8B-BaseType=Base2026.04 | 80 | |
| Qwen3-8BType=Instruct2026.04 | 80 | |
| Qwen3-14BType=Instruct2026.04 | 80 | |
| Qwen3-4B-BaseType=Base2026.04 | 74 | |
| Qwen3-4BType=Instruct2026.04 | 69 | |
| Llama3.1-8B-InstructType=Instruct2026.04 | 68 | |
| Llama3.2-1BType=Base2026.04 | 67 | |
| Llama3.1-8BType=Base2026.04 | 67 | |
| Apollo-7BType=BioMed2026.04 | 67 | |
| Qwen3-1.7B-BaseType=Base2026.04 | 66 | |
| Gaperon-8BType=Base2026.04 | 65 | |
| MedGemma-4B-itType=BioMed2026.04 | 65 | |
| Apertus-8BType=Base2026.04 | 64 | |
| EuroLLM-9B-InstructType=Instruct2026.04 | 64 | |
| GPT-OSS-20BType=Instruct2026.04 | 64 | |
| Olmo3-7BType=Base2026.04 | 63 | |
| EuroLLM-9BType=Base2026.04 | 62 | |
| Qwen3-1.7BType=Instruct2026.04 | 62 | |
| Apertus-8B-InstructType=Instruct2026.04 | 62 | |
| Mistral-7B-Instruct-v0.3Type=Instruct2026.04 | 60 | |
| Mistral-7B-v0.3Type=Base2026.04 | 59 | |
| SmolLM-3BType=Instruct2026.04 | 58 | |
| MedGemma-4B-ptType=BioMed2026.04 | 48 | |
| BioMistral-7BType=BioMed2026.04 | 48 | |
| Qwen3-0.6B-BaseType=Base2026.04 | 47 | |
| Llama3.2-1B-InstructType=Instruct2026.04 | 46 | |
| Gemma3-1B-itType=Instruct2026.04 | 44 | |
| Qwen3-0.6BType=Instruct2026.04 | 38 | |
| Gemma3-270m-itType=Instruct2026.04 | 28 | |
| Gaperon-1BType=Base2026.04 | 27 | |
| Gemma3-1B-ptType=Base2026.04 | 26 | |
| BaguettotronType=Base2026.04 | 24 | |
| Gemma3-270m-ptType=Base2026.04 | 23 |