Medical Question Answering on Medical Text QA Suite
91.3MMLU-MedClaude Sonnet 4
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 4Model Type=Closed-source2026.02 | 91.3 | 78.6 | 79.3 | 92.1 | 80.2 | 33.6 | 56.3 | 73.1 | |
| GPT-4.1Model Type=Closed-source2026.02 | 89.6 | 75.6 | 77.7 | 89.1 | 77 | 30.9 | 49.9 | 70 | |
| Gemini-2.5-FlashModel Type=Closed-source2026.02 | 84.2 | 73.8 | 73.6 | 91.2 | 77.6 | 35.6 | 53.3 | 69.9 | |
| MedMO-8BModel Type=Open-source2026.02 | 81 | 77.6 | 65 | 90.4 | 60.2 | 19.9 | 36 | 61.4 | |
| Qwen3VL-8BModel Type=Open-source2026.02 | 79.3 | 70.4 | 60 | 66.1 | 56.1 | 15.1 | 34.7 | 54.5 | |
| InternVL3-8BModel Type=Open-source2026.02 | 77.5 | 75.4 | 57.7 | 62.1 | 48.5 | 13.1 | 31.2 | 52.2 | |
| MedMO-4BModel Type=Open-source2026.02 | 75.7 | 78 | 58 | 78.5 | 57.5 | 16.4 | 29.4 | 56.2 | |
| Lingshu-7BModel Type=Open-source2026.02 | 74.5 | 76.6 | 55.9 | 63.3 | 56.2 | 16.5 | 26.3 | 52.8 | |
| InternVL2.5-8BModel Type=Open-source2026.02 | 74.2 | 76.4 | 52.4 | 53.7 | 42.4 | 11.6 | 26.1 | 48.1 | |
| Qwen2.5VL-7BModel Type=Open-source2026.02 | 73.4 | 76.4 | 52.6 | 57.3 | 42.1 | 12.8 | 26.3 | 48.7 | |
| Fleming-VL-8BModel Type=Open-source2026.02 | 71.8 | 74 | 51.8 | 53.7 | 40.5 | 12.1 | 24.9 | 46.9 | |
| HuatuoGPT-V-7BModel Type=Open-source2026.02 | 69.3 | 72.8 | 51.2 | 52.9 | 40.9 | 10.1 | 21.9 | 45.6 | |
| BioMediX2-8BModel Type=Open-source2026.02 | 68.6 | 75.2 | 52.9 | 58.9 | 45.9 | 13.4 | 25.2 | 48.6 | |
| MedGemma-4B-ITModel Type=Open-source2026.02 | 66.7 | 72.2 | 52.2 | 56.2 | 45.6 | 12.8 | 21.6 | 46.8 | |
| MedVLM-R1-2BModel Type=Open-source2026.02 | 51.8 | 66.4 | 39.7 | 42.3 | 33.8 | 11.8 | 19.1 | 37.8 | |
| Med-R1-2BModel Type=Open-source2026.02 | 51.5 | 66.2 | 39.1 | 39.9 | 33.6 | 11.2 | 17.9 | 37 | |
| LLaVA-Med-7BModel Type=Open-source2026.02 | 50.6 | 26.4 | 39.4 | 42 | 34.4 | 9.9 | 16.1 | 31.3 |