Medical Question Answering on MedQA 4-option original (test)
95.1AccuracyGemini 3 Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini 3 ProTemperature=default2026.04 | 95.1 | |
| Gemini 3 FlashTemperature=default2026.04 | 94.3 | |
| o3Size category=Large Models, Open=false2025.07 | 93.3 | |
| Gemini 2.5 ProSize category=Large Models, Open=false2025.07 | 92.6 | |
| Gemini 2.5 FlashSize category=Large Models, Open=false2025.07 | 92 | |
| DeepSeek R1Size category=Large Models, Open=true2025.07 | 90.1 | |
| MedGemma 27BSize category=Small Models, Open=true, test-time scaling=true2025.07 | 87.7 | |
| GPT-4oSize category=Large Models, Open=false2025.07 | 86.5 | |
| MedGemma 1Model Scale=27B, Temperature=0.02026.04 | 85.3 | |
| OpenBioLLM 70BSize category=Large Models, Open=true2025.07 | 78.2 | |
| Qwen3 VLModel Scale=4B, Temperature=default2026.04 | 76.8 | |
| Gemma 3 27BSize category=Small Models, Open=true2025.07 | 74.9 | |
| IQVIA Med-R1 8BSize category=Small Models, Open=false2025.07 | 73.3 | |
| MedGemma 1.5Model Scale=4B, Temperature=0.02026.04 | 69.1 | |
| MedGemma 4BSize category=Small Models, Open=true2025.07 | 64.4 | |
| MedGemma 1Model Scale=4B, Temperature=0.02026.04 | 64.4 | |
| JSL-MedLlama 3 8B v2.0Size category=Small Models, Open=Research use only2025.07 | 61.4 | |
| OpenBioLLM 8BSize category=Small Models, Open=true2025.07 | 59 | |
| BioMistral DARE 7BSize category=Small Models, Open=true2025.07 | 51.1 | |
| Gemma 3 4BSize category=Small Models, Open=true2025.07 | 50.7 |