Multimodal Medical Question Answering on MIR residency exams OOD (test)
50.4AccuracyArogyaSutra(Qwen 2.5 VL 7B)
Evaluation results
| Method | Links | |
|---|---|---|
| ArogyaSutra(Qwen 2.5 VL 7B)Zero-shot=true, Base Model=Qwen 2.5 VL 7B2026.06 | 50.4 | |
| Qwen3-VL-8B-InstructZero-shot=true2026.06 | 49.6 | |
| MedGemma-4B-itZero-shot=true2026.06 | 45.2 | |
| Llama3-OpenBioLLM-8BZero-shot=true2026.06 | 39.3 | |
| Qwen2.5-VL-3B-InstructZero-shot=true2026.06 | 38.6 | |
| BioMistral-7BZero-shot=true2026.06 | 36.9 | |
| Qwen2.5-VL-7B-Instruct (Baseline)Zero-shot=true2026.06 | 35 | |
| MedVLM-R1Zero-shot=true2026.06 | 23.6 | |
| LLaVA-v1.6-34BZero-shot=true2026.06 | 23.3 |