Medical Visual Question Answering on MMMU H&M
0.7875AccuracyGPT-o4-mini
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-o4-miniModel size=Proprietary, Prompting protocol=vanilla prompt2026.01 | 0.7875 | |
| GPT-5Model size=Proprietary, Prompting protocol=vanilla prompt2026.01 | 0.7786 | |
| Claude-4.5-sonnetModel size=Proprietary, Prompting protocol=vanilla prompt2026.01 | 0.7673 | |
| GPT-5-miniModel size=Proprietary, Prompting protocol=vanilla prompt2026.01 | 0.7428 | |
| Claude-4.5-haikuModel size=Proprietary, Prompting protocol=vanilla prompt2026.01 | 0.7214 | |
| Gemini-2.5-proModel size=Proprietary, Prompting protocol=vanilla prompt2026.01 | 0.7129 | |
| MEDVISTA-R1Model size=7-13B parameters2026.01 | 0.5643 | |
| InternVL3-8BModel size=7-13B parameters2026.01 | 0.5589 | |
| Cold-start w/o ToolsModel size=7-13B parameters, Tool-access setting=removed, SFT-stage setting=Cold-start2026.01 | 0.55 | |
| Qwen3vl-8BModel size=7-13B parameters2026.01 | 0.5286 | |
| Cold-start w/o ReasoningModel size=7-13B parameters, Reasoning-stage setting=removed, SFT-stage setting=Cold-start2026.01 | 0.5214 | |
| MEDVISTAGYM (InternVL3-8B)Model size=7-13B parameters, Tool-access setting=enabled2026.01 | 0.4857 | |
| MEDVISTA-R1Model size=< 7B parameters2026.01 | 0.4743 | |
| Gemini-2.5-flashModel size=Proprietary, Prompting protocol=vanilla prompt2026.01 | 0.4643 | |
| Qwen2.5vl-7BModel size=7-13B parameters2026.01 | 0.4643 | |
| Cold-start w/o ReasoningModel size=< 7B parameters, Reasoning-stage setting=removed, SFT-stage setting=Cold-start2026.01 | 0.4571 | |
| GRPO w/o ToolsModel size=7-13B parameters, Tool-access setting=removed, Reasoning-stage setting=GRPO2026.01 | 0.453 | |
| Cold-start w/o ToolsModel size=< 7B parameters, Tool-access setting=removed, SFT-stage setting=Cold-start2026.01 | 0.4429 | |
| Direct GRPO w/o cold-startModel size=7-13B parameters, SFT-stage setting=removed, Reasoning-stage setting=Direct GRPO2026.01 | 0.438 | |
| Internvl3-2BModel size=< 7B parameters2026.01 | 0.4357 | |
| MEDVISTAGYM (Qwen3vl-8B)Model size=7-13B parameters, Tool-access setting=enabled2026.01 | 0.4286 | |
| Direct GRPO w/o cold-startModel size=< 7B parameters, SFT-stage setting=removed, Reasoning-stage setting=Direct GRPO2026.01 | 0.42 | |
| LLaVA-Med-7BModel size=7-13B parameters2026.01 | 0.3771 | |
| GRPO w/o ToolsModel size=< 7B parameters, Tool-access setting=removed, Reasoning-stage setting=GRPO2026.01 | 0.375 | |
| MEDVISTAGYM (InternVL3-2B)Model size=< 7B parameters, Tool-access setting=enabled2026.01 | 0.3643 |