Multimodal Medical Question Answering on PMC v1.0 (test)
0.5855AccuracyGPT-4o
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4oOpen Weights=false, Open Recipe=false, Decoding=Greedy2025.08 | 0.5855 | |
| MedVLThinker-32B RLOpen Weights=true, Open Recipe=true, Training Modality=Text-only (m23k), Algorithm=RLVR, Decoding=Greedy2025.08 | 0.5437 | |
| HuatuoGPT-Vision-7BOpen Weights=true, Open Recipe=true, Decoding=Greedy2025.08 | 0.5339 | |
| Qwen2.5-VL-32B-InstructOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 0.5328 | |
| HuatuoGPT-Vision-34BOpen Weights=true, Open Recipe=true, Decoding=Greedy2025.08 | 0.5254 | |
| Gemme 3 27BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 0.5205 | |
| GPT-4o-miniOpen Weights=false, Open Recipe=false, Decoding=Greedy2025.08 | 0.519 | |
| MedVLThinker-7B RLOpen Weights=true, Open Recipe=true, Training Modality=Text-only (m23k), Algorithm=RLVR, Decoding=Greedy2025.08 | 0.5067 | |
| Qwen2.5-VL-7B-InstructOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 0.493 | |
| MedVLThinker-3B RLOpen Weights=true, Open Recipe=true, Training Modality=Text-only (m23k), Algorithm=RLVR, Decoding=Greedy2025.08 | 0.4732 | |
| Qwen2.5-VL-3B-InstructOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 0.4477 | |
| Gemme 3 4BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 0.4442 | |
| MedGemma 4BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 0.4273 | |
| MedGemma 27BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 0.3675 | |
| Llava Med v1.5 Mistral 7BOpen Weights=true, Open Recipe=true, Decoding=Greedy2025.08 | 0.3428 |