Medical Visual Question Answering on MMMU
70AccuracyMedVLThinker-32B RL
Evaluation Results
| Method | Links | |
|---|---|---|
| MedVLThinker-32B RLOpen Weights=true, Open Recipe=true, Training Modality=Text-only (m23k), Decoding=Greedy2025.08 | 70 | |
| GPT-4oOpen Weights=false, Open Recipe=false, Decoding=Greedy2025.08 | 68.82 | |
| Qwen2.5-VL-32B-InstructOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 63.92 | |
| GPT-4o-miniOpen Weights=false, Open Recipe=false, Decoding=Greedy2025.08 | 63.53 | |
| Gemme 3 27BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 60.78 | |
| MedTutor-R1Backbone=Qwen2.5VL-7B-Instruct2025.12 | 58.82 | |
| HuatuoGPT-Vision-34BOpen Weights=true, Open Recipe=true, Decoding=Greedy2025.08 | 57.06 | |
| MedVLThinker-7B RLOpen Weights=true, Open Recipe=true, Training Modality=Text-only (m23k), Decoding=Greedy2025.08 | 56.86 | |
| MedTutor-R1 w/ LLaVA-basedBackbone=LLaVA2025.12 | 56.38 | |
| MedTutor-R1 w/o RLReinforcement Learning=disabled2025.12 | 54.73 | |
| Qwen2.5-VL-7B-InstructOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 52.94 | |
| Qwen2.5VLBackbone=Qwen2.5VL-7B-Instruct2025.12 | 52.61 | |
| MedVLThinker-3B RLOpen Weights=true, Open Recipe=true, Training Modality=Text-only (m23k), Decoding=Greedy2025.08 | 52.16 | |
| HuatuoGPT-Vision-7BOpen Weights=true, Open Recipe=true, Decoding=Greedy2025.08 | 50.59 | |
| Gemme 3 4BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 46.67 | |
| Qwen2.5-VL-3B-InstructOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 44.12 | |
| MedGemma 27BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 35.88 | |
| MedGemma 4BOpen Weights=true, Open Recipe=false, Decoding=Greedy2025.08 | 32.55 | |
| Llava Med v1.5 Mistral 7BOpen Weights=true, Open Recipe=true, Decoding=Greedy2025.08 | 31.37 |