Multi-modal Multiple Choice Question Answering on PathMMU (test)
75.34Accuracy (Atlas)Patho-R1-7B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Patho-R1-7BModel Size=Large, Thinking Ability=Yes2025.05 | 75.34 | 63.37 | 66.43 | 45.4 | 66.06 | 67.93 | |
| PathGen-LLaVA-13BModel Size=Large, Thinking Ability=No2025.05 | 63.83 | 58.79 | 58.47 | 56.19 | 59.24 | 58.47 | |
| Patho-R1-3BModel Size=Small, Thinking Ability=Yes2025.05 | 62.22 | 57.93 | 62.51 | 34.99 | 61.07 | 64.07 | |
| HuatuoGPT-Vision-7BModel Size=Large, Thinking Ability=No2025.05 | 58.07 | 54.59 | 54.72 | 36.64 | 61.36 | 59.37 | |
| InternVL3-8BModel Size=Large, Thinking Ability=Yes2025.05 | 54.07 | 50.38 | 50.8 | 39.09 | 54.04 | 53.32 | |
| Llama-3.2V-11B-cotModel Size=Large, Thinking Ability=Yes2025.05 | 51.81 | 44.23 | 45.45 | 30.76 | 48.15 | 46.1 | |
| InternVL2.5-8BModel Size=Large, Thinking Ability=Yes2025.05 | 50.06 | 46.98 | 50.62 | 32.84 | 50.02 | 50.87 | |
| Qwen2.5VL-3BModel Size=Small, Thinking Ability=Yes2025.05 | 45.06 | 42.1 | 45.28 | 23.65 | 46.79 | 48.1 | |
| InternVL2-8BModel Size=Large, Thinking Ability=Yes2025.05 | 43.68 | 40.68 | 44.86 | 23.77 | 44.56 | 45.4 | |
| Quilt-LLaVA-v1.5-7BModel Size=Large, Thinking Ability=No2025.05 | 41.43 | 32.02 | 36.72 | 14.71 | 34.8 | 35.29 | |
| Qwen2.5VL-7BModel Size=Large, Thinking Ability=Yes2025.05 | 41.18 | 38.67 | 43.2 | 24.82 | 42.77 | 39.67 | |
| Llama-3.2-11B-Vision-InstructModel Size=Large, Thinking Ability=Yes2025.05 | 41.05 | 36.5 | 37.49 | 26.72 | 38.82 | 39.21 | |
| Llama3.2-3B-vlModel Size=Small, Thinking Ability=No2025.05 | 35.79 | 23.39 | 25.01 | 6.99 | 25.37 | 28.91 | |
| DeepSeek-VL2-TinyModel Size=Small, Thinking Ability=Yes2025.05 | 32.17 | 31.36 | 35.41 | 19.18 | 35.38 | 32.13 | |
| VILA1.5-3BModel Size=Small, Thinking Ability=No2025.05 | 26.53 | 26.74 | 30.6 | 15.07 | 29.1 | 30.71 | |
| LLaVA-Onevision-7BModel Size=Large, Thinking Ability=Yes2025.05 | 21.65 | 23.4 | 21.27 | 12.01 | 27.77 | 21.25 | |
| LLaVA-Med-v1.5-Mistral-7BModel Size=Large, Thinking Ability=No2025.05 | 21.4 | 18.59 | 21.51 | 7.72 | 21.31 | 20.54 | |
| Paligemma2-3b-pt-224Model Size=Small, Thinking Ability=No2025.05 | 0.5 | 1.64 | 3.15 | 0.06 | 2.01 | 1.61 |