Pathology Visual Question Answering on PathMMU (test)
72.9PubMed (Tiny) ScoreExpert performance
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Expert performanceCategory=Baseline, Source=PathMMU paper2026.06 | 72.9 | — | — | 71.5 | — | 69 | — | 68.3 | — | 78.9 | — | |
| ScaleReasoner-R1Category=Pathological VLM, Backbone=PathoR1-7B2026.06 | 71.2 | 63.8 | 67.3 | 67.6 | 67.6 | 74.9 | 69.2 | 79.3 | 76.2 | 37.9 | 38.5 | |
| Patho-R1Category=Pathological VLM, Backbone=PathoR1-7B2026.06 | 68.7 | 62.6 | 64.9 | 63.9 | 65.7 | 71 | 66.1 | 78.4 | 73.5 | 41.8 | 42.7 | |
| CLOVERCategory=Pathological VLM2026.06 | 68 | 56.6 | 59.8 | 67.6 | 60.5 | 68.6 | 62.2 | 62 | 64.1 | 36.7 | 36.6 | |
| Lingshu-7BCategory=Medical VLM, Parameters=7B2026.06 | 60.5 | 53.7 | 57.6 | 61.6 | 58.1 | 69.4 | 58.7 | 58.1 | 62.5 | 30.4 | 31.6 | |
| GPT-4VCategory=General domain VLM, Source=PathMMU paper2026.06 | 59.4 | 49.8 | 53.5 | 58.7 | 53.9 | 60.4 | 53.6 | 48.1 | 52.8 | 36.2 | 33.8 | |
| SFTCategory=Ablation Study, Backbone=PathoR1-7B2026.06 | 59.1 | 43.3 | 52.1 | 57.9 | 53.3 | 56.9 | 51.6 | 43.3 | 47.7 | 22 | 19.6 | |
| SFT + RLCategory=Ablation Study, Backbone=PathoR1-7B2026.06 | 57.7 | 44.1 | 52.7 | 56 | 52.8 | 57.6 | 52.9 | 46.6 | 48.1 | 23.7 | 19.7 | |
| Qwen2.5-VL-7BCategory=General domain VLM, Parameters=7B2026.06 | 53.4 | 44.4 | 50.5 | 50 | 47.7 | 59.6 | 48.9 | 48.6 | 46.8 | 29.4 | 28 | |
| HuatuoGPT-V-7BCategory=Medical VLM, Parameters=7B2026.06 | 49.1 | 41.2 | 47.7 | 52.3 | 46 | 52.9 | 46.1 | 44.7 | 46.9 | 19.8 | 19.2 | |
| Gemini Pro VisionCategory=General domain VLM, Source=PathMMU paper2026.06 | 43.8 | 42.7 | 44.9 | 42.4 | 42 | 43.5 | 43.7 | 49.5 | 49.4 | 32.8 | 34.7 | |
| Quilt-LLaVACategory=Pathological VLM2026.06 | 30.3 | 31.9 | 34.2 | 26.8 | 34.6 | 35.7 | 33.3 | 41.8 | 33.3 | 27.1 | 24 | |
| Frequent ChoiceCategory=Baseline, Source=PathMMU paper2026.06 | 28.8 | 25.5 | 26.1 | 27.7 | 26.7 | 29.8 | 26.5 | 28.4 | 27.5 | 22 | 21 | |
| LLaVA-Med-7BCategory=Medical VLM, Parameters=7B2026.06 | 24.6 | 22.7 | 25.1 | 23.4 | 21.1 | 25.1 | 23.6 | 17.8 | 24.5 | 20.3 | 19.2 | |
| Random ChoiceCategory=Baseline, Source=PathMMU paper2026.06 | 22.1 | 23.7 | 25.1 | 25.5 | 26.5 | 25.5 | 26 | 19.7 | 23 | 15.3 | 16.3 |