Visual Question Answering on BreastStage-Bench Closed-ended
86.81Screening Accuracy (BUS)BreastGPT (cluster)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| BreastGPT (cluster)Parameters=8B, Selection Strategy=cluster2026.06 | 86.81 | 77.21 | 75 | 82.86 | 77.13 | 68.32 | 81.12 | 61.11 | 71.38 | 75.66 | |
| BreastGPT (learn)Parameters=8B, Selection Strategy=learn2026.06 | 84.47 | 71.03 | 68.51 | 75.71 | 78.46 | 55.26 | 73.95 | 57.14 | 71.25 | 70.64 | |
| Qwen3-VL (SFT)Parameters=8B, Evaluation Mode=SFT2026.06 | 72.34 | 76.73 | 65.74 | 78.1 | 71.01 | 59.76 | 75.87 | 53.97 | 60.41 | 68.21 | |
| Gemini-3.1-ProParameters=–, Evaluation Mode=Zero-shot2026.06 | 68.09 | 73.33 | 50.21 | 47.14 | 64.36 | 23.87 | 43.88 | 44.44 | 46.53 | 51.32 | |
| Claude-sonnet-4-6Parameters=–, Evaluation Mode=Zero-shot2026.06 | 65.53 | 68 | 37.02 | 40 | 54.79 | 19.67 | 55.94 | 42.86 | 36.3 | 46.68 | |
| GPT-5.4Parameters=–, Evaluation Mode=Zero-shot2026.06 | 64.89 | 78.55 | 68.51 | 41.43 | 53.46 | 55.26 | 53.5 | 38.1 | 32.28 | 54 | |
| Gemma-4Category=Open-Source Models, #P=8B, Zero-shot=true2026.06 | 62.98 | 78.18 | 44.57 | 38.57 | 47.87 | 18.17 | 48.43 | 39.68 | 40.68 | 46.57 | |
| Hulu-MedCategory=Medical-Specific Models, #P=7B, Zero-shot=true2026.06 | 62.98 | 77.09 | 48.19 | 31.9 | 49.2 | 15.62 | 41.43 | 50.79 | 50.18 | 47.49 | |
| LingshuParameters=7B, Evaluation Mode=Zero-shot2026.06 | 58.94 | 78.55 | 39.89 | 54.29 | 58.24 | 8.56 | 45.28 | 58.73 | 51.52 | 50.44 | |
| GLM-4.6V-FlashCategory=Open-Source Models, #P=9B, Zero-shot=true2026.06 | 58.94 | 73.09 | 37.34 | 41.9 | 45.48 | 12.16 | 48.6 | 53.97 | 49.7 | 46.8 | |
| LLaVA-OneVision-1.5Category=Open-Source Models, #P=8B, Zero-shot=true2026.06 | 58.09 | 68.85 | 30.32 | 40.95 | 41.76 | 12.31 | 50.17 | 53.97 | 50.18 | 45.18 | |
| Qwen3-VLParameters=8B, Evaluation Mode=Zero-shot2026.06 | 57.87 | 78.55 | 39.68 | 51.43 | 48.14 | 25.83 | 47.9 | 34.92 | 47.02 | 47.93 | |
| Grok-4.1-FastCategory=Proprietary Models, #P=–, Zero-shot=true2026.06 | 57.23 | 76.61 | 30.74 | 40.95 | 49.47 | 22.82 | 44.23 | 31.75 | 36.42 | 43.36 | |
| Gemini-3.1-FlashParameters=–, Evaluation Mode=Zero-shot2026.06 | 55.11 | 42.67 | 49.68 | 38.57 | 37.6 | 24.32 | 31.82 | 48.41 | 35.81 | 40.44 | |
| Qwen3-VLParameters=4B, Evaluation Mode=Zero-shot2026.06 | 52.13 | 75.39 | 41.91 | 49.05 | 35.37 | 21.17 | 48.08 | 43.65 | 38.61 | 45.04 | |
| InternVL3.5Parameters=8B, Evaluation Mode=Zero-shot2026.06 | 51.91 | 77.7 | 35.85 | 34.76 | 42.55 | 16.22 | 52.27 | 44.44 | 52.98 | 45.41 | |
| Claude-opus-4-6Parameters=–, Evaluation Mode=Zero-shot2026.06 | 50.21 | 72 | 39.57 | 50.48 | 38.83 | 7.66 | 45.1 | 41.27 | 25.94 | 41.23 | |
| Qwen2.5-VLParameters=7B, Evaluation Mode=Zero-shot2026.06 | 49.15 | 79.27 | 44.47 | 44.76 | 34.31 | 14.41 | 46.85 | 37.3 | 47.62 | 44.24 | |
| MedDrCategory=Medical-Specific Models, #P=40B, Zero-shot=true2026.06 | 48.51 | 47.39 | 25.21 | 52.38 | 35.37 | 22.52 | 49.83 | 43.65 | 39.46 | 40.48 | |
| HealthGPTCategory=Medical-Specific Models, #P=32B, Zero-shot=true2026.06 | 47.02 | 24.48 | 41.38 | 45.71 | 34.04 | 21.17 | 43.01 | 42.86 | 20.22 | 35.54 | |
| Qwen2.5-VLParameters=3B, Evaluation Mode=Zero-shot2026.06 | 46.81 | 51.39 | 33.51 | 45.71 | 30.85 | 12.76 | 51.75 | 44.44 | 51.52 | 40.97 | |
| HuatuoGPT-VParameters=7B, Evaluation Mode=Zero-shot2026.06 | 45.74 | 71.39 | 43.09 | 39.05 | 35.11 | 9.61 | 47.73 | 45.24 | 49.09 | 42.89 | |
| RadFMCategory=Medical-Specific Models, #P=14B, Zero-shot=true2026.06 | 26.81 | 39.39 | 16.38 | 36.67 | 23.94 | 5.86 | 17.13 | 11.9 | 4.63 | 20.3 | |
| MiMo-VLParameters=7B, Evaluation Mode=Zero-shot2026.06 | 15.11 | 23.52 | 8.62 | 6.67 | 14.89 | 6.16 | 0.35 | 0.79 | 19.98 | 10.68 |