Medical Sycophancy Evaluation on Medical Sycophancy Benchmark 1.0 (test)
74.8Exp. Corr. ScoreMedDR
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MedDRModel Type=Medical2025.09 | 74.8 | 75.5 | 74.7 | 75.5 | 76.1 | 74.9 | 74.3 | 76.1 | 75.1 | |
| DentoBotModel Type=Medical2025.09 | 74.6 | 68.1 | 86.9 | 71.3 | 75.7 | 69.9 | 76.7 | 86.9 | 74.8 | |
| Gemma3-12BModel Type=Open-source2025.09 | 72.2 | 69 | 73 | 70.6 | 56.3 | 70.1 | 75.8 | 75.8 | 69.6 | |
| Claude-3-OpusModel Type=Commercial2025.09 | 66.9 | 47.5 | 29.2 | 38.8 | 44.1 | 34.4 | 58 | 66.9 | 45.6 | |
| LLaVA-7BModel Type=Open-source2025.09 | 65.5 | 66.2 | 70.3 | 68 | 67.1 | 64.1 | 71.9 | 71.9 | 67.6 | |
| Gemma3-4BModel Type=Open-source2025.09 | 64.2 | 63 | 69 | 55.1 | 63 | 68.8 | 57.8 | 69 | 63 | |
| MedGemma-4BModel Type=Medical2025.09 | 63.1 | 68.7 | 68.3 | 68.5 | 72.3 | 62.8 | 70.5 | 72.3 | 67.8 | |
| LLaVA-13BModel Type=Open-source2025.09 | 62.5 | 54.9 | 63.8 | 57.6 | 60.3 | 60.6 | 62 | 63.8 | 60.3 | |
| LLaVA-MedModel Type=Medical2025.09 | 62.4 | 62.4 | 64.6 | 66.6 | 65.6 | 60.6 | 68.6 | 68.6 | 64.4 | |
| Gemma3-27BModel Type=Open-source2025.09 | 61.4 | 56.4 | 74.4 | 47.3 | 32.9 | 64.2 | 52.8 | 74.4 | 55.6 | |
| Qwen2.5-VL-32BModel Type=Open-source2025.09 | 60 | 45.6 | 27 | 45.5 | 24.6 | 76.2 | 35.9 | 76.2 | 45 | |
| GPT-4oModel Type=Commercial2025.09 | 59.7 | 38.2 | 29.2 | 27.4 | 32.5 | 36.4 | 31.3 | 59.7 | 36.4 | |
| Doubao-VisionModel Type=Commercial2025.09 | 49.5 | 50.1 | 44.8 | 46.8 | 39.9 | 68.1 | 41.6 | 68.1 | 48.7 | |
| Qwen2.5-VL-3BModel Type=Open-source2025.09 | 43.6 | 37.2 | 39.6 | 37.2 | 35.2 | 35.5 | 39.3 | 43.6 | 38.2 | |
| Llama3.2-Vision-11BModel Type=Open-source2025.09 | 43.3 | 42.5 | 34.6 | 37.4 | 43.2 | 53.8 | 39.4 | 53.8 | 42 | |
| Qwen2.5-VL-7BModel Type=Open-source2025.09 | 38.5 | 43.4 | 52.3 | 46.8 | 34.7 | 44.2 | 46.7 | 52.3 | 43.8 |