Medical Sycophancy Resistance on Medical Sycophancy
94.7Mitigation Resistance RateQwen2.5-VL-32B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2.5-VL-32BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 94.7 | — | — | — | |
| Qwen2.5-VL-32BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 69.8 | — | — | — | |
| GPT-4oMitigation Strategy=VIPER, Model Category=Commercial Systems2025.09 | 69.4 | — | — | — | |
| Gemma3-27BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 60 | — | — | — | |
| MedDRMitigation Strategy=Role Playing, Model Category=Medical Specialist Models2025.09 | 57.8 | — | — | — | |
| GPT-4oMitigation Strategy=Role Playing, Model Category=Commercial Systems2025.09 | 57.1 | — | — | — | |
| MedDRMitigation Strategy=Standard Chain of Thought, Model Category=Medical Specialist Models2025.09 | 55.8 | — | — | — | |
| MedDRMitigation Strategy=Visual Cues, Model Category=Medical Specialist Models2025.09 | 55.8 | — | — | — | |
| Gemma3-4BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 54.3 | — | — | — | |
| LLaVA-MedMitigation Strategy=Role Playing, Model Category=Medical Specialist Models2025.09 | 53.7 | — | — | — | |
| Gemma3-1BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 50.6 | — | — | — | |
| Gemma3-1BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 49.5 | — | — | — | |
| Qwen2.5-VL-3BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 45.1 | — | — | — | |
| MedGemmaMitigation Strategy=VIPER, Model Category=Medical Specialist Models2025.09 | 44.2 | — | — | — | |
| LLaVA-MedMitigation Strategy=Visual Cues, Model Category=Medical Specialist Models2025.09 | 42.9 | — | — | — | |
| Qwen2.5-VL-3BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 41.7 | — | — | — | |
| LLaVA-MedMitigation Strategy=VIPER, Model Category=Medical Specialist Models2025.09 | 41.5 | — | — | — | |
| LLaVA-MedMitigation Strategy=Standard Chain of Thought, Model Category=Medical Specialist Models2025.09 | 39.5 | — | — | — | |
| Gemma3-27BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 39.1 | — | — | — | |
| Qwen2.5-VL-3BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 36.1 | — | — | — | |
| MedDRMitigation Strategy=VIPER, Model Category=Medical Specialist Models2025.09 | 34.7 | — | — | — | |
| Gemma3-1BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 34.6 | — | — | — | |
| GPT-4oMitigation Strategy=Visual Cues, Model Category=Commercial Systems2025.09 | 33.5 | — | — | — | |
| Gemma3-4BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 31.9 | — | — | — | |
| Doubao-VisionMitigation Strategy=VIPER, Model Category=Commercial Systems2025.09 | 31.1 | — | — | — | |
| Gemma3-4BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 30.5 | — | — | — | |
| Gemma3-1BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 29.7 | — | — | — | |
| Claude-3-OpusMitigation Strategy=VIPER, Model Category=Commercial Systems2025.09 | 28.8 | — | — | — | |
| Gemma3-4BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 27.9 | — | — | — | |
| GPT-4oMitigation Strategy=Standard Chain of Thought, Model Category=Commercial Systems2025.09 | 27.4 | — | — | — | |
| DentoBotMitigation Strategy=Standard Chain of Thought, Model Category=Medical Specialist Models2025.09 | 27.1 | — | — | — | |
| DentoBotMitigation Strategy=VIPER, Model Category=Medical Specialist Models2025.09 | 27.1 | — | — | — | |
| Qwen2.5-VL-32BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 27 | — | — | — | |
| Qwen2.5-VL-32BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 26.7 | — | — | — | |
| Qwen2.5-VL-3BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 26.1 | — | — | — | |
| MedGemmaMitigation Strategy=Role Playing, Model Category=Medical Specialist Models2025.09 | 25.9 | — | — | — | |
| Claude-3-OpusMitigation Strategy=Role Playing, Model Category=Commercial Systems2025.09 | 25 | — | — | — | |
| Qwen2.5-VL-7BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 24.9 | — | — | — | |
| DentoBotMitigation Strategy=Visual Cues, Model Category=Medical Specialist Models2025.09 | 24.6 | — | — | — | |
| Doubao-VisionMitigation Strategy=Standard Chain of Thought, Model Category=Commercial Systems2025.09 | 24.4 | — | — | — | |
| MedGemmaMitigation Strategy=Visual Cues, Model Category=Medical Specialist Models2025.09 | 22.8 | — | — | — | |
| Doubao-VisionMitigation Strategy=Role Playing, Model Category=Commercial Systems2025.09 | 22.7 | — | — | — | |
| Gemma3-12BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 22.4 | — | — | — | |
| DentoBotMitigation Strategy=Role Playing, Model Category=Medical Specialist Models2025.09 | 21.2 | — | — | — | |
| Llama3.2-Vision-11BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 20.8 | — | — | — | |
| LLaVA-7BMitigation Strategy=VIPER, Model Category=Open-source models2025.09 | 20.6 | — | — | — | |
| Qwen2.5-VL-7BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 19.3 | — | — | — | |
| Llama3.2-Vision-11BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 18.5 | — | — | — | |
| Qwen2.5-VL-7BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 18.2 | — | — | — | |
| Gemma3-12BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 17.5 | — | — | — | |
| LLaVA-7BMitigation Strategy=Role Playing, Model Category=Open-source models2025.09 | 17.5 | — | — | — | |
| MedGemmaMitigation Strategy=Standard Chain of Thought, Model Category=Medical Specialist Models2025.09 | 17 | — | — | — | |
| Gemma3-12BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 17 | — | — | — | |
| Qwen2.5-VL-7BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 16.8 | — | — | — | |
| Gemma3-27BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 14.8 | — | — | — | |
| Gemma3-12BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 13.8 | — | — | — | |
| Llama3.2-Vision-11BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 13.6 | — | — | — | |
| Doubao-VisionMitigation Strategy=Visual Cues, Model Category=Commercial Systems2025.09 | 13 | — | — | — | |
| Llama3.2-Vision-11BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 12.9 | — | — | — | |
| Gemma3-27BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 11.4 | — | — | — | |
| LLaVA-7BMitigation Strategy=Standard Chain of Thought, Model Category=Open-source models2025.09 | 11.1 | — | — | — | |
| Claude-3-OpusMitigation Strategy=Visual Cues, Model Category=Commercial Systems2025.09 | 7.4 | — | — | — | |
| LLaVA-7BMitigation Strategy=Visual Cues, Model Category=Open-source models2025.09 | 7.1 | — | — | — | |
| Claude-3-OpusMitigation Strategy=Standard Chain of Thought, Model Category=Commercial Systems2025.09 | 6.4 | — | — | — | |
| Claude-3-OpusModel Category=Commercial Systems2025.09 | — | 28.8 | 16.9 | 6.4 | |
| DentoBotModel Category=Medical Specialist Models2025.09 | — | 27.1 | 25 | 21.2 | |
| Doubao-VisionModel Category=Commercial Systems2025.09 | — | 31.1 | 22.8 | 13 | |
| Gemma3-12BModel Category=Open-source models2025.09 | — | 22.4 | 17.7 | 13.8 | |
| Gemma3-1BModel Category=Open-source models2025.09 | — | 50.6 | 41.1 | 29.7 | |
| Gemma3-27BModel Category=Open-source models2025.09 | — | 60 | 31.3 | 11.4 | |
| Gemma3-4BModel Category=Open-source models2025.09 | — | 54.3 | 36.1 | 27.9 | |
| GPT-4oModel Category=Commercial Systems2025.09 | — | 69.4 | 46.8 | 27.4 | |
| Llama3.2-Vision-11BModel Category=Open-source models2025.09 | — | 20.8 | 16.5 | 12.9 | |
| LLaVA-7BModel Category=Open-source models2025.09 | — | 20.6 | 14.1 | 7.1 | |
| LLaVA-MedModel Category=Medical Specialist Models2025.09 | — | 53.7 | 44.4 | 39.5 | |
| MedDRModel Category=Medical Specialist Models2025.09 | — | 57.8 | 51 | 34.7 | |
| MedGemmaModel Category=Medical Specialist Models2025.09 | — | 44.2 | 27.5 | 17 | |
| Qwen2.5-VL-32BModel Category=Open-source models2025.09 | — | 94.7 | 54.6 | 26.7 | |
| Qwen2.5-VL-3BModel Category=Open-source models2025.09 | — | 45.1 | 37.3 | 26.1 | |
| Qwen2.5-VL-7BModel Category=Open-source models2025.09 | — | 24.9 | 19.8 | 16.8 |