Ultrasound Question Answering on U2-Bench disease-diagnosis
42.22AccuracyMedGemma
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MedGemmaCondition=Pre-attack2026.03 | 42.22 | — | |
| QoQ-MedCondition=Pre-attack2026.03 | 41.46 | — | |
| Average Med-VLM BaselineCondition=Pre-attack2026.03 | 39.49 | — | |
| LLaVA-MedCondition=Pre-attack2026.03 | 34.79 | — | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=GPT-4.1 mini, Target Med-VLM=QoQ-Med2026.03 | 27.82 | 30.88 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=GPT-4.1 mini, Target Med-VLM=MedGemma2026.03 | 25.82 | 29.96 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=GPT-4.1 mini, Target Med-VLM=Avg.2026.03 | 24.6 | 28.2 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-30B, Target Med-VLM=QoQ-Med2026.03 | 23.83 | 24.6 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=GPT-4.1 mini, Target Med-VLM=LLaVA-Med2026.03 | 20.69 | 23.75 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-30B, Target Med-VLM=Avg.2026.03 | 20.46 | 21.41 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-30B, Target Med-VLM=MedGemma2026.03 | 20.08 | 21.69 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-30B, Target Med-VLM=LLaVA-Med2026.03 | 17.47 | 17.93 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-7B, Target Med-VLM=QoQ-Med2026.03 | 14.94 | 24.9 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-7B, Target Med-VLM=MedGemma2026.03 | 13.72 | 23.83 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-7B, Target Med-VLM=Avg.2026.03 | 13.41 | 22.42 | |
| LLM-driven prompt-edit attack with MCTSAttacker LLM=Qwen-7B, Target Med-VLM=LLaVA-Med2026.03 | 11.57 | 18.54 |