Sycophancy Evaluation on VISE
58.04Strong BiasGemini-1.5-Pro
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-1.5-ProModel Type=Commercial2025.06 | 58.04 | 33.96 | 47.94 | 42.05 | 41.83 | 19.59 | 22.39 | 58.04 | 37.97 | |
| Qwen2.5-VLModel Size=7B, Model Type=Open-source2025.06 | 57.66 | 38.16 | 43.41 | 45.32 | 60.54 | 30.55 | 38.79 | 60.54 | 44.92 | |
| LLaVA-OnevisionModel Size=7B, Model Type=Open-source2025.06 | 54.39 | 54.51 | 55.34 | 59.55 | 57.05 | 57.1 | 26.82 | 59.55 | 52.11 | |
| InternVL 2.5Model Size=8B, Model Type=Open-source2025.06 | 33.83 | 26.45 | 22.46 | 16.69 | 40.45 | 41.44 | 30.41 | 41.44 | 30.25 | |
| Qwen2.5-VLModel Size=32B, Model Type=Open-source2025.06 | 28.34 | 16.23 | 17.81 | 13.34 | 17.53 | 4.77 | 34.56 | 34.56 | 18.94 | |
| Qwen2.5-VLModel Size=72B, Model Type=Open-source2025.06 | 26.85 | 11.87 | 21.9 | 17.25 | 10.29 | 8.39 | 10.29 | 26.85 | 15.26 | |
| InternVL 2.5Model Size=26B, Model Type=Open-source2025.06 | 25.75 | 21.48 | 16.01 | 13.66 | 25.66 | 19.51 | 25.07 | 25.75 | 21.02 | |
| GPT 4o miniModel Type=Commercial2025.06 | 8.72 | 7.72 | 9.53 | 6.76 | 11.76 | 6.69 | 45.96 | 45.96 | 13.88 | |
| VideoChat-FlashModel Type=Open-source2025.06 | 7.55 | 5.09 | 4.16 | 2.67 | 13.36 | 52.68 | 24.39 | 52.68 | 15.7 |