Sycophancy Evaluation on Beacon benchmark
96A/B AccuracyMixtral 8×7B Instruct
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Mixtral 8×7B Instruct2025.10 | 96 | — | — | 33.33 | 66.67 | |
| DeepSeek Chat V3.12025.10 | 94.67 | 25 | — | 25 | 50 | |
| Claude 3.5 Sonnet2025.10 | 93.33 | 20 | — | 40 | 40 | |
| Gemma 2 27B IT2025.10 | 93.33 | 20 | — | 60 | 20 | |
| GPT-4o2025.10 | 93.33 | 20 | — | 40 | 40 | |
| Llama 3.1 8B Instruct2025.10 | 88 | 62.5 | — | — | 37.5 | |
| Mistral 7B Instruct2025.10 | 86.67 | 50 | — | — | 50 | |
| Cohere Command R+2025.10 | 86.67 | 30 | 10 | 60 | — | |
| Gemini 2.5 Pro2025.10 | 82.67 | — | — | — | — | |
| GPT-OSS 120B2025.10 | 16 | — | 50 | — | 50 | |
| Gemini 1.5 Pro2025.10 | 0 | — | — | — | — | |
| DeepSeek R12025.10 | 0 | — | — | — | — |