Sycophancy Assessment on BASIL Over-Update 1.0
0.016Change in Bayesian Error (RMSE)claude-haiku-4-5
Evaluation Results
| Method | Links | |
|---|---|---|
| claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.016 | |
| claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.021 | |
| llama3.2:1bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.024 | |
| mistral:7bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.025 | |
| claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.032 | |
| phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.037 | |
| phi4:14bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.041 | |
| phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.052 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.061 | |
| phi4:14bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.068 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.068 | |
| llama3.2:1bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.072 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.077 | |
| llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.078 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.079 | |
| mistral:7bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.081 | |
| gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.086 | |
| claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.087 | |
| gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.094 | |
| gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.097 | |
| llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.108 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.112 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.124 | |
| gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.132 | |
| llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.135 | |
| llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.14 | |
| mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.142 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.147 | |
| mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.191 | |
| llama3.2:3bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.213 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.216 | |
| llama3.2:3bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.257 |