Sycophancy Assessment on BASIL 1.0 (Under-Update)
-0.355Change in Bayesian Error (RMSE)mistral:7b
Evaluation results
| Method | Links | |
|---|---|---|
| mistral:7bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.355 | |
| llama3.2:1bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.329 | |
| mistral:7bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.271 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.234 | |
| mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.23 | |
| llama3.2:1bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.2 | |
| llama3.2:3bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.188 | |
| gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.171 | |
| phi4:14bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.168 | |
| mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.152 | |
| llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.151 | |
| llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.146 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.137 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.137 | |
| phi4:14bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.136 | |
| gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.132 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.129 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.129 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.116 | |
| llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.115 | |
| llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.107 | |
| gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.104 | |
| claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.103 | |
| claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.091 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.09 | |
| llama3.2:3bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.087 | |
| gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.082 | |
| claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.08 | |
| claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.079 | |
| phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.048 | |
| phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.041 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.032 |