Sycophancy Assessment on BASIL 1.0 (All)
-0.096Change in Bayesian Error (RMSE)gpt-4o-mini
Evaluation Results
| Method | Links | |
|---|---|---|
| gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.096 | |
| gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.057 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.046 | |
| mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.042 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.034 | |
| mistral:7bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.032 | |
| mistral:7bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.031 | |
| phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.025 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.025 | |
| phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.023 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | -0.023 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.023 | |
| llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.02 | |
| mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.015 | |
| gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Raw2025.08 | -0.012 | |
| phi4:14bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | -0.011 | |
| claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | -0.003 | |
| claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0 | |
| llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.001 | |
| llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.08 | 0.004 | |
| gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.004 | |
| claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.004 | |
| llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.006 | |
| claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.009 | |
| phi4:14bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.016 | |
| llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.08 | 0.018 | |
| llama3.2:3bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.028 | |
| llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.028 | |
| llama3.2:3bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.037 | |
| llama3.2:1bElicitation Method=Direct Probing, Calibration State=Raw2025.08 | 0.059 | |
| llama3.2:1bElicitation Method=Direct Probing, Calibration State=Calibrated2025.08 | 0.066 |