List your company
ResearchFormerly WizWand

Sycophancy Assessment on BASIL Over-Update 1.0

0.016Change in Bayesian Error (RMSE)claude-haiku-4-5
0.006360.071430.13650.20157Aug 23, 2025
Updated 6mo ago

Evaluation results

MethodLinks
claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.016
claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.021
llama3.2:1bElicitation Method=Direct Probing, Calibration State=Calibrated2025.080.024
mistral:7bElicitation Method=Direct Probing, Calibration State=Calibrated2025.080.025
claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Raw2025.080.032
phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.037
phi4:14bElicitation Method=Direct Probing, Calibration State=Calibrated2025.080.041
phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.052
llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Raw2025.080.061
phi4:14bElicitation Method=Direct Probing, Calibration State=Raw2025.080.068
llama3.2:1b+DPOElicitation Method=Direct Probing, Calibration State=Calibrated2025.080.068
llama3.2:1bElicitation Method=Direct Probing, Calibration State=Raw2025.080.072
llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.077
llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.078
llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Calibrated2025.080.079
mistral:7bElicitation Method=Direct Probing, Calibration State=Raw2025.080.081
gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.086
claude-haiku-4-5Elicitation Method=Direct Probing, Calibration State=Calibrated2025.080.087
gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Calibrated2025.080.094
gpt-4o-miniElicitation Method=Direct Probing, Calibration State=Raw2025.080.097
llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.108
llama3.2:1b+DPOElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.112
llama3.2:1b+SFTElicitation Method=Direct Probing, Calibration State=Raw2025.080.124
gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.132
llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.135
llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.14
mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.142
llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Calibrated2025.080.147
mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.191
llama3.2:3bElicitation Method=Direct Probing, Calibration State=Raw2025.080.213
llama3.2:1b+SFTElicitation Method=Direct Probing (Multiple Samples=5), Calibration State=Raw2025.080.216
llama3.2:3bElicitation Method=Direct Probing, Calibration State=Calibrated2025.080.257