List your company
ResearchFormerly WizWand

Sycophancy Evaluation on Sycophancy Evaluation Dataset

0.123Total Sycophancy Scoremistral:7b
0.081480.361740.6420.92226Aug 23, 2025
Updated 6mo ago

Evaluation results

MethodLinks
mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.1230.0070.172
claude-haiku-4-5Elicitation Method=Direct Probing, Calibration=Raw2025.080.1520.0830.069
claude-haiku-4-5Elicitation Method=Direct Probing, Calibration=Calibrated2025.080.1520.0830.069
mistral:7bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.1710.0250.205
claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.1760.0610.115
claude-haiku-4-5Elicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.1760.0610.115
phi4:14bElicitation Method=Direct Probing, Calibration=Raw2025.080.2940.1830.201
phi4:14bElicitation Method=Direct Probing, Calibration=Calibrated2025.080.3150.1880.189
BayesDPOBase Model=llama3.2:1b, Elicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.3410.1780.16
BayesDPOBase Model=llama3.2:1b, Elicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.3440.1780.161
gpt-4o-miniElicitation Method=Direct Probing, Calibration=Raw2025.080.3980.3230.075
gpt-4o-miniElicitation Method=Direct Probing, Calibration=Calibrated2025.080.4020.3280.075
phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.4070.210.163
phi4:14bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.4290.2170.175
llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.445-0.0780.52
llama3.2:3bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.448-0.0780.523
llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.4890.1650.333
llama3.2:1bElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.5050.170.35
llama3.2:3bElicitation Method=Direct Probing, Calibration=Calibrated2025.080.545-0.0730.613
llama3.2:3bElicitation Method=Direct Probing, Calibration=Raw2025.080.551-0.0790.624
gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.5580.480.079
gpt-4o-miniElicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.5580.480.079
mistral:7bElicitation Method=Direct Probing, Calibration=Raw2025.080.6410.2640.351
mistral:7bElicitation Method=Direct Probing, Calibration=Calibrated2025.080.6510.3080.34
BayesDPOBase Model=llama3.2:1b, Elicitation Method=Direct Probing, Calibration=Raw2025.080.6960.2940.408
BayesDPOBase Model=llama3.2:1b, Elicitation Method=Direct Probing, Calibration=Calibrated2025.080.6960.2940.406
BayesSFTBase Model=llama3.2:1b, Elicitation Method=Direct Probing (Multiple Samples=5), Calibration=Calibrated2025.080.7720.5650.211
BayesSFTBase Model=llama3.2:1b, Elicitation Method=Direct Probing (Multiple Samples=5), Calibration=Raw2025.080.7740.5640.217
BayesSFTBase Model=llama3.2:1b, Elicitation Method=Direct Probing, Calibration=Raw2025.080.7760.6570.12
BayesSFTBase Model=llama3.2:1b, Elicitation Method=Direct Probing, Calibration=Calibrated2025.080.7870.6650.114
llama3.2:1bElicitation Method=Direct Probing, Calibration=Calibrated2025.081.15-0.051.165
llama3.2:1bElicitation Method=Direct Probing, Calibration=Raw2025.081.161-0.0421.163