# Sycophancy Assessment on BASIL 1.0 (Under-Update) Source: [Sycophancy Assessment on BASIL 1.0 (Under-Update)]() Collection index: [llms.txt]() Data updated: 2026-04-21T07:04:33.579Z Evaluation of model sycophancy performance on the BASIL 1.0 (Under-Update) dataset, measured by Change in Bayesian Error (RMSE). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Sycophancy Assessment | | Dataset (including any stated split) | BASIL 1.0 (Under-Update) | | Metric | Change in Bayesian Error (RMSE) | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Change in Bayesian Error (RMSE)](): Lower is better; unit: SCALAR. ## Recorded results | Method | Change in Bayesian Error (RMSE) | Result date | Report paper | | --- | --- | --- | --- | | mistral:7b | -0.355 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | -0.329 | Not recorded | 68fa7c421e209131cea8d346 | | mistral:7b | -0.271 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | -0.234 | Not recorded | 68fa7c421e209131cea8d346 | | mistral:7b | -0.23 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | -0.2 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | -0.188 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | -0.171 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | -0.168 | Not recorded | 68fa7c421e209131cea8d346 | | mistral:7b | -0.152 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | -0.151 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | -0.146 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | -0.137 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | -0.137 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | -0.136 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | -0.132 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | -0.129 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | -0.129 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | -0.116 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | -0.115 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | -0.107 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | -0.104 | Not recorded | 68fa7c421e209131cea8d346 | | claude-haiku-4-5 | -0.103 | Not recorded | 68fa7c421e209131cea8d346 | | claude-haiku-4-5 | -0.091 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | -0.09 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | -0.087 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | -0.082 | Not recorded | 68fa7c421e209131cea8d346 | | claude-haiku-4-5 | -0.08 | Not recorded | 68fa7c421e209131cea8d346 | | claude-haiku-4-5 | -0.079 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | -0.048 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | -0.041 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | -0.032 | Not recorded | 68fa7c421e209131cea8d346 | ### mistral:7b Change in Bayesian Error (RMSE): -0.355. Result ID: 69e720a447f34ce924cccae3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): -0.329. Result ID: 69e720a447f34ce924cccaf4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### mistral:7b Change in Bayesian Error (RMSE): -0.271. Result ID: 69e720a447f34ce924cccae4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): -0.234. Result ID: 69e720a447f34ce924cccafc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### mistral:7b Change in Bayesian Error (RMSE): -0.23. Result ID: 69e720a447f34ce924cccae5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): -0.2. Result ID: 69e720a447f34ce924cccaf3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): -0.188. Result ID: 69e720a447f34ce924cccae0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): -0.171. Result ID: 69e720a447f34ce924cccaed. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): -0.168. Result ID: 69e720a447f34ce924cccae7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### mistral:7b Change in Bayesian Error (RMSE): -0.152. Result ID: 69e720a447f34ce924cccae6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): -0.151. Result ID: 69e720a447f34ce924cccaf6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): -0.146. Result ID: 69e720a447f34ce924cccaf5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): -0.137. Result ID: 69e720a447f34ce924cccafb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): -0.137. Result ID: 69e720a447f34ce924cccafe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): -0.136. Result ID: 69e720a447f34ce924cccae8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): -0.132. Result ID: 69e720a447f34ce924cccaee. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): -0.129. Result ID: 69e720a447f34ce924cccaf8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): -0.129. Result ID: 69e720a447f34ce924cccafd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): -0.116. Result ID: 69e720a447f34ce924cccafa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): -0.115. Result ID: 69e720a447f34ce924cccae2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): -0.107. Result ID: 69e720a447f34ce924cccae1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): -0.104. Result ID: 69e720a447f34ce924cccaeb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### claude-haiku-4-5 Change in Bayesian Error (RMSE): -0.103. Result ID: 69e720a447f34ce924cccaf0. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### claude-haiku-4-5 Change in Bayesian Error (RMSE): -0.091. Result ID: 69e720a447f34ce924cccaef. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): -0.09. Result ID: 69e720a447f34ce924cccaf9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): -0.087. Result ID: 69e720a447f34ce924cccadf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): -0.082. Result ID: 69e720a447f34ce924cccaec. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### claude-haiku-4-5 Change in Bayesian Error (RMSE): -0.08. Result ID: 69e720a447f34ce924cccaf1. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### claude-haiku-4-5 Change in Bayesian Error (RMSE): -0.079. Result ID: 69e720a447f34ce924cccaf2. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): -0.048. Result ID: 69e720a447f34ce924cccae9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): -0.041. Result ID: 69e720a447f34ce924cccaea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): -0.032. Result ID: 69e720a447f34ce924cccaf7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.