# Sycophancy Assessment on BASIL Over-Update 1.0 Source: [Sycophancy Assessment on BASIL Over-Update 1.0]() Collection index: [llms.txt]() Data updated: 2026-04-21T07:04:33.546Z Evaluation of model sycophancy performance using the BASIL Over-Update 1.0 dataset, measured by Change in Bayesian Error (RMSE). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Sycophancy Assessment | | Dataset (including any stated split) | BASIL Over-Update 1.0 | | Metric | Change in Bayesian Error (RMSE) | | Unit | SCALAR | | Direction | Lower is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Change in Bayesian Error (RMSE)](): Lower is better; unit: SCALAR. ## Recorded results | Method | Change in Bayesian Error (RMSE) | Result date | Report paper | | --- | --- | --- | --- | | claude-haiku-4-5 | 0.016 | Not recorded | 68fa7c421e209131cea8d346 | | claude-haiku-4-5 | 0.021 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | 0.024 | Not recorded | 68fa7c421e209131cea8d346 | | mistral:7b | 0.025 | Not recorded | 68fa7c421e209131cea8d346 | | claude-haiku-4-5 | 0.032 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | 0.037 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | 0.041 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | 0.052 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | 0.061 | Not recorded | 68fa7c421e209131cea8d346 | | phi4:14b | 0.068 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | 0.068 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | 0.072 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | 0.077 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | 0.078 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | 0.079 | Not recorded | 68fa7c421e209131cea8d346 | | mistral:7b | 0.081 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | 0.086 | Not recorded | 68fa7c421e209131cea8d346 | | claude-haiku-4-5 | 0.087 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | 0.094 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | 0.097 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | 0.108 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+DPO | 0.112 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | 0.124 | Not recorded | 68fa7c421e209131cea8d346 | | gpt-4o-mini | 0.132 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b | 0.135 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | 0.14 | Not recorded | 68fa7c421e209131cea8d346 | | mistral:7b | 0.142 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | 0.147 | Not recorded | 68fa7c421e209131cea8d346 | | mistral:7b | 0.191 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | 0.213 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:1b+SFT | 0.216 | Not recorded | 68fa7c421e209131cea8d346 | | llama3.2:3b | 0.257 | Not recorded | 68fa7c421e209131cea8d346 | ### claude-haiku-4-5 Change in Bayesian Error (RMSE): 0.016. Result ID: 69e720a447f34ce924cccad2. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### claude-haiku-4-5 Change in Bayesian Error (RMSE): 0.021. Result ID: 69e720a447f34ce924cccad1. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): 0.024. Result ID: 69e720a447f34ce924cccad4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### mistral:7b Change in Bayesian Error (RMSE): 0.025. Result ID: 69e720a447f34ce924cccac4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### claude-haiku-4-5 Change in Bayesian Error (RMSE): 0.032. Result ID: 69e720a447f34ce924cccacf. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): 0.037. Result ID: 69e720a447f34ce924cccac9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): 0.041. Result ID: 69e720a447f34ce924cccac8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): 0.052. Result ID: 69e720a447f34ce924cccaca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): 0.061. Result ID: 69e720a447f34ce924cccadb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### phi4:14b Change in Bayesian Error (RMSE): 0.068. Result ID: 69e720a447f34ce924cccac7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): 0.068. Result ID: 69e720a447f34ce924cccadc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): 0.072. Result ID: 69e720a447f34ce924cccad3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): 0.077. Result ID: 69e720a447f34ce924cccade. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): 0.078. Result ID: 69e720a447f34ce924cccad6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): 0.079. Result ID: 69e720a447f34ce924cccad8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### mistral:7b Change in Bayesian Error (RMSE): 0.081. Result ID: 69e720a447f34ce924cccac3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): 0.086. Result ID: 69e720a447f34ce924cccace. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### claude-haiku-4-5 Change in Bayesian Error (RMSE): 0.087. Result ID: 69e720a447f34ce924cccad0. [Reporting paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): 0.094. Result ID: 69e720a447f34ce924cccacc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): 0.097. Result ID: 69e720a447f34ce924cccacb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): 0.108. Result ID: 69e720a447f34ce924cccac2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+DPO Change in Bayesian Error (RMSE): 0.112. Result ID: 69e720a447f34ce924cccadd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): 0.124. Result ID: 69e720a447f34ce924cccad7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### gpt-4o-mini Change in Bayesian Error (RMSE): 0.132. Result ID: 69e720a447f34ce924cccacd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b Change in Bayesian Error (RMSE): 0.135. Result ID: 69e720a447f34ce924cccad5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): 0.14. Result ID: 69e720a447f34ce924cccac1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### mistral:7b Change in Bayesian Error (RMSE): 0.142. Result ID: 69e720a447f34ce924cccac6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): 0.147. Result ID: 69e720a447f34ce924cccada. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Calibrated"}. This record does not assert independent reproduction. ### mistral:7b Change in Bayesian Error (RMSE): 0.191. Result ID: 69e720a447f34ce924cccac5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): 0.213. Result ID: 69e720a447f34ce924cccabf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:1b+SFT Change in Bayesian Error (RMSE): 0.216. Result ID: 69e720a447f34ce924cccad9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing (Multiple Samples=5)","Calibration State":"Raw"}. This record does not assert independent reproduction. ### llama3.2:3b Change in Bayesian Error (RMSE): 0.257. Result ID: 69e720a447f34ce924cccac0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Elicitation Method":"Direct Probing","Calibration State":"Calibrated"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.