Sycophancy Detection on Social Sycophancy (Held-out)
91.9AUROCFinal SYPR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Final SYPRvariant=full metric2026.06 | 91.9 | 70.8 | 0.606 | |
| Final SYPR MetricMetric Category=Persona-calibrated formulation2026.06 | 91.2 | 71.1 | 0.671 | |
| Value-only warrant (V only)variant=V-only ablation2026.06 | 86.3 | 56.7 | 0.537 | |
| Observed praise only (P)variant=P-only ablation2026.06 | 85.1 | 56.9 | 0.522 | |
| Final SYPRvariant=full metric2026.06 | 84.3 | 57.3 | 0.478 | |
| SumMetric Category=Social Sycophancy Aggregate2026.06 | 78.6 | 36.5 | 0.472 | |
| Social Sycophancy2026.06 | 76.3 | 44 | 0.436 | |
| Observed praise onlyvariant=P-only2026.06 | 75.8 | 31.4 | 0.319 | |
| IndirectnessMetric Category=Social Sycophancy Submetric2026.06 | 73.2 | 38.5 | 0.453 | |
| Social Sycophancy2026.06 | 70.8 | 36.9 | 0.322 | |
| LLM judge w/ Contextmodel=GPT-5.4, context=full conversation2026.06 | 70 | 42.3 | 0.348 | |
| LLM judge w/ Contextmodel=GPT-5.42026.06 | 69.9 | 28.4 | 0.278 | |
| RoBERTa-base fine-tunedbase_model=RoBERTa-base, training=fine-tuned2026.06 | 61.3 | 19.5 | 0.118 | |
| DisjunctionMetric Category=Social Sycophancy Aggregate2026.06 | 50 | 18.6 | — | |
| FramingMetric Category=Social Sycophancy Submetric2026.06 | 48.7 | 17.5 | -0.063 | |
| ValidationMetric Category=Social Sycophancy Submetric2026.06 | 48.1 | 16.9 | -0.055 |