# Pressure, What Pressure? Sycophancy Disentanglement in Language Models via Reward Decomposition

Source: [Pressure, What Pressure? Sycophancy Disentanglement in Language Models via Reward Decomposition](<https://www.sota2.com/research/paper/69d5e12d7d0172140ec75bfc>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-09T05:00:04.534Z

Authors: Muhammad Ahmed Mohsin, Ahsan Bilal, Muhammad Umer, Emily Fox

Published: 2026-04-07T00:28:17.000Z

## Abstract

Large language models exhibit sycophancy, the tendency to shift their stated positions toward perceived user preferences or authority cues regardless of evidence. Standard alignment methods fail to correct this because scalar reward models conflate two distinct failure modes into a single signal: pressure capitulation, where the model changes a correct answer under social pressure, and evidence blindness, where the model ignores the provided context entirely. We operationalise sycophancy through formal definitions of pressure independence and evidence responsiveness, serving as a working framework for disentangled training rather than a definitive characterisation of the phenomenon. We propose the first approach to sycophancy reduction via reward decomposition, introducing a multi-component Group Relative Policy Optimisation (GRPO) reward that decomposes the training signal into five terms: pressure resistance, context fidelity, position consistency, agreement suppression, and factual correctness. We train using a contrastive dataset pairing pressure-free baselines with pressured variants across three authority levels and two opposing evidence contexts. Across five base models, our two-phase pipeline consistently reduces sycophancy on all metric axes, with ablations confirming that each reward term governs an independent behavioural dimension. The learned resistance to pressure generalises beyond our training methodology and prompt structure, reducing answer-priming sycophancy by up to 17 points on SycophancyEval despite the absence of such pressure forms during training.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Sycophancy Evaluation on Sycophancy Evaluation Factual](<https://www.sota2.com/research/sota/sycophancy-evaluation-on-sycophancy-evaluation-factual.md>): Sycophancy Evaluation; Sycophancy Evaluation Factual; PSS: 0.1124.

- [Sycophancy Evaluation Opinion](<https://www.sota2.com/research/sota/sycophancy-evaluation-opinion.md>): Sycophancy Evaluation; Sycophancy Evaluation Opinion; PSS: 8.14.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.05279>)

[PDF](<https://arxiv.org/pdf/2604.05279.pdf>)
