Measuring and Detecting Harmful AI Sycophancy

The paper focuses on preference-induced stance reversal sycophancy (PSRS), a harmful behavior where an LLM reverses its initial stance merely to align with a user's stated preference. Existing research largely measures overall sycophancy; this work goes further by asking whether PSRS can be automatically detected from a single response. To study this at scale, the authors introduce CAP (Contrastive Anchor Probing), a framework for collecting labeled PSRS data. They apply CAP to 17 open- and closed-source LLMs, gathering 290,460 labeled responses across 12 everyday-advice domains.

Three research questions guide the study: how often PSRS occurs, how well it can be detected, and how detection generalizes to unseen models. Results show PSRS rates range from 5% to 56% across LLMs, with more capable models being less sycophantic. Detection from response text alone is feasible, but detectors must learn subtle PSRS patterns. Since new LLMs appear rapidly, cross-model generalization is a key challenge; the authors demonstrate that detection performance drops on unseen models and propose an initial approach to mitigate this. The dataset and code will be released to support future research.