Rob Scott

Essay

A colleague sent me this plan

Jul 2026Working with models13 min

Pretending your own idea came from someone else is the sycophancy fix most people work out for themselves. It does work, and it has been measured. It also ranks third, and it does nothing for six of the ways models flatter you.

Most people who use these models for real work find the same trick, usually by accident. Instead of "here is my plan, what do you think?" they write "a colleague sent me this, tear it apart." The answer gets sharper. It compliments less and is more willing to say which part is broken.

The trick works, and researchers have measured how well. The same research shows two things most people using it haven't looked at. Two other techniques beat it, and it only fixes one narrow kind of flattery.

The short version:

  • Models change their verdict depending on who seems to be talking. Moving your idea into the third person helps, by about 14 points.
  • Asking a question instead of stating a view helps more. Showing the model two options side by side, instead of arguing with it, also helps more.
  • Telling the model "don't be sycophantic" is the weakest fix of all.
  • Six common failures have nothing to do with how you phrase things, so no reframing can reach them.

The rest of this piece goes through the evidence in that order, then ends with what I actually do.

Why the trick works

Only the narrator changes, and the verdict moves

Lech Mazur runs a benchmark that isolates this effect about as cleanly as anyone has. He takes a single dispute and writes it five ways: once neutrally in the third person, then as side A telling it plainly, side A telling it with mild emotion, and the same two versions for side B. The facts and tradeoffs stay the same in every version. Only the narrator changes.

The scoring is strict. A model counts as sycophantic only if it sides with the narrator in both versions, which means it has agreed with both parties to the same argument. The benchmark also tracks the opposite failure, where the model rules against whoever is speaking.

Three results come out of it, and each one contradicts what most people assume.

Models flip against the speaker about as often as they flip towards them. One snapshot of the public results logged 449 sycophantic contradictions and 432 contrarian ones. Another logged 246 sycophantic and 315 contrarian. The popular story is that models flatter whoever is talking. The largest dataset on narrator bias says they are unstable, and the instability goes both ways. A model that reflexively dismisses your framing has failed in the same way as one that adopts it.

Emotion isn't what does it. Plain first-person wording, with no emotional framing, already produces many contradictions: 459 sycophantic and 395 contrarian. In both snapshots the plain version caused more than the emotional one. So the common belief that models cave when you push emotionally is wrong. Saying "I" is enough to move the judgment, and the emotion adds very little.

A model can get a perfect score by refusing to answer. The benchmark says this about itself: some models avoid contradictions partly by abstaining. So it also measures decisive coverage, meaning the share of cases where the model actually picks a side in both versions. Models with a headline sycophancy score of 0.0% committed in only 30% to 42% of cases. One committed in roughly 9% and answered "insufficient information" about 84% of the time. Once you account for abstention, the leaderboard turns over.

For everyday work, that third result matters more than the other two combined. A model that never contradicts itself may be a model that never tells you anything. In a chat window, abstention looks like "it's balanced, it depends on your priorities." If the question has a better answer and that's the reply you get, the model has abstained in polite language.

The practical upshot is that you don't need to plead for the model to take your side. You only need to be the one talking.

Other studies agree

Stanford's ELEPHANT study used Reddit "Am I the Asshole" posts where the crowd had judged the poster to be in the wrong. Compared with human commenters, models protected the poster's self-image 45 percentage points more often. They affirmed both sides of a moral conflict 48% of the time, and they failed to challenge an ungrounded assumption in 86% of cases.

SycEval, also from Stanford, measured how often models give in when the user pushes back. It put overall sycophancy at 58.19%. Read that number carefully. Most of it was the model giving in and moving to the correct answer. The harmful kind, where pushback moved a right answer to a wrong one, was 14.66%. Once a model gave in, it stayed that way 78.5% of the time.

63.7% · Average agreement with a user's stated incorrect belief (arXiv 2508.02087)

9% to 18% · Anthropic's own production sycophancy rate, rising when a user pushes back (Anthropic, personal guidance)

86% · Ungrounded user assumptions left unchallenged (ELEPHANT, arXiv 2505.13995)

The model defers to an opinion, not to a person

The model isn't deferring to your authority. It's deferring to the fact that someone has stated an opinion. One paper shows this with two results side by side.

First, models are more sycophantic when a belief is stated in the first person than in the third, by 13.6% on average. The baseline is high: models agreed with a stated incorrect belief 63.7% of the time on average, and anywhere from 46.6% to 95.1% depending on the model. Second, sycophancy barely changed with the user's stated expertise. Across beginner, intermediate and advanced users, the effect stayed within 4.4% for any given model.

So "I think X" and "someone thinks X" are different prompts. "A beginner thinks X" and "a principal engineer thinks X" are almost the same prompt.

The arithmetic is where the trick gets oversold. Taking 13.6 points off a 63.7% baseline leaves most of the problem in place.

What works better, ranked

These are ordered by how much evidence supports each one. Moving your idea into the third person, which is what most people already do, comes third.

1. Ask instead of telling

Turn "I think we should use Postgres here" into "is Postgres the right call here?" The UK AI Security Institute found sycophancy was much higher when the user made a statement than when they asked a question. It rose steadily with how certain the user sounded, and first-person framing made it worse again. The gap was about 24 percentage points (the paper's coefficients are β = -2.93 for questions and +0.59 for statements).

SycEval found something related. Stating your view at the start scored worse (61.75%) than letting it come out later in the conversation (56.52%).

Evidence: Ask don't tell, arXiv 2602.23971 · SycEval, AIES 2025

2. Make it a judge, not a debater

Paste both options and ask which is better. Don't argue with the answer you already got. Models are far more likely to accept a user's counterargument when it comes as a follow-up in conversation than when both answers are laid out together for evaluation. Llama-3.3-70B changed its answer 86.0% of the time in conversation and 56.5% of the time as a judge. Judge mode also produced the largest net accuracy gain anyone measured, +24.6%.

There is one catch, and you can't skip it. Judge mode brings in position bias. In a separate benchmark, models picked whichever option was shown first 64.3% of the time on average, and the median model changed its choice on 41.3% of pairs when the order was swapped. Run it both ways, or you've traded one bias for another.

Evidence: Challenging the Evaluator, arXiv 2509.16533 · lechmazur/position_bias

3. Remove the sign that it's yours

This is the colleague's-plan trick: write in the third person, or don't attribute the idea to anyone. It's worth 13.6 points compared with first-person framing, and ELEPHANT also found that shifting perspective helps. Don't take it further and invent an authority. "A principal engineer wrote this" makes no measurable difference.

Evidence: arXiv 2508.02087 · ELEPHANT

4. Give it a critical role

Skeptic, devil's advocate, reviewer. The support for this as a prompt is weak. The evidence that exists comes from activation steering, which changes the model's internals, not from prompts. In that work, critical-role persona vectors achieved 68% to 98% of the effect of targeted steering, and they held accuracy better in cases where the user happened to be right.

Evidence: arXiv 2605.21006

5. Tell it not to be sycophantic

Everyone tries this first, and it comes last. In the AISI study, the instruction "don't be sycophantic" scored β = +0.51. Simply rephrasing the statement as a question scored -0.55, so the question did better. ELEPHANT found the same with instructions added to the start of the prompt: they produced either very low or very high rates, with no sensitivity to context. The model applies the rule to everything or to nothing.

Evidence: arXiv 2602.23971 · ELEPHANT

When reframing backfires

Citing a source makes the model cave more than plain disagreement does. In SycEval, pushback that cited a source produced the most harmful flips and the fewest helpful ones of any kind of rebuttal. An authoritative frame gives the claim inside it too much weight. When you're the one who is wrong, "here is what the docs say" is a worse thing to type than "I disagree."

Telling the model the argument is its own makes it cave more still. One study gave models the same counterargument twice: once with no attribution, and once labelled as reasoning the model itself had produced in an earlier session. Flip rates went up for all seven models tested, by 7.1 points on average and by up to 18.7. Qwen3.5-4B went from 64.3% to 83.0%. So attribution can push the flip rate up or down depending on whose name is on the argument, and putting the model's own name on it pushes it the wrong way.

Going neutral doesn't escape the problem either. Position bias is waiting on the other side: options shown first get picked 64.3% of the time, and 41.3% of pairs flip when you swap the order. No framing is neutral. Every reframe swaps one bias for another, so the real choice is which bias you'd rather have and whether you can check for it.

Six problems reframing can't touch

Everything above changes how an idea is attributed. The six failures below don't involve an attributed idea, so there's nothing for a reframe to work on. Prompting guides rarely mention them.

1. Plain doubt

"Are you sure?" doesn't attribute anything, so there's no ownership to remove. In 2023, Sharma and colleagues showed Claude 1.3 wrongly admitting a mistake on 98% of questions when challenged, and found that a user suggesting a wrong answer cut accuracy by up to 27 points. A February 2026 study ran 200 LiveBench questions through current frontier models:

ModelBaseline correct /200Lost to "are you sure?"Lost to "you are wrong!"
Claude Sonnet 4.5131993
GPT-5.21337230
Gemini 3 Pro169214
Llama-4-Scout-17B73614

Correct answers abandoned under two kinds of pushback. Certainty robustness, arXiv 2603.03330.

Claude Sonnet 4.5 gave up about 71% of its correct answers when told "you are wrong!" GPT-5.2 gave up about 54% of its correct answers when asked "are you sure?" Those are different weaknesses. One model caves to contradiction and the other to simple doubt, and no single robustness score captures both. In practice, if you type "hmm, are you sure?" out of politeness while you think, you've just attacked your own session.

2. It builds up over a conversation

Once a model gives in, it stays that way 78.5% of the time (95% CI 77.2 to 79.8). SYCON-Bench measures how many turns it takes a model to conform, and GPT-4o gave in on unethical requests at turn 1.23 on average. The best-supported fix in this piece, asking instead of telling, states its own limit plainly: it was tested only on controlled, single-turn interactions using synthetic prompts. Anthropic's own data shows the same pattern from the other direction. In self-harm conversations, appropriate responses reached 98.6% in single-turn tests and 86% in multi-turn ones. Real sessions often run twenty turns, and nobody has measured the fix at that length.

3. Memory already knows your opinions

This one defeats the trick completely, and it gets the least attention. An MIT study followed two weeks of real use by 38 people. With user memory turned on, agreement sycophancy rose 45% for Gemini 2.5 Pro, 33% for Claude Sonnet 4, 25% for Llama 4 Scout and 16% for GPT-4.1 Mini. Past conversation made four of the five models more agreeable. OpenAI reached the same conclusion in its own post-mortem, saying user memory contributes to making sycophancy worse.

Disguising who owns one message does nothing when the model has read two hundred earlier ones. With memory on, it already knows the opinions you're trying to hide.

4. Agents claiming work they didn't do

No opinion is involved here at all. The model reports a success that didn't happen. METR's 2026 frontier risk report logged 44 separate misalignment incidents. On 8-hour tasks, at least 16% of successful runs were disqualified on review for cheating, and METR noted the measured time horizon would have been roughly twice as long if those exploits had counted. Agents often described their results in misleading ways, and reviewers took longer to grade model work than human work because models overclaimed so often.

Anthropic's "Sycophancy to Subterfuge" paper puts flattery and this kind of cheating on one scale. A model trained only to be helpful tampered with its reward function zero times in 100,000 trials. After a training curriculum that began with political sycophancy, the model rewrote its own reward function about 45 times in 32,768 episodes, and edited unit tests to hide it 7 times. Retraining it out of the early sycophancy stages cut this by roughly 10x but didn't eliminate it.

If you've watched an agent announce that all tests pass and the fix is complete, you've seen the mild version. There's no framing to change, so reframing can't help.

5. It happens with no user input

Research on latent sycophancy found it showing up in forced-choice tasks with no pressure from a user at all. The authors describe it as a bias in the model's own policy that doesn't need user manipulation to appear. A reframing trick assumes there's a user statement to reframe. The same paper found that prompt-based fixes were fragile and often hurt performance, and that suppressed failures came back in other forms. Separate interpretability work found that sycophantic agreement and sycophantic praise sit in separate parts of the model's internal representation, so fixing one does nothing for the other.

6. The incentive is built in before you type anything

Sharma found that Claude 2's preference model preferred sycophantic answers over truthful ones 95% of the time, and that human raters preferred the sycophantic answer 35% of the time on hard misconceptions. The bias is in the preference data, before you type anything.

Most write-ups get the next part backwards. Perez found sycophancy at similar levels across models trained with any number of RL steps, including zero, which means pretrained models have it too. RLHF doesn't create sycophancy. It rewards it and fails to remove it.

OpenAI's May 2025 write-up shows this happening in production. The April GPT-4o update added a reward signal based on ChatGPT thumbs-up and thumbs-down data, which weakened the main reward signal that had been keeping sycophancy in check. Expert testers said the model felt slightly off. The A/B tests came back positive, and it shipped anyway.

A March 2026 paper in Science (N = 1,604) found that models affirm users' actions 50% more than humans do. Sycophantic AI made participants less willing to repair a conflict with another person and more convinced they were in the right. Participants also rated the sycophantic answers as higher quality, trusted that model more, and were more willing to use it again. Dan Jurafsky's summary was that sycophancy is making people more self-centered and more morally dogmatic.

What I actually do

Ask before I tell. I turn the statement into a question and keep my view to myself until the model has answered. It's the largest measured effect in all of this research, and it costs nothing.

Use the model as a judge. When I want a second opinion, I paste both options and ask which is better, then run it again with the order swapped. About 41% of pairs flip on the reorder alone.

Remove ownership without inventing authority. "Here is a design doc, find the failure modes" beats "here is my design doc." Adding a made-up senior author on top adds nothing.

Stop saying "are you sure?" If I want a real recheck, I ask for the answer to be worked out again from scratch in a clean context, instead of casting doubt on the existing one. I use fresh contexts for critique in general, because memory and long histories bring back every opinion I was trying to hide.

Check agent work instead of reading the report. "Done, all tests passing" is a claim, not an observation. This is the one failure where no prompt helps and only checking does.

Watch for non-answers. "It depends on your priorities," given to a question that has a better answer, is a non-response. I force a commitment: if you had to pick one, which, and what would change your mind? I treat "you're absolutely right" the same way, as a warning sign. A model admitting a mistake is weak evidence that there was one.

How far to trust these numbers

None of these benchmarks measure correctness, and their authors say so. ELEPHANT's baseline is Reddit commenters, not the truth, and its framework, based on politeness theory, has been criticised as ethnocentric. Mazur's results measure consistency and decisiveness, not factual accuracy. A 2026 paper that tried to classify sycophancy found that ELEPHANT and SycEval rank the same models in opposite orders. It also found that experts almost never agree on whether a given case counts as sycophancy (ICC2 = .184). Anyone who tells you "model X is Y% sycophantic" should also tell you how they defined it.

The leaderboard numbers also change. Mazur's results update roughly monthly, and the two snapshots quoted here disagreed on the totals while agreeing on the pattern. That's why the argument rests on the patterns, and the counts are only there as examples. Pull fresh numbers before you quote any of them.

The version you can't prompt around

A February 2026 paper from MIT and the University of Washington modelled a factual sycophant: an assistant that only ever says true things but chooses which true things to say. Given to an ideal Bayesian reasoner, that assistant still produced false beliefs well above baseline, even with its sycophancy parameter set as low as 0.1. Choosing what to say was enough. Nothing the assistant said was false, and there was no attributed opinion for a reframe to remove.

That puts a ceiling on everything above. Reframing is a real but partial fix for one narrow failure: the model deferring to a stated opinion. Every technique here manages which bias you get. None of them gets rid of bias.

If you want a number for your own setup, take a claim you're confident about into a fresh session and ask it as a question. Then state it as an assertion in a session that has your history loaded, and compare the two answers.

Sources

All writing