A coin flip, delivered as advice
The design is what makes the result hard to wave away. Every analyst faced the same four open decisions: how to measure productivity, how to compare farms, which farms the estimate should cover, and how to weight them. Neither option on any of these was wrong. They were judgment calls, the kind an experienced analyst is supposed to settle using their own reading of the problem.
For each analyst, Anthropic randomly assigned the agent a preferred option on four analytical choices and a fifth, final recommendation. The reported 86% alignment rate covers the four analytical choices, not the final recommendation.
Into each of those calls, Anthropic inserted a preference with no basis at all. A coin flip decided it. The agent then did what a good assistant does: it raised the option when the decision came up and argued for it as well as the data honestly allowed. It was not instructed to deceive, fabricate or hide anything. The analysts still followed the flip 86% of the time. Where a preference existed before the agent spoke, disagreement from the agent reversed it more often than not.
The paper also reports a baseline test without assigned preferences: in 112 of 123 runs, Claude Opus 4.8 produced the same set of five choices. That suggests the model had a consistent default approach to this particular task. The result does not show how strongly an unmodified agent would influence human analysts, but it illustrates why even unprompted recommendations can introduce a systematic directional influence.
The influence almost nobody spotted
The follow-through is the uncomfortable part. In post-task explanations, 77% of surveyed analysts used arguments that the agent had presented, sometimes closely matching its wording. Only 15% explicitly credited the agent. This shows that the agent's reasoning carried into participants' explanations without frequent attribution, but it does not establish whether participants consciously adopted, independently endorsed or failed to remember the source of each argument. Fifty-two percent rated the final recommendation as more their own than the agent's, and 35% called it mostly or entirely their own. Their confidence ran at the same level as everyone else's: analysts who followed the lean on all four decisions averaged 71 out of 100, against 73 for the rest.
Six of 48 surveyed participants suspected that the study concerned AI influence, but none identified the experimentally assigned preferences. That is evidence of limited detection of the intervention, not proof that participants were unaware of every way the agent affected their work. That combination matters more than the headline percentage, because it removes the usual defence. An expert who knows they were swayed can discount the advice next time. An expert who experienced the sway as their own judgment has nothing to correct.
What the result does not show
Anthropic lists the limits plainly, and they are real. The lean was strengthened on purpose: the hidden instructions made the agent hold a preference and voice it reliably, which real agents may or may not do. This was one task, in one field, with one model, lasting about an hour, performed by analysts who knew causal inference well and farming not at all. In their own work they would know the subject better, hold firmer views, and have colleagues and stakeholders to argue with. Across four analytical decisions, participants selected the agent's randomly assigned preferred option 86% of the time. Because the preference was randomized, the high alignment provides evidence that the agent's expressed position influenced decisions. The 86% figure describes observed agreement with the assigned lean, not the size of a treatment effect or a rate that can be generalized to ordinary workplace use.
The final recommendation also moved far less than the working decisions did, 54% against 36%. Anthropic's reading is that once an estimate existed, the recommendation mostly followed from whether it cleared the board's threshold. But that is a narrower comfort than it looks. The estimate itself was built from the four decisions the agent had already shaped. Using only the four randomized analytical preferences, a statistical model predicted 82% of final recommendations, versus 61% for always predicting the most common outcome. This calculation excluded analysts who recommended a one-year extension.
Analysts who used the agent completed the task faster and made fewer serious methodological errors than the eleven who chose not to use it. Anthropic cautions that the groups were not directly comparable: participants selected whether to use the agent, and those who declined were less experienced on average. The comparison is descriptive and cannot establish the causal benefit of AI assistance. Anthropic is not arguing for less AI assistance. Its suggestions point the other way: build agents that argue the strongest case for each option, or that ask the expert's view before offering one, and have institutions decide in advance which decisions genuinely require a person's own judgment, then protect the time it takes to form one.
Sources
- [1] Anthropic, “How AI agents shape experts' judgment calls” (Herdağdelen, Joren, Tamkin; published October 27, 2026) — the primary source, read in full for this article: study design with five randomized decisions, the 86% match rate and per-decision figures, the 54%/36% recommendation split, the 82% prediction from the four analytical leans alone with the extension exclusion, the 38-case switch result with its confidence interval and imprecise caveat, the 77%/15% echo and credit figures, the survey details, the no-agent comparison with its comparability warning, the stated limitations, and the 112-of-123 default-preference figureRead source
- [2] Decrypt, “AI Can Quietly Influence Experts' Decisions, Anthropic Study Finds” (October 2026) — secondary coverage confirming the headline figures and Anthropic's framing of the human-in-the-loop assumptionRead source