In an Anthropic experiment, 50 professional analysts often adopted an AI agent’s preference on judgment calls in a data analysis—and most did not credit the agent for shaping their choices. The analysts used Claude Code to assess whether a fictional farming cooperative should expand a paid advisory program.
The task left four analytical choices open, including how to measure farm productivity and which statistical method to use. For each choice, Anthropic secretly set the agent’s preference by coin toss. The agent was told to make the strongest honest case for its preferred option, but not to mislead or hide data. Across the four choices, analysts matched the agent’s preference 86% of the time. Anthropic says that when analysts had stated their own view first, they switched away from it 60% of the time when the agent disagreed, compared with 6% when it agreed.
The early choices shaped the final recommendation
The agent had less influence on the final decision about expanding the program. When it favored expansion, 54% of analysts recommended it; when it favored the other option, 36% did. A one-year extension counted as not expanding. The Decoder reports that this difference was not statistically significant. But the earlier analytical choices affected the estimated productivity gain, which mattered because the fictional board’s expansion threshold was roughly 3% to 4%.
Afterward, 77% of analysts used at least one of the agent’s arguments in their explanations, sometimes in nearly the same words. Only 15% mentioned AI as the source. In a survey, 52% said the final recommendation felt more like their own than the agent’s, while another 35% described it as mostly or entirely their own.
A useful tool, with limits to the findings
The experiment does not show that analysts always defer to AI in routine work. Anthropic deliberately gave agents clear preferences to measure their influence. The company notes that Claude Opus 4.8, without those assigned preferences, made the same five choices in 112 of 123 runs—evidence that a model may still have consistent leanings of its own.
The analysts using the agent finished in a median of 65 minutes. The Decoder reports serious flaws in 6% of their analyses, compared with 55% among the 11 people who worked without the agent. That comparison is uncertain: the no-agent group was small and somewhat less experienced. Anthropic says the study’s results may not reflect real-world use, where agents may not state a preference so strongly.
Why it matters
For people who use AI to analyze data or advise on policy, the study suggests that a tool can shape the assumptions behind a recommendation, not just do the coding. It does not establish how often this happens in everyday work, but it shows why reviewers may need to examine key choices as well as the final answer.
