Anthropic Research Shows 86% of Experts Follow AI Recommendations
A new study indicates that professional analysts frequently override their own initial judgments when their AI assistants suggest a different course of action.
By Riya Zhao ·
Professional analysts relying on artificial intelligence tools are likely adopting machine recommendations at a high rate, according to research published Tuesday by Anthropic. The company found that experts made choices aligned with their AI assistants 86 percent of the time in a controlled test.
Anthropic designed the experiment to measure whether professionals maintained independent judgment when assisted by AI. The researchers recruited 50 professional analysts to evaluate a fictional advisory program for a farming cooperative. Each participant faced four decision points where two options were defensible. Behind the scenes, Anthropic programmed every AI assistant to prefer a specific choice, allowing the team to track whether the software steered human thinking.
The data revealed a strong pattern of deference. When an analyst had already formed an opinion before consulting the tool, 60 percent of them changed their decision if the AI disagreed. Furthermore, 77 percent of the analysts cited arguments provided by the assistant when explaining their final policy recommendations. Only 15 percent of the participants acknowledged the AI as a source of their thinking.
Amaç Herdagdelen, Hailey Joren, and Alex Tamkin, the researchers behind the paper, noted that this tendency to follow AI suggestions may be difficult for experts to detect themselves. They stated that the findings suggest an AI system's initial biases, or priors, could influence important policy decisions, including those concerning AI safety.
The researchers argued that oversight mechanisms need to focus on the process of decision-making rather than just the final outcome. They wrote that if human judgment is to serve as a meaningful check, supervisors must surface key decision points and help experts articulate unstated assumptions. They added that a process allowing experts to discover their own preferences may be as valuable as the decision itself.
Anthropic acknowledged that the study had limitations. It covered only one specific task, and the AI assistants were deliberately configured to favor a particular result. The study arrives as Anthropic prepares for an initial public offering valued at up to $2 trillion. The company and its competitors are currently at the center of public debate regarding the management of frontier AI risks, with supporters citing productivity gains and critics raising concerns about job losses and cyberattacks.
