Experts Overwhelmingly Defer to AI Assistants in Critical Decision-Making, Anthropic Study Finds
Professional analysts, typically relied upon for their independent judgment and specialized expertise, appear to be significantly susceptible to the influence of artificial intelligence. New research from AI developer Anthropic suggests that these experts defer to AI-guided preferences 86 percent of the time, a finding that raises profound questions about the future of human oversight in high-stakes environments.
The study involved 50 professional analysts who were tasked with advising a fictional farming cooperative. The participants were asked to evaluate an advisory program and determine whether it should be expanded. Throughout the process, the analysts were required to make four intermediate choices between two equally defensible options before arriving at a final policy recommendation.
What the participants did not know was that the researchers had intentionally programmed the AI assistants to lean toward a specific decision. This hidden preference allowed researchers to measure exactly how much the AI’s underlying bias shaped the human expert’s final output. In a neutral scenario where the AI exercised no influence, researchers would have expected a 50/50 split in the decisions made.
The results were striking. Beyond the 86 percent overall agreement rate, the study found that even experts with strong initial convictions were easily swayed. Among analysts who had formed a clear preference before consulting the AI, 60 percent changed their minds when the assistant presented a conflicting view.
The research also uncovered a significant gap between how analysts used AI and how they perceived that use. While 77 percent of the participants incorporated the AI assistant’s arguments directly into their written justifications, only 15 percent actually credited the technology as the source of their reasoning. This suggests that the influence of AI may be subconscious, with experts internalizing machine-generated logic as their own.
Researchers Amaç Herdagdelen, Hailey Joren, and Alex Tamkin noted that this tendency to defer to AI might be “hard to identify, even for the experts themselves.” They warned that if AI systems have built-in “priors” or biases, those preferences could quietly shape consequential policy decisions, including those related to AI safety and ethics.
To maintain meaningful human oversight, the researchers argued that organizations must change how decisions are audited. Rather than just reviewing the final result, oversight must focus on how those results are reached, helping experts articulate the assumptions and values they would otherwise leave unstated. “A process that lets experts discover their preferences may matter for empowerment as much as the decision itself,” the researchers wrote.
Anthropic’s findings arrive as the company remains at the heart of a global debate over “frontier AI.” The firm, which is reportedly preparing for an IPO that could be valued at up to $2 trillion, is under pressure to manage the potential dangers of the technology it develops. While proponents believe AI will usher in revolutionary productivity gains, the rapid pace of development has prompted widespread concerns about mass job losses and cyberattacks.
The study’s authors acknowledged certain limitations, noting that the research focused on a single task and utilized AI assistants with deliberately forced preferences. However, the data underscores a growing concern that as AI becomes more integrated into professional workflows, the “human in the loop” may be functioning more as a rubber stamp than an independent check.
