The principle was public. The terms are private.
Last month the public statements were clean. Anthropic's chief executive said external evaluators should be able to publish key findings without editorial control. OpenAI agreed days later, in the language of editorial independence. Independent evaluation also made it into the White House Accord on Super Intelligence as one of four safety measures, although the accord did not say how the reviews should actually be run.
Reuters now reports what the private version looked like. According to two people familiar with the discussions, both labs sought significant control over what outside evaluators can publish, because embedded evaluator staff, working inside the labs, might see proprietary information and carry it to a rival. As recently as 5 October, Reuters reports, the position being sought was strict editorial control. By 8 October, talks between Anthropic and some evaluators appeared to have moved. That three-day shift is worth noticing: the terms were still being fought over last week, after the public endorsements.
Where the line gets drawn
The protective measures Reuters describes are concrete. Measures reportedly under discussion included lab employees observing embedded evaluation teams and restrictions on working inside competing labs simultaneously. Separately, Reuters reviewed contract language that can prohibit publication of confidential information without a lab's consent.
Each measure has a defensible version. Labs do have trade secrets, and an evaluator with floor access sees more than a test result. The problem is the overlap, flagged to Reuters by a person with experience of past dealings: what a lab counts as crucial intellectual property can be the same material an evaluator needs to judge a model's risk. When the two categories share pages, disputes over what can be disclosed may give the evaluated lab substantial influence over the public account of its own risks, depending on the contract and the evaluator's rights. Add the structural point made by former US Science Envoy for AI Rumman Chowdhury: evaluators depend on labs for the access that makes their work possible, and both sides know it. Leverage does not need to be exercised to shape a report. It only needs to exist.
What the labs say now
Both companies pushed back on the strongest reading, and their answers are not identical, which is itself informative.
Anthropic says evaluators it has previously worked with control their methods, criteria, conclusions and publications. For new evaluator firms, it reserves the right to comment on and approve reports while the working relationship develops. That is a two-tier system: independence as something earned through a working relationship, granted by the lab.
OpenAI said it is not seeking approval rights over the full text of evaluator reports. Its stated practice is narrower: redact confidential information, and let evaluators disclose whether major redactions were made. A right to disclose that a redaction happened is not a right to publish what was removed, but it is a check the Anthropic formulation, as reported, does not mention.
The one worked example
There is a recent case to measure this against. After autonomous agents powered by OpenAI models were involved in the incident that compromised Hugging Face's infrastructure, two safety nonprofits, METR and Redwood Research, investigated. What was planned as two days at OpenAI became six days over three visits, and a fuller set of transcripts arrived only on the last visit, after the researchers pointed to gaps. METR and Redwood said they adjusted the report's structure, emphasis, clarity and tone after OpenAI feedback, but maintained that nothing important to their conclusions was removed. They stood by their findings while acknowledging that preserving future access to developers influenced their drafting and redaction judgments. Read that last part twice. Even in the case everyone involved describes as a success, the evaluators were editing with the relationship in mind.
The evaluators have stated their own floor: a public letter last month, signed by more than 200 researchers, asks for independent control of findings, broad access and protection from retaliation. Faculty, the Accenture-owned firm named as Anthropic's first embedded evaluator, told Reuters that evaluators are only useful if people can trust what they report. The negotiation Reuters describes is, in effect, a fight over whether that trust is a property of the process or a concession from the lab. Until there are agreed standards or legislation, the labs that build the models also set the conditions under which the models get examined, and the examination is only as independent as those conditions allow.
Sources
- [1] Reuters (Aditya Soni), "Behind AI industry's call for independent reviews, a fight over what gets made public," October 13, 2026, via Northland News Radio syndicationRead source
- [2] METR — research and independent evaluations, including its published investigation of the OpenAI/Hugging Face incident: metr.org