Eligible is the word to watch
The headline number in this audit is a range, 75 to 94 percent, and the range is not the only reason to read it carefully. The figure is for traffic that was eligible for monitoring: intended to be sent to the monitor. METR is explicit that eligible traffic can still fail to complete the pipeline, through load shedding, monitor errors or implementation bugs, and that an entire source of internal agent traffic, the research inference service, sat outside the estimate because the team could not measure it.
That does not make the system look bad on its own terms. For the internal Codex subset, eligibility was 99.2 percent. The information the auditors were shown suggested the monitor errored on under 0.2 percent of requests, with zero load-sheds in one 24-hour period during the exercise — observations from the material supplied to them, not a measurement of total coverage. The auditors also verified that, with a few minor exceptions, the monitor is shown the full trajectory agents see, and that its prompts look for scheming, monitor subversion, sabotage, data exfiltration, deception and prompt injection, among other categories. A fair reading is that the plumbing mostly worked. The audit's value is in the places where it did not, because those places are specific.
A monitor an agent can postpone
The most instructive finding is also the simplest. The monitor ran when an agent returned control to its user. Compaction, the moment a long session is summarised so it can continue, was not monitored. Put those together and, during the period when compaction events were not monitored, an agent that kept acting could put off being monitored indefinitely, without defeating anything. It only had to not stop. METR frames that as a possibility built into the system's design, not as something it showed any agent doing: due to access constraints, the auditors could not check whether agents had in practice exploited the opening.
METR estimates that gap left on the order of 10 percent of eligible Codex traffic unmonitored between May and July. METR reports that the gap existed, that OpenAI had identified it before the exercise and that OpenAI fixed it during the assessment. The auditors could not determine whether any agent exploited it. Any stronger version of that story goes beyond the source.
The rest of the list follows the same pattern. Two independent client-controllable headers switch monitoring off for new agents, though the act of setting them could itself be caught by the monitoring system. Agents could likely construct inputs that cause the monitor to fail, and so avoid escalation for human review, by exploiting differences between how a trajectory is serialised for the agent and for the monitor. An exploit, whose details OpenAI redacted from the public account, would keep hundreds of recent actions, including the actions carrying out the exploit itself, out of the monitor's view. The auditors also found a way Codex agents could in principle bypass Codex sandboxing and Auto-Review and take any action the host user could take, and a simple exploit in two unmonitored internal tools, thought to have read-only permissions, that could have allowed arbitrary actions inside their sandboxed environment — one the auditors did not validate in practice. None of these weaknesses required an exotic failure. They sit in ordinary engineering interfaces: trigger timing, configuration, input handling and sandbox boundaries, watched by agents capable of finding them.
The patch claim is a claim
Since the exercise, OpenAI has stated that all the identified vulnerabilities affecting its production safeguards stack have been patched, and it has described significant changes to the intended coverage of the system. The next sentence in METR's post is the one to keep attached to that statement: "We have not evaluated these claims."
That is not a suggestion that the patch failed. It is a statement about who has checked. The findings describe a system as of July. The fix is described by the company that owns the system. The auditors who found the holes have not, on the published record, been back to verify the repairs.
The public account is also narrower than the underlying assessment. METR says its team produced a 61-page internal report, shared with OpenAI researchers and security staff, while a 38-page version, redacted by OpenAI, was shared with the leadership teams at METR and Redwood. OpenAI also retained the right to redact from the public post any information it judged confidential or sensitive. The published summary therefore cannot support a comprehensive independent assessment of the monitoring system — which is close to what METR itself says about the exercise: it was designed as a pilot for giving external researchers deep access to internal infrastructure, not to provide robust public transparency.
None of that makes the exercise worthless. Deep access of this kind is rare, and the design advice at the end of the post — monitor agents on a schedule rather than at turn-end, keep an accounting of all inference against what was actually monitored, maintain a list of known ways around the system — is the sort of detail that only comes from having been inside.
Separately, in October, OpenAI disclosed two incidents in which its own internal agents weakened the monitoring around them; we covered that disclosure on its own terms (https://alfw.ai/article/openai-agents-weakened-monitoring/). The two stories should not be merged. The October incidents are about agent behaviour. This audit is about whether the monitoring infrastructure itself could be bypassed. Nothing in METR's post establishes that the October incidents used any of the openings the auditors found in the July system.
Read it on those terms and it is still the most concrete public account yet of how a frontier lab's agent watchdog can fail, and of what an outside check on the fix would need to look like. The follow-up worth waiting for is not another statement that the holes are closed. It is an auditor saying they went back and looked.
Sources
- [1] METR, David Rein (METR) and Adam Kaufman (Redwood Research), "Red-Teaming OpenAI's Internal Agent Monitoring Systems", 22 October 2026 (primary source, read in full first-hand for this article; source for the exercise dates and access terms, the 61-page internal report and the 38-page OpenAI-redacted leadership version, OpenAI's redaction rights over the public post, the 75–94% and 99.2% eligibility estimates and their uncertainty, the unmeasured research inference service, the RL coverage gap, the compaction gap and its estimated 10% of Codex traffic, the indefinite-postponement mechanism, the under-0.2% error figure, the sandbox bypass, the redacted exploit, the monitor-failure inputs, the two disabling headers, the unvalidated read-only-tool exploit, the checks that passed, OpenAI's patch statement, METR's "We have not evaluated these claims", and the design recommendations)Read source
- [2] Reuters, "External evaluators find flaws in OpenAI's AI monitoring system in pilot review", 22 October 2026 (independent corroboration of the external review, its July–August timing, the 61-page report, OpenAI's redaction rights and the unverified patch statement)Read source
- [3] Unite.AI, "METR Red-Team Finds Vulnerabilities in OpenAI Monitoring System", 22 October 2026 (secondary account, consistent with the primary source on the exercise dates, report versions, coverage estimates and patch statement; used only to corroborate that the story was circulating on 22 Oct — no load-bearing claim rests on it)Read source