# METR and Redwood Find Monitoring Gaps in OpenAI Pilot

> Two nonprofit evaluators found ways to evade OpenAI’s internal monitoring during a pilot. OpenAI says it patched vulnerabilities affecting production safeguards.

Oossa · 2026-10-22 · https://oossa.com/en/metr-and-redwood-find-monitoring-gaps-in-openai-pilot

Nonprofit evaluators METR and Redwood Research say they found several previously unknown weaknesses in OpenAI’s internal system for monitoring whether AI models behave in line with human interests and values. One exploit could let AI agents keep “hundreds” of recent actions from being monitored, the groups said.

The findings came from a pilot in which one employee from each organization received access to OpenAI’s codebase, documentation and some technical staff. The employees were chaperoned and used OpenAI-issued devices that stayed on the company’s premises.

## What the evaluators tested

The three-week exercise ran from July 21 to August 11. METR and Redwood said they tested the monitoring system as it existed on and before July 21, while OpenAI was expanding it during the pilot. OpenAI told the evaluators it had patched the vulnerabilities affecting its production safeguards, according to the groups’ release.

The evaluators shared a 61-page report with OpenAI, but published only a high-level account. They said the pilot was not intended to provide robust public transparency about the monitoring system, and that they expect to share more detail about future exercises.

## A test of outside access

The pilot also tested what it might look like to give independent researchers deep access to OpenAI’s systems. CEO Sam Altman pledged “employee-like access” for outside evaluators in September. OpenAI said earlier in October that it was finalizing contracts with third-party safety assessors.

The model for independent evaluation is still taking shape. The report does not publicly describe the vulnerabilities in detail, and the available account does not establish whether outside researchers independently verified that the fixes work.

## The facts

- METR and Redwood’s pilot ran from July 21 to August 11.
- One employee from each group accessed OpenAI’s code, documentation and some technical staff, under supervision.
- The evaluators said one exploit could hide hundreds of recent actions from monitoring.
- The groups shared a 61-page internal report with OpenAI and published a high-level summary.
- OpenAI told the groups it had patched vulnerabilities affecting production safeguards. 

## Why it matters

For people relying on OpenAI’s safeguards, the pilot found that some recent agent actions could evade monitoring, while OpenAI says it has fixed the production-impacting weaknesses. The public summary does not provide enough detail to assess the fixes independently, so their effectiveness remains unclear.

## Sources & references

1. [Third-party evaluators find vulnerabilities in OpenAI monitoring system during test pilot](https://www.cnbc.com/2026/10/22/metr-redwood-research-open-ai-test.html) – CNBC, 2026-10-22

Last updated: 2026-10-22