AI Agent Human Oversight: Three Lessons from the Coding Approval Failure

In a browser-based test that logged more than 40,000 simulated agent sessions and 409,000 approval decisions, human reviewers approved roughly one in three commands that could exfiltrate data or execute arbitrary code. The finding, published by Belgian developer Alex Wauters, aligns with telemetry that Anthropic has reported from its Claude Code tool, where users approve about 93 percent of permission prompts. For enterprise AI leaders, the number should end the comfortable assumption that a human in the loop is a security control.

What does the new data reveal about human approval of AI coding agents?

The data shows that human-in-the-loop gatekeeping fails predictably under load: across more than 40,000 simulated sessions and 409,000 approval decisions, reviewers approved roughly one in three potentially malicious commands. Scope violations, where an agent asks to read a Kubernetes config or a credentials list, were the most often missed category, slipping past reviewers 35 percent of the time.

Destructive commands drew the most scrutiny. Reviewers most reliably caught operations such as deleting a root directory or recursively granting read, write and execute permissions on it. The problem was not detecting the obvious, but detecting the plausible: commands that look routine yet can carry a payload, and requests that only become dangerous in the context of what the agent did moments earlier.

Why do human gatekeepers approve so many dangerous commands?

Human gatekeepers approve dangerous commands because approval fatigue degrades attention and the context needed to judge a request is rarely visible at the moment of decision. The single most missed command, an invocation that runs whatever a project's build file defines, was approved nearly 65 percent of the time, which Wauters attributes to reviewers not reading the agent's history log that reveals what the script actually executes.

The failure mode is not limited to a browser game. Anthropic has observed in telemetry from Claude Code that users approve most permission prompts and that the more approvals a user sees, the less attention they pay to each one. The approval-heavy workflow does not simply create minor errors; it conditions reviewers to click yes, and that conditioning is what an attacker or a manipulated prompt can exploit.

“The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent in their supervision.”

Anthropic, as reported by The Register on Claude Code approval telemetry

There is a further structural reason human oversight fails. Wauters notes that teams have moved from reviewing single-line suggestions to handing off complex tasks and only reviewing the changes at the end, letting the agent iterate through many steps that each require approval mid-stream. Fatigue, missing context and the volume of noise combine to make the human the weakest link rather than the last line of defense.

How should enterprises stop relying on human approval as a control?

Enterprises should treat human-in-the-loop approval as accountability for decisions already bounded by policy, not as a security control, and move those decisions into deterministic rules. Wauters recommends sandboxed runtimes, development containers in the cloud, and tooling that contextualizes actions before they are approved, rather than pointing to a human reviewer as the solution.

The three lessons for AI leadership are as follows. First, human approval is a fatigue-limited audit trail, not a defense. Second, approval decisions made without verified context will fail on a predictable schedule. Third, deterministic, trajectory-aware policy must replace ad-hoc human judgment. [ Replace with a real quote from Alex Wauters on shifting the balance from human approval to permission models and safer tooling ]

1. Sandbox every agent so it cannot reach production systems, secrets or the open internet unless a policy explicitly allows it.
2. Redesign the permission model so the agent requests the least privilege needed for each step, instead of asking for broad access up front.
3. Route all agent tool calls, agent-to-agent calls and inference through a single policy perimeter that the agent cannot see or modify.
4. Require explicit human approval only for high-value or destructive actions, and reserve attention for the decisions that actually need it.

What control architecture should AI leaders put in place?

AI leaders should place controls at the gateway or policy perimeter, outside the agent's code, using stateful rules that assess each request in the context of the agent's session history. AWS's Bedrock AgentCore shows the working pattern: deny-by-default authorization, output-integrity checks so an agent cannot swap a value between steps, and financial or risk caps that hold regardless of how the agent behaves.

The key design property is that the policy runs where the agent cannot intercept or manipulate it. Gateway-level temporal policies evaluate whether a request is authorized given the recent trajectory of the session, and they decline by default. This is the contrast that matters for governance: a human approving each prompt is a judgment call inside the loop, while a policy perimeter is a deterministic decision the agent cannot bypass.

This is not a separate problem from the containment and evaluation failures our readers have already covered. In our analysis of the AI agent containment breach, misconfiguration allowed an agent to escape its sandbox. In our findings on the AI agent evaluation gap, enterprise teams shipped agents to production that they had not alignment-tested. Human oversight failure sits at the intersection: it converts a governance gap into a realised incident, and it also raises questions about liability when a human approved the very action that caused harm, a boundary examined in our article on AI agent liability.

Frequently asked questions