Rethinking "Human in the Loop" for Today's Enterprise
About the Author
Rajeev Ronanki
Chief Executive Officer
Review queues cannot keep pace with AI agents running 160 actions an hour. Rajeev Ronanki argues human judgment belongs earlier, where agent authority is granted, and lays out three steps leaders can take this quarter.
Ask your executives: of all the AI agent recommendations made in the organization, how many did they overturn in the last six months; and how many of those reversals turned out to be right?
Almost no leader can answer either question. That shines real light on what “humans in the loop” currently buys, and it’s why most companies should widen what their agents are allowed to decide. It needs to follow a trusted process, beginning with measurement of a reversal rate: how often a person overturns an agent, and how often they are right in doing so. In addition, some decisions must stay with a human permanently.
In July, an unreleased model from OpenAI got out of its test environment and into the systems of Hugging Face. About 1,200 bots ended up coordinating on a message board nobody had sanctioned, and together they took an estimated 17,600 actions over roughly four and a half days. The behavior began on July 8, and OpenAI did not detect it until July 19. Fifteen state attorneys general demanded document preservation; Alabama issued a subpoena, and the FBI got involved. The response from the business community was quick and close to unanimity: make sure there is a human in the loop.
Before accepting that advice, ask whose signature would have stopped this breach, and others like it. OpenAI had reduced its cybersecurity guardrails for that test and containment failed. The agents worked at 160 actions an hour, around the clock, for four and a half days. No approver, senior or otherwise, could be counted on to catch that from a review queue.
Keeping a human in the loop is the right instinct aimed at the wrong moment.
The oversight most companies buy is a signature
Samir Passi and Ranjit Singh published a primer for Data and Society on July 29 called The Oversight Fallacy: Why AI Agents Require More than Humans-in-the-Loop. Their argument is structural, where oversight carries meaning only when four conditions hold together for a human overseer. They have to know what the system can and cannot do, see what it is doing, hold real control over its behavior, and be able to step in before damage spreads.
Today, agents cascade actions across systems faster than those four conditions can come together. When an organization puts a human at the end of that chain, the individual is given responsibility and expectations, often without any of the four conditions.
On August 31, Cequence Security and EMA published Agents Without Guardrails, a survey of 202 enterprise IT and security leaders at companies with 1,000-plus employees. Close to 94% of those leaders were confident their agents hold no more access than they need, and 33% provisioned them that way.
Close to 94% of those leaders were confident their agents hold no more access than they need, and 33% provisioned them that way.
Cequence Security
Asked about production, 65% reported an agent acting outside its intended scope, and 29% reported measurable damage from it. Just one in three say they can contain an action considered out-of-scope within minutes, while 55% need hours plus manual steps. In 4% of cases, the first sign of trouble reached the company through a customer, which is a small share of incidents and a complete failure of detection in every one of them. Because these figures come from humans running these programs, rather than instrumentation, it most likely understates the problem, as no one reports about incidents they’ve never detected.
Christopher Steffen, VP of research at EMA, called it the space "between what's written down and what's enforced when an agent takes an action nobody approved."
None of this qualifies as an early-stage risk. Nearly half, 46%, already run agentic AI across multiple departments in production, and 31% of pilots were abandoned with credentials still live. Those controls exist on paper and not in the runtime, so adding a human approver only improves the paperwork.
Sage and IDC surveyed more than 2,000 senior finance leaders in July and found 49% spending fifteen or more hours a week checking AI output. That’s the real cost of automation, sitting in nobody's automation business case
Programmer Simon Willison, co-creator of the Django Web framework, named the challenge: an AI doing malicious things, aware that another AI is watching it, can try to trick the watcher. His conclusion: a monitor produces evidence, which must abide by an authority model that a human has signed off on.
Humans in the right loop
We have humans standing in the wrong place, holding responsibility for work nobody gave them the means to evaluate.
Put the human where authority is granted rather than the decision is executed. Before an AI agent touches anything, a named person settles four things: the outcome that agent must answer for, the evidence it must hold before it may act, the decisions it may never touch under any circumstance, and the conditions that revoke its authority. After that, the agent decides, and every call it makes is written into a record with the evidence attached and the reasoning preserved.
That’s still human judgment, applied sooner by someone with the standing to apply it. What disappears is the theater of ratifying work you did not do and cannot evaluate in ninety seconds.
I have aimed at the wrong target myself. During COVID, at a large health plan, we set out to automate half of our contact center work. The number was clean and measurable. What I never asked was what those calls were telling us. Where was care breaking down, and which questions did people keep repeating week after week? A goal shapes what an organization pays attention to, and I had aimed ours at volume. I recently raised this in a talk at FARR Summit called "Human in the Right Loop," because the same sequence shows up in how companies scope agents now. The authority question gets asked after the automation target is set, when it belongs first.
What I never asked was what those calls were telling us. Where was care breaking down, and which questions did people keep repeating week after week?
Raj Ronanki, CEO, Signal Labs
Security researchers arrive at this same answer from a different direction. Nathan Hamiel of Kudelski Security told Science News that "models by themselves don't do anything," and that "it's when we give them tools, access to systems and autonomy, that things can be dangerous." Jer Crane, a cofounder of Pocket OS, discovered in April that an agent had deleted his company's live data and its backups together. The verdict: ultimately, the blame lies with him. Every one of those judgments happens before the agent runs.
Not every agent, and not every decision
The maximalist version of this argument is wrong. Repricing a familiar contract under rules already set is different from approving a first-of-its-kind exception in a regulated market.
Calls that cannot be reversed, that have no precedent to measure against, or where being wrong lands on someone's safety should stay with a named person, and not an agent. That should be for long periods of time, in some cases permanently, as accountability must connect to someone accountable for it.
The benchmark argument
This argument was put to the test “on the road,” with autonomous vehicles, and the way that debate resolved is instructive. Waymo reported in June that it had driven more than 220 million autonomous miles through March, with 94% fewer crashes causing serious or fatal injuries than human drivers. A month later, researchers at Hunter College's Sam Schwartz program measured the same fleet against professional for-hire drivers rather than the public and found Waymo's rate of serious injuries ran 1.5 times higher. Waymo challenged the analysis and the researchers withdrew it within days. A revised version, released weeks later, found 62% fewer injury crashes and 87% fewer serious or fatal ones.
Someone proposed a benchmark, and the comparison turned out to be wrong; then, after the correction was made public, the numbers moved. What nobody proposed at any point was a supervisor in the passenger seat approving every lane change. The industry moved — from whether a human should intervene to what standard the machine must beat.
Every enterprise deploying AI agents must consider the baseline their quality is measured by, and what happens when someone shows that baseline to be wrong?
Widening the boundary on evidence
The boundary itself moves, expanding as evidence accumulates. An enterprise that keeps a record of its decisions can show the error rate on a given call. It can then widen that agent's authority with the board or other leaders watching. Without that record, the same conversation happens in a conference room and gets called governance.
How much a company can safely hand to its agents is becoming a competitive difference. Three things decide it: what each agent may decide, how much proof it needs before acting, and what happened last time. Companies that record those points can widen agent authority on evidence. Everyone else widens it under pressure and best-guessing.
Three things to do this quarter
Inventory your approvals and measure their yield. Start with the three highest-volume approval queues anywhere in the company. For the last seventy-five or more decisions in each, how many did a human reverse, and how many of those reversals proved correct? Few executives have run this, and the result settles the internal argument faster than any memo.
Move the human decision earlier and write it down. For every enterprise agent in production, a named owner sets the outcome, the evidence threshold, the never-delegate list and the revocation conditions, on one page, before the next deployment. Then bind that agent to a scope with an end date, checked when action starts rather than once at provisioning. Most companies check an agent's authority once, at provisioning, which goes a long way toward explaining the 65% who have watched an agent wander.
Choose your benchmark in public. Decide what the agent's decisions get measured against, name it in front of your board, and publish the comparison every quarter. That argument is the one piece nobody can delegate, and a company that has never had it is governing on assertion.
What comes next
"Human in the loop" is here to stay, so the meaning is worth settling. Attach it to the individual who sets an outcome, decides what evidence the agent must hold, names what it may not touch, and can pull its authority back. Defined that way, the phrase lets agents decide far more than they do today. The current definition puts a person at the end of a queue, and at 160 actions an hour that person is already outrun. A company that routes everything through a review queue has chosen to be slow while feeling careful. Decide what promise you are making before someone asks you to prove it.