Human-on-the-Loop vs Human-in-the-Loop: Supervise or Approve

Human-on-the-loop is a design pattern where an AI system acts on its own and a person supervises it: monitoring what it does, sampling what it produced, and holding the authority and the means to interrupt it. The human watches the path rather than standing in it.

That is the whole difference from human-in-the-loop, where the agent prepares an action and waits for a person to approve it before anything commits. In the loop, the system cannot move without you. On the loop, it moves and you can stop it.

Most teams building agents reach for in-the-loop by default, and for irreversible work they are right to. Applied to everything, it produces a review queue nobody can keep up with, and a queue nobody can keep up with gets approved without being read. For low-risk automation — the reversible, bounded, high-volume majority of what a business automates — on-the-loop is the better design.

The short answer

Use human-in-the-loop when the action is irreversible, or when a rule requires that a person decide. Use human-on-the-loop when a wrong action can be caught and undone before it costs anything you cannot get back.

The test is not how much you trust the model. It is arithmetic on two windows: how long until you would notice a bad action, and how long until that action does damage you cannot reverse. If the first window is reliably shorter than the second, supervision is enough. If it is not, you need a gate, and no amount of model quality substitutes for one.

Where the human sits

The three positions predate language models by decades. They come from supervisory control, the study of how operators run systems that mostly run themselves, and they describe a human's relationship to a decision loop rather than a piece of technology.

• In the loop. No action is taken without approval. The human is a required step, and the system runs at human speed.
• On the loop. The agent acts and reports. The human receives status, samples the output, and intervenes when something looks wrong. The system runs at machine speed.
• Out of the loop. The agent acts and nobody is watching. Occasionally deliberate. Usually accidental.

The third is worth naming because it is where teams land without meaning to. An on-the-loop design whose monitoring was never built, or whose alerts nobody reads, is an out-of-the-loop system with better documentation.

How they differ

Low-risk work belongs on the loop, not in it

Reviewer attention is a fixed budget. Your team has some number of people who can look at some number of cases a day, and that number does not grow because you added an approval gate. What changes is how the budget gets spent.

Gate everything and the budget spreads evenly across work that never needed it, and reviewers get faster rather than sharper. The EU AI Act names the mechanism directly, requiring that oversight measures address the risk of automation bias: the tendency to over-trust an automated system's output, which is roughly what a reviewer develops somewhere around their four-hundredth correct extraction.

An approval gate that is always approved is not a control. It is a record of attention being spent, and it is worse than no gate, because the organisation now believes a review is happening.

The version that works is the one the human-in-the-loop post already argues for: put human attention exactly where the risk is, and nowhere else. On-the-loop is what "nowhere else" looks like in practice: oversight that scales with the system rather than with the number of reviewers you can hire.

What counts as low-risk

Low-risk is not a feeling about the workload and not a claim about model quality. It is five properties, and every one of them is checkable.

• The action is reversible. Reversible meaning a path that already exists and has been tested: the record restored, the write rolled back, the message retracted. An audit log is not a reversal. It tells you precisely what you cannot take back.
• You would notice in time. Detection has to be faster than damage. If your monitoring is a report someone reads monthly, assume you would not notice.
• The blast radius is one case. One record, one message, one claim. An action that can touch an entire table, a whole customer segment, or a system other teams depend on is not one case.
• The error cost is linear. Ten wrong classifications should cost about ten times one. If a single error can trigger a step change — a regulatory finding, a lost account, a story — then the average case is not the one to plan for.
• No rule requires a person to decide. GDPR Article 22 gives people the right not to be subject to a decision based solely on automated processing where it produces legal effects or similarly significant ones. The EU AI Act goes further for parts of its high-risk category: biometric identification requires at least two competent people to separately confirm an identification before a deployer acts on it. Where a rule names a human, accuracy is not the question being asked.

Five gates, one per property. Start at human-on-the-loop and try to fail out of it, because that is where reversible, high-volume work belongs until something specific disqualifies it.

A workload that fails a gate is not condemned to in-the-loop forever. Three of the five failures describe machinery you can go and build.

What on-the-loop actually costs

Human-on-the-loop is not human-in-the-loop with the approvals deleted. It trades per-action review for a different set of work, and the trade is only good if you do the work.

Detection you did not have to go looking for. Alerts that arrive, not a dashboard somebody could open. The window that matters is time-to-notice, and a dashboard's is unbounded.

A reversal path that exists before you need it. Built and tested in advance, not designed during the incident by whoever is awake.

A sample rate somebody reads. Pick a rate, hold to it, and feed what it finds back into the system. Where volume makes that impractical, an LLM-as-a-judge can widen the sample, but only once it has been calibrated against human ratings.

A stop, and a named person who can pull it. Named as in a person rather than a team alias, and a stop they can operate without shipping a deploy.

Article 14 of the EU AI Act, for the high-risk systems it covers, requires that overseers can monitor the system well enough to detect anomalies and can interrupt it through a stop button or similar procedure that brings it to a halt in a safe state. That is a fair description of the minimum on-the-loop machinery whether or not the regulation reaches you. Teams that hear "on the loop" as "in the loop, minus the paperwork" build none of it, and end up with the throughput of autonomy and the oversight of nothing.

Both, in the same workflow

The choice is per action, not per system. Most real workflows contain both, and the boundary tends to fall exactly where something becomes irreversible.

In the insurance claims work, reading inbound email, parsing PDF and Word attachments, and scoring the confidence of each extracted field all run without anyone approving them. The gate sits at the commit: no claim reaches the system of record without an adjuster having had the chance to catch a bad extraction.

In the legal email work, classification, routing, and drafting run the same way. Every message is still approved by an attorney before it sends, because sending is the step that leaves the building.

In both, the labour-intensive majority of the pipeline is supervised rather than gated, and the gate sits on the one action nobody can retract. That is the shape to aim for.

How XY Space approaches it

We start clients in the loop and earn the way out. A new deployment gates generously, because at the start you have no evidence and the approval log is how you collect it. Once a class of action has a stable approval rate and a reversal path that has been exercised, it moves to sampling, and reviewer attention moves with it to whatever is still uncertain.

What makes that move defensible is that it is made on a record you already have, rather than on how the system has felt over a good month. The same record supports moving back the other way, which is the part teams forget to plan for.

If you are deciding which of your automations deserve a gate and which deserve a supervisor, that boundary is most of what we design. Talk to us.

Sources

• Anthropic: Building effective agents
• EU AI Act, Article 14: Human oversight
• GDPR, Article 22: Automated individual decision-making
• Safety, Trust, and Ethics Considerations for Human-AI Teaming in Aerospace Control