Article·22 October 2026·By Christina Alexandrou

A Human in the Loop is not Enough

Human agency and machine autonomy in the governance of agentic AI

Where the election disappears

In her 2026 State of the Union address, European Commission President Ursula von der Leyen observed that artificial intelligence is fast becoming a foundational layer for the European economy and security, and called for AI with human agency. The two claims are more closely connected than they first appear. Where AI operates as a tool, an individual elects whether to use it, and human agency is protected by that election. Where AI becomes infrastructure, the election disappears.

The question of governance therefore shifts from “Should humans use AI?” to a more demanding one: how can meaningful human agency be preserved when AI is already part of the environment in which decisions are made?

Four levels of action autonomy

Nowhere is this question more pressing than in agentic AI, systems that no longer merely answer questions or rank options but undertake actions. The OECD's 2026 work provides a useful vocabulary, distinguishing four levels of action autonomy: no-action autonomy, where the system recommends but the human decides; low-action autonomy, where the system acts only upon human approval; medium-action autonomy, where the system acts unless a human intervenes; and high-action autonomy, where the system acts without human involvement.

This ladder describes what AI can do. However, it does not answer the more important governance question: how much human control must be retained at each level?

A human being formally involved in a decision does not necessarily mean that the human has exercised meaningful agency. Consider a system that produces a recommendation, which a person briefly reviews and approves. Formally, the human has made the decision. Substantively, however, the human may simply have ratified the machine's judgment.

What the law already anticipates

European law anticipates the concern with Article 14 of the Artificial Intelligence Act (Regulation (EU) 2024/1689), which requires that the person assigned to oversee high-risk systems be enabled to correctly interpret the system's output, to remain aware of their own possible tendency to rely automatically or excessively on that output, and to disregard, override or reverse it in particular cases. This provision addresses automation bias, the tendency to follow a machine's suggestion even when other information points the other way. Its explicit recognition of this risk reflects the fact that a human being present in front of a screen is not necessarily exercising independent judgment.

These requirements apply to stand-alone high-risk systems from 2 December 2027, and to AI embedded in regulated products from 2 August 2028, under the Digital Omnibus on AI (Regulation (EU) 2026/1744). The Act leaves providers and deployers considerable discretion over what awareness and effective oversight is required in practice, so that implementation depends heavily on local judgement.

What the evidence shows

The empirical evidence reinforces the point. Research on automation bias shows that explainability mechanisms do not necessarily prevent over-reliance; in some circumstances, explanations can increase perceived acceptability without improving decision accuracy. Experimental evidence has also found that making AI errors more difficult to correct reduces the likelihood that people will correct them. In computational pathology, the use of AI to analyse tissue samples such as biopsy slides in support of diagnosis, AI assistance improved overall diagnostic performance while still producing instances in which initially correct human judgments were overturned by erroneous machine recommendations. Across these settings, the same problem emerges: human presence can be preserved procedurally while human agency is weakened substantively.

Four conditions for a meaningful human role

This article therefore proposes four conditions for a meaningful human role.

Understanding: the person must be able to understand what the system recommends and the limitations relevant to the decision.
Authority: the person must have genuine technical and organisational power to reject, modify or reverse the system's recommendation. An override button means little if using it is practically discouraged.
Capacity: the person must have sufficient time, expertise and information to exercise independent judgment.
Accountability: it must be clear who bears responsibility when the system is wrong.

These four conditions become increasingly difficult to satisfy as autonomy increases.

How the conditions hold up at each level

At no-action autonomy, the system recommends and the human decides. Agency can therefore remain relatively direct, provided the human has the capacity to evaluate the recommendation rather than merely accept it.

At low-action autonomy, the system acts only after human approval. Here, formal oversight can easily become symbolic. A human approval step satisfies little if the person has only seconds to review the output or lacks the information needed to challenge it. A recent experiment with 2,784 participants shows how the design of the review itself shapes judgment. Participants checked AI-extracted values in corporate emissions reports, and when correcting an AI error required the extra step of typing in the correct value themselves, they made fewer corrections and accepted more incorrect suggestions. A review step that exists on paper can become a rubber stamp when correction is made harder. Evidence suggests that concrete briefings about system errors can increase verification, whereas generic reminders about responsibility are less effective.

At medium-action autonomy, the problem changes. The human no longer approves every action but must detect problems within a continuing stream of autonomous activity. Understanding therefore requires knowledge of what the system has already done, capacity requires sufficient resources to monitor it, and authority requires a realistic opportunity to intervene before harm occurs. The theoretical ability to stop a system is not equivalent to having a meaningful opportunity to notice that it needs to be stopped.

At high-action autonomy, the system acts without human involvement at the point of decision. Human agency cannot therefore be secured through approval or intervention at that moment. It must instead be exercised earlier, by determining where the system may operate, which decisions it must never make, what constraints govern it, and how failures will be detected and attributed. This is consistent with lifecycle-oriented approaches such as the NIST AI Risk Management Framework, which treats oversight as a feature of system governance rather than merely a person standing at the final decision point.

Where agency relocates, not disappears

The central pattern is therefore not that human agency disappears as AI becomes more autonomous. Its location changes. At lower levels of autonomy, agency lies primarily in deciding. As autonomy increases, it moves toward monitoring and, eventually, toward system design and governance.

These four conditions carry consequences for regulation, organisations and system designers. This article argues that regulators should look beyond the mere presence of a human and ask whether that person has the understanding, authority, capacity and accountability necessary to exercise meaningful control. Organisations should assess these conditions before deployment, allocate sufficient time and expertise for verification, and design workflows that make correction genuinely possible. System designers should focus not simply on explaining AI outputs, but on creating interfaces and processes that encourage independent verification.

From human presence to human agency

The appropriate level of autonomy for a particular application should therefore be understood in relation to these four conditions. The question is not simply whether a human remains somewhere in the process, but whether that human can realistically exercise meaningful agency over the decisions that matter.

The standard for AI governance should therefore move from human presence to human agency. A human in the loop is not enough. What matters is whether the human has the understanding, authority, capacity and accountability to make the loop meaningful.

Sources

Back to all posts

Join Us

A Human in the Loop is not Enough

Join the Global AI³ Network