Introduction

For the past year, one recommendation has appeared in almost every discussion about AI governance:

Put a human in the loop.

The idea sounds reasonable. If an AI agent wants to execute an action, a human reviews it before approval.

Problem solved.

Or is it?

Recently, Amazon Security VP Eric Brandwine challenged that assumption, arguing that Human-in-the-Loop (HITL) governance often fails under real-world operating conditions. His concern is not that humans are unnecessary. His concern is that humans are predictable.

They experience:

• Alarm fatigue
• Approval fatigue
• Normalization of deviance
• Attention drift
• Velocity mismatch

The same patterns have been observed in emergency rooms, cybersecurity operations centers, aviation, and industrial safety systems.

When humans are required to review large volumes of routine actions, vigilance inevitably degrades.

The question is no longer whether HITL is useful.

The question is:

What governance model actually scales as autonomous systems become more capable?

I believe the answer lies in a three-layer model built around Spec-Driven Development (SDD), Runtime Assurance, and targeted Human Oversight.

Why Human-in-the-Loop Breaks Down

Traditional governance assumes a workflow like this:

Agent → Human Approval → Action

This works reasonably well at small scale.

However, modern agentic systems may generate:

• Hundreds of tool calls
• Thousands of decisions
• Continuous workflow execution
• Multi-agent coordination

At that scale, human review becomes a bottleneck.

Worse, it becomes ineffective.

A reviewer who approves ten requests per day behaves very differently from a reviewer who approves a thousand requests per day.

Eventually, approval becomes routine.

Humans stop evaluating decisions and start acknowledging them.

The governance mechanism still exists.

The governance value disappears.

This is not a failure of people.

It is a failure of system design.

The Three-Layer Governance Model

Rather than relying on humans to approve every action, governance should be distributed across three layers.

Layer 1: Design-Time Oversight, Spec-Driven Development (SDD)

This is where humans create the governance foundation before the system ever executes.

This is also where Spec-Driven Development becomes critical.

Most organizations think of specifications as requirements documents.

In an agentic environment, the specification becomes something much more important:

The governance contract.

The specification defines:

Autonomy Boundaries

What the agent may do.

Examples:

• Read account balances
• Analyze transactions
• Generate recommendations

Negative Requirements

What the agent must never do.

Examples:

• Transfer funds externally
• Modify security controls
• Access privileged systems

Escalation Rules

When human intervention becomes mandatory.

Examples:

• Confidence below threshold
• High-value transaction
• Policy conflict detected

Safety Invariants

Rules that must never be violated.

Examples:

• Fraud flags cannot be overridden
• Spending limits cannot be exceeded
• Audit trails cannot be modified

Tool Governance

What tools and resources are allowed.

Examples:

• Approved APIs
• Approved MCP servers
• Approved databases

Delegated Authority

What authority is granted and what authority remains human-only.

Examples:

• May recommend trades
• May execute trades below threshold
• Must escalate new investment strategies

This is where governance should begin.

Not at execution time.

Not after deployment.

At specification time.

Organizations that fail to define these boundaries early often discover governance gaps later when the system is already operating.

Layer 2: Runtime Oversight

Even a perfect specification is not enough.

Agentic systems operate in dynamic environments.

Runtime controls must continuously enforce the boundaries established in the specification.

This layer includes:

Sandboxing

Restricting agent access to approved resources.

Policy Enforcement

Applying rules automatically during execution.

Tool Allowlists and Denylists

Preventing unauthorized capability expansion.

Intent Verification

Validating that actions remain aligned with approved objectives.

Audit Logging

Recording decisions, tool usage, state changes, and outcomes.

Drift Detection

Identifying behavioral deviations over time.

Cost and Resource Controls

Preventing runaway execution or excessive spending.

At this layer, governance becomes operational.

The system itself helps maintain compliance with the specification.

Humans are no longer reviewing every action.

They are monitoring the health of the governance system.

Layer 3: Exception Oversight (Targeted HITL)

Human oversight still matters.

But it should be reserved for situations where human judgment creates real value.

Examples include:

Risk Threshold Exceeded

Large financial transactions.

Novel Situations

Scenarios outside previously defined boundaries.

Policy Conflicts

When multiple rules produce conflicting outcomes.

Low Confidence Decisions

When agent confidence drops below acceptable levels.

Multi-Agent Disagreement

When agents produce significantly different recommendations.

Safety Invariant Violations

When the system approaches prohibited behavior.

In this model:

Humans review exceptions, not routine operations.

This dramatically increases the value of human intervention while reducing fatigue.

Accountable Autonomy

The goal is not unrestricted autonomy.

The goal is not constant human approval.

The goal is accountable autonomy.

A system where:

• Humans define the boundaries.
• Specifications encode the governance rules.
• Runtime controls enforce those rules.
• Humans intervene when judgment is genuinely required.

The governance model becomes:

Design-Time Oversight → Runtime Oversight → Exception Oversight

instead of:

Action → Approval → Action → Approval

This scales far better as agents become more capable and autonomous.

Why SDD Matters More Than Ever

As AI systems become increasingly autonomous, governance can no longer depend solely on human vigilance.

Human attention does not scale.

Specifications do.

This is why Spec-Driven Development is becoming more important.

SDD is not simply a way to generate better code.

It is a mechanism for encoding governance before execution begins.

The specification becomes the authoritative source for:

• Autonomy boundaries
• Safety invariants
• Escalation rules
• Tool governance
• Runtime enforcement requirements

In other words:

The specification becomes the first layer of governance.

Everything else builds on top of it.

Final Thoughts

The AI industry is beginning to recognize a difficult truth:

Human-in-the-Loop governance does not scale indefinitely.

As agentic systems become more capable, organizations will need governance models that combine:

• Design-time controls
• Runtime enforcement
• Targeted human intervention

This is the path toward accountable autonomy.

And it starts long before deployment.

It starts with the specification.