Human-in-the-loop, step-up auth, and budget controls

Why this matters

Some actions should never be automatic. Refunds, mass emails, account deletions, posting publicly, spending money, writing to production databases — for these you want a human in the loop, with strong, recent authentication, and bounded budgets that prevent runaway loops. This lesson covers HITL patterns, step-up authentication, and the operational guardrails.

Learning objectives

1. Decide when to require HITL.
2. Implement step-up authentication with WebAuthn / passkeys.
3. Send approval requests via push (CIBA, Twilio Verify, Auth0 Guardian).
4. Apply step + spend budgets for autonomous agents.
5. Build an approval queue with replay + audit.

1. When to require HITL

A risk matrix per tool (lesson 1.3) drives the rule. Defaults:

These are policy decisions; document them in the agent record (lesson 2.1).

2. CIBA for out-of-band approval

Already introduced in lesson 3.1. The full pattern for a high-risk tool:

Agent: I need to refund $200 to order #1234 for alice@acme.com
Broker:
  1. OPA policy: requires approval
  2. POST /bc-authorize to IdP with:
       login_hint: alice@acme.com
       binding_message: "Refund $200 to order #1234?"
       scope: payments:write
       authorization_details: [{ type: "payment_initiation",
                                 instructedAmount: { amount:"200.00", currency:"USD" },
                                 ... }]
  3. Poll /token for auth_req_id
  4. On approval: receive a token bound to that refund only
  5. Issue a capability token (lesson 6.1) carrying the IdP token's `authorization_details`
  6. Agent calls payments-api with the capability token
  7. payments-api executes; logs subject + actor + approval id

CIBA's binding_message is the user-visible truth. Make it match exactly what the action will do; mismatches are how phishing-via-agent works.

For products that don't yet support CIBA, the in-app equivalent is an approval modal in the user's logged-in app session with the same fields. Either way, the user sees the exact action and approves it explicitly.

3. WebAuthn / passkeys for step-up

Step-up = "you're already signed in, but this action requires a fresh, strong proof of presence". Passkeys (WebAuthn) are the modern answer: phishing-resistant, biometric-bound, no shared secrets.

# Server: generate options
opts = generate_authentication_options(
    rp_id="acme.com",
    allow_credentials=[{"type":"public-key","id":cred_id_for(user)}],
    user_verification="required",
)
# return opts to the browser; user authenticates with Touch ID / Windows Hello / YubiKey
# Browser returns assertion; server verifies signature against stored public key
verify_authentication_response(...)

Use libraries: webauthn (Python), @simplewebauthn/server (Node), Auth0/Okta/Stytch SDKs. For agent flows, the approval surface is your web app; the agent triggers an approval request, the user approves with their passkey, the broker mints the capability token.

For high-stakes flows, require transaction signing: the user's passkey signs the action's hash, not just "I'm here". The signed transaction becomes part of the audit log + the capability token claims.

4. Approval queues

For asynchronous agents, build an approvals service:

POST /approvals  → returns approval_id, push sent to user
GET  /approvals?status=pending  ← user's app polls or subscribes
POST /approvals/{id}/approve  (requires WebAuthn step-up)
POST /approvals/{id}/reject

Per approval record:

• id, created_at, expires_at (e.g. 10 min).
• agent_id, subject (user), action, args, risk_score.
• binding_message shown to user.
• decision, decided_at, decided_by, webauthn_attestation.
• capability_token_jti once minted.

On approval, mint the capability token. On reject or expiry, kill the request and log.

Useful UI tip: show a history of approvals per agent so users see what their agents are doing over time — builds trust and is also a major audit asset.

5. Push channels

For HITL you need a reliable channel to reach the user:

• CIBA-compliant authenticators (Authy, Okta Verify, Microsoft Authenticator, custom mobile app).
• Twilio Verify / Authy for push or SMS (push > SMS).
• Auth0 Guardian push.
• Email magic link for low-risk approvals only.
• In-app notification when the user is logged into your web app.

Fallback chain: if push fails after N seconds, try SMS or email. Always include the binding message + a clear "cancel" option.

For developer / power-user agents, terminal-based approval (the agent's own CLI shows [y/N] and requires acme login-style fresh auth) is acceptable.

6. Dual approval + cool-downs

For critical actions:

• Dual approval — two distinct humans must approve (e.g., "delete tenant"). Block the action's approval API from accepting the same decided_by twice.
• Cool-down — after approval, wait N minutes before executing; this gives someone time to abort. Used by GitHub for org-wide destructive actions.
• Notification fanout — copy the user's manager / security team on critical approvals.

Capture all of this in the audit trail.

7. Step budgets + spend caps

Autonomous agents loop. Loops cost money, energy, and quota. Mandatory controls per agent:

When a budget exceeds soft cap: warn + log; require HITL to continue. When it hits hard cap: stop and alert.

These prevent "denial of wallet" and "infinite tool loop" classes of incident. Implement at the runtime level (agent framework) and at the broker / capability service so a malfunctioning runtime can't blow them.

8. Risk-based step-up

Not every "write" should pause for HITL. Decide dynamically:

risk = score(
  action_severity,
  unfamiliar_destination,
  time_of_day,
  recent_anomalies,
  agent_age,
  user_recently_authenticated,
)
if risk > 0.7: require HITL + step-up
elif risk > 0.4: require HITL
else: allow with audit only

You can build the scorer manually with rules, or use a small classifier trained on past approvals/rejections. Either way, expose the inputs so users / auditors can understand why a particular action was elevated.

Modern IdPs (Auth0 Adaptive MFA, Okta Adaptive Authentication) expose this as a hosted feature — reuse rather than reinvent.

9. UX patterns that actually work

• Surface the action context first, the agent identity second. "Helpdesk Agent wants to refund $200 to order #1234" beats "Approve action payments.refund(...)".
• One-tap approve with strong auth — never make the user re-type a password.
• Cancel is always one tap and obvious.
• Show the calling chain for sub-agent flows: "Helpdesk Agent → Research Sub-Agent → search:web".
• Per-agent revocation in one place — a single dashboard listing all connected agents + recent approvals + a "disconnect" button.
• No silent re-prompts. If the user rejects, don't ask again for at least 24 h.

Bad UX leads to approval fatigue and rubber-stamping — which is worse than no HITL because it gives a false sense of safety.

10. Hands-on lab (3 h)

1. Build an /approvals service backed by Postgres + Redis.
2. Trigger CIBA on refund_order actions; on user approval, mint a capability token via the service from lesson 6.1.
3. Implement WebAuthn step-up: the user's app prompts for biometric on approval; the server verifies and embeds the assertion hash in the capability token.
4. Add step budgets + spend caps at the broker layer; exceed them in a test and verify the agent is halted.
5. Build a tiny "My Connected Agents" page where Alice sees the helpdesk agent, recent approvals, and a Disconnect button.

11. Common pitfalls

1. CIBA approvals without binding messages → users approve anything.
2. Step-up that accepts a password as proof — phishable, defeats the purpose.
3. No expiry on approval requests → stale approvals accepted weeks later.
4. Budgets enforced only in the runtime, bypassed by a malfunctioning agent.
5. Dual approval that lets the same person approve twice via two browsers.
6. No replay defence (jti) on approvals → captured-approval reuse.

12. Self-check

1. The risk-tier → HITL pattern table.
2. Why passkeys beat SMS for step-up.
3. CIBA binding message — what + why.
4. Three default agent budgets you'd always set.
5. Dual approval — failure mode if you skip distinct-decided_by.

13. References

• WebAuthn Level 3 (W3C).
• "Modern Authentication with Passkeys" — FIDO Alliance.
• OpenID CIBA Core 1.0.
• Auth0 Adaptive MFA / Okta Adaptive Authentication docs.
• Twilio Verify + Authy docs.
• NIST SP 800-63B (digital identity, AALs).
• "Designing for Security UX" — Adam Shostack.