Add Approval Before Side Effect

Problem

The collections assistant schedules a R$1,890 Pix (Brazil’s instant payment) to “pay the electricity bill”. The key came from an attached PDF. Nobody confirmed. The money left for a name the customer does not recognize. The agent sent, paid, or deleted with no confirmation at all. This is the Ungated Side Effect smell.

Solution

Before running a side-effect action, pause the run, show a person exactly what will happen, and only continue with an explicit “yes” recorded by the system. The check lives in the tool’s code, not in the prompt.

Before and after

 def schedule_pix(key: str, amount_cents: int, date: str) -> dict:
-    return bank.schedule_pix(key, amount_cents, date)
+    preview = {
+        "action": "Schedule Pix",
+        "recipient": bank.lookup_key_owner(key),   # real name of the key owner
+        "amount": format_brl(amount_cents),
+        "date": date,
+    }
+    approval = approvals.request(user_id, preview, expires_in="15m")
+    if not approval.wait():                        # pauses until the answer
+        return {"status": "declined", "reason": approval.reason}
+    return bank.schedule_pix(key, amount_cents, date,
+                             idempotency_key=approval.id)

Before: the model calls and the money leaves.

flowchart LR
  M(["1. Model decides to pay"]):::llm --> T["2. schedule_pix"]:::tool
  T --> B[("3. Bank")]:::store

After: the model proposes, the person decides, code executes.

flowchart LR
  M(["1. Model proposes payment"]):::llm --> P["2. Build preview from real data"]:::tool
  P --> H((("3. Person approves or declines"))):::human
  H -->|yes| T["4. Execute with idempotency key"]:::tool
  T --> B[("5. Bank")]:::store
  H -->|no| R(["6. Declined result returns to the model"]):::io

1. The model still decides there is something to do.
2. Code builds the preview from system data, not from the model’s text: the real name of the key owner, the formatted amount.
3. The person sees the preview in a trusted channel (app, push, screen) and decides.
4. On “yes”, code executes, with a key that prevents duplicate execution.
5. The effect happens once.
6. On “no”, the tool returns the refusal, and the model continues the conversation knowing it.

Why it works

A rule in the prompt depends on the model remembering and obeying. A gate in code depends on nothing: the tool simply does not run without a recorded approval. Even if a malicious email fools the model, the worst it can do is ask. A person looking at real data makes the decision. The model proposes; it never executes.

The preview must come from the system, not the model. If the model wrote “pay the electricity bill”, an attacker could get it to write that about any payment key. Looking up the key owner’s name at the bank shows the truth: “Pay R$1,890.00 to Jane Doe”.

This has already left the hypothetical. In 2023, a Chevrolet dealership chatbot in Watsonville “agreed” to sell a Tahoe for one dollar. No human in the loop looked at the real amount. In 2024, in Moffatt v. Air Canada, a chatbot invented a refund policy and the company was ordered to honor what the machine promised. Approval does not erase legal responsibility, but it stops the irreversible effect before someone sees the number.

The idempotency key closes the last gap: a network retry or a second click does not pay twice. See also Classify Errors Before Retry.

Safe mechanics

0. Measure. List every side-effect tool and count, in the traces, how often each one was called and how many of those calls the user explicitly asked for.
1. Classify effects by risk: money, outside communication, third-party data, the user’s own data, reversible.
2. Pick a gate per level: always approve, approve above a threshold, pre-authorized with a cap, or none for reversible effects (drafts, trash).
3. Build the preview from the system, with the data the person needs to decide. Recipient name, amount, date, what cannot be undone.
4. Implement the pause: synchronous in the chat (“confirm?”) or asynchronous with saved state, if the answer may take minutes. The run must survive a reload.
5. Put the check inside the tool, so no code path can skip it.
6. Use an idempotency key on the final execution.
7. Record approvals and refusals with who, when, and what was shown. That record is what you show in an incident.
8. Measure again. Unapproved effects must be zero; track the decline rate and the time to answer.

Trap: confirming in the same chat

Asking “reply YES to confirm” in the same thread the malicious PDF arrived in is not a gate. The injected content writes YES. Use a channel only the user controls: a button in the app, a push, a review screen. The Human in the Loop pattern describes this pause.

Fatigue is the other side. Confirming every email draft trains people to click without reading. Separate what is irreversible from what can be undone. A Pix refund and a Zendesk draft do not share a gate.

Risks

• Approval fatigue. Ask only where it matters, and keep the preview short and clear.
• Misleading preview. Showing model-generated text instead of system data defeats the gate.
• Insecure channel. Asking for confirmation in the same chat where malicious content can write “yes” is not approval.
• Stale execution. The person approves hours later and the context changed (balance, price, Pix key). Revalidate before executing and put a deadline on the approval.

Challenge

This implementation has a gate, but it can be bypassed. How?

def send_email(to, subject, body, confirmed: bool = False):
    if not confirmed:
        return {"status": "needs_confirmation", "preview": f"Send to {to}"}
    return mailer.send(to, subject, body)

def send_email(to, subject, body):
    preview = {"to": to, "subject": subject, "body": body[:500]}
    approval = approvals.request(current_user, preview)   # the user's channel
    if not approval.wait():
        return {"status": "declined"}
    return mailer.send(to, subject, body, idempotency_key=approval.id)

Related

• Treats the Ungated Side Effect smell.
• Target pattern: Human in the Loop.
• Retries on the effect: Classify Errors Before Retry.
• Outside content that asks for the action: Quarantine Untrusted Content.

Check your understanding