What is Add Loop Budget?
Add Loop Budget is an AI refactoring for LLM systems, in the Moving control group. Cap steps, time, and money for each run and define what happens at the limit.
Trail content
Chapters
Transcript
Opening
You met the invoice agent in the last lesson. It opens the supplier portal, clicks next page, and reads the list. One night the button stays visible and loads nothing. With no cap, the loop clicks thousands of times, and the only alarm is the bill.
The smell told that story. This refactoring puts on the brake. And you'll see that the brake is only half of it. The other half is what the agent says when it stops.
The idea
Give every run an explicit budget, of steps, time and cost. Check it before each turn. And define what the agent delivers when the budget runs out.
Before and after
Before, the only exit is the model stopping. The task goes to the model, then a tool, then the model again, then another tool. Until the model says it's done.
After, your code has a second exit. The task arrives with a budget set by your policy, not by the model.
Before each turn, code checks the steps, the elapsed time, and the accumulated cost.
The model is called as usual, and the final answer is still a legitimate exit. Tools run, and their cost counts too, when there is one.
The task ends within budget.
Or it ends at the limit, with what was found so far, and the reason for stopping.
In code, it's a small change. While true becomes: while the budget allows. With twenty-five steps, five minutes, and eighty cents of budget.
After every call, the budget is charged with the usage the API reported.
And when it runs out, the agent wraps up, with the reason.
A real case
With the same illustrative numbers as last time. A normal run, ten turns, about fifty-two thousand tokens.
A stuck run with no cap, two hundred turns, about eight point six million.
And the same stuck run, with a cap of twenty-five, about two hundred thousand. The curve never gets to the millions.
And here's the wrap-up on the broken portal. Not an empty exception. I read the first page, invoices one to twenty of three hundred forty. Next page returned okay, and the list didn't change, three times. I stopped for repetition. Pages two to seventeen are still missing.
Operations finishes the job by hand. And the trace becomes an eval case.
Why it works
An agent loop has two sources of termination. The model's decision, and the code's decision. Without a budget, only the first exists. And it fails in exactly the bad cases. A tool that makes no progress, an impossible task, content that confuses. The budget puts a deterministic decision right next to the probabilistic one.
The wrap-up is the half that's usually missing. A cap that only throws an exception trades a high cost for a lost task. A cap with a wrap-up delivers something useful. I read fourteen of the twenty files, the likely cause is this, I still need to confirm that.
And the budget turns bad behavior into data. Every run that hits the cap is a case to understand. The cap reveals the problem. It doesn't exist so you can hide it by raising the number.
A challenge
Here's one for you. Among successful runs, ninety-five percent finish within fourteen steps. The team wants a cap of sixteen, to cut early. What do you expect to see in the eval?
Cost at the maximum drops. And the completion rate of healthy tasks drops too. Five percent of the successful runs already go past fourteen steps, and some of those go past sixteen. You'd cut today's successes, to over-save on tomorrow's stuck run.
A common starting point is two to three times that ninety-fifth percentile. The cap should catch the stuck runs, not the long ones.
Safe mechanics
Now the mechanics. Step zero, measure. Run the agent's eval, and pull the distribution of steps, time and cost per task from the traces.
Pick the caps from that data. If the ninety-fifth percentile is fourteen steps, and the healthy maximum is nineteen, a cap of twenty-five leaves room. A cap of two hundred doesn't.
Check before each call, not after. Include the cost of paid tools, like search or a browser. And use the usage the API returns, not a stale price table.
And charge subagents to the parent's budget. Pass the remaining budget down. Don't hand each child a fresh one.
Write the wrap-up. One last short call with no tools, that summarizes what was done and what's left. Or a fixed message with the state. And the text has to say it stopped at the limit.
Record the stop reason on the trace. Steps, time, cost, repetition, or done.
Then add a detector for repetition. The same tool, with the same arguments, three times in a row, also ends the loop.
And measure again. The completion rate must not drop. Cost at the worst cases should drop. And the tasks that stopped at the cap become eval cases.
Risks
A few risks. A cap too low starts failing legitimate long tasks. You'll see it in the completion rate.
A wrong cost estimate. If the math uses stale prices, or ignores cached tokens, the money cap doesn't protect you.
A wrap-up that lies. A partial answer written as if it were complete misleads the user.
A budget counted only in steps. One step that summarizes a two hundred thousand token file costs more than ten short steps put together. Limit tokens or money too.
And a fresh budget for every child. Each subagent gets twenty new steps, with no depth limit, and the loop comes back, now as a tree.
That last one hides well. Look at this loop. It has a cap of twenty steps. So it looks safe.
But every delegate call opens a brand new twenty step loop, with no depth limit, and no shared budget. With three delegations per step, the total explodes. On top of that, the subagent's result is thrown away. And ending with a timeout exception leaves no state.
The fix is to pass the same budget down to every child, and take away the delegate tool after a couple of levels. Then add each child's result to the parent's observations, and end with a wrap-up, not an exception.
One more trap. A team writes the wrap-up as one last model call, with tools, asking it to finish the task with what it has. With tools, that extra turn is the loop again, without the cap you just hit. And the text it produces pretends to be a success. A wrap-up is short, has no tools, and admits the limit.
Wrap-up
So here's the idea to take with you. Steps, time and money, chosen from your own data, checked before every call, shared down the whole tree. And when the budget runs out, the agent says what it did, what's missing, and why it stopped. That's how a surprise bill becomes an eval case.
In the next lesson, we'll look at the full pattern this refactoring points to: stop conditions, including when an agent should give up, or ask for help. See you there.
Add Loop Budget
Adicionar Orçamento ao Loop
Add Loop Budget is an AI refactoring for LLM systems, in the Moving control group. Cap steps, time, and money for each run and define what happens at the limit.
- Element
- 11/35
- Level
- Intermediate
- Read
- 5 min
- Updated
Problem
The agent runs until the model decides to stop, with no limit on steps, time, or money. This is the Unbounded Loop smell, along with its variants Subagent Recursion and No Cost Budget.
The invoice-slip agent opens the supplier portal, clicks “Next page”, and reads the list. One night the button stays visible and loads nothing. With no cap, the loop clicks thousands of times. The only alarm is the bill. The smell tells that story; this refactoring puts the brake on. The destination pattern is Stop Conditions.
Solution
Give every run an explicit budget of steps, time, and cost, checked before each turn, and define what the agent delivers when the budget runs out.
Before and after
def run_agent(task):
messages = [task]
- while True:
+ budget = Budget(max_steps=25, max_seconds=300, max_usd=0.80)
+ while budget.allows():
reply = call_model(messages, tools=TOOLS)
+ budget.charge(reply.usage)
if reply.stop_reason != "tool_use":
return reply.text
messages += run_tools(reply)
+ return wrap_up(messages, reason=budget.exhausted_reason())Before: the only exit is the model stopping.
flowchart TD
T(["1. Task"]):::io --> M(["2. Model"]):::llm
M --> D{"3. Done?"}
D -->|no| F["4. Tool"]:::tool
F --> M
D -->|yes| R(["5. Answer"]):::io
After: code has a second exit, with defined behavior.
flowchart TD
T(["1. Task"]):::io --> B{"2. Within budget?"}
B -->|yes| M(["3. Model"]):::llm
M --> D{"4. Done?"}
D -->|no| F["5. Tool"]:::tool
F --> B
D -->|yes| R(["6. Answer"]):::io
B -->|no| W(["7. Wrap-up: partial result or request for help"]):::io
- The task arrives with a budget set by your policy, not by the model.
- Before each turn, code checks steps, elapsed time, and accumulated cost.
- The model is called as usual.
- The final answer is still a legitimate exit.
- Tools run, and their cost counts too when there is one.
- The task ends within budget.
- The task ends at the limit, with what was found so far and the reason for stopping.
Worked example
The numbers below are illustrative. System prompt and tools: 3,000 tokens. Each turn adds about 400 tokens of call and result.
| Run | Turns | Tokens sent in total (approx.) |
|---|---|---|
| Normal | 10 | 52 thousand |
| Stuck, no cap | 200 | 8.6 million |
| Stuck, cap 25 | 25 | about 200 thousand |
Twenty times more turns cost far more than twenty times the price, because each turn resends the history. A cap of 25 never lets the curve reach the millions.
The wrap-up, on the broken portal, is not an empty exception. It is: “I read the first page (invoices 1 to 20 of 340). Next page returned ok and the list did not change, three times. I stopped for repetition. Pages 2 to 17 are still missing.” Ops finishes the job; the trace becomes an eval case.
The OpenAI Agents SDK offers max_turns. LangGraph offers recursion_limit. Anthropic’s “Building effective agents” (December 2024) recommends a maximum number of iterations. Those pieces are the step cap. Time, money, repetition, and an honest wrap-up you write.
Why it works
An agent loop has two sources of termination: the model’s decision and the code’s decision. Without a budget only the first exists, and it fails in exactly the bad cases: a tool that makes no progress, an impossible task, content that confuses. The budget puts a deterministic decision next to the probabilistic one.
The wrap-up (step 7) is the half that is usually missing. A cap that only throws an exception trades a high cost for a lost task. A cap with a wrap-up delivers something useful: “I read 14 of the 20 files, the likely cause is X, I still need to confirm Y”.
Finally, the budget turns bad behavior into data. Every run that hits the cap is a case to understand and, often, a new eval case. The cap reveals the problem; it does not exist so you can hide it by raising the number.
Safe mechanics
- Measure. Run the agent’s eval and pull the distribution of steps, time, and cost per task from the traces. With no eval, start with Add Eval Before Changing the Prompt.
- Pick the caps from data. A common starting point is two to three times the p95 of successful runs. The cap must not cut tasks that succeed today. If p95 is 14 steps and the healthy maximum is 19, a cap of 25 leaves room; a cap of 200 does not.
- Check before each call, not after. Include the cost of paid tools (search, browser, external APIs). Use the
usagethe API returns, not a stale price table. - Charge subagents to the parent’s budget. Pass the remaining budget down; do not hand each child a fresh one.
- Write the wrap-up. One last short call with no tools that summarizes what was done and what is left, or a fixed message with the state. The text must say it stopped at the limit.
- Record the stop reason on the trace:
stop.reason = steps | time | cost | repeat | done. - Add repetition detection: the same tool with the same arguments three times in a row also ends the loop.
- Measure again. Compare the completion rate (must not drop), cost at the p95 and the maximum (should drop), and how many tasks stopped at the cap. The ones that stopped become eval cases.
Risks
- Cap too low. Legitimate long tasks start failing. Detect it through the eval completion rate and the share of runs with
stop.reason = steps. - Wrong cost estimate. If the math uses stale prices or ignores cache tokens, the money cap does not protect you. Use the
usagethe API returns on each response. - A wrap-up that lies. A partial answer written as if it were complete misleads the user. The wrap-up text must say it stopped at the limit.
- Step-only budget. One step with a huge document costs more than ten short steps. Limit tokens or money too.
- Per-child budget. Each subagent with 20 fresh steps, and no depth cap, recreates the loop in the tree. The challenge below is that case.
Challenge
This loop has a cap, but in practice it is still an Unbounded Loop. Find out why before opening the solution.
def run(task, depth=0):
for _ in range(20):
reply = call_model(task, tools=[search, delegate])
if reply.done:
return reply.text
for call in reply.tool_calls:
if call.name == "delegate":
run(call.args["subtask"], depth + 1)
else:
execute(call)
raise TimeoutError()Show solution
Each delegate call opens a new 20-step loop, with no depth limit and no shared budget. With three delegations per step, the total explodes. On top of that, the subagent’s result is thrown away, and ending with TimeoutError leaves no state.
def run(task, budget, depth=0):
while budget.allows():
reply = call_model(task, tools=[search, delegate] if depth < 2 else [search])
budget.charge(reply.usage)
if reply.done:
return reply.text
for call in reply.tool_calls:
if call.name == "delegate":
result = run(call.args["subtask"], budget, depth + 1) # same budget
task.add_observation(call, result)
else:
task.add_observation(call, execute(call))
return wrap_up(task, reason=budget.exhausted_reason())Check your understanding
Lesson check
Answer the 6 questions to complete this lesson. Watching alone does not complete it: the check is what counts toward the trail and the certificate.
Quick answers
What problem does Add Loop Budget solve?
It treats Unbounded Loop, Subagent Recursion, and No Cost Budget.
Which pattern does Add Loop Budget lead to?
The Stop Conditions and Cost Budget design pattern.
Go further
Relations
- Smells it treats 3
- Target patterns 2
- Shows up in scenarios 4
- Part of the tracks 1