Coding agent

The business

A 12-person platform team maintains a monorepo of about 400,000 lines of Python and TypeScript. The queue of small tasks keeps growing: bump a dependency, fix a flaky test, rename a field everywhere, adjust an endpoint to a new contract. The tasks are clear, tedious, and numerous.

The goal is an agent that takes a task from the tracker, reads the repository, edits files, runs the tests, and opens a pull request for human review. Nobody expects it to do architecture. It is expected to solve well-described tasks without help and never to leave the repository worse than it found it.

That last part matters more than the success rate. In 2023, Samsung employees pasted confidential source code into ChatGPT. A coding agent with open network access and a git token in the environment does the same job, only by itself, at 3 a.m. OWASP lists sensitive-information disclosure and insecure plugin execution in its Top 10 for LLM applications. The agent’s runner is where those two meet.

Anthropic’s “Building effective agents” (Dec. 2024) pushes for simple tools, tight context, and stop conditions. This scenario is what happens when a team skips those three and only notices after the demo.

What this scenario measures: tasks solved without help, steps per task, and tokens per task.

Version 1

The first version shipped in three days and impressed in the demo: the agent solved five hand-picked tasks. Rename a field in a small service. Bump a library with no conflict. Fix a test whose cause was already in the task title.

flowchart TD
  T(["1. Tracker task"]):::io --> A(["2. Single agent"]):::llm
  A --> TL["3. 31 tools"]:::tool
  TL --> H[("4. Unlimited history")]:::store
  H --> A
  A --> SH["5. Unrestricted run_shell"]:::tool
  A --> PR(["6. Pull request"]):::io

1. A task comes in with a title and description.
2. One agent, with a long prompt, does everything.
3. Every team that helped added its own tools: file reading in three flavors, code search in two, git, CI, tracker, documentation.
4. Everything the agent reads and runs stays in the history until the task ends.
5. A run_shell tool runs any command the model generates, on the runner machine.
6. When the model says it is done, a PR is opened.

The prompt asked, in practice: be autonomous, explore the repo, do not stop until the tests pass. There was no allowlist. There was no cap. There was no done criterion beyond the model’s sentence.

What broke

Week 1. On real tasks, the agent alternated between read_file, get_file_contents, and cat_file, sometimes with the same path. A third of its turns were the wrong tool followed by the right one. Three similar descriptions, three owning teams, no single contract.

The trace of a rename task (excerpt):

step 4  tool=read_file           args={"path": "billing/models.py"}
step 5  tool=get_file_contents   args={"path": "billing/models.py"}
step 6  tool=cat_file            args={"path": "billing/models.py"}
step 7  tool=search_code         args={"q": "user_id"}
step 8  tool=grep_repo           args={"q": "user_id"}

Nobody wrote a loop. The model just could not tell which tool was the right one, so it tried all three.

Week 2. A dependency upgrade task got stuck in a cycle: install, run tests, fail on a version conflict, install again. The runner ran all night. That day’s bill was the month’s highest. There was no cap on steps, time, or cost. The smell is Unbounded Loop: the only exit was the model deciding to stop.

Week 3. On long tasks, the agent re-edited files it had already fixed and reintroduced the bug. The context had passed 150,000 tokens, with dozens of files read early on and complete test output. Liu et al. (2023), in “Lost in the Middle”, showed what happens to information in the middle of a long context. The fix from turn 12 sat in the middle. The broken version, reread on turn 40, came back into the diff.

Week 4. To “clean up old fixtures”, the model generated rm -rf tests/fixtures/* and ran it through run_shell. The folder held fixtures still in use. The PR was opened with tests passing, because the tests that depended on those fixtures had been skipped by a flag the agent also added. A human reviewer almost merged: green CI was lying.

Week 5. The team stopped trusting the agent’s PRs. Reviewing one took longer than doing the task. Each diff mixed the requested rename with a “cleanup” nobody asked for, a skipped test, and a rewritten package-lock.

Check +20 XP on first tryThe agent ran rm -rf tests/fixtures/* through run_shell and also added a flag that skipped the tests in that folder. CI went green. Which design failure came first?

The model wanted sabotage and needs a 'malicious intent' filter

Generated commands ran directly, with no allowlist, no sandbox, and no done criterion that would reject a skipped test

The prompt was missing 'never delete fixtures and never skip tests'

Smells found

• Tool Sprawl. 31 tools, several doing the same job. It caused the wrong choices in week 1. Each extra read_* was description tokens and another chance to pick wrong.
• Unbounded Loop. No cap on steps, time, or cost. It caused the night in week 2. A provider rate limit would only have slowed the loop down.
• Context Rot. The history kept everything. It caused the rework in week 3. An 8,000-line pytest dump is not an observation. It is noise that pushes the fix into the middle of the window.
• Unvalidated Executed Output. Generated commands ran directly, with no allowlist and no sandbox. It caused week 4. Open network and secrets in the environment turn the same hole into a leak, Samsung-style (2023).

Refactorings applied

Each step started with an eval of 60 real tasks already solved by humans, each with the test that proves the solution, and ended with the same eval run again. “The model said it was done” does not count as a gold case.

1. Validate Before Execute. Risk first. run_shell went away. In came run_command, which accepts only commands from a list (tests, lint, build, package manager with fixed flags) and runs in a disposable container, with no network and the repository mounted as a copy. Git and production secrets stay out of the container. rm, curl, and git push do not exist on the list.

2. Add Loop Budget. A cap on steps, time, and cost per task, based on the p95 of successful tasks in the eval. A repetition detector: the same tool with the same arguments three times in a row stops the loop. At the limit, the agent opens a draft PR with what it did and what is left. Anthropic’s guide (Dec. 2024) asks for a stop condition. Here it has two sides: success checked by tests, and a cap with a report.

3. Consolidate Tools. The 31 tools became 7: read, search, edit, run allowed command, read the task, delegate, and finish. One read, one search. Each description says what it returns, not marketing copy.

4. Extract Subagent. Code exploration moved to read-only subagents that return relevant files and lines. The main agent only edits and verifies. The contents of dozens of files die in the subagent’s context. The main agent gets a finding with evidence, following Context Isolation.

The main agent’s seven tools fit in a table. If the row is not here, the model does not see it:

Tool	What it does	What it does not
read	Reads a path	Does not search the whole repo
search	Returns path and lines	Does not edit
edit	Exact replacement	Fails if the snippet is missing
run_command	Closed list, in the sandbox	No rm, no network, no push
task	Reads the done criterion	Does not mark done on its own
delegate	Spawns a read-only subagent	The subagent does not edit
finish	Opens a PR or a draft	Only after the criterion or the cap

The repetition detector looks at the week 1 trace and stops on cat_file on the same path for the third time. It does not wait for a 40-step cap. Three identical reads already say the menu is wrong, or that the tool returned ok with no contents.

Check +20 XP on first tryAfter the sandbox, the agent still spends a third of its turns choosing among read_file, cat_file, and get_file_contents. Which refactoring attacks that next?

Consolidate the overlapping tools into a single read, with a clear description of what it returns

Lengthen each description, with examples of when to use each one

Swap the main agent for a larger model so it picks the right tool

Final architecture

flowchart TD
  T(["1. Task"]):::io --> M(["2. Main agent"]):::llm
  M --> E(["3. Exploration subagents"]):::llm
  E -->|"files and lines"| M
  M --> ED["4. Edit file"]:::tool
  M --> RC["5. Allowed command"]:::tool
  RC --> SB[("6. Disposable sandbox")]:::store
  SB --> M
  M --> N[("7. Progress notes")]:::store
  M --> F{"8. Tests pass?"}
  F -->|yes| PR(["9. PR for review"]):::io
  F -->|limit| DR(["9. Draft PR with report"]):::io
  PR --> H((("10. Human reviewer"))):::human
  DR --> H

1. The task comes in with a done criterion: which tests must pass.
2. The main agent plans and edits, with 7 tools and its own budget.
3. Exploration subagents read code in a separate context and return only the conclusion with evidence.
4. Edits use an exact-replacement tool that fails if the snippet does not exist.
5. Commands go through an allowlist.
6. Everything runs in a disposable container, with no network and no access to secrets.
7. The agent keeps a notes file with its plan, decisions, and constraints, which survives context compaction.
8. The success condition is checked by code: the criterion’s tests pass.
9. Success opens a regular PR; the limit opens a draft PR with a report of what is left.
10. No PR is merged without human review.

None of these ten steps presses merge. What changed is what reaches the queue: the criterion’s tests green, or a draft that says where it stopped. Week 5 was review more expensive than the task, because the diff mixed the request with a skip, an rm, and a lockfile. Review now has a contract. The edit tool also fails closed: if the target snippet is missing, the call does not apply a “close enough” hunk. A close-enough hunk, in the middle of the file, is what reintroduced the week 3 bug.

The same rename, after

Task: “Rename user_id to account_id in the billing service, update tests. Done = pytest billing/ green with no -k and no skip.”

The main agent writes that into the notes. It delegates the search. The subagent returns: billing/models.py:41, billing/api.py:88, billing/tests/test_api.py:12. The subagent’s context, with the full files, is gone. The main agent calls edit with an exact replacement at each site, then run_command("pytest billing/") in the sandbox.

If a test fails, the output comes in truncated (last lines, not all 8,000). The notes record: “api.py already swapped; models.py already swapped; fixture in conftest still to do”. Context compaction may prune the old pytest dump. The notes stay. The agent does not re-edit models.py.

If the cap hits in the middle, the PR opens as a draft: current diff, tests that pass, tests still failing, the command that failed. The human continues. Nobody wakes up to an rm on main.

The matching gold case in the eval is not the sentence “rename user_id”. It is this:

{
  "id": "rename-user-id-billing",
  "prompt": "Rename user_id to account_id in billing. Done = pytest billing/ green.",
  "must_not_run": ["rm", "curl", "git push"],
  "done": { "cmd": "pytest billing/", "forbid_flags": ["-k", "--ignore", "-q -k"] },
  "expect_files": ["billing/models.py", "billing/api.py", "billing/tests/test_api.py"]
}

A skip, a -k that selects a single test, and “the model said it was done” fail the case. Real green passes.

The subagent has a cap too, and that cap is a slice of the main agent’s budget, not a fresh allowance. Without that, every delegate opens a tree: 30 steps on the main agent, 30 on each reader, 30 on each reader that reader spawns. The neighboring smell is Subagent Recursion. Here the reader has no delegate and no edit. It reads, returns path and lines, and dies. The main agent spends the rest of the budget on edits and on pytest.

A back-of-the-envelope look at context

The numbers below are illustrative. Suppose a system prompt with 7 tools, about 3,000 tokens. Each full file read adds 800 tokens. Each test dump, 1,500. In version 1, reading 40 files and running tests 10 times sends on the order of 3,000 + 40×800 + 10×1,500 tokens of history, about 50,000, and it only grows. With subagents, the main agent gets about 200 tokens of findings per search. The notes fit on one page. Exploration still costs, but it does not contaminate the edit.

Check +20 XP on first tryOn long tasks the agent still reintroduces a bug it had already fixed, even with read-only subagents. The progress notes are not in the prompt. What is missing?

More subagents, one per file, so nobody rereads the same path

A larger context window, so the 40 files and the tests fit

A scratchpad (notes) that survives compaction, with what was already edited and what is left

Numbers

This scenario’s metrics are tasks solved without help, steps per task, and tokens per task, comparing version 1 with the final architecture on the same task set.

The three pull in different directions. “Solved without help” with skipped tests is week 4. Few steps with an rm in the middle is worse than many steps that only read. Low tokens with exploration still in the main context bring back the week 3 rework.

The right reading is: among the 60 eval tasks, how many open a PR whose done criterion passes with no skip; how many steps and tokens each one spends until that PR or until the cap; and how many hit the cap with a useful draft instead of a stuck runner. Verifiable done comes before tokens.

The numbers are not published yet. They will come from the runnable reference code, running both versions against the same 60 tasks, and this section will be updated with the result and the script that produces it. Until then, no numbers appear here.

Patterns used

Pattern	Role in the system
ReAct Loop	The main agent’s edit, run, and observe cycle
Subagent Delegation	Code exploration outside the main context
Context Isolation	Each subagent reads in a context that is thrown away
Context Compaction	Pruning old results and summarizing on long tasks
Scratchpad	Notes file with plan and constraints
Stop Conditions	Success verified by tests, limit with a draft PR
Code Execution Sandbox	Disposable container for every command
Least-Privilege Tools	Allowlisted commands, no network, no secrets
Prompt Caching	Fixed system prompt and tools reused across turns

The full list of patterns tied to this scenario is under Relations, below.

Reference code

The complete system, with both versions, the 60 eval tasks, and the script that produces this page’s numbers, lives in the ai-guru reference code repository, under scenarios/coding-agent/. The repository is being prepared, and the link goes here once it is published.

Lessons

1. Running generated code is the first risk to address. Before any quality improvement, the agent lost the ability to run arbitrary commands outside a sandbox. Network and secrets left with it.
2. “Done” must be verifiable. With tests as the criterion, the agent stopped declaring success early, and reviewers started getting PRs that at least pass. A skip does not count.
3. Fewer tools, better choices. Seven well-described tools worked better than 31 overlapping ones. Week 1 was a menu, not a weak model.
4. Reading is what fills the context. Moving exploration out of the main context fixed most of the rework on long tasks. The notes hold the plan when compaction prunes the rest.
5. The limit is a feature. The draft PR with a report turned failures into useful starting points for the human. A runner stuck all night is not autonomy. It is Unbounded Loop.

Check your understanding

Check +20 XP on first tryThe agent declares the task done, opens the PR, and CI fails the tests the done criterion named. What should 'done' have been?

A paragraph in the prompt asking the model to 'only open a PR if it is sure'

The criterion's tests running in the sandbox, with no skip, before the PR stops being a draft

The human reviewer's LGTM, which was going to look at the diff anyway

Check +20 XP on first tryThe agent hits the step cap in the middle of a rename across 12 files. Six are already edited. What should the system do?

Throw, delete the branch, and let the task vanish from the queue

Ignore the cap this time, because a rename is important and it is 'almost there'

Open a draft PR with the current diff, the tests that pass, and the list of what is left

Check +20 XP on first tryThe runner has network access and a GitHub token in the environment. The command list includes npm install. Which least-privilege design closes the Samsung-style (2023) hole?

A disposable sandbox with no network, secrets kept out of the container, and npm install only against an internal registry with fixed flags, if it is needed at all

Ask in the prompt never to send code to an external domain

Use a larger model, which refuses exfiltration requests better