Replace Agent with Chain

Problem

An agent with tools decides what to do on every turn, but the path is always the same. It costs more, takes longer, and sometimes skips a step. This is the Agent Where a Chain Would Do smell.

The company’s résumé screening extracts data, checks mandatory requirements, scores, and saves to the ATS. A week’s traces show the same sequence on 98% of runs: extract_resume, check_requirements, score_candidate, save_to_ats. On the other 2%, the model saved before scoring, or skipped the check. Each résumé paid for four “what do I do now?” turns to follow a path code already knew.

Anthropic’s “Building effective agents” post (December 2024) recommends workflows with fixed steps when the path is predictable, and reserves the agent for when the model needs to choose. This refactoring is that swap. The destination is Prompt Chaining.

Solution

When the path is always the same, fix the order of steps in code. Each step becomes a call with a focused prompt or a regular function, and one step’s output feeds the next.

Before and after

-agent = Agent(
-    system="You process incoming resumes. Use the tools to "
-           "extract data, check requirements, score, and save to the ATS.",
-    tools=[extract_resume, check_requirements, score_candidate, save_to_ats],
-)
-agent.run(resume_pdf)
+def process_resume(resume_pdf, job):
+    data = llm_parse(EXTRACT_PROMPT, resume_pdf, schema=ResumeData)      # 1
+    reqs = check_requirements(data, job.requirements)                     # 2. code
+    if not reqs.mandatory_ok:
+        return ats.save(data, status="does_not_meet", reasons=reqs.missing)
+    score = llm_parse(SCORE_PROMPT, data, job, schema=Score)              # 3
+    return ats.save(data, status="screened", score=score)                 # 4

Before: the agent picks the order on every turn.

flowchart TD
  IN(["1. Resume"]):::io --> A(["2. Agent picks the next step"]):::llm
  A --> T1["extract"]:::tool
  A --> T2["check"]:::tool
  A --> T3["score"]:::tool
  A --> T4["save"]:::tool
  T1 --> A
  T2 --> A
  T3 --> A

After: the order lives in code.

flowchart LR
  IN(["1. Resume"]):::io --> E(["2. Extract"]):::llm
  E --> C["3. Check requirements"]:::tool
  C --> D{"4. Meets them?"}
  D -->|yes| S(["5. Score"]):::llm
  D -->|no| G[("6. Save to ATS")]:::store
  S --> G

1. The resume arrives.
2. A focused call extracts the data into a schema.
3. Mandatory requirements are checked in code, with no model.
4. A simple condition decides whether scoring is worth it.
5. Only resumes that meet the requirements get scored.
6. All are saved, with status and reason.

Worked example

An 8-page résumé. In the agent, each turn resends the history: the extraction, the check result, the score. That is four decision calls plus the tools, with the PDF echoing through the context. If check_requirements is a tool the model “chooses” to call, it can choose not to.

In the chain, extraction is one call with EXTRACT_PROMPT and schema ResumeData. The check is a function: required degree, professional-board registration, location. Without the degree, code saves does_not_meet and does not score. There is no turn where the model “forgets” that if.

Suppose, just to see the order of magnitude, 4 thousand tokens of context per agent turn and 4 turns. The chain makes 2 model calls (extract and score) with short prompts, and zero model calls on résumés that fail the rule. Extraction and scoring quality are still measured by eval, now per step.

Why it works

In an agent, each turn is a model call to decide the next step, with the whole history resent. If the next step is always the same, those decisions are cost and risk with no benefit: the model can skip the requirements check or save before scoring.

In a chain, code decides the order. Each model call does one thing, with a short prompt and a typed output, and can be tested on its own. Steps that do not need a model (checking requirements, validating a tax id, summing values on a Brazilian electronic invoice, the NF-e) become regular code.

The chain is cheaper, faster, and more predictable. An agent is worth it when the path changes from case to case: research until you find enough, follow links on an unknown portal, pick among ten tools depending on the ticket.

Safe mechanics

0. Measure. Run the agent’s eval and record quality, cost per item, latency, and the tool sequence of each run.
1. Look at the sequences. If nearly every run calls the same tools in the same order, a chain is the right design. Group the remaining 2%: are they errors, or a genuinely different kind of input?
2. List the steps and the dependencies between them.
3. Write one focused prompt per step that needs a model, with structured output.
4. Replace with code the steps that are exact rules.
5. Wire the steps in order, with explicit conditions (the “if it does not meet them, save and stop”).
6. Handle the real exceptions: if a small share of cases needs a different path, add a branch in code or send them to a person. If the input is what changes the path (status vs. invoice), a router in front picks the right chain.
7. Measure again. Equal or better quality, lower cost and latency. Check especially the cases the agent used to skip.

Common mistake: a chain that forgets to pass context

In the agent, scoring saw the whole PDF because it was in the history. In the chain, step 3 only sees what you passed. If SCORE_PROMPT gets only the extracted schema and the job asks for “NF-e experience described on lines 40 to 80”, scoring goes blind. Pass the excerpt; do not count on memory from the previous turn.

Risks

• A case that needed flexibility. Some kind of input may depend on the agent’s dynamic decision. An eval with varied cases shows it. A résumé as an image, a résumé in another language, a zip with two PDFs: if that exists, design the branch; do not turn the main flow back into an agent.
• A chain that is too rigid. If new cases start requiring branches and loops, maybe the task changed. Reassess. One or two ifs are still a chain. A loop where the model picks the next tool is an agent again.
• Lost context between steps. In the agent, everything was in the history. In the chain, pass explicitly what each step needs.

Challenge

This agent answers questions about order status. The trace shows that on every run it calls find_order, then get_tracking, and answers. What does the chain look like?

tools: find_order(customer_id, hint), get_tracking(order_id), get_invoice(order_id)

order = find_order(customer_id, hint=extract_hint(message))   # extract the hint with a small call or regex
if order is None: return ask("What is your order number?")
tracking = get_tracking(order.id)                               # code, no model
return llm(REPLY_PROMPT, message, order, tracking)              # one call to write the reply

Check your understanding