Opening

A marketplace team had to moderate listings. Check the text, check the image, check the price, and then decide.

Someone suggested an agent with four tools. The demo looked great.

In production, every now and then, the agent decided the image looked fine, and never called the image tool. Banned listings went live.

The moderation path was always the same. Text, image, price, decision. The team gave the model a freedom the task never asked for. In this lesson, you'll learn the one question that would have prevented this, and how to pick the lowest rung that solves your problem.

The idea

Every system built on a language model answers one design question, whether the team noticed or not. Who decides the next step? Your code, or the model?

How it works

On one side, a chain. A fixed sequence of calls, with the path written in code. Step B always runs after step A. Always.

On the other side, an agent. The model gets a goal and some tools. And on every turn, it decides what to do. Call a tool, which one, with which arguments, or stop.

Between those two extremes sit workflows. Flows defined in code, that use the model at specific decision points. A router choosing among three fixed paths is a workflow. Code still owns the structure. The model only picks among options you designed.

So the listing moderation was a four box chain. Turning it into an agent didn't add intelligence. It added permission to skip the image box.

The autonomy ladder

Think of it as a ladder, with four rungs.

At the bottom, a single call. Nobody decides any path, it's one step. Classifying a ticket fits here.

Then a chain. Code decides, and it's always the same. Extract, validate, summarize.

Then a workflow. Code decides, with model choices at marked points. Routing is the classic example.

And at the top, an agent. The model decides, every turn. Like fixing a bug in a repository, where the path depends on what you discover along the way.

The rule: climb a rung only when the one below can't handle the job. Every rung up brings more flexibility. And also more cost, more latency, more variation, and more difficulty testing.

Anthropic's guide on building effective agents, from December twenty twenty-four, draws the same line. Start with workflows, where code orchestrates and the model fills the points of uncertainty. Bring in an agent when you can't write the sequence in advance.

The one-question test

So here's the test. One question. Can I draw the flowchart of this task before it runs?

If yes, and it's always the same, it's a chain. Listing moderation passes easily. Four boxes, no surprises.

If yes, but with known branches, it's a workflow. An order bot is usually like this. If the message cites an order number, look up the order. If it's about a bill, go to finance. If it doesn't fit anything, hand off to a person. The branches exist before the run.

And if no, because the next step depends on the previous result in ways you can't predict, that's an agent. Investigating why a deploy broke on a Friday night rarely has a flowchart. The next command depends on the log that just appeared. That's where an agent pays for itself.

Notice it's not about importance. Banning a listing is important too, and it asks for a chain. Importance asks for control, not freedom.

A challenge

Here's one for you. You measured that the moderation agent calls the image tool on seventy percent of runs. The policy requires one hundred percent. What change closes the hole?

Move the image call into code, before the decision, and let the model only judge the result. That seventy percent is the model using a freedom the policy never granted.

Raising the step limit wouldn't help. A limit prevents an endless loop, it doesn't require a step. And a prompt that says never skip the image lowers the skip rate, but never to zero. The skip really disappears when it stops being the model's decision.

What climbing costs

Let's put a rough number on it. Say a moderation flow has three checks, each around a thousand input tokens.

In a chain, that's always three calls.

In an agent, the model may make two, or seven, if it gets unsure and re-reads.

You trade a predictable cost for a cost with a tail.

And testing changes too. A chain has three contracts, and its eval is simple: did all three steps pass? An agent has a goal and a whole space of possible paths. Its eval needs traces, a step limit, and test cases where skipping a step was the failure.

The common mistakes

The most common mistake in practice has a name: agent where a chain would do. Autonomy for a problem that never asked for it.

Here's how it looks. A team has an agent with twelve tools to onboard new merchants. And in practice, it always asks for the same four documents, in the same order. That's not a coincidence. It's evidence. The flowchart already exists, written by the traces. Put it in code.

And the opposite mistake exists too. A chain with thirty if statements, trying to predict every question a fraud analyst might ask. Each new if is a branch you invented without data. When the branches explode, the workflow got too small. That's when you climb. But you climb with an eval, built from the cases the chain no longer covers, and with a cost limit for the agent.

The code and the stack

The same cut shows up in every stack. In LangGraph, a chain or a workflow is a graph whose edges you write. An agent is a tool loop. The agent helpers in LangChain, the OpenAI Agents SDK and the Claude Agent SDK all start from that loop.

But the library doesn't pick the rung. You do. If the flowchart fits on paper, your code draws the edges, whatever library you use.

Try it

Now take three tasks from your own product and classify each one. Fixed flowchart? Branches? Which rung?

Summarizing a meeting has a fixed flowchart, no branches. That makes it a chain.

Answering a customer about an order has a fixed flowchart, with branches. Workflow.

Investigating why a deploy failed has no fixed flowchart. Agent.

If a task you already solve with an agent landed on chain or workflow, you just found a candidate for a refactoring called replace agent with chain.

Wrap-up

So here's the idea to take with you. Autonomy isn't a feature. It's a cost you pay when you can't write the path in advance. Start at the lowest rung that solves the problem. You can always climb later, with an eval showing it was worth it.

In the next lesson, we'll build the first rung properly: prompt chaining, a few short steps, each one easy to test. See you there.