Tool Use

Intent

Give the model well-described functions to fetch data and take actions, in a loop where your code runs each request, returns the result, and decides when to stop. It is for assistants that need to answer with real data and act in the world.

Problem

Marcos’s store had a bot that handled general questions very well. Customers wanted something else: “where is my order?”, “I want to cancel”, “has my refund gone through?”. For those, the bot could only say “please contact support”.

The first patch was pasting the customer’s data into the prompt before every conversation: recent orders, status, address. The prompt bloated, the data went stale mid-conversation, and the bot still could not cancel anything.

The second patch was worse: the bot replied “order canceled” when the customer asked, but nobody actually canceled it. The model was just writing the most likely text.

This class of error has left the lab. In 2023, a Chevrolet dealership chatbot in Watsonville, California, “agreed” to sell a Tahoe for $1 after a prompt injection. The store had to explain in public that the bot did not close deals. Model text is not an action. An action is your code calling the order system.

The question this pattern answers: how do you let the model query and act on your system, with your code in control of every step?

Solution

Declare the possible actions as tools: a name, a description of when to use it, and an argument schema. The model receives the conversation and the list, and can answer by requesting a tool. That is function calling: the model does not run anything, it only asks. Your code runs it, returns the result as a message, and the model continues until it has the answer.

Three rules make the pattern work in production. Read tools run directly. Tools that change something (cancel, pay, send) go through a gate: user confirmation, human approval, or a business rule. And the loop has a turn limit, because a model can request tools forever.

Anthropic’s “Building effective agents” (December 2024) puts tools at the center of the agent, with code as the executor. OWASP’s Top 10 for LLM Applications treats ungated effects and injection through data as standard risks, not rare cases.

Worked example

The customer writes: “cancel order 4521, I bought the wrong thing”. The model does not answer “canceled”. It requests:

{ "name": "cancel_order", "arguments": { "id": "4521", "reason": "wrong purchase" } }

Code does not call orders.cancel yet. It shows a summary in the app: “Cancel order 4521? This cannot be undone.” The customer taps confirm. Then the tool runs and returns:

{ "id": "4521", "status": "canceled", "refund": "pending" }

The model reads the JSON and explains: the order was canceled, the refund is queued. If the customer does not confirm, the tool returns {"error": "the customer did not confirm the cancellation"} and the model does not pretend it canceled.

Compare with a read. “Where is order 4521?” calls get_order with no gate and returns short JSON, not the tracking page’s HTML. If the tool answers only "ok", the model does not know the status and asks again: the same signal as Unbounded Loop.

An illustrative account. Pasting the customer’s 20 orders into the prompt, at 200 tokens each, is 4,000 tokens on every turn. With a tool, the first turn carries the definition (about 200 tokens) and the second carries one order’s JSON (about 150). Across a thousand conversations a day, the difference is millions of tokens, and the status stops going stale mid-chat.

Structure

flowchart TD
  IN(["1. Message + tools"]):::io --> M(["2. Model decides"]):::llm
  M --> D{"3. Tool requested?"}
  D -->|"no"| OUT(["7. Answer"]):::io
  D -->|"read"| R["4. Run get_order"]:::tool
  D -->|"side effect"| G((("5. Gate: confirmed?"))):::human
  G -->|"yes"| W["4. Run cancel_order"]:::tool
  G -->|"no"| X["Result: declined"]:::tool
  R --> B{"6. Within budget?"}
  W --> B
  X --> B
  B -->|"yes"| M
  B -->|"no"| OUT

1. The call carries the conversation and the tool list.
2. The model decides whether to answer or request a tool, and with which arguments.
3. Code inspects the request.
4. Allowed tools run in your code, with your code’s permissions.
5. Tools with side effects go through a gate before they run.
6. Each turn counts against a step budget. When it runs out, the loop stops.
7. When the model answers without requesting a tool, the answer goes to the user.

Pseudocode

tool get_order(id: string) -> Order                  # read
tool cancel_order(id: string, reason: string) -> Ok  # side effect

function answer(conversation):
    state = conversation
    while budget.check(max_steps=8):                 # 6
        step = llm_with_tools(PROMPT, [get_order, cancel_order], state)   # 2
        if step.is_text:
            return step.text                         # 7

        for call in step.tool_calls:                 # 3
            if call.name == "cancel_order" and not human.approve(call):   # 5
                result = error("the customer did not confirm the cancellation")
            else:
                result = run(call)                   # 4
            trace.span("tool", call, result)
            state.append(call, result)

    return "I will hand you over to someone on the team."   # budget exhausted

Real-world scenarios

• Support bot with human escalation. The bot looks up orders and tracking with read tools. Exchanges and cancellations require explicit customer confirmation, and refunds above a threshold go to a person.
• Personal finance assistant with approval. Checking balance and statements is free. Scheduling a Pix transfer (Brazil’s instant payment system) is a side-effect tool, and it only runs after the user reviews the summary and approves it in the app.
• Natural language to SQL. The tool runs queries as a read-only database user with a timeout. The model sees the result and adjusts the query when there is an error.

Applicability

• The answer depends on data that changes: orders, balances, inventory, calendars.
• The assistant needs to take actions in systems you control.
• You do not know in advance which data will be needed; it depends on the conversation.
• You want the model to correct itself by reading a tool’s error.

When not to use

• The data you need is always the same and fits in the prompt. Fetching it upfront and sending it along is simpler and cheaper.
• The sequence of steps is fixed. Call the functions in code and use the model only where needed, as in Prompt Chaining.
• The work is deterministic. If the model would only call a calculator, do the math in code: see the LLM Doing Deterministic Work smell.

How to implement

1. List the actions the assistant needs. Start small: three to five tools.
2. Separate reads from side effects. Side-effect tools get a gate from day one.
3. Write each description to say what the tool does and when to use it. Usage rules live in the description, not scattered across the system prompt. See Move Instruction to Tool Description.
4. Use narrow schemas: enums for kinds and a clear format description for identifiers. Turn on strict mode so the model does not invent fields, and validate formats in code.
5. Return short, clean JSON results. Errors with a clear message (“order not found”) help the model correct itself. Raw HTML becomes the Raw HTML Tool Output smell.
6. Run tools with the permissions of the conversation’s user, never with admin permissions.
7. Put a step budget on the loop and an exit path for when it runs out. The destination pattern is Stop Conditions.
8. Record every request and result in the trace, and build an eval that checks the right tool was called with the right arguments.

Common traps

• Two almost-identical tools. get_order and get_order_status fight. Consolidate before you think about a router.
• A vague result. "ok" does not say what changed. The model repeats the call.
• A gate that lives only in the prompt. “Only cancel if the customer asks” is a suggestion. The gate is code. Without it, you have Ungated Side Effect.
• Admin permissions. The tool inherits the user’s identity. Otherwise it becomes Overprivileged Tool.

Variations

• Manual loop. You write the while and control everything. That is the Python tab below.
• SDK runner. The SDK runs the loop and calls your functions. Less code, and the gate lives inside the tool function. That is the TypeScript tab.
• Parallel tools. The model requests several reads in one turn, and your code runs them all at once. Return all results together, in one message.
• Many tools. Past a few dozen, the model starts picking wrong. The way out is the Tool Router.

Cost, latency, and failure modes

How to evaluate

Evaluate the choice separately from the final answer.

• Right tool: for each case, which tool should be called (or none)?
• Right arguments: do the arguments match what you expected?
• Steps per task: how many turns the loop took. A sudden rise usually means a confusing description.
• Gate respected: in 100% of side-effect cases, approval came first.

An eval case:

{ "input": "cancel order 4521, I bought the wrong thing", "expected_tool": "cancel_order", "expected_args": { "id": "4521" }, "requires_approval": true }

Include cases where no tool should be called, like general questions. They catch a model that calls tools by reflex.

In production

• A span per tool call with name, arguments, a short result, duration, and error.
• An alert for loops that hit the step limit.
• Error rate per tool. A tool that fails often usually has a poor description or schema.
• A separate audit log for every side-effect tool: who asked, who approved, what changed.

Security

A tool result is text the model reads. If it comes from outside (an email, a web page, a customer comment), it can carry hidden instructions trying to make the model call another tool. Greshake et al. (2023) called that indirect prompt injection. Hence: a gate on every side effect, minimal permissions per tool, and attention to the Instructions from Untrusted Data smell.

In 2023, Samsung employees pasted internal code into ChatGPT. The lesson for a tool is the inverse: the tool should not return a secret the model does not need, and the result should not go to a provider outside policy. Drop fields, mask personal data (Brazil’s LGPD applies when that data is Brazilian), and run as the user.

Pros and cons

Relations

• Smells it treats: invented answers about live data, Ungated Side Effect when the action was only text, and part of Prompt Bloat when the patch was pasting data into the prompt.
• Refactorings: Move Instruction to Tool Description, Split God Tool, Add Approval Before Side Effect.
• Neighbors: Tool Router when the list grows. Human in the Loop is the gate. Replace LLM Call with Code when the tool hides a deterministic calculation.

Code

The tabs build the same assistant: one read tool, one side-effect tool behind a gate, and a step limit. Python writes the loop. TypeScript uses the Anthropic runner. Python + LangChain, Python + OpenAI Agents, Python + Pydantic AI, and TypeScript + AI SDK are the same loop on another stack. The gate stays inside the side-effect tool in every one.

import json

import anthropic

client = anthropic.Anthropic()
MAX_STEPS = 8

TOOLS = [
    {
        "name": "get_order",
        "description": "Fetches status, items, and delivery estimate for an order. Use it when the customer mentions an order number.",
        "strict": True,
        "input_schema": {
            "type": "object",
            "properties": {"id": {"type": "string", "description": "Order number, digits only"}},
            "required": ["id"],
            "additionalProperties": False,
        },
    },
    {
        "name": "cancel_order",
        "description": "Cancels an order that has not shipped. Only use it after the customer explicitly asks to cancel.",
        "strict": True,
        "input_schema": {
            "type": "object",
            "properties": {"id": {"type": "string", "description": "Order number, digits only"}, "reason": {"type": "string"}},
            "required": ["id", "reason"],
            "additionalProperties": False,
        },
    },
]


def run_tool(name: str, args: dict, approve) -> str:
    # 5. gate for side effects
    if name == "cancel_order" and not approve(args):
        return json.dumps({"error": "the customer did not confirm the cancellation"})
    # 4. run with your system's permissions (orders is your order service)
    handler = {"get_order": orders.get, "cancel_order": orders.cancel}[name]
    return json.dumps(handler(**args))


def answer(messages: list, approve) -> str:
    for _ in range(MAX_STEPS):  # 6. step budget
        reply = client.messages.create(
            model="claude-opus-5-5", max_tokens=4096,
            system="You are the store's order assistant.", tools=TOOLS, messages=messages,
        )
        if reply.stop_reason != "tool_use":  # 7. final answer
            return next(b.text for b in reply.content if b.type == "text")

        messages.append({"role": "assistant", "content": reply.content})
        results = [
            {"type": "tool_result", "tool_use_id": b.id, "content": run_tool(b.name, b.input, approve)}
            for b in reply.content if b.type == "tool_use"
        ]
        messages.append({"role": "user", "content": results})  # all results together

    return "I will hand you over to someone on the team."

import Anthropic from "@anthropic-ai/sdk";
import { betaZodTool } from "@anthropic-ai/sdk/helpers/beta/zod";
import { z } from "zod";

const client = new Anthropic();

export async function answer(question: string, approve: (args: unknown) => Promise<boolean>) {
  const getOrder = betaZodTool({
    name: "get_order",
    description: "Fetches status, items, and delivery estimate for an order. Use it when the customer mentions an order number.",
    inputSchema: z.object({ id: z.string().regex(/^[0-9]+$/) }),
    run: async ({ id }) => JSON.stringify(await orders.get(id)), // 4. read
  });

  const cancelOrder = betaZodTool({
    name: "cancel_order",
    description: "Cancels an order that has not shipped. Only use it after the customer explicitly asks to cancel.",
    inputSchema: z.object({ id: z.string().regex(/^[0-9]+$/), reason: z.string() }),
    run: async (args) => {
      // 5. gate for side effects
      if (!(await approve(args))) return JSON.stringify({ error: "the customer did not confirm the cancellation" });
      return JSON.stringify(await orders.cancel(args.id, args.reason));
    },
  });

  // 2 to 7. the runner calls the model, runs the tools, and repeats until the final answer
  const final = await client.beta.messages.toolRunner({
    model: "claude-opus-5-5",
    max_tokens: 4096,
    system: "You are the store's order assistant.",
    tools: [getOrder, cancelOrder],
    messages: [{ role: "user", content: question }],
  });
  return final.content.find((b) => b.type === "text")?.text ?? "";
}

import json

from langchain.agents import create_agent
from langchain.tools import tool


@tool
def get_order(id: str) -> str:
    """Look up status, items, and delivery estimate for an order. Use when the customer cites an order number."""
    return json.dumps(orders.fetch(id))  # 4. read


@tool
def cancel_order(id: str, reason: str) -> str:
    """Cancel an order that has not shipped. Use only after the customer explicitly asks to cancel."""
    # 5. gate for side effects
    if not approve({"id": id, "reason": reason}):
        return json.dumps({"error": "the customer did not confirm the cancellation"})
    return json.dumps(orders.cancel(id, reason))


# "provider:model". create_agent returns a LangGraph graph.
agent = create_agent(
    "openai:gpt-4o-mini",
    tools=[get_order, cancel_order],
    system_prompt="You are the store's order assistant.",
)


def answer(question: str) -> str:
    # 2 to 7. the agent calls the model, runs the tools, and stops on the answer
    result = agent.invoke(
        {"messages": [{"role": "user", "content": question}]},
        {"recursion_limit": 16},  # 6. each graph step counts
    )
    return result["messages"][-1].text

from agents import Agent, Runner, function_tool

@function_tool
def get_order(id: str) -> str:
    """Look up status, items, and delivery estimate for an order."""
    return json.dumps(orders.fetch(id))

@function_tool
def cancel_order(id: str, reason: str) -> str:
    """Cancel an order that has not shipped."""
    # 5. gate
    if not approve({"id": id, "reason": reason}):
        return json.dumps({"error": "the customer did not confirm the cancellation"})
    return json.dumps(orders.cancel(id, reason))

agent = Agent(
    name="orders",
    instructions="You are the store's order assistant.",
    tools=[get_order, cancel_order],
    model="gpt-4o-mini",
)

def answer(question: str) -> str:
    result = Runner.run_sync(agent, question, max_turns=8)  # 6
    return str(result.final_output)

from pydantic_ai import Agent
from pydantic_ai.usage import UsageLimits

agent = Agent("openai:gpt-4o-mini", system_prompt="You are the store's order assistant.")

@agent.tool_plain
def get_order(id: str) -> str:
    """Look up status, items, and delivery estimate for an order."""
    return json.dumps(orders.fetch(id))

@agent.tool_plain
def cancel_order(id: str, reason: str) -> str:
    """Cancel an order that has not shipped."""
    if not approve({"id": id, "reason": reason}):  # 5
        return json.dumps({"error": "the customer did not confirm the cancellation"})
    return json.dumps(orders.cancel(id, reason))

def answer(question: str) -> str:
    return agent.run_sync(question, usage_limits=UsageLimits(request_limit=8)).output

import { generateText, stepCountIs, tool } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

export async function answer(question: string): Promise<string> {
  const result = await generateText({
    model: openai("gpt-4o-mini"),
    system: "You are the store's order assistant.",
    prompt: question,
    stopWhen: stepCountIs(8),
    tools: {
      get_order: tool({
        description: "Look up status, items, and delivery estimate for an order.",
        inputSchema: z.object({ id: z.string() }),
        execute: async ({ id }) => JSON.stringify(await orders.fetch(id)),
      }),
      cancel_order: tool({
        description: "Cancel an order that has not shipped.",
        inputSchema: z.object({ id: z.string(), reason: z.string() }),
        execute: async ({ id, reason }) => {
          if (!(await approve({ id, reason }))) {
            return JSON.stringify({ error: "the customer did not confirm the cancellation" });
          }
          return JSON.stringify(await orders.cancel(id, reason));
        },
      }),
    },
  });
  return result.text;
}

Framework support

Checked in October 2026: LangChain 1.x (create_agent in place of create_react_agent), OpenAI Agents (Runner.run_sync, max_turns), Pydantic AI (UsageLimits), and the AI SDK (stepCountIs).

Check your understanding