Unused Tools

Signs and symptoms

An internal data agent has 12 tools. Nobody remembered why some of them were there, so the team counted calls in the last 90 days of traces:

tool                    calls      last call
query_warehouse         18,402     today
describe_table          6,117      today
list_tables             2,954      today
export_csv              1,088      yesterday
legacy_query_v1         0          7 months ago
send_report_email       0          never
create_dashboard        0          never
refresh_cache           0          5 months ago
get_metadata_old        0          7 months ago
schedule_job            0          never
delete_temp_tables      0          4 months ago
run_python              0          never

Eight of the twelve tools have not been called in three months. Four never were. Still, all of them go out with every model call. Two of them (delete_temp_tables and run_python) have side effects: they delete data or run code.

Signs of Unused Tools:

• Tools that have not shown up in traces for months stay on the list.
• Nobody can say whether any of them is still needed, or who asked for it.
• Old versions live next to new ones (_v1, _old, legacy_).
• Side-effect tools stay available although no task needs them.
• The definitions take a meaningful share of every call’s context.

flowchart LR
  T[("1. 12 tools on the list")]:::store --> M(["2. Model reads them all on every call"]):::llm
  M --> U["3. Always uses the same 4"]:::tool
  T --> X["4. 8 unused, 2 with side effects"]:::untrusted

1. The list grew over time, one tool per request.
2. Every definition enters every call’s context, used or not.
3. Real usage concentrates on a few.
4. The rest only take up space and, when they have side effects, raise risk.

What a tool nobody calls weighs

A tool definition is text: name, description, and the JSON Schema of its parameters. The provider injects that text into the prompt on every request. A definition with a decent description and four or five parameters easily goes past a hundred tokens.

Run the math with example numbers. Suppose 200 tokens per definition and 8 unused tools: that is 1,600 extra input tokens per call. At 10,000 calls a day, you pay for 16 million tokens a day so the model can read tools it never picks. To get your real number, count the tokens of the tools array with your provider’s tokenizer.

Check +20 XP on first tryYour agent has 15 tools and the last 60 days of traces show 9 were never called. The agent answers well. What is the most direct cost of keeping those 9?

None, since the model does not pick them

Input tokens paid on every call, plus more options competing in the choice

Only the cost of maintaining the tools' code

Reasons

• Tools come in and never leave. Adding one takes five minutes. Removing one requires finding out whether anyone uses it, and nobody has that data.
• Plans that never happened. create_dashboard was built for a feature that never shipped. The feature died; the tool stayed.
• Half-done migrations. The new version came in, the old one stayed “just in case”. Six months later, nobody remembers which is which.
• No usage data. Without one span per tool call, the only source of truth is the team’s memory.
• Generous demos. To impress in a presentation, the agent got tools that “might be useful”. In production, nobody asked for them.

Cost of ignoring

• Wasted tokens on every call, as in the math above. The cost grows with volume while nothing new happens.
• Wrong choices. legacy_query_v1 and query_warehouse have similar descriptions. One day the model picks the old one, which reads a stale table, and the answer comes out wrong with no error at all.
• Attack surface. A side-effect tool nobody uses is a door nobody watches. If malicious text gets into the context, run_python is right there. The OWASP Top 10 for LLM Applications (2025 edition) calls this Excessive Agency, and “excessive functionality” is the first cause on its list.
• Ghost maintenance. An API changes and breaks the forgotten tool. Nobody notices until the day the model decides to use it.

Treatment

Refactoring	Choose it when
Consolidate Tools	Always. Remove the ones that are never called, merge the ones that overlap, and keep removals reversible for a while. Targets: Tool Router and Progressive Disclosure.

Before removing anything, you need the count. If it does not exist, start with Add Tracing and let it run for a few weeks. With per-tool spans, the count is one query:

select tool_name, count(*) as calls, max(started_at) as last_call
from spans
where kind = 'tool_call' and started_at > now() - interval '90 days'
group by tool_name
order by calls desc

Remove in stages

Taking the tool out of the schema and deleting its code on the same day is the fastest way to learn, in production, that a rare flow depended on it. Do it in three stages:

1. Hide. Remove the definition from the list sent to the model, but keep the code. Run the agent’s eval before and after to confirm nothing got worse.
2. Watch. For a few weeks, log any attempt to call the removed name. Models sometimes “remember” tools mentioned in examples or in the history.
3. Delete. With no attempts and a stable eval, remove the code.

The rare-tool trap

Zero calls in 90 days does not prove a tool is useless. Some tasks follow a calendar: quarter close, annual tax filing, disaster recovery. Before deleting, ask the owner of the flow. If the tool is rare but needed, the answer is not to remove it but to load it only when the task calls for it, with Progressive Disclosure.

Check +20 XP on first tryThe team hid export_csv_v1 from the schema on Monday. On Thursday, the log shows two attempts by the model to call that name. What does this indicate?

That removing it was a mistake and the tool should come back

That the model is hallucinating and the log can be ignored

That some prompt, example, or history still mentions the old name

Payoff

Before	After
12 definitions on every call	4 always visible, rare ones on demand
Two versions of the same query	One tool per job
run_python available with no use	No side-effect tool without an owner
A list nobody reviews	A short list with measured usage

When to ignore

• Rarely used but critical tools. A disaster-recovery tool may be called once a year. Keep it, but consider loading it only when needed.
• A freshly launched agent. After two weeks of use, the count says little. Wait for data that covers at least one full business cycle.
• A small list with no side effects. Three read-only tools, one of them rarely used, do not justify a refactoring. Write it down and revisit it with the next change.

Is it this smell?

Check +20 XP on first tryA tool was called 3 times in 90 days, always in month-end close tasks, and it solved the task all three times. Is this an Unused Tool?

No, but it can be loaded on demand

Yes, because usage is very low

Yes, and it should be removed to save tokens

Check +20 XP on first tryThe agent has 6 tools, all used every week, but two do almost the same thing under different names. Which smell?

Unused Tools

Tool Sprawl

Check +20 XP on first tryThe team has no per-tool traces and wants to know which tools to remove. What is the first step?

Remove the ones that look old by their names

Ask the model which tools it uses

Record each tool's usage for a few weeks

Check +20 XP on first tryAmong the unused tools is delete_temp_tables, which drops tables. The agent only answers read-only questions. Why does it deserve priority in the cleanup?

Because it is the one that costs the most tokens

Because it is a destructive action within the model's reach with no task that needs it

Because side-effect tools get slow when they go unused