What is Replace LLM Call with Code?
Replace LLM Call with Code is an AI refactoring for LLM systems, in the Simplifying tools group. Move calculation, formatting, and validation into a function the model calls.
Trail content
Chapters
Transcript
Opening
An invoice pipeline asked the model to do six things in one call. Extract the line items, add them up, apply the discount, compute the sales tax on each line, convert the date, and validate the issuer's tax ID.
On most invoices, the total matched. But in one batch, three invoices in a hundred reached the accounting system with the tax wrong in the second decimal place.
And there was no pattern. Sometimes it was rounding.
Sometimes a line counted twice in the sum.
And sometimes a digit of the tax ID that the model decided to fix, all on its own.
Arithmetic, sales tax and check digits aren't a matter of interpretation. They're rules. The model generates the next token by probability. A sum function doesn't. In this lesson, you'll move that work out of the model, safely, and discover a bonus you probably didn't expect.
The idea
The solution is to take the deterministic operation out of the prompt. The model extracts, or decides, what only it can. A regular function does the math, the formatting, or the validation. And when the model needs the result in the middle of a conversation, that function is offered to it as a tool.
Before and after
Before, understanding and calculating happen in the same call. The invoice goes in, numbers come out, and they go straight into the system.
After, the model reads, and code calculates and checks. The invoice arrives, and the model only extracts values, exactly as printed.
Regular functions handle the sums, the tax, the dates and the tax ID check.
Code compares the calculated total with the printed total.
And only a coherent invoice goes into the system. A mismatch goes to review.
A real case
Here's one invoice. The model returns two items. Two units at one hundred fifty dollars, and one unit at four hundred. Both with an eighteen percent tax rate. No discount. And a printed total of seven hundred dollars.
Code does the math. Two times one fifty, plus four hundred, is seven hundred. It matches the printed total. Then it computes the tax with the rounding rule the tax team chose, and validates the tax ID.
Now here's the bonus. Say the model had misread one fifty as fifteen. The sum wouldn't reach seven hundred, and the invoice would go to review.
Before, the model could quietly adjust the sum to make it close, and the reading error just disappeared. Now the math in code becomes a checker for the extraction. The error that used to slip by silently now raises a flag.
A challenge
Here's one for you. The team created a tool called sum items. It sends the amounts to a small model, and asks for the sum. The math eval improved a little. Is the refactoring done?
No. The math left one prompt and entered another. It's still token generation, it still makes mistakes, and now the mistake lives in a tool with an innocent name.
A tool, here, is a function. Sum, a rounding function, a tax ID validator. Here's the test. If the operation can't be unit tested without calling a model's API, it hasn't left the model yet. And cheaper and still wrong is still wrong.
Safe mechanics
Now the mechanics.
Step zero, measure. Run the extraction eval, and split the errors by type. Reading errors, where a value was misread. And math errors, in the sum, the tax or the format. If you don't split them, you'll think the model improved, when it only got the sum right by coincidence.
List the deterministic operations in the prompt. Every verb of calculation, conversion or validation. Add up, compute, convert, validate the check digit.
Write a function for each one, with unit tests, including the edge cases. Half a cent, a zero rate, national holidays, a tax ID whose check digit is zero. And the tax rule comes from a conversation with whoever owns tax, not from the model's guess.
Rewrite the prompt to ask only for raw data, as it appears in the document. And take the computed fields, like the tax and the total, out of what the model fills in.
Wire the functions in, right after extraction.
Or, if the model needs the result during a conversation, expose them as tools. A delivery assistant that needs the next business day mid-conversation calls a function that adds business days. The model doesn't add days. It asks.
Use mismatches as an alarm. When the code's math disagrees with the document, send it to review.
And measure again. Math errors should drop to zero. Reading errors are still there, but now the cross-check catches them.
Risks
A few risks. Interpretation hiding inside the math. Total may or may not include shipping. Settle that rule with someone who knows the business, before you code it. Otherwise you swap the model's error for yours. Deterministic, and on every single invoice.
Raw formats vary. Asking for a date as written pushes the parsing into code. Test the parser on real formats from the invoice folder.
A tool the model ignores. If the function is a tool, the model may skip it and calculate on its own. Say in the description when to use it, and check the trace for the tool call.
And more reviews, at first. The cross-check will catch errors that used to pass silently. That's the goal. But warn the operations team before they think something broke.
And if operations does complain, don't switch the alarm off. The old normal volume was invisible error. Classify the reviews. Separate real mismatches from a business rule you coded badly. And fix the right one.
There's one tempting shortcut to resist. The code's sum and the printed total differ by two percent, and someone suggests letting the model adjust the total, because it understands implicit discounts. That's the smell coming back. A real discount becomes an extracted field, not a silent adjustment.
What stays with the model
So what stays with the model? Anything with no closed rule a function could apply. Deciding whether a contract paragraph is a penalty clause or a confidentiality clause is classification. No function computes the meaning of a paragraph. Validating a check digit, or turning a date into a calendar date, is code.
Wrap-up
So here's the idea to take with you. Split reading from calculating. The model copies what's on the page, functions do every new number, and the comparison between the two becomes your best alarm for misreads. A refactoring that removes errors, and also finds the ones you couldn't see.
So far, the errors came from the model reading wrong. In the next lesson, we'll look at what happens when the call itself fails, and the code just tries again: blind retry. See you there.
Replace LLM Call with Code
Trocar Chamada de LLM por Código
Replace LLM Call with Code is an AI refactoring for LLM systems, in the Simplifying tools group. Move calculation, formatting, and validation into a function the model calls.
- Element
- 21/35
- Level
- Beginner
- Read
- 4 min
- Updated
Problem
The model is calculating, formatting, or validating something that has an exact rule, and it gets it wrong now and then with no pattern. This is the LLM Doing Deterministic Work smell.
The NF-e pipeline (Brazil’s electronic invoice) asked the model: extract the line items, add them up, apply the discount, compute ICMS (a Brazilian sales tax) on each line, convert the date, and validate the CNPJ (company tax id). On most invoices the total matched. In one batch, three invoices in a hundred reached the ERP with ICMS wrong in the second decimal. There was no pattern to the error: sometimes rounding, sometimes a duplicated line in the sum, sometimes a CNPJ digit the model “fixed” on its own.
Arithmetic, tax, and check digits are not interpretation. They are rules. The model generates the next token by probability; the sum function does not.
Solution
Take the deterministic operation out of the prompt. The model extracts or decides what only it can; a regular function does the math, formatting, or validation, or is offered to the model as a tool.
Before and after
-PROMPT = """Extract the line items, add up the amounts, apply the discount,
-compute each item's ICMS, convert dates to YYYY-MM-DD,
-and validate the issuer's CNPJ."""
-result = llm_parse(PROMPT, nfe_text, schema=InvoiceWithTotals)
+PROMPT = """Extract the line items exactly as they appear on the invoice:
+description, quantity, unit price in cents, ICMS rate.
+Also extract the printed total, the discount, and the issue date."""
+raw = llm_parse(PROMPT, nfe_text, schema=RawInvoice)
+
+items_total = sum(i.qty * i.unit_cents for i in raw.items)
+expected = items_total - (raw.discount_cents or 0)
+icms = [round_half_even(i.qty * i.unit_cents * i.icms_rate) for i in raw.items]
+issue_date = parse_date_br(raw.issue_date_text)
+cnpj_ok = validate_cnpj(raw.issuer_cnpj)
+
+if expected != raw.printed_total_cents or not cnpj_ok:
+ send_to_review(raw, reason="sum or CNPJ does not match")Before: understanding and calculating in the same call.
flowchart LR
D(["1. NF-e"]):::io --> M(["2. Extract and calculate"]):::llm
M --> E[("3. ERP")]:::store
After: the model reads, code calculates and checks.
flowchart LR
D(["1. NF-e"]):::io --> M(["2. Extract raw data"]):::llm
M --> C["3. Add, calculate, validate"]:::tool
C --> V{"4. Does it match?"}
V -->|yes| E[("5. ERP")]:::store
V -->|no| H((("Review"))):::human
- The invoice arrives.
- The model only extracts values as printed.
- Regular functions handle sums, tax, dates, and CNPJ validation.
- Code compares the calculated total with the printed total.
- Only a coherent invoice goes into the ERP. A mismatch goes to review.
One invoice, two jobs
The model returns, for example:
{
"items": [
{"qty": 2, "unit_cents": 15000, "icms_rate": 0.18},
{"qty": 1, "unit_cents": 40000, "icms_rate": 0.18}
],
"discount_cents": 0,
"printed_total_cents": 70000,
"issuer_cnpj": "12345678000190",
"issue_date_text": "03/01/2026"
}Code computes 2 * 15000 + 40000 = 70000, checks it against the printed total, applies banker’s rounding to ICMS, and validates the CNPJ. If the model had read 1500 instead of 15000, the sum would not match 70000 and the invoice would go to review. Before, the model itself could “adjust” the sum to close, and the reading error disappeared.
Why it works
A language model generates numbers the way it generates words: token by token, by probability. Most of the time that produces the right result. A function produces the right result every time, costs almost nothing, and can be tested on edge cases (R$ 0.005, a zero rate, a CNPJ whose check digit is 0).
The split has a second, often more valuable effect: the math in code becomes a checker for the extraction. If the calculated total does not match the printed one, some value was misread. The error that used to slip by silently now raises a flag.
When the calculation is needed mid-reasoning (the assistant needs the delivery date to answer), the function becomes a tool: the model asks, code calculates, the model uses the result. That is Tool Use, not a second LLM hidden behind the name calculate.
This refactoring pairs with Replace Free Text with Structured Output: the schema asks for the raw fields (unit_cents, icms_rate), and code derives the rest.
Safe mechanics
- Measure. Run the extraction eval and split errors by type: reading (value misread) and math (sum, tax, format). If you do not split, you will think the model “improved” when it only got the sum right by coincidence.
- List the deterministic operations in the prompt: everything with a verb of calculation, conversion, or validation. “Add up”, “compute”, “convert”, “validate the check digit”.
- Write a function for each, with unit tests, including edge cases (rounding, national holidays, a CNPJ with zeros). The ICMS rule comes from a conversation with whoever owns tax, not from the model’s guess.
- Rewrite the prompt to ask only for raw data, as it appears in the document.
- Adjust the schema to the raw fields. Take computed
icms_centsandtotal_centsout of what the model fills in. - Wire the functions in after extraction, or expose them as tools if the model needs the result during the conversation.
- Use mismatches as an alarm: when the code’s math does not agree with the document, send it to review.
- Measure again. Math errors should drop to zero; reading errors now get caught by the cross-check.
Risks
- Interpretation hidden in the math. “Total” may or may not include shipping. Settle the rule with someone who knows the business before coding it. Otherwise you only swap the model’s error for yours, deterministic and on every invoice.
- Varied raw formats. Asking for text as it appears (“03/12/26”) pushes parsing into code. Test the parser on real formats from the invoice folder.
- Unused tool. If the function becomes a tool, the model may ignore it and calculate on its own. Say in the description when to use it, reject a result the model “calculated” in prose, and check the trace for the tool call.
- More reviews at first. The cross-check will catch errors that used to pass. That is the goal, but warn the operations team.
Common mistake: a tool that is another prompt
Someone wraps llm("add these numbers: " + items) and calls it the sum_items tool. The math is still in a model; only the address changed. A tool, here, is a function: sum, round_half_even, validate_cnpj. If the operation has no unit test without an API, it has not left the LLM yet.
Challenge
Which parts of this HR assistant prompt should become code?
Answer questions about vacation. Employees get 30 days per year.
Calculate the balance by subtracting days already taken. If they ask
to sell 10 days, calculate the payout: salary divided by 30, times 10,
plus one third. Answer in a friendly tone.Show solution
The balance and the payout amount are calculations with exact rules, and they become tools. The model stays responsible for understanding the question and explaining the result.
tool vacation_balance(employee_id) -> { days_total, days_taken, days_left }
tool vacation_payout(employee_id, days_sold) -> { amount_cents, breakdown }
Prompt: Answer vacation questions in a friendly tone. For the balance,
use vacation_balance. For selling days, use vacation_payout and explain
the calculation using the breakdown. Never calculate amounts yourself.Check your understanding
Lesson check
Answer the 6 questions to complete this lesson. Watching alone does not complete it: the check is what counts toward the trail and the certificate.
Quick answers
What problem does Replace LLM Call with Code solve?
It treats LLM Doing Deterministic Work.
Which pattern does Replace LLM Call with Code lead to?
The Tool Use and Code Execution Sandbox design pattern.
Go further
Relations
- Smells it treats 1
- Target patterns 2
- Shows up in scenarios 2
- Part of the tracks 1