Opening

The pull request that changed the invoice extractor had one line. Extract the invoice total became: extract the invoice total, with two decimal places.

The description said: format tweak, tested on three invoices. The reviewer approved it. The change looked harmless.

A week later, accuracy on the total field for discounted invoices had dropped. And nobody knew since when, or by how much. Because there was no number from before to compare against.

Here's what happened. The three test invoices were the ones that motivated the change. Round totals, no discount.

To force two decimal places, the model started combining the goods value and the discount.

And the total came out wrong, on exactly the invoices the team never looked at. In this lesson, you'll learn to recognize this smell, and the order of work that removes it.

The diagnosis

The cycle looks like this. Someone changes the prompt to fix one case.

They test with the few examples they have on hand.

The change ships, with no comparison against the previous version.

The effect shows up on cases nobody tested.

And users discover the regression. Late, and with no data.

How do you spot it? Prompt changes go through code review as if they were text, with no results attached.

The test for a change is: I ran a few examples, and it looked fine.

There's no set of cases with expected answers stored in the repository.

Nobody can state the system's current accuracy. Only that it works. And switching models keeps getting postponed, because nobody knows what will break.

And production bugs get fixed in the prompt, and never become test cases.

Why it happens

A prompt looks like text, not code. Changing a sentence doesn't feel like changing logic. But with a model, a sentence is logic. With two decimal places changed the total field.

Building a dataset is work. Collecting and labeling fifty cases seems expensive next to a one line change. The cost shows up later, in the silent regression.

Output varies. Not knowing how to test something that isn't deterministic, the team gives up on testing. But a grading rule, like comparing fields, covers most of it.

And the bug's cases become the only test. Whoever fixes a bug tests the case that broke. And nothing else.

What it costs

The first cost is the silent regression. Quality drops on part of the cases, and nobody notices until the cost shows up somewhere else. A wrong payment slip. Tax computed on the wrong total. A seller with a rejected invoice.

Then fear. With no safety net, the team avoids improving the prompt, or switching models, even when it would be cheaper.

Then endless debates. It got better, versus it got worse, with no number to settle it.

And bugs that come back. The error fixed in March returns in June, because nothing stops it.

And here's how the hole hides, with illustrative numbers. Two hundred invoices in the eval, six of them with a discount. The overall pass rate drops from ninety-seven to ninety-four percent. It looks like noise. But in the discount category, six out of six now fail. The average hid exactly the hole that finance is going to find.

A challenge

Here's one for you. You have no eval at all, and you need to fix an extractor that gets totals wrong, by tomorrow. What's the first step you still have time to do?

Label the fifteen invoices you already know break. Measure the current prompt on them. Then change it. Fifteen cases with expected answers already are an eval. Measuring first gives you the floor. And those fifteen go into the repository, and run on the pull request.

Shipping now and filing a card to add an eval next sprint doesn't work. The card sits behind the next fix. And asking a model to generate two hundred synthetic invoices as ground truth misses the crooked discount of a real one. Start with what already failed in production.

The treatment

The treatment is a refactoring called add eval before changing the prompt. And the order matters. It's the part teams invert most often.

First, store cases with expected answers. Start with production bugs, and a couple of dozen happy examples. Split them by category. With discount, missing line item, a rare tax code.

Measure the current version. That number is the floor. Without it, every it got better is just talk.

Change the prompt, or the model, on a branch.

Measure again. Look at the overall rate and the rate per category. A small category at zero is a regression, even if the average held still.

And the case that broke goes into the set. Otherwise it comes back.

And in your pipeline, the eval runs on every pull request that touches a prompt, a template, or a model. That last part matters. A team with a forty case eval that hasn't run in four months, with half the cases in an old format, still has this smell. An eval that doesn't run protects nothing.

Close relatives

Some close relatives. Before each deploy, someone asks the bot the same eight questions and eyeballs the answers. That's a ritual, not a measurement. It has its own name: vibe check eval.

A team runs two hundred cases on every change, but only looks at the overall rate. That's not this smell. The eval exists. It just needs results per category, and an alert when any of them drops.

And a production bug, total ignores discount, fixed in the prompt. The pull request passed a two hundred case eval that had no discounted invoice at all. It passed because it couldn't see the bug. Add the invoice that broke, and others like it.

When to ignore

When can you live without it? A prototype, in its first week. Before you know what the system should do, testing by hand is reasonable. Start the dataset when the first real user shows up.

And a personal script that summarizes your meetings can live without an eval. Once someone else depends on it, it can't.

Wrap-up

So here's the idea to take with you. With a model, a sentence is logic. Treat every prompt change like a code change. With stored cases, a number from before, a number from after, and the broken case added to the set. Then switching models becomes a measured decision, made in an afternoon.

In the next lesson, we'll do that refactoring step by step: add eval before changing the prompt. See you there.