Opening

Rafael wrote a small function. It sends a product description to the model and asks for the product's category.

In testing, it returned Electronics ten times in a row. Perfect.

In production, on the very same product, it returned Electronic, singular. Then electronics, lowercase, with a period at the end. And once, Electronics and Computers.

The code that matched the answer against the marketplace's category list broke. And some new products never made it to a storefront.

Nothing was wrong with the model. Rafael treated a model call like an ordinary function, one that always returns the same output for the same input. It isn't one. In this lesson, you'll learn the four properties that explain what happened, and that almost every pattern in this catalog exists to control.

The idea

For an engineer, here's the most useful way to think about a large language model. It's a probabilistic function from text to text.

It takes a sequence of tokens, which are small chunks of text. And it returns the most likely continuation, one token at a time.

Two words in that sentence matter a lot. Probabilistic, because there's a dice roll inside. And continuation, because the model isn't looking anything up. It's completing text.

How it works

Let's walk through one call. Everything the model knows about your task is in the input. The system prompt, the conversation, any documents, and the tools you offer.

The model computes a probability for every possible next token.

Then one token is picked from those probabilities, with some randomness.

That token is appended to the text, and the cycle runs again.

It stops when the model emits an end token, or when it hits the output limit you set.

Four properties fall out of that design. And each one changes how you write the code around the model.

Four properties

First, it's not deterministic. The same input can produce different outputs, because step three is a dice roll. A small difference in an early token changes everything after it. That's what hit Rafael. Electronics and Electronic are one token apart.

So your code has to accept variation, or remove it at the boundary. That's why structured output exists.

Second, it has no memory. Every call starts from scratch. If the model seems to remember the conversation, it's because your code sent the whole history again.

Think about the third message of a chat. The customer asks: and shipping for the cheapest one to Boston? If your code sends only that last message, the model has no idea what the cheapest one means. Memory is your job. And every stored turn gets paid for again, on every call.

Third, it doesn't know what isn't in the input. The model learned a lot in training. But it doesn't know your database, your return policy, or what changed yesterday. Either you put that information in the input, or it writes something plausible.

And plausible isn't true. In twenty twenty-three, lawyers in New York filed a brief citing court decisions that a chatbot had simply made up. The judge sanctioned them. The model had no case law in its input, so it completed the text with cases that looked real. That's the problem retrieval, or RAG, solves.

Fourth, it guarantees nothing. The model has no contract. It can ignore an instruction, get the math wrong, or invent a field. Every real guarantee comes from the code around it.

In twenty twenty-four, an airline's chatbot told a grieving customer about a refund rule that wasn't the real policy. A Canadian tribunal ordered the airline to pay. It rejected the idea that the chatbot was responsible for its own words. To the customer, what the bot says is what the company said.

A challenge

Here's one for you. Your chatbot answers the customer's first question well. On the second one, and how much is shipping for it, it replies: which product do you mean? What's the most likely cause?

Your code is sending only the current message, without the previous turns. The model keeps nothing between calls, so the word it has nothing to point to.

It's not a full context window. Two short messages don't come close. And it's not temperature either. Randomness changes how the model phrases things, it doesn't erase a conversation.

What a call is

In practice, a call has four parts you control. The model. The common mistake is using the biggest one for everything, and overpaying.

The system prompt, with the fixed instructions. The mistake there is turning it into a dumping ground for rules.

The messages. Here, people send too much history, or too little.

And the output limit. Set it too short, and the reply gets cut in the middle of a sentence.

The response also tells you why generation stopped, in a field called stop reason. End turn means the model finished. Max tokens means your limit cut it off. A summary that stops mid-sentence, with a format that never closes, is almost always the second case. So check that field before you try to parse anything.

Try it

On the page there's a five minute experiment. You ask the same question five times with an open prompt, and five times with a closed list of categories.

Look at three things. Whether the five answers are identical. Whether the form changes, like capitals, plural, or a trailing period. And whether even the closed list produces an invalid answer.

Even if every answer passes today, the last line is the point. The check against your list lives in code. And that's what protects the rest of the system when the model goes off script.

Model work, code work

So here's the rule of thumb. Let the model do what only it does well. Understanding text, writing, deciding in ambiguous cases.

And let code do everything else. Math, comparing dates, looking up a table, business rules.

Picture a shipping calculator. The model reads the customer's message and pulls out the weight and the zip code. Then a ten line function looks up the carrier's table and does the math. Exact, cheap, and it never has a bad day.

The common mistake is the opposite. Pasting the rate table into the prompt, and asking the model to do the math. It works most of the time, and fails now and then, silently. When the model does that kind of work, you have a smell called LLM doing deterministic work.

In production

Now think about Rafael again, and about anything that writes to a real system. Say the model reads an invoice and your code writes the total straight into the accounting system. Where should the guarantee that the amount makes sense live?

Not in the prompt. Writing never get the total wrong, even in capitals, raises the hit rate, but it never becomes a guarantee. And not in a bigger model either. It makes fewer mistakes, and still offers no promise.

It lives in code. The total must be a positive number, and it must match the sum of the items minus the discount. If it doesn't, the invoice goes to a review queue, instead of landing wrong in your books.

Wrap-up

So here's the thing to take with you. A model call is not a function with a contract. It varies, it forgets, it only knows what you hand it, and it promises nothing. Your code is what turns almost always into always handled.

In the next lesson, we'll look at the unit everything is measured in, the token, and at the context window, the space where all of this has to fit. See you there.