Prompt as State Machine

Signs and symptoms

A WhatsApp onboarding flow signs up a fintech’s customers. The whole flow is described in the prompt (anonymized):

Follow these steps, in order:
1. Greet and ask for the full name.
2. Ask for the tax ID (CPF). Validate the format.
3. Ask for the company registration number (CNPJ).
4. Ask for a photo of an ID document.
5. Ask for proof of address (at most 90 days old).
6. Only after ALL steps, send the terms.
Do not skip steps. Do not repeat completed steps.

The trace of a real conversation shows the tax ID requested three times, because the customer answered with the tax ID and the company number in the same message and the model lost count. In another conversation, the customer said “I already sent the proof yesterday” (they had not) and the model jumped to the terms.

Signs of Prompt as State Machine:

• The prompt has numbered steps and instructions such as “do not skip” and “do not repeat”.
• Conversation state (what has been collected) only exists in the message history.
• The model skips, repeats, or reorders steps now and then.
• Out-of-order answers, or two facts in one message, confuse the flow.
• Resuming a conversation after a while leads to repeated questions.
• The user can convince the model that a step has already been done.

flowchart LR
  IN(["1. Message"]):::io --> H[("2. History as the only state")]:::store
  H --> P(["3. Prompt with the script"]):::llm
  P --> D{"4. Model guesses the step"}
  D --> OUT(["5. Next question, sometimes wrong"]):::io

1. Each message arrives.
2. There is no record of what was collected, only the conversation.
3. The prompt describes the script in text.
4. The model has to rebuild, on every message, which step it is on.
5. When it rebuilds wrongly, it skips or repeats.

Check +20 XP on first tryThe customer sends tax ID and company number in the same message. The bot asks for the tax ID again on the next turn. What is the design failure?

The prompt should stress more strongly that steps must not be repeated

State (fields already collected) does not exist outside the history, so two facts in one turn break the count

The model is weak on Brazilian documents and needs a tax-ID example in the prompt

Reasons

• Describing the flow in text is fast. One paragraph replaces a state diagram and its code.
• It works on the happy path. In tests, the user answers one thing at a time, in order.
• The model seems to understand the process. And it does most of the time, which hides the problem.
• There is nowhere for state to live. The system started as a chat, and a chat only has messages.

KYC (customer onboarding) in Brazil asks for a CPF, an ID photo, and proof of address with a date limit. LGPD (the Brazilian privacy law) asks that you know what you collected. An onboarding whose “state” is the WhatsApp history cannot answer either requirement with precision.

The trap is treating the model as a workflow engine. It is good at extracting “this is a tax ID” and at asking in a human tone. It is bad at being the source of truth for “we already have valid proof”. Someone who says “I already sent it yesterday” is negotiating with the state machine. If the machine is the model, the negotiation wins.

Cost of ignoring

• Skipped mandatory steps. A sign-up without proof of address can be a regulatory problem.
• Drop-off. Asking for the same data three times annoys people, and the customer gives up.
• Manipulation. The user convinces the model they already delivered something.
• Fragile resumes. Long or interrupted conversations lose the thread.

What code would see if state were explicit

After the message with tax ID and company number together, the record should be:

{
  "name": "Ana Souza",
  "cpf": "123.456.789-00",
  "cnpj": "12.345.678/0001-99",
  "id_photo": null,
  "address_proof": null,
  "terms_sent": false
}

The next question is the ID photo, every time. “I already sent the proof” does not change address_proof until the file is stored. The model writes the line. The graph picks the step.

Treatment

Refactoring	Choose it when
Replace Prompt Rule with Graph Edge	Always. The steps become states and transitions in code; the model extracts data and writes the questions. Targets: Routing and Prompt Chaining.
Move to Durable Execution	The flow lasts hours or days, with waits for documents or approvals. State is persisted at every step. Targets: Durable Workflow and Checkpoint and Resume.

Who does what:

1. Code stores the fields and picks the next step.
2. The model extracts what arrived in the message (one field or several) and writes the question.
3. Format checks (tax ID, proof date) are code, not “validate the format” in the prompt.
4. If the customer disappears and comes back the next day, the resumed run reads the same record, not the chat.

A common mistake: the graph lives in code, but on every message you ask the model to “read the history and say which step we are on”. That is the smell again, with a diagram on the wall.

Check +20 XP on first tryThe flow is already a graph in code, but on every message the system asks the model to read the WhatsApp thread and return the filled fields. What risk remains?

None: if the graph is in code, the prompt is no longer the state machine

Only latency, because an extra call makes the turn more expensive

Fragile state: fields rebuilt from history can be misread or invented by the user

Payoff

• Mandatory steps guaranteed by code.
• Out-of-order answers put to use, with no repeated questions.
• Exact resumes after days, because the state is saved.
• No “I already sent it” accepted unless the data is on record.

When to ignore

• Open conversations with no mandatory steps. An assistant that only answers questions does not need a state machine.
• A short, low-risk flow. Two optional questions can stay in the prompt.

Is it this smell?

Check +20 XP on first tryA scheduling bot's prompt says: 'ask for the service, then the date, then the time'. Sometimes it asks for the time before the date, but the booking always comes out right because code validates everything at the end. Is this Prompt as State Machine?

Yes, but low risk

No, because the result is right

No, this is Unbounded Loop

Check +20 XP on first tryThe sign-up flow is a graph in code, but state (collected fields) is rebuilt by reading the history on every message. What risk remains?

None, the graph is in code

Fragile state

Rule Stacking, because the graph is complex

Check +20 XP on first tryThe prompt has 20 rules in capitals, none about step order, all about tone and content. Which smell applies?

Prompt as State Machine

Rule Stacking

Routing System Prompt

Check +20 XP on first tryAn internal assistant asks two optional questions ('which team?' and 'is it urgent?') described in the prompt. If the user already states both, the bot continues. Should you split this into a state machine now?

Yes, any numbered step in a prompt is already this smell in production

Yes, because the user might lie about the team

No: a short, optional, low-risk flow can stay in the prompt