Unversioned Prompt

Signs and symptoms

A content pipeline generates posts for a brand’s blog. One of them went viral, and the marketing team wants more like it. The team channel conversation (anonymized):

mkt:  which prompt produced the remote work post? we want to repeat it
eng:  the current prompt is this one
mkt:  we tried it, the result is not similar
eng:  then it changed later. the post is from 08/12, let me check git
eng:  the prompt is in the database, in a config table. no history
eng:  someone edited it through the admin panel in august, no idea when or what

The post exists. The prompt that produced it no longer does.

The same hole shows up when a customer asks, under Brazil’s LGPD privacy law, what the assistant did with their data. The log has the answer. It does not have the text that produced it. With no version, there is no real audit.

Signs of an Unversioned Prompt:

• The prompt can be edited (in code, a database, or a panel) without creating a new version with an identifier.
• Traces and logs do not record which prompt version produced each answer.
• An old answer cannot be reproduced.
• Complaints about an answer from weeks ago cannot be investigated.
• Going back to a previous version requires remembering the old text.

flowchart LR
  E((("1. Someone edits the prompt"))):::human --> P[("2. Prompt overwritten")]:::store
  P --> M(["3. New answers"]):::llm
  M --> L[("4. Log with no version")]:::store
  L --> Q{"5. Which prompt produced this?"}

1. Someone improves the prompt.
2. The previous version stops existing.
3. The answers change.
4. The record keeps the answer, but not the version that produced it.
5. Weeks later, nobody can answer the question.

Versioning the file in git and loading latest at runtime, without storing the identifier on the span, leaves the same hole. History exists in the repository. The run does not point at it.

Check +20 XP on first tryThe viral post was on 08/12. The prompt lives in a config table, edited through a panel, with no history. Marketing wants to repeat the result. What is missing?

A larger model, more faithful to the post's tone

An immutable identifier for that prompt, stored on the post's trace

Store the generated post and use it as a few-shot

Reasons

• A prompt editable in production. An admin panel or a config table allows changes without a deploy, and without history. Speed eats the trail.
• Small changes look harmless. “I only tweaked one word” does not seem to deserve a version. That word was what held the viral post’s tone.
• The trace has no field for it. Nobody thought to record the prompt version alongside the call. The span has model, tokens, latency. It does not have prompt_version.
• latest looks simple. Code always asks for the last one. On complaint day, the last one is already something else.
• Mixing a versioned file with a versioned run. Git has the history. The run, if it does not store the hash, points at nothing.

Cost of ignoring

• Good results cannot be repeated. What worked cannot be recovered. Marketing waits a week for a clone that never arrives.
• Bad results cannot be explained. Complaints and incidents have no cause. Under LGPD, “we do not know which instruction processed the data” is a bad answer.
• No rollback. A bad change stays in production until someone rewrites the old text from memory.
• No real comparison. Without versions, no eval truly compares “before” and “after”. You compare today’s code with memory.

Treatment

Refactoring	Choose it when
Extract Versioned Template	The prompt has no place with history. It becomes a template with a name and an immutable version; publishing a change creates a new version. Target: Prompt Template Registry.
Add Tracing	The prompt has versions, but runs do not record them. Every call stores the template’s name and version on its span. Target: Audit Log.

Two parts, both required:

1. Immutable text. content.blog@12 is not edited. Publishing creates @13. Rollback points back at @12.
2. Span with the resolved version. Store content.blog@12, never latest. If code asks for latest at call time, resolve the number before the call and persist that number.

Git works. A commit hash on the trace identifies the prompt if the whole file lives in the repository. An admin panel also works, if every “save” mints a new id and the id goes on the span. What does not work is overwriting the row.

On the span, store at least this:

Field	Example
prompt_name	content.blog
prompt_version	12 (resolved, not latest)
model	the id that actually served the call

Without the trio, you reproduce the prose and still do not know which model ran it. Prompt version and model version travel together on the trail.

Check +20 XP on first tryThe registry has content.blog@1 through @14. Code always loads latest, and the span stores only the name content.blog. A complaint points at Friday's post. What breaks?

The run does not record the resolved version

Nothing, @14 is latest and the registry has history

An eval on latest is missing before each post

Payoff

• Every answer points to the exact text that produced it.
• One-click rollback.
• Evals that truly compare versions.
• Incident investigations backed by evidence.

When to ignore

• A personal, one-off script. If nobody will need to reproduce it, the repository’s version control is enough.

Is it this smell?

Check +20 XP on first tryThe prompt lives in the repository, with git history, and each deploy records the commit hash. Traces store the hash. Is this an Unversioned Prompt?

No

Yes, a template registry is missing

Yes, because a commit hash is not a semantic number

Check +20 XP on first tryThe prompt has versions in a registry, but the code always loads 'latest'. Traces record only the template name, not the version. What is the problem?

None, the registry has history

The template name should include the date

Runs do not record the version

Check +20 XP on first tryA panel lets the product team edit the prompt without a deploy, and every edit creates a numbered version that shows up on the trace. Is this an Unversioned Prompt?

Yes, editing in production is risky

No

Yes, until the eval lands in the panel the smell remains

Check +20 XP on first tryThe prompt is scattered across four files, all in git, and the span stores the commit hash. Which smell stands out?

Unversioned Prompt, because four files do not share one id

Both, in equal measure

Scattered Prompt Strings