Replace Rules with Examples

Problem

The prompt piles up style, format, and behavior rules that fight each other and still fail to produce the desired result. These are the Rule Stacking and Prompt Bloat smells.

Think of an online store that answers customer reviews with a model. Every complaint from the brand team became a new rule: “thank them by name”, “do not be curt”, “do not be too formal”. Six months later the prompt has 25 rules in capital letters, and the tone still changes from reply to reply. Nobody can say which rule the model is following in each case.

Solution

Replace the rules that try to describe a behavior with a few complete input and output examples that show it. Keep as rules only what is a short, clear constraint.

Before and after

 SYSTEM = """You reply to the store's customer reviews."""
-RULES = """
-ALWAYS thank them by name.
-NEVER use more than 3 sentences.
-For negative reviews, NEVER argue, ALWAYS offer a solution.
-DO NOT use emojis. DO NOT overuse exclamation marks.
-If they mention a delay, apologize and give the average delivery time.
-If it is praise, be brief, but DO NOT be curt.
-Use a friendly tone, but professional, but not too formal.
-"""
+EXAMPLES = """
+<example>
+<review>Ana, 2 stars: arrived 10 days late and the box was crushed.</review>
+<reply>Ana, we are sorry about the delay and the packaging. We have asked
+the carrier to review your case. If the product has any damage, reply
+here and we will exchange it at no cost.</reply>
+</example>
+<example>
+<review>Bruno, 5 stars: perfect, arrived ahead of time.</review>
+<reply>Bruno, great to hear! Thanks for shopping with us.</reply>
+</example>
+<example> ... mixed review: praise for the product, complaint about shipping ... </example>
+"""
+CONSTRAINTS = "At most 3 sentences. No emojis."

Before: behavior described through abstract rules.

flowchart LR
  R[("1. 25 rules")]:::store --> M(["2. Model interprets and prioritizes"]):::llm
  IN(["Review"]):::io --> M
  M --> OUT(["3. Unpredictable tone"]):::io

After: behavior shown through examples, with two clear constraints.

flowchart LR
  E[("1. 3 to 5 reference examples")]:::store --> M(["2. Model follows the pattern"]):::llm
  C["Short constraints"]:::tool --> M
  IN(["Review"]):::io --> M
  M --> OUT(["3. Consistent tone"]):::io

1. Instead of describing the tone, the prompt shows replies that already have the right tone.
2. The model generalizes from concrete cases that cover the hard situations (negative, positive, mixed).
3. Replies come out like the examples, including details no rule managed to describe.

Why it works

Style rules are abstract: “friendly, but professional, but not too formal” means different things to different people and models. An example is concrete: it shows length, vocabulary, the order of ideas, and what to do in a hard case, all at once.

Examples also resolve conflicts that rules create. “Be brief” and “do not be curt” fight in the abstract; the praise example shows the exact point between them: two sentences, the name, a thank-you.

What is a hard constraint (sentence limit, no emojis) stays a short rule, because it is easy to follow and easy to check in code.

The idea has old roots. The GPT-3 paper, “Language Models are Few-Shot Learners” (Brown et al., 2020), showed that a few examples in the prompt change model behavior without training. Anthropic’s documentation on prompting with examples recommends 3 to 5 varied examples, each wrapped in tags such as <example>, so the model can tell examples from instructions.

What an example teaches that a rule does not

Compare what each format says about Ana’s review:

The rule says what to avoid. The example shows what to do, and that is what the model copies.

Safe mechanics

0. Measure. Build an eval with real reviews of several kinds and a rubric for the desired tone. Run the rules version and keep the scores per review type.
1. Split the rules into two groups: objective constraints (length, clear prohibitions) and behavior descriptions (tone, how to react).
2. Pick 3 to 5 cases that cover the hard situations. Take them from real replies the team approved; do not invent them. For the store, that means at least one negative, one praise, and one mixed review.
3. Write the complete examples, with input and ideal output, marked with tags so the model can tell examples from instructions.
4. Vary the examples. If they all share one structure, the model copies that structure everywhere.
5. Check whether any example contradicts a constraint. A four-sentence example next to “at most 3 sentences” teaches the opposite of the rule.
6. Delete the behavior rules the examples cover. Keep the objective constraints.
7. Check constraints in code where possible (sentence count, emojis).
8. Measure again with the same eval. The prompt should get shorter, and the scores per type should stay equal or improve.

Risks

• Literal copying. The model may repeat phrases from the examples. Use varied examples and say they are a style reference.
• An example that teaches the mistake. An example with one bad detail becomes the pattern. Review each one with someone who knows the brand’s tone.
• Incomplete coverage. A case no example covers becomes unpredictable again. Add examples from eval failures.
• Examples that are too fixed. If cases vary a lot, picking examples by similarity for each request works better. See Few-Shot Selector.

Common mistake: examples that look too alike

Suppose all three examples open with “Hi NAME, so glad to have you here!”. The following week, a customer complains about a broken product and gets “Hi Carla, so glad to have you here!” before the apology.

The model did not learn “be warm”. It learned “open like this”, because that was the only trait repeated in every example. When a pattern shows up in all of them, it becomes a rule.

The fix is to vary on purpose: different openings, different lengths, one case where the reply thanks nobody. The only things that should repeat are the ones you want in every reply.

Challenge

Turn these rules from a ticket urgency classifier into examples.

URGENT if the whole system is down.
URGENT if customer money is involved.
NOT urgent if it is a usage question, EVEN IF the customer says it is urgent.
If the customer swears, that does NOT change the urgency.

<example><ticket>Nobody at the company can log in since 9 a.m.</ticket><urgency>high</urgency></example>
<example><ticket>My customer was charged for the invoice twice.</ticket><urgency>high</urgency></example>
<example><ticket>URGENT!!! How do I export the report as PDF?</ticket><urgency>low</urgency></example>
<example><ticket>What garbage system, the save button moved.</ticket><urgency>low</urgency></example>

Check your understanding