Extract Subprompt

Problem

A single prompt handles several responsibilities, keeps growing, and every change affects everything. This is the Prompt Bloat smell.

Picture a bank support bot on WhatsApp. The same system prompt classifies language, masks card data, chooses among billing, orders, and Pix (Brazil’s instant payments), applies 14 boleto rules, and still asks the satisfaction question. A change to a late-fee rule breaks card detection. Nobody can test one part without the others.

Solution

Pull one responsibility out of the main prompt and give it its own call, with a short prompt and a defined output. The main prompt then receives that call’s result, not the instructions for producing it.

Before and after

-SYSTEM = """You are the support assistant.
-First, identify the customer's language and reply in it.
-Detect whether the message contains card data; if so, ignore it.
-Classify the topic as billing, order, or other.
-If it is billing, follow these 14 rules: ...
-If it is an order, follow these 11 rules: ...
-Always end with the satisfaction question."""
+TRIAGE = """Read the customer's message and return the language,
+the topic, and whether it contains card data."""   # extracted subprompt
+
+SUPPORT = {
+    "billing": "You handle billing questions. [billing rules]",
+    "orders":  "You handle order questions. [order rules]",
+}
+
+t = llm_parse(TRIAGE, message, schema=Triage)
+clean = mask_card(message) if t.has_card_data else message
+reply = llm(SUPPORT[t.topic], clean, language=t.language)

Before: one call with every responsibility.

flowchart LR
  IN(["1. Message"]):::io --> M(["2. Single prompt: language, card, topic, all rules"]):::llm
  M --> OUT(["3. Reply"]):::io

After: triage has its own call, and each reply has a focused prompt.

flowchart LR
  IN(["1. Message"]):::io --> T(["2. Triage: language, card, topic"]):::llm
  T --> C["3. Mask and pick prompt"]:::tool
  C --> M(["4. Topic-focused prompt"]):::llm
  M --> OUT(["5. Reply"]):::io

1. The same message arrives.
2. A short call does only triage, with structured output.
3. Code uses the result: it masks data and picks the prompt.
4. The reply prompt handles a single topic.
5. The reply goes out, and each step shows up separately on the trace.

A worked case

Message: “good afternoon, the late fee on my PIX boleto is wrong, card 4111…”. Triage returns, for example:

{ "language": "pt-BR", "topic": "billing", "has_card_data": true }

Code masks the card. The billing prompt gets the clean text and the language, without the 11 order rules. If the late fee still comes out wrong, the trace shows whether triage picked the wrong topic or the billing prompt missed the rule. Before, both errors died in the same paragraph.

Why it works

Each responsibility in a prompt competes for the model’s attention with all the others. “Lost in the Middle” (Liu et al., 2023) showed that models use the middle of a long context poorly. A 4,000-token prompt with billing rules in the middle is exactly that middle. Splitting gives each responsibility a small prompt that the model follows more consistently and that the team can test on its own.

The extracted call’s output becomes a contract. Instead of “identify the language and reply in it” hidden in the middle of a long text, there is a language field that code reads and passes along. When something goes wrong, the trace shows whether the error was in triage or in the reply.

Finally, the parts evolve separately. Changing the billing rules no longer touches the order rules. The natural destination is Prompt Chaining. If the extraction is a classification that picks the path, Introduce Router puts the decision in code.

The extra call pays when the main prompt actually shrinks and the extracted output enters code. If the reply prompt is still huge after the split, you only added latency. A small model on the extracted call often covers the extra round-trip, as long as the main prompt lost the rules that used to live there. Measure size, accuracy per category, and p95 before you extract the next piece.

Safe mechanics

0. Measure. Run the current prompt’s eval and record accuracy per category, prompt size, and latency.
1. List the prompt’s responsibilities. Mark the ones that produce a decision or a piece of data used later (language, topic, risk).
2. Pick one to extract, ideally the one with the clearest output.
3. Write the subprompt with only that responsibility and a structured output.
4. Build an eval for the subprompt alone from real cases.
5. Delete the extracted part from the main prompt and pass the result as input.
6. Run the full eval and compare with the initial measurement.
7. Repeat for the next responsibility, if it is worth it.

Risks

• One more call. Latency and cost rise a little. Use a small model for the subprompt and check that the main prompt shrank enough to compensate.
• Lost context. The subprompt may need something that lived in the big prompt. Pass what it needs explicitly.
• Cascading errors. If triage is wrong, the reply is wrong too. Define a default path for low confidence.
• Extracting too much. Ten calls for ten small rules is the opposite smell. Extract responsibilities, not sentences. If the extra call does not change the result, Inline Trivial Step is the way back.

Common trap: extracting what code already does

Checking whether the sheet has brand, model, and voltage does not need a model. Extracting it into a subprompt still leaves an LLM doing deterministic work. The first home for that responsibility is code. The subprompt is for language: description tone, keywords, ambiguous classification.

Challenge

Which responsibility of this product description generator’s prompt would you extract first?

You write product descriptions for the online store.
Read the spec sheet and write a description of 80 to 120 words.
Check that the sheet has all required fields (brand, model,
voltage, warranty); if any is missing, do not write and list what is missing.
Use the brand's tone: direct, no exaggeration.
Also generate 5 SEO keywords.

1. Code: validate the sheet's required fields. If any are missing, return the list.
2. Call: write the description (only tone and length in the prompt).
3. Call: generate keywords in a schema { keywords: list[str] (5) }.

Related

• Treats Prompt Bloat.
• Lands on Prompt Chaining and, when the extracted piece picks the path, on Routing.
• The inverse, when the extra call does not pay for itself, is Inline Trivial Step.

Check your understanding