Sectioning

Intent

Break the task into independent parts, defined in code, and run them all in parallel, each with its own focused prompt. It is for tasks that look at the same input from several angles, or that split into pieces with no dependency between them.

Problem

Felipe maintains the team’s automated PR reviewer. The prompt asks it to review security, performance, tests, style, and API compatibility, all in one call. Reviews are slow, and quality is uneven: style comments always show up, security problems only sometimes.

When he looks at the traces, he sees why. The model has to hold five lists of criteria at once and tends to cover best what is easiest to notice. A SQL injection hides among twelve comments about variable names.

Felipe tried splitting it into five calls, one per aspect. Quality improved, but reviews took five times as long, because the calls ran one after another.

The question this pattern answers: how do you give each part of a task its own focused call without paying the sum of the latencies?

Solution

Split the task into fixed sections, decided in code: one aspect per section, or one piece of the input per section. Each section has its own short, specialized prompt. They all run at the same time. A final step, in code or with a model, combines the results.

The difference from Orchestrator-Workers is that the split does not depend on the input: code knows in advance which sections exist. The difference from Voting is that each section does something different, instead of all doing the same thing.

Anthropic’s “Building effective agents” post (December 2024) calls this parallelization, in the sectioning variant: break the work into independent subtasks and run them at the same time. The other variant is voting, where several calls do the same job.

Structure

flowchart LR
  IN(["1. PR diff"]):::io --> S1(["2. Security"]):::llm
  IN --> S2(["2. Performance"]):::llm
  IN --> S3(["2. Tests"]):::llm
  IN --> S4(["2. Compatibility"]):::llm
  S1 --> J["3. Merge and dedupe"]:::tool
  S2 --> J
  S3 --> J
  S4 --> J
  J --> F{"4. Filter by severity"}
  F --> OUT(["5. PR comments"]):::io

1. The same input goes to every section.
2. Each section runs with a prompt focused on one aspect and returns structured findings.
3. Code merges the findings, removes duplicates, and sorts them.
4. A rule decides what becomes a comment (by severity, by confidence).
5. The result goes out with each finding’s source recorded.

Worked example

Felipe’s PR changes api/users.py. Four sections run at the same time over the same diff. Outputs (simplified):

{ "section": "security", "file": "api/users.py", "line": 48, "severity": "high",
  "message": "query built with an f-string from the user's email" }

{ "section": "tests", "file": "api/users.py", "line": 48, "severity": "medium",
  "message": "no test for a malformed email on this endpoint" }

The merge sees the same line twice. A dedupe key of file plus line would keep the security high and drop the tests finding, if sorting is by severity. That is why the dedupe key needs to include the message, or you accept that a tests comment vanishes when security already spoke about the line.

With a single call, the model covered style and barely saw the injection. “Lost in the Middle” (Liu et al., 2023) helps explain: five lists of criteria in the same window starve the middle one (security, in Felipe’s prompt). A security-only section does not compete with style.

Pseudocode

schema Finding { file: string, line: int, severity: "high" | "medium" | "low", message: string }
schema Findings { items: list[Finding] }

SECTIONS = {
    "security":      SECURITY_PROMPT,
    "performance":   PERFORMANCE_PROMPT,
    "tests":         TESTS_PROMPT,
    "compatibility": API_COMPAT_PROMPT,
}

function review(diff):
    # 2. every section in parallel
    parallel {
        for name, prompt in SECTIONS:
            results[name] = llm(prompt, diff) -> Findings
            trace.span("section." + name, results[name])
    }

    # 3. merge in code
    all = flatten(results)
    unique = dedupe(all, key=(f.file, f.line, similar(f.message)))

    # 4. filter
    return [f for f in unique if f.severity in ["high", "medium"]]    # 5

Real-world scenarios

• Automated PR reviewer. Each review aspect has its own prompt and runs in parallel over the same diff. Security gets the full attention of one call.
• Content moderation. Each policy (violence, hate speech, spam, personal data) is checked by its own section, and the final decision combines the results.
• Deep research with citations. The report’s fixed questions (market, competitors, regulation) are researched at the same time, one section each.

Applicability

• The task has aspects or parts you can list before seeing the input.
• Each aspect benefits from a focused prompt.
• The parts do not depend on each other.
• Latency matters, and the queue of calls is the bottleneck.

When not to use

• A single call already does well. Sections multiply input cost, because each one reads the whole input.
• The split depends on the content. Use Orchestrator-Workers.
• The parts depend on each other. Use Prompt Chaining.
• The input is large and you want to split it into chunks and then combine a summary. That specific case is Map-Reduce.

Common mistake: sections that depend on each other

“Security first, then tests using the security findings” is not sectioning. It is a chain. Parallelism only pays off if each section can run without the others’ results.

If the split changes with the diff (an infra PR needs sections a CSS PR does not), a model decides the list, and the pattern becomes Orchestrator-Workers.

How to implement

1. List the sections and write a short prompt for each, with criteria for that aspect only.
2. Define a shared output schema, so merging needs no guessing.
3. Run the sections at the same time, with a concurrency cap.
4. Decide what happens if a section fails: continue without it or fail everything.
5. Write the merge in code: deduplication, sorting, filters.
6. Use Prompt Caching for the shared input if the provider allows it, placing the input before each section’s instruction.
7. Evaluate each section on its own and the final result.

Cap concurrency. Four sections on one PR is fine. Forty PRs at once, four sections each, is 160 calls in a burst. A semaphore of, say, 8 in-flight calls keeps you inside the provider’s rate limit. The slowest section still sets latency; the cap only spreads the burst.

Pick the model per section. Style and “missing tests” often run on a small model. Security deserves the larger one. That is a cascade inside sectioning: expensive attention only on the aspect that pays for it.

Cost of the repeated input

Four sections read the same diff four times. Suppose a 3,000-token diff and 200 tokens of prompt per section. With no cache, that is about 12,800 input tokens. With prompt caching on the shared prefix (the diff first, the instruction after), the provider charges the full diff once and later reads come cheap.

Latency does not add up: you wait for the slowest section. If security takes 8 seconds and style takes 3, the total is close to 8, plus the merge. That is why a section that got slow deserves its own alert.

Variations

• Sections by aspect. Same input, different criteria. That is the PR reviewer’s case.
• Sections by chunk. Different parts of the input, same criterion. When the merge is a summary, it becomes Map-Reduce.
• Guard section in parallel. One section checks the input’s safety while another already generates the answer. If the guard rejects, the answer is discarded.
• Sections on different models. Simple aspects on a small model, hard aspects on a larger one.

Sections by chunk look like Map-Reduce, and the border is the merge. If each chunk returns findings of the same type and code only concatenates them, it is still sectioning. If each chunk returns a summary and a model stitches one text, you have moved to Map-Reduce.

The guard section in parallel is the case where you accept throwing work away. The answer is already being generated while the guard reads the input. If the guard rejects, you discard tokens already spent. It pays off when answer latency matters more than the discarded cost, and when the guard is cheaper than generating again afterwards.

Cost, latency, and failure modes

How to evaluate

• Per section: precision and recall of each aspect on a dataset with known problems.
• Final: relevant findings per PR, useless comments per PR, and latency.
• Comparison: the same evaluation with the earlier single call.

A PR reviewer dataset is not “random PRs”. It is diffs with planted problems: an injection on line 48, an N+1, an endpoint with no test. Without planting, security recall is a guess. With planting, the security section has a golden case per failure type.

When the single call already finds the N+1 and the performance section does not, that section’s prompt is worse than the old block. Copy the slice that worked. Do not invent new criteria in the dark.

{ "diff": "fixtures/pr-311.diff", "expected": [{ "section": "security", "file": "api/users.py", "line": 48, "kind": "sql_injection" }] }

In production

• Per-section spans with finding count, tokens, and duration.
• A dashboard of findings per section over time. A section that never finds anything deserves review.
• An alert for the slowest section when it dominates total latency.

The merge is where sectioning usually degrades in silence. Deduping by file and line is too simple, as the code on this page admits. Different findings on the same line vanish. Prefer a key that includes a short hash of the message, or keep both and let the PR UI group them.

If you put a model in the merge to “write the final comment”, you bring back a serial bottleneck and a chance the model softens a security high. Merge in code, comment with the text the section already wrote. The reviewer should see who found what: a security finding stays tagged as security, even after dedupe.

Security

The diff came from the user. A security section that reads the PR can be the target of prompt injection in the commented code itself. Isolate the section’s instruction from the diff text, and do not let the security section execute anything: it only returns findings.

Merging in code also avoids a final model that “rewrites” security findings downward. The minimum severity to comment sits in a rule, not in a prompt.

Pros and cons

Code

Both tabs run four sections in parallel over the same diff and merge the findings in code.

import asyncio
from typing import Literal

import anthropic
from pydantic import BaseModel

client = anthropic.AsyncAnthropic()


class Finding(BaseModel):
    file: str
    line: int
    severity: Literal["high", "medium", "low"]
    message: str


class Findings(BaseModel):
    items: list[Finding]


SECTIONS = {
    "security": "You review security only: injection, authentication, secrets, permissions.",
    "performance": "You review performance only: queries in loops, allocation, complexity.",
    "tests": "You review tests only: missing cases for the changed code.",
    "compatibility": "You review public API compatibility only: signatures and contracts.",
}


async def section(name: str, prompt: str, diff: str) -> list[dict]:
    reply = await client.messages.parse(  # 2
        model="claude-sonnet-5-5", max_tokens=4000, system=prompt,
        messages=[{"role": "user", "content": f"<diff>\n{diff}\n</diff>"}],
        output_format=Findings,
    )
    return [{"section": name, **f.model_dump()} for f in reply.parsed_output.items]


async def review(diff: str) -> list[dict]:
    results = await asyncio.gather(
        *(section(n, p, diff) for n, p in SECTIONS.items()), return_exceptions=True
    )
    findings = [f for r in results if not isinstance(r, Exception) for f in r]  # a failed section: continue
    seen, unique = set(), []
    for f in sorted(findings, key=lambda f: ["high", "medium", "low"].index(f["severity"])):
        key = (f["file"], f["line"])  # 3. dedupe
        if key not in seen:
            seen.add(key)
            unique.append(f)
    return [f for f in unique if f["severity"] != "low"]  # 4

import Anthropic from "@anthropic-ai/sdk";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
import { z } from "zod";

const client = new Anthropic();

const Findings = z.object({
  items: z.array(z.object({
    file: z.string(), line: z.number().int(),
    severity: z.enum(["high", "medium", "low"]), message: z.string(),
  })),
});

const SECTIONS: Record<string, string> = {
  security: "You review security only: injection, authentication, secrets, permissions.",
  performance: "You review performance only: queries in loops, allocation, complexity.",
  tests: "You review tests only: missing cases for the changed code.",
  compatibility: "You review public API compatibility only: signatures and contracts.",
};

async function section(name: string, system: string, diff: string) {
  const { parsed_output } = await client.messages.parse({
    model: "claude-sonnet-5-5", max_tokens: 4000, system,
    messages: [{ role: "user", content: `<diff>\n${diff}\n</diff>` }],
    output_config: { format: zodOutputFormat(Findings) },
  });
  return (parsed_output?.items ?? []).map((f) => ({ section: name, ...f }));
}

export async function review(diff: string) {
  const settled = await Promise.allSettled(
    Object.entries(SECTIONS).map(([n, p]) => section(n, p, diff)), // 2. in parallel
  );
  const all = settled.flatMap((r) => (r.status === "fulfilled" ? r.value : []));
  const order = { high: 0, medium: 1, low: 2 };
  const seen = new Set<string>();
  return all
    .sort((a, b) => order[a.severity] - order[b.severity])
    .filter((f) => {
      const key = `${f.file}:${f.line}`; // 3
      if (seen.has(key)) return false;
      seen.add(key);
      return f.severity !== "low"; // 4
    });
}

Check your understanding