Redundant Self-Critique

Signs and symptoms

A PR reviewer generates comments and, before posting, runs three rounds of “review your review”. The code (anonymized):

review = llm(REVIEW_PROMPT, diff)
for _ in range(3):
    review = llm("Critically review the review below and improve it.", review)
post_comments(review)

A month of traces showed that on 94 of every 100 PRs, the third version had the same comments as the first, in slightly different words. On the other 6, the change was stylistic. The review cost four times as much as the first call.

Suppose, just to see the count, 2,000 tokens in generation and 2,000 in each review. Four calls send about 8,000 input tokens in this step alone, to return the comment the first call already had. Latency follows: four round trips on the critical path.

Signs of Redundant Self-Critique:

• There is a “review”, “critique”, or “improve” step that runs over its own output, with no new information.
• The review step uses the same model, the same context, and no verifiable criterion.
• Comparing the review’s input and output, almost nothing changes.
• The eval gives the same result with and without the step.
• The step doubles or triples cost and latency.

flowchart LR
  G(["1. Generate"]):::llm --> R1(["2. Review"]):::llm
  R1 --> R2(["2. Review again"]):::llm
  R2 --> R3(["2. Review once more"]):::llm
  R3 --> OUT(["3. Nearly the same output, 4x the cost"]):::io

1. The model generates the answer.
2. The same model rereads its own answer, with no new data, and rewrites it.
3. The final output is essentially the first one.

Human intuition (“reread before you send”) does not transfer. A person who rereads brings memory, fatigue, a pattern they just saw in the next file. The model rereads the same buffer.

Check +20 XP on first tryThe reviewer eval scores the same with 1 call and with 4. The three reviews only change adjectives. What is the next step?

Tell the review prompt to be stricter and more aggressive

Drop the three reviews and, if any criterion is useful, put it in the first call's prompt

Swap in a larger model for the reviews, keeping all 3 rounds

Reasons

• “Think again” always sounds good. Reviewing is a solid human habit. Copied into the pipeline, it becomes an extra call with the same input.
• Papers that show gains from reflection. Reflexion (Shinn et al., 2023) uses an external signal (the environment, a test). Self-Refine (Madaan et al., 2023) also structures the feedback. The naive copy drops the signal and keeps only “improve the text above”.
• No eval. Nobody measured whether the step changes anything. Four times the cost looks like “more quality”.
• It feels safer. One more “assurance” step brings comfort, even with no effect. On a compliance team, nobody wants to be the person who removed the review.

Cost of ignoring

• Multiplied cost and latency with no quality gain. On the reviewer, every PR waits for four calls.
• Sometimes it hurts. A review with no criterion can “improve” something that was right: it softens “this breaks production” into “might be worth a look”.
• A false sense of control. The team believes there is a quality check that does not exist. The incident walks through all three rounds and nobody distrusts the step.

Anthropic’s “Building effective agents” (Dec. 2024) describes the Evaluator-Optimizer pattern with a separate evaluator and a rubric. That is the opposite of this smell: there are criteria, independence, and, when it pays, a measured gain.

Treatment

Refactoring	Choose it when
Inline Trivial Step	The eval shows the step does not change the result. It goes, and whatever was useful in it (a criterion, a format) moves into the main step’s prompt. Target: Prompt Chaining.

If review is needed, give it new information: an external signal (Reflection) or a separate evaluator with a rubric (Evaluator-Optimizer).

Before you delete, measure. Run the eval with and without the step, on the same seed, on the same set. If the score does not move, the step goes. If it moves a little and the cost doubles, the tradeoff needs to be explicit, not a habit.

The diff the team is waiting for, in practice:

review = llm(REVIEW_PROMPT, diff)
post_comments(review)
# length checks and "file not in the diff" live in code, not in a second call

Three fewer lines on the critical path. If one day the eval asks for the round back, it comes back with a rubric or a test, not with “improve the text”.

Objective checks (length, schema, “the SQL runs”, “the quote exists in the PDF”) belong in code. They should not ask for a fourth model call.

Check +20 XP on first tryYou want to keep a review step on the SQL generator. Which design stops being empty self-critique?

The same model reads the SQL and answers whether it is good, yes or no

Run the SQL on a test database and send the error back to the model

Three rounds of 'improve the SQL' with a larger model on the last one

Payoff

• Cost and latency proportional to the real work.
• A shorter pipeline that is easier to debug.
• When there is review, it has a criterion you can measure.

When to ignore

• The eval shows a gain. If the step improves quality measurably, it is not redundant, even without an external signal. Keep it and monitor it.
• Long reasoning tasks. On some hard problems, a second pass does find errors. Measure before deciding.

Is it this smell?

Check +20 XP on first tryA SQL generator runs the query on a test database and, on error, sends the error back to the model to fix. Is this Redundant Self-Critique?

Yes, it is a review

No

Yes, because the same model generates and corrects

Check +20 XP on first tryAfter writing a summary, the same model answers 'is the summary good? yes or no'. It says yes 99% of the time. What do you do?

Remove the step

Ask the model to be stricter

Replace yes/no with a score from 1 to 10

Check +20 XP on first tryA separate evaluator, with a five-criterion rubric and a different model, rejects a third of the drafts, and the eval shows the corrected drafts are better. Is this Redundant Self-Critique?

Yes, it is a review step

No

Only if both models come from the same provider

Check +20 XP on first tryA second pass on the same model, with no new data, gains 8 points on the eval for a hard problem and doubles the cost. Is this Redundant Self-Critique?

Yes, because there is no external signal

No, if the measured gain pays for the cost and you keep monitoring

Yes, every second pass is this smell by definition