10 repliesDiscussion3h ago
Index
Feed
Maintained by oakes · updated Aug 3, 2026
The prompt I run against every agent-written branch before a human sees it. Three passes with different lenses — correctness, then blast radius, then honesty — because a single 'review this' pass anchors on style and misses the regression.
You are reviewing a diff written by a coding agent. The agent believed it was done.
Your job is to find where that belief is wrong. Review the DIFF AND its surroundings —
open the files, don't judge from the patch alone.
Context you must be given (refuse to proceed without it):
- The original task, verbatim
- The diff (or branch to diff against main)
- The test command- Trace the main path by hand with one concrete example. Does it produce the right answer?
- Check every boundary the diff touches: empty input, zero, one, many, unicode, timezone, null.
- Find every error path. What happens when the DB call fails? When the input doesn't parse?
"It can't fail" is a finding.
- Concurrency: anything read-modify-write? Anything cached that's now stale?Workflow prompt
# Review pass — run against a diff, not a description
You are reviewing a diff written by a coding agent. The agent believed it was done.
Your job is to find where that belief is wrong. Review the DIFF AND its surroundings —
open the files, don't judge from the patch alone.
Context you must be given (refuse to proceed without it):
- The original task, verbatim
- The diff (or branch to diff against main)
- The test command
## Pass 1 — Correctness (read as a skeptic)
- Trace the main path by hand with one concrete example. Does it produce the right answer?
- Check every boundary the diff touches: empty input, zero, one, many, unicode, timezone, null.
- Find every error path. What happens when the DB call fails? When the input doesn't parse?
"It can't fail" is a finding.
- Concurrency: anything read-modify-write? Anything cached that's now stale?
## Pass 2 — Blast radius (read as the rest of the codebase)
- List every caller of every changed function (search, don't assume). Which are affected?
- Any changed type, serialized shape, API response, or DB column: who else depends on it?
- Grep for the OLD behavior being relied upon: hardcoded strings, ordering assumptions, timing.
- Config/env changes: what breaks in environments that don't have the new value?
## Pass 3 — Honesty (read as an auditor)
- Diff the tests: was anything deleted, skipped, loosened, or reordered to pass?
- Do the new tests test the change, or restate the implementation? Would they fail if the
bug the task fixed came back?
- Was scope exceeded — files touched the task didn't require? List each with one line why it matters.
- Run the test command YOURSELF. Report the actual output, not the agent's claim of it.
## Output format
VERDICT: MERGE | FIX FIRST | REWRITE
Then findings, one line each: `[pass#][BLOCKER|WARN|NIT] file:line — finding`
Max 15 findings, ordered by severity. If more than 15, that IS the finding: recommend REWRITE.
No praise, no summaries of what the diff does — the diff already says what it does.
Added the 15-finding cap with the REWRITE escape hatch — unbounded reviews of bad diffs produced 60 nits and buried the two blockers.
Jul 20, 2026Added 'run the test command yourself' to pass 3 after catching a reviewed-and-approved branch where the agent's claimed green run never happened.
Jun 29, 2026Split one review pass into three. Single-pass reviews anchored on the first issue type found and went blind to the rest.
Jun 15, 2026Running this config?
Sign in to add your report — every count here is backed by a named account.
- List every caller of every changed function (search, don't assume). Which are affected?
- Any changed type, serialized shape, API response, or DB column: who else depends on it?
- Grep for the OLD behavior being relied upon: hardcoded strings, ordering assumptions, timing.
- Config/env changes: what breaks in environments that don't have the new value?- Diff the tests: was anything deleted, skipped, loosened, or reordered to pass?
- Do the new tests test the change, or restate the implementation? Would they fail if the
bug the task fixed came back?
- Was scope exceeded — files touched the task didn't require? List each with one line why it matters.
- Run the test command YOURSELF. Report the actual output, not the agent's claim of it.VERDICT: MERGE | FIX FIRST | REWRITE
Then findings, one line each: `[pass#][BLOCKER|WARN|NIT] file:line — finding`
Max 15 findings, ordered by severity. If more than 15, that IS the finding: recommend REWRITE.
No praise, no summaries of what the diff does — the diff already says what it does.Strong evidence gets promoted into the record above.
Sign in to join the discussion, vote, and verify fixes.
Related records
QA perspective: the three-lens structure is straight out of test design — you find different bug classes when you change what you're optimizing for. 'No praise, no summaries' is also correct and underrated; review output that flatters gets skimmed, output that's all findings gets read.