Where the lesson lands

Two things go wrong with retrospectives, and the second is worse than the first.

The first is that they do not get written. A retrospective is due at the exact moment a piece of work stops being interesting, so it loses to the next thing, every time. This is the same shape as every other chore I have automated: not hard, genuinely valuable, and reliably skipped.

The second is that when they do get written, the lesson lands in the wrong place. It goes into the ticket — a note on one item, in a system nobody re-reads, attached to work that is finished. Six weeks later the same mistake happens on a different ticket, and the advice that would have prevented it is sitting on the old one. The knowledge was captured and still had no effect, which is arguably worse than not capturing it, because it felt like learning.

When a pull request completes, a headless agent run starts automatically for the work it belonged to. It gathers four kinds of evidence:

• what was asked for up front — the ticket and any plan written before the work started;
• what actually shipped — the diff, anchored to the commit that was reviewed rather than to whatever the branch looks like now;
• the complete review history, including every thread that was resolved and closed;
• the log of the conversation between me and the agent during implementation, where the corrections happened.

Then it asks a single question: given only what was specified up front, could a fresh agent have got this right first time? Every place the answer is no is a finding — a divergence from the spec, a thing a reviewer had to catch, or a moment I had to intervene mid-implementation.

The report it writes is not the deliverable. What happens to the findings is.

A model can produce a plausible retrospective for almost any piece of work. That is not valuable, and treating it as the deliverable is how you end up with a generator of documents nobody reads. The design question is what happens to a finding after it exists.

One destination, chosen by a litmus test

Each finding is routed to exactly one place, and the rule for choosing is a single question: would this instruction get copy-pasted into many future tickets? If yes, it does not belong on a ticket at all. A bug better tests would have caught should not become “write more tests” boilerplate repeated forever — it should become a coverage gate that fails the build. A pattern an agent should always follow belongs in the conventions file, written once. A gap that keeps appearing in specs belongs in the template that produces the specs.

Only when a lesson is genuinely unique to one piece of work does it go on that work. That is the last resort, and it is where almost every team puts everything. The difference between the top rung and the bottom is the difference between a fix that holds automatically and a note somebody has to remember, and remembering is the thing that never scales.

The signal is in the threads people already closed

Most tooling built around pull request reviews surfaces the threads that are still open, because the use case is usually “what do I have left to fix?” For a retrospective that is exactly backwards. A resolved thread is, by definition, something the up-front spec failed to anticipate and a reviewer had to catch. The closed threads are the entire point, and reaching for the obvious existing helper would have thrown away most of the signal.

The same instinct shows up in how the diff is fetched. The analysis is anchored to the commit that was actually reviewed, not to the current state of the branch, so a retrospective still works after the branch has merged and disappeared. Getting that wrong would mean the tool only functions in the narrow window before a merge — which is precisely the window where nobody has time for a retrospective.

Context is a budget

A full review history and a full diff can each be larger than the context window they would have to fit in. The tempting fix is to truncate, and truncation on a retrospective silently removes the evidence you were looking for.

So neither is ever pulled inline. Both are written to files, and the reading and summarising is delegated to subagents that report back only what matters. It costs an extra hop and it is the difference between a tool that works on small pull requests and one that works on the large, messy ones — which are, of course, the ones with lessons in them.

This is a spike and it says so about itself. It was deliberately timeboxed to one skill, it produces prose rather than a metric, and it proposes edits without applying any of them. I would rather ship something that is honest about being a scaffold than dress a two-day experiment up as a product — but it does mean the interesting claims are unproven.

The most important one is entirely unmeasured: I do not know whether routing a lesson upward actually reduces recurrence. That is the whole thesis, and testing it needs the one thing a spike does not have — time, and enough findings to see whether the same category stops coming back after it was turned into a gate. I have not collected those numbers yet. Until I do, this page describes a structure I believe in rather than a result I can show you.

There is also a tension I have not resolved. Because the output is prose, two runs over similar work can categorise the same problem differently, which makes it hard to trend. The obvious fix — force the findings into a fixed taxonomy — would make them countable and would probably also make them worse, because the interesting findings are the ones that do not fit a category you thought of in advance. I do not yet know which side of that I would take.

Every team already generates this material. Code review is a continuous stream of things somebody had to catch, and each one is evidence about what the process failed to say up front. Almost all of it evaporates: the thread gets resolved, the pull request merges, and the only trace is a comment on a piece of work nobody will open again.

The generalisable move is not to write retrospectives. It is to ask, of every lesson you capture: what is the highest layer that could prevent this automatically? Most advice a team writes down is really a missing check, a missing convention, or a missing question in a template — and each of those, done once, outperforms a hundred well-meaning notes on individual tickets. The reason it usually does not happen is not that people disagree. It is that at the moment the lesson appears, writing a comment takes ten seconds and changing the template takes an argument.

Making that routing decision the default, and putting a concrete proposed change in front of someone at the moment the evidence is fresh, is most of the value. The agent that writes the retrospective is the cheap part.