Build the checks before you build the agent

The order is not negotiable, and it is the opposite of what feels productive.

When you are starting a new autonomic system, the natural instinct is to build the Proposer first. The Proposer is the most interesting component. It is the thing that takes a task and emits a change. It is where the magic is. Of course you start there.

You will regret it. Here is what happens:

1. You build a Proposer. It is good. It emits patches that are usually right.
2. You build a Scorer to evaluate the Proposer’s output.
3. The Scorer uses a corpus. You assemble the corpus from cases that the Proposer tends to do well on, because those are the cases you noticed.
4. The Scorer reports a high number. The Proposer is good. You ship it.
5. Six weeks later, the Proposer regresses on a case that wasn’t in the corpus. The number is still high. You don’t notice.
6. The number is high because the corpus was shaped to flatter the Proposer, not because the Proposer is good.

This is the recurring finding of the teardown series. The system that fails condition 4 fails it in exactly this way: the judge is on the same team as the proposer.

A safer build order

Build, in this order, with no skipping:

1. Verdict plane. The corpus, the rubric, the scorer, the plane layout, the separation guard. Nothing else. At this stage there is no loop; there is a way to score a change a human made. If you cannot tell whether a change helped, that is the thing to build, and it is also the thing that will still be true in a year when the Proposer has been replaced twice.

2. Ledger. Append-only, before there is anything to record. Retrofitting an audit trail onto a running loop produces an audit trail with a gap exactly where it matters.

3. Sensors and the quoting function. The trust boundary goes in before the first untrusted byte, not after the first incident.

4. Applier. Idempotent apply, discard, promote-with-token. Test discard harder than apply.

5. Proposer. Last. It is the most interesting component and the most replaceable, and building it first is how the corpus ends up shaped to flatter it.

What “build the checks first” actually means

It means: before you write a line of agent code, you write the test that decides whether a change helped. The test is held by a separate authority from the agent. The agent submits to it; it does not call it.

A practical starting point: take a small corpus of cases that you can score by hand. Run your future Proposer against the corpus. Score the output yourself. Notice which cases the Proposer gets wrong that you would have gotten right. Add those cases to the corpus. Repeat until the Proposer either improves or the corpus reveals that the Proposer is the wrong shape for the task.

This is slow. It is also the only way to build a corpus that discriminates rather than flatters.

See also

• two-plane-loop: the architecture
• closure-test: the rubric, especially condition 4