Skip to content

Reference

closure test

The closure test is a four-condition rubric for whether an agent system actually closes the loop. Pass or fail, no maturity levels: closed loop, declarative target, unattended in the ordinary case with named exceptions, and ground truth — an eval decides whether the loop converged. Condition 4 is the disqualifier.

Last reviewed 2026-08-06Maintained by Tim
  • autonomic-swe
  • evals

A membership criterion, not a maturity model. Pass or fail. Four conditions, no gradations. The recurring finding of the teardown series is that systems which look agentic — emit a change, look at the consequence, iterate — fail condition 4, and failing condition 4 means there is no Analyse stage, only Monitor.

The four conditions

  1. Closed loop. The system observes the consequence of its own change. A Sensor reads post-apply state; the Ledger records both the application and the observation. Without this, the loop is open and any “improvement” is a rumour.

  2. Declarative target. Intent is state that can be diffed, not a transcript that must be read. The Proposal is a content-addressed patch; git diff (or the equivalent on the managed subject) is the full intent. A transcript hides the intent inside the words; a diff is the intent.

  3. Unattended in the ordinary case, with an explicitly named exception class. The loop runs without a human in the loop most of the time. When it does not, the reason is named in advance, not discovered after the fact. “We watched it work” is not an exception class; “the corpus is being regenerated and a human signs off on the new cases” is.

  4. Ground truth. An eval decides whether the loop converged. Without an eval there is no Analyse stage, only Monitor. The eval is held by a separate authority from the proposer; the proposer submits to the verdict plane, it does not call it. KILL outranks aggregate. A change that improves the aggregate and breaks one previously-passing case is KILL, not PASS. This is the rule that makes the corpus mean something.

Why condition 4 is the disqualifier

Conditions 1, 2, and 3 are necessary but common. Condition 4 is the one that distinguishes autonomic software engineering from agentic engineering. Most “agent loops” fail here because the thing that proposes the change also decides whether it worked, and the loop then optimises the judge. The Tell is a monotonically rising number on a metric that the proposer also controls.

A system that fails any of the four conditions is not a closed-loop autonomic system. It may be useful, but it is not what this practice is about.

How to use the rubric

Apply the four conditions in order. The first one that fails ends the assessment. There is no “but it does 1, 2, 3 well” — partial credit is not a thing the rubric grants.

The teardown series scores a named system against the four conditions and publishes the scorecard in a fixed structure. The rubric is a component, not prose, so scores are comparable across pieces.

See also

Aliases

  • closure
  • closure rubric

See also

Keep exploring

See what this means for a real agent