--- title: Verification is not the last step kind: concept status: draft order: 60 summary: Long-running agent work needs claims, evidence, freshness, and authority as live state rather than a final pass or fail checkpoint. audience: Engineers building or reviewing long-horizon agent workflows outcome: Separate action success, task success, and intent satisfaction, then connect each claim to appropriate evidence prerequisites: - Familiarity with evidence and evaluation concepts domains: - evals - agent-systems - trust-boundary updated: 2026-09-06 author: tim --- Verification is often treated as something that happens at the end of agent work: the agent produces a result, tests run, and the result passes or fails. That model is useful for short tasks. It becomes inadequate when a task lasts long enough for its plan, environment, permissions, assumptions, or risk to change. Verification is part of the mechanism that determines what the system may do next. ## “Done” contains several different claims Consider an agent that says: > The checkout incident is fixed. That may contain at least four claims: a change was created; the change was deployed; the deployment is healthy; and the original user-visible failure is resolved. A deployment controller, health probe, telemetry stream, and synthetic checkout each establish something different. Do not collapse **action success → task success → intent satisfaction** into one state. ## Verification is a graph A long-running task is better represented as a graph of claims and evidence. At the top might be **Incident resolved**, supported by **Original symptom no longer occurs**, **No material regression was introduced**, **Expected version reached production**, and **Relevant system indicators recovered**. ```text claim ├─ evidence ├─ evidence └─ subclaim ├─ evidence └─ verifier ``` The final verdict is only as strong as the path beneath it. The graph exposes what remains unproven, which is information ordinary pass/fail systems hide. ## Evidence has provenance An agent may run its own unit tests. That is useful. An independent test service may rerun them from a clean environment; that is stronger for acceptance. A production synthetic monitor may observe the actual user path; that proves something different again. Evidence should retain what produced it, which artifact or state it refers to, when it was observed, which verifier interpreted it, and which claim it supports. Without provenance, a system accumulates green checks whose relationship to current state is unclear. ## Evidence can expire A test result attached to an immutable commit continues to describe that commit. “Production is healthy” is an observation with a short half-life. A system that measured health three days ago cannot use it as proof that production is healthy now. Evidence needs temporal semantics. A verifier should be able to say that evidence was valid when observed but is no longer current enough to authorize the next action. Verification becomes a living property of the task rather than a historical checkbox. ## Evidence can unlock authority Suppose an agent begins with production read, repository write, staging write, and no production mutation. It reproduces the issue, creates a fix, passes protected checks, and succeeds in staging. Those events can satisfy a policy condition that makes `production.deploy.request` available. After approval, `production.deploy` can become available as a temporary capability. After deployment, that permission disappears and the system returns to read-only observation. **Authority should follow evidence.** Not all evidence should automatically increase authority. Policy defines the outer envelope; verified progress can justify new capabilities inside it. ## Failure should reduce authority Three failed verifications, changed system state, an unexpected security event, or a task that consumes most of its budget without measurable progress can make the next individually permitted action a bad idea. The runtime can revoke write access, require replanning, invoke an independent reviewer, move to read-only, or escalate to a person. This is adaptive autonomy. The system manages the trajectory of delegated work, not merely whether one API call is permitted. ## The verifier is a role, not a model There is no universal verifier. A compiler, type checker, policy engine, database reconciliation query, browser test, independent model, or human can serve that role. The right verifier depends on what truth means for the claim. Prefer deterministic checks where they exist, independent observation where the environment can be measured, model judgment where semantic assessment is genuinely required, and human judgment where consequence or ambiguity demands it. “Ask another LLM whether this looks good” can help; it is not the whole assurance layer. ## Protect the verifier from the proposer If an agent can rewrite the rule that determines whether it succeeded, the verification loop becomes circular. An optimizing system naturally discovers whatever the acceptance mechanism rewards. Protected tests, held-out examples, versioned rubrics, policy rules, and independent credentials keep the target stable enough to mean something. When the task teaches that the old rules were incomplete, change the contract in a new version; do not silently rewrite the meaning of an existing pass. ## Verification is therefore state For long-horizon systems, the important runtime object may combine intent, current claims, supporting evidence, unresolved uncertainty, active policy, current authority, required verifiers, evidence freshness, and available next transitions. The model reasons inside it. The runtime governs transitions around it. Verification connects the two. ## Draft boundary This is a publishable conceptual draft. It does not report a completed implementation, live experiment, or verified production result, and it does not claim that ReactorJet ships a verification runtime. ## Related - [Controlled delegation](/guides/concept/controlled-delegation) - [A runtime for agents is an authority system](/guides/concept/runtime-as-authority-system) - [Long-Horizon Verification](/work/long-horizon-verification) - [Authority Compiler](/work/authority-compiler)