An agent says it is 95 percent confident that a change is correct. What should
happen next?

The number may help explain the agent's internal state. It does not prove that
the change works. Confidence is a belief, estimate, or model output. Evidence is
an observation tied to the thing being judged.

Operational decisions should start with evidence. Confidence can help route
uncertain cases, but it should not substitute for a test, comparison, policy
check, or human judgment.

## What evidence looks like

Useful evidence is specific enough that another person or system can inspect
where it came from and what it supports. For a software change, that may include:

- the exact input revision and candidate diff;
- the command and environment used for a check;
- the test result, including failures and skipped cases;
- a screenshot or trace from the changed user flow;
- a comparison with a saved baseline;
- the policy version used to issue a verdict;
- the deployed revision observed after release.

No single item proves everything. A unit test can show that one code path behaves
as expected. It cannot show that the correct build reached production. A
screenshot can show visible behavior at one moment. It cannot prove that an
authorization rule holds for every request.

The decision should name which claims each piece of evidence supports.

## A concrete example

Suppose an agent changes the checkout form to fix an address validation bug.

The agent runs the unit tests and they pass. That is evidence, but it answers a
narrow question: did the checks encoded in that suite pass in that environment?

A stronger evidence chain could include:

1. The issue names the failing customer case and expected behavior.
2. A regression test fails on the baseline and passes on the candidate.
3. Existing checkout cases still pass on the same candidate revision.
4. A browser run completes the affected flow at the supported viewport.
5. The candidate diff shows no change to the protected acceptance rule.
6. After deployment, an observation identifies the running revision and repeats
   the critical check.

The chain still does not prove that checkout can never fail. It gives a reviewer
or policy engine a traceable reason to accept this change within a stated scope.

Contrast that with: "The implementation is robust and I am highly confident it
is production-ready." The sentence contains no inspectable support.

## Evidence needs identity

An artifact becomes much less useful when it is detached from the run that
created it. Record enough identity to answer:

- What subject was examined?
- Which revision, environment, and configuration were used?
- Who or what produced the observation?
- When was it produced?
- Which attempt and policy does it belong to?
- Has it changed since the verdict was issued?

This is why a copied "all tests passed" line is weak. It omits the command,
revision, environment, full result, and relationship to the candidate under
review.

## Prefer comparisons to adjectives

Words such as safe, fast, accurate, and ready need a reference point.

Instead of saying "the candidate is faster," compare the candidate and baseline
under the same workload. Instead of saying "the output is accurate," score both
versions against saved examples with a declared rubric. Record regressions as
well as aggregate improvement.

A comparison can still be flawed. The case set may be too small, the environment
may differ from production, or the metric may miss what users care about. Those
limits belong beside the result, not hidden behind a larger confidence number.

## Protect the meaning of success

Evidence is weak when the thing being judged can quietly alter the judge.

A coding agent may need to add or update tests as part of legitimate work. That
does not mean every changed test should automatically count as independent proof
that the patch is correct. Keep held-out cases, policy checks, or final approval
under separate authority when the consequence warrants it.

This separation also applies to non-code work. A support agent can draft a
refund response, but it should not be able to change the refund limit used to
approve that response. A research agent can summarize sources, but its statement
that the citations are valid is not the same as checking the cited documents.

## Match the evidence to the consequence

A private draft and a production deployment do not need the same proof.

For a low-risk, reversible draft, an agent explanation and a lightweight check
may be enough. For money, identity, access, public claims, or irreversible data,
require stronger evidence and an independent authority. Name the required human
approval where automation is not appropriate.

A practical decision policy is:

- If required evidence is present and passes the declared rule, allow the named
  next step within its blast radius.
- If evidence contradicts itself, stop and surface the conflict.
- If required evidence is missing, treat the result as unknown rather than pass.
- If a previously passing protected case regresses, let that failure outrank an
  improved aggregate score.

Failing closed does not mean treating every uncertainty as a disaster. It means
uncertainty does not silently become permission.

## A small evidence packet

Before accepting consequential agent work, ask for a packet with:

1. the requested outcome and scope;
2. the candidate change or artifact;
3. the baseline used for comparison;
4. observations from the relevant checks;
5. known limits and skipped checks;
6. the verdict and the authority that issued it;
7. the next allowed action.

This packet can be a structured record rather than a document. Its purpose is
to keep the decision reproducible and stop confidence from outrunning proof.

## Related reading

- [A passing test is not always a good result](/notes/control-loop/a-passing-test-is-not-a-good-result)
- [Build the checks before you build the agent](/notes/control-loop/closing-the-loop)
- [The closure test](/reference/closure-test)
- [Control plane versus work plane](/guides/concept/control-plane-versus-work-plane)