Structure evidence for a workflow

Evidence is useful when a reviewer can trace a decision back to an observation without replaying the agent’s reasoning. A transcript may provide context, but it does not by itself prove that a file existed, a policy allowed an action, or a deployed system behaved as expected.

Build the bundle around claims. For every claim, record what was observed, how it was observed, and which version of the subject produced it.

1. List the claims before collecting artifacts

Write one claim for each condition that must be true at the decision point.

For a deployment workflow:

C-01 The candidate artifact was built from commit 8a61...
C-02 The protected evaluation passed for that artifact digest.
C-03 The production policy gate allowed that artifact and environment.
C-04 The deployed endpoint serves the expected release identifier.
C-05 The previous release can be restored through the documented rollback path.

Avoid claims such as “everything works.” A reviewer cannot tell what evidence would disprove them.

Give claims stable identifiers. Use the same identifiers in acceptance rules, evidence manifests, and verdict records.

2. Define acceptable evidence for each claim

Specify the observation method and freshness rule before the run.

Claim	Required evidence	Reject when
C-01	Build attestation naming commit and artifact digest	Digest is missing or commit differs
C-02	Verdict from protected evaluator	Verdict is absent, stale, or for another digest
C-03	Gate decision bound to digest and environment	Decision expired or target differs
C-04	Response observed from production with release identifier	Observation came only from local preview
C-05	Rollback rehearsal result against the candidate release	Procedure is documented but untested

This step keeps convenient artifacts from replacing relevant ones. A local screenshot does not prove production behavior. A passing unit test does not prove which release is deployed.

3. Give every run an identity

Create the run record before work begins. Use an identifier that remains stable across retries and append an attempt number for each execution.

run_id: deploy-2026-08-13-017
attempt: 2
workflow: production-deploy-v5
subject:
  commit: 8a61...
  artifact_digest: sha256:...
started_at: 2026-08-13T10:20:00Z
proposer:
  type: agent
  version: deploy-agent-12

Do not use a mutable branch name as the subject. Record the commit, artifact digest, database revision, message digest, or other immutable identity that the workflow actually changes.

4. Store observations as records

An observation should include:

• a unique evidence identifier;
• the claim or claims it supports;
• the subject identity;
• the observation method;
• the source or environment;
• the tool and version used;
• the observation time;
• the result and exit status;
• a digest or pointer for the raw artifact;
• any redaction applied.

evidence_id: E-04
supports:
  - C-04
subject_digest: sha256:7c9d...
method: http-release-probe
source: production
observer: release-probe-v2.3.1
observed_at: 2026-08-13T10:34:12Z
result: pass
facts:
  release_id: 8a61...
raw_artifact:
  path: observations/E-04-response.json
  sha256: 5b28...
redactions: []

Keep conclusions out of raw artifacts. The observation says what the probe received. The verdict says whether that observation satisfies the claim.

5. Preserve raw and derived evidence separately

Raw evidence is the direct output of a tool or observer. Derived evidence is a summary, comparison, score, or screenshot produced from it.

Use a simple directory shape:

run.json
claims.yaml
observations/
  E-01-build-attestation.json
  E-02-evaluation-verdict.json
  E-03-policy-decision.json
  E-04-response.json
derived/
  release-comparison.md
decision.json

Store a digest for each artifact in the manifest. If an artifact must live in another system, store a stable reference, source identity, access requirements, and digest when one is available.

Do not place secrets in the bundle. Redact at collection time, record that a redaction occurred, and keep the protected source under its existing access controls.

6. Include negative evidence

Positive evidence shows that the intended path worked. Negative evidence shows that a prohibited or broken path did not pass.

For a deployment, useful negative checks might include:

• an artifact with the wrong digest cannot reuse the gate decision;
• an expired decision cannot authorize execution;
• a failed protected case blocks promotion;
• the agent identity cannot deploy directly;
• an unreachable evaluator produces no verdict rather than a pass.

Link each negative check to a claim or boundary. Do not collect failures merely to make the bundle look complete.

7. Record the decision as a separate artifact

The final record should identify the claims, evidence, rule, and authority that produced the decision.

{
  "run_id": "deploy-2026-08-13-017",
  "decision": "promote",
  "decision_rule": "production-deploy-v5",
  "subject_digest": "sha256:7c9d...",
  "claims": {
    "C-01": ["E-01"],
    "C-02": ["E-02"],
    "C-03": ["E-03"],
    "C-04": ["E-04"],
    "C-05": ["E-05"]
  },
  "decided_by": "release-controller",
  "decided_at": "2026-08-13T10:40:00Z"
}

If evidence is missing or contradictory, record no_verdict or the equivalent workflow state. Do not fill gaps with the agent’s confidence statement.

8. Verify the bundle

Use a fresh reader or an automated validator. The producer’s ability to read its own output is not enough.

1. Start from decision.json and resolve every claim to at least one evidence record.
2. Recompute artifact digests and compare them with the manifest.
3. Confirm every evidence record names the same subject, or explicitly explains why another subject is relevant.
4. Check observation times against freshness rules and decision expiry.
5. Confirm raw artifacts exist and derived summaries point back to them.
6. Verify both the allowed path and the required negative checks.
7. Remove one required artifact and run the validator. Confirm the bundle becomes incomplete rather than silently valid.
8. Replace one artifact with evidence from another run. Confirm the subject or digest mismatch is detected.

Common failure modes

The transcript is the evidence bundle

A transcript records conversation and tool narration. Keep it as context if it helps debugging, but use direct outputs and observations for acceptance claims.

Evidence has no subject identity

“Tests passed” is ambiguous when several commits or artifacts exist. Bind each record to the immutable subject that was tested.

Screenshots have no source or time

A screenshot may be relevant visual evidence, but record the environment, URL or view, capture time, release identity, and the claim it supports.

Only successful checks survive

Store failed attempts and superseded observations with their attempt number. Do not overwrite the history that explains why the workflow retried.

Raw artifacts contain credentials or personal data

Decide what the claim needs before collection. Store the minimum evidence, redact deliberately, and record where protected source data remains.

A summary introduces facts not present in the raw output

Derived evidence should cite the evidence identifiers it used. A validator or reviewer should be able to follow each statement back to an observation.

Related reading

• A passing test is not always a good result
• Your agent should not be its own reviewer
• Two-plane loop
• Agent eval pack