Run an agent task with an independent check

An agent can inspect its own output and fix obvious mistakes. That is useful feedback, but it is not an independent reason to accept the work. In this tutorial, the agent produces a proposal in a scratch workspace. A separate checker reads that proposal, runs protected checks, and returns a verdict.

The example uses a repository task, but the pattern also fits reports, support drafts, data changes, and other work that can be saved before it takes effect.

1. Write the task contract

Start with a task that has a bounded output and at least one result you can check without asking the agent whether it succeeded.

Record these fields before the run starts:

task_id: TASK-1042
objective: update the retry policy for the import worker
allowed_scope: worker configuration and its tests
forbidden_scope: deployment settings, credentials, unrelated services
required_checks:
  - configuration parses
  - existing retry cases still pass
  - maximum delay remains within policy
budget:
  attempts: 3
  elapsed_minutes: 20

The objective tells the agent what to change. The checks tell the checker what evidence to collect. Keep them separate. If the agent can rewrite the check list during the run, the check no longer has independent authority.

Expected outcome: the task has a stable identifier, an explicit write scope, and a stopping point. A reviewer should be able to understand the task without reading the agent transcript.

2. Prepare separate proposal and verdict paths

Give the agent a disposable workspace. It may read the task inputs and write a proposal there, but it should not be able to merge, publish, deploy, or alter the protected checks.

Give the checker read access to the completed proposal and write access to a separate verdict record. Where possible, use different credentials or processes for the two roles. A prompt that tells one agent to “act as an independent reviewer” does not create separation if the same process controls the proposal, checks, and verdict.

Save the versions of the task, baseline, checker, and check data with the run:

run_id: RUN-8831
task_version: sha256:...
baseline_version: rev:...
checker_version: check-suite:17
check_data_version: retry-cases:6
workspace: scratch/RUN-8831

Expected outcome: changing the proposal cannot silently change the rules used to judge it.

3. Run the agent in proposal mode

Ask the agent to produce a saved change set and a short completion record. The record should point to evidence rather than paraphrase it.

run_id: RUN-8831
proposal_ref: sha256:...
changed_scope:
  - worker configuration
  - retry policy tests
agent_checks:
  - name: focused tests
    result_ref: evidence/agent-focused-tests.txt
uncertainties:
  - production queue timing was not observed

Self-run tests are still worth keeping. They help the agent iterate and give the checker useful context. Treat them as claims to inspect, not as the final verdict.

Stop the run if the agent writes outside the allowed scope, exhausts its budget, or cannot produce a stable proposal reference. Do not turn a partial workspace into an accepted result because the transcript sounded confident.

Expected outcome: the agent leaves one immutable proposal that can be checked or discarded without affecting the live system.

4. Submit the proposal to the checker

Pass the checker the task contract, baseline reference, and proposal reference. Do not make the transcript the source of truth. The checker should inspect the saved result and collect fresh evidence.

A useful check sequence is:

1. Confirm the proposal reference matches the saved content.
2. Reject files or fields outside the allowed scope.
3. Run the declared functional checks from a clean state.
4. Compare the proposal with the saved baseline.
5. Run at least one protected case the agent did not choose.
6. Write each result and its evidence reference to the verdict record.

The protected case does not need to be secret. It needs to be outside the agent’s write permissions for this run.

5. Turn evidence into a verdict

Use named outcomes rather than a score with an improvised threshold:

Verdict	Meaning	Workflow action
PASS	Required checks pass and no protected behavior regresses	Mark the proposal eligible for the next boundary
REVISE	The proposal is valid but a named requirement failed	Return the failed evidence to a new proposal attempt
STOP	Scope, policy, or a previously passing case regressed	Discard the proposal and require investigation
ERROR	The checker could not produce a trustworthy result	Keep the proposal isolated and retry the check later

An unreachable checker is ERROR, not PASS. Missing evidence is not neutral.

The final record might look like this:

run_id: RUN-8831
proposal_ref: sha256:...
verdict: PASS
checker_version: check-suite:17
checks:
  - name: allowed scope
    result: pass
    evidence_ref: evidence/scope-diff.txt
  - name: retry cases
    result: pass
    evidence_ref: evidence/retry-cases.xml
  - name: maximum delay policy
    result: pass
    evidence_ref: evidence/policy-result.json

PASS does not mean the work is universally safe. It means this exact proposal passed the named checks against the recorded baseline. Promotion or deployment may still need a human approval boundary.

Check the workflow

Run both positive and negative checks before relying on the design:

• Submit a valid proposal and confirm the checker emits PASS with readable evidence references.
• Make a proposal that edits a forbidden path and confirm the checker emits STOP, even if all functional tests pass.
• Break a previously passing protected case and confirm the regression cannot be hidden by a better aggregate score.
• Make the checker unavailable and confirm the proposal remains isolated.
• Change the proposal after checking and confirm its old verdict no longer applies because the content reference changed.

Handle failures

If the proposal fails a check, create a new attempt linked to the same task. Do not overwrite the failed evidence. The history should show what changed between attempts.

If the checker fails, repair or retry the checker without granting the agent permission to declare itself successful. If the task contract was wrong, close the run and start a new task version. Rewriting the contract under an existing verdict makes the record ambiguous.

If every check passes but a person still finds the result wrong, add a case that captures the missed requirement before the next run. Keep the original failure visible. It is evidence that the old check set was incomplete.

Next steps

• Add a human approval boundary
• Read why the agent should not be its own reviewer
• Use the two-plane loop as an architecture reference