## Purpose

An independent evaluator judges a proposed result without relying on the
producer's confidence. It compares the intent brief, the subject artifact, and
the evidence packet with a versioned rubric. It returns a verdict for each
criterion and for the proposal as a whole.

Independence is a control relationship, not a job title. The producer should
not be able to change the rubric, select only favorable evidence, overwrite the
verdict, or grant the capability that follows a passing result.

## When to use

Use an independent evaluator when a workflow needs more than self-review before
it can merge, publish, deploy, or hand work to another system. It is especially
useful when the worker is probabilistic, the acceptance criteria combine
several checks, or a passing test can still miss the requested outcome.

The evaluator can be deterministic code, a separate model invocation, a test
harness, a human review role, or a combination. Choose the simplest form that
can judge the stated criteria. Use deterministic checks for facts that can be
computed directly. Reserve model or human judgment for criteria that require
interpretation.

## Inputs and outputs

Inputs:

- the exact intent brief version;
- the proposed artifact or change digest;
- a sealed evidence packet;
- a versioned rubric with required and advisory criteria;
- evaluator identity and execution environment;
- saved examples or baselines when the rubric requires comparison.

Outputs:

- a verdict of `pass`, `fail`, `insufficient-evidence`, or `error`;
- one result per rubric criterion;
- evidence references and reasons for every result;
- the evaluator and rubric versions;
- recommended next action, limited to retry, revise, review, or stop.

## Smallest useful interface

```ts
type CriterionResult = {
  criterionId: string;
  result: "pass" | "fail" | "insufficient-evidence" | "error";
  evidenceRefs: string[];
  reason: string;
};

type Evaluation = {
  subjectDigest: string;
  evaluatorVersion: string;
  rubricVersion: string;
  verdict: "pass" | "fail" | "insufficient-evidence" | "error";
  criteria: CriterionResult[];
};

function evaluate(input: {
  intentRef: string;
  evidencePacketRef: string;
  rubricRef: string;
}): Evaluation;
```

The evaluator should reject a packet whose subject digest does not match the
proposal. A required criterion without evidence should not pass by default.

## Example

Rubric `checkout-timeout@2` contains two required criteria:

1. The timeout path returns the documented retry response.
2. Existing successful checkout behavior has regression evidence.

The evidence packet contains a passing focused timeout test but says the full
integration suite could not run. The evaluator returns:

```json
{
  "subjectDigest": "sha256:41de...",
  "evaluatorVersion": "checkout-evaluator@4",
  "rubricVersion": "checkout-timeout@2",
  "verdict": "insufficient-evidence",
  "criteria": [
    {
      "criterionId": "timeout-response",
      "result": "pass",
      "evidenceRefs": ["packet:run-1842#timeout-test"],
      "reason": "The focused test exercises the required timeout response and exits successfully."
    },
    {
      "criterionId": "successful-checkout-regression",
      "result": "insufficient-evidence",
      "evidenceRefs": [],
      "reason": "The required integration check did not run."
    }
  ]
}
```

The worker may repair the test environment or add valid regression evidence.
It may not convert the second criterion to advisory during the run.

## Failure modes

- **The producer also owns the rubric.** Criteria can drift toward the current
  output. Put rubric changes behind separate review and version them.
- **The evaluator reads the worker's summary instead of source evidence.** A
  persuasive explanation can replace observation. Resolve evidence references
  and judge the artifacts directly.
- **Missing evidence becomes a pass.** Use a distinct
  `insufficient-evidence` result so absence is not confused with success.
- **The evaluator changes between retries without a record.** Results cannot be
  compared. Store the evaluator version, rubric version, and subject digest.
- **A single score hides a required failure.** Report criterion-level results
  and define which criteria are mandatory.
- **The worker retries until a variable judge passes.** Set retry limits, retain
  every verdict, and route repeated disagreement to review.
- **The evaluator can execute the proposed change.** Keep judgment separate
  from the capability to merge, deploy, publish, or spend.
- **Saved examples only cover familiar successes.** Add boundary cases and
  known failures when maintaining the rubric. Do not let the producer choose
  the evaluation set for each run.

## What it does not solve

An independent evaluator does not make a weak rubric meaningful. It cannot
detect conditions that the evidence and criteria never expose. Separation also
does not guarantee different blind spots if the producer and evaluator use the
same inputs and assumptions. The component creates a reviewable authority
boundary; teams still need to test the evaluator and update its rubric through
a controlled process.

## Related reading

- [Evidence packet](/components/evidence-packet)
- [Human approval checkpoint](/components/human-approval-checkpoint)
- [Policy gate](/components/policy-gate)
- [Your agent should not be its own reviewer](/notes/control-loop/your-agent-should-not-be-its-own-reviewer)
- [Closure test](/reference/closure-test)