Agent activity is not progress

Agent dashboards are very good at showing motion. Runs started. Tools called. Tokens spent. Files changed. Tests executed. A stream of events makes the system look productive even when the original problem is still open.

Activity is an operating cost. It is not evidence of progress.

This distinction sounds obvious until a run gets stuck. An agent edits a test, runs it, reads the failure, edits another file, and repeats. The event count rises. The transcript becomes impressive. If the user-visible defect remains, the workflow has produced activity and no progress.

Progress needs a target state

You cannot measure progress from the agent’s behavior alone. You need a target state and evidence that compares it with what existed before.

For a coding task, the target may include a specific behavior in the deployed application, not merely a patch and a green local suite. For an incident agent, progress might mean the error rate has returned below a threshold and remained there for an observation window. For a support agent, drafting twelve replies is activity; resolving the right cases under the applicable policy is progress.

The unit should be a verified state transition. “Opened pull request” can be a useful workflow milestone, but it does not mean “fixed the problem.” “Sent response” does not mean “resolved the request.” The names on the dashboard should preserve those differences.

This is where many agent metrics go wrong. They measure what is easy to count inside the run and ignore what happened to the system outside it. A model can optimize tool use without improving the outcome. It can also reduce tool calls by skipping checks, which looks efficient until a bad change escapes.

Busy loops need an exit condition

An agent should not be allowed to turn uncertainty into indefinite work. Every loop needs a condition for success, a reason to continue, and a point where it stops consuming resources and asks for help.

Consider an agent investigating a slow service. Querying logs is reasonable at the start. After six similar queries produce no new distinction between likely causes, another query is not progress. The next useful action may be to request missing telemetry, run a bounded experiment, or hand the case to someone with different access.

A retry budget helps, but a raw attempt count is crude. Track whether each pass produced new evidence, eliminated a plausible cause, or changed the candidate solution. Repetition without information gain should make the workflow stop sooner than a difficult investigation that is still narrowing the problem.

Exploration can still count

Not all valuable work closes a task immediately. Research, diagnosis, and design often begin without a known route to the answer. Forcing them into a simplistic completion metric encourages premature certainty.

Exploration can count as progress when it leaves a durable change in what the team knows. A failed experiment may rule out an approach. A trace may reveal which boundary lacks telemetry. A prototype may show that the requested latency is incompatible with a dependency.

The evidence still matters. “Investigated several options” is activity. “Option B failed the saved workload because it exceeded the agreed limit” is a result someone else can use. The second statement reduces uncertainty even though no code shipped.

Activity metrics remain useful for cost and diagnosis. Tool-call counts can expose a loop. Runtime can show a slow dependency. Token use can help compare workflow designs. Just do not promote those numbers into outcome metrics. A cheaper run that fails more often is not automatically better, and a longer run that closes a hard case is not automatically worse.

Change the scoreboard

Pick one agent workflow and write its target state in observable terms. Add a completion check that runs outside the agent’s narrative, then label intermediate events for what they are: attempted, drafted, tested, reviewed, or applied. Do not call the work complete until the target-state evidence exists.

For runs that do not close, record the useful delta: evidence gathered, a cause excluded, or a precise blocker with an owner. Put activity metrics on a separate cost view. If the same run looks productive only when the event stream is on screen, the scoreboard is measuring theatre.