docs(criteria): name the budget-breach trigger on the post-failure path

`_run_evaluation_with_failure_evidence` catches BudgetExceededError alongside
AgentCrashError and TurnTimeoutError, so a `read_only` command also runs after
a token or cost breach that stopped grading part-way. The field description,
both guides and the method docstring said otherwise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: rockymadden

docs/REPORT_SCHEMA.md

@@ -188,8 +188,9 @@ fields so subclass keys round-trip.
 
 ### Post-failure criterion evidence
 
-When an agent crashes or its turn times out, coder-eval runs only deterministic,
-read-only artifact criteria while the sandbox is still live: `file_exists`,
+When an agent crashes, its turn times out, or a token/cost budget breach stops grading
+part-way, coder-eval runs only deterministic, read-only artifact criteria while the
+sandbox is still live: `file_exists`,
 `file_contains`, `file_matches_regex`, `file_check`, `json_check`,
 `reference_comparison`, and `classification_match`. A `run_command` criterion joins them
 only when the task author sets [`read_only: true`](TASK_DEFINITION_GUIDE.md#run_command)

docs/TASK_DEFINITION_GUIDE.md

@@ -905,7 +905,7 @@ Runs a command and checks the exit code, with optional stdout matching. **Binary
   stdout_match: "exact"               # "exact" (default), "contains", or "regex"
   description: "Script must output the correct text"
 
-# Graded even when the agent's turn times out or the agent crashes
+# Graded even when the turn times out, the agent crashes, or a budget breach cuts grading short
 - type: "run_command"
   command: "python graders/check_flow.py"
   read_only: true                     # declaration, not enforcement -- see below
@@ -920,7 +920,7 @@ Runs a command and checks the exit code, with optional stdout matching. **Binary
 | `expected_stdout` | `null` | When set, stdout is also checked |
 | `stdout_match` | `"exact"` | Match mode: `exact` (stripped), `contains` (substring), `regex` (pattern) |
 | `score_from_stdout` | `false` | Read a float score (0.0–1.0) from the first stdout line (remaining lines become details); a non-zero exit code or a parse failure scores 0.0. Mutually exclusive with `expected_stdout`. |
-| `read_only` | `false` | Declares the command inspects artifacts only. Its sole effect: the criterion is also graded after a terminal agent failure — see [Post-failure criterion evidence](REPORT_SCHEMA.md#post-failure-criterion-evidence). |
+| `read_only` | `false` | Declares the command inspects artifacts only. Its sole effect: the criterion is also graded on the post-failure diagnostic path — after a turn timeout, an agent crash, or a budget breach that stopped grading. See [Post-failure criterion evidence](REPORT_SCHEMA.md#post-failure-criterion-evidence). |
 
 `read_only` is an author declaration, **not** a restriction. coder-eval cannot decide
 whether a shell command is pure, so it verifies nothing and confines nothing: the

plugins/coder-eval/reference/criteria.md

@@ -249,7 +249,7 @@ Optional:
 | `expected_stdout` | Expected stdout content. When set, stdout is also checked. |
 | `stdout_match` | How to match stdout: 'exact' (stripped), 'contains' (substring), 'regex' (pattern) |
 | `score_from_stdout` | When true, read a float score (0.0-1.0) from the first line of stdout. Remaining lines are captured as details. Non-zero exit code or parse failure -> score 0.0. Mutually exclusive with expected_stdout. |
-| `read_only` | The task author's DECLARATION that this command only inspects artifacts: it writes nothing, reaches no live service, and returns the same verdict every run. Its only effect is eligibility -- the criterion is also graded after a terminal agent failure (turn timeout or agent crash), while the sandbox is still live, so a timed-out run records what the artifacts were worth. That result is diagnostic and never moves final_status or weighted_score. UNVERIFIED AND UNENFORCED: purity of a shell command is not decidable here, the command runs exactly as it would normally, and nothing stops a mutating command from being marked. Leave it false for anything that writes state or calls a live tenant (e.g. 'uip maestro flow debug', which starts a cloud job). |
+| `read_only` | The task author's DECLARATION that this command only inspects artifacts: it writes nothing, reaches no live service, and returns the same verdict every run. Its only effect is eligibility -- the criterion is also graded on the post-failure diagnostic path, while the sandbox is still live, so a run that died mid-flight still records what the artifacts were worth. Three failures reach that path: a turn timeout, an agent crash, and a token or cost budget breach that stopped grading part-way. The command runs on every one of them. That result is diagnostic and never moves final_status or weighted_score. UNVERIFIED AND UNENFORCED: purity of a shell command is not decidable here, the command runs exactly as it would normally, and nothing stops a mutating command from being marked. Leave it false for anything that writes state or calls a live tenant (e.g. 'uip maestro flow debug', which starts a cloud job). |
 
 ### `skill_triggered`
 

src/coder_eval/models/criteria.py

@@ -413,9 +413,11 @@ class RunCommandCriterion(BaseSuccessCriterion):
         description=(
             "The task author's DECLARATION that this command only inspects artifacts: it writes "
             "nothing, reaches no live service, and returns the same verdict every run. Its only "
-            "effect is eligibility -- the criterion is also graded after a terminal agent failure "
-            "(turn timeout or agent crash), while the sandbox is still live, so a timed-out run "
-            "records what the artifacts were worth. That result is diagnostic and never moves "
+            "effect is eligibility -- the criterion is also graded on the post-failure diagnostic "
+            "path, while the sandbox is still live, so a run that died mid-flight still records "
+            "what the artifacts were worth. Three failures reach that path: a turn timeout, an "
+            "agent crash, and a token or cost budget breach that stopped grading part-way. The "
+            "command runs on every one of them. That result is diagnostic and never moves "
             "final_status or weighted_score. UNVERIFIED AND UNENFORCED: purity of a shell command "
             "is not decidable here, the command runs exactly as it would normally, and nothing "
             "stops a mutating command from being marked. Leave it false for anything that writes "

src/coder_eval/orchestrator.py

@@ -893,8 +893,10 @@ async def _evaluate_post_failure_criteria(self) -> None:
         """Evaluate diagnostic criteria before the live sandbox is torn down.
 
         Results stay outside the canonical scored list. Only criteria that
-        declare themselves deterministic and read-only run on this path. This
-        excludes judges and checks that execute sandbox commands.
+        declare themselves deterministic and read-only run on this path. That
+        excludes judges and trajectory checks, and every ``run_command``
+        criterion except one the task author marked ``read_only`` -- which DOES
+        execute a sandbox command here.
         """
         if self.result is None:
             return