Skip to content

Give Benchmark Run attempt lifecycle one durable owner #193

Description

@justsml

Problem

evals/benchmark-run-lifecycle.ts exposes preflight, execute, finalize, teardown, and persist operations, but callers reconstruct final records and persistence outside the module. The matched-harness path supplies a no-op persistence operation and then persists again; browser execution similarly retains important lifecycle behavior outside the attempt module. The interface is nearly as complex as the implementation and does not yet protect the core forensic invariant.

What to build

Deepen Benchmark Run attempt lifecycle so it owns the durable transition from planned row through rejection/admission, execution, finalization, teardown, persistence, and retry lineage. Execution-specific behavior remains behind the existing adapter seam.

This ticket is the smallest enabling slice for the Browser Benchmark Run deepening ticket and should ship independently.

Acceptance criteria

  • Preflight rejection, execution failure, finalization failure, teardown failure, and success each produce one persisted attempt record.
  • Persistence is not a no-op adapter operation followed by caller-owned persistence.
  • Attempt numbering, retry lineage, and supersession are durable and cannot be reconstructed differently by each runner.
  • Teardown is attempted exactly once for every admitted execution and its verified/unverified outcome is retained even when other stages fail.
  • A returned execution result without a valid final record cannot disappear or be treated as success.
  • Browser and matched-harness callers consume the same lifecycle outcome without rebuilding competing record semantics.
  • Integration coverage observes the stable lifecycle seam, including finalize, teardown, and persistence failures; it does not assert implementation text or report prose.

Related work

Architecture evidence

The 2026-08-30 architecture review applied the deletion test: removing this module would duplicate consequential try/finally/persist ordering across current and future runners, but its current six-operation interface remains shallow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    difficulty: LLarge cross-module implementation scopeenhancementNew feature or requestpriority: highHigh-value safety or evaluation integrity workready-for-agentReady for an implementation agent

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions