Skip to content

round-16: All #164 corruption cells inject a synthetic replay failure; no end-to-end heal over genuinely corrupt on-disk evidence #174

Description

@roninjin10

Severity: p2
Area: engine-store
Slug: corruption-heal-never-driven-through-real-evidence

Claim

Every cell reaching the cache-hit corruption branch pipes a failingReplay stub whose replayOutputs fails with a hand-built BoundaryCorruption, so nothing pins that real corrupt bytes (a truncated blob, an undecodable inline row) drive detect → evict → re-execute → re-capture to a clean subsequent hit. A change to materialize's error shape or replayCorruption's cause matching could silently reclassify real corruption as reason 'host' (no evict, no record) while all cells stay green.

Evidence

  • packages/engine-store/test/ReplayCorruptionClassification.test.ts — every cell reaching the cache-hit corruption/eviction branch (lines 115-119, 167-171, 236-242, 267-271, 287-293, 330-354, 379-390, 420-427, 448, 477) provides the failingReplay stub layer (defined at line 75), failing with a hand-constructed BoundaryCorruption over literal aa…/bb… digests.
  • Real corruption evidence is tested only in isolation, by calling boundary.replayOutputs directly: MaterializedOutputBounds.test.ts:288-303 (truncated blob at the canonical address), :305-327 (tampered inline content), plus StepBoundary.test.ts:235-281 and BlobStoreAtomicity.test.ts:221/301. None joins to dispatch, classification, or eviction.
  • packages/engine-store/src/internal/ActivityPersistence.ts:101-105replayCorruption matches via reason.error instanceof StepBoundary.BoundaryCorruption on the materialized cause; used at :448 and :590.

Failure scenario

materialize's error wrapping changes (or BoundaryCorruption stops being the instance on the cause). Real truncated-blob corruption is then classified as a retryable/'host' failure — no eviction, no durable corruption record — while every stub-driven cell stays green because the stub reproduces the expected error shape by construction.

Verifier confirmation

The claim holds against the actual code; all cited line ranges verified. A sweep of engine-store tests confirms every dispatch-path test uses StepBoundary.layerTest() or the failingReplay stub; no end-to-end test does capture → corrupt on disk → dispatch under the real boundary layer → assert eviction + re-execution + clean subsequent hit. The instanceof/cause-shape coupling at ActivityPersistence.ts:101-105 is exactly the kind that breaks silently under stub-only coverage.

Missing test

Capture a real sealed output to a blob, truncate the file, dispatch with the real StepBoundary layer under Inconsistency.layerTolerant, assert record landed + cache None + one re-execution + clean third-dispatch hit.

Found by the iterate-until-dry loop, round 15 review.

Metadata

Metadata

Assignees

No one assigned

    Labels

    second-reviewsecond-pass review finding

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions