Skip to content

Check recorded run conditions; limit run output; hash inputs once - #48

Merged
Deep-Sixed merged 1 commit into
mainfrom
claude/optimistic-mendel-82kq2z
Sep 25, 2026
Merged

Deep-Sixed merged 1 commit into
mainfrom
claude/optimistic-mendel-82kq2z

Conversation

@Deep-Sixed

Copy link
Copy Markdown
Owner

Fixes the three P3s from the review of main at a986152. The P1 and P2 from that review were fixed in #44.

Recorded run conditions weren't checked against the run

record_parser_run builds RunConditions from the run itself, so it was trustworthy. But ledger_transaction and record_run accepted any RunConditions a caller passed, even though the RunConditions docstring said they were "never asserted by a caller". A replay runs the parser under the recorded conditions, so a wrong value made the replay fail or report a misleading difference.

#27 made every backend report the limits it applied, in SandboxResult.telemetry.limits. ledger_transaction now compares each stated condition with them:

Condition Compared with
device gpu
memory memory
cpus cpus
pids_limit pids
timeout_seconds timeout_seconds

A mismatch, or a condition the backend doesn't report, raises ProvenanceError before anything is recorded. The docstring now describes the check.

Parser output on disk had no size limit

  • SandboxConfig gains max_output_bytes (default 4 GiB) and max_output_files (default 100,000).
  • Both are checked when the output is collected, before anything is stored in the archive.
  • A run over either limit fails with the new FailureReason.OUTPUT_LIMIT, and its output is removed, the same way unsafe output is handled.
  • While the parser runs, its writes are still bounded only by the host filesystem that holds artifact_dir. docs/containment.md says so and recommends a volume sized for the parsers you run. Enforcing a quota during the run would need a per-backend mechanism, which this PR doesn't add.

Staged inputs were hashed twice (a cost #35 added)

snapshot_staged_input stores every blob with expected_digest, which already verifies it: new bytes are hashed while they're written, and a copy already in the store is re-hashed. It then called put_snapshot, which hashed the whole input again.

It now publishes the record through the archive's private _record_snapshot. The public put_snapshot still re-hashes everything.

Tests

  • tests/test_parsers.py:
    • A stated device, memory, cpus, pids or timeout that differs from what was applied is refused, and nothing is recorded.
    • A condition the backend doesn't report is refused, and one it does report is recorded.
    • FakeBackend now reports its limits like OciBackend.
  • tests/test_run_faults.py:
    • Output over the byte or file limit fails with OUTPUT_LIMIT, is removed, and nothing reaches the archive.
    • Output exactly at the limit is kept.
    • The default limits.
  • tests/test_archive.py:
    • Ingest doesn't run the second full re-hash, while put_snapshot still does.
    • A same-size corrupted copy already in the store is still refused during ingest.
  • The new tests fail without the fixes. Locally: 802 passed, 120 skipped.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QRK7hqAaNLkH2JhWanY5VM


Generated by Claude Code

The three P3s from the review of main at a986152.

- Run conditions: ledger_transaction (and so record_run and
  record_parser_run) compares every stated RunConditions field with the
  limits the backend reported applying (SandboxResult.telemetry.limits,
  from #27): device against "gpu", memory, cpus, pids_limit against
  "pids", timeout_seconds. A mismatch, or a condition the backend does
  not report, raises ProvenanceError before anything is recorded. The
  RunConditions docstring no longer claims callers cannot assert them
  without that check existing.
- Output limits: SandboxConfig.max_output_bytes (default 4 GiB) and
  max_output_files (default 100,000) are checked when the output is
  collected, before anything is stored in the archive. A run over either
  fails with the new FailureReason.OUTPUT_LIMIT and its output is removed,
  like an unsafe artifact. Writes during the run remain bounded only by
  the host filesystem; the docs say so.
- Snapshots: snapshot_staged_input already verifies every blob as it
  stores it (expected_digest hashes new bytes and re-verifies a stored
  copy), so it now publishes the record through the archive's private
  _record_snapshot instead of put_snapshot, which hashed the whole input
  a second time. put_snapshot itself still re-hashes everything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QRK7hqAaNLkH2JhWanY5VM
@Deep-Sixed
Deep-Sixed marked this pull request as ready for review September 25, 2026 22:23
@Deep-Sixed
Deep-Sixed merged commit 999fe3e into main Sep 25, 2026
8 checks passed
@Deep-Sixed
Deep-Sixed deleted the claude/optimistic-mendel-82kq2z branch September 25, 2026 22:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants