[tnunamak]

The Claude Code and Codex connectors store session files lossily. Large fields keep only a bounded prefix plus the true byte length: 500 characters for attachments, a bounded prefix for tool results, and GUARD_PREFIX_BYTES of 64 KB for file reads (connectors/codex/index.ts:135). The bound exists because one tool result can reach hundreds of megabytes, and reading it whole would make collector memory scale with session size. The bytes themselves are never stored, so the durable record cannot reconstruct the session.

This is the first change toward a byte archive that keeps full content without loading it into memory. It adds the contract only: schemas, dormant stream declarations, a compatibility check, and a verifier over supplied synthetic records and bytes. The PR that wires checkRetentionCompatibility in will be the one that enables capture.

No collector code path reaches the new modules. checkRetentionCompatibility and the verifier are called only from their own tests, and RETENTION_STREAMS is declared but never read by a collector, so nothing captures a file, uploads a blob or writes a cursor. retention-runtime.test.ts reads manifests/claude_code.json and manifests/codex.json and asserts neither declares a retention stream, which are the two connectors this contract is written for.

Verify, from packages/polyfill-connectors:

node --test --import tsx --test-concurrency=2 \
  src/retention-schemas.test.ts src/retention-runtime.test.ts src/retention-verifier.test.ts

36 pass, 0 fail. Note that scripts/run-tests.mjs ignores argv and globs a fixed set, so npm test -- <filter> runs the whole suite instead.

Artifact classes are declared per connector

retention-schemas.ts previously hardcoded an enum of fourteen artifact classes, eight of them connector-specific, so adding a connector meant editing a shared schema. b076a2d4 had already settled this question the other way for default stream sets, making each collector-definition.ts the single place its own streams are written. This branch predated that commit and is now reconciled with it.

RETENTION_BASE_ARTIFACT_CLASSES keeps only what means the same thing for every connector: images, documents, unknown, unclassified. Each connector declares its own classes in its own collector-definition.ts, and collector-registry.ts assembles them into CONNECTOR_RETENTION_ARTIFACT_CLASSES beside the existing definitionStreams helper.

Two classes were connector-specific despite unprefixed names. shell_snapshots (connectors/codex/index.ts:234) and the tool-results/*.txt sidecars (connectors/claude_code/index.ts:475) become codex.shell_snapshots and claude.tool_result_sidecars. A test pins the prefix in both directions: an unprefixed class fails, and so does one whose prefix names no registered connector.

The enum stays closed, which is load-bearing. A free-form class string would let an omission entry cite a rule no owner policy could have contained, defeating omission_policy_mismatch. Closing over declared classes keeps that check meaningful while letting each connector own its entries, and the inferred type is still the exact literal union, now assembled rather than written out.

Checked by adding a throwaway connector declaring two classes: both became citable in policy rules and omission entries with retention-schemas.ts unmodified, undeclared classes stayed rejected, and the prefix test caught the connector while it was declared but not yet registered.

RETENTION_STREAMS was flagged in this PR as the same kind of hardcoded table. It is not — its six names are the format's own streams, identical for every connector, so there is no per-connector copy to drift. Left as data with a comment.

Tests that would otherwise pass on broken code

The corrupt-bytes tests all ran the complete_raw path, where the whole-source source_digest_mismatch backstop catches corruption on its own — so disabling the chunk-level blob comparison entirely left the suite green. The added test verifies a complete_with_exclusions capture with omitted_bytes > 0 and no sourceBytes, which is the durable-readback case where that backstop is off, and corrupts a retained blob both by flipping a byte and by changing its length. Replacing the byte-length and SHA-256 comparison in retention-verifier.ts:94 with if (false) now fails with same-length corrupt blob bytes must not be accepted.

required root and chunk keys cannot be dropped or padded with unknown keys now drops every required key rather than a chosen subset. It previously listed 13 of the root schema's 23 required keys and 5 of the chunk schema's 15, omitting the shared provenance block on both sides, so marking classifier_version, connector_instance_id or source_root_id optional left the suite green.

It now derives each key set with Object.keys on the fixture instead of listing names. v1 root, page, chunk and receipt fixtures round-trip without defaults (retention-schemas.test.ts:89) parses each fixture and asserts it equals itself, so no field is defaulted or dropped and a fixture's key set is its schema's required set. The asserted counts, 23 root and 15 chunk, fail if a fixture stops covering its schema. Marking any of those three provenance keys optional in retention-schemas.ts now fails with chunk must require <key>.

Both sabotages were re-run after the taxonomy change and still fail.

Not verified: making blob_ref.sha256 and size_bytes optional without also loosening strictObject is behaviorally inert and no test detects it. The chunk-level .refine already requires those fields to equal their sibling chunk values, and those siblings are required, so validation still fails through the refine. Detecting that relaxation on its own would mean changing the refine, not the field modifier.

Also not caught: disabling only the first of the blob comparison's four clauses, because the other three still reject the corrupt blob. That gap predates this change and is not introduced by it.

Assisted-by: AI

[github-actions]

Schema Health Check — All Clear

All 73 scopes have consistent local schema files.

[tnunamak]

Superseded. Byte retention landed as a content-addressed spool with a four-field record reference (data-connect #106, data-connectors #100), which needs none of this contract. The chunk/coverage/omission model here is only required for owner-directed omission, which is not scheduled; the design reasoning is retained for when sparse archives are specified.

Assisted-by: AI