feat(provenance): local session logs, C2PA manifests, pod-side slice hashing (P1)#305
Draft
hthillman wants to merge 6 commits into
Draft
feat(provenance): local session logs, C2PA manifests, pod-side slice hashing (P1)#305hthillman wants to merge 6 commits into
hthillman wants to merge 6 commits into
Conversation
…mbedding, local keys Session/event JSONL logs with hash-chain checkpoints tapping the event bus; c2pa-python manifest embedding in write_track_wav/write_stem_wavs (compositeWithTrainedAlgorithmicMedia, CAWG do-not-train default, com.daydream.session assertion); local self-signed ES256 mini-CA; ledger client (no-op until DEMON_LEDGER_URL set). Import-guarded: degrades cleanly when the provenance extra is not installed. Spec: demon-provenance 02-architecture §3/§5/§7, 06-cloud-api §2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The client POSTed to a nonexistent /slices endpoint, sent a single flat
{event:...} body with no Authorization header, and parsed receipts with
the wrong field names. Rewrite it to the shipped server contract
(pipelines-provenance provenance.{routes,schema,service}.ts):
- POST {ledgerBaseUrl}/sessions/{id}/events with a batched
{events:[{seq,type,ts,ppq?,payload}]} body; per-stream seq is
contiguous from 0.
- Slice hashes are slice.pod_hash EVENTS carrying
{start_sample,sha256,channels,num_samples} (+ slice_seq), not a
separate endpoint.
- Authorization: Bearer <pod_ledger_token> from DEMON_LEDGER_TOKEN
and/or a param; the whole client no-ops unless BOTH the URL and token
are set.
- Parse {stream,chain_head,receipts:[{seq,chain_head,ts,receipt_kid,sig}]}
with the correct field names (sig, receipt_kid, receipts[]).
seq is assigned by the worker at flush time, so best-effort queue drops
never punch a seq-gap hole. Batched, fail-open, bounded queue, matching
the module's existing async/loguru style.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… hash
FIX-C: the local JSONL used a bespoke {schema,ts(ISO),session_id,event,
...flat} shape. Rewrite it to the shared wire schema (06 §2.2) so local
history replays into the ledger unchanged: {stream:"local", seq, type,
ts(epoch ms), ppq?, payload} envelopes with namespaced type names
(session.config, action.prompt, action.param, action.lora,
action.seed_audio, action.transport, session.note) matching what
ledger_client sends. Timestamps are epoch ms (06 §0: ISO never on the
wire); the file stays human-inspectable.
FIX-B (pod side): _on_slice hashed float32 fully-reconstructed audio in
a rolling chain and reported it as a slice hash — but the client hashes
the uncompressed interleaved float16 downlink bytes, so the two could
never cross-check (audit F3). Remove that masquerading chain. The tap
now only counts decoded slices; genuine slice.pod_hash reports come from
the transport codec (where the float16 bytes exist) via the new
record_pod_slice_hash, emitted through this session's ledger client so
they share the pod stream's seq.
FIX-E: add session_summary_for(session_id) and demote
latest_session_summary so manifests bind to a specific session, not the
newest tap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FIX-B (transport side): the §2.3 slice hash must be SHA-256 over the exact uncompressed interleaved float16 delta payload bytes the pod sends downstream, so the pod and rtmg-vst client hash the same bytes and can cross-check. Those bytes only exist in the per-subscriber transport codec (the bus carries float32), so hash there: - SliceCodec.encode now hashes delta.tobytes() before zstd compression, tracks a monotonic per-session slice_seq (06 §3), and exposes the report as last_slice_hash. - The WS adapter forwards it via session_log.record_pod_slice_hash after a successful send — fully guarded and fail-open so provenance never wedges the downlink (06 §7). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
FIX-D: write_track_wav / write_stem_wavs are called only from the user-upload path — on the user's uploaded source track and separated stems, i.e. human-origin dry input BEFORE any generation. They embedded digitalSourceType compositeWithTrainedAlgorithmicMedia, an affirmatively false synthetic claim (02-architecture §4, audit G5). Uploads now assert digitalCapture with no AI/seed claim; the CAWG do-not-train default still rides along to protect the user's content. The composite/trained path in build_manifest_definition is preserved for a genuine generated-output writer. FIX-E: embed_wav_manifest defaulted the session assertion to latest_session_summary() — under concurrent sessions a WAV for session A could be signed with session B's id (audit F7/G5). It now takes an explicit session_id (resolved via session_summary_for) and emits null session fields when none is given, rather than borrowing the newest tap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New acestep/provenance/receipts.py — the full §2.4 verification
procedure for the ledger's per-slice signed receipts:
- RFC 8785 (JCS) canonicalization byte-matching the server's jcs.ts,
including the ECMAScript JSON.stringify number notation (integral
floats drop ".0", -0 → "0", fixed notation for decimal exponents in
(-7, 21], e± outside; ints ≥ 2^53 are rejected because a JS peer
would hash a rounded value). Fuzz-checked against the server's
actual jcs.ts (2000 nested values) and Node JSON.stringify (7000+
doubles): zero mismatches.
- ChainMirror: local replica of the §2.1 chain rules (genesis
domain-separation head, event_hash = SHA-256(JCS(event)),
head_n = SHA-256(head_{n-1} || event_hash_n)) tracking the head
after every submitted seq — step 5 checks receipts against OUR
history, not just their signatures.
- ReceiptVerifier: fetches + caches GET {base}/keys, validates the
key-cert chain (sealing pubkey → ddp-keycert:v1 cert_sig → receipt
key), checks receipt ts against key validity, verifies the Ed25519
sig over the exact ddp-receipt:v1 framing, and requires chain_head
equality with the mirror.
LedgerClient verifies receipts as they arrive: fail-open everywhere
(one WARNING, receipts_verified / receipts_unverified counters and
last_verification_failure exposed on the client; a payload the mirror
cannot canonicalize degrades verification, never reporting).
Tests port the server's golden chain vector verbatim from
pipelines-provenance test/provenance/chain.test.ts — genesis, event
hash and all three heads byte-match the server implementation — plus
tamper coverage (sig, chain_head, ts, key-cert, unknown kid) and
end-to-end stub-ledger runs (verified, forged-head, bad-sig,
keys-endpoint-down).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collaborator
Author
|
Added |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Local provenance: session logs, C2PA manifests, pod-side slice hashing (P1)
Engine half of the provenance system: every DEMON session produces a schema-stable local event log, file writers embed C2PA manifests, and pods report hash-chained events + per-slice hashes to the provenance ledger.
What's here
acestep/provenance/: §2.2-schema JSONL session logs (epoch-ms, namespaced event types,stream:"local"); local ES256 mini-CA ("DEMON (local, self-signed)"); c2pa-python manifest embedding inwrite_track_wav/write_stem_wavs; ledger client speaking the real wire contract (batched seq-contiguous envelopes, Bearer auth,slice.pod_hashevents, receipt parsing). Optional extra:provenance(c2pa-python>=0.36). Fully no-op when the extra/env is absent.SliceCodec.encode— the same bytes the plugin hashes on receipt, making the pod/client dual-hash cross-check real.digitalCapture— nevercompositeWithTrainedAlgorithmicMedia. Do-not-train assertion on by default. Manifests bind to an explicit session id (no "latest session" races).Testing
17 tests green:
uv run --no-sync python -m pytest tests/unit/test_provenance_manifest.py tests/unit/test_provenance_session_log.py --noconftest -q— includes a real c2pa round-trip (write WAV → read manifest → assert digitalSourceType, do-not-train, com.daydream.session) and HTTP-server-backed ledger client tests. Degrade path verified with c2pa absent.Cross-repo
Spec:
demon-provenancedocs. Server: https://github.com/livepeer/pipelines/pull/2709 · Plugin: https://github.com/daydreamlive/rtmg-vst/pull/162🤖 Generated with Claude Code