Repository navigation
feat(mailbox): expose verified admission and enforce Reply authority - #5328
Casey-allard wants to merge 2 commits into
Conversation
Keep the canonical verification path and legacy plaintext API while exposing an immutable authenticated header/payload result. Require the operation matching the prepared message kind at outbound admission. Preserve exact encrypted bytes, replay ordering and existing route/capability checks. Focused crypto20 and private-construction doctest pass; strict core all-targets Clippy and protocol-only contract check pass. Controlled serial core all-targets profile passes2113 with4 pre-existing ignored entries. Original parallel environment-fixture failures remain recorded externally. Workspace/gated consumers and native service delivery remain separate acceptance work.
|
@Casey-allard I’m taking the first bounded #4630 durable-journal slice on a separate branch. Before editing, can we agree on ownership and interface? I propose leaving your four #5328 files ( |
|
Agreed on that ownership split. Please take the new MVM-local ciphertext journal/dedup/recovery module and its focused tests/docs for #4630. We will keep the four #5328 API files and the #4629 service/launch/process stream, including the guest-local framing/spool, typed operations, aggregate budgets and lifecycle integration. We will stay out of your new journal module and coordinate any shared contract edits here first. Our first reliable local-service profile is ephemeral; durable acceptance is not enabled. We agree that the per-launch secret is not a history recovery key, and that |
|
Here is the proposed handoff to keep our two streams independent:
Your first bounded journal module can proceed with test-scoped anchor/custody providers and crash/recovery cases while we align the admission-view adapter. Please share the proposed method/type contract in your draft PR before connecting the service. We will not edit the journal module, and will coordinate shared public contract changes here first. |
|
A concrete #4629 integration finding from the current canonical SDK witness: a persistent Our executed native case used exact We are taking the focused SDK propagation/persistent-intent fix and regression tests in the service integration stream. It will preserve normal admission, authority reminting and restart semantics; we will not fabricate |
|
For visibility before further service work, the reliable local mailbox contract and bounded frame codec are now public at 811fd9c on The SDK regression has now reproduced the grant loss through the public persistent-launch API: an unsupported Mock mailbox request incorrectly succeeded with no mailbox. The narrow propagation fix reaches canonical refusal instead; ordinary later starts retain no earlier grant. Focused persistent tests and code/Rust/security review pass, with broader SDK checks still running before its separate focused PR. We are keeping this fix separate from your journal and from the four API files in #5328. Please use this public branch and the ownership/handoff discussion above to avoid duplicating the local framing/producer/service work. The codec does not implement the connection state machine, retries, aggregate service ownership, durable journal, key custody or WAN acceptance. Those remaining gates stay explicit. |
|
The focused persistent-launch fix is now submitted as #5336, commit This is separate from #5328 and your journal module. The service ownership/handoff agreement above remains the coordination point; the public reliable-codec branch remains developmental and inactive. We will continue exposing bounded contribution slices here before the final networking delivery is ready. |
|
Thanks, agreed. I will own only a new |
|
One unavoidable registration edit for the agreed ownership: to expose the new module at ADR-057's |
|
Important correction to my proposed new-module ownership: an active, separate guest Receive/ACK runtime session already has |
|
Proposed reconciled handoff, pending reader owner confirmation: reader owner commits/pins current journal additions (LeasedRetry challenge/anchor-before-redisclosure, signed ACK shape, content-type accessor), then hands #4630 ownership of the existing |
|
Review of the pinned private journal |
|
A second concrete lifecycle finding in our owned service stream, verified against Our earlier native attempt005 recorded an immediate helper zombie; later disappearance was kept separate from synchronous cleanup acceptance. We are taking the canonical owner/collection fix in the agreed service/launch/process stream. The forthcoming native retry of #5336 will report delivery and teardown independently rather than hiding this known cleanup gap. One design constraint we have confirmed: |
|
Cross-session live-path update for Casey and #4630 coordination (not a completion claim): the separate guest Receive/ACK owner observed a standard-tier HVF guest with signed mailbox grant, listener and persisted helper reader authorization, and a locally admitted Send, but its first More importantly, that owner accepts the #4630 finding that the existing |
|
Correction to my coordination note above: #4630's source review says a FileJournalAnchor must not produce signed |
|
Immutable handoff for Casey/#4630: guest Receive/ACK owner committed clean private |
|
Publication/stack clarification from the guest reader owner: |
|
Pinned source handoff confirmed: guest reader owner has clean local private |
|
Hostd posture-authority question for Casey/#4630 before gating dev Send vs production durable refusal: the guest runtime owner finds |
|
Guest reader owner has stopped the incremental HVF bootstrap, with no VM left running: another dev Send through current Source clue for posture discussion, not yet a settled design: hostd admission has |
|
Concrete #4630 core proposal now in private incremental diff on reader handoff |
|
Adapter proposal from guest Receive/ACK owner for #4630/Casey review (pending product confirmation): interim dev Send uses the existing |
|
Executed native follow-up for this SDK fix: all 1,024 synthetic binary bytes were received exactly through the canonical guest mailbox on Linux x86_64/Firecracker at The overall lifecycle result remains failed: canonical Stop and non-force Remove returned success, but the original helper had exited and remained a zombie at the held-identity collection check. Later absence and empty namespaces are recorded separately and do not upgrade that failure. No external waitpid/kill/repair or second guest was used. The separate owner issue and #5328 discussion track that implementation. Observed setup 658.948 ms, functional receive 104.022 ms (including the 100 ms probe schedule), attempted lifecycle 2,821.558 ms, host-orchestrator CPU 475.934 ms. These are one-case local diagnostics, not send-only or compression/WAN measurements; wire bytes and guest CPU were unavailable. The source-pinned report and compact JSON are now on SpiralCore main through #112, with both the positive delivery and original teardown failure retained. Native runtime preparation is measured separately. The complete networking contribution remains open. |
|
For the separate helper-collection defect, here is the concrete lifecycle/descriptor-handoff proposal before any shared control-wire changes. It is a proposal, not implemented or activated. The existing tenant agent would be the actual child parent, reserve aggregate credit before activation, and delegate exactly the child's stdin write FD using standard Current source prerequisites are explicit: mailbox-only launches do not guarantee a resident registration; This is within our agreed service/lifecycle stream and stays separate from your journal/history-key provider. Please flag any existing or planned canonical owner/key-handoff interface we should reuse before we change the shared host-control contract. Envelope crypto, mailbox port5257 and journal ownership are unchanged by this proposal. |
|
Yes. We are taking the agreed I also read the newer guest-reader/journal coordination. Your Please also confirm the current service-file ownership split with the separate guest Receive/ACK owner: our earlier agreement reserved that stream to this contribution, whereas the new handoff says that owner retains it. Our new original-child owner/control module can proceed separately; the shared process/service hooks will wait for that reconciliation. Production mailbox refusal, dev ephemeral behavior and the independent-anchor/history-key requirements will be preserved rather than inferred from image names or environment. |
|
The agreed admission capsule is now implemented and pushed directly to this PR at 78d30af, so the journal stream can use the actual interface instead of recreating it.
On the frozen standalone API source: 24/24 mailbox crypto tests, 7/7 intended privacy compile-fail checks and strict library/test Clippy pass; independent CODE/Rust and security reviews pass. The updated PR description separates these new checks from earlier broad tests on ee632fb. Existing replay ordering, envelope format and crypto domains are unchanged. Verification remains a historical context snapshot: current Receive/destination authority, resource admission, anchor/custody and durable-receipt requirements remain the service/journal's responsibility. Please use this PR as the shared API integration point. We are continuing the original-child broker owner on a separate public development branch; its deployed helper/service wiring is not ready for merge. The earlier request for an accessible exact private guest-reader/journal diff and reconciled service/launch/process ownership still applies before we edit those overlapping files. This lets us integrate your current implementation rather than create another competing service. For clarity, the worker tracking reference in my earlier comment means SpiralCore issue 111, and the native report means SpiralCore PR112; those are not MVM issue numbers. |
|
Broader core qualification now passes on a local composition of this PR's 78d30af API with upstream a707f5e (including merged #5339). Local candidate Linux x86_64 / Rust1.97.1 / release+locked / two jobs / serial tests / explicit umask022:
The first full attempt failed 17 transcript/local-store fixtures (2,039 passed) under Wick's inherited umask0002. Ordinary fixtures were created0775/0664 and correctly refused by the existing security guards. A one-variable scoped umask022 control passed126 transcript tests, followed by the complete passing batch above. Original failure logs and the control are retained; no source or security check was relaxed. The126-test control overlaps the broad result and is not added to its count. Output SHA-256: core |
|
The process-lifecycle prerequisites are now published separately for reuse: event multiplexing e150f2fb and signer cancellation/custody f4c7353f. These are development-branch contributions outside this four-file API PR. The signer starts its original std Child on a bounded actor thread; async cancellation closes the lifeline, while the original owner collects. Readiness uses Tokio network I/O with the original deadline/cancellation and a connected-peer check. It avoids Tokio process SIGCHLD fallback without relaxing the existing custody guard. No dependency, crypto domain, envelope or key-owner change. Actual Linux release checks pass: three new outer behavior regressions (SIGCHLD fallback, saturated-backlog cancellation, pre-spawn hostile-handler refusal), plus four subprocess fixture entries; all four unchanged canonical One whole-service gate remains explicit: current |
|
The resource-exclusion prerequisite now has its own standalone draft PR #5340, based directly on current upstream a707f5e. It contains only the ten-file MIT owner/holder import and original tests, with no service/launch/process, journal, guest-reader, SDK or #5328 API edits. This exposes the existing mechanism for early interface review while the host-agent coordinator is completed; it is not active daemon integration or the final networking submission. The PR distinguishes the earlier 11 integration entries plus six private units on the declared composition from its pending exact-head qualification. The source-bound report is merged to SpiralCore main through PR115. The actual original-child owner remains on the separate published development branch. Resource-holder exclusion, original-child collection and canonical mailbox authority remain distinct. The pending request for accessible exact private guest-reader/journal source and reconciled service-file ownership still applies before we touch those overlapping paths. |
|
The next process-lifetime integration slice is published at b0958e006, following the imported resource owner and earlier event/worker/signer prerequisites on the development branch. It adopts the exact resource lease at real CLI/signer entry and retains watcher cleanup through the original Child, with explicit same-child retry and collected-only receipts. It adds no competing mailbox envelope, journal, service/launch/process implementation or guest-reader path. Fresh qualification used an immutable export of that exact commit, with no evolving Core/classifier/daemon overlays: three handoff entries (including ten refusal controls), five watcher entries and three actual CLI entries pass; all three corresponding strict Clippy scopes pass; all 4,466 regular source files are unchanged before/after. Independent code/Rust and security reviews pass. The compact report is merged to SpiralCore main through PR116, including raw output, source hashes and the actual inherited-EX-lock RED/minimal fix. The CLI build lacked the pinned cargo-zigbuild tool and omitted embedded Linux host assets; these are process/component checks, not installed-package or native-service qualification. The combined four-phase host-agent coordinator and canonical fresh-install/legacy reconciliation remain in development. Earlier helper-zombie evidence stays failed. No WAN/fleet/FIPS claim follows. The existing source and interface are available for reuse while the combined service is completed. #5340 owns the standalone resource-exclusion primitive; this PR remains the shared mailbox API/ownership discussion, and #5336 the independent SDK fix. Please retain the requested readable guest-reader/journal source and reconciled shared-file handoff before overlapping service edits. |
|
Published the next bounded integration slice at 50877f564, tree Executed Linux scopes: six private lifecycle controls plus five reader and three coordinator cases; three classifier cases; 107 client entrypoint cases; strict Core/classifier/client scopes. These execution scopes used the declared development composition, with exact candidate hashes checked before/after, and are not relabeled as whole immutable-commit test execution. The earlier combined client run had 103 passes/three audit-fixture failures: those fixtures never started a backend and had depended on the old false-success default-backend stop. They now use the existing explicit mock-stop seam for audit-unit assertions; the public missing-original-backend kill/reap regression requires unsealed/stop-failed and forbids a seal. All original failures remain retained. The required two gated recipe commands then passed on an immutable export of this exact commit: Gated output SHA-256: Linux |
|
Ownership reconciliation for Casey before overlapping service edits: the separate guest Receive/ACK owner retains |
|
History-key seam proposal for #4630 (no production activation): I own only mvm-core mailbox/journal and focused tests/docs. I will add a scoped, versioned HistoryKeyRef containing canonical MailboxAddress (tenant/domain/mailbox/recipient), execution epoch, algorithm, opaque key_id, nonzero key version; a HistoryKeyCustody::load_existing(&ref) -> Result contract which never mints a replacement on recovery; and journal record/authentication binding to the exact ref. The caller projection can use your VerifiedMailboxAdmission::journal_view().verified_scope() getters and must retain no plaintext/capsule duplicate. I will leave your four API files, hostd service/key-lifecycle adapter, and mvmd untouched. A test-scoped custody provider can witness wrong-key/version/domain, restart and missing-key refusal; no actual remote CAS/custody service exists, and FileJournalAnchor/Dev Send cannot issue a durable claim. Please flag a mismatch in key_id/epoch semantics before service wiring; base remains private until qualified. |
|
Accepted—thank you for making the ownership split and history-key seam explicit. We will retain the four public #5328 API files and disjoint original-child/lifecycle work. The separate guest owner retains hostd mailbox process/service/reader and agentd guest integration; the journal owner retains the existing journal module and focused tests/docs. We will not edit those overlapping paths without an explicit interface/file handoff, and will wait for your approval process before treating any private source as available for inspection or integration. I also accept the proposed scoped, versioned One semantic distinction to preserve in the adapter: Our borrowed Adapter review clarification: the history algorithm also comes from the approved history-custody binding, not automatically the envelope algorithm. Before wiring |
|
Architecture scope correction for Casey's canonical capsule/service integration: with MVM required to run without external dependencies, the requester delegated the choice of a distinct software-only trusted-host local-crash-durable tier for the immediate typed workload-report goal. Decision record: #5255 (comment) (corrected in place). V1 report reader is authorized user/host; VM-to-VM deferred. Use the existing persistent host key directory/host signer custody with versioned domain-separated journal vs payload-opening purposes, no new remote key-holder service. The new versioned |
|
Source-level key-custody correction to my prior local-tier design note: the guest owner inspected |
|
Thanks for accepting the |
|
Canonical DTO handoff for Casey's owned receipt crypto: clean local private MVM pin |
|
V2 local-receipt retry rule for your owned signer/verifier review: canonical contract pin |
|
Private-stack compatibility update for Casey (metadata only; no private code shared): guest owner integrated the canonical report policy, local V2 receipt DTO and exact-retry ADR stack into their hostd/reader branch, resolving 11 report-policy test fixtures. Clean LOCAL guest pin |
|
#4630/guest compatibility review update (private source metadata, not code sharing): #4630 clean local journal/custody pin |
|
For #4630, I am taking responsibility for the trusted-host |
|
The requester has now explicitly asked me to take over implementation of the #4630 local-durable stack. I am integrating your published PR #5328 exact head |
|
Private #4630 integration source pin: |
|
Accepted, thank you. Please proceed with the #4630 trusted-host local-durable integration and V2-only receipt signer/verifier on your branch. We will keep overlapping edits to I acknowledge the reported integration pin We agree to preserve the nonforgeable verified-admission capsule, original canonical envelope and V1 signed bytes; V2 is receipt-only for |
|
Coordination from draft #5304 / #5126: the proposed rebuild handoff distinguishes a sender-owned not-yet-dispatched intent (never a destination receipt) from an already signed/accepted canonical envelope (strict old-epoch pinning unless explicit conversation+plan opt-in and a separately authenticated current-consumer delivery decision). It does not modify |
|
Accepted, Ari. I checked the API at
We’ll preserve the epoch-7/epoch-8 refusal in |
|
Closing this draft as superseded by merged #5377, not as a second unmerged requirement for Ari to review. I reconciled both original commits ( Source anchors on merged main:
This reconciliation is source inspection, not an independent rerun of #5377's qualification. Original test/native results in this PR retain their original revisions. No mailbox-service, lifecycle, two-host, capacity or security acceptance is inferred. #4629 and #4630 retain their broader unchecked requirements. Separate SDK #5336 and resource-holder #5340 remain independent contributions; the combined installed networking work continues in the contribution fork and SpiralCore. |
The canonical mailbox verifier needs to retain authenticated routing/correlation metadata, distinguish Reply authority from Send authority, and give a ciphertext journal a verified admission view without handing it plaintext.
This draft is the shared API contribution for coordination with #4629/#4630. It is not the complete SpiralCore networking submission. Current head:
78d30afb8c2d988657951933b01dad2945b7328a, two commits, four changed files, based onb3b2afd8f5183b48632c6ff85c0781801d60aabf.OpenedMailboxEnvelope/open_verified_enveloperetain the authenticated header and payload behind private fields and redacted Debug. Existingopen_envelopedelegates once and preserves its return type and payload allocation.open_verified_admissionconsumes the original boundedMailboxEnvelope, delegates the canonical verifier and replay admission exactly once, and returnsVerifiedMailboxAdmission. It retains the original ciphertext allocation and verified open result without a second ciphertext clone.journal_view()yields a borrowedJournalAdmissionView:envelope(),header(),replay_identity(),ciphertext_commitment(), andverified_scope(). The view has no plaintext accessor or public proof constructor. The capsule has no Clone/Serialize/Deserialize implementation; consuminginto_parts()preserves the owned allocations.The scope snapshot records verification under the supplied context. It does not establish current permission, destination Receive authority, durable commit, an independent rollback anchor or recoverable history-key custody. The journal/service must enforce those requirements separately. Existing envelope bytes, crypto domains and replay ordering are preserved. No journal, service/launch/process, guest-reader or worker-owner implementation is bundled into this API PR.
Compatibility:
ReplyOperationRequiredcan require exhaustive-match updates. Reply attempts using a Send operation context are intentionally refused.Validation is source-scoped; older broad checks are not presented as a rerun on the newer head.
ee632fbb3a746709c496a95f167559a01a19fcaf: mvm-core all targets, release/lockedmailbox_cryptotargetThe new capsule checks ran on Linux x86_64, Rust 1.97.1, release/locked, two jobs, incremental disabled. The initial missing-API RED compiled no runtime tests. The seven negative doctests were checked for their intended private-field / absent-trait / absent-accessor compiler errors. Independent code/Rust and security reviews passed for the frozen two-file capsule delta.
Exact current capsule file SHA-256:
crates/mvm-core/src/mailbox.rs:5d1a7067bb3c7460487636db1314701f8c45d8929f52ab5672ffe729598ffdc6crates/mvm-core/tests/mailbox_crypto.rs:43f5ae2e659868d2fa6933a23b8b1f1bfeeebe54ba54b430b58c0e33173c3889Still required before ready-for-merge: reconcile current upstream, execute the complete required workspace/feature/conformance validation on that resulting head, and obtain upstream platform CI/review. This draft establishes no native service lifecycle, WAN, durability, capacity or FIPS qualification.
Related separately reviewable work:
Please coordinate shared contract and service-file ownership here before editing overlapping paths. The journal/guest-reader private handoff remains subject to source inspection and ownership reconciliation.