Skip to content

feat(pagebroker): implement the PVC artifact-storage backend - #496

Merged
oleg-kushniriov merged 0 commit into
test/pvc-pagebroker-lifecyclefrom
feat/pvc-storage-backend
Oct 8, 2026
Merged

oleg-kushniriov merged 0 commit into
test/pvc-pagebroker-lifecyclefrom
feat/pvc-storage-backend

Conversation

@oleg-kushniriov

@oleg-kushniriov oleg-kushniriov commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Superseded by #508 for review. The backend branch was moved below #493 in the stack. Pushing that reorder before updating PR bases caused GitHub to mark this PR merged into the lifecycle branch; it did not merge the backend into main. #508 preserves the same backend branch and reviews it against #492.

What this PR does / why we need it

Implements the PageBroker C++ artifact-storage backend (plan item 04-C) — the last piece needed before the artifact-addressed PVC flow (#474, #475, #492, #493) can run end-to-end. Until now ValidateStagedCheckpoint/ValidateStagedRestore/kGetArtifactMetadata all threw/failed "not implemented"; this PR replaces those with real dispatch.

  • identity.cpp: C++ ComputeStoreID/ComputeCommitID with a self-contained SHA-256 (no new system dependency), matching api/storage/identity.go's length-prefixed preimage encoding exactly. identity_test.cpp checks it against the same shared vectors the Go side uses (api/storage/testdata/identity.json, already copied into testdata/storage-contract/).
  • artifact_store.cpp: PVCArtifactStore publishes checkpoints into the existing artifacts/<contentUID>/containers/<name>/ layout — not a new on-disk scheme, so #474's maintenance and legacy restore are unchanged. Writes a publication.json evidence sidecar (store ID, artifact handle, deterministic commitID) next to manifest.yaml. Resolves and verifies that evidence for restore/metadata fetch, mapping failures to STORE_MISMATCH, ARTIFACT_NOT_FOUND, ARTIFACT_CORRUPT, UNSUPPORTED_ARTIFACT instead of collapsing everything into STORAGE_ERROR.
  • broker.cpp/.hpp, checkpoint_transaction_descriptor.*: wires artifact dispatch into PrepareCheckpoint/Restore/Commit, implements GetArtifactMetadata.
  • main.cpp/daemon.*: parses the sidecar's --storage-config argument (already added by #492) and resolves the configured store at startup. storage_config.cpp is a minimal parser for exactly that fixed YAML shape, not a general YAML dependency added for four string fields.
  • docs/operations/storage.md: updated to describe real activation instead of "not yet enabled," and corrected the transaction-lifetime language to match #474's fixed-constant decision (PageBroker's 2h05m lifetime is a constant now, not configuration).

Which issue(s) this PR addresses

Part of #237.

How was this tested?

No C++ toolchain is available in the environment this was developed in for the real Linux build, but a local protoc/clang++/brew-installed protobuf/abseil/googletest were enough to genuinely build and run the full test suite (not just syntax-check):

  • 52/52 tests pass: all pre-existing BrokerTest/StorageContract tests unmodified and still green, plus new Identity (exact match against the Go vectors), ArtifactStoreTest (publish/resolve/stage round-trips, atomic-replace, symlink/corruption/mismatch rejection), StorageConfig, and ArtifactBrokerTest (full checkpoint→commit→restore and metadata-fetch flows through HandleRequest).
  • Caught and fixed a real bug this way: the first version of PublishCheckpoint used fs::rename from tmpfs staging into the PVC, which fails with EXDEV in production since they're different filesystems — fixed to copy, matching how the legacy PosixCopyEngine already does it.
  • One pre-existing test needed updating (now MetadataFailsInvalidRequestWithoutCreatingTransactionWhenNoStoreIsConfigured) since it asserted the old hardcoded "not implemented" message.
  • An apparent batch of 18 test failures during development turned out to be a macOS-only /var → /private/var symlink quirk in fs::temp_directory_path(), reproduced identically against the untouched pristine code — confirmed not a regression and irrelevant to the real Linux CI build.

Special notes for your reviewer

Base is test/pvc-pagebroker-lifecycle, not #475/feat/pvc-checkpoint as the implementation plan originally decided (2026-10-06): #492/#493 were opened directly on #475 ahead of this PR, and #492 already added the sidecar's --storage-config chart argument that was this PR's own scoped startup-configuration task. Basing here instead avoids reproducing and conflicting with that hunk; see the plan doc's 2026-10-07 update for the full reasoning.

Deliberately not in this PR:

  • The reconciler-side "detect a confirmed-but-unrecorded publication and enqueue recover-metadata" trigger (#474's RepairPublication is wired to a real work-item mode as of its 2026-10-07 fix, but nothing yet calls it — needs a product decision about the detection signal, not just this backend).
  • Model Streamer engine adaptation (#265) and anything S3 — out of scope per the plan.

Does this PR introduce an API change?

PageBroker now serves artifact-addressed checkpoint, restore and metadata-retrieval requests against a configured PVC store, instead of rejecting them as not implemented. No user-facing API or deployment change — activation depends on the chart's existing --storage-config argument, already present since #492.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@oleg-kushniriov
oleg-kushniriov added this pull request to stack #473 October 7, 2026 14:15
@oleg-kushniriov
oleg-kushniriov force-pushed the feat/pvc-storage-backend branch 2 times, most recently from 41502de to f9f61b6 Compare October 8, 2026 09:15
@oleg-kushniriov
oleg-kushniriov force-pushed the feat/pvc-storage-backend branch from f9f61b6 to 9e995c2 Compare October 8, 2026 09:44
@oleg-kushniriov
oleg-kushniriov force-pushed the feat/pvc-storage-backend branch from 9e995c2 to 72efcfe Compare October 8, 2026 11:00
@oleg-kushniriov
oleg-kushniriov merged commit 72efcfe into main Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant