Skip to content

feat(operator,chart): pass PageBroker's S3 connection and transfer settings through the chart - #694

Draft
oleg-kushniriov wants to merge 1 commit into
feat/s3-operator-chart-configfrom
feat/s3-pagebroker-tuning
Draft

oleg-kushniriov wants to merge 1 commit into
feat/s3-operator-chart-configfrom
feat/s3-pagebroker-tuning

Conversation

@oleg-kushniriov

@oleg-kushniriov oleg-kushniriov commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does / why we need it

PageBroker's S3 configuration accepts more than the store identity: addressing, caFile, allowHttp and a limits object in the JSON it reads at --storage-config, two environment variables it checks that JSON against (RUNAI_STREAMER_S3_USE_VIRTUAL_ADDRESSING and AWS_CA_BUNDLE), and Model Streamer reader tuning from a second file at --model-streamer-config. The chart rendered none of these, so PageBroker always ran on its defaults, and an endpoint that needs path-style addressing, a private CA or plain HTTP could not be configured through the chart.

This adds the values and routes each one the way PageBroker expects:

  • JSON members. storage.s3.addressing, storage.s3.allowHttp, the CA file path and pageBroker.s3.limits are rendered into pagebroker-s3.yaml next to storage.yaml, kept out of api/storage.S3 because none of them is a store identity input. The operator reads that file through a new --pagebroker-s3-settings flag and merges it into the JSON it already writes, so the file keeps one author. Absent members are omitted, so an install that sets none of them produces the same JSON as before.
  • Environment pairing. The DaemonSet sets RUNAI_STREAMER_S3_USE_VIRTUAL_ADDRESSING from storage.s3.addressing (0 for path, 1 for virtual) and AWS_CA_BUNDLE from the CA mount path, from the same values, since PageBroker refuses to start when they disagree with the JSON.
  • CA bundle. storage.s3.caBundleConfigMapRef and caBundleConfigMapKey are mounted read-only at /etc/snapshot-s3-ca/<key> into PageBroker and the operator. The operator also receives its existing --artifact-cleanup-s3-ca-bundle-path, which the base PR added but did not wire.
  • Model Streamer. pageBroker.modelStreamer is written as given to model-streamer.json in the config ConfigMap, mounted, and passed with --model-streamer-config, only when set.

Two values are derived rather than typed. limits.stagingBytes defaults to the staging volume cap the chart already computes, so PageBroker refuses a staged restore the volume cannot hold instead of filling it; PageBroker's own default is 1 TiB. caFile is the fixed mount path whenever the CA reference is set.

The chart rejects what PageBroker would refuse at startup where it can say so more clearly: a limit key PageBroker does not know, an activeTransactions at or above pageBroker.maxConcurrentRequests, an addressing other than path or virtual, and a plain-http endpoint without allowHttp. PageBroker enforces the remaining bounds. A PVC install renders none of this.

Which issue(s) this PR addresses

Part of #237.

How was this tested?

  • operator: go build, go vet, go test ./..., golangci-lint (0 issues). New tests cover settings loading and validation, and the merged JSON with and without settings.
  • chart: helm unittest (103 cases, 18 new in s3_tuning_test.yaml), helm lint in PVC and S3 mode.
  • make verify-license-headers.
  • helm template in S3 mode with path addressing, a CA bundle, limits and Model Streamer tuning, confirming the rendered file, environment, mounts and flags; and in PVC mode, confirming nothing new is rendered.
  • Not run against a cluster: PageBroker's S3 engine lands independently via refactor(pagebroker): add shared transfer plans and bounded cleanup #264's stack.

Special notes for your reviewer

Base is #663, which writes the identity fields of the JSON; this PR only adds the non-identity members and their environment and mounts.

Naming follows PageBroker's own vocabulary (addressing: path | virtual) rather than SNEP-237's addressingMode: path | virtual-host, and the default when unset is PageBroker's (virtual). SNEP-237 proposes path whenever an endpoint is set, which with the chart requiring an endpoint would make every install path-style; that choice is left for the SNEP to settle and is a one-line change here.

The README rows for the base PR's storage.s3.* values were missing, so they are added alongside the new ones.

Does this PR introduce an API change?

NONE

…ttings through the chart

PageBroker's S3 configuration accepts addressing, caFile, allowHttp and a limits object in the JSON it reads at --storage-config, checks addressing and caFile against RUNAI_STREAMER_S3_USE_VIRTUAL_ADDRESSING and AWS_CA_BUNDLE in its environment, and takes Model Streamer reader tuning from a second file at --model-streamer-config. The chart rendered none of these, so PageBroker always ran on its defaults and an endpoint needing path-style addressing, a private CA or plain HTTP could not be configured.

The chart now renders the connection and transfer settings into pagebroker-s3.yaml next to storage.yaml, from storage.s3.addressing, storage.s3.allowHttp, storage.s3.caBundleConfigMapRef and pageBroker.s3.limits. The operator reads that file through --pagebroker-s3-settings and merges it into the JSON it writes, so the store identity fields keep their one author. The DaemonSet sets the two environment variables from the same values, mounts the CA bundle into PageBroker and the operator, and passes the operator its --artifact-cleanup-s3-ca-bundle-path. pageBroker.modelStreamer is rendered to its own file and passed with --model-streamer-config when set.

stagingBytes defaults to the staging volume cap, so PageBroker refuses a staged restore the volume cannot hold instead of filling it. The chart rejects an unknown limit key, an activeTransactions at or above maxConcurrentRequests, an addressing other than path or virtual, and a plain-http endpoint without allowHttp; PageBroker enforces the remaining bounds. A PVC install renders none of this.

Signed-off-by: Oleg Kushniriov <okushniriov@nvidia.com>
@oleg-kushniriov
oleg-kushniriov added this pull request to stack #691 October 22, 2026 16:54
@oleg-kushniriov oleg-kushniriov changed the title feat/s3 pagebroker tuning feat(operator,chart): pass PageBroker's S3 connection and transfer settings through the chart Oct 22, 2026
@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant