idea: code deployment fallback for clusters without ImageVolume support (alternatives to RWX PVC)

[slauger]

Context

The recommended code deployment path (OCI image volumes) requires Kubernetes 1.35+, or 1.31+ with the ImageVolume feature gate enabled manually. Most enterprise clusters (especially OpenShift) will not be there for a while. The documented fallback is a PVC, which is explicitly marked "not recommended" and effectively requires RWX storage for multi-node clusters. This means that on most production clusters today there is no recommended code deployment path. This also came up as an audience question at VoxConf.

This ticket is about rethinking the story for clusters without ImageVolume support - ideally keeping the immutability and rollout-visibility properties of the OCI approach.

Options

1. Init container pulling an OCI artifact into an emptyDir (per pod)

   • A generic init container (e.g. crane/oras or a small Go binary) pulls the code image/artifact and unpacks it into an emptyDir shared with the server container.
   • No RWX needed, works on any supported Kubernetes version, reuses imagePullSecret, and the image reference stays part of the pod spec, so rollouts still work via kubectl rollout status.
   • Could be implemented as an automatic operator fallback: use a native image volume when the cluster supports it, otherwise inject the init container transparently (same code[].image API).

2. Runnable code images as init containers

   • Require code images to be self-extracting (e.g. FROM busybox + COPY environments/ /environments/ + cp -a entrypoint) and run them as init containers writing to an emptyDir.
   • Simpler than (1), no extra tooling image, but changes the code image contract (no longer FROM scratch).

3. Per-pod sync sidecar/init container (git-based)

   • Run openvox-code (or git-sync + r10k-style resolution) per pod against an emptyDir.
   • No RWX needed and would also restore the dynamic feature-branch environment workflow, but reintroduces "is the code synced yet" ambiguity and network dependencies at pod start. Related: CLI tooling replacement for r10k/g10k with native OCI support #138.

4. Per-pod PVCs via generic ephemeral volumes

   • RWO storage per pod instead of shared RWX, populated by one of the mechanisms above. Probably only interesting for very large code bases where emptyDir memory/disk pressure is a concern.

Suggested direction

Option (1) as an operator-managed fallback keeps the current API and the immutability story intact and removes both the 1.35 floor and the RWX requirement. Options (1) and (3) are not mutually exclusive - (3) could later become the basis for a feature-branch workflow integration with openvox-code.

Acceptance

• Decision documented (ADR or docs/concepts/code-deployment.md update) on the supported path for clusters without ImageVolume
• If an operator-managed fallback is chosen: implementation issue(s) split out