Skip to content

fix: replace a dangling uv symlink instead of failing init-uv - #628

Open
stubbi wants to merge 43 commits into
mainfrom
fix/init-uv-dangling-symlink
Open

stubbi wants to merge 43 commits into
mainfrom
fix/init-uv-dangling-symlink

Conversation

@stubbi

@stubbi stubbi commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

Problem

init-uv mounts the data volume in full at /data (#450), while the gateway sees the same volume at /home/openclaw/.openclaw (root) and /home/openclaw/.local (.local subPath). An absolute ~/.local/bin/uv symlink created from the gateway, such as one that pins a newer uv under ~/.openclaw/tools/, therefore dangles inside init-uv. The old script then:

  1. fails [ -x /data/.local/bin/uv ] because the link does not resolve,
  2. runs cp /usr/local/bin/uv /data/.local/bin/uv, which refuses to write through a dangling symlink,
  3. exits 1 under set -e.

The pod stays in Init:CrashLoopBackOff until someone removes the link from the PVC by hand, and kubectl exec is not available because the gateway never starts.

Change

init-uv now handles ~/.local/bin/uv as follows (script in uvInitScript, internal/resources/statefulset.go):

  • Symlink: the target is mapped from gateway paths back to /data (/home/openclaw/.local/* to /data/.local/*, /home/openclaw/.openclaw/* to /data/*; relative targets resolve against .local/bin, which is the same in both containers).
    • If the mapped target exists, the link is kept. A pinned uv that is valid for the gateway survives restarts and is never overwritten.
    • If it does not exist, the dangling link is removed and replaced with the image's uv.
  • Regular executable file: kept, as before.
  • Missing or not executable: the image's uv is installed.
  • The binary is copied to uv.tmp and then mv -f'd into place, so the script never writes through a symlink. A stale uv.tmp from an interrupted earlier run is removed first.

This follows the reporter's suggested fix in #626, plus the gateway path mapping so a valid pinned link is not discarded.

Testing

  • New unit tests in internal/resources/uv_init_test.go run the real script with sh against a temporary directory: fresh volume, existing binary, dangling absolute link (the init-uv blocks pod start forever when ~/.local/bin/uv is a symlink that dangles under /data #626 case), dangling relative link, valid gateway links under ~/.openclaw and ~/.local (kept, never written through), stale dangling uv.tmp, and idempotency.
  • New e2e spec test/e2e/e2e_init_uv_symlink_test.go runs the operator-rendered init-uv container with its real image (ghcr.io/astral-sh/uv) in a standalone restricted pod whose volume already holds the dangling link, and asserts that it completes and leaves a regular executable uv.
  • Exercised the script by hand with sh against the dangling, replaced, and valid pinned link cases.
  • go test ./internal/resources/ and go vet ./test/e2e/ ./internal/resources/ pass locally.

Note on the required E2E check

The required E2E Tests check is red on main because the Chromium Full Integration suite crash-loops on OpenClaw's sticky /tmp check. #623 fixes that, and this PR's E2E result depends on it.

Fixes #626

🤖 Generated with Claude Code

stubbi and others added 9 commits October 11, 2026 12:51
With spec.storage.fixOwnership unset, the controller reads the instance's
Namespace to decide whether init-data-owner is admissible. When that read
failed (for example RBAC without the namespaces "get" rule), it kept the
init container enabled, which could flip a working StatefulSet in a
restricted namespace to a Pod template admission rejects.

The read failure now aborts reconciliation before any managed resource is
touched, surfaces through the ReconcileFailed event and Ready condition,
and is retried. An explicit fixOwnership still bypasses the lookup.

Adds envtest coverage: a restricted-compatible instance whose namespace
read then fails keeps its StatefulSet unchanged, and explicit values skip
the lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In `watchNamespaces` mode, #618 places `get namespaces` in tenant Roles.
Namespace objects are cluster-scoped, so those grants cannot authorize
the Pod Security label lookup. The controller then retains the ownership
helper default, potentially producing a pod rejected by restricted
admission.

Move Namespace reads into the chart's existing cluster-scoped rule set
and ClusterRoleBinding. In namespaced mode, constrain `resourceNames` to
the configured watched namespaces. Cluster-wide mode permits reading any
Namespace. Neither mode gains Namespace list/watch/write permissions.

This PR targets `fix/data-owner-pss-aware` as a focused follow-up to
#618. It can be retargeted/rebased onto main after that PR merges. The
separate namespace-lookup failure handling review on #618 remains
relevant; this change fixes the chart permission grant.

Validation:
- A regression test renders the chart, applies its actual RBAC to a
disposable envtest API server configured with RBAC, and checks
authorization as the rendered operator ServiceAccount identity.
- The original #618 template fails the watched-Namespace read assertion.
The fixed template permits the two watched namespaces, denies
unwatched/operator namespaces, and denies Namespace list/watch/patch.
Cluster-wide mode is checked too.
- Full `make test`, `make lint`, and both Helm RBAC guard scripts pass.
CI installs Helm for the authorization test.

---------

Co-authored-by: Jannes Stubbemann <jannes.stubbemann@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…en enabled

Addresses review on #617:

- Pin mcp.apps.sandboxPort to 18795 with the gateway proxy disabled too,
  and make the Service, NetworkPolicy, container port and nginx upstream
  follow the effective listener port (a user-supplied port wins). With the
  proxy disabled the Service used to target 18795 while OpenClaw listened
  on its default 18790.
- Only render the sandbox container ports, nginx listener, Service port
  and NetworkPolicy ingress when spec.config.raw enables MCP Apps, so
  instances that do not use it see no change. With configMapRef or
  mergeMode=merge the operator cannot see the effective config and keeps
  exposing the sandbox.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
init-uv mounts the data volume at /data, while the gateway sees it at
/home/openclaw/.openclaw and /home/openclaw/.local. An absolute
~/.local/bin/uv symlink created from the gateway therefore dangles in
init-uv: the -x check fails, cp refuses to write through the dangling
link, and with set -e the pod stays in Init:CrashLoopBackOff until the
link is removed from the volume by hand.

The script now maps the two gateway mount points back to /data before
inspecting a symlink. A link whose target exists is kept, so a pinned uv
survives restarts. A link whose target is gone is removed and replaced.
The binary is copied to a temporary file and renamed over the
destination, so the script never writes through a symlink.

Fixes #626

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@stubbi
stubbi force-pushed the fix/init-uv-dangling-symlink branch from 162dcb9 to 1d7b480 Compare October 11, 2026 16:56
# Conflicts:
#	api/v1alpha1/openclawinstance_types.go
#	bundle/manifests/openclaw.rocks_openclawinstances.yaml
#	charts/openclaw-operator/templates/crds/openclaw.rocks_openclawinstances.yaml
#	config/crd/bases/openclaw.rocks_openclawinstances.yaml
#	docs/api-reference.md
stubbi and others added 13 commits October 11, 2026 13:20
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#619 made every operator init container that writes pod storage use the
gateway's effective UID. Do the same for init-tmp-dir so the sticky /tmp
directory belongs to the user that writes to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Release Please has failed on every push since the repository moved from
paperclipinc to stubbi: vars.RELEASE_BOT_APP_ID and
secrets.RELEASE_BOT_PRIVATE_KEY were settings of the old organization, so
actions/create-github-app-token aborts with "Input required and not
supplied: app-id" before release-please runs.

- release-please.yaml and release.yaml mint the App token only when the
  App is configured, then fall back to RELEASE_PLEASE_TOKEN and finally
  GITHUB_TOKEN. With GITHUB_TOKEN, tag pushes and release publishing do
  not trigger other workflows, so release.yaml and operatorhub.yaml are
  dispatched explicitly (release.yaml gains a workflow_dispatch trigger
  and only runs on tag refs).
- The GHCR namespace for the image and the chart comes from one place,
  vars.IMAGE_NAMESPACE (default paperclipinc), used by release.yaml and
  .goreleaser.yaml. release.yaml previously used the repository owner
  (now stubbi) for signing and the chart while GoReleaser pushed to
  paperclipinc. A preflight fails the release if the namespace does not
  match the chart's default image.repository (#536).
- GHCR login accepts GHCR_PUSH_TOKEN / GHCR_PUSH_USER, needed to push
  into a namespace the repository owner does not own.
- GoReleaser creates the release in stubbi/openclaw-operator.
- make verify-signing accepts both the old and the new workflow identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@stubbi

stubbi commented Oct 11, 2026

Copy link
Copy Markdown
Owner Author

Merge order note (maintenance pass, 2026-10-11). The open stubbi PRs are stacked so they apply cleanly in this order: #623 → #627 → #620 → #618 → #617 → #628 → #629. This is position 6 of 7. This branch has #617 (and everything below it) merged in, so until those land the diff here also shows their changes; once they are squash-merged, "Update branch" reduces it to this PR's own changes without conflicts. #623 is at the bottom because the required E2E check cannot pass without its sticky /tmp fix (OpenClaw >= 2026.9.6 crash-loops otherwise). Branch protection requires an up-to-date branch, so after each merge click "Update branch" on the next PR and let CI re-run before merging it.

@jtatum

jtatum commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

One P2 path-mapping case remains: relative targets are not always equivalent between the gateway and init-uv.

For example, a gateway link:

~/.local/bin/uv -> ../../.openclaw/tools/uv

resolves to /home/openclaw/.openclaw/tools/uv, which is /data/tools/uv inside the init container. The new script instead checks /data/.local/bin/../../.openclaw/tools/uv, resolving to /data/.openclaw/tools/uv.

I reproduced this with the actual init script: despite the pinned executable existing at /data/tools/uv, the script declares the link dangling and replaces it with the image’s uv. This silently discards a valid version pin.

Please resolve relative targets against the gateway’s logical directory layout before mapping them onto /data, and add this cross-mount relative-link case to the regression tests.

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If this PR is still relevant, please push updates or leave a comment.

@github-actions github-actions Bot added the stale label Oct 19, 2026
@jtatum

jtatum commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Not stale

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

init-uv blocks pod start forever when ~/.local/bin/uv is a symlink that dangles under /data

2 participants