Skip to content

fix(cluster): run the join wait from the image's filesystem - #124

Merged
Mearman merged 2 commits into
mainfrom
fix/join-wait-executable
Oct 20, 2026
Merged

Mearman merged 2 commits into
mainfrom
fix/join-wait-executable

Conversation

@Mearman

@Mearman Mearman commented Oct 20, 2026

Copy link
Copy Markdown
Member

Fixes #116, which #117 did not: the wrapper it writes into /run/github-runner-node/bin is never run, because Docker mounts the service's /run tmpfs noexec (tmpfs /run tmpfs rw,nosuid,nodev,noexec in a running node) and k3s looks tailscale up with Go's exec.LookPath, which passes over a file it may not execute. On a fresh three-server cluster on Tailscale's control plane built from v2.5.0, the joining servers kept failing with "no such host" with the wrapper in place.

The wrapper now lives in /usr/local/lib/github-runner-node/bin on the image's filesystem, still only on k3s's PATH. That filesystem outlives a restart, so the entrypoint rewrites the wrapper at every start and removes it when the node joins nothing. After its wait it writes github-runner-node: <host> is a peer and resolves after tailscale up to the container's log (PID 1's standard error, best effort). With this entrypoint on the same failing server, the line appeared in its log and the "no such host" failures stopped.

tests/mesh/run.sh now fails unless every joining server logged that line, so the integration test shows k3s really ran the wrapper in a container, whichever control server it uses; on v2.5.0 it would fail. The unit tests now keep the wrapper's directory clear of every tmpfs and volume either service mounts, and cover the stale wrapper's removal and the log line. Each was broken in turn and a test failed for each.

Docker mounts a service's tmpfs noexec, and k3s finds tailscale with
Go's exec.LookPath, which passes over a file it may not execute, so the
wrapper under /run was never run and a joining node on Tailscale's
control plane still never joined. The wrapper now lives under
/usr/local/lib/github-runner-node/bin, is removed again when the node
joins nothing, and logs a line to the container's log after its wait,
which the mesh integration test now requires of every joining server.

Fixes #116
grep -q stops at the first match, and under pipefail the docker logs it
cut off with a broken pipe failed the check even when the line was there.
@Mearman
Mearman marked this pull request as ready for review October 20, 2026 21:36
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ⚠️ Failed 2026-10-20T21:37:00.961669Z fa6374d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins.

@Mearman
Mearman merged commit 105eb45 into main Oct 20, 2026
30 of 32 checks passed
@Mearman
Mearman deleted the fix/join-wait-executable branch October 20, 2026 21:39
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 2.7.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A joining server never joins on Tailscale: k3s's own tailscale up empties the peer list first

1 participant