Skip to content

fix(cluster): wait for the join host after k3s's own tailscale up - #117

Merged
Mearman merged 1 commit into
mainfrom
fix/join-after-vpn-up
Oct 20, 2026
Merged

Mearman merged 1 commit into
mainfrom
fix/join-after-vpn-up

Conversation

@Mearman

@Mearman Mearman commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Fixes #116. A node joining a cluster on Tailscale's own control plane never joined: k3s's StartVPN runs tailscale up --authkey ... --timeout=30s --reset on every start, the backend restarts from a cached network map that held no peers, and k3s looked the join host up before the peers came back, exiting with "no such host" on every restart of the container.

The entrypoint now writes a tailscale wrapper into /run/github-runner-node/bin (on the container's /run tmpfs, so it is written afresh at every start and never sits in front of the entrypoint's own tailscale calls) and starts k3s with that directory first on its PATH. The wrapper runs every command unchanged and, after up, waits until the join host is a peer and MagicDNS answers for it, the same check the entrypoint already makes before k3s starts. A server gets it for K3S_JOIN_SERVER_URL, an agent for K3S_URL, and the bootstrap server, which joins nothing, gets none. Nothing in the Compose file changes, so no running node is recreated.

tests/unit/test_cluster_join_wait.py runs the wrapper against a stand-in tailscale whose ip -4 answers only from its third call, and checks that up passes its arguments through and then waits, that other commands and a failing up are not held, where the entrypoint installs it for servers and agents, and that its directory is on a tmpfs of both services. Each was broken in turn and a test failed for each.

On a throwaway three-server cluster built from v2.4.0 on Tailscale's control plane with the pinned Tailscale, both joining servers restarted in that loop. With a hand-written wrapper of the same kind in /usr/local/sbin, each joined on its next start, the wait after up lasting about a second. The wrapper as merged here, under /run, never runs, because Docker mounts that tmpfs noexec; #124 moves it onto the image's filesystem.

k3s's --vpn-auth start runs tailscale up --reset on every start, which
restarts the Tailscale backend from its cached network map, and dials
the join host as soon as the command returns. On Tailscale's control
plane the cached map can hold no peers, so the host was not yet a peer
or a MagicDNS name, k3s exited with "no such host" and a joining node
never joined.

The entrypoint now writes a tailscale wrapper under the container's
/run tmpfs and starts k3s with it first on PATH: it runs every command
unchanged and, after up, waits until the join host is a peer and
resolves, as the wait before k3s already does. Agents get it too, for
the server they dial.

Fixes #116
@Mearman
Mearman marked this pull request as ready for review October 20, 2026 20:37
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ⚠️ Failed 2026-10-20T20:37:31.228528Z
We couldn’t complete this review. Comment @codex security review to try again.
33209ea Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins.

@Mearman
Mearman merged commit 6fae111 into main Oct 20, 2026
22 checks passed
@Mearman
Mearman deleted the fix/join-after-vpn-up branch October 20, 2026 20:38
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 2.5.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A joining server never joins on Tailscale: k3s's own tailscale up empties the peer list first

1 participant