Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -501,6 +501,63 @@ Detailed per-release notes are on the
SYN-ACK shows the outbound path was working. Traffic merely received from
the peer does not count as evidence, because a node whose outbound path is
dead still receives its peers' keepalives.
- **A node whose IP address changes recovers in about a second instead of
about half a minute.** When the host's address changed under a running
daemon (new DHCP lease, Wi-Fi to Ethernet, a VPN taking the default route, a
container moved to another address), nothing in the daemon noticed. Peers
kept sending to the old address until their own timeouts moved them to the
relay or the node's 25 s keepalive reached them, the node's own registry
connections stayed bound to the old address until a 30 s read timeout, and
the registry kept handing out the old endpoint for a minute or more. The
daemon now checks once a second which source address the kernel would use
toward the beacon (a route lookup; nothing is sent) and, when it changes,
re-registers with the beacon, sends one authenticated probe straight to
every tunnel peer so each learns the new address from it, and sends the
registry its new endpoint and LAN addresses over a fresh connection. Only
the endpoint is sent: the registry still has the node's visibility,
hostname and trust pairs, so they are not written again (the full restore
still runs if the registry has not answered for 5 minutes or its reply
shows it lost the node). The beacon registration is repeated 31 s later,
because the beacon accepts one endpoint update per node every 30 s and
drops the rest; a move within 30 s of the last keepalive registration
otherwise reached the beacon only with the next one, up to a minute later.
Measured in a three-node Docker lab, moving one node to a new address: peer
to moved node 32 s before, under 1 s after; moved node to peer 30 s before,
no failed send after; a node with no existing tunnel timed out after 30 s
before and connected directly in 0.2 s after. Peers need no update.
- An interface that drops and returns with the same address triggers
nothing, and neither does an unrelated interface appearing (a Docker
bridge, a VPN that does not take the route), nor a switch to another
beacon by the beacon-list refresh, which may be reached over another
route: the comparison starts over with the new beacon (a change still
waiting to be announced at that moment still runs). Recoveries are at
least 10 s apart, and the gap doubles up to 2 minutes while changes keep
coming, so a flapping interface cannot flood the registry; a change seen
during the gap runs when the gap ends, unless the address has gone back.
The gaps are measured on the wall clock, so time asleep counts toward
them.
- Behind NAT the local address does not change when the public one does.
The daemon also reads the address the beacon reports seeing it at (the
reply to its beacon registration, every keepalive interval, 60 s by
default) and runs the same recovery when that IP changes. A reply counts
only if it arrives within 2 s of a registration the node sent, and a new
IP only once two replies in a row agree on it; when one reply differs,
the node registers once more at once to check it. Because a NAT that maps
a node to several public IPs looks like a stream of changes, the gap
between these recoveries starts at 1 minute and grows to 1 hour. This
path has unit tests only; it was not exercised against a real NAT.
- A host waking from sleep on a new network no longer has its two
recoveries (wake and address change) abort each other's registry calls:
registry reconnects and re-registrations, the heartbeat's included, run
one at a time, and the wake or rx-watchdog recovery is skipped when a
full re-registration succeeded in the last 5 s.
- `-no-addr-watch` (config.json `no_addr_watch`) turns the watcher off.
- Publishes `tunnel.addr_changed` (`reason`: `local_address`,
`observed_endpoint` or `registry_retry`; `previous` and `current`: the
local address, or for `observed_endpoint` the IP the beacon sees;
`peers_notified`; `registry_ok`: whether the registry accepted the
re-registration). A relay-only or compat-mode node does not probe peers
directly.
- **Proxy credential hints no longer send an operator who already set
`proxy_cmd` off to set it.** When the daemon re-reads its credentials with
a proxy command and the proxy still rejects them (407, or Meta Muse's
Expand Down
2 changes: 2 additions & 0 deletions cmd/daemon/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ func main() {
beaconRTTProbe := flag.Bool("beacon-rtt-probe", false, "probe beacon RTT before selection; override hash pick when >2× slower than best (ablation test, default off)")
noRxWatchdog := flag.Bool("no-rx-watchdog", false, "disable the inbound-path watchdog that soft-recovers (beacon+registry re-registration) and, on a persistent wedge, exits non-zero for supervisor respawn")
noPathWatch := flag.Bool("no-path-watch", false, "disable the per-peer path watchdog that probes inbound-silent peers and resets a dead peer path in place (prefer-direct sequence) without a daemon restart")
noAddrWatch := flag.Bool("no-addr-watch", false, "disable the own-address watcher that notices this host's IP address changing (or the public IP the beacon sees it at) and re-announces the node to the beacon, the registry and every tunnel peer at once")
// -transport and -proxy have literal defaults: their environment
// variables beat config.json (see flagSources.envOverConfig), and -help
// must never print an environment value — PILOT_PROXY can hold proxy
Expand Down Expand Up @@ -401,6 +402,7 @@ func main() {
BeaconRTTProbe: *beaconRTTProbe,
DisableRxWatchdog: *noRxWatchdog,
DisablePathWatch: *noPathWatch,
DisableAddrWatch: *noAddrWatch,
TransportMode: *transportMode,
CompatBeaconURL: *compatBeacon,
CompatTLSTrust: *tlsTrust,
Expand Down
1 change: 1 addition & 0 deletions cmd/pilotctl/config_values.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ var configValueKinds = map[string]string{
"motd_feed_url": "String",
"motd_interval": "Duration",
"networks": "String",
"no_addr_watch": "Bool",
"no_dataexchange": "Bool",
"no_echo": "Bool",
"no_eventstream": "Bool",
Expand Down
Loading
Loading