daemon: a dial timeout while another dial succeeds is not a transport wedge

[TeoSlayer]

Problem

The rx watchdog's outbound-dial-wedged check counted every dial timeout. A burst of parallel sends to one peer whose SYN limiter dropped some of them (dial_timeouts=32 in the field) read as a wedged transport: a healthy node re-registered with the beacon and registry, and the watchdog could escalate to a restart.

Fix

A dial timeout is not counted when another dial completed while it ran. That dial's SYN reached its peer and its SYN-ACK came back, so this node's outbound path worked during the timed-out dial; the timeout says something about that peer, not about this node's transport.

• dialOKSeq counts completed dials. A dial reads it when it starts and compares on timeout. It is a counter rather than a timestamp, so a wall-clock step cannot hide timeouts.
• Traffic merely received from the peer is not evidence: a node whose outbound path is dead still receives its peers' keepalives. An earlier revision of this PR used that and was replaced.
• A dial completed before a timed-out dial started does not exempt it: only dials overlapping the timeout count as evidence.

Trade-off

A success over a path that does not share the broken part (a LAN or same-host peer) now exempts the WAN timeouts that overlap it. If such successes repeat more often than once per dial lifetime (~32s), a WAN-only wedge is not detected by this check. Main would still trip on dense timeouts between successes. Detection of a wedge where no dial succeeds is unchanged, and the restart-loop guard is untouched.

Tests

• TestRxWatchdogDialTimeoutsWhileAnotherDialSucceedsAreNotAWedge: four timeouts to a silent peer while a real dial to a second peer completes (its SYN answered with a SYN-ACK). Fails on main (counter = 4).
• TestRxWatchdogDialSuccessBeforeTheBurstDoesNotHideIt: a dial completed before the burst does not hide it. Fails against "any success ever hides timeouts".
• TestRxWatchdogPeerKeepalivesDoNotHideAWedge: keepalives from the dialled peer do not hide a wedge.
• TestRxWatchdogDialTimeoutsToSilentPeerStillCount: the 2026-07-15 partial-wedge detection still fires.

Each test takes ~32s (the full dial retry budget). They run in parallel and are skipped under -short.

Not covered

Retries that time out one after another against a single dead peer still count, as before. This fix only helps when some dial succeeds at the same time.

🤖 Generated with Claude Code

[TeoSlayer]

One trade-off to weigh in review, found after opening this:

The rule can delay the detector, not just quiet it. Peers with an established session send keepalives every 25s, and a keepalive counts as "heard from". If our outbound path is dead but a dialled peer's keepalives still arrive, roughly two thirds of the ~17s dials to it go uncounted. Uncounted timeouts do not reset the counter, so the detector still fires — after about 12 failed dials instead of 4. Whether the 2026-07-15 incident (03c1189) had keepalives arriving from the dialled peers is not known; TestRxWatchdogPartialWedgeDialTimeouts still passes.

A stricter alternative is a few lines: skip a timeout only when another dial to the same peer completed after this one started. It is immune to keepalives, but would not cover a live peer that silently drops our SYNs for another reason.

Tests added here: TestRxWatchdogDialTimeoutsToAnsweringPeerAreNotAWedge and TestRxWatchdogDialTimeoutsToSilentPeerStillCount (both ~32s, skipped under -short). The PR description names an earlier test name; these two replaced it.

[TeoSlayer]

Converted to draft after review. The change fixes the false positive it targets, but it opens a false negative: the evidence that a peer is "answering" (daemon.go around line 4134) includes the peer's unsolicited 25 s keepalive. A node whose outbound path is dead still receives those, so its dial timeouts are no longer counted. With every client frame dropped, this branch reports counter 0 and "progress", where main reaches 4 and soft-recovers. With a beacon (17 s dial budget) detection is slower; without one (32 s) it never fires, so the node stays wedged with no supervisor respawn.

Counting only frames that answer something this node sent since the dial began (rather than any decrypted frame) would keep the fix without losing the outbound-dead case.