Skip to content

Make the Home heartbeat failover tests deterministic - #4

Merged
rune-oai merged 2 commits into
mainfrom
update/heartbeat-test
Nov 16, 2026
Merged

rune-oai merged 2 commits into
mainfrom
update/heartbeat-test

Conversation

@rune-oai

Copy link
Copy Markdown
Owner

CI went red on PR #3 because TestServiceHeartbeatFailoverDoesNotDrainBlockingScope failed on one of two runs of the same commit. It is an upstream CLIProxyAPI test, and its sibling tests share the same flaw: they wait one second of real time for a Home re-subscription that takes a 100 ms heartbeat expiry plus a 500–1000 ms retry backoff, so a busy CI machine can miss the deadline.

The service gets a retry-wait hook, and the six affected tests advance retries explicitly and wait on the events themselves rather than a timer. Production behavior is unchanged: the hook defaults to the existing backoff wait.

Reproduced under CPU pressure before the fix (failed within the first few iterations). After the fix: 3,000 runs with GOMAXPROCS=1, 900 race-detector runs across 1/2/8 CPUs and ten fresh-process package runs passed, all under CPU pressure. One separate pre-existing issue remains: an auth-file replacement test fails when the whole package runs ten times in one process (-count=10), on the original code too. CI runs each test once, so it does not affect CI.

Created with Claude Opus 5.5 in Claude Code, with a GPT-6.1 Sol subagent.

🤖 Generated with Claude Code

rune-oai and others added 2 commits November 15, 2026 22:37
These upstream tests waited one second of wall-clock time for a Home
re-subscription that goes through a 500-1000 ms retry backoff after a 100 ms
heartbeat expiry, so they failed on busy CI machines. The service gets a
retry-wait hook; the tests now advance retries explicitly and synchronise on the
events themselves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Wait for the hub to invalidate and close the half-closed upstream before sending a continuation. Check the correlated 409 directly to catch unsolicited idle-loss frames, reuse the checked barrier in queued-input tests, and cover an in-flight continuation losing its outcome at EOF.
@rune-oai
rune-oai merged commit fcb56c2 into main Nov 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant