Repository navigation
test(core): de-flake the scheduler timing tests and the real-Docker container tests - #494
Merged
Merged
Conversation
Three scheduler tests waited a fixed wait(300)/wait(100)/wait(50) and then asserted the scheduler had already checked, triggered, run and persisted state.yaml. On a loaded CI runner the guess is sometimes short: all three failed on the #492 version PR (both attempts) and on e852a99, and scheduler.test.ts failures recur in older CI runs too, while the behaviour under test was fine. Replace those sleeps with waitUntil(), which polls the same condition up to a 5s deadline. The deadline uses performance.now(): two of these tests freeze Date.now to pin the cron clock, so a Date.now deadline would never expire. Assertions are unchanged; the tests only wait as long as they need to. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 40 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Deploying herdctl with
|
| Latest commit: |
914dc39
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a235f531.herdctl.pages.dev |
| Branch Preview URL: | https://test-deflake-scheduler-timin.herdctl.pages.dev |
…happen The first pass fixed the three tests seen failing on #492, and CI promptly failed a fourth: "triggers cron schedules when time is within trigger window", whose wait(500) had already been raised once "for slow CI runners". The flake is the pattern, not those three tests: any fixed sleep followed by an assertion that the scheduler got somewhere can lose on a busy runner. Every positive-condition sleep in scheduler.test.ts now polls (21 more). Sleeps that prove an ABSENCE (no second trigger, nothing after stop, max concurrency never exceeded) stay fixed, now after a waitUntil for the event they measure from. waitUntil's deadline drops to 4s, under vitest's 5s test timeout, so a real failure reports the helper's message. Mutation-checked: disabling the cron trigger window fails both window tests in 4s with "waitUntil: condition not met"; they neither hang nor pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docker-security.test.ts "Container Configuration" shells out to the real Docker daemon (docker create / inspect / rm). On a cold or busy CI runner a single `docker create` can exceed vitest's 5s default: "sets memory limits" and "sets user to non-root" both timed out that way on #492, with the other 3772 tests green. Size the per-test timeout for real Docker on that block only; the pure-config tests elsewhere in the file keep the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
De-flakes
scheduler.test.ts, which has been turning CI red, including both attempts of the version PR #492.The flake
Many tests here sleep for a fixed time, then assert the scheduler has already checked, triggered, run, or persisted
state.yaml. On a busy runner that guess comes up short. Three tests failed this way on #492 and one852a99. The first commit fixed those three, and CI promptly failed a fourth:triggers cron schedules when time is within trigger window, whosewait(500)had already been raised once "for slow CI runners". So the flake is the pattern, not any one test.The scheduler logic itself is deterministic. Cron due-checks read the mockable
Date.now(), and temp files are named randomly. What varies is how long start-up and file I/O take on the runner.The fix
waitUntil(predicate)helper polls every 10ms up to a 4s deadline. That's under vitest's 5s test timeout, so a real failure reports the helper's message rather than a bare "Test timed out".waitUntilfor the event it measures from, so the window is real rather than a guess.performance.now(), notDate.now(). Several tests freezeDate.nowto pin the cron clock, so aDate.nowdeadline would never expire.Verification
scheduler.test.ts: 51/51 on 5 plain runs and 3 runs on one core shared with 8 busy-loops.biome checkis clean.if (false && …)), both window tests fail in 4s withwaitUntil: condition not met within 4000ms. They neither hang nor pass, so polling hasn't made them toothless.Also: the real-Docker container tests
#492's rerun turned up a second flake.
docker-security.test.ts › Container Configurationshells out to the real Docker daemon (docker create/inspect/rm), and two of its tests,sets memory limitsandsets user to non-root, hit vitest's 5s default with the other 3772 tests green. Onedocker createon a cold runner can take that long by itself. That block now gets{ timeout: 30_000 }; the pure-config tests elsewhere in the file keep the default. It passes 40/40 locally against a real daemon.Test-only, so no changeset.
🤖 Generated with Claude Code