Repository navigation
Shard the dashboard's tests two ways - #579
Conversation
Since #569 shortened the backend path, the dashboard's tests entry has been its equal: on CI runs since then its job took a median 88 s against 92 s for the slowest backend shard plus backend-coverage, queue-free, and it was the longer of the two in 2 of 7 runs. It set the gate outright on main run 35876349605. So no further backend saving reaches the gate while this entry sits beside it. A vitest shard loads only its own files, so unlike pytest's the split divides nearly all of the suite's time (about 8 s fixed against 92 s divisible, measured at three workers). Two shards partition the suite exactly, 58 files each and 497 + 857 = 1354 tests, and split its CPU time about evenly, which should put each entry near 55 s. test_web_build_matrix.py pinned each workspace script to exactly one entry. It now accepts a script that runs once per shard, provided the shards are 1..N of N exactly once each, and checks that the test step hands each entry its shard after npm's --. Without the separator npm keeps --shard as its own config and both entries run the whole suite, green. Nine mutations of the shard list, the step and the labels each turn it red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
|
Claude finished @Babissimo's task in 1m 57s —— View job Review: Shard the dashboard's tests two ways
No correctness issues found. This is a tight, self-contained change and the diff matches its own description closely. What I checked:
One thing I couldn't verify directly: I wasn't able to run Nothing else to flag — the change is narrowly scoped to CI sharding, the pinning test is extended sensibly rather than weakened, and docs were kept in sync. |
Splits
Web (dashboard tests)into twovitest --shardentries, so the dashboard stops sitting level with the backend path and further backend savings can reach the gate.ClickUp: Shard the dashboard's tests two ways (split out of Cut the backend suite's per-process fixed cost).
Why now
Since #569 the dashboard's tests entry has been the backend path's equal. Queue-free job durations on CI runs since #549 split it out:
backend-coverage)Web (dashboard tests)The dashboard entry was the longer of the two in 2 of the 7 post-#569 runs, and it set the gate outright on main run 35876349605 (107 s against
backend-coverage's 102 s). Its test step is a 75 s median; checkout,setup-nodeandnpm ciadd about 11 s.Why a shard divides here
A vitest shard loads only its own files, unlike a pytest-split group, which collects the whole suite and discards the rest. At three workers locally the whole suite takes 100 s and a third 37–41 s, so about 8 s is fixed. The two shards partition the suite exactly (58 files each; 497 + 857 = 1,354 tests, the whole suite's count) and split its CPU time about evenly (roughly 245 s each). Vitest assigns files by a hash of their relative path, so the split is the same on every machine.
Expected: each entry's test step near 40 s and its job near 55 s, about 35 s under the backend path. To be confirmed on this PR's run.
The pinning test
backend/tests/test_web_build_matrix.pypinned each workspace script to exactly one entry. It now accepts a script that runs once per shard, provided the shards are 1..N of N once each, and it checks that the test step hands each entry its shard after npm's--. Without the--, npm keeps--shardas its own config (a warning), and both entries would run the whole suite while staying green.Nine mutations each turn it red: dropping a shard, duplicating one, shards of 3 across two entries, the step dropping the shard, the step dropping
--, one entry unsharded, a label without its shard, a malformed shard, and a shard on the checks entry.npm run testin place ofnpm teststays green.Verification
pre-commit run --all-filesclean.-m "not external" -n 4 --dist worksteal): 4,861 passed, 2 skipped, exit 0./code-reviewat xhigh: the--gap above and a stale ONBOARDING line were fixed; the rest needed no change.🤖 Generated with Claude Code