Repository navigation
Run the dashboard's tests on their own, and close three coverage-gate gaps - #549
Conversation
This comment has been minimized.
This comment has been minimized.
7c9f94f to
5fcc101
Compare
This comment has been minimized.
This comment has been minimized.
5fcc101 to
1af13a6
Compare
This comment has been minimized.
This comment has been minimized.
With the backend suite sharded (#547), Web (dashboard) became the job that sets the gate: on main's run for #547 the backend path finished at 95 s and the dashboard at 119 s. It runs lint (13 s), typecheck (7 s), tests (39-75 s) and build (5 s) one after another, so the tests wait on 25 s of work they do not depend on. The tests now have their own matrix entry, beside one for lint, typecheck and build. On a slow runner that entry is about 13 s of setup plus the test step, which is below the backend path and takes the dashboard off the critical path. The other workspaces keep one entry each; their scripts take 20 s together and splitting them would buy nothing. Splitting rather than sharding with vitest --shard, because the split alone is enough while the backend path stays above this entry, and sharding would add a runner for no gain today. Unlike pytest, vitest's cost does divide by shard: timed at three workers, the whole suite took 100 s and each third 37-41 s, which fits about 8 s fixed against 92 s divisible. So --shard is the next step if the backend path ever drops below this entry. The job keeps its name, web-build, so staging's needs list is unchanged. Which entry runs which script is now decided by step conditions, and a mistake there fails nothing: a script left out runs nowhere, one run twice silently undoes the split, and the two halves trading conditions keeps every count right while the tests land back on the checks entry. tests/test_web_build_matrix.py pins that every workspace in package.json has an entry, that each lint, typecheck, test and build script it defines runs in exactly one, that the dashboard is split in two, that each half runs only its own scripts, and that a split entry's name says which half it is. All eleven mutations tried against the workflow turn it red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three follow-ups to #547, where the shards' coverage is combined and the threshold applied on a runner that measured nothing. The omit test checked only test_deploy_scope.py's shallow tmp_path shape, a repo's top-level app.py. test_migrations.py also puts measured files in the report, a copied core/ two levels deeper and under an xdist worker's popen-gwN directory. A glob narrowed to a fixed depth, or to the serial basetemp layout, still matched the shallow path and would have passed while backend-coverage died on "No source for code". The test now asserts both shapes; both of those narrowings turn it red. --splits named the shard count as a literal beside the matrix list it has to equal. It now reads strategy.job-total, so the matrix alone sets the count, and the test pins that --splits and --group follow the matrix. job-total counts every combination, so the test also pins that shard is the matrix's only axis: a second one would multiply --splits while --group still spanned only the shards, leaving part of the suite unrun. The fragment count was checked by a whitespace-sensitive substring, where the rest of the file parses numbers with _threshold(). It uses that now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1af13a6 to
c0a9ce1
Compare
|
Claude finished @Babissimo's task in 5m 46s —— View job Claude finished the review
ReviewRe-ran
One unrelated CI observation: this run's No changes recommended. |
Follows #547 (backend sharding). Second half of ClickUp Shard the backend suite and split the dashboard tests, plus three follow-ups to #547's coverage gate that were parked on that ticket. Two commits, one purpose each.
1. Run the dashboard's tests in an entry of their own
Why. Now that #547 has sharded the backend suite,
Web (dashboard)sets the gate. On main's run for #547 itself (35848199811) the backend path finished at 95 s andWeb (dashboard)at 119 s. That job runs lint (13 s), typecheck (7 s), tests (39–75 s) and build (5 s) in sequence, so the tests wait on about 25 s of work they don't depend on.What.
web-buildbecomes an explicitincludematrix. The dashboard gets two entries,Web (dashboard checks)(lint, typecheck, build) andWeb (dashboard tests);packages/sharedande2ekeep one entry each and run everything as before. Stepif:s onmatrix.partchoose each entry's half.web-build, sostaging'sneeds:doesn't change. There is no branch protection or ruleset onmain, so no required check refers to the oldWeb (dashboard)name.backend/tests/test_web_build_matrix.pypins the assignment: every workspace has an entry, each lint/typecheck/test/build script runs in exactly one entry, the dashboard is split in two, each half runs only its own scripts, and a split entry's name says which half it is. Eleven mutations of the workflow all turn it red, including the two halves trading conditions (which keeps every count right).Fixed versus divisible. Timed locally at CI's three workers, the whole dashboard suite took 100 s and each
--shardthird 37–41 s. That fits about 8 s fixed against 92 s divisible, the opposite of pytest's ~47 s fixed.Expected effect. This PR's first run:
Web (dashboard tests)finished at 63 s (46 s test step) andWeb (dashboard checks)at 46 s, against a backend path of 107 s. On a slow runner the tests entry is about 13 s of setup plus a 75 s step, around 88 s, against a backend path of 94–121 s across the sharded runs so far. So the gate becomes the backend path: ~95 s rather than 119 s on #547's main run. The margin is thin on a slow runner.vitest --shardisn't included: it buys nothing while the tests entry finishes under the backend path. Given the split above, it is the cheap next step if the margin closes.Deliberate trade-off: a lint or typecheck failure no longer skips the tests, so a broken
tscnow costs a full test run. Runner minutes are free on this public repo, and wall time is what the gate measures.2. Close three gaps in the backend coverage gate's pins
test_a_tmp_path_fixture_is_omitted_from_the_reportchecked onlytest_deploy_scope.py's shallow path shape. It now also assertstest_migrations.py's deeperpopen-gwN/…/old_image/core/…shape. Two plausible narrowings of the glob (fixed depth, serial basetemp layout) matched the shallow path and would have passed whilebackend-coveragedied; both now go red.--splits 3→--splits ${{ strategy.job-total }}, so the matrix alone sets the shard count. The test pins that--splitsand--groupfollow the matrix, and thatshardis its only axis:job-totalcounts every combination, so a second axis would leave part of the suite unrun._threshold()rather than a whitespace-sensitive substring.Checks
pre-commit run --all-files(ruff, actionlint, vulture) passes.test_deploy_workflows,test_rollback_script,test_smoke_tally_shared,test_nginx_origin_marker,test_backend_coverage_gate,test_web_build_matrix): 108 passed.🤖 Generated with Claude Code