ci: run the backend, frontend and edge test suites on every pull request - #147
Open
kai-openswarm wants to merge 5 commits into
Open
ci: run the backend, frontend and edge test suites on every pull request#147kai-openswarm wants to merge 5 commits into
kai-openswarm wants to merge 5 commits into
Conversation
… a window, so reducer tests run under node:test Both modules touched window at import time (port/host derivation, the fetch interceptor install, the debug handle), so any node:test file that imports a reducer importing API_BASE died with 'window is not defined' before its first assertion; fetchSessionsStrip.test.ts has been red that way since the resilience work landed, unnoticed because nothing runs these tests in CI. In a renderer (window present) nothing changes: same port/host, same interceptor, same handle. Without one the module answers with the defaults and installs nothing.
Nothing ran any of them in CI: the 235-file backend pytest suite, the 22 renderer node:test files and the edge suite were run by hand, one file at a time, so a regression only surfaced when someone happened to run the right one. Three small workflows, hosted ubuntu, path-filtered, read-only token: - backend-tests: pytest on Python 3.13 from the locked requirements, plus a completion assertion (junit testcase count == collect-only count) so a test process that dies mid-run can never read as green - frontend-tests: tsc --noEmit + node:test via tsx over src/**/*.test.ts(x), through frontend/scripts/run-tests.mjs (the runner the tests already name) - edge-tests: pytest for openswarm-edge All three are green on the current tree: 2951 backend tests, 143 frontend tests across 22 files, 14 edge tests.
Two of five hosted runs of the backend suite stalled at 99% until the job cap with no summary and no junit: one test blocked forever on a bare ws.receive_json() (fixed on its own in a separate change). A CI lane should never depend on every test being unable to hang, so add pytest-timeout to the dev requirements and run the suite with --timeout=300. On Linux the default signal method fails just the offending test and the run continues, so the report and the "every collected test ran" assertion stay meaningful.
test_ws_endpoint_streams_a_full_turn_end_to_end read the socket with a bare
ws.receive_json() in a 40-iteration loop and broke only on the assistant reply.
When the loop ends early for any reason (fewer than 40 events, no reply), the
next receive blocks forever and the whole pytest run stalls at 99% until the
job cap. On hosted runners it does exactly that intermittently, on Linux and
Windows alike: the turn path's configure_provider_env decides whether 9Router
needs reviving from provider evidence earlier tests may leave behind, and that
revival spawns/installs the router behind a module-level asyncio.Lock; the
background turn-label aux call reaches the same machinery. Neither is part of
this test's contract ("SDK and WS auth mocked, everything else real").
Pin both out with monkeypatch, bound every receive at 5s (a regression now
fails this test instead of hanging the runner), and wait for the turn's
completed status before asserting on session.messages so the assertion cannot
race the loop's tail.
Author
|
Two follow-up commits, both surfaced by running this lane repeatedly against
Lane on hosted ubuntu with both: |
Same class as the config/backendConnection change: safeMode.ts read `window` at import, and dashboardLayoutSlice imports it, so any reducer test that imports the slice died under node:test before it ran. Guard the read; in a renderer nothing changes.
Author
|
One more of the same kind: |
This was referenced Aug 18, 2026
Draft
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Nothing in CI ran the backend pytest suite (235 files), the renderer's 22
node:testfiles, or the edge suite — they were run by hand, one file at a time, so a regression only surfaced when someone happened to run the right one. This adds three small workflows, all hosted ubuntu, path-filtered, read-only token,workflow_dispatchfor by-hand runs, triggered on every pull request (sodevPRs are gated) and on pushes tomain/dev:--require-hashes) +requirements-dev.txt, plus a completion assertion:--collect-onlycount must equal the junit testcase count. A test process that dies mid-run can exit 0 with no summary (a hard-exit shutdown path did exactly that once and silently skipped ~42% of the suite), and this makes green mean green rather than "green as far as it got".tsc --noEmit, thennode scripts/run-tests.mjs: node:test via tsx oversrc/**/*.test.ts(x). The runner is the one the test files already name in their headers; it did not exist.openswarm-edge.Separate first commit:
shared/config.tsandshared/backendConnection.tsboth touchedwindowat import time (port/host derivation, the fetch interceptor install, the debug handle), so any node:test file importing a reducer that importsAPI_BASEdied withwindow is not definedbefore its first assertion —fetchSessionsStrip.test.tshad been red that way since the resilience work landed. In a renderer nothing changes (same port/host, same interceptor, same handle); without a window the module answers with the defaults and installs nothing.Verified
Against this exact tree, on GitHub-hosted ubuntu (the same workflow files):
2966 tests collected→2951 passed, 15 skippedin 4 min 15 s; the assertion printedcollected=2966 ran=2966.# tests 143 / # pass 143 / # fail 0(22 files). Before the import-safety commit: 139/140.14 passed.smoke.spec.tsand ten fetch-heavy specs.Scope, on purpose
This is the smallest change that gives
devPRs a real test gate. Left out deliberately, each a candidate for its own PR: awindows-latestpytest leg (the suite currently carries a dozen Windows-portability assumptions in tests —charmapreads, POSIX-only asserts, native separators in API paths — which want fixing first),lintondev(a policy question:devis red on the linter today), an aggregate required-status job for branch protection, and e2e (the packaged app currently cold-boots paste2e.yml's budgets on hosted Windows).