Severity: Medium
Several test files have implicit ordering dependencies between it() blocks. Tests pass for the wrong reason; a reordering would surface flakes.
1. runner.test.ts — sequential state across describes
sisyphus-api/test/runner/runner.test.ts:62-148:
:62 starts a research session.
:75 "returns 409 when workflow already running" — only passes because :62 left a session active. (Also see the previous issue's note about whether 409 is even real.)
:86 "stops a running workflow" — depends on the session from :62 still being live.
:95 "returns 404 when no session running" — depends on :86 having just stopped it.
:135 history-list test asserts total >= 1 "From start/stop tests" — explicit ordering coupling in the assertion.
If file order or it() order changes, half the suite fails or passes for the wrong reason.
2. workflow.test.ts + connector + compile tests share a top-level let workflowId: string
test/workflows/workflow.test.ts:49, 74, 88, 104 — workflowId is declared inside the describe, assigned only by the first it("POST /workflows ..."). Subsequent it() blocks read it. If the POST test fails, every subsequent test fails with cryptic undefined 404s. Same pattern at :133 (connectorId) and :191 (workflowId inside the compile describe).
3. schemas-crud.test.ts — same pattern
createdId shared across the suite (schemas-crud.test.ts:25, 47, 77, 92, 144); the very first POST controls the rest of the suite's outcome.
Remediation
Two patterns to apply:
A) beforeEach factory. Reset the relevant collection + factory a fresh resource at the start of every it:
let workflowId: string;
beforeEach(async () => {
const res = await request(app).post('/workflows').send(validBody);
workflowId = res.body.id;
});
B) Reset the in-memory activeSessions Map and the sessions collection in beforeEach of runner.test.ts. Each test should set up its own state.
Drop the "From start/stop tests" assertion (runner.test.ts:135) — pin the count via local setup, not implicit ordering.
Coupled with #8 (test setup refactor) — fixing both is the same surgery.
Severity: Medium
Several test files have implicit ordering dependencies between
it()blocks. Tests pass for the wrong reason; a reordering would surface flakes.1.
runner.test.ts— sequential state across describessisyphus-api/test/runner/runner.test.ts:62-148::62starts aresearchsession.:75"returns 409 when workflow already running" — only passes because:62left a session active. (Also see the previous issue's note about whether 409 is even real.):86"stops a running workflow" — depends on the session from:62still being live.:95"returns 404 when no session running" — depends on:86having just stopped it.:135history-list test assertstotal >= 1"From start/stop tests" — explicit ordering coupling in the assertion.If file order or
it()order changes, half the suite fails or passes for the wrong reason.2.
workflow.test.ts+connector+compiletests share a top-levellet workflowId: stringtest/workflows/workflow.test.ts:49, 74, 88, 104—workflowIdis declared inside thedescribe, assigned only by the firstit("POST /workflows ..."). Subsequentit()blocks read it. If the POST test fails, every subsequent test fails with crypticundefined404s. Same pattern at:133(connectorId) and:191(workflowIdinside the compile describe).3.
schemas-crud.test.ts— same patterncreatedIdshared across the suite (schemas-crud.test.ts:25, 47, 77, 92, 144); the very first POST controls the rest of the suite's outcome.Remediation
Two patterns to apply:
A)
beforeEachfactory. Reset the relevant collection + factory a fresh resource at the start of everyit:B) Reset the in-memory
activeSessionsMap and the sessions collection inbeforeEachofrunner.test.ts. Each test should set up its own state.Drop the "From start/stop tests" assertion (
runner.test.ts:135) — pin the count via local setup, not implicit ordering.Coupled with #8 (test setup refactor) — fixing both is the same surgery.