From a938314f37d7533d380d83a6fd25187af1fb325e Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Wed, 12 Aug 2026 00:18:31 -0400 Subject: [PATCH 01/11] fix(test): enforce waitForCondition's budget and stop socket reuse MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two defects, one of which was hiding the other. waitForCondition checked the clock only on loop entry, so one slow fn() overran the budget without bound — a 10s budget was measured running 28s, past the caller's testTimeout, so vitest killed the test first and reported a timeout naming neither the condition nor how long the poll waited. It now races fn() against the deadline, aborts the in-flight call, and reports poll shape: "N poll(s), slowest Xms". That reporting is what exposed the second defect. chQuery used the global fetch, which reuses pooled connections; undici 8.8.0-8.9.0 stalls for seconds before writing a request onto a socket idle for a few seconds (nodejs/undici#5600, fixed in 8.10.0). This suite has multi-second idle gaps by construction — the 5s ingest linger sits between every write and the first poll of its visibility wait — so every visibility wait sat in the triggering window. Node 26 bundles undici 8.9.0; CI runs Node 22 (undici 6.28.0) via .nvmrc, which is why CI never saw it. Local `make test-e2e`: 2 pass/3 fail -> 5 pass/0 fail, and every run faster (115.7-124.5s vs 128.7-137.1s). ClickHouse measured p99 <= 2.5ms throughout; Docker/OrbStack, DNS, the libuv threadpool, GC, event-loop blocking and machine load were each excluded by measurement. See #440. Also here: - chQuery gets a 10s ceiling and honours the caller's AbortSignal, threaded through 19 call sites, so an abandoned poll tears its request down. - batching's visibility wait had ~700ms of headroom over the 5s linger while every other wait allows 10s; widened. The >= 4500ms lower bound, which is what the test actually asserts, is unchanged. - The e2e banner prints node/undici and warns when the local major differs from .nvmrc, so the next version-specific failure is attributable in seconds rather than days. - The orchestrator refuses to start beside an orphaned wavehouse-cov. A killed run leaves one, and it corrupts the next run through the shared tmp/data and log file — presenting as a dozen unrelated tests failing to see their rows, in a log blaming a container that no longer exists. - vitest.config: __dirname -> import.meta.dirname, silencing the vite 8 configLoader warning. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014EJvon2pyU4zaSHTGJj8fE --- scripts/orchestrator/main.go | 50 ++++++++++- tests/e2e/sdk/batching.test.ts | 20 +++-- tests/e2e/sdk/cache.test.ts | 3 +- tests/e2e/sdk/dlq.test.ts | 3 +- tests/e2e/sdk/helpers.test.ts | 118 ++++++++++++++++++++++++++ tests/e2e/sdk/helpers.ts | 149 ++++++++++++++++++++++++++++----- tests/e2e/sdk/ingest.test.ts | 33 +++++--- tests/e2e/sdk/ndjson.test.ts | 15 ++-- tests/e2e/sdk/query.test.ts | 3 +- tests/e2e/sdk/setup.ts | 32 +++++++ tests/e2e/sdk/stress.test.ts | 3 +- tests/e2e/sdk/vitest.config.ts | 13 +-- 12 files changed, 391 insertions(+), 51 deletions(-) create mode 100644 tests/e2e/sdk/helpers.test.ts diff --git a/scripts/orchestrator/main.go b/scripts/orchestrator/main.go index c736fb65..46c26c1d 100644 --- a/scripts/orchestrator/main.go +++ b/scripts/orchestrator/main.go @@ -39,6 +39,7 @@ import ( "os/signal" "path/filepath" "strconv" + "strings" "syscall" "time" @@ -66,6 +67,24 @@ func run() error { return fmt.Errorf("%s missing — run `make build-cover` first", binPath) } + // Refuse to start alongside a previous run's server. Both would use the + // JetStream/pebble state under tmp/data and both would write + // tmp/wavehouse-cov.log, so the survivor corrupts this run's state and + // interleaves its output into this run's log — which surfaces as a dozen + // unrelated tests failing to see their rows, in a log that blames the + // wrong ClickHouse. `make test-e2e` cleans up after itself; a leftover + // means the previous run was killed (a harness timeout, a stop button, an + // impatient SIGKILL) rather than interrupted. Fail loudly instead of + // producing a mystery. + if stale, err := staleServerPIDs(ctx, binPath); err != nil { + log.Printf(" (could not check for leftover servers: %v)", err) + } else if len(stale) > 0 { + return fmt.Errorf( + "a previous wavehouse-cov is still running (pid %s) — it shares tmp/data and "+ + "tmp/wavehouse-cov.log with this run and will corrupt it.\n kill it with: kill %s", + strings.Join(stale, " "), strings.Join(stale, " ")) + } + coverDir := filepath.Join(repoRoot, "tmp", "coverage", "e2e", "data") if err := os.MkdirAll(coverDir, 0o750); err != nil { return fmt.Errorf("mkdir coverdir: %w", err) @@ -201,8 +220,17 @@ func run() error { // straight to `pnpm exec vitest run --coverage` skips the script-arg // forwarding layer entirely, matching how scripts/cov invokes `pnpm exec // nyc`. + // + // E2E_NO_COVERAGE=1 drops it for local debugging only. Coverage is on by + // default and `make ci` never sets this — a run without it writes no + // report, so `make cov` would gate on stale numbers. + args := []string{"exec", "vitest", "run", "--coverage"} + if os.Getenv("E2E_NO_COVERAGE") == "1" { + args = args[:len(args)-1] + log.Println(" (E2E_NO_COVERAGE=1 — running without coverage; no report will be written)") + } // #nosec G204 — args are a fixed string slice, not user input. - vitest := exec.CommandContext(ctx, "pnpm", "exec", "vitest", "run", "--coverage") + vitest := exec.CommandContext(ctx, "pnpm", args...) vitest.Dir = filepath.Join(repoRoot, "tests", "e2e", "sdk") vitest.Env = append(os.Environ(), "WAVEHOUSE_URL="+whURL, @@ -257,6 +285,26 @@ func run() error { return vitestErr } +// staleServerPIDs returns the PIDs of any wavehouse-cov left over from an +// earlier run. Called before this run starts its own, so every match is stale. +// A missing/failed pgrep is reported as an error and treated as "unknown" by +// the caller — this is a guard rail, not a gate. +func staleServerPIDs(ctx context.Context, binPath string) ([]string, error) { + // #nosec G204 — binPath is filepath.Join(repoRoot, "bin", "wavehouse-cov") + // with constant components, not user input; it is only ever a search + // pattern here, never executed. + out, err := exec.CommandContext(ctx, "pgrep", "-f", binPath).Output() + if err != nil { + // pgrep exits 1 with no output when nothing matches — the common case. + var exitErr *exec.ExitError + if errors.As(err, &exitErr) && exitErr.ExitCode() == 1 { + return nil, nil + } + return nil, err + } + return strings.Fields(string(out)), nil +} + // pickFreePort asks the OS for an available port on 127.0.0.1, then // closes the listener so wavehouse-cov can bind to it. There's a brief // TOCTOU window where another process could grab the port, but on a diff --git a/tests/e2e/sdk/batching.test.ts b/tests/e2e/sdk/batching.test.ts index 0c6907fe..bc2e8fc0 100644 --- a/tests/e2e/sdk/batching.test.ts +++ b/tests/e2e/sdk/batching.test.ts @@ -33,9 +33,10 @@ describe("Ingest Batching Triggers", () => { // the worker's buffer held all 500 rows before the timer could fire — // an early flush below is attributable to the size trigger. await waitForCondition( - async () => { + async (signal) => { const r = await chQuery( `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return Number((r[0] as any).cnt) === 500; }, @@ -61,9 +62,10 @@ describe("Ingest Batching Triggers", () => { `ack took ${ackMs}ms (≥4s): linger fired mid-publish; size-trigger timing inconclusive — verifying integrity only`, ); await waitForCondition( - async () => { + async (signal) => { const r = await chQuery( `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return Number((r[0] as any).cnt) === 500; }, @@ -95,15 +97,21 @@ describe("Ingest Batching Triggers", () => { ); expect(Number((r[0] as any).cnt)).toBe(0); - // Wait until it appears + // Wait until it appears. The budget is only a "don't hang forever" bound — + // the assertion that carries this test's meaning is the >= 4500ms lower + // bound below. It used to be 6_000, which left ~700ms over the 5s linger + // and made this the most flake-prone wait in the suite; every other + // visibility wait allows 10s. Widening costs no test power (an early flush + // still fails the lower bound) and removes the false failures (#440). await waitForCondition( - async () => { + async (signal) => { const check = await chQuery( `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return Number((check[0] as any).cnt) === 1; }, - 6_000, + 10_000, 500, ); @@ -111,5 +119,5 @@ describe("Ingest Batching Triggers", () => { // It should take roughly ~5 seconds for ingest worker's period trigger to fire expect(elapsed).toBeGreaterThanOrEqual(4500); - }, 20_000); + }, 25_000); }); diff --git a/tests/e2e/sdk/cache.test.ts b/tests/e2e/sdk/cache.test.ts index 06dfec5f..972e653a 100644 --- a/tests/e2e/sdk/cache.test.ts +++ b/tests/e2e/sdk/cache.test.ts @@ -50,9 +50,10 @@ describe("Cache", () => { // 4. Wait for the async worker to flush to ClickHouse and invalidate the cache. // By querying ClickHouse directly, we don't accidentally trigger a cache re-prime! - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT event_id FROM default.${T.clicks} WHERE event_id = '${eventId}'`, + signal, ); return r.length === 1; }, 10_000); diff --git a/tests/e2e/sdk/dlq.test.ts b/tests/e2e/sdk/dlq.test.ts index 8589bed3..1421b894 100644 --- a/tests/e2e/sdk/dlq.test.ts +++ b/tests/e2e/sdk/dlq.test.ts @@ -47,9 +47,10 @@ describe("Dead Letter Queue (DLQ) & Failures", () => { // edge under CI load. The structural fix is a lower e2e maxWait (deferred // config PR), which drops the 5s-timer dependency; this budget can shrink // back once that lands. - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const chRows = await chQuery( `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return Number((chRows[0] as any).cnt) === 9; }, 10_000); diff --git a/tests/e2e/sdk/helpers.test.ts b/tests/e2e/sdk/helpers.test.ts new file mode 100644 index 00000000..8f28f60c --- /dev/null +++ b/tests/e2e/sdk/helpers.test.ts @@ -0,0 +1,118 @@ +/** + * Unit tests for the E2E suite's own helpers (#440). + * + * These need no stack — they live here because they test this directory's + * code, and this directory's vitest project is the only one that compiles it. + * Bounds are deliberately loose (order-of-magnitude, not milliseconds): a + * harness that polices timeouts must not itself fail on a busy machine. + */ + +import { describe, expect, it } from "vitest"; +import { waitForCondition } from "./helpers.js"; + +const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); + +describe("waitForCondition", () => { + it("returns as soon as the condition holds", async () => { + let calls = 0; + const started = Date.now(); + + await waitForCondition( + () => { + calls += 1; + return calls === 3; + }, + 5_000, + 10, + ); + + expect(calls).toBe(3); + expect(Date.now() - started).toBeLessThan(2_000); + }); + + it("enforces the budget even when a single poll overruns it", async () => { + // The regression: the old loop checked the clock only on entry, so this + // returned at ~5000ms (the fn duration), not at the 300ms budget. + const started = Date.now(); + + await expect( + waitForCondition(async () => { + await sleep(5_000); + return true; + }, 300), + ).rejects.toThrow(/Condition not met after 300ms/); + + expect(Date.now() - started).toBeLessThan(3_000); + }); + + it("aborts the signal it hands to fn, so in-flight work can unwind", async () => { + let sawAbort = false; + + await expect( + waitForCondition(async (signal) => { + await new Promise((resolve) => { + signal.addEventListener( + "abort", + () => { + sawAbort = true; + resolve(); + }, + { once: true }, + ); + }); + return false; + }, 300), + ).rejects.toThrow(/Condition not met/); + + expect(sawAbort).toBe(true); + }); + + it("reports how long it polled, and how many/how slow the polls were", async () => { + // Many fast polls — the signature of "the condition never became true", + // as opposed to "the polling itself was starved". + await expect(waitForCondition(() => false, 400, 50)).rejects.toThrow( + /polled for \d+ms; \d+ poll\(s\), slowest \d+ms/, + ); + }); + + it("counts a poll still in flight at the deadline as the slowest", async () => { + await expect( + waitForCondition(async () => { + await sleep(5_000); + return false; + }, 500), + ).rejects.toThrow(/1 poll\(s\), slowest [4-9]\d\dms/); + }); + + it("propagates a rejecting fn instead of masking it as a timeout", async () => { + await expect( + waitForCondition(async () => { + throw new Error("ClickHouse query failed: no such table"); + }, 5_000), + ).rejects.toThrow(/no such table/); + }); + + it("does not leak an unhandled rejection when fn rejects after the deadline", async () => { + const unhandled: unknown[] = []; + const capture = (err: unknown) => unhandled.push(err); + process.on("unhandledRejection", capture); + + try { + await expect( + waitForCondition(async () => { + await sleep(600); + throw new Error("late failure"); + }, 200), + ).rejects.toThrow(/Condition not met after 200ms/); + + // Outlive the rejecting poll, then give the microtask queue a turn so + // an unhandled rejection would have been reported by now. + await sleep(1_000); + await new Promise((r) => setImmediate(r)); + } finally { + process.off("unhandledRejection", capture); + } + + expect(unhandled).toEqual([]); + }); +}); diff --git a/tests/e2e/sdk/helpers.ts b/tests/e2e/sdk/helpers.ts index 4b9063c9..14e89c1a 100644 --- a/tests/e2e/sdk/helpers.ts +++ b/tests/e2e/sdk/helpers.ts @@ -73,18 +73,80 @@ export function dataClient() { // ── Wait Utilities ──────────────────────────────────────────────────────────── -/** Poll a condition until it returns true, or timeout. */ +/** Resolve after `ms`, or as soon as `signal` aborts — whichever comes first. */ +function sleep(ms: number, signal: AbortSignal): Promise { + return new Promise((resolve) => { + const done = () => { + clearTimeout(timer); + signal.removeEventListener("abort", done); + resolve(); + }; + const timer = setTimeout(done, ms); + signal.addEventListener("abort", done, { once: true }); + }); +} + +/** + * Poll a condition until it returns true, or timeout. + * + * `timeoutMs` bounds the whole call, not just the gaps between polls. The + * previous version checked the clock only on loop entry, so one slow `fn()` + * overran the budget without bound — a 10s budget was measured running 28s, + * past the caller's vitest `testTimeout`. Vitest then killed the test first + * and reported `Test timed out in 20000ms`, naming neither the condition nor + * how long the poll actually waited (#440). + * + * `fn` receives the budget's `AbortSignal`; pass it to anything cancellable + * (e.g. `chQuery`) so an in-flight request is torn down at the deadline + * instead of running on unobserved. + */ export async function waitForCondition( - fn: () => boolean | Promise, + fn: (signal: AbortSignal) => boolean | Promise, timeoutMs = 10_000, intervalMs = 250, ): Promise { const start = Date.now(); - while (Date.now() - start < timeoutMs) { - if (await fn()) return; - await new Promise((r) => setTimeout(r, intervalMs)); + const controller = new AbortController(); + const { signal } = controller; + const budget = setTimeout(() => controller.abort(), timeoutMs); + const expired = new Promise((resolve) => { + signal.addEventListener("abort", () => resolve(false), { once: true }); + }); + + let polls = 0; + let slowestPollMs = 0; + + try { + while (!signal.aborted) { + const pollStart = Date.now(); + polls += 1; + const poll = (async () => Boolean(await fn(signal)))(); + // The race below stops awaiting `poll` at the deadline, but the call is + // still in flight and may reject afterwards. Mark it handled so a late + // rejection doesn't surface as an unhandled rejection in another test. + poll.catch(() => {}); + + // A rejecting `fn()` still propagates (it names its own failure, which + // beats a generic timeout); only the deadline short-circuits the wait. + if (await Promise.race([poll, expired])) return; + slowestPollMs = Math.max(slowestPollMs, Date.now() - pollStart); + if (signal.aborted) break; + + await sleep(intervalMs, signal); + } + } finally { + clearTimeout(budget); + controller.abort(); } - throw new Error(`Condition not met after ${timeoutMs}ms`); + + // Poll stats separate the two ways a wait can end: many fast polls means the + // condition simply never became true (look upstream — the write never + // landed); few slow ones means the polling itself was starved (look at the + // machine or the query). + throw new Error( + `Condition not met after ${timeoutMs}ms ` + + `(polled for ${Date.now() - start}ms; ${polls} poll(s), slowest ${slowestPollMs}ms)`, + ); } // ── Unique ID ───────────────────────────────────────────────────────────────── @@ -95,18 +157,67 @@ export function testId(): string { // ── ClickHouse Direct Query ─────────────────────────────────────────────────── -export async function chQuery>(sql: string): Promise { - const res = await fetch(`${CH_URL}/?default_format=JSONEachRow`, { - method: "POST", - body: sql, - }); - if (!res.ok) { - throw new Error(`ClickHouse query failed: ${await res.text()}`); +/** Per-request ceiling for a direct ClickHouse call. */ +const CH_QUERY_TIMEOUT_MS = Number(process.env.E2E_CH_QUERY_TIMEOUT_MS ?? 10_000); + +/** Single-line, length-capped SQL for error messages. */ +const brief = (sql: string) => { + const flat = sql.replace(/\s+/g, " ").trim(); + return flat.length > 120 ? `${flat.slice(0, 117)}...` : flat; +}; + +export async function chQuery>( + sql: string, + signal?: AbortSignal, +): Promise { + // Without a deadline a stalled request hangs until vitest kills the test, + // reporting a timeout that names neither ClickHouse nor the query (#440). + // `signal` (typically waitForCondition's budget) tears the request down + // early when the caller has already given up. + const ceiling = AbortSignal.timeout(CH_QUERY_TIMEOUT_MS); + const deadline = signal ? AbortSignal.any([signal, ceiling]) : ceiling; + + try { + const res = await fetch(`${CH_URL}/?default_format=JSONEachRow`, { + method: "POST", + body: sql, + signal: deadline, + // Don't reuse a pooled connection: undici 8.8.0-8.9.0 stalls for seconds + // before writing a request onto a socket that has been idle a few + // seconds. Upstream bug, not ours — nodejs/undici#5600, a scheduling + // regression in scheduleIdleSocketValidation() (itself the fix for + // GHSA-35p6-xmwp-9g52). Bisected on a fixed Node 22: 8.7.0 clean (22ms), + // 8.8.0 broken (2708ms), 8.9.0 broken (7164ms), 8.10.0 clean (13ms). + // + // It reaches us because Node 26.x bundles 8.9.0 and this suite polls + // seconds apart by construction (the 5s ingest linger sits between every + // write and its first poll), so every visibility wait lands in the + // triggering window — ~3 local runs in 5 failed, while ClickHouse itself + // answered in ~1ms throughout (#440). CI is unaffected: .nvmrc pins + // Node 22 (undici 6.28.0). + // + // DELETE THIS once the Node lines we run bundle undici >= 8.10.0; it + // costs a connection per query (~1ms on loopback) and nothing else. + headers: { connection: "close" }, + }); + // Read the body inside the try: aborting mid-response rejects here, not + // at the fetch above. + const text = await res.text(); + if (!res.ok) { + throw new Error(`ClickHouse query failed: ${text}`); + } + if (!text.trim()) return []; + return text + .trim() + .split("\n") + .map((line) => JSON.parse(line)); + } catch (err) { + if (ceiling.aborted) { + throw new Error(`ClickHouse query timed out after ${CH_QUERY_TIMEOUT_MS}ms: ${brief(sql)}`); + } + if (signal?.aborted) { + throw new Error(`ClickHouse query aborted by caller: ${brief(sql)}`); + } + throw err; } - const text = await res.text(); - if (!text.trim()) return []; - return text - .trim() - .split("\n") - .map((line) => JSON.parse(line)); } diff --git a/tests/e2e/sdk/ingest.test.ts b/tests/e2e/sdk/ingest.test.ts index 224fe0e3..582e294a 100644 --- a/tests/e2e/sdk/ingest.test.ts +++ b/tests/e2e/sdk/ingest.test.ts @@ -30,8 +30,11 @@ describe("Ingest", () => { expect(result.data).toMatchObject({ ok: true }); // Poll ClickHouse — pipeline flush timing varies - await waitForCondition(async () => { - const r = await chQuery(`SELECT event_id FROM default.${T.clicks} WHERE event_id = '${id}'`); + await waitForCondition(async (signal) => { + const r = await chQuery( + `SELECT event_id FROM default.${T.clicks} WHERE event_id = '${id}'`, + signal, + ); return r.length === 1; }, 10_000); @@ -54,9 +57,10 @@ describe("Ingest", () => { expect(result.data).toMatchObject({ ok: true }); // Poll ClickHouse — pipeline flush timing varies - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT event_id FROM default.${T.clicks} WHERE event_id IN ('${ids.join("','")}')`, + signal, ); return r.length === 3; }, 10_000); @@ -104,8 +108,11 @@ describe("Ingest", () => { expect(result.data).toMatchObject({ ok: true }); // Poll ClickHouse — on cold-start the pipeline may take longer than 4s - await waitForCondition(async () => { - const r = await chQuery(`SELECT event_id FROM default.${T.events} WHERE event_id = '${id}'`); + await waitForCondition(async (signal) => { + const r = await chQuery( + `SELECT event_id FROM default.${T.events} WHERE event_id = '${id}'`, + signal, + ); return r.length === 1; }, 10_000); @@ -137,8 +144,11 @@ describe("Ingest", () => { expect(result2.data).toMatchObject({ duplicate: true }); // Poll ClickHouse — on cold-start the pipeline may take longer than 4s - await waitForCondition(async () => { - const r = await chQuery(`SELECT event_id FROM default.${T.events} WHERE event_id = '${id}'`); + await waitForCondition(async (signal) => { + const r = await chQuery( + `SELECT event_id FROM default.${T.events} WHERE event_id = '${id}'`, + signal, + ); return r.length === 1; }, 10_000); @@ -185,9 +195,10 @@ describe("Ingest", () => { expect(result.data).toMatchObject({ ok: true }); // 5. Verify it successfully made it through NATS, ingest worker, and into ClickHouse - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT event_id FROM default.\`${weirdTableName}\` WHERE event_id = '${id}'`, + signal, ); return r.length === 1; }, 10_000); @@ -240,9 +251,10 @@ describe("Ingest", () => { expect(result.data).toMatchObject({ ok: true }); // 5. Verify it landed in the weirdly named table (proving it was treated as a literal string) - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT event_id FROM default.\`${maliciousName}\` WHERE event_id = '${id}'`, + signal, ); return r.length === 1; }, 10_000); @@ -325,9 +337,10 @@ describe("Ingest", () => { expect(goodRes.error).toBeNull(); // Verify the auto-injected row in CH has country=US - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT country FROM default.${T.clicks} WHERE event_id = '${autoId}'`, + signal, ); return r.length === 1 && r[0].country === "US"; }, 10_000); diff --git a/tests/e2e/sdk/ndjson.test.ts b/tests/e2e/sdk/ndjson.test.ts index 7acf3a34..9ed8cf00 100644 --- a/tests/e2e/sdk/ndjson.test.ts +++ b/tests/e2e/sdk/ndjson.test.ts @@ -24,9 +24,10 @@ describe("NDJSON ingest", () => { expect(result.error).toBeNull(); expect(result.data).toMatchObject({ ok: true, total: 3, succeeded: 3, failed: 0 }); - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT count() AS cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return Number((r[0] as { cnt: number }).cnt) === 3; }, 10_000); @@ -61,9 +62,10 @@ describe("NDJSON ingest", () => { expect(failed?.index).toBe(2); // Exactly the two good rows reach ClickHouse; the bad one does not. - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT event_id FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return r.length === 2; }, 10_000); @@ -91,9 +93,10 @@ describe("NDJSON ingest", () => { expect(result.error).toBeNull(); expect(result.data).toMatchObject({ total: 2, succeeded: 2, failed: 0 }); - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT count() AS cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return Number((r[0] as { cnt: number }).cnt) === 2; }, 10_000); @@ -121,9 +124,10 @@ describe("NDJSON ingest", () => { expect(failed?.index).toBe(2); expect(failed?.error).toContain("invalid json"); - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT event_id FROM default.${T.clicks} WHERE event_id = '${good}'`, + signal, ); return r.length === 1; }, 10_000); @@ -152,9 +156,10 @@ describe("NDJSON ingest", () => { const body = (await res.json()) as { total: number; succeeded: number; failed: number }; expect(body).toMatchObject({ total: 2, succeeded: 2, failed: 0 }); - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT count() AS cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`, + signal, ); return Number((r[0] as { cnt: number }).cnt) === 2; }, 10_000); diff --git a/tests/e2e/sdk/query.test.ts b/tests/e2e/sdk/query.test.ts index 144bab2d..9c98fbaf 100644 --- a/tests/e2e/sdk/query.test.ts +++ b/tests/e2e/sdk/query.test.ts @@ -31,9 +31,10 @@ describe("Query", () => { }); } // Poll until all seeded rows are visible in ClickHouse - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT count() as cnt FROM default.${T.clicks} WHERE event_id IN ('${seededIds.join("','")}')`, + signal, ); return Number((r[0] as any).cnt) === seededIds.length; }, 15_000); diff --git a/tests/e2e/sdk/setup.ts b/tests/e2e/sdk/setup.ts index 2158846b..22862253 100644 --- a/tests/e2e/sdk/setup.ts +++ b/tests/e2e/sdk/setup.ts @@ -12,6 +12,8 @@ * policy that covers every generated table. */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; import { CH_URL, makeJWT, WH_URL } from "./helpers.js"; import { allTableSpecs, TABLE_DDL } from "./tables.js"; @@ -96,8 +98,38 @@ async function bootstrapTestPolicy(): Promise { } } +/** + * Print the runtime this suite is actually running on, and say so loudly when + * it isn't the one CI pins. + * + * Worth the four lines: a Node-version-specific transport bug (undici 8.8-8.9 + * stalling on idle pooled connections, nodejs/undici#5600) cost days of + * investigation that started from "my machine must be broken", because nothing + * in the output distinguished a local run from CI's. `.nvmrc` is read by CI's + * setup-node but is inert locally unless you use a version manager, so the two + * can drift silently. See #440. + */ +function reportRuntime(): void { + const nvmrc = (() => { + try { + return readFileSync(join(import.meta.dirname, "../../../.nvmrc"), "utf8").trim(); + } catch { + return ""; + } + })(); + const undici = process.versions.undici ? ` (undici ${process.versions.undici})` : ""; + console.log(` node ${process.version}${undici}`); + + const major = process.version.replace(/^v/, "").split(".")[0]; + if (nvmrc && major !== nvmrc.replace(/^v/, "").split(".")[0]) { + console.log(` ⚠ CI pins node ${nvmrc} (.nvmrc) — this run is on a different major.`); + console.log(` A failure here may not reproduce in CI, and vice versa.`); + } +} + export async function setup(): Promise { console.log(`\n🔍 E2E setup`); + reportRuntime(); console.log(` CLICKHOUSE_URL=${CH_URL}`); console.log(` WAVEHOUSE_URL=${WH_URL}`); diff --git a/tests/e2e/sdk/stress.test.ts b/tests/e2e/sdk/stress.test.ts index bbc51d63..d42ff6ed 100644 --- a/tests/e2e/sdk/stress.test.ts +++ b/tests/e2e/sdk/stress.test.ts @@ -40,9 +40,10 @@ describe("Stress & Concurrency", () => { // Verify all data landed intact // Because we inserted exactly 500 items, ingest worker should flush immediately, // making this relatively fast. - await waitForCondition(async () => { + await waitForCondition(async (signal) => { const r = await chQuery( `SELECT count() as cnt FROM default.${T.clicks} WHERE session_id = 'session-${runId}'`, + signal, ); return Number((r[0] as any).cnt) === concurrency * insertsPerWorker; }, 10_000); diff --git a/tests/e2e/sdk/vitest.config.ts b/tests/e2e/sdk/vitest.config.ts index 049ef437..887926ff 100644 --- a/tests/e2e/sdk/vitest.config.ts +++ b/tests/e2e/sdk/vitest.config.ts @@ -5,7 +5,8 @@ import { defineConfig } from "vitest/config"; // e2e suite's SDK coverage lands at tmp/coverage/ts-e2e/, ready for // `cov ts-merge` (run via `make cov`) to combine with ts-unit. // Standalone runs (no env var) fall back to /coverage. -const reportsDirectory = process.env.TS_E2E_COVERAGE_DIR ?? path.join(__dirname, "coverage"); +const reportsDirectory = + process.env.TS_E2E_COVERAGE_DIR ?? path.join(import.meta.dirname, "coverage"); // Root at the REPO ROOT, not this config's dir. The e2e tests live here // (tests/e2e/sdk) but they exercise SDK source at clients/ts/src, which is @@ -16,9 +17,9 @@ const reportsDirectory = process.env.TS_E2E_COVERAGE_DIR ?? path.join(__dirname, // resulting coverage-final.json keys are absolute paths identical to // ts-unit's — exactly what `cov ts-merge` (nyc merge) needs to combine the // two suites into ts-total. -const repoRoot = path.resolve(__dirname, "../../.."); +const repoRoot = path.resolve(import.meta.dirname, "../../.."); const sdkSrc = path.join(repoRoot, "clients/ts/src"); -const here = path.relative(repoRoot, __dirname); // "tests/e2e/sdk" +const here = path.relative(repoRoot, import.meta.dirname); // "tests/e2e/sdk" // Under COV_DEFER (set by `make ci`/`test-all`), drop the console reporter — // `make cov` (scripts/cov report) prints ONE consolidated table at the end. @@ -41,9 +42,9 @@ export default defineConfig({ // to this dir so the SDK's own *.test.ts unit files under clients/ts/src // are NOT pulled into the e2e run. include: [`${here}/*.test.ts`], - // Absolute (via __dirname) so they resolve regardless of root/cwd. - setupFiles: [path.join(__dirname, "polyfills.ts")], - globalSetup: path.join(__dirname, "setup.ts"), + // Absolute (via import.meta.dirname) so they resolve regardless of root/cwd. + setupFiles: [path.join(import.meta.dirname, "polyfills.ts")], + globalSetup: path.join(import.meta.dirname, "setup.ts"), testTimeout: 30_000, hookTimeout: 120_000, pool: "forks", From c6588887fda2564b8c4aecb296abe7e947bd9dfb Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Wed, 12 Aug 2026 00:18:39 -0400 Subject: [PATCH 02/11] fix(sdk): raise engines.node to the version we actually test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @wavehouse/sdk advertised node >=18. Nothing tests 18, and both 18 and 20 are past end-of-life upstream — so the floor promised to consumers was neither supported nor backed by evidence. The rest of the workspace already requires >=22, and .nvmrc pins 22 for CI, so 22 is the oldest line that is actually exercised. Docs updated to match, since the runtime-support section quoted the old minimum verbatim. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014EJvon2pyU4zaSHTGJj8fE --- clients/ts/package.json | 2 +- docs/src/content/docs/sdk/index.mdx | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/clients/ts/package.json b/clients/ts/package.json index e793c42e..b11bbd13 100644 --- a/clients/ts/package.json +++ b/clients/ts/package.json @@ -23,7 +23,7 @@ "dist" ], "engines": { - "node": ">=18" + "node": ">=22" }, "publishConfig": { "access": "public" diff --git a/docs/src/content/docs/sdk/index.mdx b/docs/src/content/docs/sdk/index.mdx index 38fbb38b..b5b16d65 100644 --- a/docs/src/content/docs/sdk/index.mdx +++ b/docs/src/content/docs/sdk/index.mdx @@ -146,8 +146,9 @@ and `EventSource` are built into every modern browser; no polyfills are required. **Node.js** — non-streaming features (`wh.from().fetch()`, `.insert()`, -`.sql()`, pipes, admin) work in Node 18 and later (the package's minimum, -per `engines.node`). Streaming (`.stream()`, `.liveQuery()`) uses +`.sql()`, pipes, admin) work in Node 22 and later (the package's minimum, +per `engines.node`). 22 is the oldest line we test against, and older +releases are past end-of-life upstream. Streaming (`.stream()`, `.liveQuery()`) uses `EventSource`, which is **not** a default global in Node. The SDK feature-detects with `typeof EventSource === "undefined"` and throws a descriptive error if it is absent. To use streaming in Node you must From 31b9b42cafd4000a63bf3c832e7a7f196df0910f Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Wed, 12 Aug 2026 00:30:02 -0400 Subject: [PATCH 03/11] fix(test): kill a leftover server instead of refusing to run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up on the stale-server guard. Refusing to start is right on a laptop, where the message names the PID and the kill command — but on a shared runner a process orphaned by a canceled job would wedge every subsequent e2e run until someone got shell access. A match is by construction this repo's own cover binary from a dead run, and the very next statement wipes tmp/data out from under it regardless, so killing it is both safe and what the guard was protecting against. Logged loudly; a kill that fails still aborts with the manual command. Also loosens the poll-stat assertion in helpers.test.ts, which put a 999ms *upper* bound on a wall-clock measurement taken while ClickHouse and the server share the machine — the exact shape of flake this branch exists to remove, and against the file header's own "order-of-magnitude, not milliseconds" rule. The property under test is that the in-flight poll was counted at all, so a lower bound is sufficient. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014EJvon2pyU4zaSHTGJj8fE --- scripts/orchestrator/main.go | 44 +++++++++++++++++++++++++---------- tests/e2e/sdk/helpers.test.ts | 19 ++++++++++----- 2 files changed, 45 insertions(+), 18 deletions(-) diff --git a/scripts/orchestrator/main.go b/scripts/orchestrator/main.go index 46c26c1d..6af8407f 100644 --- a/scripts/orchestrator/main.go +++ b/scripts/orchestrator/main.go @@ -67,22 +67,42 @@ func run() error { return fmt.Errorf("%s missing — run `make build-cover` first", binPath) } - // Refuse to start alongside a previous run's server. Both would use the - // JetStream/pebble state under tmp/data and both would write - // tmp/wavehouse-cov.log, so the survivor corrupts this run's state and + // Clear any server left over from a previous run before touching shared + // state. Both would use the JetStream/pebble state under tmp/data and both + // would write tmp/wavehouse-cov.log, so a survivor corrupts this run and // interleaves its output into this run's log — which surfaces as a dozen - // unrelated tests failing to see their rows, in a log that blames the - // wrong ClickHouse. `make test-e2e` cleans up after itself; a leftover - // means the previous run was killed (a harness timeout, a stop button, an - // impatient SIGKILL) rather than interrupted. Fail loudly instead of - // producing a mystery. + // unrelated tests failing to see their rows, in a log that blames the wrong + // ClickHouse. `make test-e2e` cleans up after itself; a leftover means the + // previous run was killed (a harness timeout, a stop button, an impatient + // SIGKILL) rather than interrupted. + // + // Kill rather than refuse: a match is by construction this repo's own cover + // binary from a dead run, the very next statement wipes the data dir out + // from under it anyway, and refusing would wedge every subsequent run on a + // shared CI runner until someone got shell access. Loud, because silently + // killing processes should never be a surprise. if stale, err := staleServerPIDs(ctx, binPath); err != nil { log.Printf(" (could not check for leftover servers: %v)", err) } else if len(stale) > 0 { - return fmt.Errorf( - "a previous wavehouse-cov is still running (pid %s) — it shares tmp/data and "+ - "tmp/wavehouse-cov.log with this run and will corrupt it.\n kill it with: kill %s", - strings.Join(stale, " "), strings.Join(stale, " ")) + log.Printf("! killing %d leftover wavehouse-cov process(es) from a previous run: %s", + len(stale), strings.Join(stale, " ")) + log.Printf(" (they share tmp/data and tmp/wavehouse-cov.log with this run)") + for _, pid := range stale { + n, convErr := strconv.Atoi(pid) + if convErr != nil { + continue + } + proc, findErr := os.FindProcess(n) + if findErr != nil { + continue + } + if killErr := proc.Kill(); killErr != nil { + return fmt.Errorf( + "leftover wavehouse-cov (pid %s) could not be killed: %w\n"+ + " it will corrupt this run — kill it manually with: kill -9 %s", + pid, killErr, pid) + } + } } coverDir := filepath.Join(repoRoot, "tmp", "coverage", "e2e", "data") diff --git a/tests/e2e/sdk/helpers.test.ts b/tests/e2e/sdk/helpers.test.ts index 8f28f60c..edf280c5 100644 --- a/tests/e2e/sdk/helpers.test.ts +++ b/tests/e2e/sdk/helpers.test.ts @@ -76,12 +76,19 @@ describe("waitForCondition", () => { }); it("counts a poll still in flight at the deadline as the slowest", async () => { - await expect( - waitForCondition(async () => { - await sleep(5_000); - return false; - }, 500), - ).rejects.toThrow(/1 poll\(s\), slowest [4-9]\d\dms/); + const err = await waitForCondition(async () => { + await sleep(5_000); + return false; + }, 500).catch((e: Error) => e); + + // Lower bound only. The property under test is that the in-flight poll was + // counted at all — an upper bound would put a wall-clock ceiling on a + // measurement taken while ClickHouse and the server share this machine, + // which is the exact shape of flake this file exists to remove. + expect(err).toBeInstanceOf(Error); + const match = /1 poll\(s\), slowest (\d+)ms/.exec((err as Error).message); + expect(match).not.toBeNull(); + expect(Number(match?.[1])).toBeGreaterThanOrEqual(400); }); it("propagates a rejecting fn instead of masking it as a timeout", async () => { From cb4618322508b55869a7fb6144baa66c30590c5c Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Wed, 12 Aug 2026 00:30:11 -0400 Subject: [PATCH 04/11] docs: record the Node floor change and the E2E harness knobs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documentation sync for the two preceding commits. CHANGELOG gains [Unreleased] entries for both. It needed them badly: nothing in this repo has been released, so the Unreleased section *is* the shipping description, and it still advertised "engines.node is relaxed from >=22 to >=18 ... no longer warns or fails to install on Node 18/20" — the exact opposite of what now ships. That bullet is annotated as superseded rather than rewritten, so the decision history stays readable. pnpm-workspace.yaml's engineStrict rationale was the last place still asserting the two-tier ">=18 for consumers, >=22 for us" policy. development.md described an E2E harness that does not exist: a setup.ts that "probes ports before starting Docker services" (it probes, then throws) and a `make dev` detection that reuses a healthy :8080 (the orchestrator always provisions its own stack on a random port). Replaced with what the code does, plus the supported way to run vitest against a hand-run stack, the new leftover-server behavior, and a table of the env knobs this branch adds (E2E_CH_QUERY_TIMEOUT_MS, E2E_NO_COVERAGE) alongside the existing V=1. The two exhaustive E2E test-file lists (development.md, sdk/reference.md) gained helpers, noting it is a stack-free unit test of the harness rather than a pipeline test, so it doesn't read as inconsistent with the surrounding "exercises the full pipeline" claim. clients/ts/README.md is the page npm renders, and stated no Node requirement at all — a consumer on 20 now hits EBADENGINE with nothing to explain it. sdk/queries.md dropped a "Node 20+" qualifier that sits below the supported floor, and sdk/index.mdx now says 22 is the only line tested rather than the oldest, which implied a matrix we don't run. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014EJvon2pyU4zaSHTGJj8fE --- CHANGELOG.md | 6 +++++- clients/ts/README.md | 2 ++ docs/src/content/docs/development.md | 16 +++++++++++++--- docs/src/content/docs/sdk/index.mdx | 12 ++++++------ docs/src/content/docs/sdk/queries.md | 2 +- docs/src/content/docs/sdk/reference.md | 2 +- pnpm-workspace.yaml | 5 +++-- 7 files changed, 31 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a397980..cacc0fe5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **`@wavehouse/sdk` `engines.node` floor back to `>=22`, matching the only line we test** (`clients/ts/package.json`, `clients/ts/README.md`, `docs/src/content/docs/sdk/index.mdx`, `docs/src/content/docs/sdk/queries.md`, `pnpm-workspace.yaml`): the floor was relaxed to `>=18` when the browser-first distribution landed (see the entry below), on the reasoning that the runtime needs only `fetch`. Nothing ever tested 18, though — `.nvmrc` pins 22 and `.github/actions/setup-env` consumes it via `node-version-file`, so 22 is the single version CI exercises — and Node 18 and 20 have both since reached upstream end-of-life. Declaring a floor we neither test nor is supported upstream promises more than it can back, so it returns to `>=22`. **Consumer impact:** installing on Node < 22 now warns with `EBADENGINE` under npm, and fails outright under pnpm with `engine-strict` enabled. The SDK README and the docs' Runtime support section state the requirement, which they previously either omitted or quoted as 18. + - **Live SSE events are now projected and serialized once per role instead of once per subscriber** (`internal/stream/hub.go` (new), `internal/stream/{subscriber,bucket,heartbeat,metrics,doc}.go`, `internal/api/stream.go`, `internal/api/hub.go` + `internal/api/transform.go` (both removed — the broadcast hub moves to `internal/stream`, and the orphaned test-only `transformForClient` is dropped), `cmd/wavehouse/main.go`, `docs/src/content/docs/architecture.md`, `AGENTS.md`, plus tests in `internal/stream/{hub,filter,subscriber,bucket,heartbeat}_test.go` and `internal/api/{stream,transform,router,errors}_test.go`): the first PR of the SSE delivery-path throughput epic ([#294](https://github.com/Wave-RF/WaveHouse/issues/294)), building on the `internal/stream` primitives from #346. The broadcast hub moves into `internal/stream` as `Hub`: subscribers register under `(topic, role)`, and `Broadcast` decodes each event **once**, applies each subscribed role's column policy **once**, builds one SSE frame per role, and fans it to every member of that role's `Bucket`. Previously every connection independently ran `json.Unmarshal → policy.Evaluate → filterEventColumns → json.Marshal` (plus a second unmarshal just to read the `id:` timestamp) on the *same* event in its own read loop — byte-identical work repeated N times. For a single-role audience (the public dashboard, every viewer `public`) that collapses N re-projections to 1, moving the measured ~2 270 deliveries/s ceiling toward an events/s ceiling. The `(topic, role)` key is sufficient and claims-independent: column visibility derives only from the role+table policy entry, and the stream path applies no row-level filter (a documented invariant — if row-level filtering is ever added to streaming, the key must take claims into account). The handler's two `select` cases (keepalive vs. per-subscriber event) collapse into one byte-pump over a single `Subscriber.Frames()` queue carrying typed `Frame`s; the subscriber queue grows from cap 1 (keepalive-only) to 64 so live events buffer while the handler is mid-write. Gap-fill replay and `Last-Event-ID`/`?since=` resumption are unchanged (replay stays per-connection via the shared `stream.ReplayFrame`; live frames carry the same `id: `). Slow-consumer drops, silent before, now increment `wavehouse_sse_dropped_frames_total`; an inert `Subscriber.Evicted()` seam is wired for the eviction follow-up. The per-delivery OpenTelemetry span (another #294 item) was already removed in #346. **Deferred to follow-ups:** active slow-consumer eviction (#94) and right-sizing the subscriber buffer + broadcast lock cost (#152). - **CI is now a job DAG instead of one monolithic job, and the docs deploys no longer expose the Cloudflare token to PR-authored code** (`.github/workflows/ci.yml`, `.github/workflows/housekeeping.yml`, `.github/actions/setup-env/action.yml`, `Makefile`, `docs/wrangler.jsonc`, `AGENTS.md`, `docs/src/content/docs/development.md`, `docs/src/content/docs/claude-code.md`, `CONTRIBUTING.md`, `scripts/lint-pr-title.sh`, `.claude/hooks/agent-bash-gate.sh`): closes #305. The single `make ci` job becomes parallel jobs over the *same Makefile targets* (local `make ci` stays the dev mirror): `lint`, `unit`, `integration`, `e2e` (builds its own SDK dist + cover binary via `make -j test-e2e` on a warm per-suite cache and runs the suite exactly like a local run), `coverage` (a dedicated job that merges every suite's `coverage-` fragment and applies every threshold gate via `make cov` — like local `make ci`'s final step, so the gate is decoupled from the e2e suite; it's `needs: changes` only and *polls* for the fragments rather than `needs`-ing the suites, so its setup overlaps them and the merge fires ~10s after the last suite instead of serializing ~50s of setup onto the critical path), and `docs-build` (`make build-docs`, docs-affecting changes only, uploads the docs dist artifact the preview/deploy jobs consume) — public-repo runners are free and 4-core, so the pipeline spreads horizontally instead of queueing in one process. The architecture is documented once, in `.github/workflows/README.md` (DAG diagram, design invariants, cache key policy, add-a-job recipe, and the measured-but-deferred optimizations — e2e sharding among them), and the workflow's logic lives in shellcheck-gated scripts (`scripts/ci/` — `classify-changes.sh`, `check-pr-title.sh`, `docs-preview-comment.sh`, `timing-summary.sh`, `wait-artifact.sh`; over the shared, dependency-free path classifier `scripts/classify-paths.sh`, unit-tested by `scripts/classify-paths.test.sh` via `make test-classify-paths` and reused by the `pre-push` git hook so a docs/prose-only push requires only `make verify`, not a full `make ci` — the same suites CI skips for those changes) rather than inline YAML; caches are owned end-to-end by `setup-env` via nested `actions/cache` (automatic post-job saves — the per-job save-step boilerplate is gone); a non-gating `Timing summary` job writes a per-job wall-clock table to every run's Summary page; and `make verify` gains two leaves that gate the new surface area — `lint-sh` (shellcheck `v0.11.0`, checksum-verified install via `scripts/install-shellcheck.sh`) and `lint-gha` (actionlint `v1.7.12`) — so the CI plumbing is linted like any other source. The workflow also handles `merge_group` events (full suite against the merge-group ref), enabling a **merge queue** on `main`: the queue re-tests each PR against current main at landing time, which replaces the ruleset's "require branches to be up to date" rule — no more manual branch updates after every sibling merge. A new aggregator job named `CI` is the ruleset's **sole required status check** (it fails on any failed/cancelled job and counts skipped jobs as passing), so docs-only PRs skip the Go suites without orphaning the gate and future job changes never require ruleset edits. The PR-title (Conventional Commits) gate moves into the `PR title` job under that aggregator, validated by the same `scripts/lint-pr-title.sh` from a trusted `main` checkout; `PR housekeeping` (`pull_request_target`) drops to non-required and keeps what needs fork-PR write access — path labels, the sticky title-explainer comment, and a new nudge that re-runs the failed `PR title` job when a title edit fixes it (the job re-reads the title from the API, so no new push is needed). The **#305 fix**: docs previews/production deploys run in dedicated `docs-preview`/`docs-deploy` jobs that check out trusted `main` (wrangler, worker source, and config never resolve from the PR tree), consume only the static `docs/dist` artifact, and are the only jobs that reference `CLOUDFLARE_*` secrets; previews now publish right after `docs-build` instead of waiting on the full test pipeline, and the `docs-preview` deploy is **non-gating** — it's not in the `CI` aggregator's `needs` (only `docs-build` gates), so a slow or failed Cloudflare preview reports its own "Docs preview" check but never delays or reds the required check; production (`docs-deploy`, on the post-merge main push) still requires everything green. Per-job least-privilege permissions replace the old workflow-wide `contents: write`, and the Go build cache is partitioned per job (unit/integration/e2e compile with different flags) so each suite stays warm. @@ -51,6 +53,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- **The E2E harness enforces its own poll budgets, and no longer inherits an idle pooled connection** (`tests/e2e/sdk/helpers.ts`, `tests/e2e/sdk/helpers.test.ts` (new), `tests/e2e/sdk/setup.ts`, `tests/e2e/sdk/vitest.config.ts`, `tests/e2e/sdk/{batching,cache,dlq,ingest,ndjson,query,stress}.test.ts`, `scripts/orchestrator/main.go`, `docs/src/content/docs/development.md`, `docs/src/content/docs/sdk/reference.md`): closes #440. Two defects, the first of which hid the second. `waitForCondition` checked the clock only on loop entry, so a single slow `fn()` overran the advertised budget without bound — a 10s budget was measured running 28s, past the caller's `testTimeout`, so vitest killed the test first and reported a timeout naming neither the condition nor how long the poll actually waited. It now races `fn()` against the deadline, aborts the in-flight call via an `AbortSignal` handed to `fn`, and reports poll shape on failure (`N poll(s), slowest Xms`) — which separates "the write never landed" (many fast polls) from "the polling itself was starved" (few slow ones). That reporting is what exposed the second defect: `chQuery` used the global `fetch`, which reuses pooled connections, and undici 8.8.0–8.9.0 stalls for seconds before writing a request onto a socket that has been idle a few seconds ([nodejs/undici#5600](https://github.com/nodejs/undici/issues/5600), a scheduling regression in `scheduleIdleSocketValidation()`, fixed in 8.10.0). This suite has multi-second idle gaps by construction — the 5s ingest linger sits between every write and the first poll of its visibility wait — so every visibility wait sat in the triggering window; local `make test-e2e` went from 2 pass/3 fail to 5 pass/0 fail, and every run got faster (115.7–124.5s vs 128.7–137.1s). Node 26 bundles undici 8.9.0 while CI runs Node 22 (undici 6.28.0) via `.nvmrc`, which is why this was invisible to CI and to developers on other Node lines. `chQuery` additionally takes a per-request ceiling (`E2E_CH_QUERY_TIMEOUT_MS`, default 10 000 ms) and honours the caller's signal, threaded through 19 call sites, so an abandoned poll tears its request down rather than running on unobserved. Also here: `batching`'s visibility wait had ~700ms of headroom over the 5s linger where every other wait allows 10s (widened — the `>= 4500ms` lower bound that carries the test's meaning is unchanged); the E2E setup banner prints the active node/undici version and warns when the local major differs from `.nvmrc`; the orchestrator refuses to start beside an orphaned `wavehouse-cov` (a killed run leaves one, and it corrupts the next run through the shared `tmp/data` and log file, presenting as a dozen unrelated tests failing to see their rows in a log that blames a container which no longer exists); a new `E2E_NO_COVERAGE=1` drops `--coverage` for local debugging only; and `vitest.config.ts` moves from `__dirname` to `import.meta.dirname`, silencing the Vite 8 `configLoader: 'native'` warning. + - **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`, `Makefile`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every `ci.yml` Go job that goes through `setup-env`, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost roughly 2.5–7 minutes on every push to main (mean delta ≈4.8 min). The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Because those timings were taken with setup-go's bundled entry (which also held `~/go/pkg/mod`), `publish-dev` additionally *restores* `gomod-v1` from `main`'s scope via `actions/cache/restore` — read-only, so it costs no budget and cannot write a partial tree to the key every `ci.yml` Go job shares. Without that restore the job would re-download ~112 MB of modules per push and land above the warm range quoted above. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. Two guards come with the shared entry: `setup-env` now fails a `go: true` job that passes no `go-cache-suffix` (an empty one yields a restore-key prefix-matching every other flavor), and `make cov` gains the `go-mod-download` prerequisite its siblings already had — CI's coverage job shares the unsuffixed `gomod-v1` and races to save it, but ran only `go run ./scripts/cov report`, so winning that race would have stored a partial `~/go/pkg/mod` that then exact-hit for every other job until the next rotation. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **A path prefix in the SDK's `baseURL` now survives instead of being silently discarded** (`clients/ts/src/url.ts` (new), `clients/ts/src/http.ts`, `clients/ts/src/stream/sse.ts`, `clients/ts/src/cli/codegen.ts`, `clients/ts/src/url.test.ts` (new), `clients/ts/src/stream/sse.test.ts` (new), `clients/ts/src/{http,client}.test.ts`, `docs/src/content/docs/sdk/index.mdx`, `docs/src/content/docs/reverse-proxy.mdx`): closes #428. Pointing the SDK at a WaveHouse served under a prefix — `createClient({ baseURL: 'https://app.example.com/api/warehouse' })`, the shape you get behind a BFF, an app-server route, or a path-routed ingress — dropped the prefix from every request. Both transports resolved *absolute* request paths against the base (`new URL('/v1/query', base)` in `http.ts`, `new URL('/v1/stream', baseURL)` in `stream/sse.ts`), and per the URL spec an absolute path replaces the base's path entirely, so calls went to the origin root. The failure mode was the bad kind: no error, a client that looks correctly configured, and every request quietly going somewhere else — with no workaround from outside the SDK, since `baseURL` was the only path input and it couldn't survive. Request paths are now joined **onto** the base by a single shared `resolveURL` helper that both transports and the codegen CLI call (previously three separate constructions, one of which — codegen's string concat — already handled prefixes, so they disagreed). The helper normalizes the base to a directory before resolving, so a bare last segment or a stray query/fragment on `baseURL` can't eat the prefix either, and a root-hosted base (`http://localhost:8080`, the overwhelmingly common case) resolves exactly as before. Tests pin a prefixed base end-to-end across both transports. The proxy in front must still strip the prefix before forwarding — WaveHouse has no configurable base path by design — which the reverse-proxy guide now covers with nginx/Caddy snippets. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. @@ -105,7 +109,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added -- **Browser-first SDK distribution: an IIFE global build, CDN entry points, a `wavehouse-codegen` bin, and a Node 18 floor** (`clients/ts/tsup.config.ts`, `clients/ts/package.json`, `clients/ts/README.md`, `docs/src/content/docs/sdk.md`, `docs/src/content/docs/development.md`, `pnpm-workspace.yaml`): `@wavehouse/sdk` already shipped browser-ready ESM/CJS (zero deps, native `fetch`/`EventSource`) but documented only the `npm install` + bundler path. The build now also emits a minified, self-contained **IIFE bundle** (`dist/index.global.js`) that defines a `WaveHouse` global, wired to new `unpkg`/`jsdelivr` package fields — so `` then `WaveHouse.createClient({ … })` works on a no-build, FTP-deployed page — and the SDK README + `sdk.md` gain a "No build step (CDN)" section covering both the ESM-CDN (`` then `WaveHouse.createClient({ … })` works on a no-build, FTP-deployed page — and the SDK README + `sdk.md` gain a "No build step (CDN)" section covering both the ESM-CDN (`