fix(test): enforce e2e poll budgets and stop reusing idle connections

clients/ts/README.md

@@ -8,6 +8,8 @@ Zero-dependency TypeScript client for [WaveHouse](https://github.com/Wave-RF/Wav
 npm install @wavehouse/sdk
 ```
 
+Requires Node 22 or newer — the only line this SDK is tested against; Node 18 and 20 are past upstream end-of-life. Browsers need no polyfill, and Node needs an `EventSource` polyfill only for streaming — see [Runtime support](https://wavehouse.dev/sdk/#runtime-support).
+
 This works in any framework that uses a bundler — React, Vue, Svelte, Angular, Astro, SolidJS, or plain Vite — with `import { createClient } from '@wavehouse/sdk'`.
 
 The package is published with [npm provenance](https://docs.npmjs.com/generating-provenance-statements) — npm shows a verified provenance badge on the [package page](https://www.npmjs.com/package/@wavehouse/sdk), and `npm audit signatures` checks the attestation against this repository's signed build.

clients/ts/package.json

@@ -23,7 +23,7 @@
     "dist"
   ],
   "engines": {
-    "node": ">=18"
+    "node": ">=22"
   },
   "publishConfig": {
     "access": "public"

docs/src/content/docs/development.md

@@ -360,7 +360,7 @@ The primary E2E integration test suite lives in `tests/e2e/sdk/`. It uses the Ty
 **Architecture**:
 
 - `scripts/orchestrator` — the E2E entrypoint behind `make test-e2e`: it starts a clean ClickHouse **testcontainer** per run, launches the `wavehouse-cov` binary on a random free port, runs the SDK suite against it, then SIGINTs the binary to flush coverage. No Compose file is involved. CI runs the exact same path.
-- `tests/e2e/sdk/setup.ts` — Smart `globalSetup` that probes ports before starting Docker services, so tests work seamlessly whether you started services manually or let the setup do it.
+- `tests/e2e/sdk/setup.ts` — `globalSetup`. Probes the `CLICKHOUSE_URL` / `WAVEHOUSE_URL` the orchestrator injects, creates the per-suite tables, refreshes the schema, and bootstraps the baseline policy. It starts nothing itself and fails fast if either URL isn't up. It also prints the active Node/undici version, warning when the local Node major differs from `.nvmrc` — a runtime-specific transport bug is otherwise indistinguishable from a code failure (see [#440](https://github.com/Wave-RF/WaveHouse/issues/440)).
 - `tests/e2e/sdk/helpers.ts` — JWT factories, typed client constructors, async wait helpers, direct ClickHouse query helper.
 
 **Running E2E tests**:
@@ -372,9 +372,29 @@ make test-e2e
 
 `make test-e2e` builds `bin/wavehouse-cov` (coverage-instrumented) and runs the orchestrator under `scripts/orchestrator/` to wire ClickHouse + the cover binary into the suite. covdata flushes on SIGINT into `tmp/coverage/e2e/data/`.
 
-**If you already have `make dev` running**, the setup detects the healthy WaveHouse on `:8080` and skips starting it via Docker — only ClickHouse is started if needed.
+The orchestrator always provisions its own stack — a fresh ClickHouse testcontainer plus `wavehouse-cov` on a random free port — so a running `make dev` on `:8080` is neither detected nor reused, and the two don't collide. To run vitest against a stack you manage yourself, start the server from the **repo root** with the E2E fixture config:
 
-**Test files** (`tests/e2e/sdk/*.test.ts`): `admin`, `auth`, `batching`, `cache`, `dlq`, `ingest`, `ndjson`, `query`, `streaming`, `stress`.
+```bash
+WH_CONFIG=tests/e2e/fixtures/config.yaml go run ./cmd/wavehouse
+```
+
+The fixture matters: the suite signs its tokens with its `sdk-dev-secret` and depends on its dedupe, DLQ, and 5s schema-refresh settings. Point the suite at a default `make dev` server (`jwt_secret: change-me-in-production`) and setup's schema calls are rejected, then global setup dies 30s later on a misleading `schema not refreshed within 30s`. The repo root matters too — the fixture's `policy.file_path` is relative to the working directory. The fixture pins no ClickHouse address, so the server looks for one on `localhost:9000`; point it elsewhere with `WH_CH_ADDR` / `WH_CH_HTTP_PORT` if yours isn't there.
+
+Prefixing the variable to `make dev` does **not** work: that recipe pins `WH_CONFIG=.config.local.yaml` inline, which overrides anything inherited from the environment.
+
+Then set `CLICKHOUSE_URL` / `WAVEHOUSE_URL` and run `pnpm test` from `tests/e2e/sdk/`; teardown is a no-op on that path, so your stack survives between iterations.
+
+If a previous run was killed (harness timeout, stop button, `SIGKILL`), it can leave a `wavehouse-cov` behind. That process shares `tmp/data` and `tmp/wavehouse-cov.log` with the next run and will corrupt it, so the orchestrator kills any leftover before starting and says so.
+
+**Environment knobs**:
+
+| Variable | Effect |
+|----------|--------|
+| `V=1` | Stream the WaveHouse subprocess log live *in addition to* capturing it to `tmp/wavehouse-cov.log`. The on-failure log excerpt is then skipped — you have already seen it |
+| `E2E_CH_QUERY_TIMEOUT_MS` | Per-request ceiling for the suite's direct ClickHouse queries (default `10000`) |
+| `E2E_NO_COVERAGE=1` | Drop `--coverage` from the vitest run (skips v8 instrumentation and report generation) while chasing a flake. **Local debugging only** — no report is written. Ignored (with a log line) under `make ci` / `make test-all`, so an exported-and-forgotten var can't produce a green coverage gate with the TS e2e report missing |
+
+**Test files** (`tests/e2e/sdk/*.test.ts`): `admin`, `auth`, `batching`, `cache`, `dlq`, `ingest`, `ndjson`, `query`, `streaming`, `stress`, plus `helpers` — a stack-free unit test of the harness's own `waitForCondition` poll helper rather than a pipeline test.
 
 ## Linting
 

docs/src/content/docs/sdk/index.mdx

@@ -146,12 +146,13 @@ and `EventSource` are built into every modern browser; no polyfills are
 required.
 
 **Node.js** — non-streaming features (`wh.from().fetch()`, `.insert()`,
-`.sql()`, pipes, admin) work in Node 18 and later (the package's minimum,
-per `engines.node`). Streaming (`.stream()`, `.liveQuery()`) uses
-`EventSource`, which is **not** a default global in Node. The SDK
-feature-detects with `typeof EventSource === "undefined"` and throws a
-descriptive error if it is absent. To use streaming in Node you must
-polyfill before calling any stream method:
+`.sql()`, pipes, admin) work in Node 22 and later (the package's minimum,
+per `engines.node`). 22 is the only line we test against, and older
+releases are past end-of-life upstream. Streaming (`.stream()`,
+`.liveQuery()`) uses `EventSource`, which is **not** a default global in
+Node. The SDK feature-detects with `typeof EventSource === "undefined"`
+and throws a descriptive error if it is absent. To use streaming in Node
+you must polyfill before calling any stream method:
 
 ```js
 import { EventSource } from 'eventsource'; // npm install eventsource

docs/src/content/docs/sdk/queries.md

@@ -62,7 +62,7 @@ await clicks.insertNDJSON('{"page":"/a"}\n{"page":"/b"}\n');
 // From a browser <input type="file"> (a File is a Blob)
 await clicks.insertNDJSON(fileInput.files[0]);
 
-// From a Node file (Node 20+: fs.openAsBlob; or read it to a string)
+// From a Node file (fs.openAsBlob; or read it to a string)
 import { openAsBlob } from 'node:fs';
 await clicks.insertNDJSON(await openAsBlob('events.ndjson'));
 ```

docs/src/content/docs/sdk/reference.md

@@ -153,6 +153,6 @@ The SDK doubles as the E2E integration test harness. Tests in `tests/e2e/sdk/` e
 make test-e2e
 ```
 
-Test files live in `tests/e2e/sdk/`: `admin`, `auth`, `batching`, `cache`, `dlq`, `ingest`, `ndjson`, `query`, `streaming`, `stress` (each `*.test.ts`).
+Test files live in `tests/e2e/sdk/` (each `*.test.ts`): `admin`, `auth`, `batching`, `cache`, `dlq`, `ingest`, `ndjson`, `query`, `streaming`, `stress`, plus `helpers` — a stack-free unit test of the harness's own `waitForCondition` poll helper rather than a pipeline test.
 
 See [Development Guide — E2E Tests via SDK](/development#e2e-tests-via-sdk) for architecture details and workflow tips.

pnpm-workspace.yaml

@@ -25,8 +25,9 @@ minimumReleaseAgeExclude:
   - "@wave-rf/*"
 
 # Fail an install when the active Node doesn't satisfy a package's engines.node.
-# The published SDK (clients/ts) declares ">=18" for external consumers; the
-# root, e2e, and docs packages require ">=22" — the dev/CI version pinned in .nvmrc.
+# Every workspace package — including the published SDK (clients/ts) — declares
+# ">=22", matching the dev/CI version pinned in .nvmrc. That is the only line CI
+# exercises, and Node 18/20 are past upstream end-of-life.
 engineStrict: true
 
 catalog:

scripts/orchestrator/main.go

@@ -39,6 +39,7 @@ import (
 	"os/signal"
 	"path/filepath"
 	"strconv"
+	"strings"
 	"syscall"
 	"time"
 
@@ -66,6 +67,47 @@ func run() error {
 		return fmt.Errorf("%s missing — run `make build-cover` first", binPath)
 	}
 
+	// Clear any server left over from a previous run before touching shared
+	// state. Both would use the JetStream/pebble state under tmp/data and both
+	// would write tmp/wavehouse-cov.log, so a survivor corrupts this run and
+	// interleaves its output into this run's log — which surfaces as a dozen
+	// unrelated tests failing to see their rows, in a log that blames the wrong
+	// ClickHouse. `make test-e2e` cleans up after itself; a leftover means the
+	// previous run was killed (a harness timeout, a stop button, an impatient
+	// SIGKILL) rather than interrupted.
+	//
+	// Kill rather than refuse: a match is by construction this repo's own cover
+	// binary from a dead run, the very next statement wipes the data dir out
+	// from under it anyway, and refusing would wedge every subsequent run on a
+	// shared CI runner until someone got shell access. Loud, because silently
+	// killing processes should never be a surprise.
+	if stale, err := staleServerPIDs(ctx, binPath); err != nil {
+		log.Printf("  (could not check for leftover servers: %v)", err)
+	} else if len(stale) > 0 {
+		log.Printf("! killing %d leftover wavehouse-cov process(es) from a previous run: %s",
+			len(stale), strings.Join(stale, " "))
+		log.Printf("  (they share tmp/data and tmp/wavehouse-cov.log with this run)")
+		for _, pid := range stale {
+			n, convErr := strconv.Atoi(pid)
+			if convErr != nil {
+				continue
+			}
+			proc, findErr := os.FindProcess(n)
+			if findErr != nil {
+				continue
+			}
+			// ErrProcessDone means it exited between pgrep and here — the
+			// common case being a Ctrl-C'd run still flushing coverage. That
+			// is the outcome we wanted, not a failure.
+			if killErr := proc.Kill(); killErr != nil && !errors.Is(killErr, os.ErrProcessDone) {
+				return fmt.Errorf(
+					"leftover wavehouse-cov (pid %s) could not be killed: %w\n"+
+						"  it will corrupt this run — kill it manually with: kill -9 %s",
+					pid, killErr, pid)
+			}
+		}
+	}
+
 	coverDir := filepath.Join(repoRoot, "tmp", "coverage", "e2e", "data")
 	if err := os.MkdirAll(coverDir, 0o750); err != nil {
 		return fmt.Errorf("mkdir coverdir: %w", err)
@@ -201,8 +243,25 @@ func run() error {
 	// straight to `pnpm exec vitest run --coverage` skips the script-arg
 	// forwarding layer entirely, matching how scripts/cov invokes `pnpm exec
 	// nyc`.
+	//
+	// E2E_NO_COVERAGE=1 drops it for local debugging only, and is ignored under
+	// the gating targets. Left unconditional, an exported-and-forgotten var
+	// would let `make ci` write a green push marker with the TS e2e report
+	// missing: test-e2e wipes tmp/coverage/ts-e2e first, ts-e2e's own threshold
+	// is informational, and ts-total then gates on ts-unit alone — an `n/a` row
+	// in the table, but a pass. COV_DEFER is exported by exactly the targets
+	// that gate (ci / test-all), so it is the right thing to key off.
+	args := []string{"exec", "vitest", "run", "--coverage"}
+	if os.Getenv("E2E_NO_COVERAGE") == "1" {
+		if os.Getenv("COV_DEFER") != "" {
+			log.Println("  (E2E_NO_COVERAGE=1 ignored — this run feeds a coverage gate)")
+		} else {
+			args = args[:len(args)-1]
+			log.Println("  (E2E_NO_COVERAGE=1 — running without coverage; no report will be written)")
+		}
+	}
 	// #nosec G204 — args are a fixed string slice, not user input.
-	vitest := exec.CommandContext(ctx, "pnpm", "exec", "vitest", "run", "--coverage")
+	vitest := exec.CommandContext(ctx, "pnpm", args...)
 	vitest.Dir = filepath.Join(repoRoot, "tests", "e2e", "sdk")
 	vitest.Env = append(os.Environ(),
 		"WAVEHOUSE_URL="+whURL,
@@ -257,6 +316,26 @@ func run() error {
 	return vitestErr
 }
 
+// staleServerPIDs returns the PIDs of any wavehouse-cov left over from an
+// earlier run. Called before this run starts its own, so every match is stale.
+// A missing/failed pgrep is reported as an error and treated as "unknown" by
+// the caller — this is a guard rail, not a gate.
+func staleServerPIDs(ctx context.Context, binPath string) ([]string, error) {
+	// #nosec G204 — binPath is filepath.Join(repoRoot, "bin", "wavehouse-cov")
+	// with constant components, not user input; it is only ever a search
+	// pattern here, never executed.
+	out, err := exec.CommandContext(ctx, "pgrep", "-f", binPath).Output()
+	if err != nil {
+		// pgrep exits 1 with no output when nothing matches — the common case.
+		var exitErr *exec.ExitError
+		if errors.As(err, &exitErr) && exitErr.ExitCode() == 1 {
+			return nil, nil
+		}
+		return nil, err
+	}
+	return strings.Fields(string(out)), nil
+}
+
 // pickFreePort asks the OS for an available port on 127.0.0.1, then
 // closes the listener so wavehouse-cov can bind to it. There's a brief
 // TOCTOU window where another process could grab the port, but on a

tests/e2e/sdk/batching.test.ts

@@ -33,9 +33,10 @@ describe("Ingest Batching Triggers", () => {
       // the worker's buffer held all 500 rows before the timer could fire —
       // an early flush below is attributable to the size trigger.
       await waitForCondition(
-        async () => {
+        async (signal) => {
           const r = await chQuery(
             `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`,
+            signal,
           );
           return Number((r[0] as any).cnt) === 500;
         },
@@ -61,9 +62,10 @@ describe("Ingest Batching Triggers", () => {
         `ack took ${ackMs}ms (≥4s): linger fired mid-publish; size-trigger timing inconclusive — verifying integrity only`,
       );
       await waitForCondition(
-        async () => {
+        async (signal) => {
           const r = await chQuery(
             `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`,
+            signal,
           );
           return Number((r[0] as any).cnt) === 500;
         },
@@ -95,21 +97,27 @@ describe("Ingest Batching Triggers", () => {
     );
     expect(Number((r[0] as any).cnt)).toBe(0);
 
-    // Wait until it appears
+    // Wait until it appears. The budget is only a "don't hang forever" bound —
+    // the assertion that carries this test's meaning is the >= 4500ms lower
+    // bound below. It used to be 6_000, which left ~700ms over the 5s linger
+    // and made this the most flake-prone wait in the suite; every other
+    // visibility wait allows 10s. Widening costs no test power (an early flush
+    // still fails the lower bound) and removes the false failures (#440).
     await waitForCondition(
-      async () => {
+      async (signal) => {
         const check = await chQuery(
           `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`,
+          signal,
         );
         return Number((check[0] as any).cnt) === 1;
       },
-      6_000,
+      10_000,
       500,
     );
 
     const elapsed = Date.now() - apiEndTime;
 
     // It should take roughly ~5 seconds for ingest worker's period trigger to fire
     expect(elapsed).toBeGreaterThanOrEqual(4500);
-  }, 20_000);
+  }, 25_000);
 });

tests/e2e/sdk/cache.test.ts

@@ -50,9 +50,10 @@ describe("Cache", () => {
 
     // 4. Wait for the async worker to flush to ClickHouse and invalidate the cache.
     // By querying ClickHouse directly, we don't accidentally trigger a cache re-prime!
-    await waitForCondition(async () => {
+    await waitForCondition(async (signal) => {
       const r = await chQuery(
         `SELECT event_id FROM default.${T.clicks} WHERE event_id = '${eventId}'`,
+        signal,
       );
       return r.length === 1;
     }, 10_000);

tests/e2e/sdk/dlq.test.ts

@@ -47,9 +47,10 @@ describe("Dead Letter Queue (DLQ) & Failures", () => {
     // edge under CI load. The structural fix is a lower e2e maxWait (deferred
     // config PR), which drops the 5s-timer dependency; this budget can shrink
     // back once that lands.
-    await waitForCondition(async () => {
+    await waitForCondition(async (signal) => {
       const chRows = await chQuery(
         `SELECT count() as cnt FROM default.${T.clicks} WHERE user_id = 'user-${runId}'`,
+        signal,
       );
       return Number((chRows[0] as any).cnt) === 9;
     }, 10_000);