diff --git a/CHANGELOG.md b/CHANGELOG.md index 019eb53d..82199699 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **HTTP customization for the SDK — `options.headers`, `options.fetchOptions`, and `options.fetch`** (`clients/ts/src/types.ts`, `clients/ts/src/client.ts`, `clients/ts/src/http.ts`, `clients/ts/src/index.ts`, `clients/ts/src/client.test.ts`, `docs/src/content/docs/sdk/index.mdx`): closes #269. `ClientOptions` exposed only `maxRetries`, so a WaveHouse behind a header-gated proxy (Cloudflare Access, an mTLS sidecar, an auth gateway) or a cookie-authenticated origin was unreachable from the SDK — a defense-in-depth gate forced consumers off the client entirely. Three knobs land together, shaped after the conventions in Supabase's, OpenAI's, and Anthropic's clients rather than invented here. **`headers`** adds static headers to every REST request; names match case-insensitively as HTTP requires, and they apply *underneath* the SDK's own — `auth` keeps `Authorization`, and a request's `Content-Type`/`Accept` can't be displaced by a global one, since a header joined rather than replaced is how you ship `Content-Type: application/json, image/png`. **`fetchOptions`** merges extra `RequestInit` fields (`credentials: "include"` for the cookie case, `mode`, `cache`, or a runtime extension like Next.js's `next: { tags }`); the fields the SDK controls — `method`, `headers`, `body`, `signal` — always win, so it cannot corrupt the request. **`fetch`** replaces the HTTP implementation outright. All three are REST-only: `.stream()` and `.liveQuery()`'s live connection go through `EventSource`, which accepts neither headers nor a `fetch`, so a header-gated deployment can query but not stream until #203 changes that transport — stated plainly in the docs rather than left implicit, since the equivalent gap in Supabase's realtime client was found the hard way by a user whose RLS policies silently stopped matching. `.liveQuery()`'s initial backfill is an ordinary REST call and *is* covered. Per-call overrides and dynamic header callbacks are deliberately deferred to #459; the per-call slot (`.fetch(opts)`) already exists, so adding them later is additive. The exported `FetchLike` matches the standard `fetch` signature, written out rather than as `typeof fetch` because that resolves differently depending on whether the consumer's TypeScript `lib` includes DOM — the same fragility behind Supabase's long tail of `node-fetch` resolution issues. `options.fetch` accepts any `fetch`-compatible function (exported as the `FetchLike` type) and is used for every request, retries included, so middleware sees each attempt. The motivating case is a runtime bug consumers can't fix themselves: undici 8.8.0–8.9.0 stalls a request before it goes out when a keep-alive socket is reused while the event loop is idle ([nodejs/undici#5600](https://github.com/nodejs/undici/issues/5600), fixed in 8.10.0), and Node 26 bundles 8.9.0 — so calls that should take milliseconds don't, with no recourse inside the SDK. Severity varies with the runtime and the idle gap: the upstream report measured ~450–465 ms, and our own runs against an instant-answering server have ranged from ~100 ms to tens of seconds. Upgrading undici is the real fix, and `options.fetch` is how you get it without waiting for a new runtime: install undici yourself and route through it, passing its dispatcher **explicitly**. That last part is load-bearing and not obvious — undici keeps its connection pool on a shared `globalThis` symbol claimed by whichever copy loads first (Node claims it for the bundled copy on the first built-in `fetch` call, not at startup), so calling an installed 8.10.0's `fetch` without a `dispatcher` resolves whatever is on that symbol and can still stall through the bundled 8.9.0's pool. Measured with both copies loaded, 1.5 s idle gaps against a 10 ms server: `21, 1514, 1495, 583 ms` with an implied dispatcher versus `17, 14, 12, 13 ms` with an explicit `new Agent()`. (Tuning `keepAliveTimeout` is *not* a workaround — measured, it changes nothing, because the retirement timer is starved by the same idle event loop; `new Agent({ pipelining: 0 })` does work for anyone pinned to an affected version, at a connection per request.) The same hook covers the ordinary reasons an SDK grows one: proxies, client certificates, tracing or circuit-breaker middleware, and mocking HTTP in a consumer's own tests without monkey-patching a global. `fetch` stays optional all the way through to the internal `HttpContext` rather than being defaulted at construction, so the default path still calls the global directly — that keeps it late-bound (replacing `globalThis.fetch` after a client exists still works, which is what `vi.stubGlobal` does) and avoids invoking a detached `fetch` reference, which is not universally safe; both properties are pinned by tests. Only the SSE transport is exempt: the live connection behind `.stream()`/`.liveQuery()` goes through `EventSource`, which this does not replace — but `.liveQuery()`'s initial backfill is an ordinary request and does go through the supplied function. Implementations shipping their own request/response declarations (undici, `node-fetch`) need casts on the URL argument, the init, and the return value, since those types are separate from the ones behind the global `fetch`; the narrow documented runtime contract is what makes them safe — a string URL and plain `RequestInit` in, and only `.ok`/`.headers` plus `.text()` on success and `.status`/`.statusText`/`.json()` on a non-`ok` response read back. Abort handling is part of that contract: `ABORTED` requires a `DOMException` named `AbortError` (platform `fetch` and undici), and any other rejection is retried as `NETWORK_ERROR`. + - **Non-JWT operator key for full-access admin + break-glass recovery** (`internal/auth/auth.go`, `internal/auth/context.go`, `internal/auth/auth_test.go`, `internal/api/router.go`, `internal/api/router_test.go`, `internal/config/config.go`, `internal/config/config_test.go`, `cmd/wavehouse/main.go`, `config.yaml`, `deployments/compose/standalone.yaml`, `tests/e2e/fixtures/config.yaml`, `docs/src/content/docs/configuration.mdx`, `docs/src/content/docs/access-control.mdx`, `docs/src/content/docs/api.md`, `docs/src/content/docs/deployment.md`, `docs/src/content/docs/development.md`, `docs/src/content/docs/architecture.md`, `docs/src/content/docs/reverse-proxy.mdx`, `AGENTS.md`, `SECURITY.md`): closes #240; partially advances the auth-hardening epic [#228](https://github.com/Wave-RF/WaveHouse/issues/228) and the management/data-plane split [#359](https://github.com/Wave-RF/WaveHouse/issues/359). Adds an optional `auth.operator_key` (`WH_AUTH_OPERATOR_KEY`): a request presenting it — via an `Authorization: Operator ` header (forwarded verbatim by proxies, no collision with Bearer JWTs) or the `X-Operator-Key` alias — is authorized as a full-access platform operator — the whole data plane *and* the `/v1/admin/*` management surface — without minting a JWT and independently of the token verifier, giving the person running the deployment a role-free credential for bootstrap and break-glass. The middleware checks it before the Bearer token with a constant-time comparison (`crypto/subtle`) and stamps two things into the request context: the live `admin_role` (so the policy evaluator's admin bypass grants unrestricted data-plane access while a policy exists) and a platform-operator bit that `RequireAdmin` honors **even when the policy is `nil`/deleted** — the one HTTP path that can restore a wiped policy, which previously required SSH access and a reboot. Empty (the default) disables it, so existing deployments are unchanged; treat it as an admin secret (load from a secret store, serve only over TLS). A successful operator authentication is audit-logged at `INFO`; a request presenting a *non-matching* operator key is logged at `WARN` and counted by a new `wavehouse_auth_operator_key_failures_total` counter (a probing/brute-force signal on the most privileged credential in the system) before falling through to the normal token/default path — the middleware still never rejects. The `Authorization` auth-scheme is matched case-insensitively (RFC 7235) via a shared `authScheme` helper, which also makes the existing `Bearer` JWT scheme case-insensitive (previously it required the canonical `Bearer` casing). Explicitly out of scope, tracked in #359: scoping the operator credential to the management surface only, and capability-scoped admin permissions. - **Missing-dedupe-id observability + optional strict mode** (`internal/api/ingest.go`, `internal/api/ingest_test.go`, `internal/config/config.go`, `cmd/wavehouse/main.go`, `config.yaml`, `deployments/compose/standalone.yaml`, `docs/src/content/docs/configuration.mdx`, `docs/src/content/docs/api.md`, `docs/src/content/docs/architecture.md`): closes #219. With dedupe enabled, a row missing the configured `id_field` can't be deduped — previously it was published with idempotency silently disabled and *no* log or metric, so a producer bug that dropped the id turned off the guarantee for those rows unnoticed. Now every such row is logged at `WARN` and counted by a new `wavehouse_ingest_dedupe_missing_id_total` counter (labeled by `table`), making the loss observable server-side. A new opt-in `dedupe.require_id` (`WH_DEDUPE_REQUIRE_ID`, default `false`) turns that signal into enforcement: a row missing the id is rejected (`400` for a single insert; a per-record failure in a batch) instead of published — a tripwire for producers that must guarantee the id (complements the client-side [#202](https://github.com/Wave-RF/WaveHouse/issues/202)). Default behavior is unchanged. - **"Durability & Storage" operations guide** (`docs/src/content/docs/durability.md` (new), `docs/src/config/sidebar.ts`, `docs/src/content/docs/reverse-proxy.mdx`, `docs/src/content/docs/configuration.mdx`, `docs/src/content/docs/deployment.md`): documents #84. A new Operations page making the embedded-JetStream durability contract explicit before the docs site publishes: a `200` from `POST /v1/ingest` means the event has been `fsync`'d to disk on the node (the server runs with `SyncAlways: true` in `internal/mq/embedded.go`), which makes the storage substrate's `fsync` tail the ingest latency floor. Covers the contract (and how it differs from JetStream's default page-cache-then-periodic-sync mode), why a slow `fsync` tail manifests as `create stream: ... context deadline exceeded` and `503` backpressure, a where-it's-cheap-vs-expensive substrate table (managed cloud block storage and PLP NVMe vs. ZFS-without-SLOG / qcow2-on-`ext4` / spinning disks), an `fio` recipe + verdict bands to measure your own storage (with the macOS `F_FULLFSYNC` honesty caveat), and the symptom checklist. Forward-references the configurable group-commit interval (`mq.sync_interval`, [#139](https://github.com/Wave-RF/WaveHouse/issues/139)) and the planned `wavehouse storage-check` preflight ([#84](https://github.com/Wave-RF/WaveHouse/issues/84)) without claiming either exists yet. Cross-linked from Configuration (Message Queue), Deployment (Persistent Storage), and the Ingest Pipeline's worker-side ack section; no code changes. @@ -23,6 +25,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **BREAKING (SDK): `PipeRef.fetch` no longer accepts a `limit` it silently ignored** (`clients/ts/src/pipes.ts`, `clients/ts/src/client.test.ts`, `docs/src/content/docs/sdk/pipes.md`, `docs/src/content/docs/sdk/reference.md`): closes #464, raised by CodeRabbit on #456. It took the same per-call options type as the query builder — which carries `limit` — but forwarded only `signal`, so `wh.pipe('top_pages').fetch({ limit: 10 })` type-checked, ran, and quietly returned whatever the pipe's SQL returned. `QueryBuilder.fetch` and `TableRef.fetch` both honour `limit`, so the inconsistency sat inside one shared type. There is nothing to forward: the endpoint binds the request body as the pipe's *parameters* (`internal/api/pipes.go` → `pipes.BindParams`), and a key the SQL doesn't declare is ignored, so a client-side row cap is not something the pipes surface offers. The parameter is now a dedicated `PipeRequestOptions` (exported) declaring `signal?: AbortSignal` and `limit?: never`, making the dead option a compile error rather than a silent no-op. `never` rather than simply omitting `limit`, because omitting it only rejects fresh object literals — TypeScript's excess-property check doesn't apply to a *variable*, so a shared `const opts: RequestOptions` carrying a limit would still have passed and still been dropped, which is the defect rather than a narrower version of it. Both cases are pinned by `@ts-expect-error` tests. **Note the collateral effect**, which is the half most consumers will actually meet: a value *declared* `RequestOptions` no longer assigns to a pipe `.fetch()` at all, even when it carries no limit at runtime, because the declared type permits one and assignability is decided on the type. Type a shared options object as `PipeRequestOptions` — the table and query-builder `.fetch()` accept it too, so it works everywhere — or inline `{ signal }` at the pipe call. Structural wrappers are unaffected: method parameters compare bivariantly, so an `interface Fetchable { fetch(opts?: RequestOptions): … }` is still satisfied by `PipeRef`. **Migration:** declare a `{{limit}}` parameter in the pipe's SQL and pass it as a pipe parameter — `wh.pipe(name, { limit })` — which is what the docs already showed. Pre-existing rather than introduced by #456, folded in there because that PR renames the type in question. + +- **BREAKING (SDK): `FetchOptions` is renamed `RequestOptions`** (`clients/ts/src/types.ts`, `clients/ts/src/index.ts`, `clients/ts/src/query-builder.ts`, `clients/ts/src/table.ts`, `clients/ts/src/pipes.ts`): the per-call options type accepted by `.fetch()`. The old name collided conceptually with the new `options.fetchOptions` — which, following OpenAI, Anthropic, and the wider ecosystem, means "extra `RequestInit` fields", not "options for our `.fetch()` method". Shipping both would have left `FetchOptions` and `fetchOptions` in the same SDK one capital letter apart, meaning unrelated things. `RequestOptions` is what Anthropic's SDK calls the identical concept. No deprecated alias: the type is unreferenced by anything consuming the pre-1.0 package, and keeping it would preserve exactly the ambiguity the rename removes. Renaming the import is the whole migration for this entry — note the separate `PipeRef.fetch` narrowing above, which is a behavioural break in the same file. The module-private `RequestOptions` in `http.ts` — the internal request descriptor — becomes `RequestSpec` to free the name. + - **`@wavehouse/sdk` `engines.node` floor back to `>=22`, matching the only line we test** (`clients/ts/package.json`, `clients/ts/README.md`, `docs/src/content/docs/sdk/index.mdx`, `docs/src/content/docs/sdk/queries.md`, `pnpm-workspace.yaml`): the floor was relaxed to `>=18` when the browser-first distribution landed (see the entry below), on the reasoning that the runtime needs only `fetch`. Nothing ever tested 18, though — `.nvmrc` pins 22 and `.github/actions/setup-env` consumes it via `node-version-file`, so 22 is the single version CI exercises — and Node 18 and 20 have both since reached upstream end-of-life. Declaring a floor we neither test nor is supported upstream promises more than it can back, so it returns to `>=22`. **Consumer impact:** installing on Node < 22 now warns with `EBADENGINE` under npm, and fails outright under pnpm with `engine-strict` enabled. The SDK README and the docs' Runtime support section state the requirement, which they previously either omitted or quoted as 18. - **Live SSE events are now projected and serialized once per role instead of once per subscriber** (`internal/stream/hub.go` (new), `internal/stream/{subscriber,bucket,heartbeat,metrics,doc}.go`, `internal/api/stream.go`, `internal/api/hub.go` + `internal/api/transform.go` (both removed — the broadcast hub moves to `internal/stream`, and the orphaned test-only `transformForClient` is dropped), `cmd/wavehouse/main.go`, `docs/src/content/docs/architecture.md`, `AGENTS.md`, plus tests in `internal/stream/{hub,filter,subscriber,bucket,heartbeat}_test.go` and `internal/api/{stream,transform,router,errors}_test.go`): the first PR of the SSE delivery-path throughput epic ([#294](https://github.com/Wave-RF/WaveHouse/issues/294)), building on the `internal/stream` primitives from #346. The broadcast hub moves into `internal/stream` as `Hub`: subscribers register under `(topic, role)`, and `Broadcast` decodes each event **once**, applies each subscribed role's column policy **once**, builds one SSE frame per role, and fans it to every member of that role's `Bucket`. Previously every connection independently ran `json.Unmarshal → policy.Evaluate → filterEventColumns → json.Marshal` (plus a second unmarshal just to read the `id:` timestamp) on the *same* event in its own read loop — byte-identical work repeated N times. For a single-role audience (the public dashboard, every viewer `public`) that collapses N re-projections to 1, moving the measured ~2 270 deliveries/s ceiling toward an events/s ceiling. The `(topic, role)` key is sufficient and claims-independent: column visibility derives only from the role+table policy entry, and the stream path applies no row-level filter (a documented invariant — if row-level filtering is ever added to streaming, the key must take claims into account). The handler's two `select` cases (keepalive vs. per-subscriber event) collapse into one byte-pump over a single `Subscriber.Frames()` queue carrying typed `Frame`s; the subscriber queue grows from cap 1 (keepalive-only) to 64 so live events buffer while the handler is mid-write. Gap-fill replay and `Last-Event-ID`/`?since=` resumption are unchanged (replay stays per-connection via the shared `stream.ReplayFrame`; live frames carry the same `id: `). Slow-consumer drops, silent before, now increment `wavehouse_sse_dropped_frames_total`; an inert `Subscriber.Evicted()` seam is wired for the eviction follow-up. The per-delivery OpenTelemetry span (another #294 item) was already removed in #346. **Deferred to follow-ups:** active slow-consumer eviction (#94) and right-sizing the subscriber buffer + broadcast lock cost (#152). diff --git a/clients/ts/src/client.test.ts b/clients/ts/src/client.test.ts index 6b201d24..9fa69ab5 100644 --- a/clients/ts/src/client.test.ts +++ b/clients/ts/src/client.test.ts @@ -6,11 +6,17 @@ import { PolicyNamespace } from "./policy.js"; import { SchemaNamespace } from "./schema.js"; import { SysNamespace } from "./sys.js"; import { TableRef } from "./table.js"; +import type { FetchLike, PipeRequestOptions, RequestOptions } from "./types.js"; let fetchSpy: ReturnType; beforeEach(() => { - fetchSpy = vi.fn().mockResolvedValue(new Response(JSON.stringify([]), { status: 200 })); + // A fresh Response per call: a Response body can only be read once, so a + // shared instance makes the second request fail and retry, and a test + // asserting on `mock.calls[1]` would silently be reading that retry. + fetchSpy = vi + .fn() + .mockImplementation(() => Promise.resolve(new Response(JSON.stringify([]), { status: 200 }))); vi.stubGlobal("fetch", fetchSpy); }); @@ -113,6 +119,28 @@ describe("WaveHouseClient.pipe()", () => { const pipe = client.pipe("top_pages", { limit: 10 }); expect(typeof pipe.then).toBe("function"); }); + + it("rejects a per-call limit, which the pipes endpoint cannot honour", async () => { + const client = createClient({ baseURL: "http://localhost:8080" }); + // Compile-time assertions: accepting `limit` here would silently drop it, + // since the endpoint binds the body as the pipe's parameters. A row cap + // belongs in the pipe's own SQL, supplied via `wh.pipe(name, { limit })`. + + // @ts-expect-error — as a fresh literal + await client.pipe("top_pages").fetch({ limit: 10 }); + + // ...and as a named value. This is the case a plain `Pick<>` would let + // through, since excess-property checking only rejects literals — so the + // limit would reach the endpoint, be ignored, and never be flagged. + const shared: RequestOptions = { signal: AbortSignal.timeout(1000), limit: 10 }; + // @ts-expect-error — `limit` is not part of PipeRef.fetch's options + await client.pipe("top_pages").fetch(shared); + + // `signal` alone is accepted, literal or named. + await client.pipe("top_pages").fetch({ signal: AbortSignal.timeout(1000) }); + const signalOnly: PipeRequestOptions = { signal: AbortSignal.timeout(1000) }; + await client.pipe("top_pages").fetch(signalOnly); + }); }); describe("WaveHouseClient.sql()", () => { @@ -143,3 +171,190 @@ describe("WaveHouseClient.sql()", () => { expect(callWithLegacyParams).toThrow(/client\.sql\(sql, params\) was removed/); }); }); + +describe("options.fetch", () => { + it("routes requests through a supplied fetch instead of the global", async () => { + // Typed as FetchLike rather than cast: a consumer's override needs no cast, + // so the test shouldn't need one either. + const custom = vi.fn(async () => new Response(JSON.stringify([]), { status: 200 })); + const client = createClient({ + baseURL: "http://localhost:8080", + auth: () => "test-token", + options: { fetch: custom }, + }); + + await client.from("clicks").select("*").limit(1); + + expect(custom).toHaveBeenCalledTimes(1); + expect(fetchSpy).not.toHaveBeenCalled(); + // The documented contract: a string URL and a complete RequestInit — + // proxy/middleware consumers rely on the whole request reaching them. + const [url, init] = custom.mock.calls[0]; + expect(typeof url).toBe("string"); + expect(url).toContain("http://localhost:8080"); + expect(init?.method).toBe("POST"); + expect(init?.headers).toMatchObject({ + "Content-Type": "application/json", + Authorization: "Bearer test-token", + }); + expect(typeof init?.body).toBe("string"); + }); + + it("falls back to the global fetch when no override is given", async () => { + const client = createClient({ baseURL: "http://localhost:8080" }); + await client.from("clicks").select("*").limit(1); + expect(fetchSpy).toHaveBeenCalledTimes(1); + }); + + it("keeps the global late-bound so it can be swapped after construction", async () => { + const client = createClient({ baseURL: "http://localhost:8080" }); + const replacement = vi + .fn() + .mockResolvedValue(new Response(JSON.stringify([]), { status: 200 })); + vi.stubGlobal("fetch", replacement); + + await client.from("clicks").select("*").limit(1); + + expect(replacement).toHaveBeenCalledTimes(1); + expect(fetchSpy).not.toHaveBeenCalled(); + }); + + it("applies the override to retries too, not just the first attempt", async () => { + const custom = vi + .fn() + .mockResolvedValueOnce(new Response("boom", { status: 500 })) + .mockResolvedValueOnce(new Response(JSON.stringify([]), { status: 200 })); + const client = createClient({ + baseURL: "http://localhost:8080", + options: { fetch: custom, maxRetries: 1 }, + }); + + await client.from("clicks").select("*").limit(1); + + expect(custom).toHaveBeenCalledTimes(2); + expect(fetchSpy).not.toHaveBeenCalled(); + }); +}); + +describe("options.headers", () => { + const headersOf = (call: number) => + fetchSpy.mock.calls[call][1].headers as Record; + + it("adds configured headers to every request", async () => { + const client = createClient({ + baseURL: "http://localhost:8080", + options: { headers: { "CF-Access-Client-Id": "abc.access", "X-Tenant": "acme" } }, + }); + + await client.from("clicks").select("*").limit(1); + await client.from("clicks").select("*").limit(1); + + // Exactly two — proves call 1 is a second request, not a retry of the first. + expect(fetchSpy).toHaveBeenCalledTimes(2); + for (const call of [0, 1]) { + expect(headersOf(call)).toMatchObject({ + "CF-Access-Client-Id": "abc.access", + "X-Tenant": "acme", + }); + } + }); + + it("cannot override the Content-Type a request needs", async () => { + // A global Content-Type outranking the request's own is a documented way to + // break uploads — it must lose, not merge. + const client = createClient({ + baseURL: "http://localhost:8080", + options: { headers: { "Content-Type": "text/plain" } }, + }); + + await client.from("clicks").select("*").limit(1); + + expect(headersOf(0)["Content-Type"]).toBe("application/json"); + expect(Object.values(headersOf(0))).not.toContain("text/plain"); + }); + + it("cannot displace the auth token, even under different casing", async () => { + const client = createClient({ + baseURL: "http://localhost:8080", + auth: () => "real-token", + options: { headers: { authorization: "Bearer impostor" } }, + }); + + await client.from("clicks").select("*").limit(1); + + const sent = headersOf(0); + expect(sent.Authorization).toBe("Bearer real-token"); + // The lowercase spelling must not ride along beside the canonical one. + expect(sent.authorization).toBeUndefined(); + }); + + it("collapses two configured spellings of one header instead of sending both", async () => { + // Left as separate keys, the Headers constructor comma-joins them at fetch + // time — `x-tenant: acme, beta` — which is the corruption this guards. + const client = createClient({ + baseURL: "http://localhost:8080", + options: { headers: { "x-tenant": "acme", "X-Tenant": "beta" } }, + }); + + await client.from("clicks").select("*").limit(1); + + const sent = headersOf(0); + const spellings = Object.keys(sent).filter((k) => k.toLowerCase() === "x-tenant"); + expect(spellings).toHaveLength(1); + expect(new Headers(sent).get("x-tenant")).toBe("beta"); + }); + + it("still applies when no auth is configured", async () => { + const client = createClient({ + baseURL: "http://localhost:8080", + options: { headers: { "X-Tenant": "acme" } }, + }); + + await client.from("clicks").select("*").limit(1); + + expect(headersOf(0)["X-Tenant"]).toBe("acme"); + expect(headersOf(0).Authorization).toBeUndefined(); + }); +}); + +describe("options.fetchOptions", () => { + it("merges configured RequestInit fields into every request", async () => { + const client = createClient({ + baseURL: "http://localhost:8080", + options: { fetchOptions: { credentials: "include", cache: "no-store" } }, + }); + + await client.from("clicks").select("*").limit(1); + + const init = fetchSpy.mock.calls[0][1]; + expect(init.credentials).toBe("include"); + expect(init.cache).toBe("no-store"); + }); + + it("cannot corrupt the fields the SDK controls", async () => { + const client = createClient({ + baseURL: "http://localhost:8080", + auth: () => "test-token", + options: { + headers: { "X-Tenant": "acme" }, + fetchOptions: { + method: "DELETE", + body: "hijacked", + headers: { "X-Tenant": "overridden", Authorization: "Bearer impostor" }, + } as RequestInit, + }, + }); + + await client.from("clicks").select("*").limit(1); + + const init = fetchSpy.mock.calls[0][1]; + expect(init.method).toBe("POST"); + expect(init.body).not.toBe("hijacked"); + // options.headers is the header channel; fetchOptions.headers is discarded + // wholesale rather than merged, so it can't smuggle an Authorization past auth. + expect(init.headers).toMatchObject({ + "X-Tenant": "acme", + Authorization: "Bearer test-token", + }); + }); +}); diff --git a/clients/ts/src/client.ts b/clients/ts/src/client.ts index eb443c42..bb3987a9 100644 --- a/clients/ts/src/client.ts +++ b/clients/ts/src/client.ts @@ -37,6 +37,9 @@ export class WaveHouseClient { auth: config.auth, options: { maxRetries: config.options?.maxRetries ?? 2, + fetch: config.options?.fetch, + headers: config.options?.headers, + fetchOptions: config.options?.fetchOptions, }, }; diff --git a/clients/ts/src/http.ts b/clients/ts/src/http.ts index 39215cb0..905a8556 100644 --- a/clients/ts/src/http.ts +++ b/clients/ts/src/http.ts @@ -2,7 +2,7 @@ import { networkError, parseErrorResponse } from "./errors.js"; import type { HttpContext, WaveHouseError } from "./types.js"; import { resolveURL } from "./url.js"; -interface RequestOptions { +interface RequestSpec { method: string; path: string; body?: unknown; @@ -25,11 +25,41 @@ export interface HttpResult { headers: Headers; } +/** + * Merge configured headers underneath the SDK's own, matching names + * case-insensitively so a caller's `authorization` can't sit alongside our + * `Authorization` and let the server pick. Canonical casing wins, and values + * replace rather than append — a header joined instead of replaced is a known + * way to produce `Content-Type: application/json, image/png`. + */ +function mergeHeaders( + base: Record, + extra: Record | undefined, +): Record { + if (!extra) return base; + const sdkNames = new Set(Object.keys(base).map((k) => k.toLowerCase())); + const merged = { ...base }; + // Spelling each configured name was last stored under, so two entries + // differing only in case replace each other here rather than surviving as + // separate keys for `Headers` to comma-join at fetch time. + const configured = new Map(); + for (const [name, value] of Object.entries(extra)) { + const lower = name.toLowerCase(); + // Skip rather than overwrite: `base` is the SDK's own, which outranks. + if (sdkNames.has(lower)) continue; + const prior = configured.get(lower); + if (prior !== undefined) delete merged[prior]; + merged[name] = value; + configured.set(lower, name); + } + return merged; +} + /** * Internal fetch wrapper with auth injection, retry, backoff, and Retry-After. * @internal */ -export async function request(ctx: HttpContext, opts: RequestOptions): Promise> { +export async function request(ctx: HttpContext, opts: RequestSpec): Promise> { const url = resolveURL(ctx.baseURL, opts.path, opts.params).toString(); const headers: Record = { "Content-Type": opts.contentType ?? "application/json", @@ -51,17 +81,29 @@ export async function request(ctx: HttpContext, opts: RequestOptions): Promis } } + // After auth, so `auth` keeps ownership of Authorization, and after the + // Content-Type/Accept this request needs. + const finalHeaders = mergeHeaders(headers, ctx.options.headers); + let lastError: WaveHouseError | null = null; const maxAttempts = ctx.options.maxRetries + 1; for (let attempt = 0; attempt < maxAttempts; attempt++) { try { - const res = await fetch(url, { + // Configured RequestInit first, so the fields the SDK controls overwrite + // it — a supplied `method` or `body` would otherwise corrupt the request. + const init: RequestInit = { + ...ctx.options.fetchOptions, method: opts.method, - headers, + headers: finalHeaders, body: requestBody, signal: opts.signal, - }); + }; + // Call the global directly when no override is configured, rather than + // capturing it — a detached `fetch` reference is not universally safe to + // invoke, and late binding is what lets tests swap `globalThis.fetch`. + const doFetch = ctx.options.fetch; + const res = doFetch ? await doFetch(url, init) : await fetch(url, init); if (res.ok) { const text = await res.text(); diff --git a/clients/ts/src/index.ts b/clients/ts/src/index.ts index 27db5667..2274b816 100644 --- a/clients/ts/src/index.ts +++ b/clients/ts/src/index.ts @@ -28,7 +28,8 @@ export type { Database, // DLQ DLQStats, - FetchOptions, + // HTTP + FetchLike, // Query FilterOp, // Ingest @@ -38,10 +39,12 @@ export type { ParamDef, // Pipes Pipe, + PipeRequestOptions, // Policy Policy, PolicyFilter, QueryFilter, + RequestOptions, Result, RolePermissions, Schemas, diff --git a/clients/ts/src/pipes.ts b/clients/ts/src/pipes.ts index 38835594..e20c4ae4 100644 --- a/clients/ts/src/pipes.ts +++ b/clients/ts/src/pipes.ts @@ -1,7 +1,7 @@ import { err, ok } from "./errors.js"; import { request } from "./http.js"; import type { StreamController } from "./stream/controller.js"; -import type { FetchOptions, HttpContext, Pipe, Result, StreamOptions } from "./types.js"; +import type { HttpContext, Pipe, PipeRequestOptions, Result, StreamOptions } from "./types.js"; type CreateStreamFn = (table: string, opts?: StreamOptions) => StreamController; @@ -24,8 +24,15 @@ export class PipeRef> implements PromiseLike> { + /** + * Execute the pipe and return results. + * + * Takes only `signal` — deliberately narrower than the `RequestOptions` the + * query builder accepts. The pipes endpoint binds the body as the pipe's + * parameters, so there is no row cap to forward; give the pipe a `{{limit}}` + * parameter in its SQL and pass it via `wh.pipe(name, { limit })`. + */ + async fetch(opts?: PipeRequestOptions): Promise> { const { data, error } = await request(this._ctx, { method: "POST", path: `/v1/pipes/${encodeURIComponent(this._name)}`, diff --git a/clients/ts/src/query-builder.ts b/clients/ts/src/query-builder.ts index b77b9192..e5ef1af1 100644 --- a/clients/ts/src/query-builder.ts +++ b/clients/ts/src/query-builder.ts @@ -5,11 +5,11 @@ import { StreamController } from "./stream/controller.js"; import { LiveQuery } from "./stream/live-query.js"; import type { Aggregation, - FetchOptions, FilterOp, HttpContext, OrderClause, QueryFilter, + RequestOptions, Result, StreamOptions, StreamSubscriber, @@ -137,7 +137,7 @@ export class QueryBuilder> implements PromiseLike< /** Default row limit when none is specified — deliberately tighter than the backend's DefaultMaxRows (10000) safety cap. */ static readonly DEFAULT_LIMIT = 1000; - async fetch(opts?: FetchOptions): Promise> { + async fetch(opts?: RequestOptions): Promise> { const effectiveLimit = opts?.limit ?? this._state.limit ?? QueryBuilder.DEFAULT_LIMIT; const ast = this._buildAST(effectiveLimit); @@ -239,7 +239,7 @@ export class QueryBuilder> implements PromiseLike< private async _fetchNext( prevRows: Row[], _limit: number, - opts?: FetchOptions, + opts?: RequestOptions, ): Promise> { // No explicit order → no keyset cursor to build; nothing to page by. const cursor = this._state.orderBy[0]; diff --git a/clients/ts/src/table.ts b/clients/ts/src/table.ts index f44c05e3..18c53146 100644 --- a/clients/ts/src/table.ts +++ b/clients/ts/src/table.ts @@ -3,10 +3,10 @@ import { request } from "./http.js"; import { QueryBuilder } from "./query-builder.js"; import type { StreamController } from "./stream/controller.js"; import type { - FetchOptions, HttpContext, InsertRecordResult, InsertResult, + RequestOptions, Result, StreamOptions, TableSchema, @@ -58,7 +58,7 @@ export class TableRef> { } /** SELECT * shortcut — fetches rows with optional pagination. */ - async fetch(opts?: FetchOptions): Promise> { + async fetch(opts?: RequestOptions): Promise> { return this.select() .limit(opts?.limit ?? 1000) .fetch(opts); diff --git a/clients/ts/src/types.ts b/clients/ts/src/types.ts index fd33a5ad..a77b9685 100644 --- a/clients/ts/src/types.ts +++ b/clients/ts/src/types.ts @@ -80,9 +80,84 @@ export interface ClientConfig<_DB extends Database = Database> { options?: ClientOptions; } +/** + * A `fetch`-compatible function. + * + * Spelled out rather than written `typeof fetch`, which resolves differently + * depending on whether the consumer's TypeScript `lib` includes DOM — the same + * signature either way, but stable across configurations. + * + * In practice the SDK only ever calls it with a string URL, and reads `.ok` and + * `.headers` always, `.text()` on success, and `.status`, `.statusText` plus + * `.json()` when the response is not `ok`. A rejection becomes a + * `NETWORK_ERROR` result and is retried with backoff — except a `DOMException` + * named `AbortError` (what the platform `fetch` and undici throw on abort), + * which becomes `ABORTED` and is not retried. Implementations that reject with + * some other abort error, such as `node-fetch`, are retried instead. + * + * Implementations shipping their own request/response declarations (undici, + * `node-fetch`) need casts on the URL, the init and the return value, since + * those types are separate from the ones behind your global `fetch`; the + * narrow runtime contract above is what makes them safe. + */ +export type FetchLike = (input: string | URL | Request, init?: RequestInit) => Promise; + export interface ClientOptions { /** Maximum retry attempts for failed requests. Default: 2. */ maxRetries?: number; + /** + * HTTP implementation used for every REST request (SSE streams excluded). + * Defaults to the global `fetch`. + * + * Provide one to route through a proxy, attach client certificates, add + * middleware (logging, tracing, circuit breaking), or bypass a transport bug + * in the runtime's bundled HTTP stack by routing through an implementation + * you install yourself — e.g. an undici new enough to be free of + * {@link https://github.com/nodejs/undici/issues/5600 | undici #5600}: + * + * ```ts + * import { Agent, fetch as undiciFetch } from "undici"; // npm install undici — 8.10.0+ + * // Pass the dispatcher explicitly: undici's pool lives on a shared + * // globalThis symbol claimed by whichever copy loaded first, so an implied + * // dispatcher can still be the runtime's affected one. + * const dispatcher = new Agent(); + * createClient({ + * baseURL, + * options: { + * fetch: (url, init) => + * undiciFetch(url as string, { ...init, dispatcher } as never) as unknown as Promise, + * }, + * }); + * ``` + * + * Only the SSE transport is exempt: the live connection behind `.stream()` + * and `.liveQuery()` uses `EventSource`, which this does not replace. + * `.liveQuery()`'s initial backfill is an ordinary request and does go + * through your function. + */ + fetch?: FetchLike; + /** + * Headers added to every REST request (SSE streams excluded) — for a + * header-gated proxy in front of WaveHouse, such as a Cloudflare Access + * service token. + * + * Matched case-insensitively, as HTTP headers are. Applied underneath the + * SDK's own headers: `auth` still owns `Authorization`, and the + * `Content-Type` and `Accept` a given request needs are not overridable + * here — a global `Content-Type` that outranked the request's own is a + * documented way to break uploads. + */ + headers?: Record; + /** + * Extra `RequestInit` fields merged into every REST request — `credentials` + * for a cookie-authenticated origin, `mode`, `cache`, `keepalive`, or a + * runtime-specific extension such as Next.js's `next: { tags }`. + * + * Fields the SDK controls (`method`, `headers`, `body`, `signal`) always + * win, so this cannot corrupt the request itself. Non-standard fields may + * need a cast, since `RequestInit` only declares the standard ones. + */ + fetchOptions?: RequestInit; } // --- Structured query AST (matches backend wire format) --- @@ -256,13 +331,36 @@ export interface ValidationResult { valid: boolean; } -// --- Fetch options --- +// --- Per-call request options --- -export interface FetchOptions { +/** + * Options for a single call, passed to `.fetch()`. + * + * Note that `await`ing a builder directly (`await wh.from("t").select("*")`) + * takes no options — use the explicit `.fetch({ … })` form for those. + */ +export interface RequestOptions { signal?: AbortSignal; limit?: number; } +/** + * Options for a single `PipeRef.fetch()` call. + * + * `limit` is declared `never` rather than simply left out. Omitting it would + * still admit a `RequestOptions` value that carries one — excess-property + * checking only rejects fresh object literals, so a variable would pass and the + * limit would be silently dropped, which is the whole defect this prevents. + * + * A pipe's row cap belongs in its SQL as a `{{limit}}` parameter, supplied via + * `wh.pipe(name, { limit })`. + */ +export interface PipeRequestOptions { + signal?: AbortSignal; + /** Not supported on pipes — pass `limit` as a pipe parameter instead. */ + limit?: never; +} + // --- Stream options --- export interface StreamOptions { @@ -276,5 +374,11 @@ export interface StreamOptions { export interface HttpContext { baseURL: string; auth?: () => Promise | string; - options: { maxRetries: number }; + // `fetch` stays optional rather than defaulted here, to keep the global late-bound. + options: { + maxRetries: number; + fetch?: FetchLike; + headers?: Record; + fetchOptions?: RequestInit; + }; } diff --git a/docs/src/content/docs/sdk/index.mdx b/docs/src/content/docs/sdk/index.mdx index 12153579..78ccd1d8 100644 --- a/docs/src/content/docs/sdk/index.mdx +++ b/docs/src/content/docs/sdk/index.mdx @@ -327,6 +327,13 @@ const wh = createClient({ | `baseURL` | `string` | — | WaveHouse server URL, optionally including a path prefix (required) | | `auth` | `() => Promise \| string` | — | Token provider. Omit for public access | | `options.maxRetries` | `number` | `2` | Retry attempts for failed/5xx requests | +| `options.headers` | `Record` | — | Headers added to every REST request ([details](#custom-headers)) | +| `options.fetchOptions` | `RequestInit` | — | Extra `RequestInit` fields merged into every REST request ([details](#extra-requestinit-fields)) | +| `options.fetch` | `FetchLike` | global `fetch` | HTTP implementation for every REST request ([details](#supplying-your-own-fetch)) | + +:::caution[These three apply to REST only] +`headers`, `fetchOptions`, and `fetch` do **not** reach the streaming transport. `.stream()` and `.liveQuery()`'s live connection go through `EventSource`, which accepts neither custom headers nor a `fetch` — which is why the token rides in `?token=` there. If WaveHouse sits behind a header-gated proxy, REST works and streaming does not; follow [#203](https://github.com/Wave-RF/WaveHouse/issues/203) for that. (`.liveQuery()`'s initial backfill is an ordinary REST call and *is* covered.) +::: :::note[How the token is transmitted] The SDK attaches your `auth` token as an `Authorization: Bearer` header on REST calls, and — because the browser `EventSource` API can't set headers — as a `?token=` query parameter on streaming connections. When both are present the server prefers the header, and it strips `?token=` from the request URL either way — so an unused credential can't survive into a later handler's log line, and the token stays out of **WaveHouse's own** logs. @@ -348,6 +355,212 @@ Trailing slashes on `baseURL` are optional, but it must be **absolute** — sche The proxy in front must **strip** the prefix before forwarding, since WaveHouse itself always serves at `/v1/…` — see [Behind a reverse proxy → Path prefixes](/reverse-proxy#path-prefixes). +### Custom headers + +`options.headers` adds headers to every REST request — the usual reason being a +header-gated proxy in front of WaveHouse, such as a Cloudflare Access service +token: + +```ts +const wh = createClient({ + baseURL: 'https://wavehouse.example.com', + options: { + headers: { + 'CF-Access-Client-Id': process.env.CF_ACCESS_CLIENT_ID!, + 'CF-Access-Client-Secret': process.env.CF_ACCESS_CLIENT_SECRET!, + }, + }, +}); +``` + +Header names are matched **case-insensitively**, as HTTP requires, so +`authorization` and `Authorization` are the same header rather than two. + +Your headers are applied *underneath* the SDK's own, and a collision means yours +is dropped rather than merged: + +- **`auth` keeps `Authorization`.** Setting it here won't displace your token + provider. Use `auth` for credentials. +- **`Content-Type` and `Accept` belong to the request.** The SDK knows what it's + sending; a global `Content-Type` that outranked it would break requests whose + body isn't JSON. + +Nothing is ever comma-joined: on a collision the SDK's value stands alone, and +two of your own entries differing only in case collapse to the last one — a +header joined rather than replaced is how you end up sending +`Content-Type: application/json, image/png`. + +From a **browser**, a cross-origin custom header must also survive CORS +preflight, and WaveHouse allow-lists a fixed set (`Accept`, `Authorization`, +`Content-Type`, `Last-Event-ID`, `X-Request-ID`) with no config knob. So custom +headers work server-side, or from a browser when the proxy in front owns CORS — +which is the same proxy the header is usually for. + +Headers are static. For a credential that rotates per request, wrap the +transport with [`options.fetch`](#supplying-your-own-fetch); a callback form is +tracked in [#459](https://github.com/Wave-RF/WaveHouse/issues/459). + +### Extra `RequestInit` fields + +`options.fetchOptions` is merged into the `RequestInit` of every REST request — +for settings that aren't headers and don't warrant replacing the whole +transport: + +```ts +const wh = createClient({ + baseURL: 'https://wavehouse.example.com', + options: { fetchOptions: { cache: 'no-store', keepalive: true } }, +}); +``` + +It carries any `RequestInit` field — `mode`, `cache`, `keepalive`, +`credentials`, `redirect` — plus runtime-specific extensions such as Next.js's +`next: { tags }`, which tags the cached response for on-demand revalidation. Only those runtime +extensions need a cast; the standard fields are declared on `RequestInit` +already. + +:::note[`credentials: 'include'` needs a proxy that owns CORS] +Sending cookies cross-origin is a common reason to reach for this, but it can't +work against WaveHouse's own CORS: it deliberately never emits +`Access-Control-Allow-Credentials` (it's a Bearer-token API), and the default +`cors_allowed_origins: "*"` makes `include` a hard browser failure regardless. +It applies where a fronting proxy answers CORS itself — see +[Behind a reverse proxy](/reverse-proxy#header-and-auth-forwarding). Same-origin +deployments already send cookies without it. +::: + +The fields the SDK controls — `method`, `headers`, `body`, and `signal` — always +win, so this can't corrupt the request itself. In particular `headers` here is +ignored; use `options.headers`, which merges properly. + +### Supplying your own `fetch` + +`options.fetch` replaces the HTTP implementation the SDK uses. Reach for it to +route through a proxy, attach client certificates, wrap requests in your own +middleware (logging, tracing, circuit breaking), stub HTTP in your own tests +without monkey-patching a global, or to work around transport behavior of the +runtime you happen to be on: + +```ts +const wh = createClient({ + baseURL: 'https://wavehouse.example.com', + options: { + fetch: async (url, init) => { + const started = performance.now(); + const res = await fetch(url, init); + console.log(res.status, url, `${Math.round(performance.now() - started)}ms`); + return res; + }, + }, +}); +``` + +Retries go through the same function, so middleware sees every attempt. + +The SDK always calls your function with a **string** URL and a plain +`RequestInit`. Off the response it reads `.ok` and `.headers` always, `.text()` +on success, and `.status`, `.statusText` plus `.json()` when the response is not +`ok` — so a hand-rolled response object needs all six. If your function +rejects, the request becomes a `NETWORK_ERROR` result and is retried with +backoff; the one exception is a `DOMException` named `AbortError` — what the +platform `fetch` and undici throw on abort — which becomes `ABORTED` and is not +retried. An implementation that signals abort some other way (`node-fetch` +throws its own `AbortError` class) gets retried instead. + +The option's type is exported as `FetchLike`. It matches the standard `fetch` +signature — written out rather than as `typeof fetch`, which resolves +differently depending on whether your TypeScript `lib` includes DOM. Let +TypeScript infer your parameters (`(url, init) => …`) and it fits without +annotation. + +Only the SSE transport is exempt: the live connection behind `.stream()` and +`.liveQuery()` uses `EventSource`, which this does not replace. `.liveQuery()`'s +[initial backfill](/sdk/streaming#live-queries) is an ordinary request and *does* +go through your function. + +#### Swapping in undici + +The motivating case is a transport bug in the runtime's own HTTP stack, which +you can't fix from inside the SDK — for example +[undici #5600](https://github.com/nodejs/undici/issues/5600): reusing a +keep-alive socket while the event loop is idle stalls the request before it +goes out. How bad it gets varies with the runtime and the idle gap: the +upstream report measured ~450–465 ms against a 10 ms server, and we have +measured anything from ~100 ms to tens of seconds on a server answering +instantly. It affects undici 8.8.0–8.9.0, and Node 26 bundles 8.9.0. + +**Upgrading undici is the actual fix** — it landed in 8.10.0. What +`options.fetch` buys you is a way to get there without waiting for a new +runtime: install undici yourself and route requests through it. The one +non-obvious part is that you must pass its dispatcher **explicitly**. + +```ts +import { createClient } from '@wavehouse/sdk'; +import { Agent, fetch as undiciFetch } from 'undici'; // npm install undici — 8.10.0+ + +// Explicit, not implied — see below. +const dispatcher = new Agent(); + +const wh = createClient({ + baseURL: 'https://wavehouse.example.com', + options: { + fetch: (url, init) => + undiciFetch(url as string, { ...init, dispatcher } as never) as unknown as Promise, + }, +}); +``` + +:::caution[Importing a fixed undici is not enough on its own] +undici keeps its connection pool on a shared `globalThis` symbol, and whichever +copy loads first claims it. Node claims it for the *bundled* copy the first time +anything touches one of its web globals — a `fetch()` call, but equally a +`new Headers()` or `new Response()` — not at startup. So which undici owns the +pool comes down to a load order you don't really control, and auditing your own +code for `fetch` calls won't tell you: any dependency can claim it first, and so +can this SDK, which constructs a `Headers` on its abort and retry-exhausted +paths. Call `undiciFetch` without a `dispatcher` and it resolves whatever is on +that symbol, so requests can go through the buggy pool even though you imported +a fixed undici. + +Measured with 8.9.0 loaded first and 8.10.0's `fetch` doing the request, 1.5 s +idle gaps against a 10 ms server: + +``` + per-request ms +no explicit dispatcher 21 1514 1495 583 ← still stalling +explicit new Agent() 17 14 12 13 +``` + +An explicit `dispatcher` wins because the request never consults the shared +symbol at all. An explicit `setGlobalDispatcher` call (below) wins the other +way round — it overwrites the symbol after the first copy claimed it. +::: + +All three casts are load-bearing, for one underlying reason: undici declares its +own request/response types, separate from the ones behind your global `fetch`, +so the two aren't structurally assignable. `url as string` is needed because +`FetchLike` accepts the full `fetch` input union while undici's `RequestInfo` +names its *own* `Request`; `{ ...init, dispatcher } as never` covers the same +split on `RequestInit`, which differs on `body` and `headers` (`never` is +assignable to either spelling, so one snippet compiles whether or not your `lib` +includes DOM); and the return cast handles the `Response` mismatch. They're safe because +of the narrow runtime contract above — a string URL and a plain `RequestInit` +in, six members read back. `undici` is a dependency you add; the SDK itself +stays dependency-free. + +If you're genuinely pinned to an affected undici, the same snippet fixes it — +your pinned version, with a dispatcher that never reuses a keep-alive socket: +`new Agent({ pipelining: 0 })`. That costs a fresh connection per request, so +prefer upgrading. Note that tuning +`keepAliveTimeout` does **not** help: the retirement timer is starved by the same +idle event loop that triggers the bug, so the socket is still there to be reused. + +To change the transport process-wide instead of per-client, an installed undici's +`setGlobalDispatcher(new Agent())` is picked up by Node's built-in global +`fetch`, no `options.fetch` needed. That's an explicit write, so it wins the +race described above — but only if the *installed* undici is 8.10.0+, since it +hands connection handling to that copy. + ### Type-Safe Tables Pass a `Database` type to get autocomplete on table names and row types: diff --git a/docs/src/content/docs/sdk/pipes.md b/docs/src/content/docs/sdk/pipes.md index 3e1cca2f..f0e5b8ed 100644 --- a/docs/src/content/docs/sdk/pipes.md +++ b/docs/src/content/docs/sdk/pipes.md @@ -21,7 +21,23 @@ const { data } = await wh.pipe('top_pages', { start_date: '2026-01-01', limit: 5 ### `.fetch(opts?)` -Execute and return results. +Execute and return results. Takes `PipeRequestOptions` — `{ signal }` only, +narrower than the `.fetch(opts?)` on a [query builder](/sdk/queries), which also +accepts `limit`. Passing a `limit` is a compile error rather than a silent +no-op. + +`limit` is typed `never` rather than left out, so the rejection also catches a +value passed in a variable — leaving it out would only reject an inline object. +That cuts both ways: a value *declared* as `RequestOptions` is rejected whether +or not it actually carries a limit, since the type permits one. If you share one +options object across calls, type it as `PipeRequestOptions` — the table and +query-builder `.fetch()` accept that too — or inline `{ signal }` at the pipe +call. + +There is no per-call row cap here: the endpoint binds your `params` as the +pipe's parameters, so a limit has to be declared in the pipe's SQL as +`{{limit}}` (see [Named Pipes](/pipes)) and passed as `wh.pipe(name, { limit })`, +as in the example above. ### `.stream(opts?)` diff --git a/docs/src/content/docs/sdk/queries.md b/docs/src/content/docs/sdk/queries.md index 0da64b91..58a6506d 100644 --- a/docs/src/content/docs/sdk/queries.md +++ b/docs/src/content/docs/sdk/queries.md @@ -223,13 +223,16 @@ if (hasMore && next) { } ``` -**Options:** +**Options** — `RequestOptions`: | Field | Type | Description | |-------|------|-------------| | `signal` | `AbortSignal` | Cancel the request | | `limit` | `number` | Override builder limit for this fetch | +A pipe's `.fetch()` takes the narrower `PipeRequestOptions` instead — see +[Pipes](/sdk/pipes#fetchopts). + ### `.stream(opts?)` Open a live stream from the builder's table. See [Streaming](/sdk/streaming). diff --git a/docs/src/content/docs/sdk/reference.md b/docs/src/content/docs/sdk/reference.md index 040c4044..71d1727e 100644 --- a/docs/src/content/docs/sdk/reference.md +++ b/docs/src/content/docs/sdk/reference.md @@ -63,7 +63,7 @@ createClient(config) → WaveHouseClient │ ├── .schema() → Promise> │ └── .stream(opts?) → StreamController ├── .pipe(name, params?) → PipeRef (PromiseLike) -│ ├── .fetch(opts?) → Promise> +│ ├── .fetch(opts?) → Promise> // { signal } only — no limit │ └── .stream(opts?) → StreamController ├── .pipes (admin) │ ├── .list() → Promise>