From 2ff4802b0434744814bfa6be5e2acb52f96e9ef1 Mon Sep 17 00:00:00 2001 From: Anna Garcia Date: Fri, 11 Sep 2026 13:48:13 -0400 Subject: [PATCH] docs(compliance): re-audit posthog-node Traces after 5.52.0 posthog-node 5.52.0 shipped native tracing, so the Traces row moves from Fail to Partial: 22 of 24 requirements met, with two retry-pacing gaps. Claude-Session: https://claude.ai/code/session_01QPLdjqTgK93WtVxnhx6aVr --- compliance/README.md | 4 ++-- compliance/posthog-node.md | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/compliance/README.md b/compliance/README.md index 22d79b6..e80d10e 100644 --- a/compliance/README.md +++ b/compliance/README.md @@ -114,7 +114,7 @@ rather than treating it as a routine new-row fill-in. | posthog-python | 14/62 fully compliant (23%; 29 contracts N/A on a server SDK) | 14 | 12 | 7 | 29 | 0 | 2026-08-17 · `95c7f6e0` | 19 | [posthog-python.md](posthog-python.md) | | posthog-android | 31/59 fully compliant (53%; **59 rows, needs +3 new contracts**) | 31 | 16 | 9 | 3 | 0 | 2026-08-10 · `8659a7b4` | 25 | [posthog-android.md](posthog-android.md) | | posthog-ios | 32/62 fully compliant (52%) | 32 | 18 | 7 | 5 | 0 | 2026-08-17 · `c0218386` | 25 | [posthog-ios.md](posthog-ios.md) | -| posthog-node | 10/62 fully compliant (16%; 28 contracts N/A on a server SDK) | 10 | 16 | 8 | 28 | 0 | 2026-08-17 · `fbdb6c7b` (posthog-js monorepo) | 24 | [posthog-node.md](posthog-node.md) | +| posthog-node | 10/62 fully compliant (16%; 28 contracts N/A on a server SDK) | 10 | 17 | 7 | 28 | 0 | 2026-08-17 · `fbdb6c7b` (posthog-js monorepo; Traces re-audited 2026-09-11 · `910b0925`) | 24 | [posthog-node.md](posthog-node.md) | | posthog-flutter | 20/59 fully compliant (34%; **59 rows, needs +3 new contracts**) | 20 | 22 | 6 | 5 | 6 | 2026-08-06 · `05b53dc` | 34 | [posthog-flutter.md](posthog-flutter.md) | | posthog-react-native | 31/59 fully compliant (53%; **59 rows, needs +3 new contracts**) | 31 | 23 | 2 | 2 | 1 | 2026-08-06 · `e1efa57` (posthog-js monorepo — now stale, see below) | 26 | [posthog-react-native.md](posthog-react-native.md) | | posthog-php | 12/59 fully compliant (20%; **59 rows, needs +3 new contracts**) | 12 | 9 | 7 | 31 | 0 | 2026-08-06 · `ed93a67` | 16 | [posthog-php.md](posthog-php.md) | @@ -191,7 +191,6 @@ run, since both were re-verified together in the same monorepo clone. | posthog-ios | Session Replay Privacy | Needs deprecation path | Password-field precedence bug in screenshot mode is now fixed, but default wireframe-capture mode still never checks `ph-no-capture` for plain `UIView`s — a silent privacy leak persists — [posthog-ios.md#n26](posthog-ios.md) | | posthog-ios | Surveys | Backward-compatible | New intro-screen requirement (`displayIntroScreen` and friends) is entirely unimplemented; the parallel trailing `thankYouMessage*` fields already exist — [posthog-ios.md#n27](posthog-ios.md) | | posthog-ios | Traces | Backward-compatible | No OTLP span/traces implementation anywhere — [posthog-ios.md#n2](posthog-ios.md) | -| posthog-node | Traces | Backward-compatible | No OTLP `/i/v1/traces` pipeline anywhere in the monorepo (industry-wide gap, matches posthog-python/-js) — [posthog-node.md#n2](posthog-node.md) | | posthog-node | Capture Exception | Needs deprecation path | Deeper Exception Event Metadata verification surfaced overlapping hard failures (hardcoded `handled`/severity, missing flat properties, inverted precedence) — [posthog-node.md#n7](posthog-node.md) | | posthog-node | Is Feature Enabled | Backward-compatible | Neither `isFeatureEnabled()` nor its successor accepts a caller `defaultValue` (hard SHALL, no server carve-out) — [posthog-node.md#n13](posthog-node.md) | | posthog-node | Set/Reset Person/Group Properties For Flags (×4) | Backward-compatible | Zero implementation; methods exist only on the client-only base class node doesn't extend — [posthog-node.md#n15](posthog-node.md) | @@ -238,6 +237,7 @@ run, since both were re-verified together in the same monorepo clone. | posthog-python | Retry Queue | Needs deprecation path | Same drop-on-failure behavior with observable `on_error`/blocking-timing implications — [posthog-python.md#n17](posthog-python.md) | | posthog-python | Is Feature Enabled | Backward-compatible | Legacy `feature_enabled()` still lacks a caller default, but its designated successor `evaluate_flags(...).is_enabled(default_value=...)` fully satisfies the spec — upgraded from ❌ last run — [posthog-python.md#n9](posthog-python.md) | | posthog-node | Capture AI | Backward-compatible | `privacyMode` is declared/documented but never wired to the client, so it silently fails to override `enableFullAiCapture` at config level as required — [posthog-node.md#n6](posthog-node.md) | +| posthog-node | Traces | Backward-compatible | Implemented in 5.52.0; automatic sends are not held back during plain backoff, and one batch goes out inside an open `Retry-After` window after a batch is retired — [posthog-node.md#n2](posthog-node.md) | | posthog-node | Local Feature Flag Evaluator | Backward-compatible | New unrecognized-operator-degrades-to-inconclusive requirement is correctly implemented in code, but the acceptance `.feature` file hasn't been updated with the new scenarios yet (test-asset sync gap, not a runtime defect) — downgraded from ✅ this run — [posthog-node.md#n24](posthog-node.md) | | posthog-ios | Exception Event Metadata | Mixed (see note) | No `exception_id`/`parent_id` tree linkage or `chained` mechanism type on nested exceptions, no `$exception_source`, no 50-entry truncation; manual `captureException` lets caller properties override SDK-owned fields (native-crash path gets this right) — [posthog-ios.md#n21](posthog-ios.md) | | posthog-android | Consent Gating | Backward-compatible | Downgraded from ✅ this run — `optOut()` never stops an active replay session, and session-replay start/stop never check `isOptedOut()`, despite session replay being named in this spec's own scope — [posthog-android.md#n18](posthog-android.md) | diff --git a/compliance/posthog-node.md b/compliance/posthog-node.md index 6cd0530..267a2f6 100644 --- a/compliance/posthog-node.md +++ b/compliance/posthog-node.md @@ -3,14 +3,14 @@ **Repo:** [PostHog/posthog-js](https://github.com/PostHog/posthog-js) (`packages/node` + `packages/core` — the standalone `posthog-node` repo is archived and its code now lives in this monorepo) **Audited commit:** `fbdb6c7b8ffdc523b19b3ea82f8ebcfa8fb065b5` ([commit](https://github.com/PostHog/posthog-js/commit/fbdb6c7b8ffdc523b19b3ea82f8ebcfa8fb065b5)) — audited on 2026-08-17 **Audited against sdk-specs commit:** `0ea0aba45170a56a8778197a3e3bd9c6e9b3dd79` -**Summary:** 10 ✅ · 16 🟡 · 8 ❌ · 28 ➖ · 0 ❓ +**Summary:** 10 ✅ · 17 🟡 · 7 ❌ · 28 ➖ · 0 ❓ -**A note on repo layout:** The `PostHog/posthog-node` repository itself has been archived/redirected — its code was moved into the `posthog-js` monorepo and now lives at `packages/node` (Node-specific integration: the `PostHog` client class, Express/NestJS middleware, error-tracking stack-trace enrichment, feature-flag polling, the new `captureAi`/`ai-capture` route) and `packages/core` (the shared `@posthog/core` library also used by `packages/browser` and `packages/react-native`: the capture/batch/flush/retry engine in `posthog-core-stateless.ts`, the richer ambient-identity `posthog-core.ts` base class used only by client SDKs, logs, error-tracking primitives). This audit clones `posthog-js` (commit above) and inspects `packages/node/src` + `packages/core/src` only — `packages/browser`, `packages/react-native`, and `packages/ai` are out of scope except where they are the *only* place a Node-facing option (e.g. `enableFullAiCapture`) is actually consumed. `packages/node`'s `PostHogBackendClient` still extends the stateless `PostHogCoreStateless`, **not** the richer client-only `PostHogCore`, so most client-only ambient-identity/session/replay primitives (`group()`, `reset()`, `getAnonymousId()`, `onFeatureFlags()`, session replay, bootstrap, etc.) simply don't exist on the Node class at all, making most N/A calls unusually clean-cut. The repo still supports both a legacy "v0" batch-capture pipeline (`POST /batch/`, default) and a newer "v1" pipeline (`POST /i/v1/analytics/events`, opt-in only via `POSTHOG_CAPTURE_MODE=v1` env var, `packages/node/src/capture-v1/config.ts`) — both share one queue/flush/retry engine in `posthog-core-stateless.ts` but differ in transport (`packages/node/src/capture-v1/*`), which matters for several findings below. Since the last audit (2026-08-06, `e1efa572`), the monorepo has added a dedicated AI-event capture route (`packages/node/src/ai-capture/`), a richer flag-evaluation snapshot API (`packages/node/src/feature-flag-evaluations.ts`), and no Traces/OTel-native-spans support in `packages/node` (still only in the separate `packages/ai` package's OTel bridge to `/i/v0/ai/otel`, not `/i/v1/traces`). +**A note on repo layout:** The `PostHog/posthog-node` repository itself has been archived/redirected — its code was moved into the `posthog-js` monorepo and now lives at `packages/node` (Node-specific integration: the `PostHog` client class, Express/NestJS middleware, error-tracking stack-trace enrichment, feature-flag polling, the new `captureAi`/`ai-capture` route) and `packages/core` (the shared `@posthog/core` library also used by `packages/browser` and `packages/react-native`: the capture/batch/flush/retry engine in `posthog-core-stateless.ts`, the richer ambient-identity `posthog-core.ts` base class used only by client SDKs, logs, error-tracking primitives). This audit clones `posthog-js` (commit above) and inspects `packages/node/src` + `packages/core/src` only — `packages/browser`, `packages/react-native`, and `packages/ai` are out of scope except where they are the *only* place a Node-facing option (e.g. `enableFullAiCapture`) is actually consumed. `packages/node`'s `PostHogBackendClient` still extends the stateless `PostHogCoreStateless`, **not** the richer client-only `PostHogCore`, so most client-only ambient-identity/session/replay primitives (`group()`, `reset()`, `getAnonymousId()`, `onFeatureFlags()`, session replay, bootstrap, etc.) simply don't exist on the Node class at all, making most N/A calls unusually clean-cut. The repo still supports both a legacy "v0" batch-capture pipeline (`POST /batch/`, default) and a newer "v1" pipeline (`POST /i/v1/analytics/events`, opt-in only via `POSTHOG_CAPTURE_MODE=v1` env var, `packages/node/src/capture-v1/config.ts`) — both share one queue/flush/retry engine in `posthog-core-stateless.ts` but differ in transport (`packages/node/src/capture-v1/*`), which matters for several findings below. Since the last audit (2026-08-06, `e1efa572`), the monorepo has added a dedicated AI-event capture route (`packages/node/src/ai-capture/`) and a richer flag-evaluation snapshot API (`packages/node/src/feature-flag-evaluations.ts`); and since this file's `fbdb6c7b` audit, posthog-node 5.52.0 ([#4579](https://github.com/PostHog/posthog-js/pull/4579), `19e78cc8`; Traces row re-audited at `910b0925`) added native, dependency-free distributed tracing — `startSpan`/`withSpan`/`getActiveSpan` and a `traces` client option exporting OTLP JSON to `/i/v1/traces`, built on `packages/core/src/traces/` — separate from the unchanged `packages/ai` OTel bridge to `/i/v0/ai/otel`. | # | Contract | Status | Note | |---|----------|--------|------| | 1 | Logs | 🟡 | [n1] | -| 2 | Traces | ❌ | [n2] | +| 2 | Traces | 🟡 | [n2] | | 3 | Tracing Headers | 🟡 | [n3] | | 4 | Alias | 🟡 | [n4] | | 5 | Capture | 🟡 | [n5] | @@ -80,11 +80,11 @@ - **Backwards compatibility:** Backward-compatible — instantiating `PostHogLogs` and exposing it via a new lazy getter/method on `PostHogBackendClient`, mirroring the existing `metrics` getter pattern, is purely additive. - **Remediation:** Add a `captureLog()` method or `logger`/`logs` getter to `packages/node/src/client.ts` that constructs `PostHogLogs` against the already-present `_sendLogsBatch` transport, the same way `metrics` is wired up today. -### n2 — Traces (❌ Fail) -- **Spec requires:** A `startSpan`/`withSpan` API producing OTLP spans shipped to `POST {host}/i/v1/traces`, explicitly distinct from LLM-analytics-as-events, requiring no OpenTelemetry runtime dependency. -- **SDK currently:** Exhaustive grep for `startSpan`/`withSpan`/`i/v1/traces`/`traceparent`/`getActiveSpan` across `packages/core/src`, `packages/browser/src`, `packages/react-native/src`, and `packages/node/src` returns zero matches anywhere in the monorepo. What exists instead is exactly the substitute the spec explicitly excludes: LLM analytics captured as `$ai_generation`/`$ai_trace_id` ordinary analytics events, plus `packages/ai/src/otel/{exporter,processor}.ts`, which posts OTel spans to `${host}/i/v0/ai/otel` (not `/i/v1/traces`), only for `gen_ai.*` AI-semantic-convention spans, and depends on real `@opentelemetry/sdk-trace-base`/`@opentelemetry/exporter-trace-otlp-http` packages — the exact OTel runtime dependency the spec says the canonical implementation must avoid. This bridge lives in the separate `packages/ai` package, which `packages/node`'s `package.json` does not depend on. No SDK in this monorepo implements the canonical contract yet — this is a monorepo-wide gap, not a node-specific regression, and is unchanged since the last audit. -- **Backwards compatibility:** Backward-compatible — a net-new subsystem; no removal needed. -- **Remediation:** Implement a dependency-free `startSpan`/`withSpan` API shipping OTLP spans to `/i/v1/traces` in `@posthog/core`, wired into `packages/node` (and browser/react-native), independent of the `packages/ai` OTel bridge. +### n2 — Traces (🟡 Partial) +- **Spec requires:** A dependency-free `startSpan`/`withSpan`/`getActiveSpan` API producing OTLP spans batched to `POST {host}/i/v1/traces`, with W3C Trace Context interop, inert (no-op / pass-through) handles when tracing can't run, per-span limits, a fail-closed `beforeSpanSend`, bounded live spans and queue, and a retry policy in which the backoff and any `Retry-After` window pace every automatic send — only a caller-driven flush may send inside them. +- **SDK currently:** Re-audited for this row at posthog-js `910b0925` against sdk-specs `4f2f48f4` (posthog-node 5.52.1; tracing shipped in 5.52.0 via [#4579](https://github.com/PostHog/posthog-js/pull/4579) `19e78cc8`, clock-basis fix [#4908](https://github.com/PostHog/posthog-js/pull/4908) `4a3c74bf`). The shared `PostHogTraces` pipeline (`packages/core/src/traces/`) is wired into `PostHogBackendClient` behind a `traces` option that keeps tracing off until set (`client.ts:677-796`), with a per-client `AsyncLocalStorage` span context on the Node entrypoint (`entrypoints/index.node.ts:29-31`) and the documented synchronous fallback on edge. 22 of 24 requirements are met: JSON OTLP with Bearer auth and gzip, a 10 MiB uncompressed pre-send check, strict `traceparent`/`tracestate` handling, has-is-remote/is-remote flags, a monotonic per-trace clock basis, attribute/event/value caps re-applied after the hook, 413 halve-and-ramp, a windowed retry budget, clamped and extendable `Retry-After`, and jitter. Two MUST gaps, both in "Error handling and retries": (1) the events pipeline's automatic flush (`flushAutomatic`, `client.ts:313-317` → `_flushEventsAndSpans(true)`, `client.ts:297-311`) holds spans back only while `throttled`, which reads the `Retry-After` window alone (`core/src/traces/index.ts:350-352`). During ordinary exponential backoff after a header-less 5xx or timeout, every events flush (interval or `flushAt`) calls `traces.flush()`, which clears the traces timer and POSTs (`index.ts:354-394`). The budget isn't charged for these sends (`index.ts:898`), so spans aren't dropped sooner, but the endpoint is hit at the events cadence through an outage. (2) When a batch exhausts its retry budget on a refusal that opens a `Retry-After` window, `_flushInner` returns (`index.ts:976-983`), but `flush()`'s outer loop sees progress and starts another pass immediately (`index.ts:354-369`), so the timer-driven drain sends the next batch inside the window. The SDK's own test pins this ("`flush()`'s outer loop is deliberately not window-aware", `core/src/traces/index.spec.ts:3100-3126`), which fails the scenario "dropping a batch does not end the Retry-After window". SHOULD-level: `captureException` does not carry the active span's trace/span ids; Node has no `captureLog` to correlate. +- **Backwards compatibility:** Backward-compatible — both fixes only hold back automatic sends the spec forbids; no public API or default changes, and the API is marked `@experimental`. +- **Remediation:** (1) Have `PostHogTraces.throttled` (or a new `backingOff` getter) also report true while `_consecutiveFlushFailures > 0` and the backoff delay has not elapsed, so `flushAutomatic` skips spans during backoff as it already does during `Retry-After`; explicit `flush()` and `waitUntil` drains stay exempt. (2) Make the automatic drain window-aware — e.g. `_flushInBackground` stops looping when `_retryAfter.isOpen()` after a pass — and change `index.spec.ts:3100` to expect one send per instant, not two. ### n3 — Tracing Headers (🟡 Partial) - **Spec requires:** `both` applicability. Request-scoped middleware extracting `X-POSTHOG-DISTINCT-ID`/`X-POSTHOG-SESSION-ID` case-insensitively; sanitize (trim/strip control chars/cap length); explicit call-site values take precedence over context; personless-capture fallback unless the caller explicitly supplied `$process_person_profile`; flag-evaluation APIs no-op/return-empty when no identity is resolvable; per-request isolation; headers never used for auth.