diff --git a/acceptance/public/capture-ai.feature b/acceptance/public/capture-ai.feature index 98d469d..83f8c8a 100644 --- a/acceptance/public/capture-ai.feature +++ b/acceptance/public/capture-ai.feature @@ -18,6 +18,22 @@ Feature: Capture AI And the call should return the enqueued event's uuid And the enqueued event should include a timestamp and uuid + @server + Scenario: AI capture drops null-valued properties on its delivery route + Given the SDK is initialized with token "test-token" + And the SDK supports AI capture with no property-changing hooks or filters + When capture_ai is called with distinct id "user-123", event "$ai_generation", and custom properties represented by JSON: + """json + {"test":null,"nested":{"drop":null},"items":["1",null,2]} + """ + And the SDK is flushed + Then one event named "$ai_generation" should be received on the AI batch endpoint + And its custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,2]} + """ + And the analytics batch endpoint should receive no events + @server Scenario: capture and capture_ai ride separate routes Given the SDK is initialized with token "test-token" diff --git a/acceptance/public/capture-exception.feature b/acceptance/public/capture-exception.feature index d449183..65b345d 100644 --- a/acceptance/public/capture-exception.feature +++ b/acceptance/public/capture-exception.feature @@ -33,6 +33,23 @@ Feature: Capture Exception | handled | true | | area | checkout | + @both + Scenario: Exception capture drops null-valued custom properties on the wire + Given the SDK is initialized with token "test-token" + And capture has a valid distinct id and no property-changing hooks or filters + And a valid handled exception + When capture exception is called for the exception with additional custom properties represented by JSON: + """json + {"test":null,"nested":{"drop":null},"items":["1",null,2]} + """ + And the SDK is flushed + Then one "$exception" event should be received + And its custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,2]} + """ + And the event should still include its SDK-generated exception data + @both Scenario: Exception capture normalizes non-standard thrown values Given the SDK is initialized with token "test-token" diff --git a/acceptance/public/capture.feature b/acceptance/public/capture.feature index f65f605..23dc041 100644 --- a/acceptance/public/capture.feature +++ b/acceptance/public/capture.feature @@ -42,6 +42,137 @@ Feature: Capture | $lib | any | And the enqueued event should include an event uuid + @both + Scenario: Queued capture drops null-valued properties and preserves array positions + Given the SDK is initialized with token "test-token" + And capture has a valid distinct id and no property-changing hooks or filters + When capture is called with event "Nullable Properties" and custom properties represented by JSON: + """json + {"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null},[null]],"empty":"","zero":0,"enabled":false,"literal":"null","emptyArray":[]} + """ + And the SDK is flushed + Then one event named "Nullable Properties" should be received + And its custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,2,{},[null]],"empty":"","zero":0,"enabled":false,"literal":"null","emptyArray":[]} + """ + And the absent custom property "missing" should remain absent + + @both + Scenario: Immediate capture drops null-valued properties and preserves array positions + Given the SDK is initialized with token "test-token" + And the SDK supports immediate delivery + And capture has a valid distinct id and no property-changing hooks or filters + And capture is configured for immediate delivery + When capture is called with event "Nullable Properties" and custom properties represented by JSON: + """json + {"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null},[null]]} + """ + And the immediate send completes + Then one event named "Nullable Properties" should be received + And its custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,2,{},[null]]} + """ + + @both + Scenario: All-null custom properties do not drop the event + Given the SDK is initialized with token "test-token" + And capture has a valid distinct id and no property-changing hooks or filters + When capture is called with event "Only Null Properties" and custom properties represented by JSON: + """json + {"test":null} + """ + And the SDK is flushed + Then one event named "Only Null Properties" should be received + And its custom properties should equal JSON: + """json + {} + """ + And it should retain its normal SDK metadata + + @both + Scenario: Null object properties introduced by before-send are omitted + Given the SDK is initialized with token "test-token" + And the SDK supports before-send with a valid distinct id + And before-send adds custom properties represented by JSON: + """json + {"hookNull":null,"hookItems":[null,{"drop":null}]} + """ + When capture is called with event "Hook Properties" and no custom properties + And the SDK is flushed + Then one event named "Hook Properties" should be received + And its custom properties should equal JSON: + """json + {"hookItems":[null,{}]} + """ + + @both + Scenario: JavaScript null and undefined object properties are omitted + Given a JavaScript SDK is initialized with token "test-token" + And capture has a valid distinct id and no property-changing hooks or filters + When capture is called with event "Null And Undefined" and JavaScript properties: + """javascript + { test: null, missing: undefined, items: ["1", null, 2] } + """ + And the SDK is flushed + Then one event named "Null And Undefined" should be received + And its custom properties should equal JSON: + """json + {"items":["1",null,2]} + """ + + @both + Scenario: Existing nullable capture inputs remain supported + Given the SDK's existing public capture API accepts null or undefined property values + And an existing caller supplies those values using the supported public property types + When the SDK adopts the serialization cleanup rule + Then the same caller code should remain accepted by the public API, including compilation/type-checking where enforced + And capture should not reject the call or require caller-side filtering solely because of those values + And serializing the captured event should omit null/undefined-valued object members while preserving array positions + + @both + Scenario: Disk-backed event serialization omits null object properties + Given the SDK is initialized with token "test-token" + And the SDK has disk-backed event persistence, a valid distinct id, and no property-changing hooks or filters + And network delivery is paused + When capture is called with event "Persisted Properties" and custom properties represented by JSON: + """json + {"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null}]} + """ + And the SDK serializes the queued event to disk + Then the persisted event's decoded custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,2,{}]} + """ + And no network request should have been sent + When the persisted event is restored and delivered + Then the received event's custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,2,{}]} + """ + + @both + Scenario: JavaScript disk serialization omits undefined object members without compacting arrays + Given a JavaScript SDK is initialized with token "test-token" + And the SDK has disk-backed event persistence, a valid distinct id, and no property-changing hooks or filters + And network delivery is paused + When capture is called with event "Persisted Undefined" and JavaScript properties: + """javascript + { test: null, missing: undefined, nested: { missing: undefined }, items: ["1", undefined, null, 2], literal: "undefined" } + """ + And the SDK serializes the queued event to disk + Then the persisted event's decoded custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,null,2],"literal":"undefined"} + """ + And no network request should have been sent + When the persisted event is restored and delivered + Then the received event's custom properties should equal JSON: + """json + {"nested":{},"items":["1",null,null,2],"literal":"undefined"} + """ + @both Scenario: Capture honors opt-out state Given the SDK is initialized with token "test-token" diff --git a/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/.openspec.yaml b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/.openspec.yaml new file mode 100644 index 0000000..7a8e2be --- /dev/null +++ b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-08 diff --git a/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/design.md b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/design.md new file mode 100644 index 0000000..448206f --- /dev/null +++ b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/design.md @@ -0,0 +1,29 @@ +## Context + +The capture contract needs a consistent serialization policy that distinguishes object members from array elements without breaking existing public input types. Backend object-null removal is planned and still in progress, not universally deployed. + +Android already drops null-valued map entries but also drops null list elements, so it does not fully match this target. Python PR #926 proposes top-level omission. SDK implementation and rollout are separate from this specification change. + +## Goals / Non-Goals + +**Goals:** Omit null/undefined-valued custom object members when serializing events for the wire or disk. Preserve array positions and existing public API acceptance across ordinary, immediate, AI, and exception capture. + +**Non-Goals:** SDK/backend implementation, widening currently non-nullable APIs, adding persistence, changing generic storage/feature-flag caches, or changing reserved-field validation and OTLP encoding. + +## Decisions + +- Define one shared capture requirement and reference it from AI and exception capture. +- Clean object members recursively, including objects inside arrays. Do not compact arrays or discard objects made empty by cleanup. Preserve `false`, `0`, empty strings, and literal strings such as `"null"` and `"undefined"`. +- Preserve null array elements. JavaScript undefined array entries follow normal JSON serialization as null without shifting positions. +- Preserve existing signatures, property value types, and runtime acceptance. Callers must not need casts, filtering, or a different API solely because of this normalization. In-memory events may retain null/undefined until serialization. +- Apply cleanup during serialization for network delivery and disk-backed event queues/caches. Do not change hook timing; the wire payload must still satisfy cleanup after enrichment and before-send processing. +- Treat AI normalization as a specific exception to payload pass-through, not permission for manual-payload redaction, truncation, or media processing. +- Validate disk contents before delivery, then validate the restored event on the wire. An event whose custom properties are all removed must still be delivered if otherwise admissible. + +## Risks / Trade-offs + +- Existing serialized output changes → require per-SDK compatibility review and rollout without narrowing supported public inputs. +- Backend work is unfinished → document it as planned behavior, not an existing universal guarantee. +- Recursive cleanup can accidentally compact arrays → fixtures include null positions and objects emptied inside arrays. +- Event normalization could affect unrelated caches → limit persistence cleanup to captured events. +- Some SDKs lack nullable inputs or disk queues → condition relevant scenarios on existing support rather than require new APIs or persistence. diff --git a/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/proposal.md b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/proposal.md new file mode 100644 index 0000000..5ca7ac2 --- /dev/null +++ b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/proposal.md @@ -0,0 +1,30 @@ +## Why + +SDKs should omit null/undefined-valued object properties when serializing captured events, while preserving array positions and existing public APIs that accept those values. The owner selected object-member omission to align with planned backend removal; that backend work is still in progress. + +## What Changes + +- Omit null/undefined-valued custom object members recursively during event serialization for wire delivery and disk-backed event queues/caches. +- Preserve null array elements, array positions, non-null values, and objects made empty by cleanup. +- Preserve public signatures, property types, and runtime acceptance for existing nullable/undefined inputs. Do not require callers to pre-filter their properties. +- Apply the shared contract to queued, immediate, AI, and exception capture, including values introduced by `before_send`. +- Add ten acceptance scenarios covering wire output, existing API compatibility, and event persistence/restore. +- **BREAKING** serialized-output change for SDKs currently retaining null-valued object members. SDK compatibility review and rollout remain separate; public inputs must not be narrowed. + +## Capabilities + +### New Capabilities + +None. + +### Modified Capabilities + +- `capture`: Define object-member omission at serialization boundaries, preserve array positions, and protect existing public input APIs. +- `capture-ai`: Apply the shared contract to the AI route. +- `capture-exception`: Apply the shared contract to additional custom properties without changing SDK-owned metadata rules. + +## Impact + +Specifications and acceptance scenarios only. No SDK or backend implementation changes. This does not require widening currently non-nullable APIs, adding disk persistence, or changing unrelated caches, privacy controls, reserved-field validation, or OTLP logs/traces rules. + +Context: https://github.com/PostHog/posthog-python/pull/926 diff --git a/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture-ai/spec.md b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture-ai/spec.md new file mode 100644 index 0000000..4d1de02 --- /dev/null +++ b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture-ai/spec.md @@ -0,0 +1,13 @@ +## ADDED Requirements + +### Requirement: AI capture drops null-valued object properties + +`capture_ai` and its immediate/awaitable variants SHALL follow capture's "Capture drops null-valued object properties" requirement for caller-supplied custom properties, including nested objects and objects inside arrays. Null array elements SHALL retain their positions. This is a specific normalization exception to manual AI capture's payload pass-through promise, not permission to add redaction, truncation, or media processing. + +#### Scenario: AI capture drops null-valued properties on its delivery route (@server) +- **GIVEN** an initialized SDK supporting AI capture with no property-changing hooks or filters +- **WHEN** capture_ai is called with distinct id "user-123", event "$ai_generation", and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2]}` +- **AND** the SDK is flushed +- **THEN** one event named "$ai_generation" should be received on the AI batch endpoint +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2]}` +- **AND** the analytics batch endpoint should receive no events diff --git a/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture-exception/spec.md b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture-exception/spec.md new file mode 100644 index 0000000..089e1d6 --- /dev/null +++ b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture-exception/spec.md @@ -0,0 +1,14 @@ +## ADDED Requirements + +### Requirement: Exception capture drops null-valued custom object properties + +`capture_exception` / `captureException` SHALL follow capture's "Capture drops null-valued object properties" requirement for caller-supplied additional custom event properties, including nested objects and objects inside arrays. Null array elements SHALL retain their positions. This MUST NOT change exception-input validation or field-specific rules for SDK-owned exception metadata. + +#### Scenario: Exception capture drops null-valued custom properties on the wire (@both) +- **GIVEN** an initialized SDK with a valid distinct id and no property-changing hooks or filters +- **AND** a valid handled exception +- **WHEN** capture exception is called for the exception with additional custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2]}` +- **AND** the SDK is flushed +- **THEN** one "$exception" event should be received +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2]}` +- **AND** the event should still include its SDK-generated exception data diff --git a/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture/spec.md b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture/spec.md new file mode 100644 index 0000000..33ca30a --- /dev/null +++ b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/specs/capture/spec.md @@ -0,0 +1,80 @@ +## ADDED Requirements + +### Requirement: Capture drops null-valued object properties + +The SDK SHALL omit custom event object properties whose values are explicit nulls (for example Python `None`, JavaScript `null`, or Swift `NSNull()`) or `undefined` where supported by the runtime, when serializing events for wire delivery or disk-backed event queues/caches. A null/undefined-valued property SHALL NOT cause an otherwise valid, previously supported capture call to be rejected. The SDK MUST NOT serialize that object member as JSON `null` or convert its value to the string `"null"` or `"undefined"`. + +This is a serialization rule, not a public API restriction. If an SDK's public capture API already accepts null/undefined property values, it SHALL continue accepting them with the same signatures and property value types. Implementations MUST NOT narrow those types, introduce null/undefined input validation errors for previously supported values, or require callers to pre-filter properties to comply. The rule does not require widening APIs that do not currently accept those values. In-memory event representations MAY retain them until serialization. This compatibility rule also applies to the AI and exception capture methods that inherit this requirement. + +This cleanup SHALL apply recursively to nested objects, including objects inside arrays. Null array elements SHALL remain JSON `null` at their original indices; cleanup MUST NOT compact or reorder arrays. JavaScript undefined array entries SHALL follow normal JSON serialization as null array elements without changing their positions. Other values, including `false`, `0`, empty strings, the literal strings `"null"` and `"undefined"`, and empty objects/arrays, SHALL remain unchanged by this cleanup. Objects made empty by removing null/undefined-valued members SHALL remain `{}`, including when they occupy an array slot. If all custom properties are removed, the SDK SHALL still send an otherwise admissible event with its normal SDK metadata. + +The same rule SHALL apply to queued and immediate/synchronous capture. The wire payload SHALL satisfy it after enrichment and `before_send` processing, including for null/undefined-valued custom object members introduced by a hook. Each disk serialization of a captured event SHALL apply the same object-member cleanup to the event being written; cleanup only at network send time is insufficient for SDKs that persist events. This does not require adding disk persistence or changing hook timing. Events restored from disk SHALL also satisfy the wire serialization rule. Missing custom keys MUST NOT be synthesized. Caller-configured privacy filtering and event drops remain authoritative; cleanup MUST NOT restore removed data. + +This requirement concerns custom event property values, not a missing/null properties container, required envelope fields, or reserved fields with their own type/validation contracts. It does not change those field-specific rules, unrelated persistent-storage/cache semantics, or the OTLP attribute rules for logs and traces. + +#### Scenario: Queued capture drops null-valued properties and preserves array positions (@both) +- **GIVEN** an initialized SDK with a valid distinct id and no property-changing hooks or filters +- **WHEN** capture is called with event "Nullable Properties" and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null},[null]],"empty":"","zero":0,"enabled":false,"literal":"null","emptyArray":[]}` +- **AND** the SDK is flushed +- **THEN** one event named "Nullable Properties" should be received +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2,{},[null]],"empty":"","zero":0,"enabled":false,"literal":"null","emptyArray":[]}` +- **AND** the absent custom property "missing" should remain absent + +#### Scenario: Immediate capture drops null-valued properties and preserves array positions (@both) +- **GIVEN** an initialized SDK supporting immediate delivery with a valid distinct id and no property-changing hooks or filters +- **AND** capture is configured for immediate delivery +- **WHEN** capture is called with event "Nullable Properties" and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null},[null]]}` +- **AND** the immediate send completes +- **THEN** one event named "Nullable Properties" should be received +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2,{},[null]]}` + +#### Scenario: All-null custom properties do not drop the event (@both) +- **GIVEN** an initialized SDK with a valid distinct id and no property-changing hooks or filters +- **WHEN** capture is called with event "Only Null Properties" and custom properties represented by JSON `{"test":null}` +- **AND** the SDK is flushed +- **THEN** one event named "Only Null Properties" should be received +- **AND** its custom properties should equal JSON `{}` +- **AND** it should retain its normal SDK metadata + +#### Scenario: Null object properties introduced by before-send are omitted (@both) +- **GIVEN** an initialized SDK supporting before-send with a valid distinct id +- **AND** before-send adds custom properties represented by JSON `{"hookNull":null,"hookItems":[null,{"drop":null}]}` +- **WHEN** capture is called with event "Hook Properties" and no custom properties +- **AND** the SDK is flushed +- **THEN** one event named "Hook Properties" should be received +- **AND** its custom properties should equal JSON `{"hookItems":[null,{}]}` + +#### Scenario: JavaScript null and undefined object properties are omitted (@both) +- **GIVEN** an initialized JavaScript SDK with a valid distinct id and no property-changing hooks or filters +- **WHEN** capture is called with event "Null And Undefined" and JavaScript properties `{ test: null, missing: undefined, items: ["1", null, 2] }` +- **AND** the SDK is flushed +- **THEN** one event named "Null And Undefined" should be received +- **AND** its custom properties should equal JSON `{"items":["1",null,2]}` + +#### Scenario: Existing nullable capture inputs remain supported (@both) +- **GIVEN** the SDK's existing public capture API accepts null or undefined property values +- **AND** an existing caller supplies those values using the supported public property types +- **WHEN** the SDK adopts the serialization cleanup rule +- **THEN** the same caller code should remain accepted by the public API, including compilation/type-checking where enforced +- **AND** capture should not reject the call or require caller-side filtering solely because of those values +- **AND** serializing the captured event should omit null/undefined-valued object members while preserving array positions + +#### Scenario: Disk-backed event serialization omits null object properties (@both) +- **GIVEN** an initialized SDK with disk-backed event persistence, a valid distinct id, and no property-changing hooks or filters +- **AND** network delivery is paused +- **WHEN** capture is called with event "Persisted Properties" and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null}]}` +- **AND** the SDK serializes the queued event to disk +- **THEN** the persisted event's decoded custom properties should equal JSON `{"nested":{},"items":["1",null,2,{}]}` +- **AND** no network request should have been sent +- **WHEN** the persisted event is restored and delivered +- **THEN** the received event's custom properties should equal JSON `{"nested":{},"items":["1",null,2,{}]}` + +#### Scenario: JavaScript disk serialization omits undefined object members without compacting arrays (@both) +- **GIVEN** an initialized JavaScript SDK with disk-backed event persistence, a valid distinct id, and no property-changing hooks or filters +- **AND** network delivery is paused +- **WHEN** capture is called with event "Persisted Undefined" and JavaScript properties `{ test: null, missing: undefined, nested: { missing: undefined }, items: ["1", undefined, null, 2], literal: "undefined" }` +- **AND** the SDK serializes the queued event to disk +- **THEN** the persisted event's decoded custom properties should equal JSON `{"nested":{},"items":["1",null,null,2],"literal":"undefined"}` +- **AND** no network request should have been sent +- **WHEN** the persisted event is restored and delivered +- **THEN** the received event's custom properties should equal JSON `{"nested":{},"items":["1",null,null,2],"literal":"undefined"}` diff --git a/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/tasks.md b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/tasks.md new file mode 100644 index 0000000..1b7a9f6 --- /dev/null +++ b/openspec/changes/archive/2026-09-08-preserve-nullable-capture-api/tasks.md @@ -0,0 +1,12 @@ +## 1. Contract and acceptance + +- [x] 1.1 Define recursive null/undefined object-member omission during event serialization while preserving array positions and existing public APIs. +- [x] 1.2 Apply the shared contract to AI and exception capture. +- [x] 1.3 Add ten acceptance scenarios for wire delivery, nullable input compatibility, and disk persistence/restore. + +## 2. Validation and archive + +- [x] 2.1 Validate OpenSpec and Gherkin syntax, scenario coverage, and JSON input/output examples. +- [x] 2.2 Sync canonical specs and consolidate the final contract into one archive relative to the PR base. + +Validation covers specification consistency and serialized JSON fixtures, not SDK/backend conformance. API compatibility was inspected statically; no SDK compilation, runtime, or persistence tests were run. diff --git a/openspec/specs/capture-ai/spec.md b/openspec/specs/capture-ai/spec.md index 24094cb..28ecdd7 100644 --- a/openspec/specs/capture-ai/spec.md +++ b/openspec/specs/capture-ai/spec.md @@ -120,9 +120,7 @@ shape), 8 MiB per-event cap, ~5 MiB batch target, compression per SDK default retries with backoff. Capture v1 re-implements the same contract on `/i/v1/ai/...` with per-event outcomes; that cutover must not change anything in the Requirements below. - ## Requirements - ### Requirement: Canonical capture_ai behavior Server SDKs that support AI capture SHALL implement `capture_ai` as described @@ -205,3 +203,15 @@ SDK's AI wrapper libraries capture. It SHALL NOT alter payloads passed to - **AND** the client has privacy mode enabled and the flag set to true - **THEN** the captured event should carry no input or output content +### Requirement: AI capture drops null-valued object properties + +`capture_ai` and its immediate/awaitable variants SHALL follow capture's "Capture drops null-valued object properties" requirement for caller-supplied custom properties, including nested objects and objects inside arrays. Null array elements SHALL retain their positions. This is a specific normalization exception to manual AI capture's payload pass-through promise, not permission to add redaction, truncation, or media processing. + +#### Scenario: AI capture drops null-valued properties on its delivery route (@server) +- **GIVEN** an initialized SDK supporting AI capture with no property-changing hooks or filters +- **WHEN** capture_ai is called with distinct id "user-123", event "$ai_generation", and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2]}` +- **AND** the SDK is flushed +- **THEN** one event named "$ai_generation" should be received on the AI batch endpoint +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2]}` +- **AND** the analytics batch endpoint should receive no events + diff --git a/openspec/specs/capture-exception/spec.md b/openspec/specs/capture-exception/spec.md index a1e6c2c..4cc3a9a 100644 --- a/openspec/specs/capture-exception/spec.md +++ b/openspec/specs/capture-exception/spec.md @@ -84,9 +84,7 @@ captureException( - **Error-tracking processors/builders** — these convert raw errors/exceptions into the structured PostHog exception property format. - **Event batcher / retry queue** — `$exception` events are delivered through the same batching/retry infrastructure as other events. - **Autocaptured exception systems** — this is the manual companion to automatic exception capture; both typically produce the same event family. - ## Requirements - ### Requirement: Canonical capture-exception behavior The SDK SHALL implement the canonical `capture-exception` behavior described by this spec. Implementations MAY adapt method names, parameter casing, type syntax, and lifecycle hooks to platform idioms where this spec explicitly allows variation, but MUST preserve the observable outcomes in the scenarios below. @@ -256,3 +254,16 @@ When the supplied error-like input already carries stack trace information (for - **THEN** one event named "$exception" should be enqueued - **AND** the enqueued exception's stacktrace frames should reflect the pre-existing stack trace, not a synthesized single-frame stack + +### Requirement: Exception capture drops null-valued custom object properties + +`capture_exception` / `captureException` SHALL follow capture's "Capture drops null-valued object properties" requirement for caller-supplied additional custom event properties, including nested objects and objects inside arrays. Null array elements SHALL retain their positions. This MUST NOT change exception-input validation or field-specific rules for SDK-owned exception metadata. + +#### Scenario: Exception capture drops null-valued custom properties on the wire (@both) +- **GIVEN** an initialized SDK with a valid distinct id and no property-changing hooks or filters +- **AND** a valid handled exception +- **WHEN** capture exception is called for the exception with additional custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2]}` +- **AND** the SDK is flushed +- **THEN** one "$exception" event should be received +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2]}` +- **AND** the event should still include its SDK-generated exception data diff --git a/openspec/specs/capture/spec.md b/openspec/specs/capture/spec.md index e07abae..083e4f0 100644 --- a/openspec/specs/capture/spec.md +++ b/openspec/specs/capture/spec.md @@ -143,9 +143,7 @@ The following steps are the canonical flow for a single capture call. Client and - **Autocapture / pageview / heatmaps** — these features internally call capture with reserved event names (`$autocapture`, `$pageview`, `$pageleave`, `$heatmap`, `$web_vitals`). - **Exception capture** — `captureException` wraps `capture` with event name `$exception` and a normalized stacktrace property. Calling `capture('$exception', ...)` directly produces a warning in SDKs that ship `captureException` (posthog-js, posthog-python). - **Before-send hook / privacy filters** — capture is the integration point for drop/mutate hooks; see the before-send hook spec. - ## Requirements - ### Requirement: Canonical capture behavior The SDK SHALL implement the canonical `capture` behavior described by this spec. Implementations MAY adapt method names, parameter casing, type syntax, and lifecycle hooks to platform idioms where this spec explicitly allows variation, but MUST preserve the observable outcomes in the scenarios below. @@ -228,3 +226,82 @@ The SDK SHALL implement the canonical `capture` behavior described by this spec. - **AND** no event named "Sensitive Event" should be enqueued - **AND** no network request should be sent - **AND** the SDK should record a before-send warning + +### Requirement: Capture drops null-valued object properties + +The SDK SHALL omit custom event object properties whose values are explicit nulls (for example Python `None`, JavaScript `null`, or Swift `NSNull()`) or `undefined` where supported by the runtime, when serializing events for wire delivery or disk-backed event queues/caches. A null/undefined-valued property SHALL NOT cause an otherwise valid, previously supported capture call to be rejected. The SDK MUST NOT serialize that object member as JSON `null` or convert its value to the string `"null"` or `"undefined"`. + +This is a serialization rule, not a public API restriction. If an SDK's public capture API already accepts null/undefined property values, it SHALL continue accepting them with the same signatures and property value types. Implementations MUST NOT narrow those types, introduce null/undefined input validation errors for previously supported values, or require callers to pre-filter properties to comply. The rule does not require widening APIs that do not currently accept those values. In-memory event representations MAY retain them until serialization. This compatibility rule also applies to the AI and exception capture methods that inherit this requirement. + +This cleanup SHALL apply recursively to nested objects, including objects inside arrays. Null array elements SHALL remain JSON `null` at their original indices; cleanup MUST NOT compact or reorder arrays. JavaScript undefined array entries SHALL follow normal JSON serialization as null array elements without changing their positions. Other values, including `false`, `0`, empty strings, the literal strings `"null"` and `"undefined"`, and empty objects/arrays, SHALL remain unchanged by this cleanup. Objects made empty by removing null/undefined-valued members SHALL remain `{}`, including when they occupy an array slot. If all custom properties are removed, the SDK SHALL still send an otherwise admissible event with its normal SDK metadata. + +The same rule SHALL apply to queued and immediate/synchronous capture. The wire payload SHALL satisfy it after enrichment and `before_send` processing, including for null/undefined-valued custom object members introduced by a hook. Each disk serialization of a captured event SHALL apply the same object-member cleanup to the event being written; cleanup only at network send time is insufficient for SDKs that persist events. This does not require adding disk persistence or changing hook timing. Events restored from disk SHALL also satisfy the wire serialization rule. Missing custom keys MUST NOT be synthesized. Caller-configured privacy filtering and event drops remain authoritative; cleanup MUST NOT restore removed data. + +This requirement concerns custom event property values, not a missing/null properties container, required envelope fields, or reserved fields with their own type/validation contracts. It does not change those field-specific rules, unrelated persistent-storage/cache semantics, or the OTLP attribute rules for logs and traces. + +#### Scenario: Queued capture drops null-valued properties and preserves array positions (@both) +- **GIVEN** an initialized SDK with a valid distinct id and no property-changing hooks or filters +- **WHEN** capture is called with event "Nullable Properties" and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null},[null]],"empty":"","zero":0,"enabled":false,"literal":"null","emptyArray":[]}` +- **AND** the SDK is flushed +- **THEN** one event named "Nullable Properties" should be received +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2,{},[null]],"empty":"","zero":0,"enabled":false,"literal":"null","emptyArray":[]}` +- **AND** the absent custom property "missing" should remain absent + +#### Scenario: Immediate capture drops null-valued properties and preserves array positions (@both) +- **GIVEN** an initialized SDK supporting immediate delivery with a valid distinct id and no property-changing hooks or filters +- **AND** capture is configured for immediate delivery +- **WHEN** capture is called with event "Nullable Properties" and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null},[null]]}` +- **AND** the immediate send completes +- **THEN** one event named "Nullable Properties" should be received +- **AND** its custom properties should equal JSON `{"nested":{},"items":["1",null,2,{},[null]]}` + +#### Scenario: All-null custom properties do not drop the event (@both) +- **GIVEN** an initialized SDK with a valid distinct id and no property-changing hooks or filters +- **WHEN** capture is called with event "Only Null Properties" and custom properties represented by JSON `{"test":null}` +- **AND** the SDK is flushed +- **THEN** one event named "Only Null Properties" should be received +- **AND** its custom properties should equal JSON `{}` +- **AND** it should retain its normal SDK metadata + +#### Scenario: Null object properties introduced by before-send are omitted (@both) +- **GIVEN** an initialized SDK supporting before-send with a valid distinct id +- **AND** before-send adds custom properties represented by JSON `{"hookNull":null,"hookItems":[null,{"drop":null}]}` +- **WHEN** capture is called with event "Hook Properties" and no custom properties +- **AND** the SDK is flushed +- **THEN** one event named "Hook Properties" should be received +- **AND** its custom properties should equal JSON `{"hookItems":[null,{}]}` + +#### Scenario: JavaScript null and undefined object properties are omitted (@both) +- **GIVEN** an initialized JavaScript SDK with a valid distinct id and no property-changing hooks or filters +- **WHEN** capture is called with event "Null And Undefined" and JavaScript properties `{ test: null, missing: undefined, items: ["1", null, 2] }` +- **AND** the SDK is flushed +- **THEN** one event named "Null And Undefined" should be received +- **AND** its custom properties should equal JSON `{"items":["1",null,2]}` + +#### Scenario: Existing nullable capture inputs remain supported (@both) +- **GIVEN** the SDK's existing public capture API accepts null or undefined property values +- **AND** an existing caller supplies those values using the supported public property types +- **WHEN** the SDK adopts the serialization cleanup rule +- **THEN** the same caller code should remain accepted by the public API, including compilation/type-checking where enforced +- **AND** capture should not reject the call or require caller-side filtering solely because of those values +- **AND** serializing the captured event should omit null/undefined-valued object members while preserving array positions + +#### Scenario: Disk-backed event serialization omits null object properties (@both) +- **GIVEN** an initialized SDK with disk-backed event persistence, a valid distinct id, and no property-changing hooks or filters +- **AND** network delivery is paused +- **WHEN** capture is called with event "Persisted Properties" and custom properties represented by JSON `{"test":null,"nested":{"drop":null},"items":["1",null,2,{"drop":null}]}` +- **AND** the SDK serializes the queued event to disk +- **THEN** the persisted event's decoded custom properties should equal JSON `{"nested":{},"items":["1",null,2,{}]}` +- **AND** no network request should have been sent +- **WHEN** the persisted event is restored and delivered +- **THEN** the received event's custom properties should equal JSON `{"nested":{},"items":["1",null,2,{}]}` + +#### Scenario: JavaScript disk serialization omits undefined object members without compacting arrays (@both) +- **GIVEN** an initialized JavaScript SDK with disk-backed event persistence, a valid distinct id, and no property-changing hooks or filters +- **AND** network delivery is paused +- **WHEN** capture is called with event "Persisted Undefined" and JavaScript properties `{ test: null, missing: undefined, nested: { missing: undefined }, items: ["1", undefined, null, 2], literal: "undefined" }` +- **AND** the SDK serializes the queued event to disk +- **THEN** the persisted event's decoded custom properties should equal JSON `{"nested":{},"items":["1",null,null,2],"literal":"undefined"}` +- **AND** no network request should have been sent +- **WHEN** the persisted event is restored and delivered +- **THEN** the received event's custom properties should equal JSON `{"nested":{},"items":["1",null,null,2],"literal":"undefined"}`