Skip to content

fix: omit null custom event properties during serialization - #314

Draft
marandaneto wants to merge 2 commits into
mainfrom
fix/null-property-serialization-sqq3ojkm
Draft

fix: omit null custom event properties during serialization#314
marandaneto wants to merge 2 commits into
mainfrom
fix/null-property-serialization-sqq3ojkm

Conversation

@marandaneto

@marandaneto marandaneto commented Sep 9, 2026

Copy link
Copy Markdown
Member

💡 Motivation and Context

Custom event properties currently send explicit null object members. This implements the serialization rule in PostHog/sdk-specs#60 for the Go SDK's legacy and v1 event paths.

The event serializer now omits null object members recursively, including objects inside arrays. It preserves null array positions, empty objects and arrays, false, zero, strings, supported numbers, and ordered duplicate JSON members. Existing nullable public inputs and caller-owned values remain unchanged. Cleanup happens after enrichment and before-send processing without restoring privacy-filtered fields.

The shared event serialization boundary covers capture, AI-named ordinary capture, exception custom properties, and identify/group custom properties. It does not change generic Properties JSON or introduce a separate AI route. Existing typed flag response nulls and exception metadata keep their field-specific behavior. There is no blanket exemption for $ or AI properties and no disk queue added.

The correction also preserves the original concrete serialization callback errors. Only the extra wrapper from the private event-property adapter is removed. User marshaler error layers and the existing exception wrapper remain intact. The zero-values fixture now expects null omission, with its nil input and all other assertions preserved.

💚 How did you test it?

Current draft status: The signed and reviewed HEAD is 28ea011afc52a967565102d20c3576671e919007. This includes the correction for the stale null fixture and callback error wrapper that caused four test failures at the previous HEAD. All 28 hosted checks passed for this exact SHA, including Test and Race jobs on Go 1.21 and Go 1.x, Public API, builds, vet, formatting, OTel, security scans, and v0/v1 compliance. Unit Tests: https://github.com/PostHog/posthog-go/actions/runs/34338572018. The bounded watch completed successfully on 2026-09-09. These results supersede the four failures on the previous SHA.

  • Passed 43 selected top-level Go tests with -race -count=1 on Go 1.26.4, Darwin/arm64. These include 84 exact-byte ordered-member cases, 8 malformed/out-of-range rejection controls, and 24 concrete error-chain and user-error identity controls across both protocols and capture/identify/group/exception paths.
  • Actual SDK enqueue/close and feature flag evaluation paths use mocked transports or numeric-loopback HTTP servers with fake keys. Tests inspect received serializer bytes for nested nulls, array positions, hooks, caller nonmutation, all-null events, typed flag metadata, exception properties, and privacy filtering. Compatibility tests check actual client failure callbacks as well as preparation errors.
  • Passed go vet -mod=readonly ., changed-file gofmt checks, and branch whitespace checks. Reused existing offline caches with module downloads disabled. No dependencies were installed.
  • The installed isolated committed-head autoreview passed at 28ea011afc52a967565102d20c3576671e919007 against main 53f54712427369388565577777d13970689395e1, with mandatory TruffleHog clean and no findings. Main is included without a merge because it has not advanced.

Draft validation limits: The full local API snapshot check remains blocked by existing uncached CLI/example dependencies (godotenv, urfave/cli, and go-md2man/v2), as recorded during the previous attempt. It was not retried or repaired here. The full suite, minimum-Go matrix, local-poller/transport-failure flags, retry/compression-specific null payloads, and performance checks were not run locally. Existing goccy rejection of 1e400 remains unchanged. Disk persistence is not applicable. Hosted results are separate from these local limits. Existing compliance harness passes do not establish the missing null-policy fixture coverage.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

An equivalent posthog-go: patch changeset was added manually. The same changeset covers the correction. No package-manager setup or release was run.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented and validated with Pi coding assistance, Git, Go, GitHub CLI, and the installed isolated autoreview helper. The human-directed scope was recursive event-property null omission without public API narrowing, array compaction, or duplicate-key loss. Existing goccy token traversal was used instead of a map round-trip so supported duplicate members and numeric values survive. The callback correction removes only the private adapter's added error layer. Private session logs are not published.

This remains a draft for human review, assigned to marandaneto. No release, auto-merge, or approval is requested by the agent.

@marandaneto marandaneto self-assigned this Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

posthog-go Compliance Report

Date: 2026-09-09 10:11:54 UTC
Duration: 169415ms

✅ All Tests Passed!

111/111 tests passed


Capture_V1 Tests

94/94 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint 26ms
Endpoint And Method.Does Not Use Legacy Endpoints 7ms
Required Headers.Has Authorization Bearer Header 11ms
Required Headers.Has Content Type Json 7ms
Required Headers.Has Posthog Sdk Info Format 18ms
Required Headers.Has Posthog Attempt Header 8ms
Required Headers.Has Posthog Request Id 7ms
Required Headers.Has Posthog Request Timestamp 8ms
Required Headers.Has User Agent 6ms
Body Format.Body Has Created At And Batch 7ms
Body Format.No Api Key In Body 7ms
Body Format.No Sent At In Body 7ms
Event Format.Event Has Required Root Fields 7ms
Event Format.Event Uuid Is Valid 7ms
Event Format.Event Timestamp Is Rfc3339 7ms
Event Format.Distinct Id Is String 6ms
Event Format.Distinct Id At Root Not Properties 7ms
Event Format.Custom Properties Preserved 7ms
Event Format.Set Properties Preserved 6ms
Event Format.Set Once Properties Preserved 7ms
Event Format.Groups Properties Preserved 7ms
Event Format.Sdk Generates Uuid If Not Provided 7ms
Event Format.Event Has Required Root Fields Batch 10ms
Event Format.Event Uuid Is Valid Batch 20ms
Event Format.Event Timestamp Is Rfc3339 Batch 12ms
Event Format.Distinct Id Is String Batch 10ms
Event Format.Distinct Id At Root Not Properties Batch 20ms
Event Format.Custom Properties Preserved Batch 11ms
Event Format.Set Properties Preserved Batch 10ms
Event Format.Set Once Properties Preserved Batch 10ms
Event Format.Groups Properties Preserved Batch 11ms
Event Format.Sdk Generates Uuid If Not Provided Batch 10ms
Batch Behavior.Multiple Events In Single Batch 32ms
Batch Behavior.Batch Envelope Smoke 13ms
Batch Behavior.Flush With No Events Sends Nothing 4ms
Batch Behavior.Flush At Triggers Batch 1008ms
Batch Behavior.Created At Reflects Batch Creation Time 9ms
Deduplication.Generates Unique Uuids 15ms
Deduplication.Different Events Same Content Different Uuids 10ms
Deduplication.Preserves Uuid On Retry 5133ms
Deduplication.Preserves Timestamp On Retry 5137ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 5138ms
Deduplication.No Duplicate Events In Batch 34ms
Header Behavior On Retry.Attempt Header Starts At One 8ms
Header Behavior On Retry.Attempt Header Increments On Retry 10133ms
Header Behavior On Retry.Request Id Preserved On Retry 5138ms
Header Behavior On Retry.Different Requests Have Different Request Ids 2013ms
Header Behavior On Retry.Request Timestamp Changes On Retry 5139ms
Response Format Validation.Success Response Has Uuid Keyed Results 8ms
Response Format Validation.Success Response Has Ok For Each Event 29ms
Response Format Validation.Success No Retry After When All Ok 30ms
Response Format Validation.Success Retry After Present When Retry Events 132ms
Response Format Validation.Success No Retry After When Drop Only 30ms
Response Format Validation.Response Echoes Request Id 8ms
Retry Behavior.Retries On 408 5135ms
Retry Behavior.Retries On 500 5137ms
Retry Behavior.Retries On 503 5134ms
Retry Behavior.Retries On 504 5136ms
Retry Behavior.Retryable Errors Have Retry After 2134ms
Retry Behavior.Respects Retry After On Retryable Error 8141ms
Retry Behavior.Does Not Retry On 400 2134ms
Retry Behavior.Does Not Retry On 401 2135ms
Retry Behavior.Does Not Retry On 402 2133ms
Retry Behavior.Does Not Retry On 413 2134ms
Retry Behavior.Does Not Retry On 415 2135ms
Retry Behavior.Non Retryable Errors Have No Retry After 2134ms
Retry Behavior.Implements Backoff 15147ms
Retry Behavior.Max Retries Respected 15134ms
Partial Batch Handling.Handles 200 Full Success 2010ms
Partial Batch Handling.Handles 200 With All Ok 3033ms
Partial Batch Handling.Does Not Retry Dropped Events 3032ms
Partial Batch Handling.Does Not Retry Limited Events 3033ms
Partial Batch Handling.Prunes Ok Events On Partial Retry 5138ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry 5134ms
Partial Batch Handling.Retries Only Retry Events From Partial 5140ms
Partial Batch Handling.Partial Retry Preserves Uuids 5137ms
Partial Batch Handling.Partial Retry Attempt Header Increments 5138ms
Partial Batch Handling.Partial Retry Request Id Preserved 5138ms
Partial Batch Handling.Respects Retry After On Partial 5138ms
Partial Batch Handling.Unknown Result Treated As Terminal 3033ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry 3035ms
Compression.Sends Gzip Content Encoding 20ms
Compression.No Content Encoding When Disabled 9ms
Compression.Compressed Body Is Decompressible 8ms
Error Handling.Does Not Retry On Unknown 4Xx 2134ms
Event Options.Cookieless Mode Override 8ms
Event Options.Disable Skew Correction Override 7ms
Event Options.Process Person Profile Override 7ms
Event Options.Product Tour Id Override 7ms
Event Options.Unset Options Omitted 7ms
Event Options.Options Override In Batch 10ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties 7ms
Geoip And Historical Migration.Historical Migration Set In Body 8ms
Geoip And Historical Migration.Historical Migration Absent By Default 9ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 17ms
Request Payload.Flags Request Uses V2 Query Param 19ms
Request Payload.Flags Request Hits Flags Path Not Decide 19ms
Request Payload.Flags Request Omits Authorization Header 18ms
Request Payload.Token In Flags Body Matches Init 18ms
Request Payload.Groups Round Trip 18ms
Request Payload.Groups Default To Empty Object 17ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 18ms
Request Payload.Disable Geoip Omitted Defaults To False 19ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 19ms
Request Lifecycle.No Flags Request On Init Alone 3ms
Request Lifecycle.No Flags Request On Normal Capture 8ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 31ms
Request Lifecycle.Mock Response Value Is Returned To Caller 19ms
Retry Behavior.Retries Flags On 502 20ms
Retry Behavior.Retries Flags On 504 20ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 21ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant