Skip to content

ci: expand faithful SDK compliance coverage - #313

Draft
dustinbyrne wants to merge 2 commits into
mainfrom
ci/sdk-harness-coverage-2026-09-07
Draft

ci: expand faithful SDK compliance coverage#313
dustinbyrne wants to merge 2 commits into
mainfrom
ci/sdk-harness-coverage-2026-09-07

Conversation

@dustinbyrne

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Make compliance results exercise the Go SDK rather than adapter-written flag requests, retries, parsing and exposure events.

  • Use public EvaluateFlags with singleton FlagKeys, then snapshot GetFlag.
  • Observe SDK-generated UUIDs and delivery callbacks; fix reset ordering and make flush wait for completion or report a timeout.
  • Pin harness 1.0.0 and add V1 deflate, Brotli and zstd profiles alongside V0/V1 gzip, including current UTC tests. Validate each profile's report inventory while keeping assertions advisory.

Changes are limited to the adapter, tests, documentation and CI. No SDK public API, shipped behavior, default or dependency changes; no SDK release needed.

💚 How did you test it?

  • Full make test passed (Go vet, race detection and coverage); nine adapter tests and two Python report-validation tests passed.
  • Built the actual SDK adapter and ran the pinned harness source against HTTP-ready local mocks on native Go 1.25.4/macOS arm64:
Profile Passed / selected
V0 gzip 45 / 47
V1 gzip 110 / 112
V1 deflate 108 / 111
V1 Brotli 108 / 111
V1 zstd 108 / 111

All profiles expose two existing GeoIP contract differences: explicit false is omitted on the wire, and the native default is true. Each alternate-codec header test additionally times out at /flush: the mock cannot decode those bodies, and the SDK supplies no callbacks for UUIDs absent from V1 results. Actual codec headers were recorded, but those header assertions were not reached. The adapter does not interpret responses to manufacture completion.

Docker execution is deferred to this PR's CI. Inspect the five separate report artifacts and inventory check, not just advisory job conclusions. The SDK has no non-closing immediate flush; the adapter waits for its configured interval and callbacks with a 30-second bound.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with a Pi coding agent using Go, Python, Git and GitHub CLI; a separate read-only agent reviewed the candidate. Human review is required. Local agent session context is available to the author, but has no public session link.

@dustinbyrne dustinbyrne self-assigned this Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

posthog-go Compliance Report

Date: 2026-09-07T20:59:39.600389+00:00
Duration: 228488ms

⚠️ Some Tests Failed

108/111 tests passed, 3 failed


Capture_V1 Tests

⚠️ 93/94 tests passed, 1 failed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint 92ms
Endpoint And Method.Does Not Use Legacy Endpoints 5ms
Required Headers.Has Authorization Bearer Header 14ms
Required Headers.Has Content Type Json 5ms
Required Headers.Has Posthog Sdk Info Format 5ms
Required Headers.Has Posthog Attempt Header 5ms
Required Headers.Has Posthog Request Id 14ms
Required Headers.Has Posthog Request Timestamp 4ms
Required Headers.Has User Agent 5ms
Body Format.Body Has Created At And Batch 5ms
Body Format.No Api Key In Body 4ms
Body Format.No Sent At In Body 5ms
Event Format.Event Has Required Root Fields 4ms
Event Format.Event Uuid Is Valid 5ms
Event Format.Event Timestamp Is Rfc3339 4ms
Event Format.Non Utc Event Timestamp Is Converted To Utc 5ms
Event Format.Distinct Id Is String 4ms
Event Format.Distinct Id At Root Not Properties 4ms
Event Format.Custom Properties Preserved 4ms
Event Format.Set Properties Preserved 5ms
Event Format.Set Once Properties Preserved 4ms
Event Format.Groups Properties Preserved 5ms
Event Format.Sdk Generates Uuid If Not Provided 4ms
Event Format.Event Has Required Root Fields Batch 8ms
Event Format.Event Uuid Is Valid Batch 6ms
Event Format.Event Timestamp Is Rfc3339 Batch 16ms
Event Format.Distinct Id Is String Batch 7ms
Event Format.Distinct Id At Root Not Properties Batch 6ms
Event Format.Custom Properties Preserved Batch 6ms
Event Format.Set Properties Preserved Batch 7ms
Event Format.Set Once Properties Preserved Batch 9ms
Event Format.Groups Properties Preserved Batch 5ms
Event Format.Sdk Generates Uuid If Not Provided Batch 7ms
Batch Behavior.Multiple Events In Single Batch 28ms
Batch Behavior.Batch Envelope Smoke 7ms
Batch Behavior.Flush With No Events Sends Nothing 3ms
Batch Behavior.Flush At Triggers Batch 1005ms
Batch Behavior.Created At Reflects Batch Creation Time 7ms
Deduplication.Generates Unique Uuids 12ms
Deduplication.Different Events Same Content Different Uuids 7ms
Deduplication.Preserves Uuid On Retry 6016ms
Deduplication.Preserves Timestamp On Retry 6021ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 6043ms
Deduplication.No Duplicate Events In Batch 30ms
Header Behavior On Retry.Attempt Header Starts At One 6ms
Header Behavior On Retry.Attempt Header Increments On Retry 12021ms
Header Behavior On Retry.Request Id Preserved On Retry 6022ms
Header Behavior On Retry.Different Requests Have Different Request Ids 2034ms
Header Behavior On Retry.Request Timestamp Changes On Retry 6022ms
Response Format Validation.Success Response Has Uuid Keyed Results 7ms
Response Format Validation.Success Response Has Ok For Each Event 26ms
Response Format Validation.Success No Retry After When All Ok 26ms
Response Format Validation.Success Retry After Present When Retry Events 1035ms
Response Format Validation.Success No Retry After When Drop Only 27ms
Response Format Validation.Response Echoes Request Id 4ms
Retry Behavior.Retries On 408 6021ms
Retry Behavior.Retries On 500 6023ms
Retry Behavior.Retries On 503 7022ms
Retry Behavior.Retries On 504 6023ms
Retry Behavior.Retryable Errors Have Retry After 3019ms
Retry Behavior.Respects Retry After On Retryable Error 11024ms
Retry Behavior.Does Not Retry On 400 2008ms
Retry Behavior.Does Not Retry On 401 2009ms
Retry Behavior.Does Not Retry On 402 2009ms
Retry Behavior.Does Not Retry On 413 2009ms
Retry Behavior.Does Not Retry On 415 2009ms
Retry Behavior.Non Retryable Errors Have No Retry After 2018ms
Retry Behavior.Implements Backoff 18023ms
Retry Behavior.Max Retries Respected 18032ms
Partial Batch Handling.Handles 200 Full Success 2008ms
Partial Batch Handling.Handles 200 With All Ok 3030ms
Partial Batch Handling.Does Not Retry Dropped Events 3029ms
Partial Batch Handling.Does Not Retry Limited Events 3030ms
Partial Batch Handling.Prunes Ok Events On Partial Retry 6042ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry 6043ms
Partial Batch Handling.Retries Only Retry Events From Partial 6043ms
Partial Batch Handling.Partial Retry Preserves Uuids 6043ms
Partial Batch Handling.Partial Retry Attempt Header Increments 6042ms
Partial Batch Handling.Partial Retry Request Id Preserved 6043ms
Partial Batch Handling.Respects Retry After On Partial 8042ms
Partial Batch Handling.Unknown Result Treated As Terminal 3031ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry 3033ms
Compression.Sends Deflate Content Encoding 30006ms
Compression.No Content Encoding When Disabled 17ms
Error Handling.Does Not Retry On Unknown 4Xx 2007ms
Event Options.Cookieless Mode Override 6ms
Event Options.Disable Skew Correction Override 5ms
Event Options.Process Person Profile Override 4ms
Event Options.Product Tour Id Override 4ms
Event Options.Unset Options Omitted 4ms
Event Options.Options Override In Batch 7ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties 4ms
Geoip And Historical Migration.Historical Migration Set In Body 5ms
Geoip And Historical Migration.Historical Migration Absent By Default 4ms

Failures

compression.sends_deflate_content_encoding

504, message='Gateway Timeout', url='http://sdk-adapter:8080/flush'

Feature_Flags Tests

⚠️ 15/17 tests passed, 2 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 15ms
Request Payload.Flags Request Uses V2 Query Param 14ms
Request Payload.Flags Request Hits Flags Path Not Decide 14ms
Request Payload.Flags Request Omits Authorization Header 14ms
Request Payload.Token In Flags Body Matches Init 14ms
Request Payload.Groups Round Trip 15ms
Request Payload.Groups Default To Empty Object 14ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 14ms
Request Payload.Disable Geoip Omitted Defaults To False 14ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 14ms
Request Lifecycle.No Flags Request On Init Alone 2ms
Request Lifecycle.No Flags Request On Normal Capture 5ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 16ms
Request Lifecycle.Mock Response Value Is Returned To Caller 14ms
Retry Behavior.Retries Flags On 502 317ms
Retry Behavior.Retries Flags On 504 318ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 15ms

Failures

request_payload.disable_geoip_false_propagates_as_geoip_disable_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['api_key', 'distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate']

request_payload.disable_geoip_omitted_defaults_to_false

Expected geoip_disable=False, got True

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant