ci: expand faithful SDK compliance coverage

.github/workflows/sdk-compliance.yml

@@ -13,20 +13,46 @@ on:
 
 jobs:
   compliance:
-    name: PostHog SDK compliance tests (capture v0)
-    # Keep the harness image version pinned separately so v0/v1 runs are reproducible.
+    name: PostHog SDK compliance (${{ matrix.profile }})
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - profile: v0-gzip
+            dockerfile: Dockerfile
+          - profile: v1-gzip
+            dockerfile: Dockerfile.v1
+          - profile: v1-deflate
+            dockerfile: Dockerfile.v1.deflate
+          - profile: v1-br
+            dockerfile: Dockerfile.v1.br
+          - profile: v1-zstd
+            dockerfile: Dockerfile.v1.zstd
     uses: PostHog/posthog-sdk-test-harness/.github/workflows/test-sdk-action.yml@03d972e49be84402c491324320b0a0f38c2ddc53
     with:
-      adapter-dockerfile: "sdk_compliance_adapter/Dockerfile"
+      adapter-dockerfile: sdk_compliance_adapter/${{ matrix.dockerfile }}
       adapter-context: "."
-      test-harness-version: "0.10.0"
-      report-name: "sdk-compliance-report-v0"
+      test-harness-version: "1.0.0"
+      sdk-type: server
+      concurrency: 1
+      continue-on-error: true
+      report-name: sdk-compliance-report-${{ matrix.profile }}
 
-  compliance-v1:
-    name: PostHog SDK compliance tests (capture v1)
-    uses: PostHog/posthog-sdk-test-harness/.github/workflows/test-sdk-action.yml@03d972e49be84402c491324320b0a0f38c2ddc53
-    with:
-      adapter-dockerfile: "sdk_compliance_adapter/Dockerfile.v1"
-      adapter-context: "."
-      test-harness-version: "0.10.0"
-      report-name: "sdk-compliance-report-v1"
+  report-inventory:
+    name: Verify compliance report inventory
+    needs: compliance
+    if: always()
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      actions: read
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
+        with:
+          pattern: sdk-compliance-report-*
+          path: reports
+      # Assertion failures remain advisory; absent/truncated/wrong-profile reports do not.
+      - run: |
+          python3 -m unittest discover -s sdk_compliance_adapter -p 'test_*.py'
+          python3 sdk_compliance_adapter/check_reports.py reports

sdk_compliance_adapter/CONTRIBUTING.md

@@ -1,54 +1,105 @@
 # Contributing
 
-This package contains the PostHog Go SDK compliance adapter used with the PostHog SDK Test Harness.
+This adapter builds against the checked-out Go SDK, not a separately published SDK dependency.
 
-## Running tests
+## Local checks
 
-Tests run automatically in CI via GitHub Actions.
+From the repository root:
 
-CI runs two jobs: `compliance` (capture v0, `Dockerfile`) and `compliance-v1`
-(capture v1, `Dockerfile.v1`). The only difference between the images is the
-`CAPTURE_MODE=v1` env var, which flips the adapter's `/health` capabilities and
-selects `posthog.CaptureModeAnalyticsV1` at init. Both jobs pin the reusable
-workflow to the 0.10.0 release commit and run the `0.10.0` harness image.
+```sh
+go test -race ./sdk_compliance_adapter
+python3 -m unittest discover -s sdk_compliance_adapter -p 'test_*.py'
+```
 
-### Locally with Docker Compose
+Adapter tests use an available loopback port; set `COMPLIANCE_TEST_PORT` to reserve
+a specific port. Run `make test` for the complete Go test suite.
 
-Run the full compliance suite from the `sdk_compliance_adapter` directory:
+## Compliance profiles
 
-```bash
-docker-compose up --build --abort-on-container-exit
-```
+CI pins the reusable workflow at `03d972e49be84402c491324320b0a0f38c2ddc53`
+and the harness image at `1.0.0` (contract 1.2). Each profile gets a separate report artifact:
 
-This will:
+| Profile | Dockerfile | Capture tests | Flag tests |
+| --- | --- | ---: | ---: |
+| v0-gzip | `Dockerfile` | 30 | 17 |
+| v1-gzip | `Dockerfile.v1` | 95 | 17 |
+| v1-deflate | `Dockerfile.v1.deflate` | 94 | 17 |
+| v1-br | `Dockerfile.v1.br` | 94 | 17 |
+| v1-zstd | `Dockerfile.v1.zstd` | 94 | 17 |
 
-1. Build the Go SDK adapters (v0 on `:8080`, v1 on `:8082`)
-2. Pull the test harness image
-3. Run the capture v0 compliance tests against the v0 adapter
+Both protocol suites include the non-UTC timestamp override test. `CAPTURE_MODE=v1`
+selects `posthog.CaptureModeAnalyticsV1`; `COMPRESSION` selects the codec used when
+`enable_compression:true`. Omitted compression retains the SDK default (none), and
+explicit false always selects none. Each process advertises only its selected codec.
+The extra V1 profiles replace the two gzip-only tests with one codec header test.
+The harness does not decode deflate, Brotli or zstd bodies and returns an empty V1
+results map for those requests. The SDK treats UUIDs absent from results as accepted
+without firing callbacks, so the adapter cannot observe their completion. The three
+codec header tests currently time out at `/flush` before reaching their assertions,
+although the SDK emits the requested encodings. These are harness decoding and
+adapter completion gaps, not failing codecs or passing header assertions. The
+adapter's own codec tests decode actual SDK requests and verify delivery callbacks
+and retries using UUID-keyed results.
 
-> **Note:** `docker-compose` currently targets the v0 adapter only. The v1
-> adapter image is built to verify it compiles, but v1 compliance tests run in
-> CI via the separate `compliance-v1` workflow job. To run v1 locally, use the
-> manual Docker instructions below with `Dockerfile.v1`.
+### Public SDK mapping
 
-### Manually with Docker
+- Capture uses `NewWithConfig` and `Enqueue(Capture)`. Timestamp input is parsed as
+  `time.Time`; UTC normalization, UUID generation, batching and retries remain SDK-owned.
+- Flags use `EvaluateFlags` with singleton `FlagKeys`, then snapshot `GetFlag`.
+  SDK transport, response parsing, retries and deduplicated `$feature_flag_called`
+  events are exercised. Each action waits for SDK exposure delivery callbacks before
+  returning, so a subsequent mock reset cannot receive the previous action's events.
+  No personal API key or local evaluator is configured, so each action evaluates
+  remotely regardless of `force_remote`.
+- `BeforeSend` observes the SDK-generated UUID without changing the event.
+  Public `Callback` notifications track successful and terminally failed events;
+  the transport passively records actual attempts, including encoded bodies.
+- The Go client has no non-closing immediate flush. `/flush` waits for the configured
+  SDK interval and terminal callbacks, bounded to 30 seconds (or request cancellation).
+  It returns HTTP 504 rather than claiming completion on timeout, including when V1
+  responses omit event UUIDs and the SDK supplies no callback. `events_flushed`
+  counts successful callbacks during that wait. It does not close/recreate the client.
+  Default adapter batching is one event / 20 ms, with explicit init options forwarded.
+- Reset and reinit close the old client before clearing its observations. Actions are
+  serialized; parallel test isolation is not supported.
 
-```bash
-# Create network
-docker network create test-network
+### Known contract differences
+
+Expected reports are 45/47 for V0 gzip, 110/112 for V1 gzip, and 108/111 for each
+alternate V1 codec (the completion timeout above plus two flag failures).
+
+All 17 flag tests remain selected. Two are expected to fail through the native SDK:
 
-# Build and run adapter (use Dockerfile.v1 to exercise capture v1)
-docker build -f sdk_compliance_adapter/Dockerfile -t posthog-go-adapter .
-docker run -d --name sdk-adapter --network test-network -p 8080:8080 posthog-go-adapter
+- `feature_flags.request_payload.disable_geoip_false_propagates_as_geoip_disable_false`:
+  the SDK omits the false field on the wire.
+- `feature_flags.request_payload.disable_geoip_omitted_defaults_to_false`:
+  the documented server SDK default is true.
 
-# Run test harness
-docker run --rm \
-  --name test-harness \
-  --network test-network \
-  ghcr.io/posthog/sdk-test-harness:0.10.0 \
-  run --adapter-url http://sdk-adapter:8080 --mock-url http://test-harness:8081
+Dedicated AI capture is not supported. Compliance assertions remain advisory, but
+`report-inventory` fails when a profile report is absent, truncated, or lacks the
+expected suite counts and UTC/codec/flag cases. Inspect each profile's artifact for
+actual failures; an advisory job conclusion or the shared workflow's PR comment
+is not a complete multi-profile result.
 
-# Cleanup
+## Run with Docker
+
+From the repository root (choose any Dockerfile from the table):
+
+```sh
+docker network create test-network
+docker build -f sdk_compliance_adapter/Dockerfile.v1.deflate -t posthog-go-adapter .
+docker run -d --name sdk-adapter --network test-network posthog-go-adapter
+docker run --rm --name test-harness --network test-network \
+  ghcr.io/posthog/sdk-test-harness:1.0.0 \
+  run --adapter-url http://sdk-adapter:8080 --mock-url http://test-harness:8081 \
+  --sdk-type server --concurrency 1
 docker stop sdk-adapter && docker rm sdk-adapter
 docker network rm test-network
 ```
+
+`docker compose up --build --abort-on-container-exit` from this directory runs V0
+only; V1 has separate CI profiles. For native development, build with
+`go build -o /tmp/posthog-go-adapter ./sdk_compliance_adapter`, then set `PORT`,
+`CAPTURE_MODE` and `COMPRESSION` when launching. `/init` accepts HTTP mock URLs on
+explicit ports at `127.0.0.1`, `localhost`, `::1`, or Docker's `test-harness` host.
+Confirm both adapter and mock HTTP readiness before invoking the pinned suites.

sdk_compliance_adapter/Dockerfile.v1.br

@@ -0,0 +1,29 @@
+FROM golang:1.21-alpine AS builder
+
+WORKDIR /app
+
+# Copy the SDK source
+COPY go.mod go.sum ./
+COPY *.go ./
+
+# Copy adapter
+COPY sdk_compliance_adapter/main.go /app/adapter/
+
+# Download dependencies
+RUN go mod download
+
+# Build adapter
+RUN cd /app/adapter && go build -o /app/adapter-server main.go
+
+FROM alpine:latest
+
+WORKDIR /app
+
+COPY --from=builder /app/adapter-server /app/adapter-server
+
+# Select the capture-v1 protocol; same binary, different runtime mode.
+ENV CAPTURE_MODE=v1 COMPRESSION=br
+
+EXPOSE 8080
+
+CMD ["/app/adapter-server"]

sdk_compliance_adapter/Dockerfile.v1.deflate

@@ -0,0 +1,29 @@
+FROM golang:1.21-alpine AS builder
+
+WORKDIR /app
+
+# Copy the SDK source
+COPY go.mod go.sum ./
+COPY *.go ./
+
+# Copy adapter
+COPY sdk_compliance_adapter/main.go /app/adapter/
+
+# Download dependencies
+RUN go mod download
+
+# Build adapter
+RUN cd /app/adapter && go build -o /app/adapter-server main.go
+
+FROM alpine:latest
+
+WORKDIR /app
+
+COPY --from=builder /app/adapter-server /app/adapter-server
+
+# Select the capture-v1 protocol; same binary, different runtime mode.
+ENV CAPTURE_MODE=v1 COMPRESSION=deflate
+
+EXPOSE 8080
+
+CMD ["/app/adapter-server"]

sdk_compliance_adapter/Dockerfile.v1.zstd

@@ -0,0 +1,29 @@
+FROM golang:1.21-alpine AS builder
+
+WORKDIR /app
+
+# Copy the SDK source
+COPY go.mod go.sum ./
+COPY *.go ./
+
+# Copy adapter
+COPY sdk_compliance_adapter/main.go /app/adapter/
+
+# Download dependencies
+RUN go mod download
+
+# Build adapter
+RUN cd /app/adapter && go build -o /app/adapter-server main.go
+
+FROM alpine:latest
+
+WORKDIR /app
+
+COPY --from=builder /app/adapter-server /app/adapter-server
+
+# Select the capture-v1 protocol; same binary, different runtime mode.
+ENV CAPTURE_MODE=v1 COMPRESSION=zstd
+
+EXPOSE 8080
+
+CMD ["/app/adapter-server"]

sdk_compliance_adapter/check_reports.py

@@ -0,0 +1,54 @@
+"""Verify the pinned harness's Markdown inventory, not advisory job conclusions."""
+
+import re
+import sys
+from pathlib import Path
+
+PROFILES = {"v0-gzip": 30, "v1-gzip": 95, "v1-deflate": 94, "v1-br": 94, "v1-zstd": 94}
+
+
+def check_report(profile: str, report: str) -> str:
+    capture_count = PROFILES[profile]
+    suite = "capture" if profile.startswith("v0-") else "capture_v1"
+    expected = {suite: capture_count, "feature_flags": 17}
+    sections = re.split(r"^## (Capture|Capture_V1|Feature_Flags) Tests\n", report, flags=re.M)
+    observed = {}
+    for name, section in zip(sections[1::2], sections[2::2]):
+        rows = re.findall(r"^\| (.+?) \| [✅❌] \| \d+ms \|$", section, re.M)
+        observed[name.lower()] = len(rows)
+        if len(rows) != len(set(rows)):
+            raise ValueError(f"{profile}: duplicate test rows")
+    if observed != expected:
+        raise ValueError(f"{profile}: expected {expected}, got {observed}")
+    total = capture_count + 17
+    summary = re.search(r"\*\*(\d+)/(\d+)\*\* tests passed", report)
+    if not summary or int(summary[2]) != total:
+        raise ValueError(f"{profile}: missing or unexpected summary total (expected {total})")
+    passed_rows = len(re.findall(r"^\| .+? \| ✅ \| \d+ms \|$", report, re.M))
+    if passed_rows != int(summary[1]):
+        raise ValueError(f"{profile}: summary pass count differs from test rows")
+    required = [
+        "non_utc_event_timestamp_is_converted_to_utc",
+        "retries_flags_on_502",
+        "retries_flags_on_504",
+        "disable_geoip_false_propagates_as_geoip_disable_false",
+        "disable_geoip_omitted_defaults_to_false",
+    ]
+    if suite == "capture_v1":
+        required.append(f"sends_{profile.split('-')[1]}_content_encoding")
+    for name in required:
+        if name.replace("_", " ").title() + " |" not in report:
+            raise ValueError(f"{profile}: missing test {name}")
+    passed = int(summary[1])
+    return f"{profile}: selected={total}, passed={passed}, failed={total-passed}"
+
+
+def main() -> None:
+    root = Path(sys.argv[1])
+    for profile in PROFILES:
+        path = root / f"sdk-compliance-report-{profile}" / "sdk-compliance-report.md"
+        print(check_report(profile, path.read_text()))
+
+
+if __name__ == "__main__":
+    main()

sdk_compliance_adapter/docker-compose.yml

@@ -21,7 +21,7 @@ services:
 
   # Test harness
   test-harness:
-    image: ghcr.io/posthog/sdk-test-harness:0.10.0
+    image: ghcr.io/posthog/sdk-test-harness:1.0.0
     command: ["run", "--adapter-url", "http://sdk-adapter:8080", "--mock-url", "http://test-harness:8081"]
     networks:
       - test-network