Skip to content

feat: add captureAi on the capture V1 AI endpoint - #939

Draft
turnipdabeets wants to merge 2 commits into
feat/v4-capture-v1from
feat/v4-capture-ai
Draft

turnipdabeets wants to merge 2 commits into
feat/v4-capture-v1from
feat/v4-capture-ai

Conversation

@turnipdabeets

@turnipdabeets turnipdabeets commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Adds captureAi(...), which sends AI observability events ($ai_generation, $ai_span, $ai_trace, $ai_embedding, ...) to the capture V1 AI endpoint (POST /i/v1/ai/events). That endpoint accepts events up to 8 MiB. It follows the capture_ai lanes in posthog-rs (PostHog/posthog-rs#250), posthog-go (PostHog/posthog-go#323) and posthog-python (PostHog/posthog-python#1017, #1022), and Ingestion's goal of shipping captureAi with every V1 release. Stacked on #938.

  • captureAi takes the same arguments as capture and goes through the same enrichment and beforeSend. Only the delivery route differs. capture never reroutes $ai_* events.
  • It uses the capture V1 transport from feat!: send analytics events through capture v1 #938: Bearer auth, V1 headers, per-event results, retry identity, same-origin redirects and the Retry-After cap. Retry identity is now tracked per endpoint, so the two lanes don't reset each other's attempt numbers. A 404 is terminal for AI: there is no /batch fallback.
  • AI events go to their own disk queue (posthog.aiFolder). It is flushed by flush(), stopped by close(), and created with the other queues so events left on disk by a previous launch still send.
  • captureAi returns the queued event's UUID (@discardableResult), or nil when the event isn't queued: opted out, disabled, dropped by beforeSend, or over the size limit.
  • The queue batches by bytes as well as count, with a 5 MiB soft limit (new internal QueueEndpoint.maxBatchBytes). It drops an event over 8 MiB + 64 KiB when it is queued, logging only the event name and size.
  • The queue keeps at most 100 events and 50 MiB on disk, dropping the oldest first (new internal QueueEndpoint.maxQueueBytes, enforced in PostHogFileBackedQueue on add and on load). Without the byte cap, 100 events at the size limit could take ~800 MiB.
  • The compliance adapter adds POST /capture_ai and advertises capture_ai_v1.

Differences from the spec and the other SDKs

The capture-ai spec only applies to server SDKs. Where iOS differs:

  • No uuid argument: iOS capture doesn't take one either. captureAi does return the UUID, like the spec.
  • Queue created at setup, not lazily: an offline mobile queue has to send events persisted by an earlier launch.
  • Queue limit is 100 events and 50 MiB, not 1000 events: keeps the disk use of multi-MB events down on devices.
  • Gzip follows config.compression: the server SDKs send AI requests uncompressed by default.

💚 How did you test it?

  • New Swift Testing suite "Capture V1 AI lane", using MockPostHogServer's new /i/v1/ai/events stub:
    • routing by method, including beforeSend and options
    • a 404 is terminal, with no /batch request
    • an oversized event is dropped when queued
    • requests split at the byte limit
    • captureAi returns the sent event's UUID, and nil for an event that isn't queued
  • PostHogFileBackedQueue tests for the byte cap: evicts the oldest, keeps an oversized newest record, trims on load.
  • Removing the 404 path guard or the byte limit makes the matching test fail.
  • make lint passes. make test: 1157 tests. The only failures are the 13 known group-container issues, which also fail on the base branch locally.
  • make buildSdk succeeds (run with xcpretty stubbed out, since it isn't installed locally). make apiCheck adds only captureAi; local Xcode also re-records unrelated @Sendable lines, which are left out.
  • SDK compliance harness 1.14.0, capture_ai_v1 suite: 95/101, the same in two runs.
    • The two UUID tests failed because captureAi didn't take or return a UUID. The adapter now returns it, as posthog-python's does; the test that passes its own uuid still can't pass. This was measured before the UUID change, so CI will have the new count. One of them aborts before flushing and its request lands in the next test (event_has_required_root_fields_batch).
    • non_utc_event_timestamp_is_converted_to_utc needs ci: run the SDK compliance harness 1.14.0 #936's adapter timestamp parsing on /capture_ai.
    • max_retries_respected and unset_options_omitted are the same mobile failures as in the analytics suite.
  • Not done: docs for captureAi on posthog.com / LLM analytics.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Built with Claude Code. The design follows the sdk-specs capture-ai spec, the V1 AI route in rust/capture/src/v1, and the posthog-rs, posthog-go and posthog-python AI lanes. Maintainer decisions: a separate AI lane with a small disk queue and no /batch fallback, PR'd separately from #938.

🤖 Generated with Claude Code

@turnipdabeets
turnipdabeets added this pull request to stack #941 October 10, 2026 15:23
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

posthog-ios Compliance Report

Date: 2026-10-10 18:17:15 UTC
Duration: 480693ms

⚠️ Some Tests Failed

107/110 tests passed, 3 failed


Capture_V1 Tests

⚠️ 91/94 tests passed, 3 failed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint ✅ 2960ms
Endpoint And Method.Does Not Use Legacy Endpoints ✅ 2802ms
Required Headers.Has Authorization Bearer Header ✅ 2760ms
Required Headers.Has Content Type Json ✅ 2873ms
Required Headers.Has Posthog Sdk Info Format ✅ 3809ms
Required Headers.Has Posthog Attempt Header ✅ 2890ms
Required Headers.Has Posthog Request Id ✅ 2685ms
Required Headers.Has Posthog Request Timestamp ✅ 2799ms
Required Headers.Has User Agent ✅ 2845ms
Body Format.Body Has Created At And Batch ✅ 2753ms
Body Format.No Api Key In Body ✅ 2826ms
Body Format.No Sent At In Body ✅ 2900ms
Event Format.Event Has Required Root Fields ✅ 2845ms
Event Format.Event Uuid Is Valid ✅ 564ms
Event Format.Event Timestamp Is Rfc3339 ✅ 2970ms
Event Format.Distinct Id Is String ✅ 2831ms
Event Format.Distinct Id At Root Not Properties ✅ 2901ms
Event Format.Custom Properties Preserved ✅ 376ms
Event Format.Set Properties Preserved ✅ 2834ms
Event Format.Set Once Properties Preserved ✅ 2788ms
Event Format.Groups Properties Preserved ✅ 325ms
Event Format.Sdk Generates Uuid If Not Provided ✅ 446ms
Event Format.Event Has Required Root Fields Batch ✅ 2934ms
Event Format.Event Uuid Is Valid Batch ✅ 2881ms
Event Format.Event Timestamp Is Rfc3339 Batch ✅ 2740ms
Event Format.Distinct Id Is String Batch ✅ 2920ms
Event Format.Distinct Id At Root Not Properties Batch ✅ 424ms
Event Format.Custom Properties Preserved Batch ✅ 2902ms
Event Format.Set Properties Preserved Batch ✅ 2765ms
Event Format.Set Once Properties Preserved Batch ✅ 2797ms
Event Format.Groups Properties Preserved Batch ✅ 2795ms
Event Format.Sdk Generates Uuid If Not Provided Batch ✅ 2840ms
Batch Behavior.Multiple Events In Single Batch ✅ 2933ms
Batch Behavior.Batch Envelope Smoke ✅ 2928ms
Batch Behavior.Flush With No Events Sends Nothing ✅ 233ms
Batch Behavior.Flush At Triggers Batch ✅ 1327ms
Batch Behavior.Created At Reflects Batch Creation Time ✅ 2779ms
Deduplication.Generates Unique Uuids ✅ 2833ms
Deduplication.Different Events Same Content Different Uuids ✅ 2866ms
Deduplication.Preserves Uuid On Retry ✅ 9207ms
Deduplication.Preserves Timestamp On Retry ✅ 9289ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 7983ms
Deduplication.No Duplicate Events In Batch ✅ 2820ms
Header Behavior On Retry.Attempt Header Starts At One ✅ 2807ms
Header Behavior On Retry.Attempt Header Increments On Retry ✅ 10439ms
Header Behavior On Retry.Request Id Preserved On Retry ✅ 8911ms
Header Behavior On Retry.Different Requests Have Different Request Ids ✅ 7708ms
Header Behavior On Retry.Request Timestamp Changes On Retry ✅ 9179ms
Response Format Validation.Success Response Has Uuid Keyed Results ✅ 2706ms
Response Format Validation.Success Response Has Ok For Each Event ✅ 3024ms
Response Format Validation.Success No Retry After When All Ok ✅ 2863ms
Response Format Validation.Success Retry After Present When Retry Events ✅ 2887ms
Response Format Validation.Success No Retry After When Drop Only ✅ 2807ms
Response Format Validation.Response Echoes Request Id ✅ 2705ms
Retry Behavior.Retries On 408 ✅ 9378ms
Retry Behavior.Retries On 500 ✅ 9226ms
Retry Behavior.Retries On 503 ✅ 11722ms
Retry Behavior.Retries On 504 ✅ 9154ms
Retry Behavior.Retryable Errors Have Retry After ✅ 6350ms
Retry Behavior.Respects Retry After On Retryable Error ✅ 10845ms
Retry Behavior.Does Not Retry On 400 ✅ 2363ms
Retry Behavior.Does Not Retry On 401 ✅ 5133ms
Retry Behavior.Does Not Retry On 402 ✅ 4843ms
Retry Behavior.Does Not Retry On 413 ✅ 4837ms
Retry Behavior.Does Not Retry On 415 ✅ 4846ms
Retry Behavior.Non Retryable Errors Have No Retry After ✅ 4741ms
Retry Behavior.Implements Backoff ✅ 21864ms
Retry Behavior.Max Retries Respected ❌ 21675ms
Partial Batch Handling.Handles 200 Full Success ✅ 4840ms
Partial Batch Handling.Handles 200 With All Ok ✅ 5913ms
Partial Batch Handling.Does Not Retry Dropped Events ✅ 5790ms
Partial Batch Handling.Does Not Retry Limited Events ✅ 5951ms
Partial Batch Handling.Prunes Ok Events On Partial Retry ✅ 7898ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry ✅ 7932ms
Partial Batch Handling.Retries Only Retry Events From Partial ✅ 8008ms
Partial Batch Handling.Partial Retry Preserves Uuids ✅ 7998ms
Partial Batch Handling.Partial Retry Attempt Header Increments ✅ 7747ms
Partial Batch Handling.Partial Retry Request Id Preserved ✅ 7920ms
Partial Batch Handling.Respects Retry After On Partial ✅ 8038ms
Partial Batch Handling.Unknown Result Treated As Terminal ✅ 6100ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry ✅ 6030ms
Compression.Sends Gzip Content Encoding ✅ 2987ms
Compression.No Content Encoding When Disabled ✅ 2865ms
Compression.Compressed Body Is Decompressible ✅ 2827ms
Error Handling.Does Not Retry On Unknown 4Xx ✅ 4847ms
Event Options.Cookieless Mode Override ✅ 2811ms
Event Options.Disable Skew Correction Override ✅ 2965ms
Event Options.Process Person Profile Override ✅ 2887ms
Event Options.Product Tour Id Override ✅ 359ms
Event Options.Unset Options Omitted ❌ 445ms
Event Options.Options Override In Batch ✅ 2928ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties ✅ 2800ms
Geoip And Historical Migration.Historical Migration Set In Body ❌ 2700ms
Geoip And Historical Migration.Historical Migration Absent By Default ✅ 2767ms

Failures

retry_behavior.max_retries_respected

Expected 4 requests, got 6

event_options.unset_options_omitted

Event option 'process_person_profile' should be absent, but options={'process_person_profile': False}

geoip_and_historical_migration.historical_migration_set_in_body

Body missing 'historical_migration' field. Body keys: ['created_at', 'batch']

Feature_Flags Tests

✅ 16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 2911ms
Request Payload.Flags Request Uses V2 Query Param ✅ 2884ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 2937ms
Request Payload.Flags Request Omits Authorization Header ✅ 3079ms
Request Payload.Token In Flags Body Matches Init ✅ 2877ms
Request Payload.Groups Round Trip ✅ 2930ms
Request Payload.Groups Default To Empty Object ✅ 2993ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It ✅ 2770ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 2823ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 2813ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 2943ms
Request Lifecycle.No Flags Request On Init Alone ✅ 131ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 3022ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 5790ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 3404ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 3169ms

turnipdabeets and others added 2 commits October 10, 2026 13:13
captureAi(...) takes capture's arguments, runs the same enrichment and
beforeSend, and queues the event on its own disk queue for
/i/v1/ai/events. It reuses the capture V1 transport (headers, per-event
results, retry identity per endpoint, redirects, Retry-After), with no
/batch fallback. The AI queue batches by bytes (5 MiB) as well as count,
keeps at most 100 events, and drops events over 8 MiB + 64 KiB logging
only the name and size. capture() never reroutes $ai_ events.

The compliance adapter gains /capture_ai and the capture_ai_v1 capability.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The AI queue now drops its oldest events once they take more than 50 MiB
on disk, on top of the 100-event limit. captureAi returns the queued
event's UUID, or nil when the event isn't queued.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant