Skip to content

docs(event-batcher): bound analytics requests by serialized size - #133

Closed
posthog[bot] wants to merge 1 commit into
docs/acceptance-step-cataloguefrom
posthog/bound-analytics-batches-by-size
Closed

posthog[bot] wants to merge 1 commit into
docs/acceptance-step-cataloguefrom
posthog/bound-analytics-batches-by-size

Conversation

@posthog

@posthog posthog Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Why

The event-batcher spec bounds a batch only by event count, flush interval, and explicit flush. The one place size appears is reactive — "Some SDKs reduce effective batch size after oversized-payload errors (413)". Ingestion refuses a request body over the deployment's limit (10 MiB on PostHog's hosted ingestion, as the logs spec already records) and refuses the whole request, so an SDK that batches by count alone can build a request it cannot send and lose every event in it.

posthog-rs#288 (merged 2026-10-18) split background analytics sends at 5 MiB for exactly this reason, matching what posthog-js capture v1 already does — and neither rule is in the contract.

Changes

Adds one event-batcher requirement, Analytics requests are bounded by serialized size:

  • An SDK-managed analytics send is split so each request stays within a byte target below the deployment's body limit, in addition to the event-count limit; the target applies to the bytes actually sent, so a compressing SDK may measure the compressed body.
  • Splitting is not dropping: every event is sent, in order, across the resulting requests.
  • The analytics lane has no per-event cap — an event larger than the target is sent alone rather than dropped locally.
  • Work deferred to send time (before-send, serialization, property stamping) stays at send time.
  • An explicit single-request API may be exempt, and the reactive 413 shrink stays as the backstop.

Proposed, applied and archived on this branch as 2026-10-19-bound-analytics-batches-by-size.

Verified SDK behavior

SDK Byte bound on analytics requests Where
posthog-js (browser-next capture v1) 5 MiB target per request alongside the event-count limit (partitionEvents) packages/browser-next/src/capture-v1.ts
posthog-rs background sends split at 5 MiB of serialized events; larger event sent alone, no per-event cap posthog-rs#288
posthog-python, posthog-node, posthog-go, posthog-ios, posthog-android count-based batching only, 413 handling after the fact —

The capture-ai spec already names an "~5 MiB batch target", but explicitly as a non-normative transport note for the AI lane.

Uncertain — worth a human call

  • No single mandated constant. The limit is deployment-dependent (2 MiB default, 10 MiB hosted) and compression changes the bytes sent, so the requirement states "a target below the limit" and records ~5 MiB as what the SDKs that measure uncompressed bodies picked. If a fixed number is wanted, it should be stated here.
  • This is a required implementation change for the count-only SDKs. Two SDKs implement it today; the requirement is written as SHALL, consistent with the repo stating the winner where SDKs diverge. Confirm that is the intent rather than a SHOULD.
  • Interacts with the open compression change (#81), which governs whether a body is gzipped at all; if that lands, the "bytes actually sent" wording may deserve a cross-reference.
  • No acceptance feature scenarios were added; the scenarios are spec-level only.

How did you test this code?

openspec validate --specs --strict reports the same 33 passed / 31 failed totals as the base branch (all pre-existing "requirement text is very long" warnings; the new requirement raises the same warning as its neighbours).


Created with PostHog Desktop

🤖 Generated with Claude Code

Adds one event-batcher requirement: an SDK-managed analytics send is split so
each request stays within a byte target below the ingestion body limit, in
addition to the event-count limit. Splitting preserves order and drops nothing,
and an event larger than the target is sent alone rather than dropped locally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: f33060ac-cc4a-4ef5-9c1d-105b692663ce
@turnipdabeets

Copy link
Copy Markdown
Contributor

Closing this one. The SDK table doesn't match what ships: posthog-python (5 MiB, consumer.py:33), posthog-go (500 KB, message.go:113), posthog-ruby (500 KB) and posthog-php (1 MB) already bound batches by size, and all four drop a single oversized event locally, which the new "no per-event cap" rule would forbid. The event-batcher spec already records both behaviors. If we want byte-bounded batches as a SHALL, we'll write it fresh from shipped SDK behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant