Skip to content

docs(traces,logs): clarify Retry-After semantics and permit proactive body measurement - #58

Merged
turnipdabeets merged 5 commits into
mainfrom
spec/clarify-otlp-retry-after-and-body-limit
Sep 4, 2026
Merged

docs(traces,logs): clarify Retry-After semantics and permit proactive body measurement#58
turnipdabeets merged 5 commits into
mainfrom
spec/clarify-otlp-retry-after-and-body-limit

Conversation

@turnipdabeets

Copy link
Copy Markdown
Collaborator

Why

Implementing Retry-After on the three OTLP export queues in posthog-js (posthog-js#4726) surfaced three places where the canonical contract either says the opposite of what the SDKs do, or says too little to keep them from diverging.

1. The two specs contradict each other on Retry-After. logs reads as replacement ("honoring Retry-After when present and otherwise exponential backoff capped at ~30s"); traces reads as additive ("exponential backoff capped at ~30s, honoring Retry-After when present"). An SDK cannot satisfy both. The implementations split accordingly:

SDK Rule Cap on the header
posthog-rs floor own ceiling (default 30s)
posthog-python floor own ceiling (30s)
posthog-go floor none
posthog-ios floor none
posthog-android replaces the backoff none
posthog-js (#4726) floor one 5 min constant

Replacement is the weaker rule: Retry-After: 1 on a queue already backed off to 30s turns it into a hot loop, which is the opposite of what the header is for.

2. Three of the six place no bound on the header, so a misconfigured proxy — the realistic source of a 429, since capture does not emit one — can strand a queue for hours.

3. traces forbids the overflow mechanism a reviewer has since asked for. Batch assembly and concurrency names "the reactive 413 path (not proactive byte measurement)". Reviewing the traces MVP (posthog-js#4579), @jonmcwest measured a span with one multi-MB attribute being uploaded up to 11 times before the halving loop isolates it — 35–47 POSTs to drain the queue around it — and asked for a client-side size check before the POST. The spec sentence predates that request.

What changes

  • Retry-After becomes a floor explicitly, in both capabilities, in the same words.
  • A clamp is required; the value stays per-SDK. Recommended default is the rule posthog-rs and posthog-python already share — clamp to the SDK's own backoff ceiling — kept as a SHOULD because an SDK whose signals have different ceilings cannot express it as one constant.
  • Both wire forms (delta-seconds and HTTP-date) are required, falling back to the SDK's own backoff — never to zero — on an unparseable value.
  • Proactive byte measurement becomes permitted in traces, as a complement to the 413 path, not a replacement: a proxy can lower the limit and a self-hosted deployment can raise it, so the SDK's constant is never authoritative.
  • The stale posthog/posthog#75090 "in flight" reference is corrected (closed as stale in August 2026, never merged).

Notes

  • No service change required; retry-queue and http-client already treat Retry-After as transport metadata without pinning semantics, so neither needs a delta.
  • Follow-ups are listed in tasks.md §4 — posthog-android is the one SDK that contradicts the clarified rule outright.
  • openspec validate clarify-otlp-retry-after-and-body-limit --strict passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC

… body measurement

Pin Retry-After as a floor with a required clamp in both capabilities, which
currently word it as replacement and as additive respectively, and allow the
pre-send size check @jonmcwest asked for on PostHog/posthog-js#4579.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC
turnipdabeets and others added 3 commits September 3, 2026 14:16
…larification

Sync the delta into the canonical specs, and correct the three stale
PostHog/posthog#75090 references in the traces server-side contract: it was
closed unmerged on 2026-08-17, so a 429 with Retry-After now only ever comes
from infrastructure in front of capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC
Drop the two entries that were not work (posthog-rs/python need no change,
posthog-js is already decided), and say plainly that merging leaves android,
ios and go non-conformant without changing their behavior.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC
metrics stays covered by the logs policy by reference, with the trigger for
revisiting named. Leaves the follow-ups as the two real cross-SDK migrations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC

@dustinbyrne dustinbyrne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved to unblock. This change is specification-only and does not itself alter shipped SDK or service behavior. I left the factual and consistency findings inline.

Comment thread openspec/specs/traces/spec.md Outdated
Comment thread openspec/specs/traces/spec.md Outdated
Review findings on #58.

The floor and the clamp had no stated order, so the scenarios contradicted
each other: `Retry-After: 120` required 120s under a maximum a conforming SDK
may set to 30s, and the HTTP-date example used a 2015 date the same paragraph
says to ignore. The wait is now `max(ownBackoff, min(parsedRetryAfter,
documentedMaximum))` — clamp the header, then floor — with scenarios that name
the maximum they assume, plus separate cases for a past HTTP-date and for
clamp-then-floor.

logs and traces stated different halves of the policy. Both now carry the
whole thing in identical words: past/non-positive handling, the cap guidance,
the caller-flush retry-budget exemption, and the reconnect rule (traces flushes
on reconnect too, trigger 6).

Inventory corrections, each checked against origin/main:

- posthog-go clamps to a fixed 30s `defaultMaxBackoff` in `retryDelayV1`
  (#255, merged 2026-07-04) — but only on the capture-v1 path. `sendBatch` on
  the legacy `/batch/` endpoint, which `CaptureModeLegacy` still makes the
  default, floors with no clamp. Split into two rows; the follow-up is now
  "extend the clamp", not "add one".
- posthog-rs' ceiling is configurable; posthog-python's and posthog-go's are
  hard-coded 30s. The shared rule was overstated as a shared number.
- posthog-android also needs HTTP-date parsing: `toIntOrNull()` takes
  delta-seconds only.
- "hot loop" overstated it — Android defaults to `maxRetries = 3` with 1/2/4s
  delays, so its 30s ceiling needs a larger configured budget. Now "aggressive
  one-second retry cadence", with the defaults stated.
- posthog-python ships a metrics queue too (`client.metrics`, alpha, #739) that
  retries 429/5xx without reading `Retry-After`. Two implementations already
  disagree, so a canonical metrics policy is now a follow-up rather than a
  deferral.
- capture does reject malformed token shapes: posthog#76501, split from the
  abandoned #75090, merged 2026-08-04 and the capture-logs authorizer runs
  `validate_token` for traces, logs and metrics alike. The gap that remains is
  the well-formed-but-unknown `phc_` key, which still gets a 200.

openspec validate --all --strict passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QKKGfBiNia4bwNFM3J43xU
@turnipdabeets
turnipdabeets merged commit 35a1ced into main Sep 4, 2026
10 checks passed
@turnipdabeets
turnipdabeets deleted the spec/clarify-otlp-retry-after-and-body-limit branch September 4, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants