fix(core): honor Retry-After on the OTLP export queues - #4726
fix(core): honor Retry-After on the OTLP export queues#4726turnipdabeets wants to merge 43 commits into
Conversation
|
Size Change: +42 kB (+0.2%) Total Size: 20.8 MB 📦 View Changed
ℹ️ View Unchanged
|
68d9c61 to
197acec
Compare
…eues Parses `Retry-After` once in the shared OTLP sender and applies it as a floor on each export queue's own backoff, and refuses a batch over the endpoint's 2 MB body limit without spending a request on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qp8JHHcGSf7mocdZtHDR29
f7ee901 to
3db04a2
Compare
…tlp-honor-retry-after
…imer The window was only consulted where a timer was armed, so an explicit flush() — the lifecycle and per-request path — sent inside it. Traces spent its whole per-batch retry budget there and dropped the spans before the wait elapsed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vkNv2jdvWeE8hoAuiwgCr
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vkNv2jdvWeE8hoAuiwgCr
Gating traces flush() on the window stranded spans: it is what the serverless waitUntil keep-alive awaits, and the recovery timer is unref'd, so a frozen isolate never sent them. Protect the head batch's retry budget instead of suppressing the send. The wall-clock deadline is now clamped, so a backward clock step can't stretch a wait past the cap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vkNv2jdvWeE8hoAuiwgCr
The deadline was re-installed by the very refusal it excused, so a host flushing faster than the window kept it open forever: the traces retry budget never advanced, the head batch never retired, and everything behind it was dropped at maxQueueSize. Measured 0 releases over 60 flushes; now releases at the intended 8 x window. A backward clock step no longer holds a window open either, and MAX_RETRY_AFTER_MS is out of the public barrel. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vkNv2jdvWeE8hoAuiwgCr
Port this PR's suites to vitest (#4739).
…three copies Share one RetryAfterWindow across the logs, metrics and traces queues, move parseRetryAfterMs off the package entry point, and stop a header-less refusal, a size verdict or a clock step from discarding an open window. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC
…s queue Metrics never retired its pending timer when a flush started, so a wait armed from a Retry-After outlived the window a later successful flush had already closed, holding every subsequent capture for up to five minutes. Logs had the mirror gap: a capture landing mid-flush armed a plain-interval timer before the window existed, and only the background wrapper re-armed it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC
…loses The re-arm helpers only ever move a timer later, so a record or series captured while a send was in flight stayed pinned to the window that send then closed — up to five minutes for a wait already over. Re-arm outright instead, and let onReconnect arm a timer during a window rather than returning with nothing scheduled, which stranded logs after an explicit flush was refused. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC
Prompt To Fix All With AI### Issue 1
.changeset/otlp-honor-retry-after.md:2-4
**Browser release metadata omitted**
These changesets omit `posthog-js`, even though the shared timer changes affect the browser package and add browser mangling metadata. This leaves the changed browser behavior without the required patch-version entry and changelog metadata; please add `posthog-js` to the applicable changeset metadata, including the sibling changeset where appropriate.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (1): Last reviewed commit: "fix(core): release a queue once the Retr..." | Re-trigger Greptile |
An explicit flush() skips the wait but was charging the refusal, so a host draining per request retired a batch in eight calls and no elapsed time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L8eFZB35NExUZyq5hGgh43
The shared timer change reaches the browser even though the window never opens there, so it needs its own changeset line. Comments that narrated the bugs behind each fix are cut back to the constraint a reader needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017SWWFUdrPWaHTpwFNr4DgC
|
Worth checking this comment as well #4726 (comment). Do we need it or should we just stay reactive to 413 path. I think it's also lower than the accepted max |
…fallback The ceiling was the 2 MB `MAX_REQUEST_BODY_SIZE_BYTES` falls back to, but the ingestion service runs with 10 MiB, so bodies between the two were refused without a request and their records dropped — a whole window, in metrics, which has no shrink path. Raise it to the largest limit any known deployment configures; the 413 path still covers the ones configured lower. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YE1TyDWGyLwUXEX3ffy83K
The parser rejects a zero delta as well as a negative one, and the window's justification named a retry-budget effect that no longer applies to traces — the rule holds for the logs and metrics gates instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YE1TyDWGyLwUXEX3ffy83K
…licated aside The test builds a circular payload, not one past the max string length, and the barrel note was the same paragraph in two files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XBUa53f3QqECUyxnkSiWGA
Traces ships new in the same release through #4579, so naming it here reads as a fix to behavior that never existed. Also drops the split-and-isolate claim from the oversized entry: metrics drops the window rather than halving. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XBUa53f3QqECUyxnkSiWGA
capture-logs caps the raw request body (main.rs DefaultBodyLimit) and the gzip output (service.rs decompress_gzip_capped) at the same value, so the comment was wrong to say the wire bytes are not capped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018AAPCyRnC5HbukEZV9hbai
|
Follow-up: background event flushes bypass the traces retry window. Node's background event flush calls public OTLP recommends honoring the header; OTel's Sources: Node flush path · OTLP throttling · OTel ForceFlush contract. Agent-assisted review. |
|
Protocol deviation: the SDK retries more statuses than OTLP allows. OTLP permits retries for 429, 502, 503, and 504, and forbids retrying other 4xx/5xx responses. The SDK additionally retries 408 and the remaining 5xx statuses. This predates the PR and supports existing tests and the backend's transient 500 responses. Document it as a compatibility deviation, and coordinate server-side status changes before narrowing client retries. Non-blocking here does not mean OTLP-conformant. Sources: SDK retry statuses · OTLP retry statuses · ingestion failure response. Agent-assisted review. |
Request decompression runs outside the body limit, so a gzip request is only measured once, after it is decoded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjT26chjUzuPSyJZqh4AWQ
The pre-commit JSON formatter added a trailing newline the api-extractor pipeline does not emit, so `Check public API references` saw a diff.
The events flush timer no longer drains spans while the traces queue is honouring a Retry-After window. A batch the SDK measured as too large itself now splits that drain only, leaving the batch size the next drain starts from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjT26chjUzuPSyJZqh4AWQ
|
@jonmcwest — on background event flushes bypassing the retry window: done in |
A refusal naming a longer wait now pushes the deadline out, bounded at five minutes from where the window was installed. Logs, metrics and traces jitter their own backoff, drawn once per failure, and metrics backs off exponentially rather than retrying on a fixed interval. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjT26chjUzuPSyJZqh4AWQ
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjT26chjUzuPSyJZqh4AWQ
…state Both capabilities document the retry delay as exponential backoff capped at ~30s. Only traces applied it, so a 5s interval reached 320s after six doublings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjT26chjUzuPSyJZqh4AWQ
…726-merge # Conflicts: # packages/core/src/traces/config.ts
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjT26chjUzuPSyJZqh4AWQ
|
@jonmcwest — on retrying more statuses than OTLP allows. Your reading of the code is right, but I do not think this is an undocumented deviation, so I have left it alone.
The other reason to leave it: that predicate is the shared transport for analytics events, not just OTLP. This PR does not otherwise touch event delivery, and dropping So I read this as a question for the ingestion team and the spec rather than something to fix here: if we want OTLP's narrower set, the spec changes first and the client follows. Happy to open that if you think it is worth doing — but flagging that it trades conformance for delivery on a path that is working today. |
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjT26chjUzuPSyJZqh4AWQ
The cap landed in 3279b97 with no changelog entry, and it changes retry cadence for posthog-js and posthog-react-native. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RWoFx4BctNmXnpnKS79Qgz
The oxfmt-data pre-commit hook appends a trailing newline to any staged .json, which the api-extractor output does not carry, so committing these through the hook fails the Check public API references job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RWoFx4BctNmXnpnKS79Qgz
Problem
Two ways the OTLP export queues spend requests the ingestion endpoint has already told them not to send.
1.
Retry-Afterwas ignored. The endpoint can tell an SDK how long to wait before retrying. The logs, metrics and traces export queues each used only their own exponential backoff, so a project told to back off for two minutes kept sending.This is conformance and defence-in-depth, not an urgent fix. PostHog's own capture path emits only 200/400/401/413/500 and never
429. The work that would have changed that, posthog/posthog#75090, was closed as stale in August without merging, and nothing has replaced it. Two reasons to do it anyway:sdk-specslogs and traces contracts require it unconditionally — "exponential backoff capped at ~30s, honoringRetry-Afterwhen present" — with no carve-out for the service not sending one yet. All three queues failed that requirement.429with aRetry-Aftertoday.2. Oversized batches were uploaded only to be refused. The endpoint caps the request body at
MAX_REQUEST_BODY_SIZE_BYTES, applied after it decompresses — 10 MiB on the deployed ingestion service (chartsshared/capture-logs/common.yaml). A batch over that could only ever come back 413, but the SDK uploaded it anyway — and the halving loop then re-uploaded progressively smaller versions, spending a full request on each attempt.Note that the traces spec still describes #75090 as "in flight" in its
Server-side contractrequirement; that reference needs updating insdk-specsindependently of this PR.Stacked on #4579 because the traces queue only exists there. The logs and metrics halves are independent of it and would apply to
mainunchanged.Changes
Retry-AfterParsed once, in the shared OTLP sender, and surfaced to whichever queue is retrying.
parseRetryAfterMsaccepts both wire forms — delta-seconds and HTTP-date — and returnsundefinedfor anything it cannot parse, a date already in the past, or a non-positive delta, so the caller falls back to its own backoff."10 minutes"is rejected rather than read as 10 seconds, and so is numeric-looking junk (-5,+5,5.5), whichDate.parsewould otherwise read as dates in 2001.PostHogFetchHttpErrorexposes it asretryAfterMs, guarded:headers.getis injected transport code, and a throwing one must not turn a retriable failure into an unhandled rejection.retry-lateroutcome of all four send wrappers gains an optionalretryAfterMs. Additive, so nothing implementing or consuming these types breaks.Three judgment calls worth reviewing:
max(ownBackoff, retryAfterMs). Taking it literally would let aRetry-After: 1turn a queue that had backed off to 30s into a hot loop; HTTP semantics are "not before this", whichmaxsatisfies in both directions.fetchWithRetryloop stops when the header is present. That loop retries on a fixed short delay (fetchRetryDelay, default 3s) up tofetchRetryCount(default 3) times. Left alone it fired four requests inside the very window the queue was about to honour, soretryChecknow hands aRetry-Afterresponse straight to the queue-level backoff. A retriable response with no header still retries there as before.Logs stores an absolute deadline rather than a duration, because the timer is not its only send trigger: the
maxBufferSizesize trigger andonReconnect()both start a flush directly, and both now check the window.onReconnect()no longer clears it — anonlineevent means the network came back, not that the endpoint's rate limit expired, and browsers fire it on every network handover.Metrics has no exponential backoff of its own, so there the header raises the next timer arm above
flushIntervalMs.Every send path, not just the timer
The first revision gated only the paths that arm a timer. A later review pass found three places where a send still went out inside the window. Each was reproduced with an executed failing test before it was touched.
posthog-nodespent a span batch's whole retry budget inside the window.flushBackground()fires everyflushInterval(10s default) while events are flowing, and node'sflush()override drains spans alongside them.MAX_RETRIES_PER_BATCHis 8, so at that cadence the budget burned out in ~80s and the spans were dropped ~220s before the endpoint would have accepted them —Dropping 1 span(s): the ingestion endpoint failed 8 times in a row. A refusal that lands inside a window the endpoint already asked us to wait out is not evidence against the batch, so it no longer counts against that budget. The send itself is deliberately not suppressed: see "Why explicit flush still sends" below.flush().flush()leaves no timer behind, so the next capture is the one that arms it — atflushIntervalMs, inside the window. React Native takes this path on every foreground and background transition._armFlushTimernow floors the delay by the remaining window.flush(). The clear lived on the background wrapper's.then, whichflush(),flushWithTimeout()andshutdown()do not go through, soonReconnectand the size trigger stayed blocked for the rest of a window the endpoint had already stopped enforcing. The window is now recorded and cleared from the outcome, the way metrics and traces do it.Metrics and traces also held the header as a duration rather than a deadline, so a timer armed while the wait was already part-served restarted the whole window instead of counting down the remainder. Both now hold an absolute deadline, matching logs.
parseRetryAfterMsadditionally reads only the first value when two hops each append one —headers.getjoins repeated headers as"60, 120"— while leaving an HTTP-date, which carries a comma of its own, intact.A refusal that arrives inside an open window may push the deadline out, but never past
MAX_RETRY_AFTER_MSfrom where the window was first installed. Sliding it forward without that ceiling meant a host flushing faster than the window kept it open forever, so the retry budget never advanced and the head batch never retired — measured 0 releases across 60 flushes, with everything behind it dropped atmaxQueueSize. It now releases at the intended bound: in a host awaitingflush()ten times a second against a 300s window, the head batch releases at 2,100,000 ms (8 x the capped window), the same figure the earlier gated revision produced.Each queue also clamps the remaining wait to
MAX_RETRY_AFTER_MSwhen it reads the deadline back, and treats a clock that has moved behind the install point as ending the window rather than extending it by the size of the step. The deadline is wall clock, andparseRetryAfterMsapplies the five-minute cap once, to the duration; without the clamp a backward step — NTP, a resumed VM, a user changing the device date — stretched a 60s wait to over an hour. Verified: a one-hour backward step returned 3660s from_retryAfterRemainingMs().Why explicit flush still sends
An earlier revision of this PR gated
PostHogTraces.flush()on the window. That was wrong, and the review that followed caught it:posthog.flush()is what the serverlesswaitUntilkeep-alive awaits. Gated, it resolved immediately with spans still queued, and the only recovery was a timer thatsafeSetTimeoutunrefs — so a frozen isolate never ran it. Verified: after the window opened, two furtherflush()calls sent nothing and the queue stayed non-empty, even though the origin had recovered 5s in.Flush triggersrequirement says the globalflush()SHALL drain the span queue, which the gate violated.So the wait is honoured where it belongs — the periodic timer — and the damage it was meant to prevent is addressed at the source, by not charging the retry budget. An explicit flush inside a window costs one request; it no longer costs the spans.
The budget now counts backoff windows rather than attempts, which is the general form of that exemption. A refusal is charged only once per
_nextFlushDelay()— the wait the timer would have taken — whoever drove the attempt. Gating onRetry-Afteralone left the same hole open whenever the endpoint named no wait, which is every refusal PostHog's capture path produces today: eightflush()calls retired a head batch in zero elapsed time, so a blip the timer would have ridden out cost the spans instead of a request. The traces spec's Error handling and retries requirement is explicit that an SDK exempting a caller-driven flush from the wait "SHALL NOT charge the resulting refusal against the batch's retry budget".The window is read from
clockNow()— monotonic where the platform has one — so an NTP correction or a resumed VM cannot hold the budget open, and it resets with the head batch rather than with the queue. Behaviour on the timer path is unchanged: eight refusals spread across the backoff, as before. This is traces-only; logs has no per-batch budget and metrics has no retry state at all, so neither can spend one early.To be precise about what is and is not honoured, the exemption is not limited to teardown. Any caller-driven flush sends inside the window, including
posthog-node's own automaticwaitUntildebounce (_onSpanQueued→scheduleDebouncedFlush, 50 ms) and React Native's AppState listener, which takes the plainflush()onactiveandinactive— onlybackgroundusesflushWithTimeout. Measured on the nodewaitUntilpath: 200 span-ends over 20 s produce 200 exports, all inside a 300 s window. That is down from 266 on the base branch, and the spans are no longer dropped, but it is not zero. Suppressing those sends is exactly what the earlier revision did, and it stranded spans in a frozen isolate.Request body limit
A payload over the limit is reported as
too-largewithout a request, so the caller's existing halving loop isolates and drops the one oversized record without spending a request on every attempt.This is @jonmcwest's ask from the #4579 review, where he measured a span carrying one multi-MB attribute being uploaded up to 11 times before the halving loop isolates it, 35–47 POSTs to drain a full 512-span queue around it, and ~480 healthy batches for the sticky shrink to ramp back up.
Worth flagging to him directly: those numbers were simulated against the 2 MB cap named in the
traces-defaultscomment he cited, and that comment was wrong — the deployed limit is 10 MiB. Against the real limit a 3 MB span is simply accepted, so the trigger is a larger record than his repro used. The saving itself is unchanged in size (see the table below); only the threshold moves.RequestDecompressionLayeris layered outsideDefaultBodyLimit::max(max_request_body_size_bytes), so a payload that gzips small is still refused on its decompressed size.byteLengthOfhelper extracted from theBuffer/TextEncoderblock already infetchWithRetry— so a CJK or emoji-heavy payload is measured the same way the server measures it.The constant is set to 10 MiB, the largest limit any known deployment configures — what the ingestion service runs with — rather than the 2 MB it falls back to when nothing configures it. @dustinbyrne and @ioannisj both caught this: an earlier revision used the fallback, which is 5× below what capture actually accepts.
That direction matters, because the two errors are not symmetric. A ceiling set too high costs one wasted request, and the 413 path catches it. A ceiling set too low costs data, with no 413 to show for it. Measured against a 10 MiB endpoint on the earlier 2 MB constant:
Metrics is the worst of the three: it has no shrink path, so a refusal discards the entire window with one
warn.The saving the check exists for is unchanged at the higher number — measured with one 12 MiB record among healthy ones:
The constant is still not authoritative — a proxy in front can lower the limit, and a self-hosted deployment runs the 2 MB fallback — so the 413 path stays the primary mechanism and handles every deployment configured below this ceiling. It is deliberately not configurable: it is internal, so it can be made configurable later without a breaking change, whereas shipping the option now would be permanent.
This departs from the traces spec as written. Batch assembly and concurrency names "the reactive 413 path (not proactive byte measurement) as the overflow mechanism" — wording that predates the review request above. PostHog/sdk-specs#58 amends it to permit a pre-send measurement that matches how the endpoint measures (uncompressed, in bytes, feeding the same shrink-and-drop path a 413 does) while keeping the 413 path mandatory.
Reviewer notes
3279b97c, because that is what the logs contract states (openspec/specs/logs/spec.md: "exponential backoff capped at ~30s, floored byRetry-Afterwhen present"). It is interval-relative, so the size of the change depends on the host: 192s to 30s on web (3s interval), 640s to 30s on React Native (10s). Measured against a permanently-refusing endpoint, that is 24 to 123 requests/hour on web and 11 to 121 on React Native, with no change to what is delivered. Now carries its own changeset.Retry-Afteris not a CORS-safelisted response header, so a cross-origin ingestion response has to sendAccess-Control-Expose-Headers: Retry-Afterbefore any browser can read it. Verified in Chromium against a two-origin harness: without that header bothfetch'sresponse.headers.get('retry-after')and XHR'sgetResponseHeader('retry-after')returnnullon a429whose status is plainly visible; adding it makes both return120. So threading headers throughRequestResponsewould be inert until the ingestion service exposes the header, which is why that work stays out of this PR rather than being a small omission from it. For comparison, the browser's own event retry queue caps at 30 minutes (packages/browser/src/retry-queue.ts), so logs is now the more aggressive of the two during an outage.429is not PostHog. With quota enforcement at capture abandoned, the header realistically arrives from a proxy, CDN or load balancer in front of capture — which is also why the cap exists.logsreads as replacement ("honoringRetry-Afterwhen present and otherwise exponential backoff capped at ~30s");tracesreads as additive. Five of the six PostHog SDKs floor (rs, python, go, ios, this one); posthog-android replaces. Floor is the only rule that satisfies both spec sentences at once.413-vs-proactive-measurement amendment below.Retry-Afterlengthens how long traces holds a failing head batch. The per-batch budget is 8 backoff windows, andRetry-Afterfloors the window, so a 5-minute header turns a ~2.5-minute hold into a ~35-minute one. Measured over 40 minutes of steady traffic against a permanently-refusing endpoint, span loss is identical either way (2200 of 2400 in both cases, sincemaxQueueSizebounds the queue regardless); the change is 127 send attempts down to 10. Deliberately left as is — the alternative, dropping on a wall-clock budget, discards spans the endpoint asked us to hold.@posthog/typeschange, and no new public surface. The outcome types live in core.parseRetryAfterMsand theRetryAfterWindowthat holds the deadline both live inutils/retry-after.ts, which is deliberately outside theutilsbarrel — that barrel is re-exported wholesale from the package entry point. Verified unreachable from@posthog/core: neither symbol appears in the generatedpackages/{node,react-native}/references/*-latest.json. The only public delta is the optionalretryAfterMs?: numberon the three already-exportedSend*BatchOutcomeunions.Verification
packages/core/src/__tests__/posthog.otlp-retry-after.spec.tsdrives it end to end from a mocked 429 through to each of the three queues' outcomes, because a unit test of the parser alone would still pass with the plumbing missing. It also covers a response with no header, a transport whoseheaders.getthrows, and the request count at the shippedfetchRetryCountdefault.Per-queue tests cover the behaviour that matters rather than the field assignment: a
Retry-Afterlonger than the 30s exponential cap actually delays the retry; a shorter one does not shorten it; a capture or span end landing mid-flush cannot pull the retry back inside the window; a steady stream of captures does not push the flush timer out indefinitely; the logs size trigger andonReconnectdo not send inside the window; and a non-retriable outcome ends the wait.Every behavioural test was mutation-checked: reverting
logs/index.ts,metrics/index.ts,traces/index.tsorposthog-core-stateless.tsfails exactly those tests and nothing else.packages/corepackages/nodepackages/react-nativepackages/browserEvery fix above was mutation-checked: reverting it to the shipped behaviour fails exactly the test written for it and nothing else. That includes the body-limit boundary, which pins both directions — a body of exactly the limit is sent, one byte more is not — because
>drifting to>=would have refused an acceptable batch without a request and, in traces, halved it down and dropped the span with no 413 to show for it. A separate case pins the range between the service's fallback and its configured limit: reverting the constant to 2 MB fails exactly those three tests, one per signal.Also exercised on an Android emulator, with the React Native example built from this branch against a mock ingestion endpoint. Answering
429+Retry-After: 30, the log exports retried at +30.08s and +30.05s rather than on the SDK's 10s flush interval, and resumed on the next attempt once the mock returned 200.Lint and format clean.
Release info Sub-libraries affected
Libraries affected
@posthog/coreis bumped (patch); it has no checkbox above.posthog-js(web) is deliberately unticked. The browser SDK does not use core's_sendOtlpBatch:posthog-logs.tsandposthog-metrics.tsbuild their outcomes from_send_request's callback, andRequestResponse(statusCode/text/json/error) carries no headers. So it never populatesretryAfterMs, both new gates are inert there, and the pre-send size check never runs. One change does reach the browser, though: the browser SDK does use core'sPostHogLogs/PostHogMetrics(it only supplies its own_sendLogsBatch/_sendMetricsBatch), and the post-flush re-arm changed from "leave a pending timer alone" to "keep whichever deadline is later". Under consecutive failures a capture-armed short timer no longer wins, so the retry follows the backoff. That is the intended "backoff is a floor" rule, and it is why the web bundle deltas are non-zero rather than only the mangled-names cache. Wiring it up means threading headers (or a parsedretryAfterMs) throughRequestResponseand both the XHR and fetch paths, which is a separate change and deliberately not in this PR.Checklist
retryAfterMsis optional on types that were already exported, so implementors and consumers are unaffected. Behaviour only changes when the endpoint sends a header it does not send today, or when a batch exceeds a limit that would have refused it anyway.If releasing new changes
pnpm changesetto generate a changeset file🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Built with Claude Code, directed by @turnipdabeets. Came out of reviewing #4579 against the
sdk-specstraces contract, which surfacedRetry-Afteras an unmet requirement shared by all three OTLP pipelines rather than anything #4579 introduced.Decisions worth flagging, beyond those above:
_sendOtlpBatchmeans one implementation of the wire format and one place to fix it.PostHogFetchHttpError, which the tagged-outcome design exists to avoid._armFlushTimerkeep whichever deadline was later. Since every capture arms the timer, that pushed the deadline out on each one and the flush never fired under a steady stream. The enqueue path now leaves a pending timer alone; only the flush-settle path extends it.MAX_REQUEST_BODY_SIZE_BYTESabove 10 MiB) is narrow. The constant is internal, so this stays reversible.The branch was reviewed by a second Claude Code agent against
sdk-specsbefore this description was written; each of its findings was independently reproduced with an executed failing test before being acted on.https://claude.ai/code/session_01Qp8JHHcGSf7mocdZtHDR29