Extract the three duplicated OTLP senders; logs drops batches on 5xx where metrics and traces retry

[turnipdabeets]

PostHogCoreStateless has three near-identical OTLP senders — _sendLogsBatch, _sendMetricsBatch, and _sendTracesBatch. Each is ~50 lines and they differ only in the endpoint path and auth style. Nothing keeps them in sync, and they have already drifted in a way that loses data.

The drift

All three classify a send failure into the same tagged outcome, but not by the same rule:

sender	retry-later when
_sendLogsBatch	err instanceof PostHogFetchNetworkError
_sendMetricsBatch	isPostHogFetchRetryableError(err)
_sendTracesBatch	isPostHogFetchRetryableError(err)

isPostHogFetchRetryableError returns true for HTTP 408, 429, and ≥500, or a network error. PostHogFetchNetworkError is network-only.

So once fetchWithRetry exhausts its internal retries against a struggling ingestion endpoint:

• logs classifies the 429/5xx as fatal and drops the batch
• metrics and traces classify it as retry-later and keep the batch queued for the next flush

Logs is the outlier — the two senders written later both use the broader predicate, and _sendMetricsBatch carries a comment explaining the choice. This looks like drift rather than a deliberate decision, but either way the behaviour isn't documented as intentional anywhere.

Why extracting helps

The bodies are otherwise identical: disabled check, JSON.stringify, gzip unless disableCompression, custom headers, fetchWithRetry with a retryCheck that excludes 413, then the same four-way outcome mapping.

A single private _sendOtlpBatch({ path, auth, payload }) returning SendBatchOutcome collapses ~150 lines to ~15 plus three thin wrappers, and makes the retry policy one decision instead of three copies that can silently disagree.

Notes

• Fixing the logs classification is a behaviour change to a shipped pipeline (batches that are currently dropped would start being retried), so it deserves its own verification rather than riding along silently in a refactor.
• Purely internal — no public API surface involved.

[turnipdabeets]

The encoder-side counterpart to this is #4571 — the logs toOtlpAnyValue emits int64 as a JSON number, which is why the traces encoder was written separately instead of sharing it.

[github-actions]

AI triage for #4570

Conclusion: Confirmed bug: logs drops batches after exhausted retryable HTTP failures, unlike metrics.
Confidence: 96%
Suggested labels: team/client-libraries

Confirmed in the current core implementation. _sendLogsBatch uses isPostHogFetchRetryableError for fetchWithRetry, but its final outcome mapping only treats PostHogFetchNetworkError as retry-later. Therefore, once internal retries are exhausted, HTTP 408/429/5xx failures are returned as fatal; PostHogLogs advances its persisted queue for fatal outcomes and drops the records.

_sendMetricsBatch already maps the same retryable predicate to retry-later, and merges the unsent window back for a later flush. The smallest safe fix is to make the logs outcome mapping use the same predicate and add focused regression coverage for exhausted 429 and 5xx responses (while preserving 413 as too-large and non-retryable 4xx as fatal).

One correction to the report: the current tree has two OTLP sender methods, logs and metrics; there is no _sendTracesBatch or traces OTLP payload implementation to extract. Since the behavior fix is small and verifiable, a broader sender abstraction should be evaluated separately rather than coupled to this data-retention fix.

Investigation

• isPostHogFetchRetryableError returns true for network errors and HTTP 408, 429, or statuses >=500 in packages/core/src/posthog-core-stateless.ts.
• Both logs and metrics pass that predicate to fetchWithRetry, excluding 413 from internal retries.
• After a send failure, _sendLogsBatch returns retry-later only for PostHogFetchNetworkError; retryable HTTP errors therefore fall through to fatal.
• _sendMetricsBatch instead returns retry-later for every isPostHogFetchRetryableError result, explicitly including exhausted 408/429/5xx failures.
• PostHogLogs._doFlush preserves queued records only for retry-later; it persists queue advancement before throwing for fatal, so the logs classification causes data loss.
• PostHogMetrics._doFlush merges a retry-later window back and re-arms its flush timer.
• The current repository contains _sendLogsBatch and _sendMetricsBatch, but no _sendTracesBatch, OTLP traces endpoint, or OTLP traces payload type.

Relevant files: packages/core/src/posthog-core-stateless.ts, packages/core/src/logs/index.ts, packages/core/src/metrics/index.ts, packages/core/src/__tests__/posthog.flush.spec.ts, packages/core/src/logs/types.ts, packages/core/src/metrics/types.ts

Fix assessment

• Straightforward: yes
• Risk: low
• Reason: The divergent outcome classification is directly evidenced and can be corrected with a one-condition change plus narrow sender-level regression tests. A generalized extraction is not required to fix the loss and is unsupported by the current two-sender implementation.
• Suggested approach: Change _sendLogsBatch to return retry-later when isPostHogFetchRetryableError(err) is true, matching metrics. Add focused tests using exhausted 429 and 5xx responses, and retain assertions that 413 maps to too-large and a non-retryable 4xx maps to fatal.

Draft PR: #4575

Pi session

The resumable JSONL pi session for this run was saved to a private gist: https://gist.github.com/marandaneto/a69e3d182381174afb30bd4a8bef2bd6

Download it locally, then fork it into your own pi session:

pi --fork path/to/session.jsonl

Saved session:

• call-1-1-2026-08-19T21-28-17-357Z_01a01bec-c6cd-71af-917c-574738cf06dc.jsonl

Close proposal

None. No issue close action was taken.

---

Generated by posthog-watcher-action using pi.