Skip to content

fix(queue): preserve durable records across retries - #740

Open
dustinbyrne wants to merge 1 commit into
mainfrom
fix/durable-queue-lifecycle
Open

fix(queue): preserve durable records across retries#740
dustinbyrne wants to merge 1 commit into
mainfrom
fix/durable-queue-lifecycle

Conversation

@dustinbyrne

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Durable event and replay queues currently couple retention to flush retry exhaustion: repeated retryable failures can clear every persisted record. Queue filenames also reuse payload UUIDs, so duplicate payload identities do not represent distinct durable entries.

This aligns the shared file-backed queue with PostHog/sdk-specs#53: retryable transport and HTTP failures retain bounded durable records, known-offline periods consume no attempts, and every enqueue has a stable queue-entry identity. HTTP 413 continues shrinking to a singleton poison record.

Cross-SDK implementation: PostHog/posthog-ios#788

💚 How did you test it?

  • ./gradlew :posthog:test --tests 'com.posthog.internal.PostHogQueueTest'
  • make testJava
  • make checkFormat
  • make api — no API snapshot changes
  • git diff --check
  • independent read-only implementation and cross-SDK reviews found no blockers

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with Pi using isolated implementation, scout, and review subagents. The human directed the durability contract and cross-SDK scope. Queue capacity and bounded backoff remain the resource controls; no public option or default was added.

@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "fix(queue): preserve durable records acr..." | Re-trigger Greptile

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

posthog-android Compliance Report

Date: 2026-09-04 20:09:40 UTC
Duration: 118518ms

✅ All Tests Passed!

46/46 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 397ms
Format Validation.Event Has Uuid 35ms
Format Validation.Event Has Lib Properties 28ms
Format Validation.Distinct Id Is String 25ms
Format Validation.Token Is Present 24ms
Format Validation.Custom Properties Preserved 26ms
Format Validation.Event Has Timestamp 23ms
Retry Behavior.Retries On 503 7030ms
Retry Behavior.Does Not Retry On 400 4031ms
Retry Behavior.Does Not Retry On 401 4026ms
Retry Behavior.Respects Retry After Header 7023ms
Retry Behavior.Implements Backoff 17023ms
Retry Behavior.Retries On 500 7019ms
Retry Behavior.Retries On 502 7021ms
Retry Behavior.Retries On 504 7019ms
Retry Behavior.Max Retries Respected 17036ms
Deduplication.Generates Unique Uuids 45ms
Deduplication.Preserves Uuid On Retry 7017ms
Deduplication.Preserves Uuid And Timestamp On Retry 12032ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 7021ms
Deduplication.No Duplicate Events In Batch 40ms
Deduplication.Different Events Have Different Uuids 26ms
Compression.Sends Gzip When Enabled 21ms
Batch Format.Uses Proper Batch Structure 22ms
Batch Format.Flush With No Events Sends Nothing 14ms
Batch Format.Multiple Events Batched Together 35ms
Error Handling.Does Not Retry On 403 4024ms
Error Handling.Does Not Retry On 413 4021ms
Error Handling.Retries On 408 5028ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 39ms
Request Payload.Flags Request Uses V2 Query Param 44ms
Request Payload.Flags Request Hits Flags Path Not Decide 22ms
Request Payload.Flags Request Omits Authorization Header 28ms
Request Payload.Token In Flags Body Matches Init 27ms
Request Payload.Groups Round Trip 30ms
Request Payload.Groups Default To Empty Object 28ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 27ms
Request Payload.Disable Geoip Omitted Defaults To False 21ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 22ms
Request Lifecycle.No Flags Request On Init Alone 14ms
Request Lifecycle.No Flags Request On Normal Capture 23ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 39ms
Request Lifecycle.Mock Response Value Is Returned To Caller 25ms
Retry Behavior.Retries Flags On 502 325ms
Retry Behavior.Retries Flags On 504 322ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 25ms

@dustinbyrne
dustinbyrne force-pushed the fix/durable-queue-lifecycle branch from e99dfe0 to 438a88a Compare September 4, 2026 20:03
@dustinbyrne
dustinbyrne marked this pull request as ready for review September 4, 2026 20:17
@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Reviews (2): Last reviewed commit: "fix(queue): preserve durable records acr..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants