feat(etl): replace durable-job pagination on repaired worker stack

[seonghobae]

Stack repair purpose

This draft replaces diverged PR #129 with a non-destructive pagination branch rooted at exact predecessor PR #143. The original #129 branch and its fail-first history remain preserved. Old #129 checks, reviews, approvals, statuses, comments, and base snapshots do not transfer to this replacement.

Exact current identity

• immediate predecessor PR: feat(etl): add lease-fenced durable job worker #143;
• base branch: repair/durable-job-lease-worker-75265fa;
• exact live predecessor/base tip: fda8ded9b07a6b0aebb3df8ef3009063655ebfc8;
• branch: repair/durable-job-cursor-list-fda8ded;
• exact current head: 2a37789e651316e9bf29cbbd68902aa827a50c63;
• current synthetic merge revision: c5ef67fdb54cb04a50398e2accf84a5680d45886;
• comparison to predecessor: ahead by 14 commits, behind by 0, merge base exactly fda8ded9b07a6b0aebb3df8ef3009063655ebfc8;
• GitHub mergeability: true;
• Draft: true;
• labels: none;
• formal reviews: none;
• unresolved review threads: zero.

Stack-divergence RCA and repair options

PR #129 targets the obsolete feat/durable-job-lease-worker branch and is non-mergeable. The durable-worker predecessor was replaced by #143 after #121 advanced, so the old pagination head no longer descends from the required exact predecessor.

Material repair options were evaluated against current repository authority and stack state:

• rejected — destructive rebase or force-push: would rewrite fail-first history and invalidate review/check provenance;
• rejected — ours/theirs conflict shortcuts: would make ancestry look linear without proving both change sets survive;
• rejected — continue the old downstream chain: would deepen a known invalid boundary;
• execute_now — replacement branch from exact feat(etl): add lease-fenced durable job worker #143: ordinary auditable commits can reapply the bounded pagination behavior and tests without new credentials, protection bypass, destructive history rewriting, or cross-repository writes.

The replacement path is the implemented option.

TDD and RCA record

RED 1 — missing pagination model on repaired ancestry

Creation head 4a555d62d950cf6d812866a89d1e74de1d88cb65 added the page-model contract before EtlJobPage and EtlJobPageResponse existed on the #143 ancestry. The test reached the intended production boundary and failed on the missing production pagination types rather than a fixture/import/setup defect.

Subsequent bounded commits reapplied owner-scoped page models, service/controller behavior, strict limit/cursor validation, deterministic keyset ordering, tenant isolation, and one-extra-row next-page detection.

RED 2 — production pagination migration missing

At exact source head 4bd27f205b1cab1210f62e33f87d105e18183606, the pagination production/controller/service tests were green, but literal-head CI 31292322341 failed on every hosted OS because EtlJobMigrationDocumentationTest reached the production migration boundary and could not find V5__add_etl_job_owner_pagination_index.sql. macOS job 93191493826 ran 384 tests with zero assertion failures and exactly two file-absence errors in:

• paginationMigrationUsesTheOwnerAndCompleteStableOrderingKey;
• paginationIndexMigrationDoesNotBlockProductionWriters.

This disproved the hypothesis that the replacement implementation was complete: the runtime query contract existed, but its nonblocking PostgreSQL index rollout artifact was absent.

The smallest feasible repair was to restore the bounded V5 migration and its migration-local Flyway transaction override from the preserved old pagination slice. Commit 266856b8a3e950534afe00abdb14f2b0da804e0b added CREATE INDEX CONCURRENTLY etl_job_owner_pagination_index over (principal_scope_hash, created_at DESC, job_record_id DESC). Commit 6e9fa4eff0459ed083f628c9682bf0bde246bac9 added V5__add_etl_job_owner_pagination_index.sql.conf with executeInTransaction=false.

Exact-head CI 31292548252 then succeeded on Ubuntu, macOS, and Windows; Dependency Review 31292548276 and CycloneDX SBOM 31292548282 also succeeded.

RED 3 — authoritative operator documentation still described the predecessor boundary

After the source and migration became green, the inherited worker runbook still said higher-level job-list pagination remained a later stack item. Commit f2ba86de2326619aaf8e6bef69497c192cb09a7c changed only the documentation contract first. Literal-head CI 31292668445, macOS job 93192431175, ran 385 tests with exactly one failure and zero errors:

• runbookDocumentsOwnerScopedKeysetPaginationAndRollback.

All production pagination, worker, controller, integration, migration, and unrelated tests passed. The RED therefore isolated documentation drift rather than a production regression.

Commit ef3c24006c4c37b4b87a28664e8b7f3a4b1d744b updated docs/etl/durable-job-intake.md with the owner-scoped list contract, canonical cursor/limit behavior, RFC 8288 next links, privacy boundary, V5 CREATE INDEX CONCURRENTLY rollout, executeInTransaction=false, invalid-index handling, and DROP INDEX CONCURRENTLY rollback. CI 31292757373 then succeeded across Ubuntu, macOS, and Windows.

Exact current head 2a37789e651316e9bf29cbbd68902aa827a50c63 records the pagination behavior and rollout in CHANGELOG.md without weakening tests or rewriting RED history.

Implemented buyer-visible boundary

The repaired slice now provides:

• authenticated principal-scoped GET /api/etl/jobs listing;
• bounded canonical limit validation;
• newest-first keyset pagination ordered by created_at DESC, job_record_id DESC;
• opaque canonical URL-safe cursor validation before database access;
• strict owner binding on every page query so cursor contents never grant authority;
• one-extra-row detection so only proven continuations produce a next cursor;
• RFC 8288 Link with rel="next" and Cache-Control: no-store;
• no payload, raw principal, raw idempotency key, internal hash, lease identifier, SQL, or exception text in list responses;
• PostgreSQL CREATE INDEX CONCURRENTLY etl_job_owner_pagination_index matching the exact owner/order access path;
• Flyway migration-local executeInTransaction=false for PostgreSQL's nontransactional concurrent-index requirement;
• operator guidance for invalid-index detection and concurrent rollback;
• descriptive multi-word snake_case database objects; and
• deterministic page-model, controller, service integration, tenant-isolation, migration, rollback, and documentation-alignment tests.

Exact-current-head evidence

For exact head 2a37789e651316e9bf29cbbd68902aa827a50c63 over unchanged exact predecessor fda8ded9b07a6b0aebb3df8ef3009063655ebfc8:

• CI 31292876884: success;
• Ubuntu job 93193007178: exact-source checkout, exact-SHA verification, and Unix tests succeeded;
• macOS job 93193007157: exact-source checkout, exact-SHA verification, and Unix tests succeeded;
• Windows job 93193007167: exact-source checkout, exact-SHA verification, and Windows tests succeeded;
• conditional test_self_hosted: skipped and not counted as positive evidence;
• Dependency Review 31292876913: success;
• CycloneDX SBOM 31292876885: success;
• CodeRabbit exact-head commit status: success;
• unresolved review threads: zero;
• formal reviews: zero;
• GitHub mergeability: true;
• exact predecessor comparison: ahead 14, behind 0.

SAST Semgrep and Security Scan are absent on this non-default stacked base and remain not passing rather than inferred from another head/base. No qualifying independent non-author formal APPROVED review exists on this exact head.

Dependency and merge boundary

This replacement is source-local GREEN and repairs the #143 -> pagination ancestry, but it is not merge-eligible. PR #121 is still the root prerequisite, and #143 remains stacked on #121. Direct-base SAST/Security evidence and independent exact-head approval are also absent for this non-default stacked target.

Do not deepen #130 or later old-stack branches from #129. After #121 and #143 integrate through protected history, retarget or re-anchor this replacement to the exact protected predecessor and regenerate every affected exact-head/base CI, Dependency Review, SBOM, SAST, Security Scan, coverage, migration, provenance, review-thread, automated-review, and independent-approval surface. Old #129 or predecessor evidence does not transfer.

Never bypass branch protection, stack order, exact-head binding, independent review, tests, coverage, security gates, writer leases, or migration rollback requirements.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3546a0ca-70fe-4452-aea3-52e2936d9409

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}