feat(streaming): add resumable result checkpoints

[seonghobae]

Product and reliability gap

Incremental streaming avoids whole-file materialization but restart recovery needs package-owned evidence before suppressing previously acknowledged records. A record number alone cannot detect prefix mutation, file replacement, framing changes, or truncation at/before the acknowledged position.

Bounded slice

This Draft adds immutable BatchResultCheckpoint / CheckpointedBatchResultRecord contracts and deterministic-close checkpoint iterators. Each checkpoint binds validated batch/endpoint identity, provider file kind/identifier, physical/logical position, exact raw line framing, and a domain-separated length-prefixed SHA-256 digest of the bounded prefix. Resume rescans from byte zero and yields nothing until the supplied checkpoint is reproduced exactly. Changed prefix/file identity or truncation at/before the checkpoint fails closed. Mutation strictly after the checkpoint remains outside the assurance boundary; this is not provider authentication, full-stream attestation, tenant authorization, or distributed exactly-once delivery.

Current exact staged state

• Head: 76b4abe68e8f2ff3deb8a05423d340dd32fef09e.
• Exact immediate base: feat(streaming): bound incremental result records #58 head c0b26eb8746820b13f7ef0f4d191827a8f7e2a94.
• Ancestry: one commit ahead / zero behind.
• CI 31284661860: success.
• Release Acceptance 31284661839: success.
• CodeRabbit status was successful and latest returned unresolved inline threads were zero; this remains staged evidence only.

Dependency chain

Current order is:

.github#790 -> #53 -> #57 -> #58 -> this PR -> #60 -> #92 -> #94 -> #95 -> #96 -> #97.

Keep Draft while predecessors are unintegrated. After protected integration, reconcile/retarget only for integration correctness and reacquire every required exact-head/exact-base quality, security, semantic-review, SBOM/provenance, supply-chain, packaging, branch-policy, and Release Acceptance gate.

Merge only when the unchanged final head has zero valid unresolved findings and every review/approval requirement actually imposed by live repository/CWL policy is satisfied. The protected code-owner gate is disabled/on hold for the current solo-maintainer state and must not be inferred as a universal approval requirement. No queued/pending/failed/cancelled/skipped/absent/stale/predecessor/generated-merge/status-only/author-only/synthetic-only evidence is final acceptance.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e177c35c-2150-4b20-b594-bb3e4f7cbf20

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

Exact-head contract finding on deb3f5ef8d679f06b226ff3594a8461999c204ee: the implementation is a prefix checkpoint, but the PR summary, ADR, changelog, and operator-facing wording repeatedly claim generic truncation detection. A prefix digest can detect mutation or truncation at or before the acknowledged checkpoint. It cannot detect mutation or truncation strictly after that checkpoint: for example, resuming after record 1 from a file that previously also contained record 2 will reproduce record 1 and then end successfully if the provider now truncates the suffix. Detecting that suffix change would require a stable provider validator or a previously established full-stream manifest, neither of which this portable streaming contract has.

Fix test-first without changing the bounded resume algorithm: add a deterministic behavior regression for suffix truncation after the checkpoint and an authoritative documentation contract requiring the limitation to be stated plainly. Narrow every generic truncation claim to the acknowledged prefix, document that successful resume does not attest the unseen suffix, and explain that hosts requiring whole-stream immutability need provider ETag/digest guarantees or a separate full-stream manifest pass. Preserve all current prefix mismatch, no-replay, confidentiality, lifecycle, and resource-limit behavior.

[seonghobae]

Exact-head verification on cc1595774ee6ab076f72dc6ba5a4175d3bec0f5d: the prefix-versus-suffix assurance finding is fully addressed. The permanent behavior regression proves that a checkpoint after record 1 can reproduce successfully even when an unseen record 2 is later truncated; the authoritative AGENTS, CLAUDE, architecture, ADR, doctoring, operator, and changelog contracts now limit fail-closed truncation detection to at or before the checkpoint and require a provider validator/authenticated digest or separate full-stream manifest for whole-stream immutability.

Exact-head CI 31110492311 and Release Acceptance 31110494315 completed successfully. The suite reports 557 passed, 4 deselected, 100% coverage of 2,094 production statements and 562 branches, 100% public docstrings, successful Python 3.10/3.12/3.14, lint, lock, package, Compose, and container gates. CodeRabbit status is successful and unresolved inline threads are zero. This author verification is not independent approval and does not make the stacked draft merge-ready; prerequisite integration and fresh integrated-base security/review evidence remain mandatory.

[seonghobae]

@coderabbitai review

Review-only request for exact current source head 76b4abe68e8f2ff3deb8a05423d340dd32fef09e against exact current stacked base c0b26eb8746820b13f7ef0f4d191827a8f7e2a94 (#58). The branch was replayed onto the current predecessor and is exactly 1 commit ahead / 0 behind. Current CI 31284661860 and Release Acceptance 31284661839 are completed/success on this source head; predecessor-head reviews do not transfer. Re-verify the prefix-checkpoint assurance boundary, exact resume/no-replay semantics, raw-line framing/digest identity, unseen-suffix limitation, pre-network identity validation, deterministic cleanup, resource bounds, compatibility, and authoritative documentation. Report source defects separately from upstream integration/review-governance prerequisites. Review only: do not mutate the branch, change Draft state, merge, release, weaken checks, or create repair workflows.

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

head_sha='76b4abe68e8f2ff3deb8a05423d340dd32fef09e'
base_sha='c0b26eb8746820b13f7ef0f4d191827a8f7e2a94'

printf '%s\n' '== Exact commits and ancestry =='
git rev-parse HEAD
git cat-file -e "${head_sha}^{commit}"
git cat-file -e "${base_sha}^{commit}"
git merge-base "$base_sha" "$head_sha"
git rev-list --left-right --count "${base_sha}...${head_sha}"
git diff --stat "$base_sha" "$head_sha"
git diff --name-status "$base_sha" "$head_sha"

printf '%s\n' '== Repository map =='
fd -H -t f -E .git | sed -n '1,240p'

printf '%s\n' '== Changed-file outlines =='
git diff --name-only "$base_sha" "$head_sha" | while IFS= read -r file; do
  case "$file" in
    *.py) ast-grep outline "$file" --items all || true ;;
    *.ts|*.tsx|*.js|*.jsx) ast-grep outline "$file" --items all || true ;;
    *) printf '%s\n' "-- $file" ;;
  esac
done

printf '%s\n' '== Checkpoint-related definitions and uses =='
rg -n -i -C 3 \
  'BatchResultCheckpoint|CheckpointedBatchResultRecord|checkpoint|resume|digest|sha256|physical.?line|logical.?record|raw.?line|close' \
  . -g '!**/.git/**' -g '!**/node_modules/**' -g '!**/dist/**' | sed -n '1,1400p'

Length of output: 201694

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

head_sha='76b4abe68e8f2ff3deb8a05423d340dd32fef09e'
base_sha='c0b26eb8746820b13f7ef0f4d191827a8f7e2a94'

printf '%s\n' '== Exact ancestry =='
printf 'HEAD: '; git rev-parse HEAD
printf 'merge-base: '; git merge-base "$base_sha" "$head_sha"
printf 'ahead/behind: '; git rev-list --left-right --count "${base_sha}...${head_sha}"

printf '%s\n' '== Changed paths =='
git diff --name-status "$base_sha" "$head_sha"

printf '%s\n' '== Exact diff =='
git diff --find-renames --find-copies --unified=80 "$base_sha" "$head_sha"

printf '%s\n' '== Relevant source and test files =='
fd -H -t f -E .git -E uv.lock \
  | rg -i '(batch|result|stream|checkpoint|openai|test.*(batch|result|stream)|adr|operator|guide|readme)' \
  | sed -n '1,500p'

Length of output: 121199

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Result streaming implementation map =='
ast-grep outline pg_llm_batch/result_streaming.py --items all

printf '%s\n' '== Full checkpoint-related implementation =='
rg -n -C 8 \
  '^(class |def |    async def |    def )|BatchResultCheckpoint|CheckpointedBatchResultRecord|iter_checkpointed_batch_records|open_checkpointed_batch_records|_validate_resume_checkpoint|_validated_stream_files|_iter_jsonl_file_lines|_parse_jsonl_line|_request|_credentials' \
  pg_llm_batch/result_streaming.py | sed -n '1,1800p'

printf '%s\n' '== Inherited validation and response lifecycle contracts =='
ast-grep outline pg_llm_batch/batch_api_client.py --items all
rg -n -C 8 \
  'def _validate_resource_id|def validate_endpoint_alias|async def _request|def _credentials|async def get_batch_status|async def __aexit__|async def close' \
  pg_llm_batch/batch_api_client.py pg_llm_batch/db.py pg_llm_batch/exceptions.py | sed -n '1,1600p'

printf '%s\n' '== Complete checkpoint tests =='
sed -n '1,620p' tests/test_resumable_result_checkpoints.py
sed -n '1,240p' tests/test_resumable_result_checkpoint_coverage.py

printf '%s\n' '== Public API compatibility references =='
rg -n -C 4 \
  'StreamingBatchAPIClient|BatchResultRecord|BatchResultCheckpoint|CheckpointedBatchResultRecord|iter_batch_records|open_batch_records' \
  README.md docs pg_llm_batch tests \
  -g '!**/uv.lock' | sed -n '1,1800p'

Length of output: 128129

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Checkpoint schema validation =='
nl -ba pg_llm_batch/result_streaming.py | sed -n '58,145p'

printf '%s\n' '== Checkpoint resume and mismatch behavior =='
nl -ba pg_llm_batch/result_streaming.py | sed -n '371,478p'

printf '%s\n' '== Endpoint alias validation =='
nl -ba pg_llm_batch/db.py | sed -n '122,165p'

printf '%s\n' '== Repository documentation requirements =='
rg -n -C 3 \
  'Update README|result streaming|checkpoint|documentation' \
  AGENTS.md CLAUDE.md ARCHITECTURE.md | sed -n '1,500p'

printf '%s\n' '== README result-streaming public API section =='
nl -ba README.md | sed -n '45,70p;280,360p'

printf '%s\n' '== Diff mentions in README =='
git diff --unified=3 \
  c0b26eb8746820b13f7ef0f4d191827a8f7e2a94 \
  76b4abe68e8f2ff3deb8a05423d340dd32fef09e \
  -- README.md

Length of output: 1216

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== Checkpoint schema validation =='
cat -n pg_llm_batch/result_streaming.py | sed -n '58,145p'

printf '%s\n' '== Checkpoint resume and mismatch behavior =='
cat -n pg_llm_batch/result_streaming.py | sed -n '371,478p'

printf '%s\n' '== Endpoint alias validation =='
cat -n pg_llm_batch/db.py | sed -n '122,165p'

printf '%s\n' '== Repository documentation requirements =='
rg -n -C 3 \
  'Update README|result streaming|checkpoint|documentation' \
  AGENTS.md CLAUDE.md ARCHITECTURE.md | sed -n '1,500p'

printf '%s\n' '== README result-streaming public API section =='
cat -n README.md | sed -n '45,70p;280,360p'

printf '%s\n' '== Diff mentions in README =='
git diff --unified=3 \
  c0b26eb8746820b13f7ef0f4d191827a8f7e2a94 \
  76b4abe68e8f2ff3deb8a05423d340dd32fef09e \
  -- README.md

Length of output: 30292

---

@seonghobae Review complete for source head 76b4abe68e8f2ff3deb8a05423d340dd32fef09e against base c0b26eb8746820b13f7ef0f4d191827a8f7e2a94.

The merge base matches the requested base. The branch is 1 commit ahead and 0 commits behind.

Source defects

1. P2 — Reject non-integer checkpoint schema versions.
   pg_llm_batch/result_streaming.py Line 80 checks schema_version != CHECKPOINT_SCHEMA_VERSION, but it does not require an int. Python treats 1.0 == 1 as true. Therefore, BatchResultCheckpoint(schema_version=1.0, ...) is accepted and compares equal to a generated version-1 checkpoint.

   This violates the stated strict, non-coercive persisted schema contract. A serialized checkpoint with a floating-point version can cross the resume boundary. Require isinstance(self.schema_version, int) and reject bool, as the other integer checkpoint fields already do. Add coverage for 1.0, Decimal-like numeric values if supported by callers, and other non-integer equal values.

2. P2 — Document the checkpointed API in README.md.
   README.md Line 56 and Line 288 document only iter_batch_records() and open_batch_records(). The file has no change in this PR. It does not expose BatchResultCheckpoint, iter_checkpointed_batch_records(), or open_checkpointed_batch_records().

   This is a public streaming API and changes the recovery, lifecycle, and assurance contract. AGENTS.md Line 104 requires a README update for these changes. Add a short checkpointed usage example and state these limits clearly:

   • resume rescans from byte zero and suppresses records only after exact checkpoint reproduction;
   • early-exit consumers must use open_checkpointed_batch_records();
   • successful resume does not attest mutation or truncation strictly after the acknowledged checkpoint.

Verified source behavior

• The checkpoint digest uses domain-separated, length-prefixed framing.
• The digest includes ordered file identity, raw line bytes, line number, and newline termination.
• Blank lines and CRLF framing affect the digest.
• Transport chunk boundaries do not affect the digest.
• Resume yields no record until checkpoint reproduction succeeds.
• The implementation detects changed prefixes, changed file identifiers, and truncation at or before the checkpoint.
• The implementation and authoritative documents correctly limit the assurance claim for the unseen suffix.
• Request identity validation occurs before credentials and network access.
• The context-managed checkpoint iterator closes the outer iterator and nested response path.
• Existing aggregate and non-checkpointed streaming APIs remain unchanged.

Upstream integration and review-governance prerequisites

No source defect above concerns the stacked branch relationship.

This draft remains non-merge-ready until the required predecessor sequence merges, the PR is reconciled to integrated main, and all required exact-head and exact-base quality, security, supply-chain, packaging, release-acceptance, and independent-review gates run again. The stated CI and Release Acceptance successes are useful staged evidence. They do not replace final integrated-base acceptance or an independent non-author approval.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[seonghobae]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[seonghobae]

Superseded by current-main replacement #177. Fresh verification shows #59 is still a Draft on obsolete stacked base agent/bounded-jsonl-result-streaming at head 683a3e7f52e5529817180a6ace0c2b2176d85412, while #177 is the active main-based rebuild at head e716b9e9f4a7f4f8613d19de25e15cb81ca0761a. The replacement owns the same checkpoint API/ADR/doctoring/result-streaming/test surfaces and additionally isolates the checkpoint implementation on top of the protected #172 streaming result. No checks, reviews, approvals, or generated-merge evidence from this PR transfer to #177.