Skip to content

[PR] perf: avoid repeated fold closure scans within a batch - #6747

Merged
ranxianglei merged 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-14_linear-fold-registration
Nov 15, 2026
Merged

ranxianglei merged 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-14_linear-fold-registration

Conversation

@Afloat16

@Afloat16 Afloat16 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Closes #6744

Problem

recordCacheFoldsFromBlocks() calls pushFold() for each newly discovered block. Every call scans the complete retained ledger to freeze open k counters. After the first new record, only the newly appended tail can remain open within that synchronous batch, so the repeated full scans do unnecessary work on long histories and larger block batches.

Change

Keep a closure-scan starting index local to the current batch. The first eligible block still visits all old records, including multiple open records in a restored ledger. Each later eligible block begins at the previous new tail. Advance the index only after a successful append.

This preserves closed counters, fractional requestsAfter, record order and identity, duplicate/high-water filtering, and an open final successful fold if a later block throws. It adds no persisted fields and retains the complete history. Prefix-token calculation and subsequent sample accounting are unchanged.

Validation

  • Unchanged master a8bbb8d23108ba9148d09512927041364625d910: the 10 new tests report 2 bounded-work failures and 8 behavior controls passed.
  • With the change: 54 tests passed across new batch registration, existing cache-ledger tests, and coverage accrual.
  • The public API visits 599 fold records for 600 new blocks, versus 179,700 on the base. With 500 existing + 600 new, it visits 1,099 versus 479,700. The test counts actual array-record access instead of timing an assertion.
  • Behavior controls cover complete ledger/report equality against individual public calls, legacy open/closed records and fractional counters, empty/all-skipped batches, timestamp fallback and geometry, duplicate/out-of-order IDs, interrupted batches followed by retry, array/object identity, and lazy plugin discovery through recordCacheSample().
  • Official npm run typecheck and npm run build pass. The benchmark also passes strict TypeScript with the repository's NodeNext/ES2024 settings. git diff --check passes.

Public-function benchmark

Node 24.22.1 / Linux x64; one warmup per fresh process and seven alternating base/patch pairs. Fixture construction and full ledger/report checksums are outside the timed registration call. The report clock is fixed, with no output fields removed; complete ledger and report hashes match for every paired run.

Existing folds New folds Base median Patched median
0 6,000 36.94 ms 2.20 ms
6,000 6,000 116.28 ms 4.90 ms
6,000 1 0.104 ms 0.132 ms
0 3 0.015 ms 0.031 ms

The small inputs do not establish a consistent speedup and have microsecond-scale measurement variation. This change bounds the closure scan within a batch; the existing prefix-token calculation can still depend on message history.

The committed scripts/bench-fold-registration.ts reproduces one warmed process. Run it on each revision with the test preloads, alternating revisions for paired measurements:

node --import ./tests/preload-test-env.mjs --import ./tests/preload-storage-baseline.mjs --import tsx scripts/bench-fold-registration.ts

Existing work checked

Afloat16's historical production patches and non-PR issues were checked. The only earlier same-file patch, #6149, changes credential identity. #6256/#6687 concern the separate report minimum argument limit. The actual upstream patches in #1289, #1490, #2231 and #5651 preserve this repeated closure scan, and the open cache-ledger file-patch audit and function-specific searches contain no existing batch-closure optimization.

CI follow-up: deterministic report comparison

The Ubuntu and Windows Node 22 test shards in run 43602693147 exposed a fixture race in the new full-report comparison: the two calls to buildSessionCacheReport() could cross a millisecond boundary, so only generatedAt differed. The test now fixes Date.now() through its own TestContext; the complete ledger/report deep-equality assertions and production implementation are unchanged.

An independent advancing-clock preload deterministically reproduces the previous failure and passes after this fixture correction. The 54 focused tests, official typecheck and build pass on the tested tree. The follow-up is included in current head 3664e6ef06ac3d023e2962d6bfd19986f6f14ed2; its hosted CI result remains separate from these local checks.

@ranxianglei ranxianglei changed the title perf: avoid repeated fold closure scans within a batch [PR] perf: avoid repeated fold closure scans within a batch Nov 14, 2026
Louie0806 pushed a commit to Louie0806/billion-context that referenced this pull request Nov 14, 2026
@ranxianglei
ranxianglei merged commit 9a1c679 into ranxianglei:master Nov 15, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Batch cache-fold registration repeatedly rescans the complete ledger

2 participants