Repository navigation
[PR] fix: prevent SimHash feature vote overflow - #6046
ranxianglei merged 2 commits into
Conversation
The storage driver persists fold-anchor SimHash values. Regenerate its four wire goldens after correcting signed-vote overflow. Verification restricts the persisted-state delta to foldAnchors.*.sh; outbound bodies and all other state remain unchanged.
(ai: vllm-qwen/qwen3.8-27b) CI re-verified live — ALL GREEN on current head What I found and did:
结构化结论(八问)
判定:CI 全绿,可进入人工 review/merge;merge 权归 owner。 |
[bot] 🏷 Dup screening: not a duplicate. #6036 is the originating bug report ("[Bug] Prevent SimHash feature vote overflow", Afloat16); #6046 is its only fix PR. No other open/closed issue or PR touches the SimHash vote overflow or the Independent code review of PR #6046 (head What I independently verified:
Behavior change to flag (mandatory disclosure): for inputs where Assessment: high-quality external contribution — minimal clean diff, correct root-cause fix, genuine regression coverage, no quality defects requiring rework (§②a not triggered). The PR is currently unlabeled; if labeling, I'd suggest 结构化结论(八问)
判定:方向正确、实现最小、测试真实、无回归风险(除已披露的一次性安全瞬态)。可进入 merge——仅需例行 rebase 到当前 master(应用干净、golden 不 stale),merge 权归 owner。CI 本轮镜像缓存实测 30 项全绿。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.1145-master.8095a49 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Problem
Signed 8-bit vote counters overflow within the existing 256-feature sampling budget. Repeating one shingle can reverse its sketch at 128 votes or collapse it to zero at 256 votes, allowing unrelated repeated messages to collide.
Changes
Use signed 16-bit vote counters, which cover the full sampled vote range without changing feature sampling or the 16-character serialized sketch format.
The regression suite checks repeated Latin and CJK shingles around the 127/128 and 255/256 boundaries and across sampling strides, plus distinct unrelated sketches at the cap.
Storage golden update
fold-reconcile.tsstores these same SimHash values inmetadata.foldAnchors.<id>.sh. Correcting the arithmetic intentionally changes those persisted fingerprints. Regenerate the four storage-render golden files with the repository'sscripts/update-storage-render-goldens.ts.The before/after verification compared complete normalized persisted state for all four wire scenarios. Only 13 fold-anchor
shvalues changed; outbound bodies, fold/edit landmarks, state shape, block contents, stats and all other persisted fields remained unchanged. Each golden changes onlystateDigest,stateDigestAlignedandsectionDigests.metadata. No assertion or normalization rule was weakened.Before/after verification: the unchanged base passed all 17 storage golden/restart tests; the corrected fingerprints and regenerated goldens passed all 36 SimHash, alignment, storage-render and restart tests. Typecheck and production build also passed.
Validation
Exact commit
c26c68a93fb11035a0e02cd08b171a295abbddb4passed all 36 tests,npm run typecheck, andnpm run buildin a fresh Node 22 checkout: committed-version verification. The tracked source and golden files remained unchanged during validation. This checks the targeted and related suites plus the four-wire storage and restart contracts; the upstream full CI matrix runs independently.Related work
Original feature #2266; no matching vote-overflow correction was found in the upstream or prior contribution history.
Closes #6036.