Repository navigation
[PR] fix: cache-attribution root cause (OpenAI omitted cache fields + cache-diff byte offsets) - #6517
ranxianglei wants to merge 0 commit into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9517-pr.6517.5556042This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6517.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Progress update — my worker process stalled (~34 min, likely a hanging dependency install) and was restarted. Here's where the review of PR #6517 (mirror #6518) stands: Code review — both fixes verified correct by reading the surrounding code:
Live state (just checked): PR head Next: I'll do a bounded local run of the targeted suites ( |
[bot] 🏷 PR #6517 review — both fixes verified correct; rebase needed for full-suite green I reviewed the code, ran the targeted suites locally (bounded), and verified the regression + golden claims. Code review — both fixes correct:
Verification (local, bounded):
CI (live-checked): 12 guard/lint lanes green (kernel-guard, version-guard, config-surface-guard, docs-sync, unused-exports, pi-subagents-guard, test-import-graph, lifecycle-ledger-guard, plan, lane-guard, docs-coverage, lock-sync); the 18 test/e2e/static lanes are skipped because the branch is 18 commits behind current master ( Verdict: code is correct and the fixes are sound. The branch must be rebased to current master so the CI 8问审核(中文,供人工 reviewer):
合并判定:代码正确、回归测试有效、黄金变更已验证、typecheck 干净、4 个定向套件全绿(63 测试 0 失败)、CI guard 车道全绿。唯一阻碍:分支落后 master 18 个提交,CI 的 18 个 test/e2e/static 车道被 skip——需 rebase 到当前 master 让 CI test 车道跑全量回归后才能给出权威 green 判定。代码可合(pending rebase + CI 全量绿)。 |
4fc5dfb to
6a641f3
Compare
(ai: vllm-qwen/qwen3.8-27b) Resolved the merge conflict. Key finding: this PR is fully redundant — every line of it is already in master, so after rebasing it had zero net-new content and GitHub closed it as a consequence (head became identical to base, nothing to merge). No code was changed by me. What happened:
七问:
说明:force-push 是任务明确授权的冲突 rebase 操作;PR 的关闭是 head 被推到与 base 一致后 GitHub 的自然结果(无内容可合),我没有执行任何显式 close 动作。该 PR 的内容全部已在 master,无需后续处理,建议 owner 确认关闭即可。 |
Root-cause fix for the cache-attribution batch: two defects that both corrupt cache hit-rate attribution, fixed in one PR.
What
1. OpenAI routing: omitted cache fields are now a measured zero, not "unmeasured" (#6329)
sglang returns
prompt_tokens_details: null(and OpenAI omits the field whencached_tokens = 0). The OpenAI adapter (src/loop/adapter-openai.ts) treated a missing field as "no data" (undefined→ unmeasured), so those requests were excluded from hit-rate attribution entirely. Result: the UI session hit rate was pulled down but the attribution panel could not explain the gap (a 12.6pp unexplained miss in the reported case), and TTL/eviction cost was systematically underestimated.Now:
prompt_tokenspresent + both cache fields absent ⇒cached = 0(a measured miss, counted in the denominator). A usage object missingprompt_tokensstays unmeasured.2.
bili acp-cache diff: byte offsets now read in original-text coordinates (#6495)acp-cache-diffcompared a UTF-8 byte mismatch offset against UTF-16 JSON span indices and sliced evidence at that byte index. A long CJK/emoji prefix inflated the byte offset past the UTF-16 end of the last shared element, so a pre-array field rewrite (e.g. a system-prompt change) was misclassified as a healthy append, and the divergencemessageIndex/evidence snippet were lost.textOffsetAtByteconverts the mismatch byte to an original-text position (anchoring inside a multibyte code point to its start) and is used only for the span comparisons and evidence slicing; public byte offsets and byte-based attribution are unchanged.Provenance
Both fixes were already prepared as separate PRs by external contributors; this PR cherry-picks them (original authorship preserved) so the root-cause batch closes in one merge:
8113815c. Only the OpenAI Routing: Requests where the upstream omits cache fields are misrecorded as "unmeasured" and excluded from attribu | OpenAI 路由:上游省略缓存字段的请求被误记为"未测量"并剔除出归因,命中率与归因面板差 12.6pp 无法解释 #6329-relevant changes are taken (adapter + regression test + golden); the unrelated storage/update changes that were bundled in [PR] fix: OpenAI cache attribution — treat omitted cache fields as measured zero #6340 are not included.b6c73fa3.Behavior change (intended)
cached = 0in the hit-rate denominator (was: excluded as unmeasured). The headline hit-rate number is unchanged; the attribution panel's "unattributed backfill" now accounts for these requests (e.g. −0.4% → ~−12.9% in the reported case), and TTL/eviction seam detection now covers them.acp-cache diff(offline diagnostic only): for multibyte-prefixed dumps, a genuinely pre-array divergence is now classifieddiverge(was: misreportedappend),preArrayis restored, anddivergence.messageIndex/evidence resolve correctly. PublicbyteOffsetvalues are unchanged. No live compression/cache path is touched.Verification
Targeted (local) — the full-suite verdict is the remote CI
testlanes:tests/loop-core.test.ts(OpenAI Routing: Requests where the upstream omits cache fields are misrecorded as "unmeasured" and excluded from attribu | OpenAI 路由:上游省略缓存字段的请求被误记为"未测量"并剔除出归因,命中率与归因面板差 12.6pp 无法解释 #6329 regression): pass — fails without the adapter fix.tests/golden-storage-render.test.ts: pass — the newchat.jsongolden is correct with only the OpenAI Routing: Requests where the upstream omits cache fields are misrecorded as "unmeasured" and excluded from attribu | OpenAI 路由:上游省略缓存字段的请求被误记为"未测量"并剔除出归因,命中率与归因面板差 12.6pp 无法解释 #6329 fix (the digest moves from the cachedTokens 0-vs-undefined change, not from any storage change).tests/acp-cache-diff-unicode.test.ts(10 new): pass;tests/acp-cache-diff.test.ts(19 existing): pass.tsc --noEmit: clean.Fixes #6329
Fixes #6495