Repository navigation
[PR] fix(billing): build the normalized cost table once per config load, not per lookup (#7276) - #7280
[PR] fix(billing): build the normalized cost table once per config load, not per lookup (#7276)#7280ranxianglei wants to merge 1 commit into
Conversation
…ot per lookup (#7276) resolveBillingPrice is hot (per-request stamping + the pricing UI) but called overrideCosts() on every invocation, allocating a fresh CostsShape each time. That new table identity defeated costLowerIndex's identity cache, so the lowercase index was rebuilt on every lookup too — even an exact override hit. Build the normalized table once in setBillingOverrides (which every config reload path already funnels through) and reuse it until the next load. The setter rebuilds unconditionally (even for a repeated object) so in-place mutation between sets can't leak stale rows; an empty/cleared set yields null so the registry fallback applies immediately. No config field, pricing formula, matching order, request wire, or kernel behavior changes.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.10959-pr.7280.79ce547This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr7280.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) Verified: the fix is correct and CI is in progress (0 failures) — but this PR duplicates the still-open PR #7279 by the original contributor, and per the no-replace policy #7279 should be the one that gets merged. Your call is needed. This is a [PR] pickup, so I verified rather than re-did: Code (head abc252d) — correct, no behavior change
CI (live, mirror cache): 30 checks — 0 failed, 13 in progress (11 test shards + 2 static-gates); every completed lane green. Verdict pending. Labels: applied to the mirror issue on pickup — bug, p2, scope/other (plus the existing pr).
Recommendation: merge #7279 and close this PR (+ delete the upstream branch 结构化八问
判定:代码本身可合(等 CI 收尾转绿);合并路径卡在重复 PR 决策上——建议合 #7279、关本 PR。 |
What
resolveBillingPriceis a hot path (per-request billing stamping + the pricing UI), but it calledoverrideCosts()on every invocation, allocating a freshCostsShapeobject each time. That new table identity defeatedcostLowerIndex's identity cache (src/registry.ts:564), so the lowercase lookup index was rebuilt on every lookup too — even for an exact override hit. A price table with N configured models paid for N full row conversions plus a full re-index on each of its lookups.Fix
Build the normalized cost table once in
setBillingOverrides(which every config-reload path already funnels through — startup, web Apply onChanged,/__bili/configreload) and reuse it until the next load:overrideCostRows(module-level) is rebuilt in the setter, unconditionally — even for a repeated input object — so in-place mutation between sets can't leak stale rows.null, so the registry fallback applies immediately.resolveBillingPricenow reads the cached table instead of rebuilding it.Evidence
Synthetic exact-override-hit workload (1000 configured models, 5000 lookups, fresh process, 100-lookup warmup), this host:
Identical result hashes before/after. (The reporter's 4850 ms is the same mechanism on a shared box.)
Regression coverage
Two new tests in
tests/billing-prices.test.ts:output/cacheRead/cacheWriteexposed immediately); case-folded model names still hit.21/21 billing tests pass; typecheck clean.
Scope / no behavior change
No config field, pricing formula, matching order, request wire, or kernel behavior changes. The resolved prices are byte-identical to before — only the per-lookup rebuild is removed.
Related
costLowerIndexis a single-slot identity cache that thrashes between the two tables. Out of scope here (touchesregistry.ts, needs a small eviction-policy decision).Credit
Approach and regression tests follow the implementation proposed in #7276 by @Afloat16 (branch
2026-11-19_cache-billing-cost-rows); independently verified and re-based onto master.