Repository navigation
[PR] perf: root-cause fix for billing-registry-lookup-perf (Fixes #7276, #7278, #7284) - #7363
Closed
ranxianglei wants to merge 4 commits into
Closed
ranxianglei wants to merge 4 commits into
ranxianglei wants to merge 4 commits into
Conversation
(cherry picked from commit 4ebcc22)
…ling path scans (#7278) The billing price path scans two cost tables per lookup (the overrides table, then the registry fallback). costLowerIndex kept only ONE cached index, so an overrides-miss lookup evicted the overrides index for the registry index and back again — rebuilding the (large) registry cost index on every lookup, O(registry rows). Make costLowerIndex a small LRU Map (cap 2: the two tables the path scans), refreshing recency on hit and evicting the oldest on overflow. The registry index is now built once and reused; bounded memory. The model-registry index (regLowerIdx) stays a single slot — it has only one table. Regression test counts index builds via a Proxy ownKeys trap and pins the registry index to exactly one build across repeated overrides-miss lookups (fails on the old single-slot cache). (cherry picked from commit 4521b10)
The committed test exercised stable identities for both cost tables (the post-#7276 shape). At the PR head the real path (overrideCosts, pre-#7276) hands scanRegistryCostRow a fresh overrides identity per lookup; a FIFO cache would evict the stable registry index every other lookup and rebuild it, yet the stable-identity test cannot see that. Add a fresh-identity case that pins the recency-refresh: passes on the LRU, fails (25x rebuilds) on a FIFO variant. (cherry picked from commit a5b32a2)
(cherry picked from commit cee5b4e)
Owner
Author
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 审核结论:可合(CI 全量回归进行中,当前 0 失败)独立复核了 PR #7363(镜像 #7366)的全部声明——本地实测,非转述 issue 结论: 验证记录
代码审查(独立读 diff)
一处 PR 正文小误(非代码问题):#7282 的 git author 实为 结构化八问
判定:可合(定向测试 + typecheck + build 本地绿,解析结果字节级不变,代码审查无回归)。全量回归以 CI 为准,当前进行中(0 失败);若 CI 红我再定向排查。 合并后 #7279 / #7282 / #7285 / #7280 变冗余,请 owner 关闭(agent 不关 PR)。 |
Owner
Author
This was referenced Nov 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
根因修复:billing-registry-lookup-perf(3 个 issue,一个 PR)
本 PR 是根因汇聚单 [Root Cause] billing-registry-lookup-perf 的唯一根因修复 PR:cherry-pick 三个
已有 PR 的全部 commit(保留原作者署名,
-x标注来源),一次闭环。4ebcc229859resolveBillingPrice每次调用重建整张 override 归一化表 → 改为setBillingOverrides一次构建、之后复用4521b10954c+a5b32a22777costLowerIndex单槽 identity 缓存被「每次扫两张表」互踢 → 改为 cap=2 的 LRU Mapcee5b4e0810endsWith扫描 → 首次 relay fallback 时懒建 slash 边界后缀桶三个 commit 同基于
04ecbb42;master 自那以后未触碰src/billing.ts/src/registry.ts及相关测试,无冲突按序 cherry-pick(7279 → 7282 → 7285)。
机制(为什么是一个根因)
计费/registry 查找路径上的三处「每次查找重复建表 / 建索引 / 全表扫描」:
overrideCosts);costLowerIndex);三者叠加:一次 overrides-miss 的 relay 模型查找 = 重建 override 表 + 重建 registry 成本索引 + 全表
后缀扫描(成本行 + 模型行各一遍)。表越大、配置模型越多,每次查找越贵。
验证
tests/billing-prices.test.ts/tests/issue7278-registry-cost-index.test.ts/tests/registry-price-profile.test.ts/tests/registry.test.ts;npm run typecheck+npm run build绿;scripts/bench-registry-suffix-index.ts(捆绑真实 roster:7126 成本行 / 428 模型):500 名 × 5000 次 CPU 65 ms(同机修复前 ~400 ms);未收录模型 100 名 × 5000 次 19 ms(修复前 ~251 ms);
输出 SHA-256 与 Relay registry lookups scan every row for each model suffix #7284 审查记录一致(
7aeaa317…/93a07756…)——解析结果字节级不变;与既有 PR 的关系
请 owner 关闭(agent 不关 PR)。
行为变更
无:无配置面 / 线协议 / 持久化 / kernel / 模型可见文本变更;解析出的价格、窗口、输出上限逐字节不变
(基准 digest 钉住)。
Fixes #7276
Fixes #7278
Fixes #7284