Skip to content

[PR] perf(registry): index relay model suffix candidates - #7285

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-19_index-registry-suffix-candidates
Nov 20, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-19_index-registry-suffix-candidates

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Fixes #7284.

Problem and change

Relay lookup scans all lowercase-index entries for each model suffix even when the underlying registry is unchanged. Repeated pricing/context/output queries therefore keep walking the entire roster for a small matching subset or a missing model.

Build suffix buckets once, lazily on the first relay fallback. Index every slash boundary so nested model ids retain the existing matching behavior. Buckets preserve Object.entries order and all case-collision rows; exact keys, known-provider precedence, variant order, first usable price, maximum context/output ceiling and conflict-warning order stay unchanged. The suffix map lives with the existing identity-checked table index and is rebuilt when that index is replaced. Its entries share the same immutable row records as the lowercase map.

This targets the remaining suffix scan, independently of #7279's billing-row normalization and #7282's two-table cache. No kernel, request-wire, configuration, persistence or model-facing text changes.

Validation

Base: 04ecbb42dd38543e9c407bd40eb773ae5ec8b60b, still current upstream master at publication. Commit: cee5b4e08100a5a3d54f5c666a82413b845d08ca.

  • 88/88 targeted registry, output-ceiling, configured-window precedence, model-window, billing and pricing-web tests passed, with no skips.
  • Two new parity tests pass on both the baseline and this implementation. They pin nested identifiers, case-collision rows, slash-boundary misses, provider isolation, exact-key precedence, variant order, first usable prices, maximum limits and replacing/revisiting table identities.
  • npm run typecheck, npm run build (including the kernel/pi-subagents builds and dist checks), and git diff --check passed.
  • Independent read-only review checked suffix-match equivalence, ordered candidate selection, callers and cache lifetime.

The full regression verdict remains the upstream CI on the pushed head.

Measurements and cost

The included scripts/bench-registry-suffix-index.ts uses the real bundled 7,126 cost rows and 428 model entries, no network. Each measured iteration resolves pricing, context and output. Each case warms 5,000 iterations before measuring another 5,000. Four isolated processes ran before/change/change/before, pinned to one CPU on Linux with Node.js 24.22.3. Output arrays are SHA-256 hashed after timing.

Run 500 real model basenames: CPU / wall ms 100 unlisted model names: CPU / wall ms
Before 1 1004.583 / 1008.162 1022.686 / 1024.144
Change 1 176.912 / 159.680 94.806 / 94.642
Change 2 154.903 / 154.741 90.627 / 90.313
Before 2 1484.580 / 1484.708 842.872 / 842.876

All four real-model output hashes are 7aeaa317ff2464199514139965e687d41566318002a77381b9ed73fc9b3ad7de; all missing-model hashes are 93a07756a5faaa16afac511398bdccefff61bcd36003e5ca0fcfc2e8292eac1b.

There is a storage/setup tradeoff. After explicit GC, the measured indexes retain about 0.4 MB more heap for this roster (roughly 1.2–1.4 MB before versus 1.6–1.8 MB after). First-lookup construction is outside the steady-state measurement: the real-model case ranged 16.4–44.5 ms before and 29.0–31.5 ms after; the missing-model case ranged 15.6–17.5 ms before and 20.1–23.0 ms after. Exact/provider-only lookups do not allocate the suffix buckets. Timings vary on this shared host; no proportional end-to-end request speedup is claimed.

Reproduce from either revision, copying the benchmark to the baseline when comparing:

node --expose-gc --import ./tests/preload-test-env.mjs --import tsx scripts/bench-registry-suffix-index.ts

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: opencode/vllm-qwen/qwen3.8-27b)

查重: 不重复. Same performance domain as the open #7279 (billing-row normalization) and #7282 (cost-index cache eviction), but a different mechanism: those fix repeated normalization and alternating-table index eviction; this PR removes the per-lookup O(n) suffix row scan itself, which neither of them touches. Source issue #7284 states the distinction explicitly. No other open PR targets the suffix scan.

标签: #7284 labeled p2 + scope/wiring on pickup (deterministic cost issue on the per-request pricing/context path, not an availability defect → p2; registry lookup is request-pipeline, cross-lane → no host-lane label).

Review — independent verification performed

  • Base is current: PR base 04ecbb42 == live origin/master at review time; mergeable: true.
  • CI (live, mirror cache): 30 checks, 0 failed, 10 test shards still in progress at check time. Final verdict remains the full CI run on the pushed head.
  • Equivalence, three independent ways:
    1. Construction: both call sites build suffix = /${name.toLowerCase()}`` (always slash-prefixed, name provably non-empty via `modelVariants`), and a bucket keyed by `lower.slice(slash)` over every slash position contains exactly the entries the old `lower.endsWith(suffix)` filter kept, in the same `Object.entries` order.
    2. Fuzz: 7,468 random (roster, suffix) pairs on the reachable slash-prefixed domain — old scan vs new buckets matched exactly (membership + order).
    3. Parity claim verified locally: the two new tests pass on the baseline src/registry.ts (36/36) and on this head, so they pin pre-existing behavior rather than the new mechanism.
  • Targeted suites on head: 84/84 (registry, price-profile, output-limit, configured-window precedence, model-windows, billing-prices, billing-web, price-profile-stamp) + 29/29 adjacent (registry-unsupported-proxy, issue-393, pi-subagent-registry, identity-registry, fallback-window-floor, preflight-window-clamp). npm run typecheck clean.
  • Lifetime/staleness: bySuffix lives on the identity-cached index object (regLowerIdx/costLowerIdx); a table swap replaces the whole index, so the suffix map can never outlive its table. _resetForTest clears both slots. No stale-state path found.
  • Interaction with [PR] fix(registry): cap the cost-row index cache at the two tables the billing path scans (#7278) #7282 (owner, open): [PR] fix(registry): cap the cost-row index cache at the two tables the billing path scans (#7278) #7282 rewrites costLowerIdx from a single slot to a 2-entry LRU Map — adjacent hunks in src/registry.ts, different lines, no semantic conflict (the LRU caches the same index objects this PR extends). Whichever merges first, the other needs only a context-shift rebase. Suggest merging [PR] fix(registry): cap the cost-row index cache at the two tables the billing path scans (#7278) #7282 first so this one rebases onto the final cache shape.
  • One examined non-issue: for a suffix without a leading /, the new helper returns [] where the old endsWith scan would have matched. Unreachable: both call sites always pass a slash-prefixed suffix, the helper is module-private, and its documented contract is "slash-delimited suffix". No change requested.
  • Tradeoff accounting: ~0.4 MB retained heap for the bundled roster (buckets share the existing immutable row records — no duplication), lazy construction so exact/provider-only lookups pay nothing, one-time cold cost disclosed. Against a measured ~6× cut on the relay-fallback scan, that is a sound trade.
  • Quality gates: no kernel/wire/config/persistence/model-facing-text changes; bench script follows the existing scripts/bench-*.ts convention and is not shipped (scripts/ outside files); diff is 4 files / +130 −12, every line on-topic; naming, comments and style match the file.

结构化八问

  1. 是什么问题: relay(未知 provider)回退路径上,每次 pricing/context/output 后缀查询都要线性扫全表(7,126 行),重复查询持续付 O(n) 成本。
  2. 什么原因导致的: 每表小写索引只有 byLower/entries 两个维度,没有后缀维度,endsWith 只能逐行过滤——索引缺维度是根因,不是查询写得慢。
  3. 影响范围与严重程度: 所有非 known-provider 宿主的模型定价/上下文窗口/输出上限查询(每请求路径);严重度为确定性性能成本(实测 5,000 次组合查询 ~1.0–1.5s CPU → ~0.15–0.18s),无可用性影响,p2。
  4. 怎么修复的: 首次 relay 回退时惰性构建斜杠边界后缀桶(嵌套 model id 的每个 / 位置都入桶),挂在既有 identity-checked 索引上随表替换而重建;exact/provider 命中路径不构建。
  5. 是否有回归问题: 无。逐项审过既有行为:exact-key 优先、case-fold 命中、known-provider 隔离(不跨 provider 扫)、variant 剥离顺序、first usable price、max context/output、冲突告警顺序与 once-per-name 去重、表身份替换后重建——全部保持;等价性经构造证明 + 7,468 组 fuzz + 基线/head 双向 parity 测试三重验证。
  6. 是否破坏缓存: 不。后缀桶生命周期与既有索引缓存完全一致(表身份不变则复用,替换则整体重建);无新的失效面。与 [PR] fix(registry): cap the cost-row index cache at the two tables the billing path scans (#7278) #7282 的 LRU 化可组合,无冲突。
  7. 是否改了设计方向: 没有。同一索引架构上加一个惰性维度,数据流、边界、存储布局均未动。
  8. 是否引入了新的配置: 没有。无新 env/config/默认值。

判定: 可合(human merge)——perf: 不属于 §7.4 纯 fix: 自动合并档,需人工合并;等 CI 全绿即可。建议顺序: 先合 #7282,本 PR rebase 后合入。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: opencode/vllm-qwen/qwen3.8-27b)

CI status update (live-verified): the full CI run on the pushed head cee5b4e (run triggered at PR creation) has completed green — all 12 test shards (ubuntu/windows × node 22/24), both static-gates, mysql-contract, and all 13 guard jobs success (red-signal skipped by design). A second re-triggered batch is still in progress (guards green so far, test shards queued, 0 failures) — no action needed on it.

Updated verdict: 可合 (mergeable) — full regression green on the pushed head; human merge per §7.4 (perf: is not auto-merge territory). Suggested order unchanged: merge #7282 first, rebase this PR onto it (context-shift only), then merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Relay registry lookups scan every row for each model suffix

2 participants