Skip to content

[PR] docs(#3548): model-level nudgeGrowthTokens override + coupled threshold effects - #3567

Closed
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-17_model-level-compress-efficiency-docs
Closed

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-17_model-level-compress-efficiency-docs

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Refs #3548

Part 1 of #3548's deliverables — the no-decision-dependency part: docs only, zero code/behavior change.

What changed

CONFIGURATION.md + CONFIGURATION.zh-CN.md, nudgeGrowthTokens section (mirrored):

  1. Coupled effects of changing the step — the growth gate is max(20000, 0.45 × step) (kernel minGrowthFloor/minGrowthRatio), the T1 token-mass trigger defaults to one full step, T2/T3 derive at step × 1.5 unless pinned by tierNudgeTokens. Raising the step (e.g. 50k → 100k) therefore moves gate → 45k, T1 mass → 100k, derived T2/T3 → 150k in ONE change, and each fold covers a larger span. To shift only the T1 mass trigger, set tierNudgeTokens.t1 instead.
  2. Per-model override pattern for an expensive-summary model sharing a route with cheaper models: key by the EXACT wire model id under providers[<url>].models["<id>"].compress (exact match, no substring matching; unmatched models keep route/global settings), with a neutral example whose value is flagged as a candidate to validate on your own workload — no performance claims.
  3. Verification pointer: each request's nudge decision log line prints the effective interval= and reason=, so the resolved settings are observable live.

Verified before writing (v0.1.197 source, commit 3aeddc1)

  • mergeCompress/resolveCompress do exact-key, per-field, global→provider→model merging (src/compress-settings.ts) — model-level nudgeGrowthTokens already works as documented.
  • Kernel decideNudge computes growthFloor = max(minGrowthFloor=20000, minGrowthRatio=0.45 × step) and tier thresholds t1 ?? step, t2/t3 ?? step × 1.5 (kernel/src/compress.ts).
  • tests/compress-settings.test.ts already pins model-level resolution, unmatched-model fallback, and growthFloor/growthCap flattening — so Part 1 needs no code or test changes.

No new config fields, no default changes. Parts 2–4 of #3548 (summary-prompt evaluation, reasoning/effort-scope verification, cost-attribution observability) await owner decision in the issue thread; this is the issue's single PR.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-17_model-level-compress-efficiency-docs (e0e18c3)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4218-pr.3567.5835901

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3567.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 在途项里唯一匹配「nudgeGrowthTokens 文档」的就是本 PR(#3567)。相关但不同者:#3548 是本 PR 的母 issue(本 PR 为其 Part 1,该 issue 的单一 PR);#1997/#2035 是「动态自适应 nudge 节奏」的功能请求(机制不同——动态自适应 vs 静态模型级覆盖,且是 feature 非文档);#2225/#2226、#1265 是已关闭的其它主题文档(compaction 时机 / flat-50K 设计记录);#2378 是当初引入 tierNudgeTokens 字段的 feature PR(已合并)。无并列重复。

审核结论:可合。 这是纯文档 PR——零代码、零行为、零配置面变更;所有技术断言我已对当前 master(@ b10c525)源码逐条独立核验通过,zh/en 逐句镜像一致。

结构化八问:

  1. 是什么:给 nudgeGrowthTokens 配置项补三段说明——改步长的联动效应、按精确模型 id 做模型级覆盖的写法、如何在日志里核对生效值。
  2. 什么原因:非缺陷修复。High compression cost on DeepSeek Flash: model-level efficiency config, output control and A/B validation proposal | [pi native][0.1.197] DeepSeek Flash 压缩成本较高:模型级效率配置、输出控制与对照验证建议 #3548 反映「高成本摘要模型与便宜模型共用一条路由时无法单独调该模型的压缩节奏」,Part 1 是把「现有已支持但没写清」的能力补齐进文档。
  3. 影响范围/严重程度:仅文档读者;对用户零行为影响(不改任何代码/默认值/字段)。严重程度不适用(无功能变更)。
  4. 怎么修的:在 CONFIGURATION.md 与 CONFIGURATION.zh-CN.md 的 nudgeGrowthTokens 小节各追加约 20 行镜像文本,描述既有机制,不新增任何配置面。
  5. 是否有回归:无。docs-only,不触碰任何代码路径/wire/缓存/重试/增量游标。真正的文档类风险是「文档说 A、实际是 B」——已逐条核验(见下),不存在反向失实。
  6. 是否破坏缓存:不涉及(无运行时改动)。
  7. 是否改设计方向:不涉及(纯说明性文档)。
  8. 是否引入新配置:不涉及。只文档化已存在字段(providers[url].models[id].compress.nudgeGrowthTokens、tierNudgeTokens),未新增/改名/改语义任何字段——符合 Configuration Surface Discipline,故无需 config-surface 章节。

技术断言逐条核验(master @ b10c525):

  • 增长门 max(20000, 0.45×step) → kernel/src/compress.ts:1456-1458(Math.max(minGrowthFloor, minGrowthRatio*nudgeGrowthTokens))+ kernel/src/config.ts:18-19(minGrowthFloor=20000, minGrowthRatio=0.45)✅
  • T1 质量触发默认=整步、T2/T3 默认=step×1.5、可被 tierNudgeTokens 钉死 → kernel/src/compress.ts:1482-1489(tier2GrowthMultiplier ?? 1.5;t1 ?? step / t2,t3 ?? tier2Threshold)+ src/compress-settings.ts:302-308(tierNudgeTokens→内核 nudge.tierGrowthTokens 映射)✅
  • 精确模型 id 匹配、不做子串、未命中回退路由/全局 → src/compress-settings.ts:122 route?.models?.[model]?.compress(直接键访问,精确相等)+ mergeCompress 的 pick = model ?? provider ?? global(:41-42,:73)✅
  • 步长扁平化 growthFloor=growthCap → src/compress-settings.ts:295-297 ✅
  • 日志行打印 interval= 与 reason= → src/server.ts:2166 diagNudge 输出格式;四个 lane(prepare-anthropic/google/openai/responses)均以 log("info", …) 每请求输出,非 debug 门控,interval 即生效步长 ✅
  • 示例 JSON 结构 providers[url].models[id].compress.nudgeGrowthTokens → 与 resolveCompress 路径一致,tests/compress-settings.test.ts:180-187(parseRouteEntry)已钉住模型级 compress 解析 ✅
  • PR 声称「测试已覆盖、无需新增」属实 → tests/compress-settings.test.ts:15(model>provider>global)、:44/:59/:78(URL 最长前缀 + 模型覆盖路由 + 未命中回退)、:88(flattening)✅

diff 整洁度:仅 2 文件 +20/−1,每行都服务本 PR 目的;zh/en 逐句镜像对齐;锚点 [tierNudgeTokens](#tiernudgetokens) 指向已存在的 #### tierNudgeTokens 小节(CONFIGURATION.md:1258)可解析;示例值明确标注「candidate value — validate on YOUR workload」无性能断言。干净。

非阻塞观察:分支 mergeable_state: behind(落后于 master),但 mergeable: true 无冲突;本 PR 仅触碰 2 个文档文件且无文本冲突,rebase 一次即可干净合入。建议 owner 合入前 rebase 刷新基线(非阻塞)。

判定:可合。 建议 owner 自行 rebase 后合并(按规则我不代合 PR)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

孤儿票清理:空挑验证——分支对 master 已无独立提交(内容已全部落地)。关单;误判请 reopen。

@ranxianglei ranxianglei changed the title docs(#3548): model-level nudgeGrowthTokens override + coupled threshold effects [PR] docs(#3548): model-level nudgeGrowthTokens override + coupled threshold effects Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant