Repository navigation
Hermes 插件上报宿主 per-model 上下文窗口(修 #3261:自定义模型按默认 200K 计算) - #3262
Conversation
…#3261) The hermes native plugin stamped model/max-output but never the context window, so custom-alias models (absent from every registry) sized sessions at the 200K default even when the host knew the real window. Now the plugin reads HERMES_HOME/context_length_cache.yaml (the host's own per-model window cache, keyed <model>@<base_url>) best-effort and reports it on both #955/#956 channels: x-bili-plugin-context-window header in the llm_request middleware and contextWindow in the runtime-info body (dedup key extended so a changed window re-reports). Base URL resolution: hook/request-supplied fields first, then the provider entry in config.yaml; matching is exact-URL then origin-granularity (cache keeps /v1, configs often omit it); conflicting per-upstream windows are never guessed (fail-open to the proxy's normal chain, like every other miss). Adjacent fix in the same call path: on_pre_api_request read 'table' unconditionally while binding it only inside the max_tokens-present branch — a hook fire without max_tokens raised NameError into the swallowing except and silently skipped the whole runtime-info report. Hoisted the binding.
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3762-pr.3262.9a44dc6This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3262.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 检索了 open+closed issue 与 PR(按失败签名:hermes / context window / context_length_cache / runtime-info):无其他条目做同一件事;master 上最近的 hermes-plugin 提交(#2905 attach、#2838 YAML 双后端、#2455 非 ASCII model 头保护)都是不同机制;引用 #3261 的只有本 PR。 正在独立审核(不复读 issue 侧结论):已拉取分支核对代码、确认 proxy 侧两条链路( |
(ai: vllm-qwen/qwen3.8-27b) PR #3262 独立审核结论(八问)审核方式:不复读 issue 侧分析,自己拉分支重读 diff、对照 master 旧代码、核 proxy 侧链路、本地定向验证。CI 39/39 全绿(head
小观察(不拦):每次模型调用最多触发 2×(cache+config) 次小 YAML 解析(两个 hook 各一次)——文件极小且 best-effort 设计本就如此,不值得为此加缓存失效复杂度,不立项。 判定:可合。 证据链完整(CI 39/39 + 本地定向 30/30 + 回归清点),diff 每行都服务于本 PR 目的,docs 四件套 en/zh 同步且与代码事实相符。按仓库规则合并按钮在你手里:#3262 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.445-master.c2a73cc # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #3261
是什么问题
Hermes 原生插件只上报 model/max-output,从不带 context window。自定义 provider 别名(如
group/sol)不在任何模型目录里,注册表兜底解析不到 →native=none→ 会话按默认 200K 算预算;而 Hermes 宿主自己明确知道真实窗口(HERMES_HOME/context_length_cache.yaml里就有group/sol@http://…: 500000)。同一代理下 pi 正常,因为 pi 两条通道都盖了。修法(纯 Python 插件侧,零 TS 变更)
proxy 侧两条链路本来都是通的(头通道
pluginReportedContextWindow+pluginHeadersMatchModel门控;runtime-info 的contextWindow字段handlePluginRuntimeInfo已解析),缺的只是插件侧把宿主真值带上来:context_window_for(model, base_url):best-effort 读HERMES_HOME/context_length_cache.yaml(宿主自维护、键<model>@<base_url>),复用既有双 YAML 后端模式(PyYAML/ruamel,[Bug] mitm_domains_from_config() bare 'import yaml' returns [] when PyYAML is absent -> silent blind tunnel (ruamel.yaml fallback suggested) #2838)。任何缺失/解析失败/无匹配一律 fail-open 回 None,proxy 走原有窗口链——最坏情况 = 现状。provider参数从config.yamlproviders 表解析(与mitm_domains_from_config()同源同键名)。匹配两级:归一化全 URL 精确匹配 → origin 粒度(缓存键带/v1后缀而配置常省略)。on_llm_request盖x-bili-plugin-context-window头(与 model 头同刻,天然满足pluginHeadersMatchModel门控);runtime-info body 加contextWindow字段,去重键从(model, maxOutput)扩为(model, maxOutput, contextWindow),窗口变化会重新上报。顺手修的同路径缺陷(独立标注)
on_pre_api_request里table只在max_tokens存在的分支内绑定、却在分支外无条件读取——hermes 以无max_tokens触发该 hook 时抛 NameError 被外层 except 吞掉,整条 runtime-info 上报静默丢失(包括 maxOutput 本身)。已把绑定提升到分支外。行为变更:旧行为 = 无 max_tokens 的 hook 触发不上报任何 runtime-info;新行为 = 正常上报(maxOutput 缺席、contextWindow 若有则带上)。纯修复,无回归面。行为变更披露
测试
tests/hermes-native.test.ts(定向本地全绿 30/30,typecheck 干净;全量回归以 CI 为准):attach-window场景:cache 含group/sol@http://127.0.0.1:3425/v1: 500000+ config provider 指向同 origin(不带/v1,覆盖 origin 粒度匹配腿)→ 断言头通道"500000"+ runtime-info{…, contextWindow: 500000};hook 调用刻意不带 provider kwarg(已知最小 hook 面),验证单条目无歧义回退腿。测试机无 YAML 后端时正腿跳过并 warn(负腿照跑)。attach-window-ambiguous场景:同模型两个上游两个窗口 → 断言两通道都不猜。配置面 / kernel / 发布注意
bili plugin update hermes刷新插件副本(安装器逐字节拷贝源文件,测试已钉住)。