Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions CLIENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,11 +245,14 @@ pure-stdlib Python module shipped inside the npm package:
- **Plugin-mode stamping:** an `llm_request` middleware stamps
`x-bili-plugin: hermes` + conversation id (= the hermes session id, so
gateway multi-session stays safe) + model, and `x-bili-plugin-max-output`
once known — ONLY after the ACP tools are registered against the live proxy
manifest; round 1 rides wire mode. A `pre_api_request` hook captures the
effective `max_tokens` and pushes runtime-info (model + max output) to the
proxy. `compress` / `decompress` / `acp_status` are registered as real
hermes tools served by the proxy's existing plugin endpoints.
once known — plus `x-bili-plugin-context-window` when the host's own
per-model window is known (read from `HERMES_HOME/context_length_cache.yaml`,
keyed `<model>@<base_url>`; ambiguous entries are never guessed, #3261) —
ONLY after the ACP tools are registered against the live proxy manifest;
round 1 rides wire mode. A `pre_api_request` hook captures the effective
`max_tokens` and pushes runtime-info (model + max output + context window)
to the proxy. `compress` / `decompress` / `acp_status` are registered as
real hermes tools served by the proxy's existing plugin endpoints.
- **Known limitations:** requests going out hermes' Codex-wire transport may
drop the per-request header surface, so such setups stay in wire mode until
that transport exposes headers. Inert when `BILLION_CONTEXT_PROXY` is set
Expand Down
2 changes: 1 addition & 1 deletion CLIENTS.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ fork 继承同一面;Antigravity 自带用户插件系统 —— `plugins/<name>

- **安装:** `bili plugin install hermes` 把 `plugin.yaml` + `__init__.py` 拷进 `~/.hermes/plugins/billion-context/`,写一个机器自管的 `bili.json` sidecar(指向全局 bili 安装的 `dist/index.js` + node 路径),并经 hermes 自己的通道启用插件(`hermes plugins enable billion-context` —— CLI 不在 PATH 上时改为打印同一条命令)。新开一个 hermes 会话生效。卸载:`bili plugin remove hermes`;全局更新后刷新:`bili plugin update hermes`。
- **生命周期:** 加载时插件先附着到健康的运行中代理 —— #2905 起按泳道:本 hermes lane 的实例第一,其他已声明 lane 的实例绝不 attach(要搭车需在 `native.attachLanePriority` 里列出),8787 用户区守护是最后兜底 —— 否则在临时端口自拉起(声明 `lane: hermes`,别的泳道骑不上来;父进程 pid 看门狗在 hermes 退出时收掉它;并发启动走与启动器相同的 starting-marker 仲裁协议)。只有代理确认健康后,才用 `HTTPS_PROXY` / `https_proxy` + `SSL_CERT_FILE`(bili 组合 CA bundle —— 当前 hermes 经此解析环境信任;`HERMES_CA_BUNDLE` 保留给旧版本)把 hermes 的 httpx 栈指向它 —— **从不改动** `~/.hermes/config.yaml`。provider 的 https 域名从 hermes 配置读出并加入 MITM 白名单;其余域名与启动器模式一样盲隧道。拉不出健康代理时插件静默退场,流量直连(不压缩、无死端口)。
- **Plugin 模式盖章:** `llm_request` 中间件打 `x-bili-plugin: hermes` + 会话 id(= hermes session id,gateway 多会话安全)+ 模型,已知后再加 `x-bili-plugin-max-output` —— 且只在 ACP 工具已对着存活代理清单注册完之后;第一轮走 wire 模式。`pre_api_request` hook 捕获生效的 `max_tokens`,把 runtime-info(模型 + 最大输出)推给代理。`compress` / `decompress` / `acp_status` 注册为真正的 hermes 工具,由代理既有的插件端点提供。
- **Plugin 模式盖章:** `llm_request` 中间件打 `x-bili-plugin: hermes` + 会话 id(= hermes session id,gateway 多会话安全)+ 模型,已知后再加 `x-bili-plugin-max-output`,宿主 per-model 窗口已知时再加 `x-bili-plugin-context-window`(读自 `HERMES_HOME/context_length_cache.yaml`,键为 `<model>@<base_url>`;同模型多上游窗口不一致时宁缺毋猜,#3261)—— 且只在 ACP 工具已对着存活代理清单注册完之后;第一轮走 wire 模式。`pre_api_request` hook 捕获生效的 `max_tokens`,把 runtime-info(模型 + 最大输出 + 上下文窗口)推给代理。`compress` / `decompress` / `acp_status` 注册为真正的 hermes 工具,由代理既有的插件端点提供。
- **已知局限:** 走 hermes Codex-wire 传输发出的请求可能丢掉逐请求头面,这类配置在该传输暴露头之前停留在 wire 模式。`BILLION_CONTEXT_PROXY` 已设置(启动器管着代理)或定义了 `BILI_PROVIDER_REWRITES` 时插件整体退场。退出开关:`BILI_NATIVE_HERMES=0`。

## ZCode(Z.ai / bigmodel coding plan)
Expand Down
8 changes: 5 additions & 3 deletions TECHNICAL-NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,11 @@ Implementations: `src/agent/pi.ts` (covers pi and omp),
`src/agent/dsh-native.ts`, `src/kimi/native-mcp.ts` (bootstrap-time report
only — kimi's provider `custom_headers` are static, so per-request headers
would go stale on model switch), `hermes-plugin/__init__.py` (Python plugin:
per-request headers via an `llm_request` middleware, max output captured by a
`pre_api_request` hook) — other client integrations should follow the same
protocol.
per-request headers via an `llm_request` middleware, max output and the host's
per-model context window — read from hermes' own
`HERMES_HOME/context_length_cache.yaml`, fail-open on any miss (#3261) —
captured by a `pre_api_request` hook) — other client integrations should
follow the same protocol.

**DSH `maxOutput` reporting is declaration-gated — an undeclared custom model empties the whole ceiling chain (#2904).** `src/agent/dsh-native.ts` forwards `maxOutput` **only** when the host's `resolveModelInfo(provider, model)` returns a positive `defaultMaxTokens`, and DSH populates that **only** when the model item explicitly declares `maxTokens` — provider-level output defaults are never reported. For a user-added custom model with no declared `maxTokens`, `defaultMaxTokens` is `undefined`, so BOTH runtime-info channels come back empty. With no models.dev entry and no bili-configured `output` either, the four-source ceiling chain (#1840) resolves to nothing, and `restoreOutputBudget` lifts a client-starved budget to the session high-water mark — which on a long session has decayed through small positives to a poisoned low value. Every turn then runs reasoning-only and stops at `length`/`max-tokens`; switching to a model WITH a known ceiling masks it (the high-water is floored up to the ceiling), so the switch is a trigger, not a cause. The real fix home is **DSH-side**: report its configured provider/model output default instead of `undefined` (the immediate user workaround is setting `maxTokens` on the model item). bili deliberately does not guess a quota floor (#1842); instead #2904 surfaces the degraded state persistently on the `/acp` panel, the Web UI session page (data source `/__bili/sessions/<id>/detail`), and `/__bili/plugin/status?conversationId=` (`outputBudgetDegraded`) — clearing automatically once a healthy value or ceiling arrives — and extends `degenerateTurnWarning` to `length`/max-output terminations so the truncation correlates with its cause in the log.

Expand Down
2 changes: 1 addition & 1 deletion TECHNICAL-NOTES.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ hook 附着前先探测候选者的 `/__bili/health`:armed → 附着并注册 w
| 逐请求头(门控在 `x-bili-plugin`) | 每次模型请求 | `x-bili-plugin-context-window`、`x-bili-plugin-max-output`、`x-bili-plugin-model` |
| `POST /__bili/plugin/runtime-info`(回环地址) | 插件自举 + 任一上报字段变更 | `{agent, model, contextWindow?, maxOutput?, baseURL?, conversationId?, source}` |

窗口解析顺序:`anthropic-beta` 协商 > 逐请求 plugin 头 > runtime-info > launcher 环境变量 > 路由配置 > models.dev 注册表 > 内置表。runtime-info 这一步:带 `x-bili-plugin` 头的请求读**按 agent 的条目**(agent+model 必须匹配);不带该头的请求解析以 `conversationId` 记录的**会话级条目**,键与会话绑定的同一会话信号一致(客户端会话头、自定义 session 头或请求体的 `prompt_cache_key`)—— 无论哪种,model 都必须匹配(#1531:omp 打 `prompt_cache_key` 但不打 plugin 头,且主/子代理会话共用 agent 名却跑不同模型)。上报的 `maxOutput` 仅在请求体自带输出预算缺席时兜底。现有实现:`src/agent/pi.ts`(覆盖 pi 与 omp)、`src/agent/opencode-native.ts`(v1)、`src/agent/opencode-v2.ts`、`src/agent/dsh-native.ts`、`src/kimi/native-mcp.ts`(仅自举时上报 —— kimi 的 provider `custom_headers` 是静态的,逐请求头会在模型切换后过期)、`hermes-plugin/__init__.py`(Python 插件:经 `llm_request` 中间件打逐请求头,`pre_api_request` hook 捕获最大输出)—— 其他客户端接入请遵循同一协议。
窗口解析顺序:`anthropic-beta` 协商 > 逐请求 plugin 头 > runtime-info > launcher 环境变量 > 路由配置 > models.dev 注册表 > 内置表。runtime-info 这一步:带 `x-bili-plugin` 头的请求读**按 agent 的条目**(agent+model 必须匹配);不带该头的请求解析以 `conversationId` 记录的**会话级条目**,键与会话绑定的同一会话信号一致(客户端会话头、自定义 session 头或请求体的 `prompt_cache_key`)—— 无论哪种,model 都必须匹配(#1531:omp 打 `prompt_cache_key` 但不打 plugin 头,且主/子代理会话共用 agent 名却跑不同模型)。上报的 `maxOutput` 仅在请求体自带输出预算缺席时兜底。现有实现:`src/agent/pi.ts`(覆盖 pi 与 omp)、`src/agent/opencode-native.ts`(v1)、`src/agent/opencode-v2.ts`、`src/agent/dsh-native.ts`、`src/kimi/native-mcp.ts`(仅自举时上报 —— kimi 的 provider `custom_headers` 是静态的,逐请求头会在模型切换后过期)、`hermes-plugin/__init__.py`(Python 插件:经 `llm_request` 中间件打逐请求头,`pre_api_request` hook 捕获最大输出与宿主 per-model 上下文窗口 —— 窗口读自 hermes 自己的 `HERMES_HOME/context_length_cache.yaml`,任何缺失/歧义 fail-open(#3261))—— 其他客户端接入请遵循同一协议。

**omp native:当 OMP 的 Cowork 传输接管出口时,不打 `prompt_cache_key`(#2583)。**native omp 模式下,打上的 `prompt_cache_key` 是唯一的逐请求会话信号,其唯一消费者是 bili 自己的 fetch 层拦截器(把模型 URL 改写成 `<proxy>/bili/<url>` 并在转发前剥掉该字段)。较新的 OMP 构建带有 "Cowork 传输"(2026-07-30 引入,can1357/oh-my-pi `642892ae`;代理处理 `e22a5ff2`):默认 `anthropic`/`anthropic-messages` 请求经 node:https **直发** —— 除非显式代理环境变量(`PI_PROXY` / `PI_PROXY_ANTHROPIC`)把它们推回 Bun 的 fetch,否则根本不经过 `globalThis.fetch`。在那条路径上打戳会让字段原样进入 Anthropic 严格 schema(HTTP 400 `prompt_cache_key: Extra inputs are not permitted`)。打戳闸门(`src/agent/pi.ts` 的 `stampPromptCacheKey` → `ompCoworkTransportBypassesFetch`)镜像 OMP 自己的按模型出口选择:证明直发时不打戳,并每进程每模型警告一次。后果:这些直发请求不经压缩、对代理匿名(身份/面板/账单退化 —— 与 #1403 同一取舍);其余 lane(launcher `/bili/` 改写、MITM 白名单、其他 provider/api、配置了代理的宿主)不受影响。已知残留:`NO_PROXY`/`no_proxy` 规则命中模型 host 时,即使设置了 `PI_PROXY` / `PI_PROXY_ANTHROPIC`,OMP 的代理处理也会被静默 —— 请求直发但已打戳(同类 400);这类 host 应从 `NO_PROXY` 中移除而非依赖打戳闸门。反向不对称是 fail-safe 的:OMP 的按形状 fetch 回退(Request 对象入参、非 record headers、不支持的 body)即使没有任何代理环境变量也会走回 `globalThis.fetch`,那里放弃打戳只退化身份、不会硬失败。完整覆盖需要宿主侧变更(Cowork 出口改走 `globalThis.fetch`)—— 作为边界注记记录在 #2583。

Expand Down
132 changes: 126 additions & 6 deletions hermes-plugin/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,115 @@ def add(url: Any) -> None:
return hosts


def _norm_base_url(url: Any) -> str:
"""Lowercase, trailing-slash-stripped form used to compare a configured/hook base URL
against the URL half of a context-cache key."""
u = str(url or "").strip().lower()
return u.rstrip("/")


def _base_url_origin(url: Any) -> str:
"""scheme://host[:port] of a base URL — the cache keeps a path suffix (/v1) that some
configs omit, so a full-URL miss retries at origin granularity."""
u = _norm_base_url(url)
scheme, sep, rest = u.partition("://")
if not sep:
return u
return scheme + "://" + rest.split("/", 1)[0]


def _provider_base_url(provider: Any) -> Optional[str]:
"""The configured base URL of one hermes provider (best effort; same config keys as
mitm_domains_from_config). None when absent/unreadable."""
name = str(provider or "").strip()
if not name:
return None
cfg = hermes_home() / "config.yaml"
try:
data = _yaml_loader()(cfg.read_text(encoding="utf-8"))
except Exception:
return None
if not isinstance(data, dict):
return None

def pick(entry: Any) -> Optional[str]:
if not isinstance(entry, dict):
return None
for key in ("base_url", "url", "api"):
if entry.get(key):
return str(entry[key]).strip()
return None

providers = data.get("providers")
if isinstance(providers, dict):
found = pick(providers.get(name))
if found:
return found
legacy = data.get("custom_providers")
if isinstance(legacy, list):
for entry in legacy:
if str((entry or {}).get("name") or "").strip() == name:
found = pick(entry)
if found:
return found
return None


def _request_base_url(request: Dict[str, Any], context: Dict[str, Any]) -> Optional[str]:
"""Best-effort base URL of an outgoing request: hook/request-supplied fields first,
then the provider entry in config.yaml resolved by the hook's provider name."""
for source in (context, request):
for key in ("base_url", "url"):
v = source.get(key)
if isinstance(v, str) and v.strip():
return v.strip()
return _provider_base_url(context.get("provider"))


def context_window_for(model: Any, base_url: Any = None) -> Optional[int]:
"""#3261: the host's own known context window for a model, read from
HERMES_HOME/context_length_cache.yaml (flat mapping "<model>@<base_url>" -> tokens).
Fail-open to None on any miss/error so the proxy falls back to its normal window chain.
A stage matching several DIFFERENT windows yields nothing rather than a guess: an
overestimate risks preflight overflow, an underestimate over-compresses — both worse
than the default the proxy would otherwise use."""
name = str(model or "").strip()
if not name:
return None
cache_file = hermes_home() / "context_length_cache.yaml"
try:
data = _yaml_loader()(cache_file.read_text(encoding="utf-8"))
except Exception:
return None
if not isinstance(data, dict):
return None
pairs: List[Tuple[str, int]] = []
for key, value in data.items():
k = str(key)
at = k.find("@")
if at <= 0 or k[:at].strip() != name:
continue
if isinstance(value, bool) or not isinstance(value, (int, float)) or value <= 0:
continue
pairs.append((_norm_base_url(k[at + 1 :]), int(value)))
if not pairs:
return None

def sole(values: List[int]) -> Optional[int]:
return values[0] if len(set(values)) == 1 else None

wanted = _norm_base_url(base_url) if base_url else ""
if wanted:
for stage in (
lambda u: u == wanted,
lambda u: _base_url_origin(u) == _base_url_origin(wanted),
):
got = sole([v for u, v in pairs if stage(u)])
if got is not None:
return got
return sole([v for _, v in pairs])


def _spawn_child(sidecar: Dict[str, str], port: int, token: str) -> Optional[subprocess.Popen]:
log_path = os.path.join(tempfile.gettempdir(), f"bili-proxy-{port}.log")
env = {k: v for k, v in os.environ.items() if k not in _PROXY_ENV_KEYS}
Expand Down Expand Up @@ -777,28 +886,35 @@ def on_llm_request(request: Optional[Dict[str, Any]] = None, **context: Any) ->
max_output = _state["max_output"].get(str(session_id))
if isinstance(max_output, (int, float)) and max_output > 0:
merged["x-bili-plugin-max-output"] = str(int(max_output))
window = context_window_for(model, _request_base_url(request, context))
if window is not None:
merged["x-bili-plugin-context-window"] = str(window)
updated = dict(request)
updated["extra_headers"] = merged
return {"request": updated, "source": PLUGIN_ID, "reason": "plugin-mode headers"}
except Exception:
return None


def report_runtime_info(model: str, max_output: Optional[int]) -> None:
def report_runtime_info(model: str, max_output: Optional[int], context_window: Optional[int] = None) -> None:
try:
origin = _state.get("origin")
if not origin:
return
key: Tuple[Any, Any] = (model, int(max_output) if isinstance(max_output, (int, float)) else None)
mo = int(max_output) if isinstance(max_output, (int, float)) else None
cw = int(context_window) if isinstance(context_window, (int, float)) else None
key: Tuple[Any, Any, Any] = (model, mo, cw)
sent: set = _state["runtime_info_sent"]
if key in sent:
return
sent.add(key)
if len(sent) > MAX_TRACKED_SESSIONS:
sent.clear()
body: Dict[str, Any] = {"agent": AGENT_NAME, "model": model[:256], "source": "hermes-native"}
if isinstance(max_output, (int, float)) and max_output > 0:
body["maxOutput"] = int(max_output)
if mo is not None and mo > 0:
body["maxOutput"] = mo
if cw is not None and cw > 0:
body["contextWindow"] = cw

def send() -> None:
http_post_json(origin.rstrip("/") + "/__bili/plugin/runtime-info", body, RUNTIME_INFO_TIMEOUT_S)
Expand All @@ -813,13 +929,17 @@ def on_pre_api_request(*, session_id: str = "", model: str = "", max_tokens: Any
max output tokens for the NEXT request's header, and report runtime info to the proxy."""
try:
session = str(session_id or "")
table: "OrderedDict[str, int]" = _state["max_output"]
if session and isinstance(max_tokens, (int, float)) and max_tokens > 0:
table: "OrderedDict[str, int]" = _state["max_output"]
table[session] = int(max_tokens)
while len(table) > MAX_TRACKED_SESSIONS:
table.popitem(last=False)
if model:
report_runtime_info(str(model)[:256], table.get(session) if session else None)
report_runtime_info(
str(model)[:256],
table.get(session) if session else None,
context_window_for(model, _request_base_url({}, _)),
)
except Exception:
pass

Expand Down
Loading
Loading