Skip to content

fix(#2592): sniff the unattributed raw-URL branch for the dsh compaction envelope - #2763

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-12_raw-url-compaction-sniff
Oct 11, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-12_raw-url-compaction-sniff

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

Closes the last lane an agentless dsh compaction call could land through (#2592 field-report shape): the native fetch patch's #1117 unattributed-raw-URL branch.

Leak chain being closed: dsh background/manual compaction is agentless (no initiator attribution) → takeoverGate refuses it → without the settings overlay nothing routed the URL to /bili/ → the fetch sent direct to the provider → the checkpoint landed and destroyed the compression substrate. Previously only the two proxy-side lanes were guarded (pipeline #1835 + passthrough #2709 — "45 refused, 4 landed" report), and after landing only the post-hoc rebase floor (#2742, just merged) contained the damage.

Change (client-side divert, server stays decisive)

In src/agent/native-intercept.ts, the unattributed-raw-URL branch now body-sniffs POSTs for the dsh compaction directive (client mirror of DSH_COMPACTION_MARKER_SNIFF — first 48 ASCII chars of DSH_COMPACTION_INSTRUCTION_PREFIX) and diverts only marker hits into the proxy's unattributed passthrough lane, stamped x-bili-passthrough: 1 exactly like the overlay-routed case. The #2709 guard there runs the marker-decisive last-user-message check:

Fail-open discipline (unchanged #1117 semantics for everything else):

  • plain traffic (no marker) → direct send, byte-identical to before;
  • body forms that cannot be read without disturbing the dispatch (ReadableStream, FormData) → direct send, fix(#2592): pin the rebase ref floor + name the rebase in stale-ref receipts #2742 backstops a landing;
  • proxy not ready → direct send;
  • over-match (a conversation merely quoting the directive) → one loopback hop, the server-side decisive check relays verbatim — no false positive.

Request-object inputs are probed through a clone so the original stays sendable (#2496). init.body wins over the Request's own body (spec order, mirrors fetchMethodOf).

Tests

Together with #2709 (both proxy lanes), #2658/#2474 (auto-compaction off), #2622-lineage routing and #2742 (rebase floor + receipts), dsh compaction now has no unguarded path to a provider while bili is alive.

Refs #2592, closes the residual leak tracked there. Follow-up: none known.

@github-actions

github-actions Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-12_raw-url-compaction-sniff (62f41f7)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3039-pr.2763.bc08d66

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2763.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Reviewing PR #2763 now — reading the original issue #2592, the touched paths in src/agent/native-intercept.ts, the #2709 passthrough-lane wiring, and running the targeted suites. Full structured conclusion to follow in this thread.

ework-agent and others added 2 commits October 11, 2026 18:46
…ion envelope

The #1117 unattributed-raw-URL branch was the last lane an agentless dsh
compaction call could LAND through: no settings overlay routes it, the
takeover gate refuses attribution, so the fetch sent direct to the
provider (the 4/49 field-report leak shape; only the post-hoc rebase
floor #2742 caught it afterwards).

Divert ONLY marker-sniffed POSTs into the proxy's unattributed
passthrough lane (same x-bili-passthrough stamp the overlay-routed case
uses), where the #2709 guard runs the marker-decisive last-user-message
check: refused (403, dsh logs 'step compaction failed' and continues the
turn) or, with allowDshCompaction (#2028), relayed verbatim by explicit
user decision.

- client prefilter mirrors DSH_COMPACTION_MARKER_SNIFF (first 48 chars
  of the directive); over-matches cost one loopback hop, the server
  stays decisive
- body forms: string / URLSearchParams / Uint8Array / ArrayBuffer /
  Blob inspected; Request-object input probed via clone (#2496-safe);
  ReadableStream/FormData fail OPEN to today's direct send (#2742
  backstops)
- dead proxy (origin not ready) fails open to direct send

Verified live end-to-end: unattributed raw-URL envelope -> 403
dsh_compaction_refused with ZERO bytes reaching the upstream; ordinary
unattributed traffic unaffected (200, verbatim at the upstream).
…rror instead of re-deriving slice(0, 48)

Review finding on #2763: the client-side fetch patch re-derived the same 48-byte dsh compaction directive prefix locally (DSH_COMPACTION_INSTRUCTION_PREFIX.slice(0, 48)). If the server sniff length ever changed, the two mirrors would silently diverge (client stops diverting what the server would refuse). Export the server constant and import it — no semantic change. Verified: tsc clean, unused-export gate clean, native-intercept + dsh-compaction guard/#2709/#2658 suites 111/111, build bundles the marker into dist/agent/dsh-native.js.
@ranxianglei
ranxianglei force-pushed the 2026-10-12_raw-url-compaction-sniff branch from a2a6e15 to 62f41f7 Compare October 11, 2026 10:49
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review 完成 — 结论:可合(merge 由人执行)

审核过程:先对原 issue #2592(机制族 F1–F5)做方向关——本 PR 关闭的是该族最后一个"compaction checkpoint 能落地到 provider"的通道(client fetch patch 的 #1117 unattributed-raw-URL 分支),与 #2709(passthrough 道)、#1835(pipeline 道)、#2742(rebase floor 兜底)共同封死;方向正确,修的是机制(未归因通道缺少内容判定)而非症状。逐行读了两个文件的 diff + 周边 dispatch 路径(takeoverGate、apig-resign 早退、routed 分支、正常 rewrite 路径),两种压缩模式都推演过:passthrough lane 在 session 之前、plugin/proxy 模式均不受影响。测试有效性:7 个新 case 断言有牙(divert sink URL、x-bili-passthrough stamp、dispatch action、各 fail-open 形态、bodyUsed === false)。

结构化八问:

  1. 是什么问题:dsh 无 agent 的背景/手动 compaction 调用若走裸 provider URL(无 settings overlay 路由、无 initiator 归因),会绕过 bili 直连 provider——checkpoint 落地后销毁 ACP ref 压缩底座(现场报告 49 次中 4 次落地)。
  2. 什么原因导致的:「BUG」native-intercept 进程级 globalThis.fetch 补丁按 URL 形状接管所有模型流量,无关调用者(如 MemOS)被卷入宿主压缩流程 #1117 的设计决策是"URL 形状不能认领请求,未归因者保留原 URL 直连"——这个 fail-open 设计早于"dsh 自身 compaction 也可能未归因且走裸 URL"的发现,于是安全关键的一类请求静默继承了直连语义;此前只有 proxy 侧两道被守。
  3. 影响范围与严重程度:影响 market/profile 安装中 overlay 未生效/未收敛的 dsh 宿主;数据损坏级(ref 永久不可解析 → circuit breaker),非全面不可用;触发条件=未归因+裸 URL+marker body 三者同时成立。
  4. 怎么修复的:client-side divert、server stays decisive——在未归因分支里对 POST body 做 marker 预嗅(与服务端 DSH_COMPACTION_MARKER_SNIFF 同一组字节),仅命中者改写进 loopback /bili/<url> passthrough lane(与 overlay 路由情形同款 stamp),由既有 [Bug] #1001 归因缺口:客户端原生压缩落地后只剩「未确认档」——bili 已记录的 17 次拒绝(dshCompactionRefusals)与 10 次 DIRECT 旁路未被用于归因 #2709 marker-decisive 检查做最终裁决:拒(403,dsh 记日志继续回合)或 allowDshCompaction 下逐字节转发。其余流量保持 「BUG」native-intercept 进程级 globalThis.fetch 补丁按 URL 形状接管所有模型流量,无关调用者(如 MemOS)被卷入宿主压缩流程 #1117 原语义;Request 输入经 clone 探测([omp-native] fetch recovery that lands on the same healthy origin is treated as give-up: BILLION_CONTEXT_PROXY is deleted for the whole process #2496),init.body 按 spec 序优先。
  5. 是否回归问题:审过的既有行为逐项——非 marker 未归因流量 byte-identical 直连(有测试钉住);有归因流量代码路径未触碰;签名 APIG 流量在新分支之前早退、不变。两处有意代价已披露:①每个未归因 model-URL POST 现在付一次全 body 扫描(string 扫描廉价;Uint8Array/Blob/Request-clone 有 ~1× body 瞬时分配);②marker 命中路径没有 attributed 路径的 TypeError→recover→retry 兜底,loopback 中途死亡时 dsh 收到网络错误而非落地 checkpoint——更安全方向,dsh 对两者恢复行为相同。未发现语义漂移/fail-fast 变吞错/阈值时序输出格式变化。
  6. 是否破坏缓存:不涉及 session 状态、ref 空间、prefix cache、重试/去重键;pasethrough lane 不碰 session。
  7. 是否改了设计方向:不转向——把既有的"client 预筛 + server 裁决"两级防线扩到第三条道;client sniff 按构造 non-decisive(over-match 只多一跳 loopback,服务端逐字节转发,无误报伤害)。
  8. 是否引入新配置:无。复用 在bili网页配置界面支持关闭和开启dsh的自动压缩功能 #2028 allowDshCompaction,无新 env/config/表面。

本地验证(rebased head):分支原落后 master 46 个提交,已 rebase 到当前 master f3d8c4e36(clean,期间 master 仅改了 handle.ts 的 durable-guard 区域,与本 PR 零交互);native-intercept 54/54,dsh-compaction guard + #2709 两 lane + #2658/#2432/#2451/#2028 + fix-1117 共 57/57(合计 111/111),tsc clean,unused-export gate clean,build 绿且 marker 确认打包进 dist/agent/dsh-native.js。说明一点:旧 base 上本地跑 #2658 有两条腿红(4 !== 3)——纯 stale-base 假象,master 已在 897541933(#2747)把 rig 计数对齐 #2742 的 ref-floor pin(CI 跑 merge-ref 所以一直绿),rebase 后本地转绿。旧 head 的 CI 全绿(test matrix ubuntu/windows 22+24、全部 e2e lanes 含 real-dsh-headless、各 guard);PR 提到的 opencode-native 单测失败系其本地环境特有,CI 矩阵干净,不构成阻塞。

顺手小修(已推,labeled commit 62f41f702):review 发现 client 侧本地重算 PREFIX.slice(0, 48) 与服务端私有常量构成镜像漂移隐患(将来服务端改嗅探长度,client 会悄悄漏拦)。已把 DSH_COMPACTION_MARKER_SNIFF 导出并在 native-intercept 直接 import,去掉重算;无语义变化,上述验证即在该 commit 上通过。

可选加固(不阻塞):live full-chain 证明(真 startServer + fake upstream:envelope→403、upstream 零字节)是 ad-hoc 跑的,未落成常驻 e2e test;两端各有 unit 层钉住,组合是直通管道,可接受。

新 head 62f41f702 的 CI 正在跑,全量矩阵以 CI 为准。分支现已与 master 齐平,mergeable 状态稍后即转 clean。建议合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 补充:CI 全绿,状态更新

新 head 62f41f702 的 CI 已跑完:28/28 全部 success(含 4 个全量 test matrix ubuntu/windows 22+24、全部 real-client e2e lanes、kernel/pi-subagents/unused-exports/config-surface/version 各 guard)。

期间 master 又前进了一个大批次(#2690 unified SQLite storage + #2765 v0.1.192 元数据同步,64 文件 +6088/-44),PR 因此显示 BEHIND。已核查:该批次与本 PR 触碰的 native-intercept.ts / dsh-compaction-guard.ts / passthrough lane 零文件重叠,git merge-tree 无冲突——BEHIND 仅为版本窗口的正常漂移,不构成阻塞(不追移动中的 target;v0.1.192 release 流程进行中,master 可能还会动)。

结论不变:可合,merge 由人执行。#2763

@ranxianglei
ranxianglei merged commit 4d13c4a into master Oct 11, 2026
28 checks passed
@ranxianglei ranxianglei mentioned this pull request Oct 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.284-master.4d13c4a (rolling master tag).

npm install -g billion-context@0.0.284-master.4d13c4a  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant