Skip to content

feat: stable head-system anchor for prefix-cache preservation (#1085) - #1092

Merged
ranxianglei merged 4 commits into
masterfrom
2026-09-21_stable-system-anchor
Sep 23, 2026
Merged

ranxianglei merged 4 commits into
masterfrom
2026-09-21_stable-system-anchor

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Sep 21, 2026 •

Copy link
Copy Markdown
Owner

Closes #1085

Problem

Every conversation starts with the client's head system block, which typically embeds volatile files (AGENTS.md & co.). When those files change mid-session, the re-sent conversation diverges at byte zero and the upstream provider's KV prefix cache misses in full — the failure mode reported in #1085.

Scope (owner conclusion)

This is a best-effort wire-layer fallback, not the root fix — the root fix belongs client-side (the client owns its history and decides how instruction changes are presented; anomalyco/opencode does this in the client). The proxy can only stop itself from letting a changed head invalidate the whole cached prefix. Per that scoping, anchoring is plain-proxy mode only: plugin-mode agents (x-bili-plugin) own their context management and are never anchored, so clients that already inject cache-friendly updates are not double-processed.

Approach

Freeze each session's first-seen head system/instructions text as a sticky anchor and keep forwarding those bytes verbatim. A detected change is appended instead as a trailing [System context update] … user message; each note composes sequentially onto the previous one.

  • Localized change (file-style edit sharing ≥70% of lines with the version in effect so far): note carries a compact line diff (- removed / + added, 2 lines of context).
  • Non-localized change (structural reshuffle, tool-def churn, timestamped banners, heads over 400 lines): adopted outright — one deliberate cache miss beats appending diff noise that would mislead the model about its instructions.
  • Churn guard: more than 8 accumulated notes likewise replace the anchor with the newest text and clear the log.
  • Anchor + append-only note log live in session.metadata (persisted across restarts, immune to compression/compaction) and are re-injected every turn on all four wire surfaces (anthropic / openai / google / responses), including internal compress-loop rounds.
  • Excluded from anchoring: title-gen micro-requests (OpenAI/Google), Responses compaction-trigger requests, auto-mode classifier requests.
  • On the Responses wire, sessions are keyed on conversation identity alone when anchoring is on — otherwise an instructions change forks into a fresh |sub:<fp> session via kernel subagentNamespace and orphans the anchor state exactly when needed.

What changed

  • src/system-anchor.ts (new): reconcileSystemAnchor() + exported diffLines() (LCS line diff with hunk rendering); constants ANCHOR_MAX_NOTES=8, DIFF_MIN_SHARED=0.7, DIFF_MAX_LINES=400.
  • src/server.ts: per-surface hooks at the system-text extraction points, all gated stableSystemAnchor && !pluginMode; anchor flows into rebuilt payloads, token estimates, and loop rounds automatically.
  • src/config.ts: opt-in stableSystemAnchor (file key) + BILI_STABLE_SYSTEM_ANCHOR env, default OFF, env wins.
  • src/loop/*: adapters accept optional notes, appended as trailing user messages in buildRequest.
  • Tests: tests/system-anchor.test.ts (unit: capture / no-op repeat / localized diff + sequential composition / non-localized adoption / removal / churn guard / persistence round-trip / surface independence / third-party interop zero-injection) and tests/e2e-stable-system-anchor.test.ts (real proxy + mock upstream: OpenAI V1→V2→V2 byte-stable prefix with one diff note; interop session zero-injection; Responses wire lifecycle; anthropic-wire cache_control breakpoint stability — breakpoints ride the same logical blocks across turns and never land on a note, per feat: stable head-system anchor for prefix-cache preservation (#1085) #1092 review; plugin-mode negative: verbatim passthrough, zero notes).
  • Docs: CONFIGURATION.md / CONFIGURATION.zh-CN.md env-table rows.

Known residual limitation

Client-placed cache_control breakpoints may still misalign after a head swap — the proxy cannot move the client's breakpoints; only the head-swap itself is guaranteed stable.

Rebase onto master c9e4572 (2026-09-23, head df499d1)

Rebased per review; four conflicts resolved:

Behavior interaction with #1106 (flagged for the merge decision)

With stableSystemAnchor ON, #1106's persona split for claude-over-Responses (and the codex root lane) is bypassed: an instructions change under one session id no longer forks a |sub:<fp> session — the anchor absorbs it instead (localized change → diff note; non-localized → one deliberate anchor swap = one cache miss). That is the natural consequence of the "drift = same conversation" semantics anchoring assumes; neither PR reviewed this combination explicitly. With the knob OFF (default), #1106 behavior is unchanged.

Pre-flight (rebased onto current master c9e4572)

  • npm run typecheck ✅
  • npm test ✅ 2282 pass / 0 fail / 2 skipped (pre-existing skips)
  • npm run build ✅
  • e2e-codex real-client regression: not run in the agent environment (no upstream reachable); full phase available via the manual-dispatch E2E workflow (repo secrets) — complete before merge per gate.

Wire-shape change behind an off-by-default knob → human review/merge per §7.4.

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-21_stable-system-anchor (df499d1)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1092

Each push to this PR publishes a new version under the pr-1092 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1092.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — verified, one small fix pushed, ready for human merge.

Per the [PR] wake rule I did not redo the work — I pulled the branch, reviewed the full diff against current master, and independently re-ran the pre-flight.

Branch state

  • Exactly 1 commit ahead of origin/master; base 9c01dbe (= current master HEAD, release v0.1.134). Fully rebased — no stale-base risk.

Code verification (beyond CI)

  • Anchor persistence: state lives in session.metadata[stableSystem.<surface>], which src/persist.ts:578 serializes and :634 restores — survives restarts and compaction as claimed.
  • Responses wire: devContent is derived from projection.systemParts after the reconcile (src/server.ts:2857/2869), so the injected developer message inherits the anchored bytes; update notes are appended before the nudge (:2873 vs :2889) so volatile content stays last.
  • OpenAI wire: notes land right after injectOpenaiSystem and before the nudge user message (src/server.ts:2442-2445 vs :2460).
  • Anthropic failure path: when the kernel transform throws, systemOut still equals the anchored parsed.system (initialized pre-try; catch at src/server.ts:2284-2290 keeps notes too) — matches the documented asymmetry.
  • coreToGoogle (acp-kernel/wire) builds a fresh array per call — the adapter's contents.push(notes) cannot alias shared state.
  • The Responses session-keying change (skip subagentNamespace on instruction drift when anchoring is on) follows logically: without it, drift would fork an orphaned |sub:<fp> session and reset the anchor.

Pre-flight re-run on the rebased head

  • npm run typecheck ✅ · npm test ✅ 2172 pass / 0 fail / 2 skipped (pre-existing) · npm run build ✅
  • CI already built billion-context@pr-1092 from 04c39e4 (green).

Issue found & fixed directly on the branch (commit d308362, pushed to 2026-09-21_stable-system-anchor)

  • The docs diff introduced a stray leading space before | on three env-table rows in both CONFIGURATION.md (:512-514) and CONFIGURATION.zh-CN.md (:510-512) — including a needless rewrite of the two pre-existing BILI_SUBAGENT_SPLIT / BILI_FORK_ADOPTION rows (unrelated churn, §7.2 clean-diff rule). Stripped the leading space on all three rows per file; the two pre-existing rows are now byte-identical to master and the new row matches the rest of the table. Docs-only, no behavior impact.

Non-blocking observations (report only)

  • The failure-path asymmetry (anthropic still forwards the anchored system on kernel-transform failure; openai/google/responses forward unchanged) is deliberate and documented in the PR body — acceptable for a rare path.
  • Default OFF + opt-in via file/env with env winning is the right rollout posture for a wire-shape change.

中文摘要:审查了 #1085 的 head-system 锚定 PR(逐路径核对了四条线路的锚点注入、note/nudge 顺序、失败路径与持久化,并在 rebase 后的分支上重跑 typecheck/test/build 全绿);修复了文档中三行表格多余的行首空格(含两行无关行的无谓改动,已恢复与 master 一致并推回分支);功能设计合理、默认关闭,可以合并(按 §7.4,wire-shape 变更需人工合并:https://github.com/ranxianglei/billion-context/pull/1092)。

@BsoBird

BsoBird commented Sep 21, 2026

Copy link
Copy Markdown

I think this feature needs to be taken further. You can refer to headroom's CacheAligner functionality, at: https://github.com/headroomlabs-ai/headroom

doc:https://docs.headroomlabs.ai/docs/cache-optimization

@BsoBird

BsoBird commented Sep 21, 2026

Copy link
Copy Markdown

For example:

  1. Volatile sub-segment detection (the most worthwhile one)
    The 1092 strategy has a hidden risk: what it freezes is the bytes seen the first time. If the client's system prompt embeds content that changes every turn (many clients inject the current time, a random session identifier, memory usage, and the like into system), then the "change" isn't really a semantic change — it's a volatile field refreshing. The consequence of the current implementation is: every turn detects a "change" → every turn appends a [System context update] → within 9 turns the churn guard triggers → the whole anchor is replaced → the cache is wiped, and then the next round starts this cycle all over again.
    Enhancement: do field-level volatile detection on the anchor (regex/structural scanning for patterns like timestamps, UUIDs, monotonically increasing counters), and mask out volatile sub-segments before comparing fingerprints. Genuine semantic changes (AGENTS.md being edited) can still be captured, while volatile refreshes no longer trigger delta appends. This amounts to grafting CacheAligner's "detection" onto 1092's "policy."

  2. Change attribution report
    Right now 1092 only knows "it changed," not "why it changed." You could add an attribution log: when the anchor changes, diff out exactly which substring changed; if it matches a volatile pattern, mark it volatile and prompt the user ("your client injects a timestamp into system every turn — consider turning that off on the client side"). This is more valuable than silently swallowing it — fixing the source beats working around it in the proxy. The bili ecosystem already has the per-request miss-attribution infrastructure from acp_cache, and this report can hang off the same set of statistics.

  3. Coverage extrapolation
    1092 only handles the head system; if volatile content appears in a tool result in the middle of the history (logs carrying timestamps are very common), the prefix cache will break at that position just the same, but that's handled by acp-kernel's fold/absorb. The enhancement direction is to feed volatile-detection results into absorb's priority — large results that contain volatile fields and are otherwise low-value get absorbed first.

At least these three features can be enhanced.

Of course, the specific evaluation requires you to read headroom's code.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 @BsoBird Thanks — before answering I read the headroom source (crates/headroom-proxy/src/cache_stabilization/, the Phase E PR-E1..E6 module) plus the docs page. Verdict on the three points:

1. Volatile sub-segment detection — real gap, but the "graft" is bigger than it looks.

Confirmed the failure mode against #1092's code: a per-turn-volatile system (client injecting a timestamp / session id / memory usage) makes reconcileSystemAnchor() see a distinct change every turn → appends a full-text [System context update] note each turn — and since all accumulated notes are re-injected every turn, the tail grows by ~one full system-text size per turn — → the 9th distinct change trips the churn guard (ANCHOR_MAX_NOTES = 8) → anchor replaced outright → one deliberate miss → the cycle restarts from the freshly-frozen volatile bytes. Steady state = perpetual churn, zero cache benefit, growing note noise. Your diagnosis is correct.

What the headroom code actually offers: volatile_detector.rs (PR-E5) is a pure observer. It scans for three families — ISO-8601 timestamps (byte-position checks, deliberately no regex), UUID v4 (version nibble at pos 14), ID-named JSON fields (request_id/trace_id/session_id/correlation_id substring match) — caps findings at 10/request, truncates samples to 80 chars, emits one structured WARN per finding, and never mutates a byte (explicit invariant; its advice to customers is literally "move per-request IDs/timestamps to message metadata or post-prefix fields"). So there is no masking implementation to port — the pattern taxonomy is portable, the repair logic is new work.

The key design constraint if we build it: masking must gate change detection, not content forwarding. Forwarding fresh volatile values each turn changes nothing — any changed byte at position P invalidates everything downstream of P, and system sits at byte zero. The mechanism is therefore: compare masked fingerprints (volatile spans blanked); masked-equal ⇒ no semantic change ⇒ keep forwarding the frozen anchor bytes, emit no note. Consequence to accept: the model sees a stale timestamp indefinitely, which argues for a bounded refresh (refresh the anchor after N turns / T idle minutes, accepting one deliberate miss) — a product call. And the matcher must be conservative: a false positive that masks a genuine AGENTS.md edit would hide a real semantic change behind the frozen anchor; the safe rule is to suppress a change only when every diff span falls inside a recognized volatile pattern. (headroom bans regex in that codebase for cold-start reasons; bili has no such constraint, so the set can be broader — monotonic counters, hex digests.)

2. Change attribution report — valid, cheap, strong precedent on both sides.

headroom's drift_detector.rs (PR-E6) is exactly this shape: per-session SHA-256 over the cache hot zone (system / tools / first 3 messages, canonicalized — sorted keys, cache_control stripped), prefix-aware comparison (conversation growth into the window is benign, a settled message changing is drift), names the drifted dimensions in cache_drift_observed, LRU-bounded (1000 sessions), secrets hashed before leaving the module, ~6 digests/request (<1ms), log-only on every request. On the bili side the plumbing already exists: [acp-usage] per-round cache-hit telemetry (src/loop/core.ts:221) and the once-per-session collapse warning in src/cache-warn.ts (#499). An anchor-change attribution line (which region moved, how many lines, whether the delta matched a volatile pattern, actionable hint) fits that diagnostics-only posture and costs almost nothing. Two things worth stealing from headroom: fingerprint tools[] alongside system (clients reorder/re-schema tool lists nondeterministically — that's why headroom ships tool_def_normalize), and hash-before-log for secret-shaped values.

3. Feeding volatile detection into absorb priority — premise doesn't hold, I'd push back.

Tool results mid-history are immutable across turns: produced once, re-sent byte-identical by the client. They cannot cause prefix drift — unlike the head system, which the client re-renders fresh each turn. The actual drift sources are: head system (#1092), client-side history rewrites (what #1001 detects), and tool-list/schema changes (client-controlled). Timestamped log content inside a tool result is stable turn-over-turn, so it does zero incremental cache damage beyond what the content already costs. Worse, folding anything mid-history busts the cache at the fold point by construction — that's inherent to compression (compress = a deliberate cache reset of the compressed span, prefix before it preserved). Absorb/fold prioritization serves window pressure, not cache stability; coupling volatile-content scoring into acp-kernel absorb priority mixes those concerns and lands as a cross-repo kernel change (manual/human territory per §7.4). For what it's worth, headroom's own answer to "the prefix went cold" is its cold-prefix recompaction hook (recompact the whole prefix once after TTL lapse) — not volatile-aware absorption either.

Recommendation: ship points 1+2 as one follow-up PR, separate from #1092 — shared volatile-pattern library; masked comparison gates the anchor's change-detection while the frozen anchor keeps flowing; attribution warning rides the existing [acp-usage]/cache-warn channel; bounded anchor refresh as a config knob. Scope it behind stableSystemAnchor or its own off-by-default flag. Point 3: reject — if the goal is cheaper volatile-heavy sessions, the lever is telling the client to stop injecting timestamps into system, which is exactly headroom's own philosophy (detect + advise, fix at the source).

@ranxianglei — this is feature-class, so it's your call per triage rules. Happy to spec/implement the 1+2 follow-up once you confirm direction here or open the issue.

中文摘要:读了 headroom 的 cache_stabilization 源码后确认——其 E5/E6 检测器均为纯观测、绝不改写请求字节(volatile_detector 只报 timestamp/UUID/ID 字段三类模式并建议客户把易变内容移出缓存前缀;drift_detector 按会话对 system/tools/前 3 条消息做 SHA-256 指纹并点名漂移维度),所以"嫁接"的只是模式分类法,修复逻辑需新写;结论:第 1、2 点成立且有双侧先例,建议合并为一个后续 PR(掩码比较仅用于变更判定、冻结锚定字节继续原样转发、归因告警挂现有 [acp-usage]/cache-warn 通道、加有界锚刷新配置项),第 3 点前提不成立(历史中部的 tool 结果跨轮逐字节不变、不会造成前缀漂移,fold 本身就在折叠点破缓存,且属跨仓内核改动),不予采纳。

@BsoBird

BsoBird commented Sep 21, 2026

Copy link
Copy Markdown

Follow-up to #1085 / #1092. Two gaps found while comparing against opencode's cache work (anomalyco/opencode#29672, #43507):

1. Volatile-subfield masking before change detection

reconcileSystemAnchor uses strict byte comparison (system-anchor.ts: "prefix caching is a byte game"). That is right for the forwarded bytes, but wrong for change detection: if the client's head system embeds per-turn volatile values (current timestamp, random session id, runtime stats — all confirmed real in opencode#29672), every turn counts as a "change":

  • a new [System context update] note appended per turn,
  • churn guard (ANCHOR_MAX_NOTES = 8) trips on turn 9 → anchor replaced → one deliberate full prefix-cache miss, then the cycle restarts.

Net effect for volatile-headed clients: a guaranteed full cache bust every 9 turns, plus a growing note tail — strictly worse than no anchoring.

Proposal: compute a masked fingerprint for comparison only (normalize timestamp/UUID/monotonic-counter-shaped substrings to placeholders). outbound still forwards the anchor verbatim — the comment's desync concern doesn't apply since masking never touches what we send. Log volatile hits with attribution so users can fix the client at the source. Semantic changes (an edited AGENTS.md) must still be detected through the mask.

2. Anthropic-wire cache_control regression test

Update notes are appended as trailing user messages — the same position as the opencode plugin notices that broke prompt caching in opencode#43507 (breakpoint landed on a message that can never prefix-match). bili's breakpoint placement is explicit (coreToAnthropic(..., cacheControls)), and the nudge (#451) already rides the same trailing position safely — but the current e2e only exercises the OpenAI chat and Responses wires.

Request: add an e2e on the anthropic wire with a client-sent cache_control breakpoint set + a pending update note — assert forwarded breakpoint positions are unchanged and no breakpoint lands on a note message. This is the one wire where breakpoint semantics actually bind.

Non-goals

  • Reordering the client's system content (opencode's S1/S2 split) — that's the client's job; the proxy forwards client bytes verbatim by design.
  • Replacing the anchor with the newest text on churn — current churn-guard behavior is fine for genuinely churning heads once volatile noise no longer triggers it.

@BsoBird

BsoBird commented Sep 21, 2026

Copy link
Copy Markdown

so what do u think

ranxianglei pushed a commit that referenced this pull request Sep 21, 2026
…otes (#1092)

Session D of the #1085 e2e: a client sending cache_control breakpoints on the system block and an early history user block gets exactly those same logical blocks marked on every turn (identity-keyed carry-over through the kernel round-trip), no breakpoint ever lands on an injected [System context update] note, and steady turns keep the note count at one. Closes the opencode#43507-class regression gap flagged in review.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

`[bot] 🏷 My take, point by point:

1. Volatile-subfield masking — agreed, and your framing is exactly right

Masking for comparison only while outbound keeps forwarding the frozen anchor verbatim is the correct split — you're right that the desync objection doesn't apply, because masking never touches what we send. The prefix stays byte-stable by construction; only the fingerprint used to decide "did it change" gets normalized.

Three design constraints I'd bake into any implementation:

  • Conservative suppression rule: treat a turn as "no change" only when every diff span between incoming and anchor falls inside a recognized volatile pattern. A semantic edit that happens to sit next to a timestamp must still surface as a real change. Anything less strict silently swallows genuine AGENTS.md edits.
  • Pattern taxonomy, not fuzzy scoring: headroom's E5 detector (the one you're pointing at) settled on exactly three families — ISO-8601 timestamps detected by byte-position checks (no regex), UUIDv4, and id-named JSON fields (request_id/trace_id/session_id/correlation_id substring needles). That's a deliberately minimal, fast, fully unit-testable set. Worth porting the taxonomy as-is rather than inventing a broader matcher — broader = more false-suppression risk.
  • Attribution ships in the same PR (your earlier point 2): when a volatile hit suppresses a change, log a warn-once-per-session note naming the matched pattern family — "your client injects timestamps into system every turn; fix it client-side". Diagnostics-only, same posture as bili's 重启/会话分裂后上游 prompt cache 命中掉到 0% —— 每请求全量付费(成本回归) #499 cache-warn detector. Without this, the proxy quietly papers over a client bug forever.

One open product question to flag for @ranxianglei: with masking on, the model sees the frozen volatile values indefinitely (a timestamp from turn 1 on turn 500). Acceptable for most ambient-instruction cases, but a bounded anchor refresh (age/turn cap → one deliberate miss) may want to exist as an escape hatch. My recommendation: don't build it until someone actually hits the staleness problem.

Status: this is feature-class work — I won't start implementing until the owner confirms direction (and opens the issue, per repo rules). If confirmed, I'd scope it as ONE follow-up PR behind stableSystemAnchor (masking + attribution together, shared pattern library), separate from #1092 so this PR stays mergeable on its own.

2. Anthropic-wire cache_control regression test — done, pushed

Added as Session D of tests/e2e-stable-system-anchor.test.ts (commit c0b041c, on the PR branch). A client sends cache_control breakpoints on the system block and an early history user block across three turns (V1 head → V2 head → steady), and the test asserts:

  • breakpoint count is invariant (exactly the two client markers, turns 1–3);
  • the system breakpoint follows the anchored head (turn 2's system text contains V1, not V2);
  • the history-block breakpoint stays on the same logical block (this is the key property — bili's carry-over is identity-keyed via content-hash ids in anthropicToCore/coreToAnthropic, not index-based, so it survives history growth);
  • no breakpoint ever lands on an injected [System context update] note, and the note count stays at one on steady turns.

I also read the mechanism first, and the opencode#43507 class of failure can't occur structurally in bili: notes/nudges are plain string-content user messages appended after the kernel round-trip, and cache_control is re-applied strictly by message identity lookup — new ids get nothing. The test exists to pin that guarantee against future refactors (that's the whole value of this kind of e2e).

Pre-flight on the pushed head: typecheck ✅, full suite 2172 pass / 0 fail / 2 pre-existing skips ✅. Merge stays with a human per §7.4.

Non-goals

Agree with both: reordering client system content is the client's job (bili forwards client bytes verbatim — that's the wire-fidelity invariant), and the churn guard's replace-on-trip behavior is fine once volatile noise stops triggering it.


中文摘要:已把 anthropic 线 cache_control 断点稳定性的回归测试补进 PR 分支(c0b041c,断点按消息身份而非索引保持、永不落在注入的更新备注上,全量测试通过);volatile 子段掩码方案我认可且给出了三条实现约束(保守抑制规则、移植 headroom 三族模式库、归因日志同 PR 交付),但属新功能,需 owner 确认方向并开 issue 后再动手。

@ranxianglei

Copy link
Copy Markdown
Owner Author

审查结论:HOLD 维持(与 PR 自标一致);前提更正 + 两块可拆走的 salvage

深审(base 9c01dbe 对拍、锚机制逐路径走查、9/9 单测 + e2e OpenAI/Responses 复核)后的结论:维持 hold,不合。但封存理由里有一处前提需要更正,且有两块资产值得在重设计时拆走。

前提更正:#1094-hold 的"同一替换点"冲突不属于本 PR

BsoBird 在 #1094 上的 hold 分析(同一替换点、互斥不变式:都在到达时替换 tool-result 字节)针对的是 #1098(pre-crush)vs #1097——不是 #1092。本 PR 的锚替换的是 head-system 字节,不是 tool-result 字节;且 #1097 自己的文本是建在 #1092 的产物之上的:设计约束 4 明说 fetched content 走 trailing request-only message(与 nudge、systemNotes 同通道),约束 1 说走 stableSystem 元数据同持久化路径,风险节说要扩 #1085 follow-up 的 anthropic cache_control 回归测试。锚与 #1097 在精神上是互补的,不是竞争。

真正要为 #1097 后重设计记录的摩擦点:锚 state 在 session.metadata["stableSystem.<surface>"](src/system-anchor.ts:31-33,66)——#1097 约束 1 要统一 content store,先合 #1092 会制造一次迁移;subagentNamespace skip(见下)把锚 state 与 session keying 纠缠,重设计可能重 key;ANCHOR_MAX_NOTES=8 的 churn-guard replace 故意中途改 head 字节(一次全量 cache miss),要对 content-addressed store 不变量重验。

Responses 身份修复(commit 04c39e4,src/server.ts:1350-1366)不能原样拆走

opts.stableSystemAnchor 开启时 Responses session key 跳过 subagentNamespace(<identity>, instructions)。单独拆出去会在无锚的前提下把真不同人格(同 conversation id、不同 instructions)并进一个 session/fold 空间——正是 master 上 instructions 指纹要防的。它修复的真 bug(opencode system-context reconcile:每次 AGENTS.md 编辑都重置压缩状态——master 今天就存在)值得独立 issue + 独立设计(如把 trusted-codexTurn 待遇扩展到提供原生 turn/conversation 标记的客户端,或加 config gate)。

值得无条件拆走

  1. c0b041c 的 anthropic cache_control 断点稳定覆盖(tests/e2e-stable-system-anchor.test.ts):Feature: content-addressed message store — ID-referenced tool results + acp_retrieve (built-in CCR) #1097 明说要扩这个测试;抽出来(断言无锚时的字节稳定转发)成本低、直接解锁 Feature: content-addressed message store — ID-referenced tool results + acp_retrieve (built-in CCR) #1097 的风险项。
  2. Prepared.systemNotes 通道形状(+ pickAdapter(..., systemNotes) 穿线,src/loop/index.ts:29-42、各 adapter +7):Feature: content-addressed message store — ID-referenced tool results + acp_retrieve (built-in CCR) #1097 约束 4 要的正是这个通道做 acp_retrieve 注入。保留形状、丢弃锚语义。

锚机制轻审备注(供重设计)

  • plugin 模式无 !pluginMode 守卫(src/server.ts:2172-2176, 2338-2378, 2561-2586, 2757-2806)——notes+冻结锚在 plugin 模式同样生效而 tool/nudge 注入被抑制;wire 一致(notes 每轮重注入)但相对 nudge 的不对称缺测试 + README "Two compression modes" 缺一行(AGENTS §6)。
  • 失败路径 note 追加安全(rebuiltMessages 在 try 前从 parsed.messages 初始化)。
  • 豁免清单在代码里核实:classifier passthrough(:2157)、compaction trigger(:2800-2806)、title-gen(:2375-2380)。
  • 单测 9/9 过;e2e 覆盖 OpenAI+Responses,anthropic/google 的 plugin-mode 车道未覆盖。

另:上方 BsoBird 提到的 headroom CacheAligner 思路与本 PR 的锚目标(head 字节稳定)同族,重设计时可一并参考。

@ranxianglei

Copy link
Copy Markdown
Owner Author

@ework — please re-review the current head c0b041c in full. Additional context since your last pass: an independent deep review is at #1092 (comment) (verdict: hold maintained, with a premise correction re #1098-vs-#1097 and a salvage list; the instructions-drift bug is now #1102). Note BsoBird's CacheAligner suggestions above are also worth folding into your assessment.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Full re-review of head c0b041c`, incorporating the independent deep review and BsoBird's CacheAligner thread. Verdict: agree with HOLD — and I executed salvage item 1 (PR #1103).

Scope of this pass

c0b041c = 04c39e4 (feature, byte-identical to my round-1 review) + d308362 (docs whitespace fix, mine) + c0b041c (anthropic cc e2e, mine). No feature code changed since round 1. Pre-flight re-run on this exact head: typecheck ✅, 2172 pass / 0 fail / 2 pre-existing skips ✅; GitHub CI green on c0b041c (both checks incl. the real-codex zero-token e2e) ✅.

Independent verification of the deep review's claims

I checked each one against the code rather than taking them on trust:

  • subagentNamespace skip — confirmed src/server.ts:1353-1364: with stableSystemAnchor on, Responses keying takes the verbatim conversation identity and skips subagentNamespace(identity, instructions). Agreed it cannot be carved out: without the anchor absorbing instruction drift, same-conversation-id/different-instructions personas would collapse into one session/fold space — precisely what master's fingerprint exists to prevent. [Bug] opencode system-context reconcile:每次 AGENTS.md 编辑使 Responses 会话 key 里的 instructions 指纹漂移,压缩状态被重置(master 现存) #1102 now tracks the underlying bug (opencode reconcile resets compression state on AGENTS.md edits — live on master today), so nothing further is owed from this thread.
  • Anchor state location & friction points — confirmed session.metadata["stableSystem.<surface>"] (src/system-anchor.ts:38-40 key, :57-59 write; ANCHOR_MAX_NOTES = 8 at :33). All three recorded friction points stand: the unified content store (Feature: content-addressed message store — ID-referenced tool results + acp_retrieve (built-in CCR) #1097 constraint 1) would force a migration if feat: stable head-system anchor for prefix-cache preservation (#1085) #1092 lands first; the keying entanglement may be re-keyed in redesign; the churn-guard replace deliberately mutates head bytes mid-session (one full miss) and needs re-validation against content-addressed-store invariants.
  • Plugin-mode asymmetry — confirmed: all four hooks gate on opts.stableSystemAnchor only (src/server.ts:2173 anthropic, :2376 openai + !isTitleGen, :2584 google + !isTitleGen, :2803 responses + !isCompactionTrigger) — none reference pluginMode. So frozen anchor + per-turn note re-injection is active in plugin mode while tool/nudge injection is suppressed. Wire-consistent, but the nudge asymmetry is untested and the README "Two compression modes" section (README.md:76) has no anchor mention (grep-confirmed) — AGENTS §6 requires reasoning about both modes. Both go on the redesign checklist.
  • Exclusion list — classifier passthrough :2155-2158, compaction-trigger :2800-2806, title-gen :2375-2380: all present as claimed. Failure-path note appending safe (rebuiltMessages initialized before the try) — confirmed in round 1.
  • Test status — unit 9/9 re-run on c0b041c: pass ✅. Lanes: OpenAI + Responses (original), Anthropic added by my c0b041c commit (now also asserts no breakpoint lands on a note with anchoring active); Google lane remains uncovered — noted.

Nothing in this pass contradicts the deep review or changes the verdict. The premise correction (#1094-hold's replacement-point conflict targets #1098-vs-#1097, not this PR) is accepted — the anchor replaces head-system bytes, not tool-result bytes, and #1097 is built on top of this PR's artifacts, so complementary-not-competing holds.

Salvage status

  1. Done → PR test: anthropic-wire cache_control breakpoint stability across turns #1103. The anchor-free half of the coverage, off master, test-only (55 lines in tests/e2e-anthropic.test.ts): client-sent breakpoints on system + early history block ride the same logical blocks across turns under a legitimately changing head and growing history — identity-keyed, count-invariant, no migration to newer blocks. Passes on master today (typecheck ✅, 2163/0/2 ✅), so it merges independently of the held PR and delivers exactly the coverage Feature: content-addressed message store — ID-referenced tool results + acp_retrieve (built-in CCR) #1097's risk section names.
  2. Recorded, not shipped. Keeping the Prepared.systemNotes + pickAdapter(..., systemNotes) channel shape for acp_retrieve injection (discard anchor semantics) is the right redesign constraint — but shipping it alone would be dead plumbing (no producer without the anchor), so it stays in the brief, not in a PR.

Folding in the CacheAligner/volatile-masking thread

Under HOLD this becomes part of the redesign brief instead of a follow-up against this PR. My prior constraints stand: conservative suppression rule (suppress a turn only when every diff span falls inside a recognized volatile pattern), port headroom E5's minimal 3-family taxonomy (ISO-8601 byte-position timestamps / UUIDv4 / id-named fields — no regex, no fuzzy scoring), attribution warn-once log in the same change. One interaction worth recording for the redesign: masking makes churn-guard trips rare in practice, which simultaneously mitigates the churn-replace full-miss concern above — volatile masking and the ANCHOR_MAX_NOTES re-validation are coupled design inputs, not independent items.

PR remains held per your verdict; merge is human-only regardless (§7.4 wire-shape change).


中文摘要:已对 c0b041c 完成完整复审(功能代码与首轮一致、CI 绿、单测 9/9 复跑通过),并逐条独立核实了深审的全部论断(subagentNamespace 纠缠、metadata 存储位置、plugin 模式缺守卫、豁免清单、失败路径安全)——均属实,结论同意维持 HOLD;salvage 第一项已落地为独立 PR #1103(无锚版 anthropic 断点稳定性 e2e,master 上直接通过,解锁 #1097 风险项),第二项(systemNotes 通道形状)记入重设计要点、不单独发死代码。

antclt pushed a commit to antclt/billion-context that referenced this pull request Sep 22, 2026
Anchor-free e2e salvaged from held PR ranxianglei#1092 per review direction: client-sent cache_control breakpoints (system block + early history block) must ride the same logical blocks on every turn — identity-keyed carry-over through the kernel round-trip, invariant under legitimate head changes and history growth, never migrating to newer blocks. Prerequisite coverage named in ranxianglei#1097's risk section.
@ranxianglei

Copy link
Copy Markdown
Owner Author

独立审查结论:approve(rebase 后合并,wire-shape 变更按 §7.4 仍需人工)。 我把 PR head 1eb4ade rebase 到当前 master c9e4572(已含 #1084)做了完整验证。

语义审查(通过)

  • 门控完整:默认关(env BILI_STABLE_SYSTEM_ANCHOR wins / file stableSystemAnchor:true,其余一律 false);四个 prepare 面全部 opts.stableSystemAnchor && !pluginMode;session-id 巷道再加 pluginAgentHeader === undefined。title-gen、compaction-trigger、auto-classifier 请求全部排除。
  • 注入顺序正确:history → anchor notes(stable,逐轮原样重注入)→ nudge(volatile)——stable-before-volatile,前缀最大化;四面一致,loop 重建路径也带 notes。
  • wire-ephemeral 纪律:notes 如 nudge(0.1.69 回归: plugin 模式 nudge 注入被 !pluginMode 静默禁用 — INJECT 日志照打但从不投递, 自动压缩只剩 preflight 应急 #451)从不进 kernel fold space、不占 message refs;anchor+log 存 session.metadata.stableSystem.<surface>(重启/压缩免疫)。这与 proxy 模式 acp_summary re-voice(user,锚位)互不干扰——两模式都推演过,无 [Bug] ACP: ≥2 compressed blocks render as multiple system messages → SGLang (vLLM-compatible) upstreams return 400 #377 类冲突。
  • 保守方向正确:localized 门槛(≥70% 共享行、≤400 行)LCS 实现边界安全(400² Uint16);非局部/超限/churn>8 一律有意 miss 换锚,宁可 miss 不污染——正好是 owner 定位("误判宁可 miss,不可污染")的实现。head 移除→REMOVED_NOTE 局部处理,不断缓存。
  • 第三方 interop:自带 cache-friendly 更新的客户端零额外注入(有单测+e2e)。

⚠️ rebase 实操(ework 必须处理)

  1. 真语义三方冲突 src/server.ts:~1404(Responses 巷道 session-id):master 在 PR fork 后落了 Responses instructions fingerprint should be an allowlist (codex/claude only), not a default — generic relay clients fork on every system-prompt change #1106 instructionsFingerprintApplies 反向白名单(codex/claude-over-Responses 才指纹分家)。正确解是嵌套并集:anchor 开 → verbatim(锚定吸收漂移);anchor 关 → Responses instructions fingerprint should be an allowlist (codex/claude only), not a default — generic relay clients fork on every system-prompt change #1106 门控照旧。我本地已按此解并全量验证通过,直接 rebase 会掉语义。
  2. prepareAnthropic Prepared:master 的 imageBillingFor 第三参 与 PR 的 systemNotes 并存(union)。
  3. compress-loop systemPrompt:master 的 withConversationIdNote 包裹 与 PR 的 pickAdapter 第 10 参 systemNotes 并存(union)。
  4. config.ts / CONFIGURATION 两文件:相邻字段/表格行 union,注意去重(机械并集会留双份)。

修复后验证(rebased head 21bcc11,含上述 4 处解冲突)

typecheck ✅;全量 2282 pass / 0 fail / 2 skip ✅;build ✅。

需 owner 知悉的交互(非阻塞,默认关可控)

anchor 开启时 #1106 的 claude-over-Responses 人格分家会被旁路:同一 session-id 的人格切换不再拆 sub:<fp> 会话,而是走 diff 注记(非局部则换锚=一次 miss)。这是 "漂移=同一对话" 语义的自然推论,但两个 PR 都没显式评审过这个组合,合并前请确认这是你想要的。codex root 线同理。

另:e2e-codex 真客户端回归在 ework 环境因上游未起没跑——合并前按门禁补跑(CI 或本地)。

ework-agent added 4 commits September 23, 2026 14:44
When a client's head system/instructions block changes mid-session (e.g. an embedded AGENTS.md is edited), the forwarded prefix diverges at byte zero and the provider's KV prefix cache misses in full. With stableSystemAnchor enabled (default off; env BILI_STABLE_SYSTEM_ANCHOR=1 or config file), bili freezes each session's first-seen head-system text as a sticky anchor and re-sends those bytes verbatim; detected changes are appended as trailing '[System context update]' user messages carrying the new full text, each superseding the previous ones. The anchor and append-only log live in session metadata (persisted, compaction-proof) and are re-injected every turn across all four wire surfaces (anthropic/openai/google/responses) including internal compress-loop rounds. Churn guard: more than 8 distinct changes replace the anchor outright (one deliberate cache miss). Excluded: title-gen micro-requests, Responses compaction-trigger requests, auto-mode classifier requests. Clients implementing their own variant of this mechanism need no extra injection — their in-history updates pass through as ordinary history (covered by test). On the responses wire, anchoring keys sessions on conversation identity alone so instruction drift no longer forks into orphaned |sub:<fp> sessions.
…otes (#1092)

Session D of the #1085 e2e: a client sending cache_control breakpoints on the system block and an early history user block gets exactly those same logical blocks marked on every turn (identity-keyed carry-over through the kernel round-trip), no breakpoint ever lands on an injected [System context update] note, and steady turns keep the note count at one. Closes the opencode#43507-class regression gap flagged in review.
Owner conclusion: proxy-side anchoring is a best-effort wire-layer fallback, not the root fix (that belongs client-side). Re-scoped accordingly:

- Plain-proxy mode only: plugin-mode agents (x-bili-plugin) own their context management and are never anchored — avoids double-processing clients that already inject cache-friendly updates (claude-code style). Gates in prepareAnthropic/prepareOpenai/prepareGoogle/prepareResponses plus the Responses identity-derivation site.

- Diff-based notes instead of full-text resend: localized head changes (file-style edits sharing >=70% of lines, LCS ratio) append a compact line diff (- removed / + added, 2-line context) against the version in effect so far; each note composes sequentially onto the previous one.

- Non-localized changes (structural reshuffle, tool-def churn, timestamped banners, heads >400 lines) are adopted outright: one deliberate cache miss beats appending noise that would mislead the model about its instructions.

- Removal keeps flowing as the anchor with a removal note (previously fell into the replace path).

- Tests: unit suite rewritten for diff semantics (hunk format, sequential composition, non-localized adoption, churn cap, interop zero-injection); e2e heads are now multi-line AGENTS.md-style texts differing in one line, plus a plugin-mode negative session asserting verbatim passthrough.

- Docs: CONFIGURATION.md/.zh-CN.md rows updated with the fallback positioning, mode restriction, eligibility thresholds, and the cache_control residual limitation.
@ranxianglei
ranxianglei force-pushed the 2026-09-21_stable-system-anchor branch from 1eb4ade to df499d1 Compare September 23, 2026 06:52
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Rebase completed — branch 2026-09-21_stable-system-anchor now at df499d14 on master c9e4572e, fully verified.

What was done

Rebased the 4-commit branch onto current master (c9e4572e, incl. #1084 session-file GC, #1170/#1172) and resolved all four predicted conflicts per your spec:

  1. Responses-lane session-id (src/server.ts:~1404) — nested union: opts.stableSystemAnchor && pluginAgentHeader(req.headers) === undefined → verbatim identity (anchor absorbs drift); otherwise Responses instructions fingerprint should be an allowlist (codex/claude only), not a default — generic relay clients fork on every system-prompt change #1106's instructionsFingerprintApplies gating runs exactly as on master. Both comments preserved ([Feature] Optimize the cache by reorganizing the context. #1085 anchor rationale + Responses instructions fingerprint should be an allowlist (codex/claude only), not a default — generic relay clients fork on every system-prompt change #1106 allowlist rationale). Note: this hunk conflicted twice — once at commit 1/4 (original skip) and again at 4/4, where the rescope commit layered the pluginAgentHeader === undefined restriction onto my commit-1 union; final shape = nested union + plugin-lane guard, as specified.
  2. prepareAnthropic Prepared — union: master's imageBillingFor(opts, upstreamOrigin) third arg kept alongside this PR's systemNotes.
  3. Compress-loop systemPrompt — union: master's withConversationIdNote(…) wrap kept alongside this PR's pickAdapter(…, systemNotes) 10th arg.
  4. config.ts / both CONFIGURATION docs — adjacent fields/table rows unioned and deduplicated; env table carries SUBAGENT_SPLIT / FORK_ADOPTION / STABLE_SYSTEM_ANCHOR / CHAIN_CONTENT exactly once each, no leading-space residue (d308362 applied cleanly on top).

Verified in the committed tree: zero conflict markers, hunk shape at src/server.ts:1404 matches spec.

Verification (rebased head df499d14)

  • npm run typecheck ✅
  • npm test ✅ 2282 pass / 0 fail / 2 skipped — identical to your local rebased numbers
  • npm run build ✅
  • e2e-codex real-client regression: not run here (no upstream reachable from this environment). Per the gate, please run the manual-dispatch E2E workflow (repo secrets) or locally before merge.

PR body updated with the rebase section, conflict resolutions, and the interaction disclosure below.

⚠️ Behavior interaction needing your confirmation (old → new)

With stableSystemAnchor=ON: claude-over-Responses persona switches under one session id no longer fork |sub:<fp> sessions (old behavior per #1106) — instruction drift is absorbed by the anchor instead (localized change → diff note; non-localized → anchor swap = one deliberate cache miss). Codex root lane likewise. Why: anchoring assumes "drift = same conversation", so fingerprint-forking would orphan the anchor state exactly when it is needed. With the knob OFF (default) #1106 behavior is unchanged byte-for-byte. Neither PR reviewed this combination explicitly — please confirm this is what you want before merging; if not, it is a one-line condition change (keep fingerprinting active for persona-split clients even under anchoring).

Merge stays human-only per §7.4 (wire-shape change behind an off-by-default knob).

中文摘要:已把 PR rebase 到最新 master c9e4572,四处冲突全部按你的方案解决(Responses 巷道 session-id 与 #1106 门控做嵌套并集,其余为字段并集+文档去重),新 head df499d1 上 typecheck/build 全绿、测试 2282 pass / 0 fail / 2 skip(与你本地数字一致);e2e-codex 真客户端回归因本环境无上游未跑,请合并前补跑;另标记一个需你确认的行为交互——锚定开启时 #1106 的 claude-over-Responses 人格分家被旁路(漂移改由锚点吸收,非局部则一次有意缓存失效),关闭时零变化——请合并前确认是否符合预期。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Addendum: CI on df499d14 is fully green — all 8 checks passed (test ×4 across ubuntu-22/ubuntu-24/windows-22/windows-24, real-codex zero-token preflight, codex MCP registration, build-artifact, version-guard). Remaining pre-merge items are the two flagged in my previous comment: your confirmation of the #1106 persona-split bypass interaction, and the full e2e-codex regression (manual-dispatch E2E workflow or local, since no upstream is reachable here). Merge itself stays with you per §7.4.

@ranxianglei
ranxianglei merged commit f350307 into master Sep 23, 2026
8 checks passed
@ranxianglei
ranxianglei deleted the 2026-09-21_stable-system-anchor branch October 15, 2026 05:32
cskwork pushed a commit to cskwork/billion-context that referenced this pull request Oct 21, 2026
cskwork pushed a commit to cskwork/billion-context that referenced this pull request Oct 21, 2026
…otes (ranxianglei#1092)

Session D of the ranxianglei#1085 e2e: a client sending cache_control breakpoints on the system block and an early history user block gets exactly those same logical blocks marked on every turn (identity-keyed carry-over through the kernel round-trip), no breakpoint ever lands on an injected [System context update] note, and steady turns keep the note count at one. Closes the opencode#43507-class regression gap flagged in review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Optimize the cache by reorganizing the context.

2 participants