Skip to content

Feature: tool-result valuator + deterministic compression plugin interface (two-tier absorb gate in kernel) #354

Closed

Description

@ranxianglei

来源: ranxianglei/billion-context#1094 分析 pre-crush 通道时发现 —— owner 分层裁定(2026-09-22):估价归内核、压缩方式开放为插件接口,内核先行。

Feature: tool-result valuator + deterministic compression plugin interface (two-tier absorb gate in kernel)

Filed per owner ruling on billion-context#1094 (2026-09-22):

内核负责估价 —— token/成本估算、是否超阈、路径决策(先确定性 crush,仍超阈才走模型蒸馏的两层门)在内核做;压缩方式开放为插件接口 —— 内核只定义接口,提供一些默认实现(JSON 字段方差统计、源码 AST 级裁剪这类确定性压缩作为默认插件)。配置开关 + 默认关闭。

Motivation

bili's absorb path makes the size decision host-side today (per-turn countTokens + minToolTokens/contextThresholdPct in bili's applyAbsorbView), and a deterministic pre-crush prototype (billion-context PR #1098, currently blocked) runs host-side in front of it. Problems with host-side ownership:

  1. The two-tier gate (deterministic crush first; model distillation only if still over threshold) must be atomic with the kernel's absorb hide/prompt/adjudication nodes — a host-side pre-pass races the prompt decision (bili had to strip/re-bake the [ACP absorb] marker around the crush).
  2. Both compression modes (proxy vs plugin) and any future host must get identical decisions; duplicating valuation per host drifts.
  3. The deterministic strategies are payload-pure and host-agnostic — they belong where the message lifecycle already lives.

Coordinate with billion-context#1097 (content-addressed store + acp_retrieve): agreed precedence is ID-reference wins for stored results — the valuator then sees placeholder size (small) and naturally skips them; the plugin interface must be callable from BOTH the arrival path and the retrieve path.

Proposal

1. Valuator — kernel-owned, pure, deterministic

  • Input: tool-result text + absorb config (+ context-pressure signal). Output: a decision — skip (below threshold) · crushed-done (deterministic compression brought it under threshold; no model round-trip) · distill (still over → existing [ACP absorb] prompt path, optionally fed the crushed text).
  • Reuses existing estimation (defaultCountTokens) and minToolTokens/contextThresholdPct semantics; adds a minimum-reduction floor (minReduction, suggested default 0.1) so marginal savings don't alter wire bytes.
  • Same payload + config ⇒ same decision, always (bili resends history every turn; prefix-cache stability depends on byte-stable outputs).
  • Integration point: the absorb node chain in processTurn (where appendAbsorbPrompts/hideAbsorbedMessages run), so hide/prompt/ref bookkeeping stays coherent; hosts pass config, not logic.

2. Compression plugin interface — open, with defaults

  • Narrow synchronous contract: (text, meta) => { text, strategy, lossy, stats? } | null; null = pass-through. Mandatory contracts: pure & deterministic, input never mutated, fail-open on any anomaly, sub-minimum-size inputs untouched.
  • Registration: ordered list via config (hosts may add/replace/disable); kernel ships defaults.
  • Default implementations (zero new runtime deps, pure TS):
    • json-fold — constant-field hoisting + identical-row/run collapse into annotated envelopes; LOSSLESS (decodable back byte-for-byte).
    • code-trim — Python/JS-TS comment & docstring elision behind a string-literal-aware scanner (template literals, assigned triple-quoted strings survive verbatim; unterminated constructs → pass-through). The interface leaves room for a stronger AST implementation to register later (owner's "AST 级" wording — the scanner is the zero-dep default, AST is a swappable upgrade).
    • log-select — level-classified line selection (ALL ERROR/FAIL lines kept first, warnings deduped, summary lines kept, runtime stack frames collapsed, honest [N lines omitted] footer instead of CCR hashes); mechanics ported from headroom (Apache-2.0).
  • Guarantees enforced by the kernel, not each plugin: dispatch guards (prose/JSON/source never reach the wrong strategy), ERROR lines never dropped, sub-threshold inputs untouched.

3. Config + safety

  • Block-shaped config (like absorb): enabled, minReduction, per-strategy toggles/exclusions; resolved through the existing config-resolution path; DEFAULT OFF everywhere; hosts (bili) keep their own multi-level merge and hand the resolved block down.
  • No CCR-style retrieval markers inside crushed output (no dangling refs without a retrieval channel).

Non-goals

No ML, no query-aware relevance ranking (stateless-per-payload requirement), no new runtime dependencies, no wire-shape change beyond tool-result text, byte-identical behavior when disabled.

Phasing suggestion

  1. Valuator + interface + json-fold default (lossless, lowest risk).
  2. code-trim + log-select defaults.
  3. Retrieve-time application once #1097's store lands (same registry, second call site).

Starting material

  • Working implementations of all three strategies + a 29-test suite exist on the billion-context PR #1098 branch (src/pre-crush.ts, tests/pre-crush.test.ts) — ready to port as the default plugins. That PR stays blocked pending this issue.
  • headroom (Apache-2.0) crates/headroom-core/src/transforms/log_compressor.rs for log-selection mechanics.

Activity

  1. ranxianglei commented on Sep 22, 2026

    @ranxianglei
    OwnerAuthor

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 开工。这是你在 billion-context#1094 的分层裁定(估价归内核、压缩方式开放为插件接口、内核先行)的正式立项,方向已拍板,我按 feature triage 复核了一遍:合理且与内核架构契合——bili 侧 applyAbsorbView 的 size 决策和 #1098 的 pre-crush 前置通道都是 host-side 重复实现,收进内核后两种 proxy 模式与未来 host 共享同一决策路径,且 crush 节点与 absorb hide/prompt/adjudication 同处一条 processTurn 节点链,消除了 #1094 里 strip/re-bake [ACP absorb] marker 的竞态。

    实现计划(对应 issue 的 phasing):

    • Phase 1+2 本次一起做(三个默认策略的工作实现都在 #1098 分支上,移植成本低,拆开只会多两轮 review):
      1. Valuator(evaluateToolResult):纯函数,输入 tool-result 文本 + absorb/crush 配置 + 上下文压力信号,输出 skip | crushed | distill 三态决策;复用 minToolTokens/contextThresholdPct 语义,新增 minReduction(默认 0.1)。
      2. 插件接口:同步契约 (text, meta) => { text, strategy, lossy, stats? } | null;注册表模式(对齐现有 message-filter registry),host 可加/换/禁;内核保证 dispatch guard(JSON/源码/log 互不串)、ERROR 行不丢、sub-threshold 不动、fail-open。
      3. 三个默认插件:json-fold(无损)、code-trim(字符串字面量感知的注释/docstring 裁剪)、log-select(headroom 机制移植,诚实省略 footer)。
      4. 配置块 config.crush(block-shaped like absorb):enabled / minReduction / per-strategy toggles+exclusions,默认全关,关闭时字节级不变。
    • Phase 3(retrieve-time 应用)等 #1097 的 store 落地,接口已预留(crushText 独立导出,两个 call site 共用同一 registry)。

    一个设计决策先报备:crush 采用 view-only 每轮确定性重算(不写 state)——与 absorb prompt 的 view-only 语义一致,bili 每轮重发历史时输出字节稳定(prefix-cache 友好),且无旧持久化 state 的迁移问题。代价是每轮重算一次字符串变换(廉价)。

    开始实现,完成后带 PR 回复本 thread。

  2. ranxianglei commented on Sep 22, 2026

    @ranxianglei
    OwnerAuthor

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 Done — PR: #359 (Closes #354, branch 2026-09-23_tool-result-crush-gate)

    中文摘要:实现了内核侧两级 absorb 门的第一级——工具结果估价器 + 确定性压缩插件接口(json-fold / code-trim / log-select 三个内置策略),默认关闭、开启后字节稳定,807/807 测试通过,可合并待审。

    What landed (phases 1+2)

    • Valuator — pure, deterministic evaluateToolResult: skip / crushed (under minToolTokens, no model round-trip) / distill (still over or uncrushable → [ACP absorb] path, fed the crushed payload when crushing succeeded). Reuses defaultCountTokens + absorb gates; new minReduction floor (default 0.1). Same payload+config ⇒ same bytes.
    • Plugin interface — narrow sync contract (text, meta) => { text, lossy, stats? } | null; ordered registry with host add/replace/disable; per-strategy enabled/excludeTools in config. Kernel-enforced: dispatch guards per payload kind, fail-open on any throw/sub-floor/invariant violation.
    • Built-ins — json-fold (lossless const-hoist + run collapse, byte-decodable envelopes), code-trim (literal-aware comment/docstring elision; template literals & assigned triple-quoted strings verbatim; unterminated → pass-through), log-select (level-classified selection; warnings deduped; stack collapse; honest [N lines omitted] footer).
    • Pipeline — crush node between absorb-hide and absorb-prompt, so prompt decisions run on post-crush sizes atomically (stale markers stripped + re-decided same turn). View-only recomputation, no persisted-state mutation.
    • Config — top-level config.crush { enabled, minReduction, strategies? } through the existing resolution/validation path. Zero new runtime deps.

    Phase 3 (retrieve-time application) waits on billion-context#1097's store; the standalone crushText export is the shared call site.

    Behavior changes flagged (old → new)

    1. Feature DEFAULT OFF (crush.enabled:false, requires absorb.enabled:true) — output byte-identical when off (tested). No host behavior changes until opt-in.
    2. New pipeline node crush (canonical list 11→12; order test updated).
    3. Deviations from the reference prototype (billion-context PR #1098 branch, still unmerged):
      • log error budget LOG_MAX_ERRORS 10 → 20, plus a strict invariant: every distinct ERROR/FAIL line must survive verbatim in lossy log output or the result is rejected (fail-open). The prototype could silently drop middle errors beyond its budget of 10; the kernel never will. Logs with >20 distinct errors fail open instead.
      • Annotation vocabulary __pre_crush/[pre-crush:] → __acp_crush/[acp-crush:] (kernel-owned naming; nothing shipped depends on the old names).
      • Config at top-level config.crush, not nested absorb.preCrush.

    Verification

    typecheck clean · 807/807 tests (was 766; +41 new in tests/crush.test.ts) · build clean. Touched files Prettier-formatted; repo-wide format:check failure is pre-existing on master.

    One nit found while working (minor, not filed separately): the AGENTS.md module map predates several modules (absorb, config, pipeline, state, filter/, …) — I added the crush.ts line but left the rest untouched to keep this diff minimal; happy to file a docs issue if you want the map refreshed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions