Repository navigation
Description
Summary
In the DSH plugin lane, compression itself can still reduce context, but obsolete compress call/result pairs may survive their intended lifecycle and raise the post-compression floor. The DSH adapter invokes forwardTool without the host toolCallId, so blocks get synthetic plugin_* IDs while replayed tool calls keep call_* IDs. hideConsumedCompressCalls then treats bare compress calls as orphans when IDs do not match, and a numeric-ref sweep cannot retire BLOCKED refs (refNum(BLOCKED) is null).
This is a DSH-lane recurrence of the mechanism fixed for Pi in #3587, not a request to reopen that issue, and not a request to drop protected history unconditionally. A fix must keep a complete summary carrier for every active block in both plugin and proxy modes (see #5866).
A separate parser issue, #7240, covers invalid newline-joined JSON tuples being accepted as a single range. The two defects were observed together in one incident and amplify each other; they should stay separate repair scopes.
Verified environment
- Incident: billion-context 0.1.215, DSH 0.2.2-rc.3, plugin-native lane, OpenAI-compatible wire. DSH is deployed on Ubuntu (release not provided), using a self-hosted upstream serving
qwen3.8-flash-next(reporter-confirmed). - The identity / BLOCKED branch was also reproduced offline at 0.1.216, commit
6a641f378537e06e7430ec90c9abf035ffa12377, with Node.js 24. - These tests are function-level synthetic probes, not a full DSH wire capture. Later releases, including current master, have not been rechecked here.
Related work (not duplicates)
- [Bug][Pi] lost native compress call identity evicts active carriers and resurrects early summary anchors #3587 (closed) and [PR] fix: preserve native Pi compress carrier identity and cached prefix #3592, landed via [PR] merge: tier-5 train — dsh prepare/streaming domain (7 PRs: bili_final terminal marker, auto-fold storm gate, wire-only guard) #4590: the same missing-id mechanism on Pi. That fix carries Pi's native call id through the plugin transport. This report is the DSH adapter path still omitting toolCallId at the versions above, plus inactive leftovers that stay visible when refs are
BLOCKED. - The plain text content of MCP compress is not simplified, causing the full summary to be redundantly retained. | MCP compress 的纯文本 content 未被简化,导致完整摘要重复保留 #3975 (open) and [PR] fix(kernel): stub plain-text line-form compress calls in consumed-call cleanup (#3975) #4032 (open, not merged): consumed-call cleanup does not parse documented line-form
content, so even a matching active call may keep full summary text. That overlap is already tracked; this issue does not relitigate it. One synthetic control below (matching IDs, line-form, call unchanged) is consistent with The plain text content of MCP compress is not simplified, causing the full summary to be redundantly retained. | MCP compress 的纯文本 content 未被简化,导致完整摘要重复保留 #3975 and is not the headline defect. - Plugin-lane array-form folds end with zero full summaries on the wire after the first turn (call stubbed + render stripped) #5866 (open): plugin-lane array-form can end with zero full summaries after a call is stubbed and the render is stripped. That is the opposite failure mode. Retiring or rewriting IDs must not delete the only complete carrier.
Mechanism and code
Source links are pinned to the checked 0.1.216 commit; the implicated 0.1.215 logic was also inspected.
- The DSH adapter calls
forwardTool(base, sid, tool.name, args, exec.signal, true)without the optional toolCallId. - Plugin execution then creates a
plugin_-prefixed synthetic ID when none is supplied. hideConsumedCompressCallsresolves barecompressby call ID; unmatched bare calls become orphans. A sweep requires a numeric ref, whilerefNum(BLOCKED)is null. This describes the tested branch, not a claim that a later range can never cover these messages.
Minimal synthetic reproduction
Function-level only: isolated checkout, Node.js 24, no production data, and no running DSH. Please attach reproduce.mjs, observed-0.1.215.json, and observed-0.1.216.json (the two result files are byte-identical). The issue-1 cases remain understandable from the table below if the files are missing.
git clone --depth 1 --branch v0.1.215 https://github.com/ranxianglei/billion-context.git bili-0215
node reproduce.mjs ./bili-0215
git clone https://github.com/ranxianglei/billion-context.git bili-0216
git -C bili-0216 checkout --detach 6a641f378537e06e7430ec90c9abf035ffa12377
node reproduce.mjs ./bili-0216Results on both checked versions:
| Fixture | hidden | kept | notes |
|---|---|---|---|
| Inactive block, mismatching IDs, BLOCKED refs | 0 | 2 | headline case |
| Same fixture, matching IDs | 2 | 0 | diagnostic |
| Same fixture, mismatching IDs but numeric refs | 2 | 0 | diagnostic, not a proposed patch |
| Active matching ID with documented line-form content | 0 | 2 (call unchanged) | consistent with #3975; not claimed as new |
A script failure after export or API changes on a newer revision is not evidence that the defect is gone. The ID/ref variants are diagnostic counterfactuals, not safe patches.
Sanitized incident evidence
Private session content is not attached. These aggregates come from a 0.1.215 + DSH 0.2.2-rc.3 OpenAI-compatible session:
- 20 compress calls / 20 blocks; every block ID differed from the corresponding tool-call ID; 15 blocks stayed active.
- 18 call/result pairs were outside later block coverage: 97,774 characters of call text plus 15,911 characters of results. Two other pairs were covered by a later range, so BLOCKED is not an absolute barrier to later coverage.
- Five inactive blocks had uncovered original calls totaling 20,032 call-text characters.
- Offline replay of those 18 uncovered calls against actual block metadata retained all 36 messages.
- A carrier-strip helper comparison retained 15 summary anchors (61,358 summary characters) with the actual IDs, versus two anchors (5,822 characters) in a diagnostic ID-matched run — a 55,536-character duplicate-carrier opportunity in that replay, not a measured historical wire or token saving.
- Actual upstream input usage once fell from 120,630 to 67,328 tokens, so ordinary compression was not completely ineffective. Later post-fold baselines rose; the last available raw input usage was 142,244 tokens. A session-accounting display of about 385,104 tokens is not evidence of a 385k outbound request.
Why the two issues together are serious
On their own, leftover identity mismatches keep obsolete compress machinery in context. In the same incident, invalid multiline JSON-tuple content was silently accepted as one range (#7240), so long summary text moved into topic and batch execution was easy to over-count. Combined, oversized topics and unretired call/result text can raise the post-compression floor even when a fold still reduces usage. This is not a claim of irreversible data loss, and not causal proof that the later 142,244 baseline was produced only by these two defects.
Expected behavior
- Propagate the host's real call identity where the DSH API provides it; do not guess field names.
- Reconcile historical synthetic IDs only when the match is unambiguous. Fixing future calls is separate from repairing existing ledgers.
- Retire obsolete successful exchanges at the intended fold boundaries, while keeping failure diagnostics and recent protections.
- After any compaction or anchor suppression, every active block must still have a complete summary carrier (Plugin-lane array-form folds end with zero full summaries on the wire after the first turn (call stubbed + render stripped) #5866).
Suggested labels: bug, p1, dsh, scope/kernel (the omitted toolCallId is in the DSH adapter; maintainers may also want scope/wiring). Theme: C (compression engine) and F (dsh lane).
Appendix: synthetic reproduction files
GitHub's issue API cannot attach files, so the three approved synthetic files are included below verbatim. observed-0.1.215.json and observed-0.1.216.json are byte-identical.
SHA-256:
reproduce.mjs:df04e74c77c1c25382a1aef6a85771d806dc2c3511ab508fcaeed8112310979fobserved-0.1.215.json/observed-0.1.216.json:0f6ca47adb3ae63a0d2159004bc6e436191fb34bc3d006fd78f448c8c3c39c06
reproduce.mjs
// Synthetic fixtures only. Reads an isolated upstream checkout; no server or database access.
import { registerHooks } from 'node:module';
import { existsSync } from 'node:fs';
import { resolve, sep } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import assert from 'node:assert/strict';
const input = process.argv[2];
if (!input) throw new Error('Usage: node reproduce.mjs /path/to/isolated/billion-context-checkout');
const root = resolve(input);
registerHooks({ resolve(specifier, context, nextResolve) {
if (specifier.startsWith('.') && specifier.endsWith('.js') && context.parentURL?.startsWith('file:')) {
const parent = fileURLToPath(context.parentURL);
if (parent.startsWith(root + sep)) {
const candidate = new URL(specifier.slice(0, -3) + '.ts', context.parentURL);
if (existsSync(fileURLToPath(candidate))) return nextResolve(candidate.href, context);
}
}
return nextResolve(specifier, context);
}});
const load = relative => import(pathToFileURL(resolve(root, relative)).href);
const { parseCompressArgs } = await load('kernel/src/parse-compress-input.ts');
const { hideConsumedCompressCalls } = await load('kernel/src/hide-consumed.ts');
const malformed = { content: '["m00001–m00002","TITLE ONE","' + 'A'.repeat(24000) + '"]\n["m00003–m00004","TITLE TWO","BODY TWO"]' };
const summarize = p => ({ ok: p.diagnostics.ok, ranges: p.ranges.map(r => ({ startRef: r.startRef, endRef: r.endRef, topicChars: r.topic?.length ?? 0, summaryChars: r.summary.length })) });
const correct = { content: 'm00001–m00002 First\nSummary one.\n\nm00003–m00004 Second\nSummary two.' };
const good = parseCompressArgs(correct);
assert.equal(good.ranges.length, 2, 'Control: documented line format should parse two ranges');
const call = { id: 'synthetic-message-call', role: 'assistant', contentType: 'tool-call', toolName: 'compress', toolCallId: 'call_synthetic', text: JSON.stringify({ content: 'm00001–m00002 Short topic\n' + 'S'.repeat(1000) }) };
const result = { id: 'synthetic-message-result', role: 'tool', contentType: 'tool-result', toolName: 'compress', toolCallId: 'call_synthetic', text: 'Synthetic successful receipt' };
const block = { blockId: 'b1', startRef: 'm00001', endRef: 'm00002', compressCallId: 'plugin_synthetic', active: false, topic: 'Short topic', summary: 'S'.repeat(1000) };
function run({ matchedId = false, numericRefs = false, active = false } = {}) {
const state = { blocks: [{ ...block, active, compressCallId: matchedId ? call.toolCallId : block.compressCallId }], messageRefs: { byRaw: { [call.id]: numericRefs ? 'm00003' : 'BLOCKED', [result.id]: numericRefs ? 'm00004' : 'BLOCKED' }, byRef: {} }, pendingOrphanSweepMinRef: 1 };
const out = hideConsumedCompressCalls(state, [call, result]);
return { hidden: out.hidden, kept: out.messages.length, callUnchanged: out.messages.some(m => m.id === call.id && m.text === call.text) };
}
console.log(JSON.stringify({ fixture: 'Synthetic, function-level only; no historical wire capture and no carrier-strip E2E assertion.', issue1: { inactiveIdMismatchBlocked: run(), inactiveIdMatchedBlocked: run({ matchedId: true }), inactiveIdMismatchNumeric: run({ numericRefs: true }), activeIdMatchedLineForm: run({ matchedId: true, active: true }) }, issue2: { malformed: summarize(parseCompressArgs(malformed)), correctLineForm: summarize(good), nestedTupleDiagnostics: parseCompressArgs({ content: JSON.stringify([['m00001–m00002', 'Title', 'Summary']]) }).diagnostics } }, null, 2));observed-0.1.215.json
{
"fixture": "Synthetic, function-level only; no historical wire capture and no carrier-strip E2E assertion.",
"issue1": {
"inactiveIdMismatchBlocked": {
"hidden": 0,
"kept": 2,
"callUnchanged": true
},
"inactiveIdMatchedBlocked": {
"hidden": 2,
"kept": 0,
"callUnchanged": false
},
"inactiveIdMismatchNumeric": {
"hidden": 2,
"kept": 0,
"callUnchanged": false
},
"activeIdMatchedLineForm": {
"hidden": 0,
"kept": 2,
"callUnchanged": true
}
},
"issue2": {
"malformed": {
"ok": true,
"ranges": [
{
"startRef": "m00001",
"endRef": "m00002",
"topicChars": 24017,
"summaryChars": 38
}
]
},
"correctLineForm": {
"ok": true,
"ranges": [
{
"startRef": "m00001",
"endRef": "m00002",
"topicChars": 5,
"summaryChars": 12
},
{
"startRef": "m00003",
"endRef": "m00004",
"topicChars": 6,
"summaryChars": 12
}
]
},
"nestedTupleDiagnostics": {
"ok": false,
"kind": "no-valid-ranges",
"invalidItems": 1,
"keys": [
"content"
],
"invalidReasons": [
"entry 0: not an object"
]
}
}
}observed-0.1.216.json
This file is byte-identical to observed-0.1.215.json. The same bytes are repeated here so the filename is present.
{
"fixture": "Synthetic, function-level only; no historical wire capture and no carrier-strip E2E assertion.",
"issue1": {
"inactiveIdMismatchBlocked": {
"hidden": 0,
"kept": 2,
"callUnchanged": true
},
"inactiveIdMatchedBlocked": {
"hidden": 2,
"kept": 0,
"callUnchanged": false
},
"inactiveIdMismatchNumeric": {
"hidden": 2,
"kept": 0,
"callUnchanged": false
},
"activeIdMatchedLineForm": {
"hidden": 0,
"kept": 2,
"callUnchanged": true
}
},
"issue2": {
"malformed": {
"ok": true,
"ranges": [
{
"startRef": "m00001",
"endRef": "m00002",
"topicChars": 24017,
"summaryChars": 38
}
]
},
"correctLineForm": {
"ok": true,
"ranges": [
{
"startRef": "m00001",
"endRef": "m00002",
"topicChars": 5,
"summaryChars": 12
},
{
"startRef": "m00003",
"endRef": "m00004",
"topicChars": 6,
"summaryChars": 12
}
]
},
"nestedTupleDiagnostics": {
"ok": false,
"kind": "no-valid-ranges",
"invalidItems": 1,
"keys": [
"content"
],
"invalidReasons": [
"entry 0: not an object"
]
}
}
}
Activity
ranxianglei commentedon Nov 19, 2026
(已收)本单已归入根因聚合 issue(token 见聚合单 body),机器人已停止在本单上的工作,修复走聚合 issue 统一出 PR;本单保留不关,修复验证后统一关闭。(聚合单:[Root Cause] dsh-lane (10 items),GitHub #6446)