Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ acp-kernel/
├── src/
│ ├── index.ts # Barrel export
│ ├── compress.ts # Core: processTurn, applyCompression, decideNudge, pipeline nodes
│ ├── crush.ts # Tool-result valuator + deterministic crush strategies (two-tier absorb gate)
│ ├── boundaries.ts # Range boundary resolution (startId/endId → message indices)
│ ├── filter.ts # Protected tool message filtering
│ ├── nudge.ts # Nudge text rendering
Expand Down
34 changes: 32 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@ const state = createInitialState();
const config = defaultConfig(200000); // modelContextLimit (positional); optional overrides as 2nd arg

// processTurn runs the canonical node pipeline every turn:
// assign-refs → sync-blocks → merge-blocks → prune → filter →
// hide-compress-calls → nudge-inject → emergency-truncate → render-refs
// assign-refs → sync-blocks → prune → absorb-hide → crush → absorb-prompt
// → filter → hide-compress-calls → recommend → nudge-inject
// → emergency-truncate → render-refs
const { messages, state: nextState, nudge } = core.processTurn({
messages, state, config, tokenCount,
});
Expand Down Expand Up @@ -162,6 +163,35 @@ The nudge system tells the model *when* to compress. It implements:
- **Compressible-range computation**: reports the actual compressible ranges (excluding covered + preserved-recent messages) so the model knows what to target.
- **Baseline reset on compress**: `applyCompression` clears the growth baseline on success, preventing the feedback-loop bug where the nudge re-fires post-compress.

### Tool-result crush gate

Deterministic first tier of the two-tier absorb gate. When a tool result qualifies as an absorb candidate and context pressure is on, the `crush` node (between `absorb-hide` and `absorb-prompt`) re-evaluates it **in view** with pure, deterministic strategies before the absorb-prompt decision runs:

- **skip** — below `absorb.minToolTokens` or under `absorb.contextThresholdPct`; bytes untouched.
- **crushed** — deterministic compression brought it under `minToolTokens`; no model round-trip. The wire carries the crushed payload (e.g. `__acp_crush` rows/identical-run envelopes for JSON, lossless — decodes back to an equal JSON document).
- **distill** — still over (or uncrushable); forwarded to the `[ACP absorb]` prompt path, carrying the crushed payload when crushing succeeded.

Built-in strategies form an ordered plugin registry (hosts can add/replace/disable via `registerCrushPlugin` / config):

| Strategy | Kind | Lossy | What it does |
|----------|------|-------|--------------|
| `json-fold` | json | no | constant-field hoisting + identical row/run collapse into annotated envelopes |
| `code-trim` | code | yes | comment/docstring elision behind a string-literal-aware scanner; template literals and assigned triple-quoted strings survive verbatim; unterminated constructs fail open |
| `log-select` | log | yes | level-classified line selection; **every distinct ERROR/FAIL line survives verbatim** (kernel-enforced invariant — output dropping one is rejected); warnings deduped, stack frames collapsed, honest `[N lines omitted]` footer |

Kernel-enforced guarantees: dispatch guards keep each payload kind on its own strategies; plugins must be pure and deterministic (same payload + config ⇒ same bytes, prefix-cache friendly); any throw, sub-`minReduction` result, or invariant violation fails open to the next candidate or pass-through. **Default off** (`crush.enabled: false`) and requires `absorb.enabled: true`; when disabled the pipeline is byte-identical to pre-crush behavior.

```json
"crush": {
"enabled": false,
"minReduction": 0.1,
"strategies": {
"json-fold": { "enabled": false },
"code-trim": { "excludeTools": ["bash"] }
}
}
```

## Wire codec (`acp-kernel/wire`)

The `acp-kernel/wire` subpath ships the provider wire-body codecs (lossless
Expand Down
29 changes: 29 additions & 0 deletions src/compress.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import type { ResolvedRange } from "./boundaries.js";
import { truncateLargeToolOutputs } from "./truncate-tools.js";
import { hideConsumedCompressCalls } from "./hide-consumed.js";
import { appendAbsorbPrompts, hideAbsorbedMessages } from "./absorb.js";
import { applyCrushToMessages } from "./crush.js";
import { applyMessageFilters, listMessageFilters } from "./filter/index.js";
import { activeBlockSpans } from "./block-map.js";
import { createRenderRefsNode } from "./render-refs.js";
Expand Down Expand Up @@ -531,6 +532,7 @@ export function createCore(ports: Ports = {}): CompressionCore {
syncBlocksNode,
pruneNode,
absorbHideNode,
crushNode,
absorbPromptNode,
filterNode,
hideCompressCallsNode,
Expand Down Expand Up @@ -625,6 +627,33 @@ const absorbPromptNode: PipelineNode = {
},
};

// Tier 1 of the two-tier absorb gate: deterministic compression of eligible
// oversized tool results. Runs after absorb-hide (already-absorbed pairs are
// gone) and before absorb-prompt, which then re-decides on post-crush sizes —
// results crushed below minToolTokens stop triggering model round-trips.
const crushNode: PipelineNode = {
name: "crush",
enabled: (_io, ctx) =>
ctx.config.crush?.enabled === true && ctx.config.absorb?.enabled === true,
run(io, ctx) {
const applied = applyCrushToMessages(
io.messages,
ctx.config,
ctx.tokenCount,
ctx.countTokens,
);
return {
...io,
messages: applied.messages,
effects: {
...io.effects,
crushCount: applied.crushedCount,
crushDistilledCount: applied.distilledCount,
},
};
},
};

const filterNode: PipelineNode = {
name: "filter",
enabled: (_io, ctx) =>
Expand Down
34 changes: 34 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ export function defaultConfig(
contextThresholdPct: 0,
excludeTools: [],
},
crush: {
enabled: false,
minReduction: 0.1,
},
};
return {
...base,
Expand All @@ -50,6 +54,7 @@ export function defaultConfig(
absorb: overrides.absorb
? { ...base.absorb, ...overrides.absorb }
: base.absorb,
crush: overrides.crush ? { ...base.crush, ...overrides.crush } : base.crush,
};
}

Expand Down Expand Up @@ -129,5 +134,34 @@ export function validateConfig(config: Config): string[] {
errors.push("rules.maxRuleChars must be >= 1");
}
}
if (config.crush) {
if (
!Number.isFinite(config.crush.minReduction) ||
config.crush.minReduction <= 0 ||
config.crush.minReduction > 1
) {
errors.push("crush.minReduction must be in (0, 1]");
}
if (config.crush.strategies) {
for (const [id, ov] of Object.entries(config.crush.strategies)) {
if (!ov || typeof ov !== "object" || Array.isArray(ov)) {
errors.push(`crush.strategies.${id} must be an object`);
continue;
}
if (ov.enabled !== undefined && typeof ov.enabled !== "boolean") {
errors.push(`crush.strategies.${id}.enabled must be a boolean`);
}
if (
ov.excludeTools !== undefined &&
(!Array.isArray(ov.excludeTools) ||
ov.excludeTools.some((t) => typeof t !== "string"))
) {
errors.push(
`crush.strategies.${id}.excludeTools must be a string array`,
);
}
}
}
}
return errors;
}
Loading
Loading