Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions PLUGIN.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,11 +115,21 @@ Context-level visibility for plugin UIs (status bars / slash commands):

`contextTokens` is the last reported context size (input + cache-read) — the same value the nudge decision reads. Errors: `400` missing `conversationId`, `404` unknown conversation.

### 5. MITM transparent-proxy mode
### 5. `POST /__bili/plugin/compact`

Notify the proxy that the agent performed an **in-session native compaction** (e.g. omp's `/compact` or its auto threshold): the next model request re-sends a shortened history (compaction summary + retained tail) under the SAME conversation id. Fire-and-forget is fine — a failed notification must never break the agent's compaction.

```json
{ "conversationId": "the same value you send as x-bili-plugin-conversation" }
```

Effect: the proxy marks a one-shot compaction boundary on the session. On the next model request, blocks that were active before the compaction but no longer anchor into the shortened history are downgraded to a pre-compaction archive (listed by `acp_status` with a reason; `decompress` on one returns an explicit "unavailable" error instead of failing silently), and stale `byRaw`/`byRef` mappings are pruned to the live ids. Errors: `400` invalid JSON / missing `conversationId`, `404` unknown conversation.

### 6. MITM transparent-proxy mode

The `/bili/` prefix is absent in MITM mode, so URL-based detection cannot work. Instead the proxy's own launcher (`bili pi` / `bili codex` / `bili claude`) exports `BILLION_CONTEXT_PROXY=http://127.0.0.1:<port>` in the child env, next to the `HTTPS_PROXY` + CA vars it already sets. A plugin detects cooperative mode by reading that env var (the proxy origin for all `/__bili/plugin/*` calls); everything else (headers, tool forwarding, status) is identical — the `x-bili-plugin*` headers pass through the MITM tunnel into the same pipeline. This is also where `x-bili-plugin-context-window` matters most: MITM upstreams are often private relays the models.dev registry doesn't know.

### 6. Lifecycle of one compression (what the plugin does)
### 7. Lifecycle of one compression (what the plugin does)

1. Model replies with a native `compress` tool call (args contain `startId`/`endId` refs it read from the tag-annotated context).
2. The agent ends the assistant turn; the plugin's tool handler fires.
Expand Down
16 changes: 16 additions & 0 deletions src/agent/pi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,22 @@ export function createBiliPlugin(agentOverride?: string, opts?: { retryIntervalM
state.sid = undefined;
void registerTools(pi, ctx, state, agent).catch((err: unknown) => console.error(`bili-plugin(${agent}): ${err instanceof Error ? err.message : String(err)}`));
});
// omp fires session_compact on in-session native compaction (sid does
// not rotate), so the proxy reuses stale state — notify it to archive
// the now-unreachable blocks (#395). Fire-and-forget: a failed
// notification must never break the agent's compaction.
pi.on("session_compact", (_event, ctx) => {
const proxyBase = proxyBaseForCtx(ctx);
if (proxyBase === undefined) return;
const sid = sessionIdOf(ctx);
if (sid === undefined || sid.length === 0) return;
fetch(`${proxyBase}/__bili/plugin/compact`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ conversationId: sid }),
signal: AbortSignal.timeout(5000),
}).catch(() => {});
});
};
}

Expand Down
6 changes: 5 additions & 1 deletion src/decompress-shared.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import {
import { mkdirSync, unlinkSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { tmpdir } from "node:os";
import type { Session } from "./session.js";
import { preCompactionArchiveOf, type Session } from "./session.js";

/** Bounded retention for large-decompress temp files. Each decompress with
* body > 10000 writes one file under tmpdir(); the reaper unlinks oldest past
Expand Down Expand Up @@ -78,6 +78,10 @@ export function resolveDecompress(
const blockId = rawBlockId.trim();
const block = ctx.core.decompress(blockId, ctx.session.state);
if (!block) return `[Block ${blockId} not found]`;
const archived = preCompactionArchiveOf(ctx.session);
if (archived[blockId] !== undefined) {
return `[decompress FAILED: block ${blockId} is a pre-compaction archive — its content was in the history BEFORE the client's native compaction and is no longer reachable (replaced by the client's compaction summary). decompress is unavailable for archived blocks.]`;
}

const full = args.full === true;
const cached = ctx.session.blockContents.get(blockId);
Expand Down
11 changes: 10 additions & 1 deletion src/loop/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import {
type CoreMessage,
type NudgeDecision,
} from "acp-kernel";
import { lastCompressSuffix, type Session } from "../session.js";
import { lastCompressSuffix, preCompactionArchiveOf, type Session } from "../session.js";
import type { BiliMessage } from "acp-kernel/wire";
import {
parseCompressInput,
Expand Down Expand Up @@ -148,6 +148,8 @@ function handleAcpStatus(args: Record<string, unknown>, ctx: LoopCtx): string {
const nudge = ctx.nudge;
const ranges = nudge?.compressibleRanges ?? [];
const protectedRanges = nudge?.protectedRanges ?? [];
const archive = preCompactionArchiveOf(ctx.session);
const archivedIds = Object.keys(archive);
const extra: string[] = [];
if (nudge) {
extra.push("");
Expand All @@ -157,6 +159,13 @@ function handleAcpStatus(args: Record<string, unknown>, ctx: LoopCtx): string {
extra.push("");
extra.push(formatRanges(ranges, protectedRanges));
}
if (archivedIds.length > 0) {
extra.push("");
extra.push(`PRE-COMPACTION ARCHIVE — ${archivedIds.length} block(s): content was replaced by the client's native compaction summary, so it is no longer in the session history and decompress is unavailable.`);
for (const id of archivedIds) {
extra.push(` ${id} — ${archive[id].reason}`);
}
}
return extra.length > 0 ? `${base}\n${extra.join("\n")}` : base;
}

Expand Down
29 changes: 28 additions & 1 deletion src/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { buildStatusPanel } from "acp-kernel/panel";
import { fileURLToPath } from "node:url";
import fs from "node:fs";
import path from "node:path";
import { acquireInFlight, listSessions, markDirty, peekSession, releaseInFlight, withSessionLock, type Session } from "./session.js";
import { acquireInFlight, listSessions, markCompactionBoundary, markDirty, peekSession, releaseInFlight, withSessionLock, type Session } from "./session.js";
import { ACP_TOOLS_ANTHROPIC, ACP_TOOLS_OPENAI, ACP_TOOLS_RESPONSES, PROXY_TOOL_NAMES } from "./compress-tool.js";
import { executeProxyTool } from "./loop/core.js";
import { normalizeSseLineEndings } from "./sse-util.js";
Expand Down Expand Up @@ -299,6 +299,33 @@ export function handlePluginRegister(payload: string, res: import("node:http").S
res.end(JSON.stringify({ ok: true, conversationId, agent }));
}

export function handlePluginCompact(payload: string, res: import("node:http").ServerResponse): void {
let parsed: { conversationId?: unknown };
try {
parsed = JSON.parse(payload) as { conversationId?: unknown };
} catch {
res.writeHead(400, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: "invalid JSON body" }));
return;
}
const conversationId = typeof parsed.conversationId === "string" ? parsed.conversationId.trim() : "";
if (!conversationId) {
res.writeHead(400, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: "conversationId is required" }));
return;
}
const entry = conversations.get(conversationId);
const session = entry ? peekSession(entry.sessionId) : undefined;
if (!entry || !session) {
res.writeHead(404, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: "unknown plugin conversation (no model request has arrived with this conversation id yet)" }));
return;
}
markCompactionBoundary(session);
entry.lastSeen = Date.now();
res.end(JSON.stringify({ ok: true, conversationId }));
}

export function handlePluginManifest(res: import("node:http").ServerResponse): void {
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({
Expand Down
19 changes: 17 additions & 2 deletions src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ import {
conversationSignalResponses,
subagentNamespace,
} from "acp-kernel/wire";
import { getSession, listSessions, type Session, initSessions, markDirty, flushAllSessions, acquireInFlight, releaseInFlight, withSessionLock, markNativeCompactionBoundary, reconcileNativeCompactionBoundary, snapshotMessages } from "./session.js";
import { getSession, listSessions, type Session, initSessions, markDirty, flushAllSessions, acquireInFlight, releaseInFlight, withSessionLock, markNativeCompactionBoundary, reconcileNativeCompactionBoundary, snapshotMessages, applyCompactionArchive } from "./session.js";
import { COMPRESS_TOOL, ACP_TOOLS_ANTHROPIC, ACP_TOOLS_OPENAI, ACP_TOOLS_RESPONSES, ACP_READONLY_TOOLS_RESPONSES, COMPRESS_TOOL_NAME, buildCompressSystemPrompt, buildCompressHybridSystemPrompt, withStagedCompressGuidance } from "./compress-tool.js";
import { rewriteJsonResponse, type RewriteCtx } from "./stream.js";
import { applyRanges } from "./stream.js";
Expand All @@ -64,7 +64,7 @@ import { observeResponsesTerminalState } from "./stream-terminal.js";
import { emitStreamError } from "./stream-error.js";
import { affinityToken, clientConversationHeader, codexTurnIdentity, preferPromptCacheKeyIdentity, type ConversationIdentity } from "./session-id.js";
import { prefixAffinity, type AnonymousAffinity } from "./prefix-affinity.js";
import { consumePluginRegisterFor, flushConversations, handlePluginManifest, handlePluginRegister, handlePluginStatus, handlePluginTool, loadConversations, pipePluginChatWithStrip, pipePluginJson, pipePluginResponsesWithStrip, pluginAgentHeader, pluginConversationHeader, pluginReportedContextWindow, recordPluginSession, rememberPluginMessages, takePendingPluginRegister } from "./plugin.js";
import { consumePluginRegisterFor, flushConversations, handlePluginCompact, handlePluginManifest, handlePluginRegister, handlePluginStatus, handlePluginTool, loadConversations, pipePluginChatWithStrip, pipePluginJson, pipePluginResponsesWithStrip, pluginAgentHeader, pluginConversationHeader, pluginReportedContextWindow, recordPluginSession, rememberPluginMessages, takePendingPluginRegister } from "./plugin.js";
import { setupMitm, readMitmUpstream } from "./mitm.js";
import type { BiliMessage } from "acp-kernel/wire";
import { systemToUser, isLoopbackAddress, inspectContextOverflow, reserveOutputHeadroom, shouldReserveOutputHeadroom, usageTotals, type WireProtocol } from "./util.js";
Expand Down Expand Up @@ -754,6 +754,17 @@ async function handle(
return;
}
}
if (req.method === "POST" && req.url === "/__bili/plugin/compact") {
try {
const body = await readBody(req);
handlePluginCompact(body.toString("utf8"), res);
return;
} catch (err) {
res.writeHead(err instanceof BodyTooLargeError ? 413 : 400, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: String(err) }));
return;
}
}
// Unknown /__bili/ or /__acp/ path → 404 locally. These are bili's own
// management prefixes; forwarding would leak the internal path to the
// upstream (which 403s it) — #346.
Expand Down Expand Up @@ -1429,6 +1440,7 @@ function prepareAnthropic(
// estimates to the kernel.
extractSystem(parsed.system);
const tokenCount = session.stats.lastInputTokens;
const activeBefore = new Set(session.state.blocks.filter((b) => b.active).map((b) => b.blockId));
const turn = core.processTurn({ messages: msgs, state: session.state, config, tokenCount, renderTags: "text-only" });
session.state = turn.state;
// The fold from last turn's compress has now materialized in state —
Expand All @@ -1448,6 +1460,7 @@ function prepareAnthropic(
const willInjectNudge = opts.compress.injectNudge && !!turn.nudge?.shouldInject;
log("info", diagNudge(turn, sessionId, tokenCount, config.modelContextLimit, parsed.model, willInjectNudge));
processedMessages = stripKernelSummaries(turn.messages, turn.state);
applyCompactionArchive(session, activeBefore, new Set(msgs.map((m) => m.id)), log);
reapOrphanBlocks(session, msgs, deactivateBlock);
rebuiltMessages = coreToAnthropic(processedMessages as BiliMessage[], cacheControls);

Expand Down Expand Up @@ -1529,6 +1542,7 @@ function prepareOpenai(
// tokenCount = upstream's real input_tokens from the previous turn
// (see anthropic branch comment). Never an estimate.
const tokenCount = session.stats.lastInputTokens;
const activeBefore = new Set(session.state.blocks.filter((b) => b.active).map((b) => b.blockId));
const turn = core.processTurn({ messages: msgs, state: session.state, config, tokenCount, renderTags: "text-only" });
session.state = turn.state;
// The fold from last turn's compress has now materialized in state —
Expand All @@ -1548,6 +1562,7 @@ function prepareOpenai(
const willInjectNudge = opts.compress.injectNudge && !!turn.nudge?.shouldInject && shouldInject;
log("info", diagNudge(turn, sessionId, tokenCount, config.modelContextLimit, parsed.model, willInjectNudge));
processedMessages = stripKernelSummaries(turn.messages, turn.state);
applyCompactionArchive(session, activeBefore, new Set(msgs.map((m) => m.id)), log);
reapOrphanBlocks(session, msgs, deactivateBlock);
rebuiltMessages = systemToUser(coreToOpenai(processedMessages as BiliMessage[]));

Expand Down
76 changes: 76 additions & 0 deletions src/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,82 @@ export function reconcileNativeCompactionBoundary(session: Session): boolean {
return true;
}

/** Mark a client-side native compaction (omp /compact on the anthropic wire).
* The sid does NOT rotate on in-session compaction, so the per-sid registry
* reuses the same key with stale state. Consumed by the NEXT processTurn via
* applyCompactionArchive (#395). Distinct from nativeCompactionBoundary
* (Responses/codex, which rebases by resetting state). */
export function markCompactionBoundary(session: Session): void {
session.metadata.compactionBoundary = {
at: Date.now(),
pending: true,
};
markDirty(session);
}

type PreCompactionArchive = Record<string, { at: number; reason: string }>;

function readPreCompactionArchive(session: Session): PreCompactionArchive {
const raw = session.metadata.preCompactionArchive;
if (raw && typeof raw === "object" && !Array.isArray(raw)) {
return raw as PreCompactionArchive;
}
return {};
}

export function preCompactionArchiveOf(session: Session): PreCompactionArchive {
return readPreCompactionArchive(session);
}

// Must run AFTER the processTurn that followed markCompactionBoundary (so
// syncBlocks has deactivated the blocks whose raw ids left the shortened
// history). Blocks active in `activeBefore` but inactive now are archived;
// byRaw/byRef are pruned to liveRawIds (stops the #390 additive leak).
// nextIndex is left alone so a freed ref slot is never re-allocated onto a
// retained tail's live tag.
export function applyCompactionArchive(
session: Session,
activeBefore: Set<string>,
liveRawIds: Set<string>,
log: (level: string, msg: string) => void,
): void {
const boundary = session.metadata.compactionBoundary;
if (!boundary || typeof boundary !== "object" || !(boundary as Record<string, unknown>).pending) {
return;
}
const activeAfter = new Set(session.state.blocks.filter((b) => b.active).map((b) => b.blockId));
const deactivated = [...activeBefore].filter((id) => !activeAfter.has(id));
if (deactivated.length > 0) {
const archive = readPreCompactionArchive(session);
const at = Date.now();
for (const id of deactivated) {
archive[id] = { at, reason: "content replaced by client native compaction summary" };
}
session.metadata.preCompactionArchive = archive;
}

const { byRaw, byRef } = session.state.messageRefs;
const prunedByRaw: Record<string, string> = {};
for (const [rawId, ref] of Object.entries(byRaw)) {
if (liveRawIds.has(rawId)) prunedByRaw[rawId] = ref;
}
const prunedByRef: Record<string, string> = {};
for (const [ref, rawId] of Object.entries(byRef)) {
if (liveRawIds.has(rawId)) prunedByRef[ref] = rawId;
}
session.state.messageRefs.byRaw = prunedByRaw;
session.state.messageRefs.byRef = prunedByRef;

session.metadata.compactionBoundary = {
...(boundary as Record<string, unknown>),
pending: false,
archivedAt: Date.now(),
archivedBlocks: deactivated,
};
markDirty(session);
log("info", `[${session.id}] native compaction boundary: archived ${deactivated.length} pre-compaction block(s)${deactivated.length > 0 ? ` (${deactivated.join(", ")})` : ""}; pruned ref maps to ${prunedByRaw.length} live raw id(s)`);
}

/** Flush a session to disk and drop it from memory (LRU eviction). Refuses to
* evict sessions that are in-flight or whose flush failed (would lose a
* never-persisted session permanently). Returns true if a slot was freed. */
Expand Down
Loading
Loading