Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions src/knobs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -568,13 +568,13 @@ export function codexCompactMode(): CodexCompactMode {
}

/** BILI_DSH_AUTHORED_COMPACTION — #3733 bili-authored dsh client compaction.
* Default OFF ("0"/unset): the #1729 guard keeps refusing dsh's native
* compaction summarize calls, exactly as before. "1": when the session
* holds active fold blocks, the guard site answers the call LOCALLY with
* Default ON (the #3723/#3901 flip): when the session holds active fold
* blocks, the guard site answers dsh's native compaction call LOCALLY with
* our folded snapshot (zero egress) instead of refusing — dsh records our
* checkpoint and its file shrinks (#3733/#3723). Read per-request so a
* running proxy flips without a restart. Env-only for the rollout window;
* the #2028-style file/web toggle follows once adopted. */
* checkpoint and its file shrinks (#3733/#3723). "0"/false is the escape
* hatch back to the #1729 refusal path. Read per-request so a running proxy
* flips without a restart. Env-only for the rollout window; the #2028-style
* file/web toggle follows once adopted. */
export function dshAuthoredCompactionEnabled(): boolean {
const envRaw = process.env.BILI_DSH_AUTHORED_COMPACTION;
if (envRaw !== undefined) return envRaw.trim() !== "0";
Expand Down
38 changes: 21 additions & 17 deletions src/server/budget.ts
Original file line number Diff line number Diff line change
Expand Up @@ -306,23 +306,27 @@ export function clampOutputBudget(requested: number, inputEstimate: number, nati
return clamped;
}

/** #2490: translate the refused dsh compaction replay size (recorded on the
* session by the guard in handle.ts) into a token floor for the clamp's
* planning basis. bytes/4 mirrors the default estimator's chars/4 caliber
* (the replay is JSON text riding the same wires). 0 when nothing was
* recorded — legacy behavior.
* STICKY BY DESIGN: the guard records a high-water mark and nothing decays
* or clears it while the session lives. While compaction stays refused the
* raw ledger only grows (refused checkpoints never land), so the max bounds
* every future re-send; decaying the floor would re-open the #2490 kill
* chain. Lifetime ends with the session (metadata eviction). Known
* conservative residue: if a checkpoint lands anyway (direct-bypass path;
* the #2028 opt-out was removed, #3666) and shrinks the ledger, the floor
* stays at its old high until eviction — fail-closed over-tightening,
* accepted.
* Note (#3384): recording stays sticky; what changed is APPLICATION — the
* clamp gates where this floor may drive planning (see clampOutgoingOutput),
* so a stale high-water mark can no longer starve a healthy session. */
/** #2490: translate the recorded dsh raw-ledger scale (stamped on the session
* by the guard in handle.ts) into a token floor for the clamp's planning
* basis. bytes/4 mirrors the default estimator's chars/4 caliber (the replay
* is JSON text riding the same wires). 0 when nothing was recorded — legacy
* behavior.
* Two stamp regimes (#5289): the REFUSAL path stamps a STICKY high-water over
* the envelope bytes — while compaction stays refused the raw ledger never
* shrinks (refused checkpoints never land), so the max bounds every future
* re-send, and decaying it would re-open the #2490 kill chain. The AUTHORED
* path (#3733) stamps the EXACT bytes of the folded snapshot just handed to
* dsh — that answer genuinely shrinks dsh's store to the snapshot, so the
* floor follows reality instead of ratcheting to the pre-answer peak (which
* starved small-window sessions permanently: 128K window − 93K stale floor
* ≈ 33K max output forever while the folded wire sat at ~49K). Lifetime ends
* with the session (metadata eviction). Known conservative residue: a
* checkpoint landing through a path invisible to bili (direct bypass; the
* #2028 opt-out was removed, #3666) leaves the old high until eviction —
* fail-closed over-tightening, accepted.
* Note (#3384): APPLICATION is gated by the clamp (see clampOutgoingOutput) —
* a stale high-water mark can no longer starve a healthy session outside the
* giant-request / enforced-target regimes. */
export function dshLedgerFloorTokens(metadata: Record<string, unknown> | undefined | null): number {
const bytes = metadata?.["dshCompactionRefusedBytes"];
if (typeof bytes !== "number" || !Number.isFinite(bytes) || bytes <= 0) return 0;
Expand Down
38 changes: 23 additions & 15 deletions src/server/handle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2036,23 +2036,18 @@ async function handleRequest(
const shapeDrift = inboundMsgs !== null && inboundMsgs > DSH_COMPACTION_SHAPE_MSGS ? `, shape drifted from the ≤${DSH_COMPACTION_SHAPE_MSGS}-msg rc.1 envelope — rc.2 replays the full shadowed region (#2193)` : "";
log("warn", `[${session.id}] dsh native compaction call identified (final user message = COMPACTION_INSTRUCTION, ${inboundMsgs} msgs${shapeDrift}) — REFUSED, not forwarded: bili owns compression on this lane; a landed dsh checkpoint would durably shadow the raw history (#1729, cf. #1206/#1772)`);
}
// #2490: remember the replayed envelope SIZE. The host's raw ledger is
// invisible to bili's folded view (folds never shrink what dsh itself
// counts), so the next turn's output clamp must plan against the raw
// scale or a monster output rides a small folded input out the door
// (the 384K-token/7.5MB turn-37 single-message kill chain). High-water
// mark, not a sum: the replay is the whole ledger every time, so the
// max already bounds every future re-send; a sum would compound on each
// retry. Write-only by design: nothing decays or clears this value —
// dshLedgerFloorTokens() in budget.ts documents why the mark stays
// sticky and consumes it.
const prevBytes = typeof session.metadata["dshCompactionRefusedBytes"] === "number" ? session.metadata["dshCompactionRefusedBytes"] as number : 0;
session.metadata["dshCompactionRefusedBytes"] = Math.max(prevBytes, inboundBytes);
if (authored !== null) {
// #3733 counters + the same raw-ledger high-water as a refusal
// (the envelope still proves the raw scale the output clamp must
// plan against); NOT counted in dshCompactionRefusals.
// #3733 counter; NOT counted in dshCompactionRefusals.
// #5289: the authored answer SHRINKS dsh's raw store to the snapshot we just
// sent (dsh records our tagged checkpoint), so that snapshot IS the ride-out
// scale from now on — the planning floor follows it instead of ratcheting to
// the pre-answer raw peak. Ratcheting is exactly what starved small-window
// sessions permanently: 128K window − 93K stale floor ≈ 33K max output
// forever while the folded wire sat at ~49K. The refusal path below keeps
// the sticky high-water: there dsh's ledger really never shrinks and the
// #2490 monster-output protection must stay intact.
session.metadata["dshAuthoredCompactions"] = (typeof session.metadata["dshAuthoredCompactions"] === "number" ? session.metadata["dshAuthoredCompactions"] as number : 0) + 1;
session.metadata["dshCompactionRefusedBytes"] = Buffer.byteLength(authored.body);
markDirty(session);
if (!res.headersSent && !res.writableEnded && !res.destroyed) {
res.writeHead(200, { "content-type": authored.contentType });
Expand All @@ -2061,6 +2056,19 @@ async function handleRequest(
logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0);
return;
}
// #2490: on the REFUSAL path, remember the replayed envelope SIZE. The host's
// raw ledger is invisible to bili's folded view (folds never shrink what dsh
// itself counts), so the next turn's output clamp must plan against the raw
// scale or a monster output rides a small folded input out the door (the
// 384K-token/7.5MB turn-37 single-message kill chain). High-water mark, not a
// sum: the replay is the whole ledger every time, so the max already bounds
// every future re-send; a sum would compound on each retry. Sticky here by
// design — while compaction stays refused the ledger only grows (refused
// checkpoints never land); the authored path above resets to the snapshot
// scale instead (#5289). dshLedgerFloorTokens() in budget.ts documents
// consumption.
const prevBytes = typeof session.metadata["dshCompactionRefusedBytes"] === "number" ? session.metadata["dshCompactionRefusedBytes"] as number : 0;
session.metadata["dshCompactionRefusedBytes"] = Math.max(prevBytes, inboundBytes);
session.metadata["dshCompactionRefusals"] = (typeof session.metadata["dshCompactionRefusals"] === "number" ? session.metadata["dshCompactionRefusals"] as number : 0) + 1;
// #3013/#3022 (A3): after the once-warn above, refusals were fully
// silent — the field storm burned 5051 refusals with zero further
Expand Down
2 changes: 1 addition & 1 deletion tests/dsh-authored-compaction.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,7 @@ test("e2e: authored compaction answers locally — zero egress, tagged checkpoin
assert.equal(h.bodies.length, before, "upstream never contacted (zero egress)");
const s = h.sessionOf("s-auth")!;
assert.equal(s.metadata["dshAuthoredCompactions"], 1, "authored counter");
assert.ok(typeof s.metadata["dshCompactionRefusedBytes"] === "number", "raw ledger high-water still marked");
assert.equal(s.metadata["dshCompactionRefusedBytes"], Buffer.byteLength(r.body), "#5289: floor follows the snapshot we sent, not the ~600K pre-answer envelope peak");
assert.equal(s.metadata["dshCompactionRefusals"], undefined, "not counted as a refusal");
});
});
Expand Down
Loading
Loading