kv-stable: pyramid deepens only on merge-threshold cadence — no demand path when the full-fold floor crosses the hard wall (recurring OverBudget wedge windows, empty merge queue at every exhaustion)

[Anarchid]

Summary

At a tight operating point, kv-stable's pyramid can only deepen on threshold merge cadence (6^k accumulation), and no component can demand a deeper merge when the full-fold floor crosses the hard wall. When history grows faster than the cadence delivers relief, the agent enters recurring OverBudget wedge windows — in the deterministic repro below, 39 of 80 compile points threw across turns 105–400 (~half the agent's lifetime unreachable), and the merge queue was empty at every single exhaustion: the system is wedged while holding nothing that could unwedge it.

Trivially easy to trigger. Found by accident while replicating #37 — the harness wedged before the cache measurements finished.

Distinct from #50 and #56

• kv-stable emits no produce ops → permanent infeasibility at tight operating points (L1-holdback deadlock) #56 (closed): solver emitted no produce ops when foldable chunks had no L1 coverage; the fix (PR fix(kv-stable): demand-side L1 production under escalation (#56) #61) emits L1 requests for uncovered ranges under escalation — and only L1.
• kv-stable: from-scratch ideal cut never folds past L3 — L4/L5 summaries unused; bootstrap after resolutions wipe can OverBudget-wedge an agent #50 (open): deep summaries exist (intact L4/L5 chains) but the from-scratch ideal cut won't use them.
• This issue: L1 coverage is complete (so the kv-stable emits no produce ops → permanent infeasibility at tight operating points (L1-holdback deadlock) #56 path demands nothing) and deeper summaries do not exist and cannot be demanded — they can only arrive when ≥mergeThreshold orphans accumulate at a level. Between cadence events, every compile hard-fails.

Mechanism (code pointers, master @ 0.6.3)

1. Merge production is exclusively cadence-driven. checkMergeThresholdRecursive (autobiographical.ts:~5798) — the "speculative pre-producer" that kv-stable.ts's header defers deeper folding to — enqueues an L_{k+1} merge only when ≥mergeThreshold (default 6) orphans exist at level k (a relaxed ≥2 applies only to non-newest runs, and a single contiguous run is always the newest, so 6 it is).
2. Escalation demands only uncovered L1s. The kv-stable emits no produce ops → permanent infeasibility at tight operating points (L1-holdback deadlock) #56 fix in kv-stable.ts (~:165–185) emits produce ops solely for foldable ranges with no L1. With full coverage, result.produced is empty on every failing compile.
3. The shadow pre-producer inherits the limitation. productionBudgetTokens (autobiographical.ts:~6836) runs the same solver at a tighter budget — its produce ops are the same L1-only escalation set, so it cannot deepen the pyramid either (it also defaults to unset).
4. Relief is deferred by design, but nothing is ever deferred-to. The hard-fail site (~:6895) correctly notes "a produce op only schedules a missing summary; it does not make the current raw plan feasible" — relief must come from later drains. But at every observed exhaustion mergeQueue=0, compressionQueue≤1: there is nothing draining toward relief.

Deterministic repro (LLM-less, no API key)

Built on test/_harness/strategy-runner.ts mechanics (real Membrane + MockAdapter + NativeFormatter, real Chronicle store):

• Strategy: { adaptiveResolution: true, foldingStrategy: 'kv-stable', compressionModel: 'mock', targetChunkTokens: 1500, headWindowTokens: 0, recentWindowTokens: 3000 } (defaults otherwise: mergeThreshold 6, speculativeProduction true, productionBudgetTokens unset)
• Workload: generateWorkload({ turns: 400, avgUserTokens: 200, avgAssistantTokens: 200, toolCallProbability: 0, seed: 7 })
• Compile every 5th turn at { maxTokens: 11_000, reserveForResponse: 1_000 } → hard wall 10,200
• Mock compressor returns summaries whose length scales with the consolidated input (clamp(0.08 × input_tokens, 180..700) words) — matching observed live-fire summary sizes; fixed short summaries mask the wedge by keeping the fold floor artificially low

Result: first failure at turn 105; 39/80 compiles throw through turn 400, interleaved with 21 successes — a sawtooth where each cadence merge briefly dips the floor under the wall and growth re-crosses it. Representative failures with the pyramid at that instant (orphan/total per level):

turn 105: exhausted 10506 > 10200 (deepest fold L2)   pyramid L1:4/10  L2:1/1        mergeQ=0
turn 175: exhausted 11158 > 10200 (deepest fold L2)   pyramid L1:1/19  L2:3/3        mergeQ=0
turn 205: exhausted 15247 > 10200 (deepest fold L2)   pyramid L1:5/23  L2:3/3        mergeQ=0
turn 400: (final)                                     pyramid L1:5/47  L2:1/7 L3:1/1 mergeQ=0

At turn 175 the plan needs L3; only 3 orphan L2s exist (6 required); nothing is queued and nothing can be demanded. The first L3 arrives by cadence around turn ~330 and does not end the failure windows — raw growth outruns each relief.

Confirmed against the real API as well (Haiku 4.5, same geometry, 200 turns): identical signature from turn 175 on — Adaptive picker exhausted but 11850 tokens still exceed hard budget 10200 (… deepest fold level=L2) — while an in-process #37 cache replication was running; the wedge is not a mock artifact.

Why this matters operationally

This is the "agent unreachable, every compile throws" failure mode of #50, but reachable without a resolutions wipe, a config change, or any store surgery — steady organic growth at a tight budget is sufficient, and the windows recur indefinitely. productionBudgetTokens — suggested in #50 as insurance — does not help here, since the shadow pick cannot demand merges either.

Suggested direction

Under escalation (plan.escalated), when the deepest existing level still exceeds the hard wall, emit demand-side merge produce ops for the deepest-level orphan runs — accepting short-count merges (< mergeThreshold) in that mode, mirroring how #56 accepted demand-side L1 production as the lesser evil versus a hard wedge. Acceptance: the repro above completes 400 turns with zero failed compiles, and every exhaustion that does occur leaves a non-empty production queue behind (i.e., the system is always draining toward relief).

Diagnostics side-note: #59's "deepest AVAILABLE vs deepest PLANNED" would have made this distinguishable from #50 at a glance; this issue is a second motivating case for it — here available == planned == L2, and the missing datum is why nothing deeper is coming.

🤖 Generated with Claude Code