[nissa-seru]

Problem

Follow-up to #138, for the two minor findings of the maintainers' round-2 review (R2-1, R2-2).

• R2-1: a run of transient failures outlived every call that could end it. While a chunk has a run, pendingRetrySpans treats its span as a hole a retry will fill, so contiguousMergeCandidates keeps the summaries before it waiting rather than merging them as an interior run. That hold means this process expects to retry the chunk. A run ended only when its own rung answered or decided, or when its chunk stopped being pending, and a quarantined chunk stays pending. So when another process quarantined the request this process's run was retrying, tick skipped the chunk with its run still standing, and the merges beside the span waited for the life of the process.
• R2-2: the spec's "Durable refusal quarantine" paragraph said a family exhausts when every variant "throws a provider/validation error". It also described only one of the ways a family exhausts, which the review didn't raise.

Changes

• src/strategies/autobiographical.ts:
  • Where a chunk's runs end. The expectation of a retry ends where the chunk compresses or its request can't go again, so tick, which every attempt's outcome passes through, ends the chunk's runs there (endChunkFailureRuns):
    • The chunk compressed: a mint, an adoption, the dedup guard, a thin stub, or the L1 of a call it joined. tick reads this on the chunk object it took, which a mint marks even where a rebuild during the call cut another. It ends runs by the chunk's key, its sources, which both objects share.
    • The request was found or made unable to go. compressChunkHierarchical takes a fourth, optional notification, onRequestStopped, and gives it at three places:
      • an exhaustion whose record the ledger holds, written by this ladder ('written') or found already written by another process ('held');
      • a call that joined another instance's ladder, where the reloaded ledger holds its family by the gate's own test, now one method (durableQuarantineFor) that the gate and the joined call share;
      • the overlap guard, which refuses every attempt while the overlap stands.
    • tick's quarantine skip ends them too.
    • Elsewhere runs stand, since a retry is coming: tools not yet pushed, a family dropped while its ladder ran, or a hold placed during a joined call, which discarded another instance's answer. This ladder's own answer ends its rung's run when it comes, before a hold can discard it, so no run stands for such a hold.
  • Merge discovery runs again after every attempt, whether its call returned or threw, and after a skip that ended a run. A rung's answer can end the last run inside the ladder before a later call throws, so the pair that run held is found at once rather than at the next landing. After a thrown attempt, a failure of discovery's own is logged, and the attempt's error is still the one thrown: tick's callers act on it (a withdrawal that a host counts as no failure, a window wait after which the scan goes on), and a merge discovery queued is in memory before the write that failed.
  • A rung's answer ends its runs inside the ladder, as a fresh answer already did:
    • An answer reused from the family's progress ends them too. Another ladder, perhaps another process's, decided the rung, so a run this process opened on it before then stood, and a failure of that rung's next request, once the frontier moved, counted onto it.
    • A carrier rejection ends the run of the request that carried the reasoning, whether it comes now or is reused. That request's failures ran under the rung's own label, and the outcome the rejection notes, under the carrier label, ended no run.
    • A split-stitch that answers ends the run a failure inside it opened under the rung's own label. Its pieces' answers, each under a label of its own, never ended it, so where a hold placed during its calls kept the summary from being written, it stood.
    • Split-stitch's outcome reused from progress ends nothing itself: the family's exhaustion follows it with no call in between, and tick ends the chunk's runs where that exhaustion stops the request. The runs field's and endRungRuns' docs name this exception.
  • The docs:
    • The runs field, endChunkFailureRuns, pendingRetrySpans, the exhaustion's doc and tick's comment say where runs end and where they stand. tick's comment also says that a dropped family's recorded outcomes go with it, while the chunk's runs carry into the next attempt with their counts. contiguousMergeCandidates no longer calls a quarantined chunk's hole stranded for good: it's stranded until a later request shape or a clear lands it, and once tick's skip ends the chunk's runs, the interior run before it merges at two.
    • The failure classifier and the ladder's progress comment say a transient failure spends no rung by itself, since COMPRESSION_PERSISTENT_FAILURES of them decide one.
    • The progress docs say split-stitch is re-run whole after a transient failure inside it.
    • Older comments say what the code does: a tool-prose rung, after which the canonical's plain-prose retry is skipped, follows source-only-final's refusal as well as the canonical's, so it can follow an incomplete canonical answer; the split rung's root is source-only-final's request, which gave no summary, whatever its outcome, though their instructions differ where the chunk is part of a read document; and the quarantine gate's cap is per chunk and request regime, cleared for the whole span by the success-clear and the paid-off sweep, while an operator's clear by key clears one shape and leaves the span's others counting.
• test/overbudget-diagnostics.test.ts: its override of compressChunkHierarchical passes the fourth argument on.
• test/compression-recall-curve.test.ts, test/source-only-l1.test.ts and test/compression-transient-ladder.test.ts: three test titles say what their tests show. The recall-curve test's title says two transport failures with nothing landing between them quarantine nothing, which is what it runs.
• changelog.d/compression-transient-ladder.fixed.md (fix(compression): progress past quarantined chunks; transient failures never spend the ladder; pace the lane #138's fragment): its outage sentence no longer says an outage never decides anything. An outage adds at most one failure to a run however long it lasts, and that one decides the rung where it is the run's eighth.
• docs/compression-recall-curve-fallback-spec.md:
  • The quarantine paragraph names the three ways a family exhausts:
    • every later rung the ladder reaches failing after a refusal or an incomplete answer. The canonical's tool-prose hoist runs only when the canonical's own first call refused, so a refusal of its no-tools or tools-less retry leaves the stage incomplete;
    • source-only-final, where it's configured, after a request-attributable rejection (or at once where it isn't);
    • an accepted answer that strips to empty. After the canonical stage's, a recall variant's or split-stitch's answer, the canonical's plain-prose retry is asked once; after source-only-final's or a tool-prose hoist's, the family exhausts at once.
  • The errors that exhaust a family are the request-attributable ones (context_length, invalid_request, safety, unsupported) or one its failures decided. A transient failure ends no rung by itself, and COMPRESSION_PERSISTENT_FAILURES of them on one rung decide it. An abort ends no rung and records nothing.
  • The algorithm no longer says the canonical's non-refusal behaviour is unchanged.
  • Invariant 10 names two exceptions to "never requested again": split-stitch, which is re-run whole after a transient failure inside it, and a compression hold placed during a call, which discards the call's answer before anything records it, so the chunk is asked for again once the hold is released. Its outage sentence says what the fragment's now does.

What each part is for

• Ending runs in tick, on what the call reports. A run's hold is this process's expectation of a retry. Every attempt's outcome passes through tick, so the rule lives in one place, and a return the code adds later doesn't need its own call to keep a hold from outliving its chunk.
  • Why not every resolved call: a return that decided nothing (tools not yet pushed, a family dropped meanwhile, a joined call whose answer a hold discarded) leaves a retry coming.
  • Why not only where a quarantine is found or made: a chunk that compressed, by a mint on an object a rebuild replaced or by a joined call's L1, would keep its runs until the next rebuild's prune.
  • Cost to keep: a nine-line helper, one optional parameter and its three calls, and the gate's test as a method.
• The shared quarantine test. A joined call and the gate then agree on what stops a request.
• Rerunning merge discovery. Without it, the pair a run held waits for the next landing elsewhere. One call where tick's call returns, one where it throws.
• Ending a rung's runs at its answer, however it comes. A run counts failures of a rung that hasn't answered. One helper ends both of a rung's runs (with and without its reasoning carriers), and it's called where an answer is noted, where one is reused and where split-stitch completes. The carrier rejection's line ends the run of the request that carried the reasoning, now for a reused rejection too. Cost to keep: a three-line helper, two calls of it, and one line moved out of a branch.
• What this gives up. A later request shape the durable gate lets through, a retry after a clear, or one after the overlap is mended starts its count afresh. If the pair beside the span merged meanwhile, its summary lands beside their parent, as after a restart: a lopsided merge, with nothing lost. A clear lifts a quarantine and ends no run.
• The tests. Each pins one way a run ends here, or doesn't. One also pins the hold rule above a retry's level, which an internal review found untested.
• The texts. Each says what the code now does.

Other open pull requests

This and #245 conflict in two places in src/strategies/autobiographical.ts. #215, which conflicts with main in ten places there, gains an eleventh with this change.

Tests

test/compression-transient-ladder.test.ts:

• "a chunk whose request is quarantined leaves its runs":
  • A skip for another process's quarantine. Four chunks: the first two land, the third fails with a 503 and opens a run, and the fourth lands. A second strategy on the same store gets a context_length rejection for the third chunk's request and quarantines it. The first strategy's next tick skips the third chunk. Its run is gone, and the pair of L1s before the span merges into an L2 in the same tick. The model is asked for that merge, never for the third chunk again.
  • An exhaustion this ladder writes. Runs are seeded under keys of rungs other than the one that exhausts, two for the first chunk and one for the second. The first chunk's canonical request meets context_length, and its family exhausts. Both of its runs end, and the second chunk's stays.
  • An exhaustion after the rung's own rejection ended its run. The third chunk's 503 opens a run. Its retry, after the fourth lands, meets context_length. The rejection ends the run as its outcome, and the family exhausts. No run is left, and the pair of L1s before the span is queued in the same tick.
  • Past the speculative cap, a skip that queues an L1 merge. The skip test's first steps under a cap of two unmerged L1s, so the fourth chunk's landing puts three past it with no L1 merge queued. The third chunk is no longer failing, as a return that decided nothing leaves it, and keeps its run; a second strategy quarantines its request; and a fifth chunk is queued behind it. The next tick skips the third, which ends its run and queues the pair before its span. The fifth isn't compressed: the queued merge runs in the same tick instead.
  • A call that resolves without a notification, as an override that predates the fourth parameter does: the chunk's run stands.
• "a chunk's runs end where this process won't retry its request, and stand where a retry is coming":
  • A tick that joins another instance's call. The skip test's first steps, then a second strategy on the same store asks for the third chunk, and its call is held until the first strategy's tick has joined it. Released, it meets context_length, and its family exhausts. The joined tick made no call of its own. Its run is gone, and the pair before the span is queued.
  • The overlap guard. An L1 seeded over the first chunk's first message, and a run on the chunk. The tick makes no call, and the run is gone.
  • Tools not yet pushed. A first chunk holding a tool call and its result, with no tools pushed, and a run on it. The tick makes no call, and the run stands.
  • A chunk that compresses while a rebuild cuts it again. A run seeded on another rung of the first chunk. Its call is held, a message is added during it, and the call answers. The chunk's L1 is written, and the run is gone.
• "a family dropped while its ladder ran is not quarantined by its exhaustion": the skip test's first steps with source-only-final on. The third chunk's retry: its canonical is refused, which ends its run, and the family is dropped as source-only-final's call arrives.
  • With no other run, the pair before the span is queued in the same tick.
  • With a run an earlier rung left, that run stands and keeps holding the pair.
• "every rung keeps its own identity across a resume": a 503 on the request carrying reasoning opens its run. At the next attempt the carrier rejection is followed by the stripped retry's answer, and the run is gone.
• "a rung's answer ends its runs, whether it came now or is reused from progress":
  • A reused refusal, for a run of the request as sent and for one of its stripped copy. The canonical is refused and source-only-final's call fails, so the refusal is kept in progress. A run is seeded on the canonical, as this process's own failures leave one before another ladder's refusal reaches the progress. At the next attempt the canonical isn't asked again, source-only-final's call is withdrawn by the host (which ends no run), and the canonical's run is gone.
  • A reused carrier rejection. The carrier request is rejected and its stripped retry fails. A run is seeded on the carrier request. At the next attempt the carrier request isn't sent again, the stripped request is withdrawn, and the run is gone.
  • A split-stitch that answers. The canonical and source-only-final refuse, and the split rung's first piece fails, opening the rung's run. At the next attempt both pieces answer, and a hold placed during the second keeps the stitched summary from being written. The run is gone.
  • A tick whose later call throws. The skip test's first steps with source-only-final on. The third chunk's retry, after the fourth lands: its canonical is refused, which progress keeps, and its source-only-final call fails with a 503. A run is seeded on the third chunk's canonical. Its next tick reuses the refusal, which ends the run, and source-only-final's call is withdrawn, so the tick throws. The pair before the span is queued all the same.
  • A discovery that throws after a failed attempt. A chunk's call is withdrawn, and merge discovery is made to throw. tick throws the withdrawal, and the discovery's failure is logged.
• The joined call, two more ways: the second strategy's call lands the chunk's L1, and the joined tick ends the run; or a hold placed during that call discards its answer, so it ends with neither an L1 nor a quarantine, and the run stands. The first joined-call test now also checks that the tick took no quarantine skip, which ends runs too.
• "an exhaustion whose key another process quarantined meanwhile …" (both cases): the record another process writes during the second ladder's last call makes its exhaustion find it ('held'), and the test now also checks that this stops the request.

In "a hole that a pending retry will fill …", the test now also pins that a chunk's L1 retry holds the hole among L2 summaries, as #138's code already does.

Checks (Linux x86_64, Node 24.22.0), at this head, one commit on main 0547be76:

• The head: typecheck and build are clean, and npm test passes 2,184 of 2,184.
• Against main's source, with main's overbudget-diagnostics test, whose override this change widens: 18 of the 430 tests in the five files below fail, all of them tests this change adds or changes in compression-transient-ladder.test.ts. They are the skip, the past-the-cap skip, four exhaustions (this process's, one after the rung's own rejection, and both of one another process wrote first), the joined call's two endings, the overlap guard, the chunk compressed while a rebuild cut it, the dropped family's case with no other run, both carrier tests, both reused refusals, the split-stitch, the thrown tick and the discovery failure. The tests that pin where a run stands pass there: the call that resolves without a notification, tools not yet pushed, the run an earlier rung left, the joined call that ends with neither an L1 nor a quarantine, and the hole test.
• Each piece taken out on its own fails at least one test in the five files (compression-transient-ladder, speculation-cap, overbudget-diagnostics, compression-recall-curve and source-only-l1), 21 pieces in all:
  • tick's quarantine skip ending the chunk's runs, and the rediscovery after it: the skip test and the past-the-cap skip test;
  • endChunkFailureRuns matching the chunk: eleven tests; going on past the first match: the exhausting family's test;
  • the hole rule's "or above": the hole test;
  • the overlap guard's notification: its test; a joined call's: the joined call that's quarantined; the exhaustion's: the exhausting family's test and both tests of an exhaustion another process wrote first;
  • tick ending a compressed or stopped chunk's runs: five tests; its merge discovery after every return: three;
  • the carrier rejection's end: both carrier tests, and its end for a reused rejection: the reused one;
  • the end at a reused answer: both reused refusals and the thrown tick; at a complete stitch: the split-stitch test;
  • the exhaustion's notification for one another process wrote: both of those tests; the joined call's quarantine condition: the joined call that ends with neither an L1 nor a quarantine; the joined L1 marking the chunk compressed: the joined-L1 test and two concurrency tests;
  • the discovery after a thrown attempt: the thrown-tick and discovery-failure tests; the try around it: the discovery-failure test;
  • the stripped request's run: the stripped reused-refusal case; the cap's check before each candidate: the past-the-cap skip test.
• Controls: a notification put where a retry is coming fails the test that pins it standing: after a dropped family, the run an earlier rung left; for tools not yet pushed, the tools test. And with the joined-call tests' tick started after the other instance's call has settled, both joined-call cases fail, so they do reach the join.

Not verified

• The spec paragraph: it was checked by reading the ladder, not by a run of each exhaustion path. The existing test "canonical rejection → source-only-final once" covers the second path's first half.

No changelog entry of its own: no release carries #138 yet, and its fragments describe the behaviour this completes. This change corrects one sentence of #138's fragment, the outage's.

— Oscar-1005 (Claude Opus 5.5). Myrtle-2415 (Claude Opus 5.5) read its last round, and internal reviews read the rounds before it. Checked and opened by Fiona-2757 (Claude Opus 5.5) through the shared nissa-seru account.

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling.

[greptile-apps]

Required label not found on this PR.