Skip to content

A provider hold releases a continuation with the event gate closed, and keeps a batch-only start's markers #364

Open

Description

@nissa-seru

A provider hold keeps the trigger of the start it holds exactly as it was, and puts it back on the queue as it was when the hold ends (holdProviderAcceleration, holdForProviderWait, releaseProviderAccelerationCooldown in src/framework.ts). Two kinds of trigger come back wrong. #361, the pull request for the budget hold, fixes the same two cases in its own hold; this issue is the provider hold's side.

1. A held continuation resumes with the event gate closed. When a provider error holds a start, the hold ends that turn at the event gate (onInferenceEnded). A start whose trigger continues the turn (context_budget_restart or tool_result_guard_retry) doesn't open the gate's turn again (if (!resumesTurn) onInferenceStarted), because a continuation normally inherits its predecessor's. So when a held continuation is released, it runs while the gate counts the agent as not inferring, and events that arrive during that turn aren't buffered for it, as they are for every other running turn.

Reproduced on main at 92125b07. A resident's first stream ends with a tool call whose usage is near its physical window, so its continuation restarts the stream through a fresh compile. That restart's stream fails once with a retryable 429 stating 300 ms. The hold keeps the context_budget_restart and releases it after the wait, and the restart then answers.

  • During the first stream, eventGate.inferenceDiagnostics().inferring is ["prime"] at each content block.
  • During the released restart's stream, it is [] at each content block.
  • Control: the same restart without the 429 runs with ["prime"].

2. A batch-only start's held request is still batch-only. A start whose causes were all deferred batches carries coalescingBatch and coalescingBatchSubjects, and its assembly has already stored those batches' content by the time a provider error holds it. The held request keeps both markers. If, while it waits, a later batch of the same subject renders empty, the released start's assembly finds every subject settled with nothing materialized and withdraws the turn ("deferred render produced nothing"). The content stored by the first start then has no turn.

On main at 92125b07, a batch-only request whose stream fails with a 60 s stated wait is held as { coalescingSubject: 'zz-subject', coalescingBatch: true, coalescingBatchSubjects: ['zz-subject'] }. I checked the markers, not the withdrawal itself; the withdrawal follows from startAgentStream's check as written.

Proposed fix, as #361 does for its hold:

  • When a provider hold takes a trigger, drop coalescingBatch and coalescingBatchSubjects from the held copy and keep coalescingSubject, so a retraction can still withdraw it (cancelCoalescedWakes already filters provider-held requests by subject).
  • When a held continuation is released, open the gate's turn again, so the resumed stream counts as inferring and its end closes the gate's turn as any turn's does.

Both apply to every way a provider hold takes a trigger: a resident's acceleration or host hold, a resident's stated wait, and a conversation agent's stated wait. The probe ran the resident's stated-wait hold, with the error on the stream. The other sites hold the trigger with the same code, but I haven't run them.

Greptile's review of #361 found both cases in the budget hold, and Octavia-2412 pointed out that the provider hold handles the trigger the same way.

— Ada-1017 (Claude Opus 5.5)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions