fix(opencode): recover turns after event stream EOF - #2272
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| private async recoverForegroundPendingRequests(turnId: string): Promise<boolean> { | ||
| const requestEvents = await this.readForegroundPendingRequestEvents(); | ||
| let hasPendingRequest = false; | ||
| for (const event of requestEvents) { | ||
| const translated = await this.translateEvent(event); | ||
| for (const translatedEvent of translated) { | ||
| if (isOpenCodeProviderInternalEvent(translatedEvent)) { | ||
| this.notifySubscribers(translatedEvent, null); | ||
| continue; | ||
| } | ||
| this.notifySubscribers(translatedEvent, turnId); | ||
| } | ||
| hasPendingRequest ||= this.pendingPermissions.has(event.properties.id); | ||
| } | ||
| return hasPendingRequest; |
There was a problem hiding this comment.
recoverForegroundPendingRequests calls notifySubscribers for every pending permission it finds without checking seenPermissionRequestIds. When a permission was already delivered via the live stream (which adds the ID to seenPermissionRequestIds), and the stream then drops, this recovery path emits it again — once for the bottom-of-loop reconcile call and again for the top-of-loop reconcile on the reconnected stream. Subscribers receive the same permission_requested event two or three times. The check added to the regular stream path (in consumeOpenCodeStreamEvent) should also be applied here before calling notifySubscribers.
There was a problem hiding this comment.
Verified against the code and this one isn't reachable: the seenPermissionRequestIds dedup lives inside translateEvent itself (not consumeOpenCodeStreamEvent), and recoverForegroundPendingRequests routes every re-listed request through translateEvent, so a permission already delivered on the live stream produces an empty translated list during recovery — notifySubscribers is never reached for it. The reconnect decision is unaffected because hasPendingRequest reads pendingPermissions, not the emitted events. Added an explicit regression test for the exact scenario (permission delivered live, stream drops, permission.list returns it again): "does not re-emit a live-delivered permission re-listed during stream recovery" asserts exactly one permission_requested across the turn.
- trace + warn when a post-reconnect reconcile is unresolved (review) - document why seenPermissionRequestIds is session-lifetime (review) - settle eventStreamReady when close aborts the initial connect - abort-aware reconnect backoff and reconcile reads so close() is not blocked - per-part text bookkeeping: suppress buffered deltas replaying recovered text, and bridge every assistant message in multi-message turns - route recovered tool parts through subagent registration - reset accumulatedUsage after recovered completions like live ones - serialize stream errors in trace payloads so pino keeps name/message - snapshot pre-turn message ids after turn activation to keep terminal events consumed during startTurn from being dropped Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Superseded by #2684, which preserves the bounded EOF-recovery outcome but is rebuilt directly from the latest stable release (v0.2.5), split from unrelated fleet/material-progress work, and independently re-reviewed. I am closing this older branch without rewriting its history so the original discussion and evidence remain available. |
1 participant
Linked issue
Closes #2271
Type of change
What does this PR do
Since #916, the OpenCode adapter fails the active foreground turn the moment the global event stream (
client.global.event,sseMaxRetryAttempts: 0) ends or errors — one dropped SSE connection kills an otherwise healthy turn, even though the OpenCode session keeps running and usually completes. Details and evidence in #2271.This PR makes stream EOF recoverable instead of fatal:
session.status,session.messages, and pending question/permission requests before deciding anything.The stream remains the source of truth. The messages API is consulted once per disconnect to bridge the gap — this is not a return to the pre-#916 polling path.
How did you verify it
Reproducing the bug: on my daemon (Paseo 0.1.110, OpenCode 1.18.3, macOS arm64), host load reliably triggers it: 9 concurrent OpenCode agents on one
opencode serveproduced 9 turn failures in ~5 minutes, allprovider.opencode.stream.eoftraces, whilesession.statusstill reported the sessions busy (log excerpts in #2271).Unit tests:
New coverage includes: reconnect while busy (stream EOF mid-turn → turn completes with full text), recovery of the missing assistant suffix when EOF finds the session idle, pending-permission recovery across EOF, failure when reconciliation is impossible, usage recovery, and reconnecting through consecutive failed connects (backoff path).
Production soak: this change (cherry-picked onto
v0.1.110) has been running my real daemon workload — ~19 OpenCode agents, same load pattern that triggered the failures — since 2026-07-20. No EOF turn failures since deployment.Also ran: server-package
npm run typecheck(clean),npm run linton changed files (clean),npm run format:check(clean). I did not run the*.real.e2esuites (they need live model access).Disclosure: this patch was developed with AI assistance (Claude Code), then verified as described above — the test suite, the load repro, and the production deployment are real runs on my machine, not generated claims.
Checklist
npm run typecheckpassesnpm run lintpassesnpm run formatran (Biome)🤖 Generated with Claude Code