Skip to content

fix: bound every await in the OpenAI/Codex reviewer path - #24

Merged
Suprhimp merged 1 commit into
masterfrom
duul-errors-investigation
Jul 31, 2026
Merged

fix: bound every await in the OpenAI/Codex reviewer path#24
Suprhimp merged 1 commit into
masterfrom
duul-errors-investigation

Conversation

@Suprhimp

Copy link
Copy Markdown
Member

Problem

Reviews on the Codex/ChatGPT (stateless) backend went silent for 565s/750s with zero [duul] Retry logs — longer than the theoretical 363s worst case of duul's own 3×120s retry loop.

Root causes

  1. SDK defaults were live. buildClient passed no timeout/maxRetries, so the openai SDK used its defaults: 600s timeout + 2 silent internal retries. Multiplied by duul's 3 attempts, one flaky connection could burn up to 30 minutes with no log output. This is where the 565s/750s went.
  2. Abort/connection errors were never retryable. isRetryable matched error.name === 'AbortError', but the SDK's APIUserAbortError/APIConnectionError don't override name (it stays 'Error'). So even when the 120s abort fired, the error was thrown immediately without retry — explaining the missing Retry logs.
  3. The Codex OAuth refresh fetch was unbounded. On a 401, await this.refresh() runs outside the per-attempt AbortController and its fetch had no signal — a stalled OAuth endpoint hung the entire review forever, silently. This is the only code path consistent with an eternal single-await hang.

Fix

  • timeout: 120_000, maxRetries: 0 on the OpenAI client — duul's loop is the single owner of retries, and every retry is logged.
  • Retryability keyed on missing HTTP status (abort/connection failure) instead of error.name.
  • AbortSignal.timeout(30_000) on the Codex token refresh fetch.
  • Stateless stream: explicit Promise.race against the 120s abort + stream.abort() in finally, as a backstop for mid-SSE stalls.
  • Observability: log before each API call (→ openai stream (attempt 1/3, timeout 120000ms)) and each tool execution, plus the three previously-silent tool-loop continue paths (cache hit / repeat limit / budget block). A stall now names its await.

Anthropic/Google providers are untouched: they use native fetch, whose aborts throw real AbortError DOMExceptions, so their existing checks work.

Test

New regression test injects a stream that ignores the request signal and never ends; asserts the call rejects at the review deadline and the stream is aborted. 97/97 tests pass.

🤖 Generated with Claude Code

Root causes of the silent 565s/750s review hangs:

- buildClient left SDK defaults active: 600s timeout + 2 silent internal
  retries, multiplying with duul's own 3 attempts into up to 30min of
  unlogged waiting. Now timeout: 120s, maxRetries: 0 — duul owns retries.
- isRetryable matched error.name === 'AbortError', but the SDK's
  APIUserAbortError/APIConnectionError keep name 'Error', so aborts and
  connection failures were never retried (hence zero Retry logs).
- The Codex OAuth refresh fetch had no timeout and runs outside the
  review AbortController — a stalled endpoint hung the review forever.
  Now bounded at 30s via AbortSignal.timeout.
- Stateless (ChatGPT backend) streams now race the 120s abort explicitly
  and abort the SDK stream controller in finally, covering mid-SSE stalls.
- Log before each API call / tool execution and on the previously-silent
  tool-loop continue paths (cache hit, repeat limit, budget block).

Adds a regression test that injects a hanging stream and asserts the
call fails at the deadline instead of awaiting forever.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Suprhimp
Suprhimp merged commit 46e9c7a into master Jul 31, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant