Improved ChatGPT/OpenAI-compatible connection rate-limit handling, caching, and diagnostics

CHANGELOG.md

@@ -1,3 +1,10 @@
+- Fixed the OpenAI-compatible and ChatGPT Subscription connection types silently ending a prompt with no visible response when the provider returned an empty completion (typically because the conversation grew too large for the model's context/output limit) — Claude Code now shows an explanatory message, including the size of the request that triggered it, instead of just returning to the idle prompt
+- Added a "Session Statistics…" context menu item (right-click the prompt area or the terminal) showing cumulative prompt-cache and token-usage statistics plus the size of the last request, broken down per model used during the session; only available for sessions using the OpenAI-compatible or ChatGPT Subscription connection types
+- Added an experimental "Explicit Cache" option per model in the Model Aliases dialog (for the OpenAI-compatible and ChatGPT Subscription connection types), enabled by default for GPT-5.6-family models, to reduce token costs on long conversations where that model family's automatic caching is unreliable; older GPT models get an extended 24-hour cache retention instead
+- Fixed prompt caching for the OpenAI-compatible and ChatGPT Subscription connection types not surviving "Continue last session"/"Resume session", by keying the cache to Claude Code's own session id instead of an internal id that changed on every restart; also enabled prompt caching for the OpenAI-compatible connection type, which previously had none
+- Clarified in the user manual that the `/usage` command shows only local, per-session statistics and never reflects ChatGPT subscription quota, since it does not query any account-wide usage endpoint
+- Fixed the OpenAI-compatible and ChatGPT Subscription connection types not clearly reporting rate-limit/quota errors from the provider — Claude Code now shows the actual cause immediately instead of silently retrying up to 10 times when a ChatGPT subscription's usage limit is exhausted for an extended period
+
 # 1.3.5 (2026-07-19)
 
 - Fixed Ctrl++/− terminal and Markdown Preview zoom shortcuts not working when using the numpad +/− keys

claude-launch-tests/README.md

@@ -18,6 +18,10 @@ All scripts unset CLAUDECODE and CLAUDE_CODE_ENTRYPOINT to avoid the
 | `test_term_env.py` | Does presence/absence of TERM affect output buffering? |
 | `test_stdin_close_after_output.py` | Can we send a response via stdin if we close it after receiving output? |
 | `test_interactive_choices.py` | Can claude ask clarifying questions and receive answers via stdin in --print mode? |
+| `test_429_retry_headers.py` | Does a response header stop Claude Code CLI's built-in 429 retry loop? |
+| `test_usage_command_endpoints.py` | Does the `/usage` slash command hit a live network endpoint? |
+| `test_session_id_header_stability.py` | Does `X-Claude-Code-Session-Id` stay stable across `--continue`? |
+| `test_system_prompt_prefix_stability.py` | Does the `system` field stay byte-identical turn-to-turn (needed for cache breakpoints)? |
 
 ## Conclusion: interactivity in --print mode
 
@@ -32,3 +36,19 @@ All scripts unset CLAUDECODE and CLAUDE_CODE_ENTRYPOINT to avoid the
 of the CLI, not a PTY issue. PTY is only needed for permission prompts when running without
 `--dangerously-skip-permissions`. For the plugin, `--dangerously-skip-permissions` is an acceptable
 default (with a warning in the UI), deferring PTY support to a later optional stage.
+
+## Conclusion: 429 retry suppression
+
+| Response header | Requests before giving up | Wall time |
+|---|---|---|
+| none (baseline) | 8+ (still retrying at 90s test cap) | 90s+ (would reach ~10min for all 10 attempts) |
+| `Retry-After: 0` | 8+ (behaves like baseline) | 90s+ |
+| `Retry-After: 999999` | 1 | ~0.1s |
+| `X-Should-Retry: false` | 1 | ~0.1s |
+
+**A large `Retry-After` value or an explicit `X-Should-Retry: false` header both make Claude Code
+CLI give up immediately** instead of retrying up to 10 times, surfacing
+`{"error":"rate_limit","is_error":true,"api_error_status":429,...}` right away. A small
+`Retry-After` value does not suppress retries. Applied in
+`OpenAIProxyServlet.setRetryAfterIfHardLimit()` — only for genuine hard-quota-exhaustion errors
+that carry a `resets_in_seconds` field, not for ordinary transient 429s.

claude-launch-tests/test_429_retry_headers.py

@@ -0,0 +1,143 @@
+"""
+Hypothesis: Claude Code CLI's built-in retry behavior on HTTP 429 responses can be
+influenced by response headers (Retry-After, or a should-retry-style convention),
+so a proxy could tell the CLI "don't bother retrying, this is a hard quota limit"
+instead of the CLI blindly retrying up to 10 times with its own backoff schedule.
+
+Background: the NetBeans plugin's OpenAI-compatible proxy already returns a clean
+Anthropic-format `rate_limit_error` on 429 (see OpenAIProxyServlet.toCodexAnthropicError),
+but captured logs from a real session still showed the CLI retrying
+"attempt 1/10" .. "attempt 10/10" with 1s-39s backoff before giving up — even though
+the underlying quota wouldn't reset for ~29 days. This test probes whether any
+response header changes that behavior, run against a local fake server standing in
+for ANTHROPIC_BASE_URL (same mechanism the plugin's proxy uses).
+
+Run: python3 claude-launch-tests/test_429_retry_headers.py
+
+Result: CONFIRMED — both mechanisms work.
+- baseline (no extra headers): 8 requests over 90s, then still hadn't finished
+  (timed out at our 90s test cap; the real backoff schedule is 1s/2s/4s/8s/16s/32s/39s(capped)
+  and would keep going up to attempt 10/10, matching the ~10min real-session logs)
+- Retry-After: 999999 (a value far larger than any sane retry budget): 1 request,
+  ~0.1s, CLI immediately surfaces `{"error":"rate_limit","is_error":true,
+  "api_error_status":429,...}` and exits — no retries at all
+- Retry-After: 0: behaved like baseline (8+ requests, still retrying) — a small
+  value does NOT suppress retries, only a large one does
+- x-should-retry: false: 1 request, ~0.1s, same immediate-surface behavior as
+  the large Retry-After case
+
+Conclusion: Claude Code CLI's SDK (Stainless-generated) honors both a large
+`Retry-After` value and an explicit `X-Should-Retry: false` header to skip its
+own retry loop and surface the error immediately. Applied in
+OpenAIProxyServlet.setRetryAfterIfHardLimit(): when a Codex `usage_limit_reached`
+error body carries `resets_in_seconds` (a genuine, long-lived hard quota limit),
+the proxy now forwards it as `Retry-After` on the HTTP response — but only for
+that specific hard-limit case, not for ordinary transient 429s (which have no
+`resets_in_seconds` field), so those still get the CLI's normal short-backoff
+retry behavior. This only applies to non-streaming responses (where the proxy
+sets the real HTTP status/headers); the streaming (SSE) error paths in
+OpenAIProxyServlet already commit to `HTTP 200` before the upstream error is
+known, so there's no way to retroactively attach a Retry-After header there —
+the error is only conveyed via the SSE `event: error` body in that case.
+"""
+
+import http.server
+import json
+import os
+import subprocess
+import threading
+import time
+
+CLAUDE = '/usr/local/bin/claude'
+CWD = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+ENV_BASE = {k: v for k, v in os.environ.items()
+            if k not in ('CLAUDECODE', 'CLAUDE_CODE_ENTRYPOINT',
+                         'ANTHROPIC_BASE_URL', 'ANTHROPIC_AUTH_TOKEN', 'ANTHROPIC_API_KEY')}
+
+RATE_LIMIT_BODY = json.dumps({
+    "type": "error",
+    "error": {"type": "rate_limit_error", "message": "Rate limit test — should not resolve"},
+}).encode('utf-8')
+
+CMD_TAIL = ['--print', 'say hi', '--output-format', 'stream-json', '--verbose',
+            '--dangerously-skip-permissions']
+
+
+class CountingHandler(http.server.BaseHTTPRequestHandler):
+    """Always answers /v1/messages with 429 + RATE_LIMIT_BODY, optionally with
+    extra headers. Records every request's arrival time on the class-level list
+    supplied via `server.hits`."""
+
+    extra_headers = {}
+
+    def log_message(self, fmt, *args):
+        pass  # keep test output focused on our own summary
+
+    def do_POST(self):
+        self.server.hits.append(time.monotonic())
+        self.send_response(429)
+        self.send_header('Content-Type', 'application/json')
+        for k, v in self.extra_headers.items():
+            self.send_header(k, v)
+        self.send_header('Content-Length', str(len(RATE_LIMIT_BODY)))
+        self.end_headers()
+        self.wfile.write(RATE_LIMIT_BODY)
+
+
+def run_variant(label, extra_headers, timeout=90):
+    handler_cls = type('Handler', (CountingHandler,), {'extra_headers': extra_headers})
+    server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), handler_cls)
+    server.hits = []
+    port = server.server_address[1]
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+
+    env = dict(ENV_BASE)
+    env['ANTHROPIC_BASE_URL'] = f'http://127.0.0.1:{port}'
+    env['ANTHROPIC_AUTH_TOKEN'] = 'sk-test-fake'
+
+    print(f'\n=== {label} (headers={extra_headers or "none"}) ===')
+    start = time.monotonic()
+    try:
+        proc = subprocess.run([CLAUDE] + CMD_TAIL, env=env, cwd=CWD,
+                               stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+                               timeout=timeout)
+        elapsed = time.monotonic() - start
+        timed_out = False
+    except subprocess.TimeoutExpired as e:
+        elapsed = time.monotonic() - start
+        timed_out = True
+        proc = None
+
+    server.shutdown()
+    thread.join(timeout=5)
+
+    hit_count = len(server.hits)
+    gaps = [round(b - a, 2) for a, b in zip(server.hits, server.hits[1:])]
+    print(f'requests to fake server: {hit_count}')
+    print(f'gaps between requests (s): {gaps}')
+    print(f'wall time: {elapsed:.1f}s{" (TIMED OUT)" if timed_out else ""}')
+    if proc is not None:
+        tail = proc.stdout.decode('utf-8', errors='replace')[-500:]
+        print(f'stdout tail: {tail!r}')
+        err_tail = proc.stderr.decode('utf-8', errors='replace')[-300:]
+        if err_tail.strip():
+            print(f'stderr tail: {err_tail!r}')
+    return hit_count, elapsed, timed_out
+
+
+if __name__ == '__main__':
+    results = {}
+    results['baseline (no extra headers)'] = run_variant(
+        'baseline (no extra headers)', {})
+    results['Retry-After: 999999 (far future)'] = run_variant(
+        'Retry-After: 999999 (far future)', {'Retry-After': '999999'})
+    results['Retry-After: 0'] = run_variant(
+        'Retry-After: 0', {'Retry-After': '0'})
+    results['x-should-retry: false'] = run_variant(
+        'x-should-retry: false', {'x-should-retry': 'false'})
+
+    print('\n\n=== SUMMARY ===')
+    print(f'{"variant":40s} {"requests":>9s} {"seconds":>9s}')
+    for label, (hits, elapsed, timed_out) in results.items():
+        print(f'{label:40s} {hits:9d} {elapsed:9.1f}{"  TIMEOUT" if timed_out else ""}')

claude-launch-tests/test_session_id_header_stability.py

@@ -0,0 +1,101 @@
+"""
+Hypothesis: Claude Code CLI's `X-Claude-Code-Session-Id` request header stays
+identical across a `--continue` restart of the same conversation, since it's
+the same identifier used for the `~/.claude/projects/<hash>/<session-id>.jsonl`
+file that `--continue` reopens. If true, this header is a better source for the
+OpenAI proxy's `prompt_cache_key` than the plugin's own per-process UUID (which
+is regenerated on every PTY restart, defeating prompt-cache continuity across
+"Continue last session").
+
+Method: run `claude --print` twice from the same working directory against a
+local fake HTTP server that echoes a valid Anthropic message response and
+records the `X-Claude-Code-Session-Id` header of every request — first as a
+new session, then with `--continue`.
+
+Run: python3 claude-launch-tests/test_session_id_header_stability.py
+
+Result: CONFIRMED. Both runs sent the identical session id header
+(e.g. `3ff9518a-bfbd-4de7-92fe-13c419eba9f5` in one run), matching the
+`session_id` field in the CLI's own `system/init` stream-json event. Applied in
+OpenAIProxyServlet: `prompt_cache_key` is now sourced from this header (falling
+back to the proxy's internal per-process UUID when absent), so prompt caching
+survives "Continue last session" / "Resume session" instead of starting a fresh
+cache key on every restart.
+"""
+
+import http.server
+import json
+import os
+import subprocess
+import threading
+
+CLAUDE = '/usr/local/bin/claude'
+CWD = '/tmp/claude_session_id_stability_test'
+ENV_BASE = {k: v for k, v in os.environ.items()
+            if k not in ('CLAUDECODE', 'CLAUDE_CODE_ENTRYPOINT',
+                         'ANTHROPIC_BASE_URL', 'ANTHROPIC_AUTH_TOKEN', 'ANTHROPIC_API_KEY')}
+
+RESPONSE_BODY = json.dumps({
+    "id": "msg_test", "type": "message", "role": "assistant", "model": "claude-sonnet-4-5",
+    "content": [{"type": "text", "text": "hi there"}],
+    "stop_reason": "end_turn", "stop_sequence": None,
+    "usage": {"input_tokens": 1, "output_tokens": 1},
+}).encode('utf-8')
+
+
+class RecordingHandler(http.server.BaseHTTPRequestHandler):
+    def log_message(self, fmt, *args):
+        pass
+
+    def do_POST(self):
+        session_header = self.headers.get('X-Claude-Code-Session-Id', '<MISSING>')
+        self.server.session_ids.append(session_header)
+        length = int(self.headers.get('Content-Length', 0))
+        self.rfile.read(length)
+        self.send_response(200)
+        self.send_header('Content-Type', 'application/json')
+        self.send_header('Content-Length', str(len(RESPONSE_BODY)))
+        self.end_headers()
+        self.wfile.write(RESPONSE_BODY)
+
+
+def run(env, prompt, extra_args=()):
+    cmd = [CLAUDE, '--print', prompt, '--output-format', 'stream-json',
+           '--verbose', '--dangerously-skip-permissions', *extra_args]
+    proc = subprocess.run(cmd, env=env, cwd=CWD, capture_output=True, timeout=30)
+    return proc.stdout.decode('utf-8', errors='replace')
+
+
+def main():
+    os.makedirs(CWD, exist_ok=True)
+    server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), RecordingHandler)
+    server.session_ids = []
+    port = server.server_address[1]
+    threading.Thread(target=server.serve_forever, daemon=True).start()
+
+    env = dict(ENV_BASE)
+    env['ANTHROPIC_BASE_URL'] = f'http://127.0.0.1:{port}'
+    env['ANTHROPIC_AUTH_TOKEN'] = 'sk-test-fake'
+
+    print('=== First run (new session) ===')
+    out1 = run(env, 'say hi')
+    ids_after_first = list(server.session_ids)
+    print(f'session ids after first run: {ids_after_first}')
+
+    print('\n=== Second run (--continue) ===')
+    out2 = run(env, 'say hi again', extra_args=('--continue',))
+    ids_after_second = list(server.session_ids)
+    print(f'session ids after second run: {ids_after_second}')
+
+    server.shutdown()
+
+    first_unique = set(ids_after_first)
+    second_new = set(ids_after_second) - set(ids_after_first)
+    print(f'\nfirst-run session id(s): {first_unique}')
+    print(f'session id(s) seen only in second run: {second_new}')
+    stable = first_unique and second_new <= first_unique
+    print(f'\nSTABLE ACROSS --continue: {stable}')
+
+
+if __name__ == '__main__':
+    main()