Gateway 500 (NoneType TypeError) = downstream model out of capacity — LiteLLM surfaces it as a crash, not a 429

[Julia-Lex]

Root cause (confirmed)

This is a LiteLLM gateway bug triggered by downstream model capacity exhaustion: when the model behind litellm.local.lexmount.net has insufficient capacity under concurrent load, the gateway's limiter/accounting path crashes with

500 '>' not supported between instances of 'NoneType' and 'int'

instead of returning a clean 429/rate-limit response.

If you see this signature: the downstream model is out of capacity. It is not a bench bug, not an OpenClaw bug, and not worth client-side debugging — request more capacity for that model.

Status: the model hit in the 2026-07-03 runs (gpt-5.4) has since been scaled up and should no longer trip this. Other models on the same gateway can still hit it, so this issue stays open as the lookup entry for the signature.

How it presents

• Tasks die on their first LLM call (steps=0, env_status=failed, "No result JSON from OpenClaw").
• OpenClaw retries 6 times over ~33-40s, then surfaces FailoverError: 500 '>' not supported... in stderr.
• Zero 429 / rate limit / too many requests strings anywhere — do not grep for rate-limit keywords, grep for NoneType.
• Concurrent runs only; single-stream never trips it. Intermittent (2 of 3 c10 batches on 2026-07-03; an identical rerun 10 minutes later was clean).

Occurrences (2026-07-03/04, openclaw + lexmount + gpt-5.4)

Run	Load	Result
20260703_224231	first_n 20, c10	15/20 hit the 500
20260703_233910	first_n 10, c10	6/10 hit the 500
20260703_234909 (identical rerun)	first_n 10, c10	0/10 — clean

When another model hits it

1. Confirm the stderr signature (experiments/.../tasks/<id>/stderr.txt).
2. Report it as model capacity shortage for that model — escalate to whoever owns the model deployment, with the time window and concurrency.
3. Optionally ask the gateway admin to fix LiteLLM's limiter so exhaustion returns a proper 429 (which clients could back off on) instead of a 500 crash.

Reference stderr (task 5, run 20260703_233910):

[agent/embedded] embedded run agent end: runId=48e7ef34-... isError=true model=gpt-5.4 provider=bench error=LLM request failed. rawError=500 '>' not supported between instances of 'NoneType' and 'int'
[diagnostic] lane task error: lane=main durationMs=35381 error="FailoverError: 500 '>' not supported between instances of 'NoneType' and 'int'"

🤖 Generated with Claude Code

[Julia-Lex]

New evidence (2026-07-07): this 500 signature is NOT (only) capacity — it has a deterministic trigger.

Repro against litellm.local.lexmount.net, model dmx-claude-opus-4-8, /v1/messages:

• words=100/4000, stream=false or true → 200
• words=12000, stream=false → 200
• words=12000, stream=true → 500 '>' not supported between instances of 'NoneType' and 'int' — reproducible every time.

So: streaming requests above an input-size threshold crash the adapter (looks like a token-count vs None limit comparison on the streaming path). Capacity pressure may still produce the same signature, but large-prompt streaming does so deterministically.

Impact: claude-code site-skills experiment arm (bigger prompts) failed 10/20 tasks with this 500 while the control arm passed 20/20 — the skill injection pushed first-call prompts over the threshold.

Client-side workaround in the bench strip proxy: rewrite stream=true → stream=false upstream and synthesize Anthropic SSE events from the complete response for the CLI. Verified: the failing repro now returns 200.

Suggested gateway fix: guard the streaming-path token comparison (treat None limit as unlimited).

[Julia-Lex]

Correction/refinement to my previous comment: the size threshold is NOT fixed — it drifts with gateway state.

Probe history today (dmx-claude-opus-4-8, /v1/messages, filler words):

• ~13:00: 12k words nostream 200 / stream 500; 20k words nostream 500
• ~13:16: 6k words 500 in BOTH modes (2k still 200)

So the None-comparison crash fires at a limit that shrinks as the gateway degrades (same pattern as the 2026-07-03 openclaw incident: intermittent, state-dependent, self-recovers). Streaming just fails earlier than non-stream at any given moment. This morning's clean control-arm window was timing luck.

Practical guidance: when this signature appears, probe with a mid-size (6-12k words) request; if it 500s, back off and rerun in a quieter window — client-side prompt shrinking cannot chase a moving wall.

[Julia-Lex]

Root cause confirmed by model owner: the dmx-claude-opus-4-8 upstream is out of credits/quota — the drifting size threshold was quota-driven throttling surfacing through the gateway's broken None-comparison (still worth fixing: it should be a clean 4xx, not a 500 TypeError). claude-code experiment arm halted; no further runs against this model until quota is restored.