Skip to content

an idle keep-alive closes at DEBUG, and an upstream that never answered says so - #5

Merged
itswl merged 1 commit into
mainfrom
sidecar-idle-timeouts
Oct 14, 2026
Merged

itswl merged 1 commit into
mainfrom
sidecar-idle-timeouts

Conversation

@itswl

@itswl itswl commented Oct 14, 2026

Copy link
Copy Markdown
Owner

The sidecars' shared HTTP handler logged every socket timeout as one WARNING line. In the work stack's MCP gate most of those lines were an MCP client's keep-alive connection idling past timeout between calls — the timeout doing its job — and a peer with a real problem, or an upstream that hung, had to be picked out from among them.

What changes

  • common.HttpHandler notes where the read was. Waiting for the next request line after at least one answer is the idle close: one DEBUG line, idle keep-alive closed peer=… requests_seen=N. A socket that connected and never spoke, or a request whose headers or body never arrived, stays the WARNING it was, with the count and the last request line saying which.
  • gate.py logs an upstream failure at WARNING as well as writing call.failed, and names a timeout as one with the limit it ran into — upstream timeout: no answer within 60s — in both, so it cannot be read as the client-side timed out. Other failures keep their reason as before.

Nothing about the timeouts themselves changes: the same 30 s on the client side, the same UPSTREAM_TIMEOUT on the relay.

Tests

  • test_an_idle_keep_alive_closes_at_debug_not_as_a_warning replaces the test that pinned the idle close at WARNING.
  • test_a_body_that_never_arrives_names_the_request_it_belonged_to: the mid-request stall stays a WARNING naming its request.
  • test_a_chat_server_that_hangs_is_a_502_that_says_it_timed_out: a stalled upstream is a 502, the ledger row and a WARNING line both say upstream timeout.

The two new behaviours fail without the change. bash scripts/gate.sh: STACK GATE GREEN.

…ver answered says so

The shared HTTP handler logged every socket timeout as one WARNING line. Most of
them were an MCP client's keep-alive connection idling past `timeout` between
calls — the timeout doing its job — and a peer with a real problem, or an
upstream that hung, was a line among them. Where the read was now decides:
waiting for the next request line after at least one answer is the idle close,
logged at DEBUG; a socket that never spoke, or a request whose headers or body
never arrived, stays one line at WARNING with the count and the last request
line.

The gate's upstream failures were in the ledger only. They are logged at
WARNING as well, and a timeout is named as one with the limit it ran into
("upstream timeout: no answer within 60s"), so it cannot be read as the
client-side `timed out`.
@itswl
itswl merged commit d5428ca into main Oct 14, 2026
2 checks passed
@itswl
itswl deleted the sidecar-idle-timeouts branch October 19, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant