Connection.response() raises TimeoutError when the response has already arrived
Describe the bug
Connection.response() can report a timeout for a request whose answer has
already been received and correctly stored.
_dequeue_messages files every response it dequeues into self.responses,
whatever request it was watching for — only the break out of its loop is
specific to watch_for (python/src/xstudio/connection/__init__.py:490-493):
req_id = msg[-2]
if req_id in self.responses:
self.responses[req_id] = msg # filed regardless of watch_for
if req_id == watch_for:
break # only the break is specific
else:
self.handle_broadcast(msg)
response() then lets TimeoutError propagate without consulting
self.responses (:401-406):
def response(self, req_id, timeout_milli=None):
if timeout_milli is not None:
self._dequeue_messages(timeout_milli, req_id) # may raise
return self._response(req_id) # never reached
So when any other consumer of the connection's queue takes our response first,
the answer is stored correctly and the caller is told the request timed out.
Not having dequeued it ourselves is being treated as no answer arriving.
To Reproduce
from xstudio.connection import Connection
from xstudio.core import version_atom
conn = Connection(auto_connect=True)
req_id = conn.request(conn.remote(), version_atom())
# Any other consumer of this connection's queue. Doing it on this thread
# removes the race without changing the mechanism: dequeue_messages() pumps
# with watch_for unset, so it files the response and carries on rather than
# breaking on it.
conn.dequeue_messages(300)
assert conn.responses[req_id] is not None # the answer IS here
conn.response(req_id, 300) # ...and this raises TimeoutError
Expected behavior
TimeoutError should be raised only when no response for that request has been
recorded. Where one has been recorded — by any consumer of the queue — it should
be returned.
Impact
Three ways this shows up, all measured on develop @ cb0e9b6b:
Connection(background_processing=True) is effectively unusable. It
starts exactly such a consumer via start_background_processing →
process_events_forever. With one running, 20 of 20 bounded reads raised
TimeoutError with the answer already sitting in self.responses.
- A request issued from inside a broadcast callback, with no threads
involved at all. The nested request_receive re-enters the same pump on the
calling thread, so an outer frame files the answer while the inner call
concludes it timed out. Measured 1 in 20.
- The deterministic single-threaded case above.
It matters beyond a spurious exception: callers treat TimeoutError as evidence
that an actor is unresponsive and act on it — dropping cached handles,
re-acquiring, marking a peer unhealthy.
Desktop
- OS: macOS 15 (arm64)
- xSTUDIO version:
develop @ cb0e9b6b, built from source
- Python 3.11 (bundled interpreter)
Additional context
A fix and a regression test are in the linked PR. The same investigation turned
up a second, independent defect on the same code path — a read's latency being
charged for dispatching unrelated broadcasts — filed separately, since it is
architectural and has no small fix.
Connection.response()raisesTimeoutErrorwhen the response has already arrivedDescribe the bug
Connection.response()can report a timeout for a request whose answer hasalready been received and correctly stored.
_dequeue_messagesfiles every response it dequeues intoself.responses,whatever request it was watching for — only the
breakout of its loop isspecific to
watch_for(python/src/xstudio/connection/__init__.py:490-493):response()then letsTimeoutErrorpropagate without consultingself.responses(:401-406):So when any other consumer of the connection's queue takes our response first,
the answer is stored correctly and the caller is told the request timed out.
Not having dequeued it ourselves is being treated as no answer arriving.
To Reproduce
Expected behavior
TimeoutErrorshould be raised only when no response for that request has beenrecorded. Where one has been recorded — by any consumer of the queue — it should
be returned.
Impact
Three ways this shows up, all measured on
develop@cb0e9b6b:Connection(background_processing=True)is effectively unusable. Itstarts exactly such a consumer via
start_background_processing→process_events_forever. With one running, 20 of 20 bounded reads raisedTimeoutErrorwith the answer already sitting inself.responses.involved at all. The nested
request_receivere-enters the same pump on thecalling thread, so an outer frame files the answer while the inner call
concludes it timed out. Measured 1 in 20.
It matters beyond a spurious exception: callers treat
TimeoutErroras evidencethat an actor is unresponsive and act on it — dropping cached handles,
re-acquiring, marking a peer unhealthy.
Desktop
develop@cb0e9b6b, built from sourceAdditional context
A fix and a regression test are in the linked PR. The same investigation turned
up a second, independent defect on the same code path — a read's latency being
charged for dispatching unrelated broadcasts — filed separately, since it is
architectural and has no small fix.