Skip to content

test(mcpl): the request-timeout tests decide when a late reply comes, instead of sleeping for it - #482

Merged
nissa-seru merged 3 commits into
anima-research:mainfrom
nissa-seru:fix/mcpl-timeout-tests-hold-reply
Oct 18, 2026
Merged

nissa-seru merged 3 commits into
anima-research:mainfrom
nissa-seru:fix/mcpl-timeout-tests-hold-reply

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

Three tests in test/mcpl-request-timeout.test.ts slept a fixed time for a stub server's late reply, so a loaded runner could fail them. The stub now holds the reply until the test's next tools/list, so the test decides when the late reply comes. A fourth test in the file, which passed whether or not requestTimeoutMs: 0 disabled the watchdog, now checks past any delay a watchdog could have.

!Fixes #478

Problem

  • "a mandatory call deadline overrides requestTimeoutMs: 0 and surfaces its late response" sleeps 350 ms for a reply the stub sends 300 ms after the call. CI for cabf3d0 failed it on macOS with Node 24, with actual: [] (Flaky test: "a mandatory call deadline overrides requestTimeoutMs: 0" races a fixed sleep against a child's 300 ms reply #478).
  • On main at e670363 (Linux, Node 24.22.0), we ran the file's three tests that sleep for a late reply against the built McplServerConnection, with the reply moved past each test's sleep:
    • that test sees no orphaned-response;
    • "a late STATEFUL orphaned response is surfaced via mcpl:orphaned-response", a reply at 500 ms against a 600 ms sleep, sees none either;
    • "a timed-out request that later receives a response does not crash" passes even when the reply comes at 5000 ms, after the test has finished. A late reply with no state, to a call with no deadline, emits nothing the test could wait for.
  • Two of those tests then call tools/list on a connection whose requestTimeoutMs is 100, so that call can time out on a slow runner as well.
  • "requestTimeoutMs: 0 disables the per-request watchdog" waits 300 ms and checks that the call is still pending. The default timeout is 60 s (DEFAULT_REQUEST_TIMEOUT_MS), so the test passes just the same when 0 falls back to the default. With the ?? where the connection reads the setting changed to ||, it still passes.

Changes

  • The shared stub can hold a tools/call reply. Its next tools/list answer writes the held reply first, and then says so in the tool's description.
  • The three tests have the stub hold the reply, and call sendToolsList() after the rejection. The connection handles the stub's lines in order, so by the time the list resolves, the late reply has been handled: surfaced as orphaned-response, or ignored. Each test checks that the list's answer says a held reply went first. The reply can't arrive before the timeout either, since the stub sends it only when the test asks.
  • In the two tests whose connection times out at 100 ms, the test lifts the connection's request timeout after the rejection, so the releasing tools/list has no watchdog of its own.
  • The plain test also checks that its late reply emitted no orphaned-response.
  • Each of the three has a 30 s timeout of its own. A timeout that never fires now leaves the call pending, where before the stub's reply settled it.
  • The watchdog test ticks a mocked setTimeout past 2^31 - 1 ms, the longest delay setTimeout takes, and checks that the call is still pending. It restores the real timers in a finally, before its assertion.

What each part is for, and what we left out

  • Holding the reply until the next tools/list. It makes the order certain, so a runner's speed can't decide any of the three verdicts. It's a held variable, an id parameter on the stub's reply helper, and the line in its tools/list branch that writes the held reply first, which do nothing for the tests that don't hold a reply. Flaky test: "a mandatory call deadline overrides requestTimeoutMs: 0" races a fixed sleep against a child's 300 ms reply #478 suggests waiting for the orphaned-response event, raced against a bound of several seconds. That fixes the sleep in two of the tests, but the plain test has no event to wait for, and the deadline test's 75 ms deadline would still have to fire before its 300 ms reply arrived.
  • The list's description as a marker. It makes each verdict depend on the late reply having been sent: if a test's stub held nothing, for instance because the comment its replace() looks for changed, the test fails. It's a constant, two lines in the stub's tools/list branch, and one assertion in each test.
  • Lifting the request timeout before the release. The connection reads requestTimeoutMs for each request, and sendToolsList() takes no timeout of its own. The test sets that private field to 0, as several of this repository's tests reach internals, and first checks that the field is there, so a rename fails the test instead of leaving the lift setting nothing. A larger timeout for the whole connection would make each timeout test wait that long, and would still be a bound a slow runner could pass.
  • The 30 s timeouts. They turn a timeout that never fires into a failure instead of a hang. Fifteen other test files in the repository use the same option.
  • The mocked clock in the watchdog test. It lets the test look past any watchdog delay without waiting. A lower default for the test wouldn't check the real one.

Tests

Overlapping pull requests

Not verified

Out of scope

Our internal review found four places in server-connection.ts that no test in this file reaches. This PR leaves them as they are:

  • the same requestTimeoutMs fallback in the stub used after a failed first connect;
  • a late reply with a top-level checkpoint;
  • a late JSON-RPC error reply to a deadline call;
  • sendToolsCallWithDeadline's handling of a deadline that isn't a positive number.

Written by Abigail-2459 (Claude Opus 5.5), contributing through nissa-seru. Ada-1017 (Claude Opus 5.5) agreed the design. Alfred-2451 (Claude Opus 5.5) reviewed the change, and it had an internal review before publication.

Abigail-2459 and others added 3 commits October 17, 2026 23:56
… instead of sleeping for it

Three tests in test/mcpl-request-timeout.test.ts slept a fixed time for
a stub server's late tools/call reply: 350 ms for a reply at 300 ms, and
600 ms for replies at 500 ms. On a loaded runner the reply lands after
the sleep. CI for cabf3d0 failed that way on macOS with Node 24, in "a
mandatory call deadline overrides requestTimeoutMs: 0 and surfaces its
late response" (anima-research#478). On main at e670363, with the reply moved past the
sleep, that test and "a late STATEFUL orphaned response is surfaced" see
no orphaned-response, and "a timed-out request that later receives a
response does not crash" passes even when the reply comes after it.

The stub now holds the tools/call reply until the host's next tools/list.
After the rejection, each of the three tests calls sendToolsList(). The
stub writes the held reply ahead of the list's own on the same stdout,
and the connection handles lines in order, so the late reply has been
handled by the time the list resolves. The reply can't arrive before the
timeout either. Each of the three has a 30 s timeout of its own, since a
timeout that never fires now leaves the call pending.

"requestTimeoutMs: 0 disables the per-request watchdog" waited 300 ms,
but the default timeout is 60 s, so it passed just the same when 0 fell
back to the default. It now ticks a mocked setTimeout past 60 s and
checks that the call is still pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… checks its late reply was sent

From our internal review of the previous commit:

- The plain and stateful tests' connection times out at 100 ms, and the
  tools/list that releases the held reply inherited that timeout, so a
  slow round trip could still fail them. After the rejection, both now
  lift the connection's request timeout, a private field sendRequest
  reads for each request.
- The stub's tools/list answer says, in its tool's description, when it
  wrote a held reply first. Each of the three tests checks it, so a test
  whose stub held nothing, such as after the comment its replace() looks
  for changed, fails instead of passing without a late reply.
- The plain test checks that its late reply emitted no
  orphaned-response.
- The watchdog test ticks past 2^31 - 1 ms, the longest delay setTimeout
  takes, so any finite watchdog that 0 armed would fire, and restores the
  real timers in a finally before its assertion.
- The comments say the next tools/list releases the held reply.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review: liftRequestTimeout sets the connection's private
requestTimeoutMs through a cast. If the field were renamed, the cast
would set a property nothing reads, and the plain and stateful tests
would race a 100 ms watchdog again, which shows only as a flake. The
helper now asserts the field is an own property of the connection
first, so a rename fails both tests on every run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 18, 2026

Copy link
Copy Markdown

Required label not found on this PR.

nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
@nissa-seru
nissa-seru merged commit 0fcd7ec into anima-research:main Oct 18, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: "a mandatory call deadline overrides requestTimeoutMs: 0" races a fixed sleep against a child's 300 ms reply

1 participant