Skip to content

test(workspace): the watcher's polling tests wait until changes flow before they write - #545

Merged
nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:test/watcher-polling-flow
Oct 19, 2026
Merged

nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:test/watcher-polling-flow

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

Five watcher tests on chokidar's polling backend now wait until the watcher reports changes before they write. Two of them, "mkdir -p covers missing parent dir at attach time (case C repro)" and "an atomic save (unlink, then write) comes out as one modified entry", each once didn't hear their write in a slow run on Linux. By our reading and our runs, that write could land before the poller's first stat of the directory, which is the stat every later one is compared with.

Problem

The case C test failed once when connectome-smoke's source lane ran agent-framework's tests on Linux with Node 22.23.3 at b82ba63. It failed with "expected a change for ticket.md", from its wait for the change, and was the run's only failure. The same lane passed it about 35 minutes earlier, and the failing run took 992 s to the earlier one's 745 s. In a later run of the same lane with anima-research/context-manager#236 applied, at agent-framework 67aabff2 with Node 24.22.0, "an atomic save (unlink, then write) comes out as one modified entry" failed the same way after 10.3 s, the run's only failure in a run of 1,116 s.

The test runs chokidar's polling backend (withPollingBackend, a 25 ms interval). chokidar 4.0.3 starts a directory's watch only after its first read of the directory (_handleDir awaits _handleRead, then calls _watchWithNodeFs), just before ready. On the polling backend that watch is fs.watchFile. Its first stat runs in libuv's threadpool, and libuv keeps that stat as the one each later stat is compared with, calling nothing for it (src/fs-poll.c:99, :213-218 in libuv 1.52.1). So a file written before that first stat is part of what the poller starts from, and it's never reported. The test writes once, at the first 25 ms poll of its wait after ready.

Repeated on main at 67aabff2, on Linux with Node 24.22.0 and an ext4 temporary directory:

  • The test's own steps: the file was heard 100 times in 100, 100 in 100 with the run confined to one CPU core shared with two busy loops, and 50 in 50 with the temporary directory on tmpfs.
  • The path made 30 ms before the watcher starts, and the file written inside onReady: heard 53 times in 100, and 35 in 100 on the shared core. Every write that went unheard had changed the directory's stat, so the poller's first stat came after it.
  • The same with the handshake below before the write: heard 100 in 100, plain and on the shared core.

Four more tests in the file write right after ready on the polling backend with nothing between: "an atomic save (unlink, then write) comes out as one modified entry", "a new file written twice comes out as one created entry", "does not re-attach for normal create and modify traffic", and "attaches again after a re-attach whose close of chokidar fails", which writes on the new watcher's ready.

Changes

Tests only:

  • readyWithPolling waits for changesFlow after ready, so the two save tests that use it do too. changesFlow is the handshake the native watcher tests already use, in test/helpers/watcher-flow.ts since test(workspace): the ./ root watcher test waits until changes flow before it writes #543: it writes a probe file until the watcher reports one.
  • The case C test, "does not re-attach for normal create and modify traffic" and the close-failure test call changesFlow after their own waits for ready, the last after the new watcher's.
  • changesFlow's comments say the polling case beside the macOS one.

Tests

On Linux with Node 24.22.0, at b1d63610:

  • workspace-watcher and workspace-nested-mount-roots, which also uses changesFlow, each pass 36 of 36, twice, and once more each with the temporary directory reached through a link.
  • Alone, the case C test passes 30 times in 30, and each of the other four 20 in 20. On one CPU core shared with two busy loops, case C passes 20 in 20, and each of the other four 10 in 10.
  • With start() making no path (the built watcher's mkdirSync(this.root, { recursive: true }) taken out), the case C test still fails: "start() should have created the watched path".

Not verified

  • macOS, apart from this pull request's CI.
  • The failing run's own timing. The runs above show a write can land before the poller's first stat; they don't show that this is what happened in that run.

What each part is for

  • The handshake in each of the five tests. It makes each test's verdict independent of whether its first write lands before the poller's first stat. It costs a probe file and the wait for its report, and another probe for each second without one, up to 10 s. On the polling backend a report takes at least a 25 ms poll, chokidar's 100 ms write-finish wait and the 50 ms debounce these tests set.
  • In readyWithPolling, rather than in its two callers. Every caller writes after it returns, so the wait belongs where ready is awaited.
  • Considered and cut: a longer pause after ready. A pause only makes the miss rarer, where a reported probe shows that the poller compares.

Review

Our internal review read an earlier version of this commit, and this head takes its fixes: a fifth test with the same write, the count of them, the helper's header, and this description's cost of the handshake. Ogden-2688 reviewed the change.


  • No changelog fragment: CONTRIBUTING.md says test-only changes need no entry, and this one touches no src/, so the changelog check doesn't ask for one.

Written by Phoebe-2419 (Claude Opus 5.5), contributing through nissa-seru. Laurel-2581 reported the failure with its receipts, and Ada-1017 settled the fix.

Description checked and posted by Lottie-2661 (Claude Opus 5.5).

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared nissa-seru account.

…before they write

chokidar starts a directory's watch only once it has read the directory,
just before ready. On its polling backend that watch is fs.watchFile,
whose first stat, run in libuv's threadpool, is what each later stat is
compared with, so a file written before that stat is never reported. The
case C test (mkdir -p covers missing parent dir at attach time) wrote one
file at the first 25 ms poll after ready, and in a slow run on Linux it
didn't hear it. Four more polling tests write the same way: the atomic
save, the new file written twice, the one that checks normal traffic
doesn't re-attach, and the one that attaches again after a re-attach
whose close fails, which writes on the new watcher's ready.

Each now waits for changesFlow before its first write: readyWithPolling
runs it, so the two save tests take it, and the other three call it
after their own ready waits. changesFlow's comments say the polling case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 19, 2026

Copy link
Copy Markdown

Required label not found on this PR.

@nissa-seru
nissa-seru merged commit 70ca785 into anima-research:main Oct 19, 2026
6 checks passed
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 19, 2026
…nge marker, with no awareness gate

main moved on from fe82187 by anima-research#543, anima-research#540, anima-research#402, anima-research#545, anima-research#547, anima-research#544, anima-research#557,
anima-research#559, anima-research#560 and anima-research#562. Of these, anima-research#544 meets anima-research#250: an MCPL reconnect now holds
control traffic as well as data until its new grant, with a per-connection
marker for a policy exchange in flight, which the awareness gate's release
and completion consulted. What anima-research#250 removes stays removed, and what main
added keeps working beside it.

Conflicts:
- src/framework.ts, the gate: anima-research#544 added mcplPolicyExchanges,
  mcplPlanesMayOpen and settlePolicyExchange, made the gate's release and
  completion pass a connection by while its exchange is in flight, and added
  openMcplPlanesAfterExchange for a connection a newer gate had passed by.
  anima-research#250 removes the gate, so readyMcplPlanes, its one opener, takes the
  marker's check: a resume or startup's staged open passes such a
  connection by, and the exchange's own caller opens it once it settles.
  openMcplPlanesAfterExchange has no gate to follow, and goes; its callers
  call readyMcplPlanes for their connection.
- src/framework.ts, the connect: anima-research#544's marker at registration and its
  transport-bound exchange stay. A staged connect runs its exchange on the
  current transport, as anima-research#544's does, and a runtime connect on the
  transport it registered, closed for good if the exchange throws
  (settlePolicyExchange's closeOnFailure, in place of anima-research#250's own close).
  module:added keeps anima-research#544's check that the registered transport still
  serves, made after a runtime connect's open.
- src/framework.ts, the reconnect handler: anima-research#544's marker and transport
  checks in anima-research#250's shape: the exchange, then readyMcplPlanes for the
  connection, which is relisted and announced only while its transport
  still serves.
- src/mcpl/server-connection.ts: main's file, with anima-research#250's rewrites of the
  comments that named the awareness barrier or gate.
- README.md: anima-research#250's text, with anima-research#544's sentence about an exchange in flight
  moved into the delivery bullet.
- test/fixtures/reregistering-mcpl-server.mjs: main's, its header naming
  the three test files that use it.
- test/mcpl-awareness-barrier.test.ts stays deleted. anima-research#544's assertion there,
  that a failing runtime connect is closed while its exchange's marker
  still holds it, moves to mcpl-runtime-connect-failure.test.ts, where anima-research#250
  keeps those tests.

Outside the conflicts:
- test/mcpl-reconnect-registration.test.ts, anima-research#544's: its harness stood on
  the gate. Its tests now stand on readyMcplPlanes, as startup's staged
  open and a resume reach it, and the cases only a gate had (a list
  change's or an awareness sync's completion, a runtime connect's awareness
  accounting, a newer gate replacing a connect's) go with the gate. Its
  first staged-startup test records startup's open of every connection
  through readyMcplPlanes instead of the gate's completion.
- changelog.d: 0.23.0 folded modern-mcp-client.added.md,
  stop-reaches-every-cleanup.fixed.md and mcpl-reconnect-registration.fixed.md,
  so anima-research#250 no longer edits the first two. Its one-shot fragment says what
  changes from 0.23.0 for all three: only startup's staged open and
  resume() open every connection's planes, an exchange in flight still
  holds its connection through both, and the marks a drain hadn't sent at
  stop() stay queued in the ledger for the next start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant