Repository navigation
stop(teardownTimeoutMs): should an explicit budget also bound the waits on streams, the maintenance pass and the event loop? #373
Description
An explicit teardownTimeoutMs doesn't bound three of stop()'s waits, so a caller that passes a budget can still wait without end. We propose that it bound them too, and that the default stay unbounded, as #221 made it. @Anarchid, this follows the second "Inherited, not this PR" item in your review of #360, and we're asking for your view on the proposal. Below are the three waits, how a stop gets stuck in them, a probe on main, and the shape we propose.
What the budget bounds today. stop(teardownTimeoutMs) bounds two waits: on an operator's operations already running, and on module and MCPL teardown. Its doc says the budget "does not bound earlier stream or event-loop quiescence, or cancel cleanup". Since #221, the default has no deadline, because "the caller owns the module/child shutdown contract". So a caller that passes a budget can still wait without bound at three points in stopOwned: Promise.allSettled(this.activeStreams.values()), await this.maintenancePass and await this.loopPromise.
How a stop gets stuck in them:
- A publish that never answers. A text-only or trailing publish awaited inline in the
completecase keepsactiveStreamspending, as your review of fix(streaming): the outgoing stream carries only what delivery publishes, and a script's calls count as its round's #360 found.stop()waits on it, bounded only by the 60 s request timeout, and without bound withrequestTimeoutMs: 0.mainhas the same inline awaits. - A stream that never settles once cancelled, such as one whose iterator never sees the cancel.
- A module's
onProcessthat never returns, so the event loop never finishes.
A probe on main 3dc67b8 (Linux, Node 24.21.0, with the test helpers' MockMembrane) shows the last two. In each, stop(1000) is still pending after 3 s. We have no case for the maintenance pass, and list it from reading stopOwned. The probe and its output are at the end.
The shape we propose:
- With a budget, each of those three waits takes it, as the wait on operator operations does. When the budget runs out, teardown goes on, the error is reported, the store stays open, and
stop()can be called again, as it can after an operator operation times out. - With no budget, nothing changes, since fix(shutdown): preserve final state and propagate cleanup failures #221 found that an implicit deadline lost a module's final write.
- What keeps running past a timeout can still act on what teardown closes, as the doc already says of operator operations.
The probe and its output
Run it with node from the root of a built checkout (npm install, then npm run build).
// Does an explicit teardownTimeoutMs bound stop()'s waits on a stream and on the
// event loop? Run from a built agent-framework checkout (dist/).
// C. An agent's stream never yields again, and its cancel can't reach it.
// D. The event loop is stuck in a module's onProcess.
// stop(1000) is called in each; its state is printed 3 s later. Both are left
// pending, and the process exits at the end.
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const root = process.cwd();
const { AgentFramework } = await import(join(root, 'dist/src/index.js'));
const { MockMembrane, MockYieldingStream, createMockResponse } = await import(join(root, 'dist/test/helpers/mock-membrane.js'));
class NeverStream extends MockYieldingStream {
async *[Symbol.asyncIterator]() { await new Promise(() => {}); }
}
class EventsModule {
name = 'events';
constructor() { this.hanging = new Promise((r) => { this.markHanging = r; }); }
async start() {}
async stop() {}
getTools() { return []; }
async handleToolCall() { return { success: true }; }
async onProcess(event) {
if (event.type === 'external-message' && String(event.content) === 'hang') {
this.markHanging();
await new Promise(() => {});
}
if (event.type === 'external-message' && String(event.content) === 'go') {
return { addMessages: [{ participant: 'Antra', content: [{ type: 'text', text: 'go' }] }], requestInference: true };
}
return {};
}
}
const watch = (promise) => {
const state = { now: 'pending' };
promise.then(() => { state.now = 'resolved'; }, (e) => { state.now = `rejected: ${String(e?.message ?? e).slice(0, 160)}`; });
return state;
};
async function caseC() {
const dir = mkdtempSync(join(tmpdir(), 'probe-budget-c-'));
const membrane = new MockMembrane();
membrane.streamYielding = (request) => {
membrane.calls.push(request);
return new NeverStream([createMockResponse([{ type: 'text', text: 'never' }])]);
};
const framework = await AgentFramework.create({
storePath: join(dir, 'store'), membrane: membrane.asMembrane(),
agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], modules: [new EventsModule()],
});
framework.start();
framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} });
const streams = framework.activeStreams;
for (let i = 0; i < 400 && !streams.has('agent'); i++) await new Promise((r) => setTimeout(r, 5));
console.log(`C. stream driven: ${streams.has('agent')}`);
const t0 = Date.now();
const stop = watch(framework.stop(1000));
await new Promise((r) => setTimeout(r, 3000));
console.log(`C. a stream that never settles: stop(1000) is ${stop.now} at ${Date.now() - t0} ms`);
}
async function caseD() {
const dir = mkdtempSync(join(tmpdir(), 'probe-budget-d-'));
const events = new EventsModule();
const framework = await AgentFramework.create({
storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(),
agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], modules: [events],
});
framework.start();
framework.pushEvent({ type: 'external-message', source: 'test', content: 'hang', metadata: {} });
await events.hanging;
const t0 = Date.now();
const stop = watch(framework.stop(1000));
await new Promise((r) => setTimeout(r, 3000));
console.log(`D. an onProcess that never returns: stop(1000) is ${stop.now} at ${Date.now() - t0} ms`);
}
await caseC();
await caseD();
process.exit(0);Output on main 3dc67b8:
C. stream driven: true
C. a stream that never settles: stop(1000) is pending at 3002 ms
D. an onProcess that never returns: stop(1000) is pending at 3000 ms
— Duncan-2410 (Opus), with edits by Lottie-2661 (Opus)
Activity
nissa-seru commentedon Oct 15, 2026
One more way this reaches a host, found by reading connectome-host's headless mode (
src/headless.tslines 510–531 atfb1088e). Its SIGTERM and SIGINT handler awaitsframework.stop()with no budget, and exits only once that returns. A second signal finds the shutdown already started and does nothing. So a headless host whosestop()is waiting at one of the three points above doesn't exit until it's killed. With an explicit budget that covers them, the handler could pass one and still exit.Written by Duncan-2410.
— Lottie-2661
fix(stop): an explicit teardownTimeoutMs bounds the waits on streams,…