diff --git a/README.md b/README.md index a54efc619..371607f58 100644 --- a/README.md +++ b/README.md @@ -301,6 +301,72 @@ A pattern that matches no tool is reported once, as a console line and an A pattern that could name a server's tools is not judged until that server has listed them, so late connects and reconnects don't cause early reports. +#### Modern MCP servers (2026-07-28) + +The same `mcplServers` list also takes servers that speak modern MCP +(revision `2026-07-28`), reached through the official SDK client. The +configuration chooses the family, and nothing is probed: + +| Configuration | Family | Transport | +|---|---|---| +| `url: 'ws://…'` / `'wss://…'` | MCPL (MCP `2024-11-05`) | WebSocket | +| `url: 'http://…'` / `'https://…'` | modern MCP | Streamable HTTP (`token` / `accessProvider` as bearer) | +| `command` | MCPL | stdio | +| `command` + `protocol: 'modern'` | modern MCP | stdio | + +```typescript +{ + mcplServers: [ + { id: 'docs', url: 'https://mcp.example.com/mcp', accessProvider: getToken }, + { id: 'search', command: 'search-mcp', protocol: 'modern', reconnect: true }, + ], +} +``` + +A configuration with both `command` and `url` needs `transport` to say which +one is used; without it, it is refused rather than silently ignoring one. + +A modern server's tools get the same prefix, `enabledTools`/`disabledTools`, +class overrides and dispatch as an MCPL server's, and `listMcplServers()` +shows its `family`, `protocolVersion` and `transport`. Its config differs in +three ways: +- It has no MCPL surface (grant, push, channels, inference requests, feature + sets). A setting that asks for that surface can't apply, so it is a + configuration error: `allowHostCommands: true`, an `enabledCapabilities` + pattern that matches a capability denied by default, `channelSubscription: + 'auto'` or a non-empty list, a scope's non-empty `whitelist`, a + `toolLifecycle` `observe` or `inputs` block, and a + `toolLifecycle.maxInputBytes` or an `autofetch` bound above its default. + A setting + that only takes something away, or repeats a default, is already + satisfied, so it is accepted: `enabledFeatureSets` and the rest of + `enabledCapabilities`, which are allow-lists, `disabledFeatureSets`, + `disabledCapabilities`, a scope's `blacklist`, `channelSubscription: + 'manual'`, a `toolLifecycle.maxInputBytes` or an `autofetch` bound at or + below its default, and `shouldTriggerInference`. + So a host that puts a deny-list on every server it deploys can deploy a + modern one. +- `requestTimeoutMs` is one deadline per tool call and must be 1 to 2^31−1; + 0 is refused, where MCPL reads it as no watchdog. +- Over stdio it takes no credential: `token` or `accessProvider` on one is + refused. + +At the deadline, cancellation is requested for a leg still in flight; +between the rounds of an `input_required` continuation nothing is in flight +to cancel. Either way the outcome is reported as unknown (`no-response`), +and the call is never retried. The protocol has no cancel for a paused call, +so whatever the server holds for it, beyond the `requestState` it gave this +client, stays until the server lets it go. A call that ends for any other +reason after the server asked for another round, before its final answer, is +reported the same way, since the server may already have acted. An MCPL +server that refuses `2024-11-05` with `-32022` fails with +`McplProtocolVersionError`, and is not retried. The error names the +revisions the server says it supports ("none listed" for an empty list), or its error text when it sends no list, +and when the list includes `2026-07-28` it says to configure the server with +`protocol: 'modern'` (stdio) or reach it over http(s). + +Server and transport text the framework records has the server's configured credentials replaced, in both engines. That covers connection traces, console lines, stderr traces, ops alerts, and a tool call's failure as its caller, the model and the `tool:failed` trace get it. The credentials are those the configuration holds and those its dials sent: from the url as each dial sends it (a WebSocket dial sets the token in the query, which writes the whole query again, form-encoded, and an access provider's token can be in it), credential-like path segments and query names, credential-shaped or credential-named query values, and a WebSocket url's userinfo, in their encoded and decoded forms, replaced wherever they occur; every non-empty query parameter as its whole `name=value`, where no letter or digit adjoins it; the static `token`, and each token an access provider handed a dial, the newest sixteen, raw and url-encoded; and a stdio server's `env` values whose names have a part ending in a credential word, such as `GITHUB_TOKEN`, `PGPASSWORD` or `SESSION_COOKIE`, raw and url-encoded. A stdio server's command-line arguments aren't derived. Standalone strings shorter than eight characters stay unchanged because they may be ordinary words; short query values are still removed with their parameter names. Treat a submitted configuration as immutable while the server uses it: apply changes with `restartMcplServer(id, newConfig)`. Secrets neither the configuration holds nor a dial sent remain outside this filtering. JSON-RPC tool answers, and the error a host's own connect raises, pass through as sent. + #### Feature sets `enabledFeatureSets` and `disabledFeatureSets` select which of a server's diff --git a/changelog.d/cleartext-mcp-credential-warning.added.md b/changelog.d/cleartext-mcp-credential-warning.added.md new file mode 100644 index 000000000..3a58775c4 --- /dev/null +++ b/changelog.d/cleartext-mcp-credential-warning.added.md @@ -0,0 +1,16 @@ +- Connecting to an MCP server warns once when a credential would cross the + network unencrypted, to an `http://` or `ws://` url whose host isn't this + machine: its `token` or `accessProvider`, or a secret the url carries + itself (a `ws://` url's userinfo, a query parameter whose name says + credential: `token`, `key`, `secret`, `password`, `auth` or `signature`, + or whose value is shaped like one under a plain name, a query parameter + name that may itself be one, or a path segment that looks like one, as + the redacted url judges it). A name that may itself be + a credential is counted in the warning, never quoted. An `http(s)://` url + with userinfo is refused instead, since the HTTP client can't send it and + its error would quote it. Loopback hosts are + exempt, and so are the unspecified addresses `0.0.0.0` and `[::]`, which + reach this host too. It warns rather than refuses, since a private network + or a TLS-terminating proxy can make that deliberate. + `serverConfigWarnings` is exported beside `serverConfigProblems`, so a host + can apply the same check in its own configuration validation. diff --git a/changelog.d/legacy-mcp-protocol-boundary.breaking.md b/changelog.d/legacy-mcp-protocol-boundary.breaking.md new file mode 100644 index 000000000..80ffaa865 --- /dev/null +++ b/changelog.d/legacy-mcp-protocol-boundary.breaking.md @@ -0,0 +1,37 @@ +- **MCPL/MCP server authors:** the legacy engine now enforces the revision it + speaks. `McplServerConnection` offers MCP `2024-11-05` and now also requires + it back: an `initialize` result with any other `protocolVersion`, or with + none, fails as `McplProtocolVersionError` (`kind: 'mismatch'`), stating what + each side said. Every server family probed answers `2024-11-05`, but a + hand-written server that omits the field must add it. An `initialize` + answered with JSON-RPC `-32022` fails as `McplProtocolVersionError` + (`kind: 'rejected'`), keeping the server's `supported` list, code and data. + When that list names modern MCP, the message says to configure the server + with `protocol: 'modern'`. A protocol-version verdict is never retried: no + reconnect stub, and a reconnecting connection stops its backoff loop + (`reconnect-failed` carries `permanent: true` and raises the ops alert at + once). At startup the verdict raises the same `mcpl-down` ops alert, so a + server refused for its revision doesn't drop out with only a log line. The + established revision is on `connection.protocolVersion`. +- `tools/list` now follows `nextCursor` across pages, so a paginating server's + whole inventory reaches the agent; before, every page after the first was + silently dropped. A repeated cursor, or more than 100 pages, is an error + rather than a partial inventory. +- **Hosts and operators:** a server configuration with both `command` and + `url` and no `transport` is now refused before anything is spawned or + dialed. Before, it was launched as stdio and the url was silently + ignored. Migration: drop the field you don't use, or set `transport` to + say which one is used. Unchanged: a configuration with only one of them, + or with both and a `transport`, resolves as before (a host can still + layer a url and `transport: 'websocket'` over a file's command). A legacy + `requestTimeoutMs` above 2^31−1 is refused too: Node ran it as a timer of + about 1 ms, so every request timed out at once. Every value that worked, + `0` included, passes. +- **Hosts:** `stop()` now rejects, after its whole teardown, when an MCP + server's child couldn't be confirmed exited, naming each such child and + its server. Before, closing a stdio server returned as soon as a signal + was sent, so `stop()` never failed for this. Code that chains work on + `stop()`, such as exiting a TUI or switching sessions, should catch that + rejection: the framework is stopped either way. As with any failed + teardown, an owned store stays open, and a later `stop()` retries the + reap, completing once the child has exited. diff --git a/changelog.d/mcp-standard-tool-results.changed.md b/changelog.d/mcp-standard-tool-results.changed.md new file mode 100644 index 000000000..b0a5da276 --- /dev/null +++ b/changelog.d/mcp-standard-tool-results.changed.md @@ -0,0 +1,164 @@ +- What the model sees of an MCP tool result from a server that isn't an + MCPL peer (an MCP-only legacy server, or a modern server) now follows its + content types: + - text stays text; + - an inline image stays native when a provider can take it: png, jpeg, + gif or webp, within 5 MB decoded; + - audio, binary resources, other images, and media with inline data but + no type are saved to the workspace (`tool-results/`), each under a + unique name, with a bounded stub saying where. If a payload can't be + saved, the result is reported as incomplete (`isError`), noting the tool + may already have completed, never as a success with part of it missing, + and the host's log says why the save failed; + - a failed result is read as its text alone, so there an inline image is + saved and stubbed like any other payload; + - a `resource_link` is shown as a link, with its name and URI, and never + fetched; + - an embedded text resource shows its text; + - an unknown block is shown as JSON, bounded, saying how much was left + out. A malformed entry is named and left out, and content that isn't a + list of blocks makes the result a failure. + + Before, any non-text block turned the whole result into raw JSON. + `structuredContent` is kept by presence on the new `ToolResult.structured` + (`false`, `0` and `null` are values). The model sees it as JSON when the + server's content has no text block of its own; a structured-only result + used to reach the model as an empty string. MCPL peers keep their RFC-005 + reading and gain `structured`. +- A script whose tool call returns `structuredContent` receives the + server's result itself, as one JSON object: `{ content, structuredContent, + isError, error? }`. From a server that isn't an MCPL peer, `content` is the + blocks as sent, so images and other payloads survive. `structuredContent` + is the tool's own JSON, as sent. Content that is present but isn't a + list of blocks, from such a server, is the malformed result the model + reads (`isError`, with the malformed notice as `error`), its structured + content kept, where it was an empty success. +- An MCPL peer's blocks in a script's `content` get the treatment every + lane the model reads gives them, since what a script prints comes back to + the model. An RFC-005 reference arrives without its URI, as the host's + reference id (`refId`), what the server said about it in that block + (`name`, `mimeType`, `sizeBytes`, `digest`, `expiresAt`, with control and + bidi characters stripped from the text fields, as in the model's stub), + and `savedTo`, the workspace path the framework fetched its payload to for + the script (`null` when it couldn't; `fetch_reference` takes the id as its + `ref_id` argument). An unsaved reference that counts as expired also + carries `expired`, the reason the model's stub gives, and isn't fetched. + A block the classifier rejects arrives as the notice the model reads in + its place, and inline data that also claims a bulk disposition as the + withheld notice. #181, by @Meganeuridae, first + found scripts losing `structuredContent`, and it kept the saved paths of + RFC-005 payloads for them by falling back to the text rendering. Here the + paths are in the structured object, as `savedTo`. +- If a script's structured result would exceed the 5 MB script cap, the + framework saves it to the workspace and the script gets + `{ isError, oversized: { chars, savedTo } }`. If it can't be saved, the + script gets a failure that says the tool may have completed. Results + without structured content keep the existing script contract. +- A result that can't be converted to JSON, such as a cyclic object or a + `BigInt` from an in-process module, reaches the script as an error that + names the tool and says the call itself completed: for a structured + result, the object above with `content: []`, `structuredContent: null`, + `isError: true` and the `error`, and `Error: …` otherwise. Before, the + script got the runner's bare error, such as `Converting circular + structure…`. +- The `code_execution` tool's description now says what a script's tool + function returns: the text, or for a structured result the JSON object + above, with references and the oversized form. These two changes, and an + end-to-end test with real Python adapted from #181's, come from + @Meganeuridae's review of this change against #181. +- The direct tool path (`executeToolCall`, `ModuleContext.callTool`) adds + `structured` to an MCP tool's result, success or failure, when the server + sent `structuredContent`. A modern server's result reaches it as an MCPL + server's does: an error result is a failure with its text, and a success + is the server's content array as `data`. +- Closing a stdio MCP/MCPL server now waits for the child to exit, sending + SIGKILL after 2 s. A child still alive 2 s after SIGKILL fails `close()` + with an explicit error. Before, close returned as soon as a signal was + sent. `stop()` still finishes shutting everything else down after such a + failure, and then rejects (see the breaking notes). Concurrent `close()` + calls share one teardown, and a + later call re-checks a child that couldn't be reaped. A launch whose exit + can't be confirmed is never orphaned, and nothing launches beside it: + - `McplServerConnection.connect()` and `connectWithReconnect()` throw the + new `McplUnreapedLaunchError`, whose `connection` owns the launch; its + `close()` retries the reap. A reconnect attempt halts the loop instead + (`reconnect-failed`, `permanent: true`). Reconnect attempts run one at + a time, and never start before the previous child has exited. + - `reconnectAfterFailure()` on a connection without reconnect ends it + with `close()`, so a child it can't reap stays owned. + - `close()` during a reconnect handshake ends that launch before it + resolves, a WebSocket dial still opening included. In the framework, + `stop()` and a disconnect of the id end a first connect still in its + handshake the same way, and a WebSocket dial still waiting for its + access provider: `connect()`, `connectWithReconnect()` and + `McplServerRegistry.addServer()` take an optional `AbortSignal`, and an + aborted `connectWithReconnect()` makes no retry stub. + - `McplServerRegistry` holds an id until its child is confirmed exited: + `removeServer` frees it only once the close succeeds, a second + `addServer` for an id still connecting is refused, and `closeAll()` + keeps each connection it couldn't reap, reporting every failure. + - In the framework, a disconnect or failed connect whose cleanup can't + reap the child keeps the server registered, closed and listed as + disconnected, and its error says so; its tools leave the agents' list. + A connect for that id is refused until a disconnect, retried once the + child is gone, confirms the exit and completes the removal. A failed + startup connect that leaves such a child raises the `mcpl-down` ops + alert. +- One owner per server id, across both families: a connect is refused while + another connect for that id is in flight (a legacy handshake included) or + a disconnect is still tearing it down. A disconnect closes a registered + connection immediately, and a legacy connect still handshaking is cleaned + up when its handshake settles (a modern connect in flight, or such a + legacy one, fails with "was not connected"); then it removes what was + registered. A teardown never removes what a later connection registered. + `restartMcplServer` refuses, before it tears the running server down, what + its connect would refuse: whatever the configuration check refuses (such as + an unusable binding, userinfo on an http(s) url, or MCPL policy that asks + for what a modern server lacks), a new id another server holds, a tool + prefix another server or a module holds or that meets another server's, a + framework whose `stop()` has begun, and a connect or disconnect of the + server in flight. Before, a restart disconnected first, + so a refused connect left the server gone; now the server keeps running. + From those checks through its connect, the restart holds the id and the + tool prefix it moves the server to: a connect or a restart started + meanwhile is refused them, or a prefix that meets that one, and a module + added meanwhile is refused that prefix, each naming the restart, rather than taking them while the old + server is torn down. The id and prefix it moves the server from are free + once its disconnect is done. A disconnect of the id it moves the server + to (its own, for a restart under the same id), or a `stop()`, that begins + during the restart ends it, as a disconnect ends a connect in flight, and + the restart's error says the server is down. A restart whose connect + fails leaves the server down, configured under the new configuration and + its id, and holding its tool prefix, as any failed connect does: it's + listed as not connected, and a restart or a disconnect under that id + tries it again or removes it. `addModule` also refuses a name a + server's tool prefix holds, as `create()` does at startup. A disconnect + releases the tool-prefix claims its own server holds. Treat a submitted + configuration as immutable while the server uses it: change settings with + `restartMcplServer(id, newConfig)`, and rotate a network server's + credentials through `accessProvider` (a stdio server's changed `env` goes + through a restart with the new configuration). +- A server configuration naming no usable transport is now an error before + anything is spawned or dialed. That covers a `transport` that doesn't + match the url's scheme, `transport: 'http'` without a url, and an + unrecognized scheme. `McplServerConnection.connect()` and + `connectWithReconnect()` apply the framework's whole check (so `protocol` + on a WebSocket url is refused too), and a configuration error is never + turned into a retry stub. `openTransport()` and `isWebSocketTransport()` + decide by the same rule, so an http(s) url is never dialed as a WebSocket. +- A configuration error, a WebSocket dial failure and `listMcplServers()` + show a server's url redacted, since a credential may ride in it: no + userinfo, each query parameter's value replaced, and so is a path + segment that looks like a credential (16 or more characters with a + digit, as a token or uuid has, or 32 or more of anything), such as the + per-user secret some hosted servers put in the path. A url that doesn't + parse isn't shown. An `accessProvider` that throws is logged by + its error's class and its message's length, not its text, which may quote + the credential. +- `listMcplServers()` lists a configuration that connecting would refuse + with `refused`, which says why and is the verdict to read. Its `family` + and `transport` are null only when the configuration names no usable + transport, where the listing guessed a legacy binding before. A usable + binding refused for something else, such as an https url with + `requestTimeoutMs: 0`, keeps them. Its `policyEstablished` is null for a + modern server, which has no MCPL policy. diff --git a/changelog.d/mcpl-connect-in-flight.fixed.md b/changelog.d/mcpl-connect-in-flight.fixed.md index f972c8ff0..f3f6d9961 100644 --- a/changelog.d/mcpl-connect-in-flight.fixed.md +++ b/changelog.d/mcpl-connect-in-flight.fixed.md @@ -1 +1 @@ -- A runtime connect, a restart or a disconnect of an MCPL server whose connect is still in flight is now refused before it changes anything, with `MCPL server "" has a connect in flight; try again once it settles`. Nothing stops a connect in flight. A second connect went ahead beside it, and whichever settled last replaced the other in the registry without closing it. A disconnect removed the server's config and tool prefix, and the connect then registered the server without them, where no call reached it and only another disconnect cleared it. A restart begins with that disconnect. +- A runtime connect, or a restart of another server into an id whose legacy connect is still in its handshake, is now refused before it changes anything, with `MCPL server "" has a connect in flight; try again once it settles`. While a modern connect is in flight, the id reads as already registered. Before, a second connect went ahead beside a legacy one, and whichever settled last replaced the other in the registry without closing it. A disconnect of the id, in either engine, now ends the connect in flight and removes whatever it registered. Before, the disconnect removed the server's config and tool prefix, and the connect then registered the server without them, where no call reached it and only another disconnect cleared it. diff --git a/changelog.d/modern-mcp-client.added.md b/changelog.d/modern-mcp-client.added.md new file mode 100644 index 000000000..6b276304c --- /dev/null +++ b/changelog.d/modern-mcp-client.added.md @@ -0,0 +1,49 @@ +- Modern MCP servers (revision `2026-07-28`) can now be configured beside + MCPL ones, through the official SDK 2.x client. A server's URL decides its + family: `ws://`/`wss://` stays MCPL (MCP `2024-11-05`), and + `http://`/`https://` is modern MCP over Streamable HTTP. A stdio server + stays legacy unless its config sets `protocol: 'modern'`. A modern server's + tools use the same prefix, tool policy, RFC-008 class hints and dispatch + paths as any MCPL server's, and its tool prefix meets the same rules: it + can't be a module's name or another server's prefix, or meet another + server's, whichever family either server is. It has no MCPL surface: no grant, push, + channels, inference requests, quiesce plane or awareness gate. MCPL policy + that asks for that surface on one is a configuration error, such as + `allowHostCommands: true` or a scope's non-empty `whitelist`. Policy that + only takes something away or repeats a default, such as + `disabledCapabilities` or an allow-list like `enabledFeatureSets`, is + accepted as already satisfied, so a host that puts a deny-list on every + server it deploys can deploy a modern one. A modern `requestTimeoutMs` + outside 1..2^31−1 is a configuration error too, since 0 means "time out + now" to the SDK. Details: + - **stdio:** runs through the framework's own spawner (env allowlist, + `inheritEnv`, stderr lines including startup), launched once per connect. + - **HTTP credentials:** come from `token`/`accessProvider`, as a cached + bearer refreshed once on 401. + - **Deadline:** each call gets one `requestTimeoutMs` deadline across every + leg and the pauses between them. At the deadline cancellation is + requested for a leg in flight (between continuation rounds none is), the + outcome is reported as unknown, and nothing is replayed. + - **Failures:** reported as `McplRequestError`, with the outcome taken from + what actually crossed the transport: `not-sent`, `error-response` (HTTP + 401/403 with `data.httpStatus`) or `no-response`. Only an error or a + `complete` result is a final answer: a call that ends after an + `input_required` answer, or after a result type this revision doesn't + define, is `no-response`. A complete result that can't be used is a + plain error. + - **Lifetime:** reconnects use the legacy backoff settings. A tool-list + change subscription is kept open, reopened if refused or lost, and its + changes go through ordinary admission, so they park under quiesce like + any wake. `close()` and `stop()` end any connect in flight. A + `ModernMcpConnection` starts once: a second `start()` returns the first + call's promise. When a stdio launch's first attempt fails and its cleanup + can't confirm the child exited, `ModernMcpConnection.connect()` throws + `McplUnreapedLaunchError`, whose `connection` owns the launch; its + `close()` retries the reap. + - **Status:** `listMcplServers()` entries gain `family`, `protocolVersion` + and `transport`. +- A server lost for good (it closed and won't reconnect), MCPL or modern, + now leaves the agents' tool list, as a disconnected one does. So does one + whose reconnecting halts for good (`reconnect-failed` with + `permanent: true`, which also raises the ops alert). An MCP tool + result marked as an error is traced as `tool:failed`, on both engines. diff --git a/changelog.d/redact-recorded-mcp-credentials.fixed.md b/changelog.d/redact-recorded-mcp-credentials.fixed.md new file mode 100644 index 000000000..98c9a85d6 --- /dev/null +++ b/changelog.d/redact-recorded-mcp-credentials.fixed.md @@ -0,0 +1 @@ +- Server and transport text the framework records has the server's configured credentials replaced, in both engines: connection traces, console lines, stderr traces, the last of a server's stderr that a failed handshake keeps (in its trace and console line), the ops alert (and so `logs/failures.log` and the webhook), an MCP tool call that fails without an answer, as its caller and the model get it and in the `tool:failed` trace's message and stack. The credentials are those the configuration holds and those its dials sent: from the url as each dial sends it (a WebSocket dial sets the token in the query, which writes the whole query again, form-encoded, and an access provider's token can be in it), credential-like path segments and query names, credential-shaped or credential-named query values (each as written and decoded), every non-empty parameter as its whole `name=value`, and a WebSocket url's username and password, its userinfo as one unit before its `@` (the username alone when there's no password), with no length floor, so a short password goes with its username and a short username alone goes too, and the Basic credential a WebSocket client sends for it; the static `token`, and each token an access provider handed a dial (the newest sixteen), raw, as JSON encoders write it (with `JSON.stringify`'s escapes, and with its slashes escaped too, as some encoders write them), and as a url carries it, percent- or form-encoded with either case of hex; and a stdio server's `env` values whose names have a part ending in a credential word, singular or plural, with any digits after it (`GITHUB_TOKEN`, `API_KEYS`, `PGPASSWORD`, `DB_PASS`, `AUTHORIZATION`, `SESSION_COOKIE`, `TOKEN2`), raw, as JSON encoders write them, and percent-encoded with either case of hex. A stdio server's command-line arguments are not derived. A string replaced on its own has at least eight characters, since a shorter one can't be told from ordinary text, and is replaced wherever it occurs. Every non-empty query parameter is also replaced as its whole `name=value`, only where no letter or digit adjoins the pair (so `v=2` leaves `dev=2` alone); a short value goes only that way. A connection's event records use a redactor built once for it, and built again once a dial has sent something new. An operation's records replace the credentials of the configurations present when it began and of those present when each record is made, so a server removed, or replaced by a restart, while the operation waits is still covered, and so is one connected meanwhile. Every place a credential matches is replaced, places that overlap included, and text that may come from any server is matched against every server's strings at once, so two servers' secrets that overlap are both replaced whole. Over HTTP, which only the modern engine speaks, a 401 or 403's response body is left out of the request error's data; where the HTTP client builds the error's message from the body, as it does for most statuses, the message is redacted like any record. Credentials neither the configuration holds nor a dial sent are outside this, and so are a stdio server's command-line arguments. JSON-RPC answers, and the error a host's own connect raises, pass as they are. diff --git a/changelog.d/stdio-pipe-failure-sigkill.fixed.md b/changelog.d/stdio-pipe-failure-sigkill.fixed.md new file mode 100644 index 000000000..250cc00fd --- /dev/null +++ b/changelog.d/stdio-pipe-failure-sigkill.fixed.md @@ -0,0 +1,20 @@ +- **A stdio MCP server whose stdin or stdout fails is now ended even when + it ignores SIGTERM, in both engines.** Since 0.22.2 such a failure sends + the child SIGTERM, and its exit closes the connection. A child that + ignored SIGTERM kept running, the connection stayed connected, and each + request waited out its timeout. The transport now sends SIGKILL after 2 s, + as its `close()` does, in the one attempt `close()` shares, so the + connection closes, and reconnects if it would after any exit, its close + still naming the pipe's error. + - Nothing more is written to the child after the failure. A request made + then is refused as not sent (`not-sent`), where it was written to the + failed pipe and reported at its deadline as possibly executed. One + written before the failure is still reported as possibly executed. + - A connected server's child that outlives SIGKILL too is held as a + launch that couldn't be reaped, as a reconnect holds one: reconnecting + halts, the `mcpl-down` alert is raised, its tools leave the agents' + list, and its id stays held until a disconnect or `stop()` retries the + reap and confirms the exit. + - Under Bun, a child that closes its stdin isn't reported at all, since + Bun emits no `'error'` event for the writes to it, so request timeouts + still bound it, as they do a child that stops reading. diff --git a/changelog.d/stop-ends-mcp-connect-in-flight.fixed.md b/changelog.d/stop-ends-mcp-connect-in-flight.fixed.md new file mode 100644 index 000000000..36105525a --- /dev/null +++ b/changelog.d/stop-ends-mcp-connect-in-flight.fixed.md @@ -0,0 +1,8 @@ +- **`stop()` ends an MCPL connect still in flight, in either engine, + instead of waiting for it.** The connect rejects with "was not connected: + the framework stopped while it was connecting", and `stop()` doesn't wait + on a server for it, in its handshake (its launch reaped), its setup or its + tool listing, which asks every server. A restart in flight ends too, its + connect refused once `stop()` has begun, so the server it was restarting + is down. A disconnect admitted before `stop()` began still finishes + first. diff --git a/package.json b/package.json index 80a3da6dd..ae20f1312 100644 --- a/package.json +++ b/package.json @@ -50,12 +50,14 @@ "@animalabs/chronicle": "^0.4.0", "@animalabs/context-manager": "^0.14.1", "@animalabs/membrane": "^0.5.87", + "@modelcontextprotocol/client": "^2.3.1", "chokidar": "^4.0.3", "discord.js": "^14.25.1", "ws": "^8.18.0" }, "devDependencies": { "@babel/parser": "^7.29.9", + "@modelcontextprotocol/server": "^2.3.1", "@types/node": "^22.0.0", "@types/ws": "^8.5.13", "tsx": "^4.21.0", diff --git a/src/code-execution/tool-definition.ts b/src/code-execution/tool-definition.ts index 65109e8cc..dbf37dae9 100644 --- a/src/code-execution/tool-definition.ts +++ b/src/code-execution/tool-definition.ts @@ -33,8 +33,15 @@ export function buildCodeExecutionToolDefinition(opts?: { "non-identifier character replaced by '_' " + "(e.g. tool 'mcpl--discord--fetch_history' is the function mcpl__discord__fetch_history, " + "and tool 'mcpl--dog-events--status' is mcpl__dog_events__status). " + - 'Each function takes a single dict of arguments and returns a string — the same text ' + - 'the tool would have returned to you directly; parse structured results with json.loads. ' + + 'Each function takes a single dict of arguments and returns a string: the text the tool ' + + 'would have returned to you directly, or "Error: ..." if the call failed. For a result ' + + 'with structured content, the string is a JSON object to parse with json.loads: ' + + '{"content": [...], "structuredContent": ..., "isError": false}, with "error" added when ' + + `"isError" is true. "content" holds the server's content blocks. A reference to a payload ` + + 'appears there as {"type": ..., "refId": ..., "savedTo": ...}, where "savedTo" is the ' + + `workspace path the payload was saved to (null if it wasn't) and "refId" is fetch_reference's ` + + 'ref_id. A structured result over 5 MB is saved to the workspace instead, and the string is ' + + '{"isError": ..., "oversized": {"chars": ..., "savedTo": ...}}. ' + "An exact-name lookup dict is also available: tools['mcpl--discord--fetch_history']({...}). " + 'Use top-level await; run independent calls in parallel with asyncio.gather. ' + 'Only what you print() (plus stderr and the exit code) comes back to you — intermediate ' + diff --git a/src/framework.ts b/src/framework.ts index c9b6f18f2..b06394192 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,6 +1,6 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { dirname, join } from 'node:path'; -import { INLINE_WITHHELD_TEXT, REFERENCE_LIMITS, classifyBlock, isInlineContradiction, isReferenceExpired, referenceRegistry, referenceStubOrNull, sanitizeLabel } from './mcpl/references.js'; +import { INLINE_WITHHELD_TEXT, REFERENCE_LIMITS, classifyBlock, isInlineContradiction, isReferenceExpired, referenceBlockForScript, referenceRegistry, referenceStubOrNull, sanitizeLabel } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, @@ -108,7 +108,7 @@ import { ConversationRouter, CONVERSATION_ROUTING_DEPRECATION_NOTICE } from './m import { INBOUND_SOURCE_KEY, type InboundAcceptanceObserver, type InboundChannelSource, type InboundLane, type InboundSource, type InboundUnscopedSource } from './mcpl/inbound-source.js'; import type { IncomingMessageFacts, RoutingDecision } from './mcpl/conversation-router.js'; import { safeSlice } from './safe-slice.js'; -import { boundedErrorMessage as thrownText } from './error-message.js'; +import { boundedErrorMessage, errorMessage } from './error-message.js'; import { ProviderWaits, EVERY_MODEL, isReleaseModel, withoutControls, type ProviderWait } from './provider-waits.js'; import type { WorkspaceModule } from './modules/workspace/index.js'; import type { ToolResultGuard, WithheldSpill } from './tool-result-guard.js'; @@ -155,9 +155,9 @@ function mcplServerGoneText(serverId: string, toolName: string): string { "so the call didn't run. If you need it, ask whoever runs this host."; } -/** An MCPL tool call refused while this host connects to its server, in a +/** A tool call refused while this host connects to its server, in a * runtime connect or a restart's: the server's prefix routes calls to it - * before the registry holds it, which it does once the handshake is done. */ + * before the server is registered, which it is once its connect is done. */ function mcplServerConnectingText(serverId: string, toolName: string): string { return `This host is connecting to the "${serverId}" server that provides ${toolName}, so the call didn't run. ` + 'Try again in a moment.'; @@ -171,13 +171,14 @@ function mcplToolDeniedText(serverId: string, toolName: string): string { "and calling it again won't change that. If you need it, ask whoever runs this host."; } -/** A runtime connect, restart or disconnect refused because a connect of the - * server is in flight (mcplConnecting). The registry takes a server only once - * its handshake is done, and nothing can stop that connect meanwhile, so it - * would register the server after whatever ran beside it. Whoever asked reads - * it: an operator, or an agent through a host's tool that passes it on. */ -function mcplConnectInFlightError(serverId: string): Error { - return new Error(`MCPL server "${serverId}" has a connect in flight; try again once it settles`); +/** A runtime connect or restart refused because a legacy connect of the + * server's id is still in its handshake, before its server is registered + * (mcpServerIdTaken). A modern connect registers its connection first, so an + * id it holds reads as already registered. A disconnect isn't refused: it + * ends the connect in flight, in either engine. Whoever asked reads it: an + * operator, or an agent through a host's tool that passes it on. */ +function mcplConnectInFlightText(serverId: string): string { + return `MCPL server "${serverId}" has a connect in flight; try again once it settles`; } /** Detect a supported image media type from magic bytes (the model API @@ -387,14 +388,15 @@ function mcplResultText(content: McpToolResultContent[] | undefined): string { */ function mcplToolFailure(content: McpToolResultContent[] | undefined, serverId: string, callName: string): ToolResult { const said = mcplResultText(content); - return { - success: false, - error: said.trim() - ? said - : `The "${serverId}" server reported an error for ${callName} without saying what went wrong. ` + - 'Whether it did anything is unknown: check before calling it again.', - isError: true, - }; + return { success: false, error: said.trim() ? said : mcplToolNoWordsText(serverId, callName), isError: true }; +} + +/** What an agent reads for a tool's error result with no words in it, or + * only whitespace, from a server of either engine: whether the call did + * anything is unknown. `callName` is the tool's name as it was called. */ +function mcplToolNoWordsText(serverId: string, callName: string): string { + return `The "${serverId}" server reported an error for ${callName} without saying what went wrong. ` + + 'Whether it did anything is unknown: check before calling it again.'; } /** @@ -533,7 +535,10 @@ import { isToolAllowed } from './mcpl/tool-policy.js'; import { EventGate, formatShadowWarning } from './gate/event-gate.js'; import { UsageTracker, persistedUsageProblem, type PersistedUsageState } from './usage/usage-tracker.js'; import type { SessionUsageSnapshot, UsageUpdatedEvent } from './usage/types.js'; -import { stderrTailOf, withStderr, type McplConnectionCloseInfo, type McplServerConnection } from './mcpl/server-connection.js'; +import { McplProtocolVersionError, McplUnreapedLaunchError, stderrTailOf, withStderr, type McplConnectionCloseInfo, type McplServerConnection } from './mcpl/server-connection.js'; +import { ModernMcpConnection } from './mcpl/modern-connection.js'; +import { asText, credentialRedactor, redactUrl, redactServerText, resolveServerBinding, serverConfigProblems, serverCredentialPatterns, serverTextRedactor } from './mcpl/protocol-family.js'; +import { MALFORMED_CONTENT_NOTICE, normalizeStandardToolResult } from './mcpl/tool-result-normalize.js'; import { boundedForLog, describeCause } from './mcpl/errors.js'; import { assertServerConfigBooleans, buildWebSocketUrl, isWebSocketTransport } from './mcpl/transport.js'; import type { @@ -639,6 +644,15 @@ interface DeferredWrite { } const bySeq = (a: { seq: number }, b: { seq: number }): number => a.seq - b.seq; +/** A server configuration refused before anything was spawned or dialed: + * only a corrected configuration helps, so nothing retries it. */ +class McpConfigurationRefusal extends Error { + constructor(message: string) { + super(message); + this.name = 'McpConfigurationRefusal'; + } +} + /** The reply helpers an MCPL connection hands an inbound request's handler. */ type InboundResponder = { id: string | number; @@ -672,7 +686,6 @@ function asCompiled(landed: ReadonlyArray<{ content: ContentBlock[] }>): Content const text = blocks.filter((b) => b.type === 'text').map((b) => (b as { text: string }).text).join(''); return [...blocks.filter((b) => b.type !== 'text'), { type: 'text', text } as ContentBlock]; } - /** Stamp a deferred write's durable id into the metadata it is stored with * (boot recovery dedups replays by it). Idempotent for an already-stamped * message. */ @@ -1643,6 +1656,28 @@ interface EphemeralRun { toolCallsCount: number; } +/** + * What a script gets for a tool result that can't be converted to JSON, + * such as a cyclic object or a BigInt that an in-process module returned. + * It names the tool, and says the call itself completed, so a script that + * reports it doesn't invite a blind retry. The host's log gets the same + * line, since the tool that returned it needs fixing whether or not the + * script reports it. + */ +function unconvertibleScriptResult(toolName: string, err: unknown, ownError?: string): string { + let reason: string; + try { + reason = err instanceof Error ? err.message : String(err); + } catch { + reason = 'a value with no text form'; // String() throws on an object with no prototype + } + const own = ownError ? `, with this error: ${ownError}` : '.'; + const text = `tool '${toolName}' returned a result that can't be converted to JSON, so the script can't receive it ` + + `(${reason}). The call itself completed${own}`; + console.error(`[code_execution] ${text}`); + return text; +} + /** * Longest failure text the inference-failure pipeline writes to one sink: the * reason in the exhaustion stderr line, a field of a logs/failures.log record, @@ -1907,6 +1942,22 @@ export function shedMayHelp(errorType: string | undefined, httpStatus: number | return errorType === 'invalid_request' && (httpStatus === undefined || httpStatus === 400 || httpStatus === 422); } +/** + * What a thrown value says, safely and briefly, as boundedErrorMessage gives + * it, with `recorded` applied before the cut, so no part of a credential + * across the cut is kept. `recorded` is a server's redactor where the text + * can quote what that server sent; by default the text is kept as it is. + */ +function thrownText(error: unknown, recorded: (text: unknown) => string = asText): string { + let text: string; + try { + text = recorded(errorMessage(error)); + } catch { + text = '(a value that could not be read)'; + } + return boundedErrorMessage(text); +} + /** A stored value that read but isn't in a form this code takes, as against * a read that failed: starting again changes nothing until it's repaired, * so the refusal that quotes it says so instead of "once it can be read". */ @@ -1921,6 +1972,11 @@ function jsonKind(value: unknown): string { : `a ${typeof value}`; } +/** A restart's hold on the id and tool prefix it moves its server to; + * `restarting` names the server it restarts. Compared by identity. + * `endedBy` says what ended it before its connect: a disconnect of that id. */ +type McpRestartClaim = { readonly restarting: string; endedBy?: string }; + /** Why the framework cancelled a stream it owns: the mark it records, by * agent and stream, before the cancel, for the stream's terminal to read. */ type FrameworkCancelKind = 'turn_ended' | 'budget_restart' | 'quiesce_abandoned' | 'shutdown'; @@ -2418,6 +2474,15 @@ export class AgentFramework { private mcplToolServers: Map = new Map(); /** Servers whose tools/list answered in the latest refresh. */ private mcplListedServers: Set = new Set(); + /** refreshMcplTools' starts, and the start whose result was published + * last: a refresh that a later-started one has overtaken publishes + * nothing, since that one listed every server after it. */ + private mcplToolsRefreshStarted = 0; + private mcplToolsRefreshPublished = 0; + /** How many times each server's connection has closed. A listing that + * began before its server's latest close is of a session that's over, + * and refreshMcplTools publishes none of it. */ + private mcplServerCloses: Map = new Map(); /** Zero-match tool-pattern diagnostics (checkToolPatterns): set at the end * of create(), when the tool universe is first complete. */ private toolPatternChecksArmed = false; @@ -2428,13 +2493,27 @@ export class AgentFramework { private toolPatternKnownPrefixes: Map = new Map(); /** Maps serverId → McplServerConfig for prefix lookup. */ private mcplServerConfigs: Map = new Map(); - /** Servers whose connect is in flight (connectMcplServerInternal), from the - * registry's handshake until it settles: a call their prefix routes then - * finds no server registered, and is told this host is connecting to it. - * One mark per id: the registry holds one connection per id, and a - * second connect, a restart or a disconnect of an id whose connect is in - * flight is refused (refuseMcplConnect, runDisconnectMcplServer). */ - private mcplConnecting: Set = new Set(); + /** Modern-MCP (2026-07-28) servers, by id; read through modernServers(). + * They are kept apart from the MCPL registry on purpose: they have no + * grant, planes or server→host traffic, so the MCPL machinery (quiesce's + * plane pauses, awareness gates, policy) never sees them. They take part + * only through the tool paths, and their list changes reach agents + * through ordinary admission. */ + private modernMcpConnections: Map | undefined; + /** Connects and disconnects in flight, by server id (holdMcpServerId); + * created on first use, like modernMcpConnections. */ + private mcpServerOps: Map }> | undefined; + /** Each legacy connect in flight's own abort (connectHeldMcplServer): + * stop() and a disconnect of its id end it at once, its launch reaped, + * rather than waiting out its handshake. Created on first use, as the + * restart claims are (legacyConnectAborts()). */ + private legacyConnectAbortMap: Map | undefined; + /** The ids and tool prefixes restarts hold, each with the claim of the + * restart holding it, from the restart's checks through its connect + * (restartMcplServer): no other connect or restart takes them while the + * old server is torn down. Created on first use (restartClaims), like + * mcpServerOps. */ + private mcpRestartClaims: { ids: Map; prefixes: Map } | undefined; /** Host capabilities advertised during the MCP handshake — stored so servers * can be connected at runtime (connectMcplServer) after initialization. */ private mcplHostCapabilities: McplHostCapabilities | null = null; @@ -3240,6 +3319,53 @@ export class AgentFramework { return !!this.stopPromise || !!this.queue?.isClosed; } + /** stop()'s first step for each connect still in flight, in either engine: + * end it, so it fails at once and its operation ends, its launch reaped, + * rather than holding stop()'s wait until its handshake's deadline. A + * modern connect's connection is registered before it starts and is + * closed; a legacy one's handshake, and the access provider a WebSocket + * dial waits for, are aborted (legacyConnectAborts). A modern close's + * outcome is the teardown's to report (closeModernMcpConnections). */ + private endConnectsInFlight(): void { + for (const [id, op] of this.mcpServerOps ?? []) { + if (op.kind !== 'connect') continue; + this.modernServers().get(id)?.close().catch(() => { /* reported by the teardown's close */ }); + const abort = this.legacyConnectAborts().get(id); + if (abort) { + abort.abort(); + } else if (this.mcplServerRegistry?.getServer(id)) { + // Past its handshake, setting up: the registered connection's close, + // as a disconnect's, fails what the setup waits for. + this.mcplServerRegistry.removeServer(id).catch(() => { /* reported by the teardown's close */ }); + } + } + // A connect waiting on other servers, listing every server's tools, stops + // waiting (untilStopBegins). + for (const wake of [...(this.stopWaiters ?? [])]) wake(); + } + + /** Wakers for the waits a connect makes that a stop() ends: see + * untilStopBegins. Made on first use. */ + private stopWaiters: Set<() => void> | undefined; + + /** `work`, or undefined once stop() has begun, whichever comes first: a + * connect's last step (its tool listing, which asks every server) needn't + * hold stop()'s wait for operator operations on a server that doesn't + * answer. The work goes on; the teardown's close ends what it waits for. */ + private async untilStopBegins(work: Promise): Promise { + work.catch(() => { /* its caller has stopped waiting, or reads it below */ }); + if (this.stopHasBegun()) return undefined; + const waiters = (this.stopWaiters ??= new Set()); + let wake!: () => void; + const stopped = new Promise((resolve) => { wake = () => resolve(undefined); }); + waiters.add(wake); + try { + return await Promise.race([work, stopped]); + } finally { + waiters.delete(wake); + } + } + private refuseOperatorAfterStop(verb: string): void { if (this.stopHasBegun()) throw new Error(verb + ' refused: the framework is stopping'); } @@ -3280,6 +3406,13 @@ export class AgentFramework { const step = (run: () => void): boolean => { try { run(); return true; } catch (error) { shutdownErrors.push(error); return false; } }; + // No MCP server is admitted from here on (stopHasBegun): none is launched + // or dialed while the rest are being closed, so none outlives the + // framework. A connect still in flight is ended now, in either engine, + // so it fails at once and its operator operation ends, rather than + // holding the wait for operator operations below until its handshake's + // deadline. + step(() => this.endConnectsInFlight()); step(() => this.pushCoalescer?.suspend()); step(() => this.flushCoalescingSnapshot()); this.running = false; @@ -3472,11 +3605,12 @@ export class AgentFramework { // the mount turn it holds, bounded only by teardownTimeoutMs. this.guardSpillsClosedFor = stoppingGeneration; - // Stop modules and MCPL servers in parallel + // Stop modules and both engines' MCP servers in parallel const shutdownPromises: Promise[] = [this.moduleRegistry.stopAll(teardownTimeoutMs)]; if (this.mcplServerRegistry) { shutdownPromises.push(this.mcplServerRegistry.closeAll(teardownTimeoutMs)); } + shutdownPromises.push(this.closeModernMcpConnections(teardownTimeoutMs)); // Timed-out or failed teardown retains its context and may still write. // Sync available state best-effort, but leave its store open for a retry. for (const result of await Promise.allSettled(shutdownPromises)) { @@ -4616,6 +4750,17 @@ export class AgentFramework { * past it. */ async addModule(module: Module): Promise { + // A server's tool prefix and a module's name share one namespace, as + // create() checks at startup: a module isn't added under a prefix a + // server holds, or one a restart is moving its server to. + const server = this.mcplPrefixMap.get(module.name); + if (server !== undefined) { + throw new Error(`Module "${module.name}" collides with MCPL server "${server}"'s toolPrefix "${module.name}"`); + } + const claimedBy = this.restartClaims().prefixes.get(module.name); + if (claimedBy) { + throw new Error(`Module "${module.name}" collides with the toolPrefix "${module.name}" the restart of "${claimedBy.restarting}" holds`); + } this.resetCompletedStop('addModule()'); await this.moduleRegistry.addModule(module); this.emitTrace({ type: 'module:added', moduleName: module.name }); @@ -4970,13 +5115,24 @@ export class AgentFramework { return this.referenceFetcherInstance; } - /** Register every RFC-005 reference in a tool-result content array against - * its server, then eagerly fetch the ones inside the autofetch envelope - * (below the eager ceiling, connection origin — never worse than the - * server having inlined the bytes). Failures are non-fatal: the stub then - * says fetch_reference instead of a path. */ - private async autoFetchReferences(content: unknown[] | undefined, serverId: string, eager: boolean): Promise { - if (!content) return; + /** Register every RFC-005 reference in an MCPL peer's tool-result content + * against its server, then fetch by `mode`: + * - `register`: nothing. An in-process caller (executeMcplToolCall) may + * never show the result to an agent, and an eager fetch would spend the + * server's fetch budget, which the agent's own fetch_reference needs, on + * bytes nobody asked for. A stub later made of a registered reference is + * bound, so fetch_reference can fetch it. + * - `model`: eagerly fetch the ones inside the autofetch envelope (below + * the eager ceiling, connection origin — never worse than the server + * having inlined the bytes). Failures are non-fatal: the stub then says + * fetch_reference instead of a path. + * - `script`: as `model`, then fetch every reference ("eagerly lazy"): a + * script receiving a result is the strongest available signal that the + * bytes are wanted (scripts process, they do not browse), so the stub it + * receives carries a workspace path it can open directly. + * A uri the result calls expired is fetched in no mode. */ + private async autoFetchReferences(content: unknown, serverId: string, mode: 'register' | 'model' | 'script'): Promise { + if (!Array.isArray(content)) return; // Register every block first, then judge each uri once, by its latest // record. The registry keys records by uri and evicts the one registered // least recently, so a uri this result mentions keeps its record through @@ -4993,18 +5149,23 @@ export class AgentFramework { const expiredHere = byUri.get(c.testimony.uri)?.expiredHere === true || isReferenceExpired(c.testimony); byUri.set(c.testimony.uri, { record, expiredHere }); } - if (!eager) return; + if (mode === 'register') return; const fetcher = this.getReferenceFetcher(); - const eligible = [...byUri.values()] - .filter(({ record, expiredHere }) => !expiredHere && fetcher.isEagerEligible(record)) - .map(({ record }) => record); + const live = [...byUri.values()].filter(({ expiredHere }) => !expiredHere).map(({ record }) => record); // Parallel with a count cap: N small fetches cost one round-trip of // wall-clock, and anything past the cap stays lazy (stub says so). const MAX_EAGER_PER_RESULT = 8; + const eligible = live.filter((record) => fetcher.isEagerEligible(record)); await Promise.all(eligible.slice(0, MAX_EAGER_PER_RESULT).map(async (record) => { const outcome = await fetcher.fetch(record, { timeoutMs: EAGER_FETCH_TIMEOUT_MS }); if (!outcome.ok) console.error(`[rfc005] eager fetch of ${record.refId} failed: ${outcome.error}`); })); + if (mode !== 'script') return; + // A record the eager pass saved returns at once. + for (const record of live) { + const outcome = await fetcher.fetch(record); + if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); + } } /** Core agent_settings keys — extension keys must not collide with these. */ @@ -11859,7 +12020,14 @@ export class AgentFramework { */ private coalescingBinding(serverId: string): string { const config = this.mcplServerConfigs.get(serverId); - return createHash('sha256').update(JSON.stringify([serverId, config?.url ?? null, config?.command ?? null, config?.args ?? null])).digest('hex').slice(0, 16); + // A stdio server switched to the modern family is another peer, so its + // binding changes. The marker is added only for protocol: 'modern', so + // every binding stored for a legacy configuration stays the same. A + // network server's family follows its url's scheme, which is hashed. + const family = config?.protocol === 'modern' ? ['modern'] : []; + return createHash('sha256') + .update(JSON.stringify([serverId, config?.url ?? null, config?.command ?? null, config?.args ?? null, ...family])) + .digest('hex').slice(0, 16); } /** @@ -17859,38 +18027,46 @@ export class AgentFramework { if (!this.mcplServerRegistry) { return { success: false, error: 'MCPL not initialized', isError: true }; } - const server = this.mcplServerRegistry.getServer(serverId); - if (!server) { + const modern = this.modernServers().get(serverId) ?? null; + const server = modern ? null : this.mcplServerRegistry.getServer(serverId); + if (!modern && !server) { return { success: false, error: this.mcplServerAbsentText(serverId, call.name), isError: true }; } + const config = this.mcplServerConfigs.get(serverId); const toolName = call.name.slice(prefix.length + 2); // Strip prefix + '--' - if (!isToolAllowed(toolName, this.mcplServerConfigs.get(serverId))) { + if (!isToolAllowed(toolName, config)) { return { success: false, error: mcplToolDeniedText(serverId, call.name), isError: true }; } + const args = (call.input && typeof call.input === 'object') ? call.input as Record : {}; try { - const result = await server.sendToolsCall(toolName, call.input as Record); + const result = modern ? await modern.callTool(toolName, args) : await server!.sendToolsCall(toolName, args); // A server's error is a failure here too, as the model's dispatch - // reads it. A success is the server's own blocks, for the caller to - // use in process. - if (result.isError) return mcplToolFailure(result.content, serverId, call.name); - // RFC-005 §6.2: register the result's references with this server, so + // reads it: its text (mcplToolFailure), without its other blocks, and + // its structured content, kept by presence. A success is the server's + // own blocks and structured content, for the caller to use in process. + const direct = AgentFramework.directToolResult(result, serverId, call.name); + if (direct.isError) { + const { data: _blocks, ...failure } = direct; + return failure; + } + // RFC-005 §6.2: register an MCPL peer's references with this server, so // a stub later made of them is bound and fetch_reference can fetch // them through this server's address, where it has one. Nothing is // fetched here: an in-process caller may never show them to an agent, // and an eager fetch would spend the server's fetch budget, which the // agent's own fetch_reference needs, on bytes nobody asked for. A - // failure keeps only its text (mcplToolFailure), so no stub is made of - // its references and they aren't registered. - await this.autoFetchReferences(result.content, serverId, false) - .catch((e) => console.error('[rfc005] reference registration error:', (e as Error).message)); - return { - success: true, - data: result.content, - }; + // failure keeps only its text, so no stub is made of its references and + // they aren't registered. Only an MCPL peer's blocks are references; a + // modern server's and an MCP-only peer's are standard. + if (server && server.capabilities !== null) { + await this.autoFetchReferences(result.content, serverId, 'register') + .catch((e) => console.error('[rfc005] reference registration error:', (e as Error).message)); + } + return direct; } catch (err) { return { success: false, - error: err instanceof Error ? err.message : String(err), + error: redactServerText(err instanceof Error ? err.message : err, config ?? {}), isError: true, }; } @@ -18648,12 +18824,18 @@ export class AgentFramework { /** * Resolve one script-inner tool call to the string the script's tool - * function returns. Never rejects — errors become "Error: ..." strings, + * function returns. Never rejects: errors become "Error: ..." strings, * matching the managed PTC runtime ("Claude's code receives this error"). * - * Results are serialized with the HISTORY serializer (images become - * placeholders): programmatic tool results are text-only by contract, and - * a script variable holding megabytes of base64 helps nobody. + * A result without structured content is serialized with the HISTORY + * serializer: images become placeholders, since a script variable holding + * megabytes of base64 helps nobody. A result with structured content is + * one JSON object holding both of the server's views + * (scriptStructuredResult), so its content blocks reach the script as the + * dispatch gave them, images and payloads included. A module's result that + * can't be stored or sent, such as a cyclic object or a BigInt, is already + * an error naming the tool when the dispatch returns it (storableToolResult), + * so a script reads that error's text. */ private async handleScriptToolCall( agentName: string, @@ -18676,21 +18858,29 @@ export class AgentFramework { // stream while the script still runs and wedge the tool round. script.endTurn = true; } - if (result.isError) { + // An MCP structured result reaches the script whole, as one JSON object, + // so code can use structuredContent directly and doesn't have to parse + // it back out of text. + const structured = Object.prototype.hasOwnProperty.call(result, 'structured'); + if (result.isError && !structured) { return `Error: ${result.error ?? 'tool call failed'}`; } - if (result.data === undefined || result.data === null) return nonErrorResultText(result, 5_000_000); + if ((result.data === undefined || result.data === null) && !structured) { + return nonErrorResultText(result, AgentFramework.SCRIPT_RESULT_MAX_CHARS); + } // RFC-005 "eagerly lazy": a script receiving a result is the strongest // available signal that the bytes are wanted (scripts process, they do - // not browse) — materialize the references of a content array now, so - // the stub the script receives carries a workspace path it can open - // directly. Only a reference registered against its server can be - // fetched: an MCPL server's result content is, on arrival, including - // content a module got from a call of its own (ctx.callTool) and returns. - // A reference a module makes itself has no server, so its fetch is - // refused (and logged here) and the script gets its stub, as it does for - // a reference nested in other data, which isn't fetched. - if (Array.isArray(result.data)) { + // not browse), so the stub the script receives carries a workspace path + // it can open directly. An MCPL peer's references were fetched for the + // script at dispatch (autoFetchReferences). Any other tool's content + // array, a module's, is materialized here. Only a reference registered + // against its server can be fetched: a module's result can carry content + // it got from a call of its own (ctx.callTool), whose references were + // registered on arrival (executeMcplToolCall). A reference a module makes + // itself has no server, so its fetch is refused (and logged here) and the + // script gets its stub, as it does for a reference nested in other data, + // which isn't fetched. + if (Array.isArray(result.data) && this.resolveMcplTool(toolName) === null) { const fetcher = this.getReferenceFetcher(); for (const block of result.data) { const c = classifyBlock(block); @@ -18706,7 +18896,65 @@ export class AgentFramework { // reference is its stub at any depth and inline data claiming a uri or a // disposition is withheld (RFC-005). 5MB protocol safety cap only; no // context-cap truncation, no file spill. - return toolResultDataToHistoryString(result.data, 5_000_000); + if (structured) return this.scriptStructuredResult(result, toolName); + try { + return toolResultDataToHistoryString(result.data, AgentFramework.SCRIPT_RESULT_MAX_CHARS); + } catch (err) { + return `Error: ${unconvertibleScriptResult(toolName, err)}`; + } + } + + /** The 5MB protocol safety cap on what a script receives. */ + private static readonly SCRIPT_RESULT_MAX_CHARS = 5_000_000; + + /** + * A structured MCP result as a script receives it: one JSON object holding + * both of the server's views, `{ content, structuredContent, isError }`, + * plus `error` (the text) on a tool error. `content` is the content array + * the dispatch gave this call (the direct reading): the server's blocks as + * sent, so payloads and images survive, except that an MCPL peer's blocks + * get the treatment RFC-005 gives them on every lane the model reads + * (referenceBlockForScript). `structuredContent` is the tool's own JSON, as + * sent. Past the size cap the result stays valid JSON: the whole object is + * saved to the workspace, and the script gets + * `{ isError, oversized: { chars, savedTo } }`. If it can't be saved, the + * script gets a failure to materialize the result, saying the tool may + * already have completed, never a success missing its result. A result + * that can't be converted to JSON (a cyclic object, a BigInt), which the + * dispatch's own check makes unlikely here, gets the same object empty, + * `{ content: [], structuredContent: null, isError: true, error }`, the + * error naming the tool. + */ + private async scriptStructuredResult(result: ToolResult, toolName: string): Promise { + const object = { + content: Array.isArray(result.data) ? result.data : [], + structuredContent: result.structured, + isError: result.isError === true, + ...(result.isError ? { error: result.error ?? '' } : {}), + }; + let json: string; + try { + json = JSON.stringify(object); + } catch (err) { + const error = unconvertibleScriptResult(toolName, err, result.isError ? result.error : undefined); + return JSON.stringify({ content: [], structuredContent: null, isError: true, error }); + } + if (json.length <= AgentFramework.SCRIPT_RESULT_MAX_CHARS) return json; + const workspace = this.getWorkspaceModule(); + const mount = workspace ? this.firstWritableMountName(workspace) : null; + if (workspace && mount) { + const path = `${mount}/tool-results/${new Date().toISOString().slice(0, 10)}-script-${randomUUID()}.json`; + const written = await workspace.writeBinary(path, Buffer.from(json, 'utf8'), 'application/json').catch(() => null); + if (written?.success) { + return JSON.stringify({ isError: object.isError, oversized: { chars: json.length, savedTo: path } }); + } + } + return JSON.stringify({ + isError: true, + error: `the result (${json.length} chars) exceeds the ${AgentFramework.SCRIPT_RESULT_MAX_CHARS}-char script result cap ` + + `and could not be saved to the workspace, so it could not be delivered; the tool may already have completed`, + oversized: { chars: json.length, savedTo: null }, + }); } /** @@ -19039,6 +19287,13 @@ export class AgentFramework { this.store.appendToStateJson(PROCESS_LOG_ID, entryToStore); } + /** Count one server's close (see mcplServerCloses). Partially-constructed + * frameworks (tests build them with Object.create) have no map yet. */ + private noteMcplServerClosed(serverId: string): void { + this.mcplServerCloses ??= new Map(); + this.mcplServerCloses.set(serverId, (this.mcplServerCloses.get(serverId) ?? 0) + 1); + } + /** Forget one server's RFC-008 class declarations (see the close handler). */ private dropMcplToolClasses(serverId: string): void { // Tolerate partially-constructed frameworks (tests build them with @@ -21066,20 +21321,32 @@ export class AgentFramework { if (error instanceof DiscordAwarenessAccountingError) throw error; // Fail-open: log and continue with remaining servers const err = error instanceof Error ? error : new Error(String(error)); - console.error(`Failed to connect MCPL server "${config.id}": ${err.message}${withStderr(err)}`); - // Reaching this catch means no stub was created, so nothing will - // retry — except in the edge case where addServer succeeded (with a - // reconnecting stub) and a later setup step threw; config.reconnect - // is the best cheap approximation of that. + const message = redactServerText(err.message, config); + console.error(`Failed to connect MCPL server "${config.id}": ${message}${withStderr(err, (text) => redactServerText(text, config))}`); + // Whether anything retries is the registered connection's to say: + // usually nothing is left registered, but a legacy connect whose + // setup threw after its handshake leaves a reconnecting stub, and a + // launch that couldn't be reaped leaves a connection that won't + // reconnect. A protocol-version verdict and a refused configuration + // never retry. + const left = this.modernServers().get(config.id) ?? this.mcplServerRegistry?.getServer(config.id); const stderrTail = stderrTailOf(err); this.emitTrace({ type: 'mcpl:server-connect-failed', serverId: config.id, - error: err.message, - ...(stderrTail === undefined ? {} : { stderrTail }), + error: message, + ...(stderrTail === undefined ? {} : { stderrTail: redactServerText(stderrTail, config) }), attempt: 0, - willRetry: config.reconnect === true, + willRetry: !!left?.willReconnect, }); + // So the server doesn't vanish quietly: alert at once, since no + // attempt follows, for what needs the operator. That's a + // protocol-version verdict, as the reconnect path alerts for one, or + // a launch that couldn't be reaped, whose reap nothing retries on its + // own. + if (error instanceof McplProtocolVersionError || left?.holdsUnreapedChild) { + this.opsAlert('mcpl-down', config.id, `MCPL server unreachable at startup: ${message}`); + } } } @@ -21098,6 +21365,91 @@ export class AgentFramework { await this.refreshMcplTools(); } + /** The modern connections, created on first use, so a framework assembled + * field by field (as unit tests do) needs no setup for them. */ + private modernServers(): Map { + return (this.modernMcpConnections ??= new Map()); + } + + /** The restarts' claims (mcpRestartClaims), created on first use, as the + * modern connections are. */ + private restartClaims(): { ids: Map; prefixes: Map } { + return (this.mcpRestartClaims ??= { ids: new Map(), prefixes: new Map() }); + } + + private legacyConnectAborts(): Map { + return (this.legacyConnectAbortMap ??= new Map()); + } + + /** + * Hold a server id for one connect or disconnect, until `release()`. + * Together with the live connections (the MCPL registry and the modern + * map) this is what holds an id: from a connect's admission until its + * teardown is done, one owner at a time. So a connect is refused while + * anything holds its id, and a teardown never removes what a later + * connection registered. A disconnect takes the hold over, and waits for + * the operation it displaced (`previous`); a connect that finds itself + * `displaced` ends rather than setting up a server being removed. Each + * release frees only its own hold. + */ + private holdMcpServerId(id: string, kind: 'connect' | 'disconnect'): { + previous: Promise | undefined; + displaced: () => boolean; + release: () => void; + } { + const ops = (this.mcpServerOps ??= new Map()); + const previous = ops.get(id)?.settled; + let settle!: () => void; + const op = { kind, settled: new Promise((resolve) => { settle = resolve; }) }; + ops.set(id, op); + return { + previous, + displaced: () => ops.get(id) !== op, + release: () => { + if (ops.get(id) === op) ops.delete(id); + settle(); + }, + }; + } + + /** Whether a server's connection still serves it, or is reconnecting to + * it. One the host is closing or has closed doesn't, and neither does one + * lost for good, whose reconnecting has stopped. */ + private static mcpConnectionServes(connection: McplServerConnection | ModernMcpConnection): boolean { + if (connection instanceof ModernMcpConnection && connection.isClosed) return false; + return connection.isConnected || connection.willReconnect; + } + + /** + * A connect whose failed launch couldn't be confirmed exited. Its + * connection stays registered, closed, and holds the id, so nothing + * launches beside a child that may still be running; a disconnect retries + * the reap, and completes once the exit is confirmed. Once stop() has + * begun it refuses a disconnect, and retries the reap itself, so no + * disconnect is offered then. + */ + private unreapedConnectFailure(id: string, failure: Error): Error { + return new Error( + `${failure.message}. MCP server "${id}" stays registered, closed, until its exit is confirmed` + + (this.stopHasBegun() ? '' : ': disconnect it to retry the reap'), + { cause: failure }, + ); + } + + /** Close every modern-MCP connection, including any connect still in + * flight, so none outlives a failed startup or stop(). One whose close + * fails or is still pending stays registered, holding its id, as a failed + * connect's does, and every failure is thrown, as the MCPL registry's + * closeAll does. `timeoutMs` bounds the wait without cancelling cleanup. */ + private async closeModernMcpConnections(timeoutMs = DEFAULT_SHUTDOWN_TIMEOUT_MS): Promise { + await waitForShutdown([...this.modernServers()].map(([id, connection]) => ({ + label: `mcp:${id}`, + promise: Promise.resolve().then(() => connection.close()).then(() => { + if (this.modernServers().get(id) === connection) this.modernServers().delete(id); + }), + })), timeoutMs); + } + /** * Reconcile the durable ledger with Chronicle, then deliver every server's * work. A drain still running is waited for and then followed by a fresh @@ -21510,10 +21862,31 @@ export class AgentFramework { `[discord-awareness] could not recycle unhealthy connection ${connection.id}:`, closeError instanceof Error ? closeError.message : closeError, ); + // One without reconnect is closed, keeping a child it couldn't + // reap, so it won't serve again: its tools leave the agents' list, + // as a failed disconnect's do. One with reconnect is its reconnect + // loop's to settle, which halts (and refreshes) or serves again. + if (!connection.willReconnect) this.handleToolsListChanged(connection.id); }); })); } + /** + * Close a legacy connection for good through the registry. A close that + * can't confirm its child exited keeps the connection registered, holding + * its id, and emits no 'close', so no close listener takes its tools off + * the agents' list: this does, then rethrows the failure for the caller to + * report. A server the host has given up on leaves the list either way. + */ + private async closeMcplServerForGood(connection: McplServerConnection): Promise { + try { + await this.mcplServerRegistry!.closeServer(connection.id, connection); + } catch (error) { + this.handleToolsListChanged(connection.id); + throw error; + } + } + /** * Close a runtime connect's new connection for good, through the registry, * after a step of that connect failed. A close that fails keeps the entry, @@ -21524,7 +21897,7 @@ export class AgentFramework { connection: McplServerConnection, failedStep: string, ): Promise { - await this.mcplServerRegistry!.closeServer(connection.id, connection).catch((closeError) => { + await this.closeMcplServerForGood(connection).catch((closeError) => { console.error( `[mcpl] could not close MCPL server "${connection.id}" after its ${failedStep} failed; ` + 'it stays registered until a disconnect, restart or shutdown closes it:', @@ -21542,18 +21915,54 @@ export class AgentFramework { private async connectMcplServerInternal( config: import('./mcpl/types.js').McplServerConfig, deferAwareness = false, - ): Promise { + claim?: McpRestartClaim, + ): Promise { + if (this.stopHasBegun()) { + throw new Error(`MCP server "${config.id}" was not connected: the framework is stopping or stopped`); + } if (!this.mcplServerRegistry || !this.mcplHostCapabilities) { throw new Error('MCPL subsystem is not initialized'); } - // Every connect needs a readable awareness ledger: a runtime connect's - // awareness barrier reconciles it, and a stdio child taking the + // Configuration mistakes fail here, before any routing is registered: a + // combination naming no usable transport (an http(s) url with transport + // 'websocket', transport 'http' with a ws url or with no url at all), + // protocol on a URL, a modern deadline of 0, MCPL policy that asks for + // what a modern server lacks. Every valid legacy shape resolves exactly + // as before. + const family = AgentFramework.usableConfigFamily(config); + const taken = this.mcpServerIdTaken(config.id, claim); + if (taken) throw new Error(taken); + const hold = this.holdMcpServerId(config.id, 'connect'); + try { + return await this.connectHeldMcplServer(config, family, deferAwareness, hold.displaced); + } finally { + hold.release(); + } + } + + /** The part of a connect that runs holding its id, handing back the + * connection it registered. `displaced`: whether a disconnect has taken + * the id over and is waiting for this connect, which a legacy connect asks + * once its handshake is done. */ + private async connectHeldMcplServer( + config: import('./mcpl/types.js').McplServerConfig, + family: 'legacy' | 'modern', + deferAwareness: boolean, + displaced: () => boolean, + ): Promise { + if (!this.mcplServerRegistry || !this.mcplHostCapabilities) { + throw new Error('MCPL subsystem is not initialized'); + } + + // Every legacy connect needs a readable awareness ledger: a runtime + // connect's awareness barrier reconciles it, and a stdio child taking the // protective default derives that from it. Read it first, so an // unreadable ledger refuses the connect before anything is registered // under its id or spawned (a reconnecting server would otherwise be left // registered as a closed stub). With no outbox configured nothing is read. - this.getPlacedReactionBaseline(); + // A modern connection has no awareness barrier and no reaction default. + if (family === 'legacy') this.getPlacedReactionBaseline(); // Register prefix + config for tool dispatch routing (idempotent with // the pre-registration pass in initializeMcpl). One config gives an id @@ -21566,6 +21975,47 @@ export class AgentFramework { this.mcplPrefixMap.set(prefix, config.id); this.mcplServerConfigs.set(config.id, config); + if (family === 'modern') { + // Registered and wired before its first connect, so the connect in + // flight is already the framework's: stop() and disconnect close it, + // and a second connect for the same id is refused above. + const modern = ModernMcpConnection.create(config); + this.modernServers().set(config.id, modern); + this.wireModernEvents(modern, config); + let failure: Error | null = null; + try { + await modern.start(); + } catch (error) { + failure = error instanceof Error ? error : new Error(String(error)); + } + if (!failure && !modern.isClosed) { + this.emitTrace({ type: 'module:added', moduleName: `mcpl:${config.id}` }); + return modern; + } + // It failed, or stop() or a disconnect closed it while connecting: it + // never serves. The connect error is the one to report. + const reason = failure ?? new Error( + `MCP server "${config.id}" was not connected: ` + + (this.stopHasBegun() ? 'the framework stopped while it was connecting' : 'it was disconnected while connecting'), + ); + try { + await modern.close(); + } catch (cleanup) { + // A cleanup failure is reported beside it, never in its place. The + // launch may still be running, so the connection stays registered + // and holds the id: nothing launches beside it, and a disconnect + // retries the reap. + const message = cleanup instanceof Error ? cleanup.message : String(cleanup); + this.emitTrace({ type: 'mcpl:server-error', serverId: config.id, error: redactServerText(message, config) }); + if (this.modernServers().get(config.id) === modern) { + throw this.unreapedConnectFailure(config.id, new Error(`${reason.message}; its cleanup also failed: ${message}`, { cause: reason })); + } + throw reason; + } + if (this.modernServers().get(config.id) === modern) this.modernServers().delete(config.id); + throw reason; + } + // Record per-server channel subscription policy before the server // registers channels — handleRegister fires during the handshake. if (this.channelRegistry) { @@ -21591,6 +22041,10 @@ export class AgentFramework { `replaces the framework's protective default`, ); } + // Only a stdio server's configuration is copied, and its launch notes + // nothing. A dialing configuration stays the object the framework + // records the server's text with, since each dial notes what it sent + // against the object it read (noteDialSent). connectionConfig = { ...config }; Object.defineProperty(connectionConfig, 'env', { enumerable: true, @@ -21601,22 +22055,52 @@ export class AgentFramework { }), }); } - // The prefix above routes this server's tool calls already, and addServer - // registers the server only once its handshake is done, so until it - // settles a call is told this host is connecting to it. A connect that - // fails without reconnect leaves it unregistered, and a call is told it's - // gone; with reconnect, the stub it leaves is registered and answers. - this.mcplConnecting.add(config.id); let connection: McplServerConnection; + const abort = new AbortController(); + this.legacyConnectAborts().set(config.id, abort); try { - connection = await this.mcplServerRegistry.addServer(connectionConfig, this.mcplHostCapabilities); + connection = await this.mcplServerRegistry.addServer(connectionConfig, this.mcplHostCapabilities, abort.signal); + } catch (error) { + // Ended by stop() or by a disconnect of its id (its abort): said as a + // modern connect ended that way says it, with the handshake's own + // error as the cause. + const ended = abort.signal.aborted + ? `MCP server "${config.id}" was not connected: ` + + (this.stopHasBegun() ? 'the framework stopped while it was connecting' : 'it was disconnected while connecting') + : null; + // The registry keeps a failed launch that may still be running, as the + // closed connection that owns it. + if (error instanceof McplUnreapedLaunchError) { + throw this.unreapedConnectFailure(config.id, ended === null + ? error + : new Error(`${ended}; its cleanup also failed: ${error.cleanup.message}`, { cause: error })); + } + if (ended !== null) throw new Error(ended, { cause: error }); + throw error; } finally { - this.mcplConnecting.delete(config.id); + if (this.legacyConnectAborts().get(config.id) === abort) this.legacyConnectAborts().delete(config.id); + } + // stop() began while the handshake ran: its teardown has already + // collected the registry, so this connection would outlive it. Or a + // disconnect took the id over and is waiting for this connect. Either + // way, close it rather than set up a server that is being removed. + if (this.stopHasBegun() || displaced()) { + const why = this.stopHasBegun() ? 'the framework stopped while it was connecting' : 'it was disconnected while connecting'; + // A cleanup that can't reap the child is reported with the refusal, + // not in place of it and not dropped. The registry keeps such a + // connection, so the id stays held. + let cleanup = ''; + try { + await this.mcplServerRegistry.removeServer(config.id); + } catch (error) { + cleanup = `; its cleanup also failed: ${error instanceof Error ? error.message : String(error)}`; + } + throw new Error(`MCP server "${config.id}" was not connected: ${why}${cleanup}`); } // Wire listeners before either startup staging or the runtime global gate // releases control traffic needed for registration and marker service. - this.wireMcplEvents(connection); + this.wireMcplEvents(connection, config); if (deferAwareness) { // Staged startup: both planes are closed, so the §5.3 policy @@ -21624,7 +22108,7 @@ export class AgentFramework { // grant now, before staging ever releases. await this.registerMcplServerFeatures(config, connection); this.emitTrace({ type: 'module:added', moduleName: `mcpl:${config.id}` }); - return; + return connection; } const awarenessBarrier = this.installMcplDataPlaneGate(); @@ -21688,6 +22172,7 @@ export class AgentFramework { this.completeMcplDataPlaneGate(awarenessBarrier); this.emitTrace({ type: 'module:added', moduleName: `mcpl:${config.id}` }); + return connection; } /** Per-connection §17.8 fetch limiter: at most one in-flight fetch, at @@ -21704,13 +22189,17 @@ export class AgentFramework { * branch actually taken (anything but `close` is treated as `mcp-only`) * and a non-string `reason` is dropped. */ - private static policyRefusalFields(receipt: { fallback?: unknown; reason?: unknown }): { + private static policyRefusalFields( + receipt: { fallback?: unknown; reason?: unknown }, + recorded: (text: unknown) => string, + ): { fallback: 'close' | 'mcp-only'; reason: string | null; } { return { fallback: receipt.fallback === 'close' ? 'close' : 'mcp-only', - reason: typeof receipt.reason === 'string' ? receipt.reason : null, + // The server's own words, recorded as every server text is. + reason: typeof receipt.reason === 'string' ? recorded(receipt.reason) : null, }; } @@ -21724,6 +22213,16 @@ export class AgentFramework { connection: McplServerConnection, announce: unknown, ): Promise { + // The server's configuration as it is now, before any await: a record + // made later keeps it, whatever becomes of the server meanwhile. A server + // with no configuration has been removed, and its manifest isn't + // refreshed. + const config = this.mcplServerConfigs.get(connection.id); + if (!config) return; + // A queued run is contained as the announcement's own run is + // (wireMcplEvents), so its failure, such as a 'close' fallback whose + // close failed, is reported, through this server's redactor. + const recorded = serverTextRedactor(config); // The announcement's params are the server's hints, which a notification // may leave out: read the one the receipt reports here, before anything // is applied, so no later step can throw on them. @@ -21746,7 +22245,7 @@ export class AgentFramework { const cur = this.manifestRefreshState.get(connection.id); if (cur?.queued && !cur.inFlight) { cur.queued = false; - this.rerunManifestRefresh(connection, announce); + this.rerunManifestRefresh(connection, announce, recorded); } }, AgentFramework.MANIFEST_FETCH_FLOOR_MS - sinceLast).unref?.(); } @@ -21755,9 +22254,6 @@ export class AgentFramework { st.inFlight = true; try { - const config = this.mcplServerConfigs.get(connection.id); - if (!config) return; - // §17.4 fetch. A server that cannot answer keeps its previous // manifest in force — an unparseable/failed fetch teaches nothing and // narrows nothing (§17.5 step 1). @@ -21765,7 +22261,7 @@ export class AgentFramework { try { fetched = await connection.sendManifestRequest(); } catch (err) { - console.error(`[mcpl] ${connection.id}: mcpl/manifest fetch failed — previous manifest stands (§17.5): ${err instanceof Error ? err.message : err}`); + console.error(`[mcpl] ${connection.id}: mcpl/manifest fetch failed — previous manifest stands (§17.5): ${recorded(err instanceof Error ? err.message : err)}`); return; } if (!fetched || typeof fetched !== 'object') { @@ -21828,23 +22324,27 @@ export class AgentFramework { // feature state, mirroring the §5.3 initial path; // unanswered → interim grant and interim-derived state stand. let negotiated = false; + // A 'close' fallback closes after this try, so a close that fails is + // reported as itself, not as an unanswered policy. + let closeForGood = false; try { const receipt = await connection.sendFeatureSetsUpdateRequest(updateParams); if (receipt && receipt.accepted === false) { - const fallback = receipt.fallback ?? 'mcp-only'; + const refusal = AgentFramework.policyRefusalFields(receipt, recorded); + const fallback = refusal.fallback; console.error(`[mcpl] ${connection.id} refused post-manifest policy — fallback: ${fallback}`); this.emitTrace({ type: 'mcpl:policy-refused', serverId: connection.id, phase: 'manifest-change', - ...AgentFramework.policyRefusalFields(receipt), + ...refusal, }); if (fallback === 'close') { - await this.mcplServerRegistry!.closeServer(connection.id, connection); - return; + closeForGood = true; + } else { + connection.establishGrant(CapabilityGrant.empty()); + this.featureSetManager!.initializeServer(connection.id, maskedOrEmpty, cfg, CapabilityGrant.empty()); } - connection.establishGrant(CapabilityGrant.empty()); - this.featureSetManager!.initializeServer(connection.id, maskedOrEmpty, cfg, CapabilityGrant.empty()); } else { connection.establishGrant(newGrant); this.featureSetManager!.initializeServer(connection.id, maskedOrEmpty, cfg, newGrant); @@ -21854,6 +22354,10 @@ export class AgentFramework { } catch { console.error(`[mcpl] ${connection.id} did not answer post-manifest policy — expansion does not activate (§6.7); interim state stands`); } + if (closeForGood) { + await this.closeMcplServerForGood(connection); + return; + } // Adopt the validated manifest + tracking. droppedCapabilities is // REPLACED unconditionally — an empty new mask must clear stale paths @@ -21891,7 +22395,7 @@ export class AgentFramework { st.lastCompletedAt = Date.now(); if (st.queued) { st.queued = false; - setTimeout(() => { this.rerunManifestRefresh(connection, announce); }, + setTimeout(() => { this.rerunManifestRefresh(connection, announce, recorded); }, AgentFramework.MANIFEST_FETCH_FLOOR_MS).unref?.(); } } @@ -21905,9 +22409,10 @@ export class AgentFramework { private rerunManifestRefresh( connection: McplServerConnection, announce: unknown, + recorded: (text: unknown) => string, ): void { this.containedInboundHandler(connection.id, 'manifest-changed', - (params: unknown) => this.handleManifestChanged(connection, params))(announce); + (params: unknown) => this.handleManifestChanged(connection, params), recorded)(announce); } /** @@ -21930,6 +22435,10 @@ export class AgentFramework { connection: McplServerConnection, ): Promise { if (!connection.capabilities) return; + // Taken before any await, so every record of what the server says below + // uses one derivation, whatever becomes of the server meanwhile (removed, + // or replaced by a restart). + const recorded = serverTextRedactor(config); // §5.4: compute the effective grant from the (config-masked) // advertisement. Not yet active — expansion activates only after the @@ -21957,35 +22466,39 @@ export class AgentFramework { updateParams.deniedCapabilities = [...grant.deniedPaths]; } + // A 'close' fallback closes after this try, so a close that fails is + // reported as itself, not as an unanswered policy. + let closeForGood = false; try { const receipt = await connection.sendFeatureSetsUpdateRequest(updateParams); if (receipt && receipt.accepted === false) { - const fallback = receipt.fallback ?? 'mcp-only'; + // The fallback the host takes, which is the one logged; the server's + // reason is its own text. + const refusal = AgentFramework.policyRefusalFields(receipt, recorded); + const fallback = refusal.fallback; console.error( - `[mcpl] ${config.id} refused initial policy (${receipt.reason ?? 'no reason'}) — fallback: ${fallback}`, + `[mcpl] ${config.id} refused initial policy (${refusal.reason ?? 'no reason'}) — fallback: ${fallback}`, ); this.emitTrace({ type: 'mcpl:policy-refused', serverId: config.id, phase: 'initial', - ...AgentFramework.policyRefusalFields(receipt), + ...refusal, }); - if (fallback === 'close') { - await this.mcplServerRegistry!.closeServer(connection.id, connection); - return; - } // mcp-only (§3.2): grant stays empty, MCPL privileged surface stays // dark, plain MCP tools keep working. MUST NOT widen in response to // the refusal — a human can change config; the policy engine cannot. - return; - } - // Expansion activates on receipt (§6.7). - connection.establishGrant(grant); - if (receipt?.accepted === true && receipt.mode === 'degraded') { - for (const uf of receipt.unavailableFeatures ?? []) { - console.error( - `[mcpl] ${config.id} degraded: ${uf.featureSet} (${uf.effect}; missing ${uf.missingCapabilities.join(', ')})`, - ); + if (fallback !== 'close') return; + closeForGood = true; + } else { + // Expansion activates on receipt (§6.7). + connection.establishGrant(grant); + if (receipt?.accepted === true && receipt.mode === 'degraded') { + for (const uf of receipt.unavailableFeatures ?? []) { + console.error(recorded( + `[mcpl] ${config.id} degraded: ${uf.featureSet} (${uf.effect}; missing ${Array.isArray(uf.missingCapabilities) ? uf.missingCapabilities.join(', ') : 'unknown'})`, + )); + } } } } catch (err) { @@ -21997,10 +22510,14 @@ export class AgentFramework { console.error( `[mcpl] ${config.id} did not answer the initial policy Request — ` + `MCPL privileged surface stays disabled (§6.7 unanswered expansion): ` + - (err instanceof Error ? err.message : String(err)), + recorded(err instanceof Error ? err.message : err), ); return; } + if (closeForGood) { + await this.closeMcplServerForGood(connection); + return; + } if (config.scopes) { console.error(`[mcpl] ${config.id}: config \`scopes\` is ignored — §7 Scoped Access removed in 0.5.0`); @@ -22129,8 +22646,11 @@ export class AgentFramework { return this.trackOperatorOperation('mcpl:connect', () => this.runConnectMcplServer(config)); } + /** connectMcplServer's work, for a restart's connect too: the id and tool + * prefix its `claim` holds are free to it, and refused to everyone else. */ private async runConnectMcplServer( config: import('./mcpl/types.js').McplServerConfig, + claim?: McpRestartClaim, ): Promise { assertServerConfigBooleans(config); // Lazily bring up the MCPL subsystem — a framework that started with zero @@ -22139,60 +22659,39 @@ export class AgentFramework { await this.initializeMcpl([], this.mcplInferenceRoutingConfig ?? undefined); } - this.refuseMcplConnect(config); + const collision = this.mcplPrefixCollision(config, undefined, claim); + if (collision) throw new Error(collision); // The server's restored host-managed trees are read before the server is // added, so a read that fails leaves it unconnected; MCPL itself stays // up, even when this connect brought it up. A feature set with no // restored tree, as after a disconnect removed it, is read when it - // registers. A server already registered was refused just above, so - // the read never reaches a connected server's trees. - this.checkpointManager?.loadHostStates(config.id); + // registers. A server already connected is refused by addServer below, + // so its trees are left as they are. + if (!this.mcplServerRegistry?.getServer(config.id)) this.checkpointManager?.loadHostStates(config.id); const oldToolNames = new Set(this.mcplTools.map(t => t.name)); - await this.connectMcplServerInternal(config); - await this.refreshMcplTools(); - this.emitMcplToolDiff(oldToolNames, config.id); - } - - /** - * Refuse a runtime connect, or a restart, before it writes anything: an id - * whose connect is in flight or that's already registered, or a tool prefix - * that equals a module name or another server's prefix, or meets another - * server's (prefixesMeet, #265). create() applies the same prefix rules at - * startup. The connecting id's own entries don't count, nor, for a restart, - * those of the server it replaces (`replacing`), which it disconnects first. - */ - private refuseMcplConnect( - config: import('./mcpl/types.js').McplServerConfig, - replacing?: string, - ): void { - // A connect in flight isn't registered until its handshake is done, so - // the registry's check below misses it. Its id is taken whether this - // connects it or a restart replaces it: the restart's disconnect can't - // stop a connect in flight, which would register the server after it. - const inFlight = [config.id, replacing].find((id) => id !== undefined && this.mcplConnecting.has(id)); - if (inFlight !== undefined) throw mcplConnectInFlightError(inFlight); - // The registry refuses a second server under one id too, but only once - // the connect has written its config and prefix over the live server's. - if (config.id !== replacing && this.mcplServerRegistry?.getServer(config.id)) { - throw new Error(`MCPL server "${config.id}" is already registered`); - } - const own = (id: string) => id === config.id || id === replacing; - const prefix = config.toolPrefix ?? `mcpl--${config.id}`; - if (this.moduleRegistry.getAllModules().some(m => m.name === prefix)) { - throw new Error( - `MCPL server "${config.id}" toolPrefix "${prefix}" collides with module "${prefix}"`, - ); - } - const prefixOwner = this.mcplPrefixMap.get(prefix); - if (prefixOwner && !own(prefixOwner)) { - throw new Error( - `MCPL server "${config.id}" toolPrefix "${prefix}" collides with server "${prefixOwner}"`, - ); - } - for (const [other, otherId] of this.mcplPrefixMap) { - if (!own(otherId) && prefixesMeet(prefix, other)) throw prefixesMeetError(config.id, prefix, otherId, other); + const connection = await this.connectMcplServerInternal(config, false, claim); + // A stop() that began during the connect's setup or its tool listing + // ends the connect: it doesn't wait on a server for a framework that is + // stopping, and the teardown closes what the connect set up. + const stopped = () => new Error(`MCP server "${config.id}" was not connected: the framework stopped while it was connecting`); + if (this.stopHasBegun()) throw stopped(); + await this.untilStopBegins(this.refreshMcplTools()); + if (this.stopHasBegun()) throw stopped(); + // The connect let go of its id once its setup was done, so a disconnect + // during the listing wasn't held off. A disconnect under way ends the + // connect, as one during its setup does, and so does a connection that + // no longer serves: closed by a disconnect, its child reaped or not, or + // lost for good. Whatever removed or replaced the connection closed it + // first, since an id has one owner at a time. A connection the server + // closed that is reconnecting still serves. A disconnect takes the id + // before its close marks the connection, a turn later for a legacy one, + // so a disconnect under way is asked of as well. A restart reports the + // connect's failure as its own. + if (this.mcpServerOps?.get(config.id)?.kind === 'disconnect' || !AgentFramework.mcpConnectionServes(connection)) { + throw new Error(`MCP server "${config.id}" was not connected: it was disconnected or lost while its tools were being listed`); } + this.emitMcplToolDiff(oldToolNames, config.id); } /** @@ -22201,9 +22700,21 @@ export class AgentFramework { * transport close, which preserves checkpoints for the reconnect), remove * its channels from the registry, drop routing entries, and refresh tools. * No-op-ish if the server is not connected (still clears routing state). + * + * It holds the id throughout (holdMcpServerId): a connect for it is + * refused meanwhile, and a connect already in flight is ended (closed, or + * displaced when its legacy handshake finishes) and settles before + * anything is removed. A close that can't confirm a child exited fails the + * disconnect before anything is removed: the server stays registered, + * closed, so nothing launches beside a child that may still be running, + * and disconnecting again retries the reap. */ async disconnectMcplServer(id: string): Promise { this.refuseOperatorAfterStop('disconnect MCPL server'); + // A restart moving its server to this id is ended by this disconnect, + // even one that finishes before the restart's connect begins. + const claim = this.restartClaims().ids.get(id); + if (claim) claim.endedBy = `MCP server "${id}" was disconnected, which ends the restart of "${claim.restarting}"`; return this.trackOperatorOperation('mcpl:disconnect', () => this.runDisconnectMcplServer(id)); } @@ -22211,31 +22722,65 @@ export class AgentFramework { if (!this.mcplServerRegistry) { throw new Error('MCPL subsystem is not initialized'); } - // A connect in flight would register the server after this disconnect, - // with its config and prefix gone, where only another disconnect reaches - // it; refused as a second connect of it is (refuseMcplConnect). - if (this.mcplConnecting.has(id)) throw mcplConnectInFlightError(id); - const config = this.mcplServerConfigs.get(id); - const oldToolNames = new Set(this.mcplTools.map(t => t.name)); + const registry = this.mcplServerRegistry; + const hold = this.holdMcpServerId(id, 'disconnect'); + try { + // Close what the id has now, so a connect in flight ends rather than + // being waited out: a modern connect is ended by its close, a legacy + // one still in its handshake (or a WebSocket dial waiting for its + // access provider) by its abort, its launch reaped, and a legacy one + // already registered (setting up after its handshake) fails its + // requests and finds itself displaced. The registry's removal is the + // legacy close: a failed one stays registered. + this.legacyConnectAborts().get(id)?.abort(); + const modern = this.modernServers().get(id); + const closing = [modern?.close(), registry.removeServer(id)]; + for (const pending of closing) pending?.catch(() => { /* its verdict is awaited below */ }); + // What held the id before (a connect still running, an earlier + // disconnect) settles first, so what it registered is removed whole. + await hold.previous; + const oldToolNames = new Set(this.mcplTools.map(t => t.name)); - await this.mcplServerRegistry.removeServer(id); - this.channelRegistry?.removeServer(id); + try { + for (const pending of closing) await pending; + // Whatever was registered since: a connect that was still + // handshaking, whose own removal couldn't reap, is re-checked here. + await registry.removeServer(id); + } catch (error) { + // Closed either way, so its tools leave the agents' list. The server + // keeps its id until a disconnect confirms the exit. + await this.refreshMcplTools(); + this.emitMcplToolDiff(oldToolNames, id); + const message = error instanceof Error ? error.message : String(error); + throw new Error( + `${message}. MCP server "${id}" stays registered, closed, until its exit is confirmed: disconnect it again to retry the reap`, + { cause: error }, + ); + } + if (modern && this.modernServers().get(id) === modern) this.modernServers().delete(id); + this.channelRegistry?.removeServer(id); - // Permanent removal: also destroy feature-set and checkpoint state - // explicitly. The 'close' handler usually does this, but a connection that - // already transiently closed (reconnect pending) emits no second 'close' - // from close(), and the transient path deliberately preserves checkpoints. - this.featureSetManager?.removeServer(id); - this.checkpointManager?.removeServer(id); + // Permanent removal: also destroy feature-set and checkpoint state + // explicitly. The 'close' handler usually does this, but a connection that + // already transiently closed (reconnect pending) emits no second 'close' + // from close(), and the transient path deliberately preserves checkpoints. + this.featureSetManager?.removeServer(id); + this.checkpointManager?.removeServer(id); - // Only `id`'s own entry: an id with no config falls back to the default - // prefix, which another server may have set as its own. - const prefix = config?.toolPrefix ?? `mcpl--${id}`; - if (this.mcplPrefixMap.get(prefix) === id) this.mcplPrefixMap.delete(prefix); - this.mcplServerConfigs.delete(id); + // Only this server's own claims, as the prefix map holds them rather + // than as its configuration reads now: a server connected since under + // a prefix it held keeps it, and a configuration changed after its + // connect leaves no claim behind. + for (const [prefix, owner] of this.mcplPrefixMap) { + if (owner === id) this.mcplPrefixMap.delete(prefix); + } + this.mcplServerConfigs.delete(id); - await this.refreshMcplTools(); - this.emitMcplToolDiff(oldToolNames, id); + await this.refreshMcplTools(); + this.emitMcplToolDiff(oldToolNames, id); + } finally { + hold.release(); + } } /** @@ -22243,8 +22788,28 @@ export class AgentFramework { * same (or an updated) config. This actually respawns a stdio child — * unlike `reconnect: true`, which only retries after transport-level * failures. Throws if the server has no stored config and none is given. - * A config `connectMcplServer` would refuse for its boolean settings is - * refused before the running server is disconnected, so it stays up. + * What its connect would refuse is refused before the running server is + * torn down. Until the connect, a connect or a restart started meanwhile + * is refused the id and the tool prefix the restart moves the server to, + * and a module added meanwhile is refused that tool prefix. The id and + * prefix it moves the server from are free once its disconnect is done, as + * after any disconnect. + * + * A disconnect of the id the restart moves the server to (its own id, for + * a restart under the same id), or a `stop()`, that begins during the + * restart ends it, as a disconnect ends a connect in flight: the restart + * rejects, saying the server is down, and its holds are released, and the + * server isn't left configured, as after any disconnect. A restart whose + * connect fails on its own leaves the server down, configured under the + * new configuration and its id, and holding its tool prefix, as any failed + * connect does: `listMcplServers()` lists it as not connected, and a + * restart or a disconnect under that id tries it again or removes it. + * + * Treat a submitted configuration and its nested values as immutable for + * as long as the server uses them. Apply changed settings with + * `restartMcplServer(id, newConfig)`; use `accessProvider` for a network + * server's rotating credentials (a stdio server's changed `env` goes + * through a restart with the new configuration). */ async restartMcplServer( id: string, @@ -22258,25 +22823,181 @@ export class AgentFramework { id: string, newConfig?: import('./mcpl/types.js').McplServerConfig, ): Promise { + // What the connect would refuse is refused here, before the running + // server is torn down for it: the configuration's binding and settings, + // a new id another server holds, and a tool prefix another server or a + // module already holds. (A stop() already begun refused the restart at + // its entry, as it refuses every operator operation.) From these checks + // through its connect, the restart holds the id and the tool prefix it + // moves the server to (mcpRestartClaims), so no connect, restart or + // module added meanwhile takes them while the old server is torn down. + // A stop(), or a disconnect of the id it moves the server to, that + // begins after these checks isn't held off: it ends the restart, as it + // would end a connect. const config = newConfig ?? this.mcplServerConfigs.get(id); if (!config) { throw new Error(`MCPL server "${id}" is not configured`); } - // Checked before the running server is disconnected, so a config this - // refuses leaves it up, as refuseMcplConnect below does for its prefix. + const family = AgentFramework.usableConfigFamily(config); assertServerConfigBooleans(config); - // Every connect reads the awareness ledger: its awareness barrier + const inFlight = this.mcpServerOps?.get(id); + if (inFlight) { + // A connect or disconnect of the server being restarted, in flight, + // whatever id the restart moves it to: the restart's own disconnect + // would wait behind it, and its connect would then meet whatever that + // operation left, after the server was torn down. + throw new Error(`MCP server "${id}" was not restarted: it is being ${inFlight.kind === 'disconnect' ? 'disconnected' : 'connected'}; restart it once that has finished`); + } + const claimedBy = this.restartClaims().ids.get(id); + if (claimedBy) { + // Another restart is moving its server to this id, or restarting this + // server already. + throw new Error(claimedBy.restarting === id + ? `MCP server "${id}" was not restarted: it is being restarted; restart it once that has finished` + : `MCP server "${id}" was not restarted: the restart of "${claimedBy.restarting}" holds its id; restart it once that has finished`); + } + if (config.id !== id) { + const taken = this.mcpServerIdTaken(config.id, undefined, id); + if (taken) throw new Error(taken); + } + const collision = this.mcplPrefixCollision(config, id); + if (collision) throw new Error(collision); + // A legacy connect reads the awareness ledger: its awareness barrier // reconciles it, and a stdio child taking the protective default derives // that from it. Read it before disconnecting, so an unreadable ledger // refuses the restart and leaves the running child up, whatever the // server's baseline or transport. (No outbox configured: nothing to read.) - this.getPlacedReactionBaseline(); - // What the connect would refuse refuses the restart the same way, before - // anything is disconnected. `id`'s own entries don't count, since the - // disconnect removes them whatever id the new config carries. - this.refuseMcplConnect(config, id); - await this.runDisconnectMcplServer(id); - await this.runConnectMcplServer(config); + // A modern connection reads neither. + if (family === 'legacy') this.getPlacedReactionBaseline(); + + const claim: McpRestartClaim = { restarting: id }; + const prefix = config.toolPrefix ?? `mcpl--${config.id}`; + this.restartClaims().ids.set(config.id, claim); + this.restartClaims().prefixes.set(prefix, claim); + try { + await this.runDisconnectMcplServer(id); + try { + await this.runConnectMcplServer(config, claim); + } catch (error) { + // The old server is gone, whatever ended the connect: a disconnect or + // a stop() that began during the restart, or the new server's own + // failure. The caller learns that first, then the connect's reason. + const reason = error instanceof Error ? error.message : String(error); + throw new Error(`MCP server "${id}" is down: it was disconnected for its restart, and the restart's connect failed: ${reason}`, { cause: error }); + } + } finally { + if (this.restartClaims().ids.get(config.id) === claim) this.restartClaims().ids.delete(config.id); + if (this.restartClaims().prefixes.get(prefix) === claim) this.restartClaims().prefixes.delete(prefix); + } + } + + /** + * A redactor for text that may come from any configured server, with every + * server's credentials replaced: those of the configurations present when + * it's made, then those present when it's used. An operation makes it + * before it waits, so a server removed meanwhile, and one connected + * meanwhile, both have their credentials replaced. + */ + private everyServerRedactor(): (text: unknown) => string { + const snapshot = [...this.mcplServerConfigs.values()]; + // The configurations present when the operation began, then those present + // now: a server removed meanwhile and one connected meanwhile are both + // covered, each with what its dials have sent by the time the text is + // recorded. One pass over all their strings, so a secret that begins + // another server's longer one can't cut that one short. + return (text) => credentialRedactor( + [...new Set([...snapshot, ...this.mcplServerConfigs.values()])].flatMap(config => serverCredentialPatterns(config)), + )(text); + } + + /** Why a connect for this id is refused, or null when the id is free. One + * owner per id, across both engines (holdMcpServerId): a connect or + * disconnect in flight holds it, and so does a registered connection, + * including one kept because its launch couldn't be confirmed exited, and + * a restart moving its server to the id (mcpRestartClaims), except to + * that restart's own connect (`claim`). That connect is still refused + * while a disconnect of the id is in flight: the restart's own disconnect + * has finished by then, so that one began during the restart, and it ends + * the restart. `restarting` words the refusal for a restart of that + * server checking the id it would move to. */ + private mcpServerIdTaken(id: string, claim?: McpRestartClaim, restarting?: string): string | null { + const refused = restarting === undefined ? '' : `MCP server "${restarting}" was not restarted: `; + const retry = restarting === undefined ? 'connect it' : `restart "${restarting}"`; + const claimedBy = this.restartClaims().ids.get(id); + if (claimedBy && claimedBy !== claim) { + return `${refused}MCP server "${id}" is held by the restart of "${claimedBy.restarting}"; ${retry} once that has finished`; + } + const op = this.mcpServerOps?.get(id); + if (op?.kind === 'disconnect') { + if (claim) return `MCP server "${id}" is being disconnected, which ends the restart of "${claim.restarting}"`; + return `${refused}MCP server "${id}" is still being disconnected; ${retry} once that has finished`; + } + // A disconnect of the id began during the restart, and has finished. + if (claim?.endedBy) return claim.endedBy; + const registered = this.modernServers().get(id) ?? this.mcplServerRegistry?.getServer(id); + // A connect in flight that hasn't registered its server yet. + if (op && !registered) return `${refused}${mcplConnectInFlightText(id)}`; + if (op || registered) { + const idle = !op && registered && !registered.isConnected && !registered.willReconnect; + return `${refused}MCPL server "${id}" is already registered${idle ? ' (not connected and not reconnecting: disconnect it first)' : ''}`; + } + return null; + } + + /** The collision rules create() enforces at startup, as a runtime connect + * (or a restart, before its disconnect) meets them, in either engine: the + * tool prefix must not shadow a module name or another server's prefix, + * or meet another server's (prefixesMeet, #265), and must neither be nor + * meet one a restart is moving its server to (mcpRestartClaims). The + * server's own prefix is free to keep, so is that of the server a restart + * is about to disconnect (`leaving`), and so is the prefix a restart's own + * connect holds (`claim`). */ + private mcplPrefixCollision( + config: import('./mcpl/types.js').McplServerConfig, + leaving?: string, + claim?: McpRestartClaim, + ): string | null { + const prefix = config.toolPrefix ?? `mcpl--${config.id}`; + if (this.moduleRegistry.getAllModules().some(m => m.name === prefix)) { + return `MCPL server "${config.id}" toolPrefix "${prefix}" collides with module "${prefix}"`; + } + const claimedBy = this.restartClaims().prefixes.get(prefix); + if (claimedBy && claimedBy !== claim) { + return `MCPL server "${config.id}" toolPrefix "${prefix}" is held by the restart of "${claimedBy.restarting}"`; + } + const prefixOwner = this.mcplPrefixMap.get(prefix); + if (prefixOwner && prefixOwner !== config.id && prefixOwner !== leaving) { + return `MCPL server "${config.id}" toolPrefix "${prefix}" collides with server "${prefixOwner}"`; + } + for (const [other, otherId] of this.mcplPrefixMap) { + if (otherId !== config.id && otherId !== leaving && prefixesMeet(prefix, other)) { + return prefixesMeetError(config.id, prefix, otherId, other).message; + } + } + for (const [other, holder] of this.restartClaims().prefixes) { + if (holder !== claim && other !== prefix && prefixesMeet(prefix, other)) { + return `MCPL server "${config.id}" toolPrefix "${prefix}" meets the toolPrefix "${other}" the restart of ` + + `"${holder.restarting}" holds: two prefixes can give one tool name when one is the other's followed by "-", ` + + 'or begins with the other\'s followed by "--"'; + } + } + return null; + } + + /** The protocol family a configuration resolves to, or a refusal naming + * every problem with it, before anything is spawned, dialed or torn down. */ + private static usableConfigFamily(config: import('./mcpl/types.js').McplServerConfig): 'legacy' | 'modern' { + let family: 'legacy' | 'modern'; + try { + family = resolveServerBinding(config).family; + } catch (error) { + throw new McpConfigurationRefusal(`MCP server "${config.id}" configuration: ${error instanceof Error ? error.message : String(error)}`); + } + const problems = serverConfigProblems(config); + if (problems.length > 0) { + throw new McpConfigurationRefusal(`MCP server "${config.id}" configuration: ${problems.join('; ')}`); + } + return family; } /** @@ -22287,10 +23008,22 @@ export class AgentFramework { id: string; connected: boolean; retrying: boolean; + /** Which engine serves it: `legacy` (MCP 2024-11-05 + MCPL) or `modern` + * (MCP 2026-07-28), as its configuration selects; null when the + * configuration names no usable transport (see `refused`). */ + family: 'legacy' | 'modern' | null; + /** The MCP revision its last handshake established; null until one has. */ + protocolVersion: string | null; + /** How it is reached; null when the configuration names no usable transport. */ + transport: 'stdio' | 'websocket' | 'http' | null; + /** Why connecting refuses this configuration, before anything is spawned + * or dialed; null when it is usable. */ + refused: string | null; toolPrefix: string; toolCount: number; - /** True only after the §5.3 policy receipt activated the grant. */ - policyEstablished: boolean; + /** True only after the §5.3 policy receipt activated the grant. Null for + * a modern server, which has no MCPL policy. */ + policyEstablished: boolean | null; /** Sole normative authorization allowlist for the connection (§5.4). */ effectiveGrant: string[]; /** Advertised paths removed by host enabled/disabled capability config. */ @@ -22324,11 +23057,17 @@ export class AgentFramework { lastNegotiatedAt: number | null; }; command?: string; + /** The url as {@link redactUrl} shows it, since a credential may ride in + * it: no userinfo, each query parameter's value replaced, and any path + * segment or query parameter name that looks like a credential + * replaced. */ url?: string; }> { const result: Array<{ id: string; connected: boolean; retrying: boolean; - toolPrefix: string; toolCount: number; policyEstablished: boolean; + family: 'legacy' | 'modern' | null; protocolVersion: string | null; + transport: 'stdio' | 'websocket' | 'http' | null; refused: string | null; + toolPrefix: string; toolCount: number; policyEstablished: boolean | null; effectiveGrant: string[]; maskedCapabilities: string[]; deniedCapabilities: string[]; allowHostCommands: boolean; toolObserveFilter: import('./mcpl/tool-lifecycle.js').ToolObserveRule[] | null; @@ -22342,8 +23081,18 @@ export class AgentFramework { }> = []; for (const [id, config] of this.mcplServerConfigs) { const prefix = config.toolPrefix ?? `mcpl--${id}`; - const connection = this.mcplServerRegistry?.getServer(id) ?? null; - const connected = connection?.isConnected ?? false; + const modern = this.modernServers().get(id) ?? null; + const connection = modern ? null : this.mcplServerRegistry?.getServer(id) ?? null; + const connected = modern?.isConnected ?? connection?.isConnected ?? false; + // What connecting would decide, before anything is spawned or dialed: a + // configuration it refuses is listed as refused, not as a guessed binding. + let binding: { family: 'legacy' | 'modern'; transport: 'stdio' | 'websocket' | 'http' } | null; + try { + binding = resolveServerBinding(config); + } catch { + binding = null; + } + const problems = serverConfigProblems(config); // Attribute through the dispatch resolver, the one rule that picks a // tool's server. const ownTools = (this.mcplTools ?? []).flatMap((t) => { @@ -22353,10 +23102,14 @@ export class AgentFramework { result.push({ id, connected, - retrying: !connected && (connection?.willReconnect ?? false), + retrying: !connected && (modern?.willReconnect ?? connection?.willReconnect ?? false), + family: binding?.family ?? null, + protocolVersion: modern?.protocolVersion ?? connection?.protocolVersion ?? null, + transport: binding?.transport ?? null, + refused: problems.length > 0 ? problems.join('; ') : null, toolPrefix: prefix, toolCount: ownTools.length, - policyEstablished: connection?.policyEstablished ?? false, + policyEstablished: binding?.family === 'modern' ? null : connection?.policyEstablished ?? false, effectiveGrant: connection?.grant.effectiveList() ?? [], maskedCapabilities: [...(connection?.droppedCapabilities ?? [])].sort(), deniedCapabilities: [...(connection?.grant.deniedPaths ?? [])].sort(), @@ -22370,7 +23123,7 @@ export class AgentFramework { ? { ...connection.manifestState } : { lastValidatedRevision: null, lastFetchedAt: null, lastNegotiatedAt: null }, command: config.command, - url: config.url, + url: config.url === undefined ? undefined : redactUrl(config.url), }); } return result; @@ -22382,11 +23135,15 @@ export class AgentFramework { * rejection at this boundary, including failures after channels/register or * channels/changed has acknowledged the request. It's invoked at once, so * control-plane replies keep their acknowledge-before-reconcile order. + * Each text a failure records, in its trace and its log line, goes through + * `recorded`, the server's redactor where the caller has one, since a + * handler's error can quote what the server sent. */ private containedInboundHandler( serverId: string, event: string, handler: (params: T, responder?: InboundResponder) => void | Promise, + recorded: (text: unknown) => string = asText, ): (params: T, responder?: InboundResponder) => void { return (params: T, responder?: InboundResponder) => { // Whether the handler wrote a reply. A reply counts once its write @@ -22441,7 +23198,7 @@ export class AgentFramework { ...(respondError ? { respondError: (code: number, message: string, data?: unknown) => writeResponse(() => respondError(code, message, data), { - kind: 'error', code, message: typeof message === 'string' ? message : describeCause(message), + kind: 'error', code, message: recorded(typeof message === 'string' ? message : describeCause(message)), }), } : {}), }) as InboundResponder | undefined; @@ -22458,14 +23215,14 @@ export class AgentFramework { ? handlerError.error : writeFailed ? writeFailure : new Error('the handler tried to answer a second time after its reply'); reported = true; - const message = describeCause(primary); + const message = recorded(describeCause(primary)); // What the handler's own writes did, before any internal-error // reply sent here. const handlerResponseAttempted = responseAttempted; // A reply write the handler's own attempt failed, read before the // internal-error reply below, whose own failure is responseError. const responseFailure = writeFailed && handlerError && handlerError.error !== writeFailure - ? describeCause(writeFailure) : undefined; + ? recorded(describeCause(writeFailure)) : undefined; let responseError: string | undefined; if (tracked && !responseAttempted) { // The server gets the standard message, not the handler's own @@ -22482,7 +23239,7 @@ export class AgentFramework { stopping ? 'the host stopped while handling this request' : 'Internal error', ); } catch (sendError) { - responseError = describeCause(sendError); + responseError = recorded(describeCause(sendError)); } } this.emitTrace({ @@ -22550,22 +23307,25 @@ export class AgentFramework { * Wire event listeners on an MCPL server connection. * Push events and inference requests are deferred to Steps 6/7. */ - private wireMcplEvents(connection: McplServerConnection): void { + private wireMcplEvents(connection: McplServerConnection, config: McplServerConfig): void { + const recorded = serverTextRedactor(config); type Responder = InboundResponder; // Every inbound handler, request or notification, is contained - // (containedInboundHandler). + // (containedInboundHandler). What a failure records goes through this + // server's redactor, since a handler's error can quote what the server + // sent. const onRequest = ( event: string, handler: (params: T, responder?: Responder) => void | Promise, ): void => { - connection.on(event, this.containedInboundHandler(connection.id, event, handler)); + connection.on(event, this.containedInboundHandler(connection.id, event, handler, recorded)); }; // Forward subprocess stderr lines as trace events so consumers (conhost, // log sinks, TUI badges) can persist and surface them. connection.on('stderr', (params: { line: string }) => { - this.emitTrace({ type: 'mcpl:server-stderr', serverId: connection.id, line: params.line }); + this.emitTrace({ type: 'mcpl:server-stderr', serverId: connection.id, line: recorded(params.line) }); }); // featureSets/changed is REMOVED in 0.5.0 — the connection answers it @@ -22685,7 +23445,7 @@ export class AgentFramework { const parsed = parseToolObserveParams(params); if (!parsed.ok) { if (responder?.respondError) responder.respondError(-32602, parsed.message, parsed.data); - else console.error(`[mcpl] ${connection.id}: tools/observe rejected — ${parsed.message}`); + else console.error(`[mcpl] ${connection.id}: tools/observe rejected — ${recorded(parsed.message)}`); return; } connection.toolObserveFilter = parsed.rules; @@ -22790,7 +23550,7 @@ export class AgentFramework { } catch (error) { console.error( `MCPL server "${connection.id}" re-registration after reconnect failed:`, - error instanceof Error ? error.message : error, + recorded(error instanceof Error ? error.message : error), ); } this.releaseMcplDataPlaneGate(awarenessBarrier); @@ -22827,36 +23587,41 @@ export class AgentFramework { this.emitTrace({ type: 'mcpl:server-connect-failed', serverId: connection.id, - error: params.error, - ...(params.stderrTail === undefined ? {} : { stderrTail: params.stderrTail }), + error: recorded(params.error), + ...(params.stderrTail === undefined ? {} : { stderrTail: recorded(params.stderrTail) }), attempt: params.attempt, willRetry: connection.willReconnect, }); }); - connection.on('reconnect-failed', (params: { error: string; stderrTail?: string; attempt: number }) => { + connection.on('reconnect-failed', (params: { error: string; stderrTail?: string; attempt: number; permanent?: boolean }) => { this.emitTrace({ type: 'mcpl:server-connect-failed', serverId: connection.id, - error: params.error, - ...(params.stderrTail === undefined ? {} : { stderrTail: params.stderrTail }), + error: recorded(params.error), + ...(params.stderrTail === undefined ? {} : { stderrTail: recorded(params.stderrTail) }), attempt: params.attempt, willRetry: connection.willReconnect, }); - // The reconnect loop never gives up (backoff caps at ~300s), so - // "the server is effectively down" is an attempt-count judgment: - // 5 failed attempts ≈ a few minutes of outage. Throttled per - // (serverId, kind), so a long outage re-posts every ~15 min. - if (params.attempt >= 5) { + // The reconnect loop gives up only on a protocol-version verdict or a + // launch that couldn't be reaped (`permanent`), which no later attempt + // changes — alert at once, since no fifth attempt will come. Otherwise + // backoff caps at ~300s, so "the server is effectively down" is an + // attempt-count judgment: 5 failed attempts ≈ a few minutes of outage. + // Throttled per (serverId, kind), so a long outage re-posts every + // ~15 min. + if (params.permanent || params.attempt >= 5) { this.opsAlert( 'mcpl-down', connection.id, - // Not the server's stderr (params.stderrTail): the webhook posts - // outside the process, and a server's stderr can hold a credential - // its configuration gives it, such as a token, which nothing here - // redacts. - `MCPL server unreachable (attempt ${params.attempt}): ${params.error}`, + // The failure, redacted, and not the server's stderr + // (params.stderrTail): the webhook posts outside the process, and a + // server's stderr is its own text. + `MCPL server unreachable (attempt ${params.attempt}): ${recorded(params.error)}`, ); } + // Halted for good: its tools leave the agents' list, since it won't + // serve again on its own. + if (params.permanent) this.handleToolsListChanged(connection.id); }); // A late answer to a request the connection keeps track of: a push/render @@ -22919,7 +23684,7 @@ export class AgentFramework { this.emitTrace({ type: 'mcpl:server-error', serverId: connection.id, - error: err.message, + error: recorded(err.message), }); }); @@ -22936,7 +23701,10 @@ export class AgentFramework { this.featureSetManager?.removeServer(connection.id); // RFC-008: a closed provider's class hints are stale — on reconnect it // may class its tools differently. Until the next tools/list its tools - // are unclassed, the restrictive answer (RFC-007 §4.3). + // are unclassed, the restrictive answer (RFC-007 §4.3). A listing + // already under way is of the closed session, so it doesn't count as + // that next one (see refreshMcplTools). + this.noteMcplServerClosed(connection.id); this.dropMcplToolClasses(connection.id); this.emitTrace({ type: 'mcpl:server-closed', @@ -22949,9 +23717,103 @@ export class AgentFramework { willReconnect: connection.willReconnect, }); this.emitTrace({ type: 'module:removed', moduleName: `mcpl:${connection.id}` }); + // Lost for good (no reconnect, and the registry has dropped it): its + // tools leave the agents' list, as a disconnect's do. + if (!connection.willReconnect && this.mcplServerRegistry?.getServer(connection.id) !== connection) { + this.handleToolsListChanged(connection.id); + } }); } + /** + * Wire a modern-MCP connection's events. There's less here than for MCPL + * on purpose: no push, inference, channels or policy exist on this family. + * A list change, including the relist after a reconnect, takes the legacy + * engine's refresh path (`handleToolsListChanged`), so any wake it causes + * goes through ordinary admission: it parks under quiesce or a hold like + * every other wake. It installs no awareness gate, since that gate orders + * MCPL data-plane traffic, which this family doesn't carry. + */ + private wireModernEvents(connection: ModernMcpConnection, config: McplServerConfig): void { + const recorded = serverTextRedactor(config); + connection.on('stderr', (params: { line: string }) => { + this.emitTrace({ type: 'mcpl:server-stderr', serverId: connection.id, line: recorded(params.line) }); + }); + connection.on('tools-list-changed', () => { + this.handleToolsListChanged(connection.id); + }); + connection.on('reconnect', (info?: { attempts?: number }) => { + this.emitTrace({ type: 'mcpl:server-reconnected', serverId: connection.id, attempts: info?.attempts ?? 0 }); + this.emitTrace({ type: 'module:added', moduleName: `mcpl:${connection.id}` }); + this.handleToolsListChanged(connection.id); + }); + const failed = (params: { error: string; attempt: number; permanent?: boolean }) => { + this.emitTrace({ + type: 'mcpl:server-connect-failed', + serverId: connection.id, + error: recorded(params.error), + attempt: params.attempt, + willRetry: connection.willReconnect, + }); + // `permanent`: no later attempt will come, after a launch that + // couldn't be reaped or a protocol-version verdict. Alert now, and the + // server's tools leave the agents' list: it won't serve again on its + // own. + if (params.permanent || params.attempt >= 5) { + this.opsAlert('mcpl-down', connection.id, `MCP server unreachable (attempt ${params.attempt}): ${recorded(params.error)}`); + } + if (params.permanent) this.handleToolsListChanged(connection.id); + }; + connection.on('connect-failed', failed); + connection.on('reconnect-failed', failed); + connection.on('error', (err: Error) => { + this.emitTrace({ type: 'mcpl:server-error', serverId: connection.id, error: recorded(err.message) }); + }); + connection.on('close', () => { + // A closed provider's class hints are stale (RFC-008), as for MCPL, + // and so is a listing already under way. + this.noteMcplServerClosed(connection.id); + this.dropMcplToolClasses(connection.id); + this.emitTrace({ + type: 'mcpl:server-closed', + serverId: connection.id, + code: null, + signal: null, + willReconnect: connection.willReconnect, + }); + this.emitTrace({ type: 'module:removed', moduleName: `mcpl:${connection.id}` }); + // Without reconnect a server lost on its own is gone, as the MCPL + // registry drops one, once its child's reap is confirmed: until then it + // holds the id, so nothing launches beside a child that may still run, + // and stop() and a disconnect still reach it. A reap that fails keeps + // it registered, holding the id, as a failed disconnect does, for a + // disconnect to retry; that failure is a permanent one, which takes + // its tools out of the list. Its configuration stays listed as + // disconnected. One the host is closing stays until that teardown is + // done: whoever closes it removes it. + if (!connection.isClosed && !connection.willReconnect) { + void connection.lossReaped().then(() => { + if (connection.isClosed || connection.holdsUnreapedChild || this.modernServers().get(connection.id) !== connection) return; + this.modernServers().delete(connection.id); + // Gone for good: its tools leave the agents' list, as a + // disconnect's do. + this.handleToolsListChanged(connection.id); + }); + } + }); + } + + /** An MCP tool call's trace: a result the server marked an error (or one + * the reading couldn't complete) is tool:failed, as on the other tool + * paths; anything else is tool:completed. */ + private traceMcpToolOutcome(serverId: string, tool: string, callId: string, result: ToolResult, durationMs: number): void { + if (result.isError) { + this.emitTrace({ type: 'tool:failed', module: `mcpl:${serverId}`, tool, callId, durationMs, error: result.error ?? 'tool returned an error' }); + } else { + this.emitTrace({ type: 'tool:completed', module: `mcpl:${serverId}`, tool, callId, durationMs }); + } + } + /** * Discover tools from all connected MCPL servers and cache them. * Tools are namespaced as `{toolPrefix}--{toolName}` per server config. @@ -22964,17 +23826,22 @@ export class AgentFramework { */ private async refreshMcplTools(): Promise { if (!this.mcplServerRegistry) return; + const generation = ++this.mcplToolsRefreshStarted; const tools: import('./types/index.js').ToolDefinition[] = []; const toolFeatureSets = new Map(); const toolClasses = new Map(); const toolServers = new Map(); const listedServers = new Set(); + const closesOf = (serverId: string): number => this.mcplServerCloses?.get(serverId) ?? 0; + // Each server's close count as its listing began. + const closesAtListing = new Map(); const repeats: Array<{ serverId: string; tools: string[] }> = []; - for (const server of this.mcplServerRegistry.getAllServers()) { + for (const server of [...this.mcplServerRegistry.getAllServers()]) { const config = this.mcplServerConfigs.get(server.id); const prefix = config?.toolPrefix ?? `mcpl--${server.id}`; + closesAtListing.set(server.id, closesOf(server.id)); try { const result = await server.sendToolsList(); listedServers.add(server.id); @@ -23024,7 +23891,84 @@ export class AgentFramework { } } - this.mcplTools = tools; + // Modern servers: the same namespacing, tool policy and RFC-008 class + // hints. Feature sets are MCPL and don't apply. A server whose listing + // fails stays unlisted (unknown), never "listed nothing". + // The servers there now: one connected while this refresh runs is + // listed by its own connect's refresh. + for (const server of [...this.modernServers().values()]) { + if (!server.isConnected) continue; + const config = this.mcplServerConfigs.get(server.id); + const prefix = config?.toolPrefix ?? `mcpl--${server.id}`; + closesAtListing.set(server.id, closesOf(server.id)); + try { + const listed = await server.listTools(); + listedServers.add(server.id); + for (const tool of listed) { + if (!isToolAllowed(tool.name, config)) continue; + const namespacedName = `${prefix}--${tool.name}`; + toolServers.set(namespacedName, server.id); + const declaredClasses = parseDeclaredClasses(tool._meta, namespacedName); + if (declaredClasses) toolClasses.set(namespacedName, declaredClasses); + tools.push({ + name: namespacedName, + description: tool.description ?? '', + inputSchema: tool.inputSchema as import('./types/index.js').ToolDefinition['inputSchema'], + }); + } + } catch (error) { + // A listing the teardown closed under a stop() isn't the server's + // error (untilStopBegins lets such a listing go on after stop()). + if (!this.stopHasBegun()) { + this.emitTrace({ + type: 'mcpl:server-error', + serverId: server.id, + error: `tools/list failed: ${redactServerText(error instanceof Error ? error.message : String(error), config ?? {})}`, + }); + } + } + } + + // A listing counts only while the session it came from is still its + // server's live one, asked of the servers there now. A server removed + // while this refresh awaited its lists (a disconnect, or a server lost + // for good) contributes nothing, so whichever refresh finishes last + // can't bring back its tools. Nor does one whose connection no longer + // serves: the host is closing it or failed to reap it, or it was lost + // for good. Nor does one whose connection closed after its listing + // began, even if it has reconnected since: the close forgot its class + // hints as stale (RFC-008), and its reconnect lists the new session. + const connections = new Map([ + ...(this.mcplServerRegistry?.getAllServers() ?? []).map((server) => [server.id, server] as const), + ...this.modernServers(), + ]); + const serving = (serverId: string): boolean => { + const connection = connections.get(serverId); + return !!connection && AgentFramework.mcpConnectionServes(connection); + }; + const over = new Set([...listedServers].filter((serverId) => !serving(serverId) || closesOf(serverId) !== closesAtListing.get(serverId))); + if (over.size > 0) { + for (const [name, serverId] of [...toolServers]) { + if (!over.has(serverId)) continue; + toolServers.delete(name); + toolFeatureSets.delete(name); + toolClasses.delete(name); + } + for (const serverId of over) listedServers.delete(serverId); + } + // A refresh started after this one has published already: it began + // listing after this one did, a server replaced by a restart under the + // same id included, so this one's listings are older. A server this one + // listed can still have no listing there, such as one that refused the + // later refresh's request. Its tools would stay missing until + // something else refreshed, so one more refresh lists it again. + if (generation < this.mcplToolsRefreshPublished) { + const unlisted = [...listedServers].find((id) => !this.mcplListedServers.has(id)); + if (unlisted !== undefined) this.handleToolsListChanged(unlisted); + return; + } + this.mcplToolsRefreshPublished = generation; + this.mcplTools = over.size > 0 ? tools.filter((tool) => toolServers.has(tool.name)) : tools; this.mcplToolFeatureSets = toolFeatureSets; this.mcplToolClasses = toolClasses; this.mcplToolServers = toolServers; @@ -23139,13 +24083,14 @@ export class AgentFramework { this.mcplToolRefreshInFlight = true; const oldToolNames = new Set(this.mcplTools.map(t => t.name)); + const recorded = this.everyServerRedactor(); this.refreshMcplTools() .then(() => { this.emitMcplToolDiff(oldToolNames, serverId); }) .catch((error) => { - console.error('MCPL tool refresh error:', error); + console.error('MCPL tool refresh error:', recorded(error instanceof Error ? (error.stack ?? error.message) : error)); }) .finally(() => { this.mcplToolRefreshInFlight = false; @@ -23181,10 +24126,14 @@ export class AgentFramework { } } - /** The refusal of an MCPL tool call whose server isn't in the registry, for - * the model's call and a script's alike. */ + /** The refusal of a tool call whose server isn't registered, for the + * model's call and a script's alike: "connecting" while a connect of its + * id is in flight (mcpServerOps) or a restart holds the id, and "gone" + * otherwise. A legacy connect registers its server once its handshake is + * done; a modern connect registers its connection first, so a call during + * it reads that connection's own refusal. */ private mcplServerAbsentText(serverId: string, toolName: string): string { - return this.mcplConnecting.has(serverId) + return this.mcpServerOps?.get(serverId)?.kind === 'connect' || this.restartClaims().ids.has(serverId) ? mcplServerConnectingText(serverId, toolName) : mcplServerGoneText(serverId, toolName); } @@ -23195,9 +24144,10 @@ export class AgentFramework { */ private dispatchMcplToolCall(agentName: string, call: ToolCall, serverId: string, prefix: string): void { const toolName = call.name.slice(prefix.length + 2); // strip "{prefix}--" - const server = this.mcplServerRegistry!.getServer(serverId); + const modern = this.modernServers().get(serverId) ?? null; + const server = modern ? null : this.mcplServerRegistry!.getServer(serverId); - if (!server) { + if (!modern && !server) { // RFC-007: refused before executing — no lifecycle events. this.toolLifecycleEmitter?.refuse(agentName, call.id); this.pushEvent({ @@ -23233,6 +24183,65 @@ export class AgentFramework { // when the send's result, or the open that follows, comes after the turn. const sendingTurn = SEND_ENGAGEMENT_TOOLS.has(toolName) ? this.turnRouting.get(agentName) : undefined; + const dispatchRedactor = serverTextRedactor(config ?? {}); + const failDispatch = (error: unknown) => { + const raw = error instanceof Error ? error : new Error(String(error)); + // Server and transport text, as every record of it is, with the + // configuration the call was dispatched under. + const recorded = dispatchRedactor; + const err = { message: recorded(raw.message), stack: raw.stack === undefined ? undefined : recorded(raw.stack) }; + this.emitTrace({ type: 'tool:failed', module: `mcpl:${serverId}`, tool: toolName, callId: call.id, error: err.message, stack: err.stack }); + // RFC-007: transport error, closed connection or timeout — no result. + this.toolLifecycleEmitter?.markDispatchFailure(agentName, call.id); + this.pushEvent({ + type: 'tool-result', + callId: call.id, + toolName: call.name, + agentName, + moduleName: `mcpl:${serverId}`, + result: { success: false, error: err.message, isError: true }, + }); + }; + + // A script's call that returns structured content gets the server's + // result itself (the direct reading): the script receives both views + // whole, so nothing the model reading lowers (stubs, image placeholders, + // generated JSON) stands in for what the server sent. An MCPL peer's + // content blocks are the exception: what a script prints comes back to + // the model, so each gets the treatment RFC-005 gives it on every lane + // the model reads. A standard peer's blocks reach the script as sent. + const forScript = this.scriptToolWaiters?.has(call.id) ?? false; + const reading = async (result: { content?: unknown; isError?: boolean; structuredContent?: unknown }, mcpl: boolean): Promise => { + if (!forScript || !Object.prototype.hasOwnProperty.call(result, 'structuredContent')) { + return this.mcpToolResult(result, call.id, mcpl, serverId, call.name); + } + const direct = AgentFramework.directToolResult(result, serverId, call.name); + if (mcpl) return Array.isArray(direct.data) ? { ...direct, data: direct.data.map((block: unknown) => referenceBlockForScript(block, serverId)) } : direct; + // A standard peer's content that isn't a list of blocks is a malformed + // result on the model's reading (normalizeStandardToolResult), so the + // script reads it that way too, its structured content kept. + if (!result.isError && result.content !== undefined && !Array.isArray(result.content)) { + return { success: false, isError: true, error: MALFORMED_CONTENT_NOTICE, data: [], structured: result.structuredContent }; + } + return direct; + }; + + if (modern) { + // Modern MCP: no state exchange, no channels, no RFC-005 references. + modern.callTool(toolName, args) + .then((result) => reading(result, false)) + .then((result) => { + this.traceMcpToolOutcome(serverId, toolName, call.id, result, Date.now() - startTime); + this.pushEvent({ type: 'tool-result', callId: call.id, toolName: call.name, agentName, moduleName: `mcpl:${serverId}`, result }); + }) + .catch(failDispatch); + return; + } + const legacy = server!; + // An MCPL peer's results get the RFC-005 reading; an MCP-only peer's + // the standard one, the same as a modern server's. + const mcplPeer = legacy.capabilities !== null; + // Build state params for stateful tools (Step 8). // A tools/list attribution selects that exact stateful set. Untagged tools // retain the host-managed or unique server-managed fallback, but never guess @@ -23280,38 +24289,25 @@ export class AgentFramework { } } - server.sendToolsCall(toolName, args, stateParams) + legacy.sendToolsCall(toolName, args, stateParams) .then(async (result) => { - // RFC-005: register references and eagerly fetch the small ones - // before serialization, so their stubs carry a saved path. - await this.autoFetchReferences(result.content, serverId, true) - .catch((e) => console.error('[rfc005] autofetch error:', (e as Error).message)); - return result; + // RFC-005: register references and fetch the small ones, and every + // one a script will read, before serialization, so their stubs carry + // a saved path. Only an MCPL peer's blocks are references; an + // MCP-only peer's are standard. A script reads the blocks unless the + // result is an error without structured content, which reaches it as + // its error text alone. + if (mcplPeer) { + const scriptReads = forScript + && (!result.isError || Object.prototype.hasOwnProperty.call(result, 'structuredContent')); + await this.autoFetchReferences(result.content, serverId, scriptReads ? 'script' : 'model') + .catch((e) => console.error('[rfc005] autofetch error:', (e as Error).message)); + } + return { result, toolResult: await reading(result, mcplPeer) }; }) - .then((result) => { + .then(({ result, toolResult }) => { const durationMs = Date.now() - startTime; - // Convert MCP tool result to framework ToolResult. A server's error - // is a failure (mcplToolFailure, which an in-process caller's error - // takes too). When a success contains non-text blocks (e.g. images - // from an MCP tool like zulip-mcp's fetch_attachment), pass the full - // content array through so toMembraneToolResult can preserve image - // blocks natively. Text-only results still collapse to a joined - // string for backward compatibility with callers that expect data to - // be string-ish. - const converted: ToolResult = result.isError - ? mcplToolFailure(result.content, serverId, call.name) - : { - success: true, - data: result.content?.some((c) => c.type !== 'text') - ? result.content - : (mcplResultText(result.content) || undefined), - isError: false, - }; - this.emitTrace({ - type: converted.isError ? 'tool:failed' : 'tool:completed', - module: `mcpl:${serverId}`, tool: toolName, callId: call.id, durationMs, - ...(converted.isError ? { error: converted.error } : {}), - }); + this.traceMcpToolOutcome(serverId, toolName, call.id, toolResult, durationMs); // Record checkpoint from stateful tool response (Step 8). // Explicit response attribution is authoritative. Otherwise use the @@ -23380,24 +24376,103 @@ export class AgentFramework { toolName: call.name, agentName, moduleName: `mcpl:${serverId}`, - result: converted, + result: toolResult, }); }) - .catch((error) => { - const err = error instanceof Error ? error : new Error(String(error)); - this.emitTrace({ type: 'tool:failed', module: `mcpl:${serverId}`, tool: toolName, callId: call.id, error: err.message, stack: err.stack }); - // RFC-007: transport error, closed connection or timeout — no result. - this.toolLifecycleEmitter?.markDispatchFailure(agentName, call.id); + .catch(failDispatch); + } - this.pushEvent({ - type: 'tool-result', - callId: call.id, - toolName: call.name, - agentName, - moduleName: `mcpl:${serverId}`, - result: { success: false, error: err.message, isError: true }, - }); - }); + /** + * The direct path's reading of an MCP tool result (executeToolCall, + * ModuleContext.callTool). `data` stays the raw content array, as module + * callers have always received it. Two things change: a tool error + * (`isError: true`) is now a failure, where every answer used to count as + * success, and `structured` is added by presence. + */ + private static directToolResult( + result: { content?: unknown; isError?: boolean; structuredContent?: unknown }, + serverId: string, + callName: string, + ): ToolResult { + const structured = Object.prototype.hasOwnProperty.call(result, 'structuredContent') + ? { structured: result.structuredContent } + : {}; + if (result.isError) { + // Its text as the model's dispatch reads it (mcplToolFailure). The raw + // content rides along for a script's structured result, the server's + // own account of the error; an in-process caller gets the text alone + // (executeMcplToolCall). + const content = Array.isArray(result.content) ? result.content as McpToolResultContent[] : undefined; + const { error } = mcplToolFailure(content, serverId, callName); + return { success: false, data: result.content, error, isError: true, ...structured }; + } + return { success: true, data: result.content, ...structured }; + } + + /** + * The framework's ToolResult for an MCP tool result on the model path + * (model dispatch, and scripts, which ride it). A standard peer, meaning an MCP-only legacy + * server or any modern server, gets the standard reading + * (tool-result-normalize.ts): typed content blocks, workspace-saved + * payloads, and `structured` kept by presence. An MCPL peer keeps the + * RFC-005 reading. Its content array passes through when it holds + * non-text blocks, so references and images render natively. It gets + * `structured` too, with the JSON view when its content is empty. The + * direct path (a module's callTool, a channel's tool call) doesn't come + * here: it returns the server's result itself (directToolResult), for + * either engine, as does a script's call whose result is structured, with + * an MCPL peer's blocks given RFC-005's treatment (referenceBlockForScript). + */ + private async mcpToolResult( + result: { content?: unknown; isError?: boolean; structuredContent?: unknown }, + callId: string, + mcplPeer: boolean, + serverId: string, + callName: string, + ): Promise { + const hasStructured = Object.prototype.hasOwnProperty.call(result, 'structuredContent'); + if (!mcplPeer) { + // The date and call id make a saved payload's name readable; the UUID + // makes it unique. Sanitizing and truncating map distinct ids to one + // label (`call:a` and `call/a`), and a later write would replace the + // earlier result's file. + const label = `${new Date().toISOString().slice(0, 10)}-${callId}`.replace(/[^A-Za-z0-9._-]/g, '_').slice(0, 80) + `-${randomUUID()}`; + const workspace = this.getWorkspaceModule(); + const mount = workspace ? this.firstWritableMountName(workspace) : null; + // A save that fails resolves null, never rejects: the payload is then a + // lost part of a completed call's result, not a dispatch failure. Why it + // failed goes to the host's log; the stub says only that it did. + const save = workspace && mount + ? async (fileName: string, bytes: Buffer, mimeType: string) => { + const path = `${mount}/tool-results/${fileName}`; + try { + const written = await workspace.writeBinary(path, bytes, mimeType); + if (written.success) return path; + console.error(`[mcp] saving tool payload ${path} (${JSON.stringify(mimeType)}) failed: ${written.error ?? 'the workspace refused the write'}`); + } catch (error) { + console.error(`[mcp] saving tool payload ${path} (${JSON.stringify(mimeType)}) failed:`, error); + } + return null; + } + : null; + return normalizeStandardToolResult(result, label, save, mcplToolNoWordsText(serverId, callName)); + } + const blocks = (Array.isArray(result.content) ? result.content : []) as McpToolResultContent[]; + // When the result contains non-text blocks (RFC-005 references, images), + // pass the full content array through so toMembraneToolResult can + // preserve them natively. Text-only results collapse to a joined string + // for callers that expect data to be string-ish (mcplResultText). + let textContent = mcplResultText(blocks); + if (blocks.length === 0 && hasStructured) textContent = JSON.stringify(result.structuredContent) ?? 'null'; + const hasNonText = blocks.some((c) => c.type !== 'text'); + const structured = hasStructured ? { structured: result.structuredContent } : {}; + return { + success: !result.isError, + data: result.isError ? undefined : hasNonText ? blocks : (textContent || undefined), + error: result.isError ? (textContent.trim() ? textContent : mcplToolNoWordsText(serverId, callName)) : undefined, + isError: result.isError ?? false, + ...structured, + }; } /** diff --git a/src/index.ts b/src/index.ts index ef57e6eff..fd34068f3 100644 --- a/src/index.ts +++ b/src/index.ts @@ -97,10 +97,23 @@ export type { // connectMcplServer / disconnectMcplServer / restartMcplServer) export type { McplServerConfig } from './mcpl/index.js'; -// MCPL client connection (exposed so external tooling — server playtests, -// health probes, protocol harnesses — can dial an MCPL server exactly the -// way the framework host does, over stdio or WebSocket) -export { McplServerConnection, McplRequestError } from './mcpl/index.js'; +// Both engines' client connections, and the configuration helpers and +// protocol revisions they share (exposed so external tooling, such as server +// playtests, health probes and protocol harnesses, can reach a server +// exactly the way the framework host does: MCPL over stdio or WebSocket, and +// modern MCP over stdio or Streamable HTTP) +export { McplServerConnection, McplRequestError, McplProtocolVersionError, McplUnreapedLaunchError } from './mcpl/index.js'; +export { ModernMcpConnection } from './mcpl/index.js'; +export type { ModernToolCallResult } from './mcpl/index.js'; +export { + LEGACY_MCP_PROTOCOL_VERSION, + MODERN_MCP_PROTOCOL_VERSION, + resolveServerBinding, + serverConfigProblems, + serverConfigWarnings, + checkServerConfig, +} from './mcpl/index.js'; +export type { McpProtocolFamily, McpTransportKind, ServerBinding } from './mcpl/index.js'; export type { McplHostCapabilities, McplConnectionCloseInfo } from './mcpl/index.js'; // MCPL tool lifecycle (RFC-007) and tool classes (RFC-008): the types a host diff --git a/src/mcpl/capability-grant.ts b/src/mcpl/capability-grant.ts index 728757d1a..ae5f9c292 100644 --- a/src/mcpl/capability-grant.ts +++ b/src/mcpl/capability-grant.ts @@ -223,7 +223,7 @@ export function capabilityPatternMatches(pattern: string, path: string): boolean /** §13.4: denied unless explicitly enabled by host config. RFC-007 adds both * tool-lifecycle leaves: observing other servers' tool use (and, above all, * their arguments) is an operator decision, never a default. */ -const DENY_BY_DEFAULT: readonly string[] = [ +export const DENY_BY_DEFAULT: readonly string[] = [ 'contextHooks.beforeInference.inject.system', 'toolLifecycle.observe', 'toolLifecycle.inputs', diff --git a/src/mcpl/index.ts b/src/mcpl/index.ts index 6d8862e9d..f04cab42a 100644 --- a/src/mcpl/index.ts +++ b/src/mcpl/index.ts @@ -106,9 +106,22 @@ export { } from './errors.js'; // Server connection and registry -export { McplServerConnection, McplRequestError } from './server-connection.js'; +export { McplServerConnection, McplRequestError, McplProtocolVersionError, McplUnreapedLaunchError } from './server-connection.js'; export type { McplConnectionCloseInfo } from './server-connection.js'; export type { TransportCloseInfo } from './transport.js'; +export { ModernMcpConnection } from './modern-connection.js'; +export type { ModernToolCallResult } from './modern-connection.js'; +export { + LEGACY_MCP_PROTOCOL_VERSION, + MODERN_MCP_PROTOCOL_VERSION, + MAX_TIMER_MS, + mcplPolicyGrants, + resolveServerBinding, + serverConfigProblems, + serverConfigWarnings, + checkServerConfig, +} from './protocol-family.js'; +export type { McpProtocolFamily, McpTransportKind, ServerBinding } from './protocol-family.js'; export { McplServerRegistry, type McplCapabilityQuery } from './server-registry.js'; // Feature set management (permission layer) diff --git a/src/mcpl/modern-connection.ts b/src/mcpl/modern-connection.ts new file mode 100644 index 000000000..348327fe6 --- /dev/null +++ b/src/mcpl/modern-connection.ts @@ -0,0 +1,1079 @@ +/** + * ModernMcpConnection: a server that speaks modern MCP (2026-07-28), reached + * over stdio or Streamable HTTP through the official SDK client. + * + * Responsibilities divide like this. The SDK owns the modern wire: + * `server/discover`, the per-request `_meta` envelope, `resultType`, + * continuation of `input_required` results, cancellation, list-change + * subscriptions, and the HTTP binding's headers, auth retry and + * header-mismatch handling. This class owns what Connectome decides: + * - **stdio** runs through Connectome's own spawner, so the child gets the + * same env allowlist, `inheritEnv` and stderr surfacing as a legacy server, + * and it starts once per connect. The SDK's own stdio transport starts a + * throwaway second copy for discovery. + * - **HTTP credentials** come from `token`/`accessProvider`: a cached bearer + * token, re-resolved once when the server answers 401. + * - **One deadline per tool call** (`requestTimeoutMs`, always positive) + * bounds every leg of the call and the pauses between them. At the + * deadline the SDK requests cancellation of a leg in flight; between + * continuation rounds nothing is in flight. Either way the outcome is + * unknown. Calls are never replayed. The protocol has no cancel for a + * call paused at `input_required`, so whatever the server holds for it, + * beyond the `requestState` it gave this client, stays until the server + * lets it go. + * - **Request outcomes from the wire.** A failed call is classified by what + * crossed Connectome's transport boundary, never by the error's class. The + * SDK raises the same error classes locally (an invalid outputSchema before + * dispatch, output validation after a successful result) as it does for + * server answers. A call's answer is the server's final one: an + * `input_required` answer asks for another round, so a call that ends + * after one has no answer. + * - **Lifetime.** On a lost transport the connection restarts with the same + * backoff settings as legacy, when `reconnect` is set, once the lost + * launch is reaped, so a second child never runs beside the first. A + * stdio child that stays alive but silently stops reading its stdin is + * no loss the pipes show: each call's deadline is the only bound on it. + * A server that shares no revision with this client is a configuration + * verdict, as for legacy (`McplProtocolVersionError`): it is never + * retried. List-change + * awareness is kept while the connection lives: a subscription the server + * advertises but that didn't open, or that ended, is reopened. Each connect + * is a generation, so a superseded client never touches a newer one. + * `close()` owns a connect in flight too. + * + * Nothing here is MCPL: a modern server has no grant, no planes and no + * server→host requests. The framework uses this connection only through its + * tool paths, and the MCPL machinery never sees it. + */ +import { AsyncLocalStorage } from 'node:async_hooks'; +import { EventEmitter } from 'node:events'; + +import { + Client, + InsufficientScopeError, + SdkError, + SdkErrorCode, + SdkHttpError, + StreamableHTTPClientTransport, + UnsupportedProtocolVersionError, + parseJSONRPCMessage, + type AuthProvider, + type CallToolResult, + type JSONRPCMessage, + type McpSubscription, + type MessageExtraInfo, + type Transport, + type TransportSendOptions, +} from '@modelcontextprotocol/client'; + +import { StdioTransport, type McplTransport } from './transport.js'; +import { McplProtocolVersionError, McplRequestError, McplUnreapedLaunchError } from './server-connection.js'; +import { LEGACY_MCP_PROTOCOL_VERSION, MAX_TIMER_MS, MODERN_MCP_PROTOCOL_VERSION, checkServerConfig, describeCredentialError, noteDialSent, redactServerText, serverTextRedactor } from './protocol-family.js'; +import type { McplServerConfig, McpToolDefinition } from './types.js'; + +const DEFAULT_REQUEST_TIMEOUT_MS = 60_000; +/** Bound on establishing the connection (`server/discover`), like the legacy + * engine's initialize timeout. */ +const CONNECT_TIMEOUT_MS = 30_000; +/** Inventory listing has its own bound: it is host work, not a resident's call. */ +const LIST_TIMEOUT_MS = 30_000; +/** Startup stderr kept for the first listener. */ +const STDERR_BACKLOG_LINES = 200; + +/** A modern tool result as the framework consumes it. `structuredContent` is + * kept by presence: `false`, `0` and `null` are values, not absence. */ +export interface ModernToolCallResult { + content: CallToolResult['content']; + isError?: boolean; + structuredContent?: unknown; + _meta?: Record; +} + +/** + * What crossed Connectome's transport boundary for one logical request: were + * any of its requests handed to the transport, and how did the server answer + * the most recent one? A logical call can take several legs: a + * header-mismatch retry, or the rounds of an `input_required` continuation. + * Only an error or a `complete` result is the server's final answer. + * `input-required` asks for another round, and a result of any other type, + * or of none (`unrecognized`), doesn't say the call finished. + */ +interface WireRecord { + readonly method: string; + handedOff: boolean; + currentId?: string | number; + /** The current leg's answer. */ + answer: 'none' | 'error' | 'complete' | 'input-required' | 'unrecognized'; + error?: { code: number; message: string; data?: unknown }; + /** An earlier leg's answer asked for another round (or wasn't a complete + * result): the server may have acted on the call, whatever a later leg + * meets. A new leg's send never clears it. */ + continued: boolean; +} + +/** + * A send that Connectome's own transport refused before writing anything (its + * stdio line is closed): nothing of that request reached the server. + */ +class NotWrittenError extends Error { + constructor(message: string) { + super(message); + this.name = 'NotWrittenError'; + } +} + +/** The logical request a send belongs to. The SDK hands each request to the + * transport inside the caller's async chain, so the scope reaches `send`. */ +const wireScope = new AsyncLocalStorage(); + +/** + * The boundary between the SDK and the actual transport. It records, for the + * logical request in scope, each request of the observed method that it + * hands on, and the answer that comes back for it. Everything else passes + * through untouched: the SDK sees the inner transport's own members. + */ +class ObservedTransport implements Transport { + onclose?: () => void; + onerror?: (error: Error) => void; + onmessage?: (message: T, extra?: MessageExtraInfo) => void; + private readonly awaiting = new Map(); + + constructor(private readonly inner: Transport) {} + + get sessionId(): string | undefined { + return this.inner.sessionId; + } + + get hasPerRequestStream(): boolean | undefined { + return this.inner.hasPerRequestStream; + } + + setProtocolVersion(version: string): void { + this.inner.setProtocolVersion?.(version); + } + + setSupportedProtocolVersions(versions: string[]): void { + this.inner.setSupportedProtocolVersions?.(versions); + } + + async start(): Promise { + this.inner.onmessage = (message, extra) => { + // An answer to a request in scope is handed to the SDK inside that + // request's scope. The SDK drives an input_required continuation from + // this very callback, outside the caller's async chain, and the next + // leg's send must still be recorded against the right call. + const record = this.observe(message); + if (record) wireScope.run(record, () => this.onmessage?.(message, extra)); + else this.onmessage?.(message, extra); + }; + this.inner.onerror = (error) => this.onerror?.(error); + this.inner.onclose = () => this.onclose?.(); + await this.inner.start(); + } + + async send(message: JSONRPCMessage, options?: TransportSendOptions): Promise { + const record = wireScope.getStore(); + let undo: (() => void) | undefined; + if (record && 'method' in message && 'id' in message && message.method === record.method) { + const prior = { handedOff: record.handedOff, currentId: record.currentId, answer: record.answer, error: record.error }; + // Handed off from here on: past this point nothing proves it unsent, + // unless the transport says it wrote nothing. + record.handedOff = true; + record.currentId = message.id; + record.answer = 'none'; + record.error = undefined; + this.awaiting.set(message.id, record); + undo = () => { + this.awaiting.delete(message.id); + Object.assign(record, prior); + }; + } + try { + await this.inner.send(message, options); + } catch (error) { + // Refused before any write: this leg never reached the server, so the + // record is as it was before it. Any other throw (an HTTP send can fail + // after part of the request went out) leaves it handed off. + if (error instanceof NotWrittenError) undo?.(); + throw error; + } + } + + async close(): Promise { + await this.inner.close(); + } + + /** Drop a finished request's entries (unanswered legs included). */ + forget(record: WireRecord): void { + for (const [id, entry] of this.awaiting) if (entry === record) this.awaiting.delete(id); + } + + /** Record an answer to a request in scope; returns its record when it + * answers that record's current leg. */ + private observe(message: JSONRPCMessage): WireRecord | undefined { + if (typeof message !== 'object' || message === null) return undefined; + if (!('id' in message) || (!('result' in message) && !('error' in message))) return undefined; + const record = this.awaiting.get(message.id as string | number); + if (!record) return undefined; + this.awaiting.delete(message.id as string | number); + // A superseded leg's late answer says nothing about the current one. + if (record.currentId !== message.id) return undefined; + if ('error' in message) { + record.answer = 'error'; + record.error = message.error as WireRecord['error']; + } else { + const result = (message as { result?: unknown }).result; + const type = typeof result === 'object' && result !== null ? (result as { resultType?: unknown }).resultType : undefined; + record.answer = type === 'complete' ? 'complete' : type === 'input_required' ? 'input-required' : 'unrecognized'; + if (record.answer !== 'complete') record.continued = true; + } + return record; + } +} + +/** + * The diagnostic for a line that is JSON but not a JSON-RPC message: the + * line's first 200 characters, after the configuration's credentials are + * replaced, so a credential across the cut can't leave its first part. + */ +export function malformedLineDiagnostic( + config: McplServerConfig, + text: string, + error: unknown, + recorded: (text: unknown) => string = serverTextRedactor(config), +): string { + // The parser's message can quote the line, so it's redacted too. + const shown = recorded(text); + const why = recorded(error instanceof Error ? error.message.split('\n')[0] : error); + return `MCP server "${config.id}" sent a malformed JSON-RPC message (${shown.length > 200 ? `${shown.slice(0, 200)}…` : shown}): ${why}`; +} + +/** + * SDK `Transport` over Connectome's stdio spawner. Framing is the same + * newline-delimited JSON-RPC as legacy. Lines that are not JSON are ignored, + * as the legacy engine ignores them. Because this is a custom transport, the + * SDK runs `server/discover` on this very process instead of starting a + * throwaway copy, and cancels with `notifications/cancelled`, which is the + * stdio binding's mechanism. + */ +class SpawnerStdioTransport implements Transport { + onclose?: () => void; + onerror?: (error: Error) => void; + onmessage?: (message: T, extra?: MessageExtraInfo) => void; + private line: McplTransport | null = null; + /** The spawn start() awaits, so a close() made meanwhile closes its child. */ + private spawning: Promise | null = null; + + /** Built once for the transport: every malformed line uses it. */ + private readonly recorded: (text: unknown) => string; + + constructor( + private readonly config: McplServerConfig, + private readonly onStderr: (line: string) => void, + /** The child's pipe failed and it outlived SIGKILL too, so the line + * stays open, owning it (the line's 'unreaped'): `reason` names the + * pipe's error. */ + private readonly onUnreaped: (reason: string) => void, + ) { + this.recorded = serverTextRedactor(config); + } + + async start(): Promise { + // A command that can't be spawned rejects here, with its spawn error, so + // the connect fails with it rather than waiting out its discover probe. + this.spawning = StdioTransport.spawn(this.config); + const line = await this.spawning; + this.line = line; + line.on('line', (text: string) => { + // A line that isn't JSON is ignored, as the legacy engine ignores it. + // A line that is JSON but not a JSON-RPC message (`null`, an array, a + // bare value) is a diagnostic. It must never reach a handler that + // assumes the shape: thrown from this callback, it would end the host. + let parsed: unknown; + try { + parsed = JSON.parse(text); + } catch { + return; + } + let message: JSONRPCMessage; + try { + message = parseJSONRPCMessage(parsed); + } catch (error) { + this.onerror?.(new Error(malformedLineDiagnostic(this.config, text, error, this.recorded))); + return; + } + this.onmessage?.(message); + }); + line.on('stderr', (text: string) => this.onStderr(text)); + line.on('error', (error: Error) => this.onerror?.(error)); + line.on('close', () => this.onclose?.()); + line.on('unreaped', () => this.onUnreaped(`its stdio pipe failed with ${line.pipeFailure?.message ?? 'an error'}`)); + } + + async send(message: JSONRPCMessage): Promise { + if (!this.line || this.line.closed) throw new NotWrittenError(`MCP server "${this.config.id}" stdio transport is closed`); + // Once the child's stdin or stdout has failed, the line is ending it and + // writes nothing more to it, so a message is refused here as not sent. + const failed = this.line.pipeFailure; + if (failed) throw new NotWrittenError(`MCP server "${this.config.id}" stdio pipe failed with ${failed.message}`); + this.line.writeLine(JSON.stringify(message)); + } + + async close(): Promise { + // A close() during the spawn closes the child it yields. start() + // awaited the spawn first, so the line is wired by then, and its close + // ends what the SDK has in flight, as it does after the spawn. + if (!this.line && this.spawning) await this.spawning.catch(() => null); + await this.line?.close(); + } +} + +/** Connectome's credential plumbing as the SDK's minimal bearer seam: a + * cached token, fetched on first use and once again after a 401. */ +function bearerAuth(config: McplServerConfig): AuthProvider | undefined { + const provider = config.accessProvider; + if (!provider && !config.token) return undefined; + let cached: string | undefined; + const resolve = async (): Promise => { + if (provider) { + try { + const fresh = await provider(); + if (fresh) { + // Sent in the Authorization header: the server's text is recorded + // with it replaced. + noteDialSent(config, { token: fresh }); + return fresh; + } + } catch (error) { + // Same fallback as the WebSocket dial: use the static token, and let + // the server refuse a stale credential loudly. The provider's own + // text may quote the credential, so it isn't logged. + console.error( + `[mcp] server "${config.id}": access provider failed (${describeCredentialError(error)}) — using configured fallback`, + ); + } + } + return config.token; + }; + return { + token: async () => (cached ??= await resolve()), + onUnauthorized: async () => { + cached = await resolve(); + }, + }; +} + +/** One connect generation's client and the boundary it talks through. */ +interface Session { + readonly generation: number; + readonly client: Client; + readonly wire: ObservedTransport; + /** Its transport has closed, whether or not the session was adopted yet. */ + lost: boolean; +} + +export class ModernMcpConnection extends EventEmitter { + readonly id: string; + readonly family = 'modern' as const; + readonly transportKind: 'stdio' | 'http'; + /** The pinned revision, once a connect has established it; null before. */ + protocolVersion: string | null = null; + + private session: Session | null = null; + /** A connect in flight, owned so `close()` can end it. */ + private opening: { session: Session; done: Promise } | null = null; + private generation = 0; + private closedByHost = false; + private reconnectEnabled: boolean; + private readonly reconnectIntervalMs: number; + private readonly reconnectMaxIntervalMs: number; + private reconnectTimer: ReturnType | null = null; + private reconnectAttempts = 0; + /** At most one pending reopen, and only ever for the current generation. */ + private relisten: { generation: number; timer: ReturnType } | null = null; + private relistenAttempts = 0; + /** + * Generations whose cleanup could not confirm their child exited. While any + * is held, no replacement is started: launching another would orphan the + * first. close() retries them and reports what still fails. + */ + private unreaped: Array<{ session: Session; failure: Error }> = []; + /** A lost session's reap in flight: the reconnect follows it, and + * close() waits for it. */ + private losing: { session: Session; done: Promise } | null = null; + /** Child stderr from before anyone listens (the caller attaches only once + * connect() returns), delivered to the first listener. Bounded: the + * newest lines are kept. */ + private stderrBacklog: string[] = []; + + private constructor(private readonly config: McplServerConfig) { + super(); + this.id = config.id; + const binding = checkServerConfig(config); + if (binding.family !== 'modern') { + throw new Error(`MCP server "${config.id}" is not configured for modern MCP; it belongs to McplServerConnection`); + } + this.transportKind = binding.transport === 'http' ? 'http' : 'stdio'; + this.reconnectEnabled = config.reconnect === true; + this.reconnectIntervalMs = config.reconnectIntervalMs ?? 5000; + this.reconnectMaxIntervalMs = config.reconnectMaxIntervalMs ?? 300_000; + // An 'error' with no listener would throw out of an SDK callback and take + // the host down; out-of-band errors here are diagnostics, never fatal. + this.on('error', (error: Error) => { + if (this.listenerCount('error') === 1) console.error(`[mcp] ${this.id}: ${redactServerText(error.message, this.config)}`); + }); + this.on('newListener', (event: string | symbol) => { + if (event !== 'stderr' || this.stderrBacklog.length === 0) return; + const pending = this.stderrBacklog; + this.stderrBacklog = []; + // After the listener is added (newListener fires just before). + queueMicrotask(() => { for (const line of pending) this.emit('stderr', { line }); }); + }); + } + + private deliverStderr(line: string): void { + if (this.listenerCount('stderr') > 0) { + this.emit('stderr', { line }); + return; + } + this.stderrBacklog.push(line); + if (this.stderrBacklog.length > STDERR_BACKLOG_LINES) this.stderrBacklog.shift(); + } + + /** The per-call deadline: `requestTimeoutMs`, validated positive. */ + get requestTimeoutMs(): number { + return this.config.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS; + } + + get isConnected(): boolean { + return this.session !== null; + } + + /** Whether a lost connection will be re-established in the background. */ + get willReconnect(): boolean { + return this.reconnectEnabled && !this.closedByHost && this.unreaped.length === 0; + } + + /** Whether close() has been called: closed for good, or closing. A + * connection lost on its own is not closed in this sense. */ + get isClosed(): boolean { + return this.closedByHost; + } + + /** Whether a child this connection launched couldn't be confirmed exited. + * While it's true the connection owns that child, so whatever registered + * it keeps its id held; close() retries the reap. It's settled only once + * a reap has finished: while a lost session's reap or a close runs, it + * reads false, so read it once lossReaped() or close() has settled. */ + get holdsUnreapedChild(): boolean { + return this.unreaped.length > 0; + } + + /** The reap of a session lost on its own, settled once it's done, whether + * or not it confirmed the child exited (holdsUnreapedChild says which); + * settled at once when no such reap is running. */ + lossReaped(): Promise { + return this.losing?.done ?? Promise.resolve(); + } + + /** A failed generation's cleanup couldn't reap its child: keep owning it, + * and halt reconnecting until close() settles it. */ + private holdUnreaped(session: Session, failure: Error): void { + if (this.unreaped.some((held) => held.session === session)) return; + this.unreaped.push({ session, failure }); + this.emit('error', new Error( + `MCP server "${this.id}": ${failure.message}. Not starting another launch while it may still be running; close the connection to retry the reap.`, + )); + } + + /** The halt, as a permanent failure for the reconnect events. */ + private unreapedFailure(): string | null { + const held = this.unreaped[0]; + if (!held) return null; + const halted = this.reconnectEnabled ? '; reconnecting is halted' : ''; + return `the previous launch could not be reaped (${held.failure.message})${halted}`; + } + + /** + * A connection that hasn't started: configuration is validated (and + * throws), nothing is launched or dialed. An owner can register it, and + * wire its events, before {@link start}, so a connect in flight is + * already the owner's to close. + */ + static create(config: McplServerConfig): ModernMcpConnection { + return new ModernMcpConnection(config); + } + + /** The first connect, shared by every call to start(). */ + private starting: Promise | null = null; + + /** + * The first connect. Without `reconnect` a failure throws. With it, the + * connection keeps retrying in the background, as the legacy engine's + * stub does, and reports `connect-failed`, unless the failed launch + * couldn't be reaped: no retry can come then, so it throws too. A + * connection closed before or during its first connect resolves + * unconnected. A connection starts once: + * later calls return the first call's promise, so a second session never + * opens beside the first. + */ + start(): Promise { + return (this.starting ??= this.firstConnect()); + } + + private async firstConnect(): Promise { + try { + await this.open(); + } catch (error) { + if (this.closedByHost) return; + // No common revision is a configuration fact, not an outage: a retry + // would restart the server to hear the same answer. + if (error instanceof McplProtocolVersionError) { + this.reconnectEnabled = false; + throw error; + } + // A launch that couldn't be reaped halts reconnecting, so no retry + // will come: the first connect fails, as one without reconnect does. + if (!this.reconnectEnabled || this.holdsUnreapedChild) throw error; + console.error(`MCP server "${this.id}" initial connect failed, will retry:`, redactServerText((error as Error).message, this.config)); + this.reconnectAttempts = 1; + this.emit('connect-failed', { error: (error as Error).message, attempt: 0 }); + this.scheduleReconnect(); + } + } + + /** + * {@link create} and {@link start} in one step. A first connect that fails + * and whose cleanup couldn't confirm its child exited throws + * {@link McplUnreapedLaunchError}, whose connection owns that launch, as + * the legacy engine's connect does: its close() retries the reap. + */ + static async connect(config: McplServerConfig): Promise { + const connection = ModernMcpConnection.create(config); + try { + await connection.start(); + } catch (error) { + const held = connection.unreaped[0]; + if (held) throw new McplUnreapedLaunchError(connection, error instanceof Error ? error : new Error(String(error)), held.failure); + throw error; + } + return connection; + } + + private isCurrent(generation: number): boolean { + return generation === this.generation && !this.closedByHost; + } + + /** One connect generation: a fresh client over a fresh transport. */ + private async open(): Promise { + // Closed before this connect began: launch nothing. + if (this.closedByHost) return; + const generation = ++this.generation; + this.cancelRelisten(); + // A new generation's subscription has failed nothing yet: its first + // reopen backs off from the base interval, not from the previous + // generation's failures. + this.relistenAttempts = 0; + const inner: Transport = this.transportKind === 'stdio' + ? new SpawnerStdioTransport( + this.config, + (line) => { if (this.isCurrent(generation)) this.deliverStderr(line); }, + // A child whose pipe failed and that outlived SIGKILL: a loss, whose + // reap finds the child still running and holds the launch, as for any + // launch it can't reap. During a connect there is no session to lose + // yet: the connect's own cleanup holds it, once the connect fails. + (reason) => { if (this.isCurrent(generation)) this.handleLost(generation, reason); }, + ) + : new StreamableHTTPClientTransport(new URL(this.config.url!), { authProvider: bearerAuth(this.config) }); + const wire = new ObservedTransport(inner); + + const client = new Client( + { name: 'agent-framework', version: '1.0.0' }, + { + versionNegotiation: { mode: { pin: MODERN_MCP_PROTOCOL_VERSION } }, + // Nothing advertised: this client supplies no sampling, elicitation + // or roots, so a conforming server cannot require them. + capabilities: {}, + listChanged: { + tools: { + autoRefresh: false, + debounceMs: 0, + onChanged: () => { if (this.isCurrent(generation)) this.emit('tools-list-changed'); }, + }, + }, + }, + ); + const session: Session = { generation, client, wire, lost: false }; + client.onerror = (error: Error) => { + // The SDK reports late responses to deadline-abandoned calls here + // ("unknown message ID"), and a list-change subscription it could not + // open at connect, among other out-of-band errors. They are emitted as + // this connection's 'error', so they reach the host's error listeners + // (and are logged when there are none). They never change a call's + // outcome, which was already reported. + if (this.isCurrent(generation)) this.emit('error', error); + }; + client.onclose = () => { + // Marked before the session is adopted too: a close during the connect + // makes it a failed connect (see establish). + session.lost = true; + if (this.isCurrent(generation)) this.handleLost(generation, 'transport closed'); + }; + + // `done` settles only once this connect's own cleanup has finished, on + // every path: close() waits on it, and must not resolve while that + // cleanup could still find a child it can't reap. + let settle!: () => void; + this.opening = { session, done: new Promise((resolve) => { settle = resolve; }) }; + try { + await this.establish(session); + } finally { + if (this.opening?.session === session) this.opening = null; + settle(); + } + } + + /** Connect one generation's client, and adopt it if it's still current. */ + private async establish(session: Session): Promise { + try { + await session.client.connect(session.wire, { timeout: CONNECT_TIMEOUT_MS }); + } catch (error) { + const failure = await ModernMcpConnection.closeSession(session); + if (failure) this.holdUnreaped(session, failure); + throw this.versionVerdict(error) ?? error; + } + if (!this.isCurrent(session.generation)) { + // Superseded or closed while connecting: this generation never serves. + const failure = await ModernMcpConnection.closeSession(session); + if (failure) this.holdUnreaped(session, failure); + return; + } + if (session.lost) { + // The transport closed while the client connected, such as a child + // that exited at the connect-time list-change subscribe: the SDK + // reports that subscribe's failure only as an error, and resolves the + // connect anyway (R1-2). Adopted, the session would read as connected + // with nothing behind it, and no later close would come to reconnect + // it. So the connect failed. + const failure = await ModernMcpConnection.closeSession(session); + if (failure) this.holdUnreaped(session, failure); + throw new Error(`MCP server "${this.id}" closed its connection while connecting`); + } + this.session = session; + this.protocolVersion = MODERN_MCP_PROTOCOL_VERSION; + this.reconnectAttempts = 0; + this.keepListening(session, session.client.autoOpenedSubscription); + } + + /** + * The server refused the pinned revision (no common one in its + * `server/discover` answer, or an unsupported-version error): a + * configuration verdict, as for the legacy engine, in the same type, so + * every reader treats both engines alike. Retrying can't change it. + */ + private versionVerdict(error: unknown): McplProtocolVersionError | null { + if (!(error instanceof UnsupportedProtocolVersionError)) return null; + const data = (error as { data?: unknown }).data as { supported?: unknown } | undefined; + const supported = Array.isArray(data?.supported) && data.supported.every((v) => typeof v === 'string') + ? (data.supported as string[]) + : null; + // Legacy MCP is reached over stdio or WebSocket. A stdio server's + // family is its configuration's choice; an http(s) url is modern by its + // scheme, so there is nothing to reconfigure on it. + const legacyHint = !supported?.includes(LEGACY_MCP_PROTOCOL_VERSION) + ? '' + : this.transportKind === 'stdio' + ? ` It speaks legacy MCP: configure it without protocol: 'modern'.` + : ` It speaks only legacy MCP, which this client reaches over stdio or WebSocket (ws://, wss://), not http(s).`; + return new McplProtocolVersionError( + `MCP server "${this.id}" does not support MCP ${MODERN_MCP_PROTOCOL_VERSION}` + + (supported ? ` (it supports: ${supported.join(', ') || 'none listed'}).` : ` (${error.message}).`) + + legacyHint, + 'rejected', + null, + supported, + (error as { code?: unknown }).code === undefined ? undefined : Number((error as { code?: unknown }).code), + (error as { data?: unknown }).data, + MODERN_MCP_PROTOCOL_VERSION, + ); + } + + /** + * Keep list-change awareness while this generation lives. The SDK opens the + * subscription at connect when the server advertises tool list changes. If + * that open failed (the SDK reports it through `onerror` and connects + * anyway), or the subscription later ends without our asking, it is + * reopened with backoff. Once it's back the inventory is refreshed, since + * changes may have been missed. A lost transport is the reconnect path's + * job, not this one's. + */ + private keepListening(session: Session, subscription: McpSubscription | undefined): void { + if (!session.client.getServerCapabilities()?.tools?.listChanged) return; + if (!subscription) { + this.scheduleRelisten(session.generation); + return; + } + void subscription.closed.then((reason) => { + if (reason === 'local' || this.session !== session || !this.isCurrent(session.generation)) return; + this.scheduleRelisten(session.generation); + }); + } + + private cancelRelisten(): void { + if (this.relisten) clearTimeout(this.relisten.timer); + this.relisten = null; + } + + private scheduleRelisten(generation: number): void { + if (!this.isCurrent(generation)) return; + if (this.relisten?.generation === generation) return; + this.cancelRelisten(); + const timer = setTimeout(() => { + if (this.relisten?.timer === timer) this.relisten = null; + void this.reopenSubscription(generation); + }, this.backoffDelay(Math.max(1, this.relistenAttempts))); + timer.unref?.(); + this.relisten = { generation, timer }; + } + + private async reopenSubscription(generation: number): Promise { + const session = this.session; + if (!session || session.generation !== generation || !this.isCurrent(generation)) return; + try { + const subscription = await session.client.listen({ toolsListChanged: true }, { timeout: CONNECT_TIMEOUT_MS }); + if (this.session !== session || !this.isCurrent(generation)) { + await subscription.close().catch(() => {}); + return; + } + this.relistenAttempts = 0; + this.keepListening(session, subscription); + this.emit('tools-list-changed'); + } catch (error) { + if (this.session !== session || !this.isCurrent(generation)) return; + this.relistenAttempts++; + this.emit('error', new Error(`MCP server "${this.id}" could not reopen its list-change subscription: ${(error as Error).message}`)); + this.scheduleRelisten(generation); + } + } + + /** The transport went away (child exit, or a closed HTTP client). */ + private handleLost(generation: number, reason: string): void { + const session = this.session; + if (session?.generation !== generation) return; + this.session = null; + this.cancelRelisten(); + // Reap before any reconnect, so a second child never runs beside the + // first. A transport usually closes once its child has exited, but a + // loss read from a failed pipe can come while the child still runs + // (R1-1). A launch that can't be reaped halts reconnecting, as a failed + // reconnect's does, and is a permanent failure, as that one is: no + // later attempt will come. The reap starts, and is recorded, before + // 'close' is emitted, so a listener that throws can't skip it. + const done = ModernMcpConnection.closeSession(session).then((failure) => { + if (this.losing?.session === session) this.losing = null; + if (failure) { + this.holdUnreaped(session, failure); + // A close() the host made meanwhile reports the failure itself. + if (!this.closedByHost) { + this.emit('reconnect-failed', { error: `${reason}; ${this.unreapedFailure()}`, attempt: Math.max(1, this.reconnectAttempts), permanent: true }); + } + } else if (this.willReconnect) this.scheduleReconnect(); + }); + this.losing = { session, done }; + this.emit('close', { reason }); + } + + /** The legacy engine's backoff: doubling from the base interval, capped, + * with ±25% jitter, and never past the timer limit (a longer delay would + * fire at once, turning the backoff into a tight loop). */ + private backoffDelay(failures: number): number { + const exponent = Math.max(0, Math.min(failures - 1, 30)); + const capped = Math.min(this.reconnectIntervalMs * 2 ** exponent, this.reconnectMaxIntervalMs); + return Math.min(capped * (0.75 + Math.random() * 0.5), MAX_TIMER_MS); + } + + private scheduleReconnect(): void { + if (!this.willReconnect || this.reconnectTimer) return; + this.reconnectTimer = setTimeout(() => { + this.reconnectTimer = null; + void this.attemptReconnect(); + }, this.backoffDelay(this.reconnectAttempts)); + } + + private async attemptReconnect(): Promise { + if (!this.willReconnect || this.session) return; + const attempt = Math.max(1, this.reconnectAttempts); + try { + await this.open(); + if (!this.session) return; + console.error(`MCP server "${this.id}" reconnected`); + this.emit('reconnect', { attempts: attempt }); + } catch (error) { + if (this.closedByHost) return; + this.reconnectAttempts = attempt + 1; + const halted = this.unreapedFailure(); + if (halted) { + this.emit('reconnect-failed', { error: `${(error as Error).message}; ${halted}`, attempt, permanent: true }); + return; + } + // No common revision: every further attempt would hear the same + // answer. Stop; only the configuration can fix it. + if (error instanceof McplProtocolVersionError) { + this.reconnectEnabled = false; + this.emit('reconnect-failed', { error: error.message, attempt, permanent: true }); + return; + } + this.emit('reconnect-failed', { error: (error as Error).message, attempt }); + this.scheduleReconnect(); + } + } + + /** + * The server's complete tool inventory. The SDK follows every page; one + * budget, {@link LIST_TIMEOUT_MS}, bounds the whole walk, not each page. + * Its failures use the request-outcome contract with `tools/list` as the + * observed method. A relist the SDK makes inside a tool call is auxiliary + * to that call, and leaves the call's evidence alone. + */ + async listTools(): Promise { + const { tools } = await this.request('tools/list', 'tools/list', (client) => + client.listTools(undefined, { + cacheMode: 'refresh', + timeout: LIST_TIMEOUT_MS, + signal: AbortSignal.timeout(LIST_TIMEOUT_MS), + }), LIST_TIMEOUT_MS); + return tools.map((tool) => ({ + name: tool.name, + ...(tool.description !== undefined ? { description: tool.description } : {}), + inputSchema: tool.inputSchema as Record, + ...(tool._meta !== undefined ? { _meta: tool._meta as Record } : {}), + })); + } + + /** + * Call a tool within the configured deadline. One budget covers the whole + * logical call: the SDK's per-leg timer and its `maxTotalTimeout` across + * continuation rounds, and an abort signal around any retry the SDK makes + * before dispatch (an auth refresh, or a header-mismatch relist). + */ + async callTool(name: string, args: Record): Promise { + const timeoutMs = this.requestTimeoutMs; + const result = await this.request('tools/call', `tools/call "${name}"`, (client) => + client.callTool( + { name, arguments: args }, + { timeout: timeoutMs, maxTotalTimeout: timeoutMs, signal: AbortSignal.timeout(timeoutMs) }, + ), timeoutMs); + return { + content: result.content ?? [], + ...(result.isError !== undefined ? { isError: result.isError } : {}), + ...('structuredContent' in result ? { structuredContent: result.structuredContent } : {}), + ...(result._meta !== undefined ? { _meta: result._meta as Record } : {}), + }; + } + + /** Run one logical request in a wire scope, and classify its failure by + * what that scope saw. */ + private async request(method: string, what: string, run: (client: Client) => Promise, timeoutMs?: number): Promise { + const session = this.session; + if (!session) { + // Connectome refuses before anything reaches the SDK: provably not sent. + throw new McplRequestError(`Cannot send ${what}: connection to "${this.id}" is not established`, 'not-sent'); + } + const record: WireRecord = { method, handedOff: false, answer: 'none', continued: false }; + try { + return await wireScope.run(record, () => run(session.client)); + } catch (error) { + throw this.requestFailure(what, record, error, timeoutMs); + } finally { + session.wire.forget(record); + } + } + + /** + * Close one generation: the transport first (during discovery the SDK + * hasn't attached it yet, so only closing it ends the probe), then the + * client. The transport's close is the reaping verdict, and its failure is + * returned, never swallowed. It never rejects, which lossReaped() and the + * framework's unguarded wait on it rely on. + */ + private static async closeSession(session: Session): Promise { + let failure: Error | null = null; + try { + await session.wire.close(); + } catch (error) { + failure = error instanceof Error ? error : new Error(String(error)); + } + await session.client.close().catch(() => { /* SDK teardown; the verdict is the transport's */ }); + return failure; + } + + /** The close in flight, which later calls wait behind. */ + private closing: Promise | null = null; + + /** Set when the host closes a live session; cleared, with the one + * 'close', by the close that confirms its teardown. */ + private closeIncomplete = false; + + /** + * Close for good: no reconnect, no reopened subscription, no connect left + * in flight. Awaits the in-flight connect's end, and rejects while a child + * can't be confirmed exited: with that failure itself, the transport's + * own error, or an AggregateError when there are several, as the MCPL + * engine does. Calls queue behind the one in flight, so none resolves + * while teardown is still pending. A call after a failed close re-checks + * what wasn't reaped. 'close' is emitted once, when the teardown of a + * session the host closed while live completes, not on a failed close. + */ + close(): Promise { + // Closed from this call on: a connect or start() in the same turn sees + // it, and launches nothing. + this.closedByHost = true; + const previous = this.closing; + const next = (previous ? previous.catch(() => {}) : Promise.resolve()).then(() => this.closeStep()); + this.closing = next; + return next; + } + + private async closeStep(): Promise { + this.closedByHost = true; + if (this.reconnectTimer) clearTimeout(this.reconnectTimer); + this.reconnectTimer = null; + this.cancelRelisten(); + const opening = this.opening; + const losing = this.losing; + const session = this.session; + this.session = null; + if (session) this.closeIncomplete = true; + // Every generation this connection still owns: a connect in flight, the + // live one, a lost one still being reaped, and any a failed cleanup + // couldn't reap. + const targets: Session[] = []; + if (opening) targets.push(opening.session); + if (session) targets.push(session); + if (losing) targets.push(losing.session); + for (const held of this.unreaped) if (!targets.includes(held.session)) targets.push(held.session); + this.unreaped = []; + const failures: Error[] = []; + for (const target of targets) { + // Closing the transport ends a probe in flight too: its connect then + // rejects and its own cleanup runs (sharing this verdict). + const failure = await ModernMcpConnection.closeSession(target); + if (failure) { + failures.push(failure); + if (!this.unreaped.some((held) => held.session === target)) this.unreaped.push({ session: target, failure }); + } + } + // The connect's own cleanup, and a lost session's reap, close their + // generation again, after the close above; each `done` settles once it + // has. A child that either cleanup couldn't reap is this close's failure + // too. + for (const pending of [opening, losing]) { + if (!pending) continue; + await pending.done; + for (const held of this.unreaped) if (!failures.includes(held.failure)) failures.push(held.failure); + } + if (failures.length === 1) throw failures[0]; + if (failures.length > 1) { + throw new AggregateError(failures, `MCP server "${this.id}" did not close cleanly: ${failures.map((f) => f.message).join('; ')}`); + } + if (this.closeIncomplete) { + this.closeIncomplete = false; + this.emit('close', { reason: 'closed by host' }); + } + } + + /** + * Map a failure onto the request-outcome contract by what crossed the + * transport boundary for this request, claiming no more than that shows. + * The rules apply in this order. The call's answer is the server's final + * one: an error or a `complete` result. + * - nothing handed to the transport: `not-sent`, whatever was raised (a + * pre-dispatch validation, a signal already aborted, no transport, a send + * Connectome's stdio line refused before writing, an HTTP refusal before + * any request went out); + * - the most recent leg answered with a JSON-RPC error: `error-response` + * with that error's data, and its code in the message; + * - the most recent leg answered with a complete result: the server + * finished, but what it returned can't be used (output validation). + * That's a plain error, not a request outcome; + * - the most recent leg answered without finishing the call: `no-response`, + * since the server may have acted on what it received. An + * `input_required` answer asks for another round, and the call can end + * before that round is sent: at the deadline, when the connection is + * gone, or when this client can't go on (input it can't supply, the + * rounds cap). Nothing is in flight then, so nothing is cancelled. A + * result of a type this client doesn't recognize ends the call where it + * is; + * - an HTTP 401/403 refusal of the current leg: `error-response`, the + * status in `data.httpStatus` (the server answered at the HTTP layer, + * outside the JSON-RPC code space). After an earlier leg + * asked for another round it is `no-response` instead: the server may + * already have acted on the call; + * - handed off and unanswered: `no-response`. At the deadline, cancellation + * was requested and the outcome is unknown. + * A `tools/list` failure says so: listing changes nothing on the server, so + * a retry is safe, where a tool call's retry may duplicate a side effect. + */ + private requestFailure(what: string, record: WireRecord, error: unknown, timeoutMs?: number): Error { + const prefix = `MCP server "${this.id}"`; + const message = redactServerText(error instanceof Error ? error.message : String(error), this.config); + const advice = record.method === 'tools/list' + ? 'Listing changes nothing on the server, so it is safe to retry.' + : 'Verify state before retrying; a blind retry of a side-effecting tool may duplicate it.'; + if (!record.handedOff) { + return new McplRequestError(`${prefix} ${what} was not sent: ${message}`, 'not-sent'); + } + if (record.answer === 'error' && record.error) { + const { code, message: wireMessage, data } = record.error; + return new McplRequestError(`${prefix} returned error for ${what}: [${code}] ${wireMessage}`, 'error-response', data); + } + if (record.answer === 'complete') { + return new Error(`${prefix} answered ${what}, but the result can't be used: ${message}`); + } + const deadline = error instanceof SdkError && error.code === SdkErrorCode.RequestTimeout; + const within = deadline && timeoutMs ? ` within ${timeoutMs}ms` : ''; + if (record.answer === 'input-required' || record.answer === 'unrecognized') { + const reason = /[.!?]$/.test(message) ? message : `${message}.`; + const ending = record.answer === 'unrecognized' + ? `its answer was not a complete result: ${reason}` + : deadline + ? 'it asked for another round (input_required), and the deadline passed before that round was sent, so nothing was in flight to cancel.' + : `it asked for another round (input_required), and the call ended before that round was sent: ${reason}`; + return new McplRequestError( + `${prefix} did not finish ${what}${within}: ${ending} ` + + `The outcome is unknown: the server may already have acted on the call. ${advice}`, + 'no-response', + ); + } + // An HTTP refusal of the credential: 401 or 403, however the SDK raised + // it. It raises a plain 403 under another code than its step-up limit's, + // and an insufficient_scope 403 as InsufficientScopeError when no OAuth + // provider is configured. + const refusedWith = error instanceof SdkHttpError && (error.status === 401 || error.status === 403) + ? error.status + : error instanceof InsufficientScopeError ? 403 : null; + if (refusedWith !== null) { + if (record.continued) { + return new McplRequestError( + `${prefix} did not finish ${what}: it asked for another round (input_required), and the server refused a later one: ` + + `HTTP ${refusedWith} (${message}). The outcome is unknown: the server may already have acted on the call. ${advice}`, + 'no-response', + ); + } + // The response body (the SDK's `text`) is the server's text and can + // echo the request's url, so it's left out of the data; the status stays. + const { text: _body, ...data } = (error instanceof SdkHttpError && error.data && typeof error.data === 'object' ? error.data : {}) as Record; + return new McplRequestError(`${prefix} refused ${what}: HTTP ${refusedWith} (${message})`, 'error-response', { ...data, httpStatus: refusedWith }); + } + if (deadline) { + const outcome = record.method === 'tools/list' + ? 'the list did not arrive in time.' + : 'the outcome is unknown: the tool may still complete server-side.'; + return new McplRequestError( + `${prefix} did not answer ${what}${within}. Cancellation was requested; ${outcome} ${advice}`, + 'no-response', + ); + } + return new McplRequestError(`${prefix} ${what} failed: ${message}`, 'no-response'); + } +} diff --git a/src/mcpl/protocol-family.ts b/src/mcpl/protocol-family.ts new file mode 100644 index 000000000..c93454b7d --- /dev/null +++ b/src/mcpl/protocol-family.ts @@ -0,0 +1,652 @@ +/** + * Which MCP protocol family a configured server speaks, and over which + * transport. Both are decided from the configuration alone, before anything + * is spawned or dialed: there is no cross-family probing and no fallback. + * + * - **Legacy**: MCP 2024-11-05, with MCPL riding on it as an experimental + * capability. `McplServerConnection` speaks it, over stdio or MCPL's + * WebSocket binding. + * - **Modern**: MCP 2026-07-28, which is stateless. It has no `initialize`; + * every request carries its version and client capabilities in `_meta`, and + * server-to-client requests are replaced by in-band continuation. + * `ModernMcpConnection` speaks it, over stdio or Streamable HTTP. + * + * A URL's scheme decides a network server's family: `ws://`/`wss://` is + * legacy, `http://`/`https://` is modern, because neither family has the + * other's binding. A stdio (`command`) server can be either. It is legacy + * unless its config says `protocol: 'modern'`, so every existing stdio + * configuration opens exactly as before. + * + * Why no auto-detection: the modern spec's discover-first probe makes some + * real MCPL servers exit (heartbeat-mcpl closes on any first message that + * isn't `initialize`), the SDK's stdio probe launches the server twice, and a + * server's era is a stable fact its configuration can state. + */ +import { DENY_BY_DEFAULT, capabilityPatternMatches } from './capability-grant.js'; +import { DEFAULT_EAGER_FETCH_MAX_BYTES, DEFAULT_MAX_TOTAL_BYTES } from './reference-fetcher.js'; +import { DEFAULT_MAX_INPUT_BYTES } from './tool-lifecycle.js'; +import type { McplServerConfig, ScopeConfig } from './types.js'; + +/** The one revision the legacy engine offers and accepts. */ +export const LEGACY_MCP_PROTOCOL_VERSION = '2024-11-05'; +/** The revision the modern engine pins. */ +export const MODERN_MCP_PROTOCOL_VERSION = '2026-07-28'; + +export type McpProtocolFamily = 'legacy' | 'modern'; +export type McpTransportKind = 'stdio' | 'websocket' | 'http'; + +export interface ServerBinding { + family: McpProtocolFamily; + transport: McpTransportKind; +} + +/** The largest delay a Node/Bun timer honors (2³¹−1 ms, about 24.8 days). */ +export const MAX_TIMER_MS = 2_147_483_647; + + +type BindingFields = Pick; + +/** + * A path segment that may be a credential. Some hosted servers put a + * per-user secret in the path (`…/mcp/s//mcp`). A random token + * (base64url, hex, a uuid) is long and carries a digit; anything very long + * is treated as one too. A short or plain-word segment can't be told apart + * from a name, so it stays. + */ +function credentialLikeSegment(segment: string): boolean { + return (segment.length >= 16 && /[0-9]/.test(segment)) || segment.length >= 32; +} + +/** Query parameter names that say they carry a credential. */ +const CREDENTIAL_PARAM = /token|key|secret|password|auth|signature/i; + +/** + * A query parameter name that may itself be a credential, which + * {@link redactUrl} replaces and a warning never quotes: one shaped like a + * token, or one with no value whose name says it carries a credential, since + * a bare token rides as a name alone (`?`). + */ +function credentialLikeQueryName(name: string, value: string): boolean { + return credentialLikeSegment(name) || (value === '' && CREDENTIAL_PARAM.test(name)); +} + +/** + * A url as text a person or a log reads it: its scheme, host and port, and + * its path with any credential-like segment replaced. The userinfo is + * removed and each query parameter's value replaced, since a credential + * rides there too, and so is a parameter name that may itself be one + * ({@link credentialLikeQueryName}). A url that doesn't parse can't be + * redacted, so it isn't shown at all. + */ +export function redactUrl(url: string | undefined): string { + let parsed: URL; + try { + parsed = new URL(url ?? ''); + } catch { + return '(an invalid url, not shown)'; + } + parsed.username = ''; + parsed.password = ''; + parsed.hash = ''; + parsed.pathname = parsed.pathname.split('/').map((segment) => (credentialLikeSegment(segment) ? 'redacted' : segment)).join('/'); + parsed.search = new URLSearchParams([...parsed.searchParams].map(([name, value]): [string, string] => [credentialLikeQueryName(name, value) ? 'redacted' : name, 'redacted'])).toString(); + return parsed.toString(); +} + +/** + * Server and transport text as the framework records it, with the strings a + * server's configuration knows to be credentials replaced, and those its + * dials sent: + * - from the url as each dial sends it ({@link urlAsDialed}), and decoded: + * the configured one, and each one dialed with an access provider's token + * ({@link noteDialSent}): credential-like path segments and + * query names, credential-shaped or credential-named query values, every + * other non-empty parameter as its whole name=value, and a WebSocket url's + * username and password, its userinfo as one unit before its @ (the + * username alone when there's no password), with no length floor, so a + * short password goes with its username and a short username alone goes + * too, and the Basic credential a WebSocket client sends for that userinfo; + * - the static token, and each token an access provider handed a dial for + * a header, raw, as JSON encoders write it (JSON.stringify's escapes, and + * with its slashes escaped too, as some encoders write them), and percent- + * or form-encoded with either case of hex; + * - a stdio server's environment values whose names say they're + * credentials, raw, as JSON encoders write them, and percent-encoded with + * either case of hex. + * A string replaced on its own has at least 8 characters, since a shorter + * one can't be told from ordinary text, and it is replaced wherever it + * occurs, after an escape or inside a longer token too. A query parameter is + * also replaced as its whole name=value, only where no letter or digit + * adjoins the pair, so v=2 leaves dev=2 alone; a short value goes only that + * way. Every place one of these matches is replaced, overlapping places + * included, and the replacement is never scanned again; text that may come + * from several servers takes one pass over all their configurations' + * strings (serverCredentialPatterns), so strings of different servers that + * overlap are both replaced whole. A credential neither the configuration + * holds nor a dial of it sent, such as another origin's, is outside it, and + * so are other encodings of the ones it holds. JSON-RPC answers, and the + * error a host's own connect raises, pass as they are. + */ +export function redactServerText(text: unknown, config: Pick): string { + return serverTextRedactor(config)(text); +} + +/** + * {@link redactServerText} for one configuration, its strings found and its + * pattern built once, and again once a dial has sent something new + * ({@link noteDialSent}): for a connection's records, such as each stderr + * line. + */ +export function serverTextRedactor(config: Pick): (text: unknown) => string { + let built = -1; + let redact!: (text: unknown) => string; + return (text) => { + const version = dialsSent.get(config)?.version ?? 0; + if (version !== built) { + redact = credentialRedactor(serverCredentialPatterns(config)); + built = version; + } + return redact(text); + }; +} + +/** + * What a configuration's dials sent that the configuration doesn't hold: the + * urls dialed with an access provider's token, and the provider's tokens, + * sent in a url or a header. Kept by the configuration object the dial read. + * For the dials that note anything, a WebSocket dial and an HTTP bearer, that + * is the object the framework records the server's text with. A provider can + * hand each dial a new token, so only the newest sixteen of each are kept, + * newest by when a dial last sent it. + */ +const dialsSent = new WeakMap(); +const DIAL_SENT_KEPT = 16; + +/** Note what one dial sent: the url it dialed, or a token it sent in a + * header. That configuration's patterns cover it from then on. */ +export function noteDialSent(config: object, sent: { readonly url?: string; readonly token?: string }): void { + let entry = dialsSent.get(config); + if (!entry) { + entry = { urls: [], tokens: [], version: 0 }; + dialsSent.set(config, entry); + } + const kept = entry; + const keep = (list: string[], value: string | undefined): void => { + if (!value) return; + const at = list.indexOf(value); + if (at >= 0) { + // Sent again: it moves to the newest place, and the set is unchanged. + list.splice(at, 1); + list.push(value); + return; + } + list.push(value); + if (list.length > DIAL_SENT_KEPT) list.shift(); + kept.version += 1; + }; + keep(kept.urls, sent.url); + keep(kept.tokens, sent.token); +} + +/** + * The url a dial sends. A WebSocket dial sets the token as the `token` + * parameter, and setting it makes URLSearchParams write the whole query + * again, form-encoded (a space as `+`), so the url as sent can differ from + * the url as configured. An http(s) dial sends the url as configured. + */ +export function urlAsDialed(url: URL, token: string | null | undefined): URL { + const dialed = new URL(url); + if (token && (dialed.protocol === 'ws:' || dialed.protocol === 'wss:')) dialed.searchParams.set('token', token); + return dialed; +} + +/** A string {@link redactServerText} replaces, and the pattern that finds it. */ +export interface CredentialPattern { + readonly value: string; + readonly source: string; +} + +/** + * The strings {@link redactServerText} replaces for one configuration, as + * patterns for {@link credentialRedactor}. Text that may come from several + * servers goes through one redactor over all their patterns, so a secret + * that begins inside another configuration's, or shares its start, can't + * leave any of that one in place. + */ +export function serverCredentialPatterns(config: Pick): CredentialPattern[] { + // Strings replaced wherever they occur (after an escape or inside a longer + // token too), and name=value pairs, replaced only where no letter or digit + // adjoins them, so a pair such as v=2 leaves dev=2 alone. A pair's value is + // also replaced on its own when it's a credential, wherever it occurs. + const secrets = new Set(); + const pairs = new Set(); + const userinfo = new Set(); + const standalone = (value: string): void => { if (value.length >= 8) secrets.add(value); }; + const decode = (value: string, query = false): string => { + try { return decodeURIComponent(query ? value.replace(/\+/g, ' ') : value); } + catch { return value; } + }; + // Percent-escapes in lowercase hex, as some encoders write them. + const lowerEscapes = (value: string): string => value.replace(/%[0-9A-F]{2}/g, (escape) => escape.toLowerCase()); + // A string as written; as JSON encoders write it, with JSON.stringify's + // escapes (a quote, a backslash, a control character) and with its + // slashes escaped too, as some encoders write them; and percent-encoded in + // either case of hex (a string encodeURIComponent refuses, such as one + // with an unpaired surrogate, has no encoded form). + const withEncoded = (value: string): void => { + const json = JSON.stringify(value).slice(1, -1); + for (const written of [value, json]) { + standalone(written); + standalone(written.replace(/\//g, '\\/')); + } + let encoded: string; + try { encoded = encodeURIComponent(value); } catch { return; /* no percent-encoded form */ } + standalone(encoded); + standalone(lowerEscapes(encoded)); + }; + // A token as sent: raw in a header, and percent- or form-encoded where a + // transport puts it in a url. + const addToken = (token: string): void => { + withEncoded(token); + const formEncoded = new URLSearchParams({ t: token }).toString().slice(2); + standalone(formEncoded); + standalone(lowerEscapes(formEncoded)); + }; + addToken(config.token ?? ''); + // A stdio server's environment values whose names say they carry a + // credential: a part of the name that ends with a credential word + // (GITHUB_TOKEN, API_KEY, PGPASSWORD, ACCESSTOKEN), and not one where the + // word only begins or sits inside a part (GIT_AUTHOR_NAME, KEYBOARD_LAYOUT, + // TOKENIZERS_PARALLELISM). + for (const [name, value] of Object.entries(config.env ?? {})) { + // A JavaScript host can pass a value that isn't a string; only strings are derived. + if (typeof value === 'string' && name.split(/[_\-.]/).some((part) => CREDENTIAL_NAME_PART.test(part))) withEncoded(value); + } + // A url as a dial sends it. Nothing quotes the configured url before a + // dial, so its form as configured needs no strings of its own. + const addUrl = (url: URL): void => { + for (const segment of url.pathname.split('/')) { + if (credentialLikeSegment(segment)) { + standalone(segment); + standalone(decode(segment)); + } + } + for (const parameter of url.search.slice(1).split('&')) { + const equals = parameter.indexOf('='); + const encodedName = equals < 0 ? parameter : parameter.slice(0, equals); + const encodedValue = equals < 0 ? '' : parameter.slice(equals + 1); + const name = decode(encodedName, true); + const value = decode(encodedValue, true); + if (credentialLikeQueryName(name, value)) { + standalone(encodedName); + standalone(name); + } + if (CREDENTIAL_PARAM.test(name) || credentialLikeSegment(value)) { + standalone(encodedValue); + standalone(value); + } + if (value !== '') { + pairs.add(`${encodedName}=${encodedValue}`); + pairs.add(`${name}=${value}`); + } + } + if (url.protocol === 'ws:' || url.protocol === 'wss:') { + for (const value of [url.username, url.password]) { + standalone(value); + standalone(decode(value)); + } + // The userinfo itself as one unit wherever it stands before an @ with + // no letter or digit just before it: username:password, or the + // username alone when there's no password, as a token written as the + // username is. So a password too short to replace on its own still + // goes with its username, and a short username alone goes too. The @ + // and that boundary anchor the unit as a pair's boundaries do, so it + // has no length floor. And the Basic credential a WebSocket client + // sends for it. Each from the userinfo as written and as decoded. + for (const [username, password] of [[url.username, url.password], [decode(url.username), decode(url.password)]]) { + if (!username && !password) continue; + userinfo.add(password ? `${username}:${password}` : username); + standalone(Buffer.from(`${username}:${password}`).toString('base64')); + } + } + }; + let url: URL | undefined; + try { url = new URL(config.url ?? ''); } catch { /* Still remove the static token. */ } + if (url) addUrl(urlAsDialed(url, config.token)); + // What the dials sent beyond that: urls dialed with an access provider's + // token, and its tokens sent in a header. + const sent = dialsSent.get(config); + for (const dialed of sent?.urls ?? []) { + try { addUrl(new URL(dialed)); } catch { /* a dialed url always parses */ } + } + for (const token of sent?.tokens ?? []) addToken(token); + const escape = (value: string): string => value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + return [ + ...[...secrets].map(value => ({ value, source: escape(value) })), + ...[...pairs].map(value => ({ value, source: `(? ({ value, source: `(? string { + if (patterns.length === 0) return asText; + // Longest first, so at each place the longest string that matches there is + // the match. + const sources = [...patterns].sort((a, b) => b.value.length - a.value.length).map(({ source }) => source); + const pattern = new RegExp([...new Set(sources)].join('|'), 'g'); + return (text) => { + const input = asText(text); + const spans: Array<[start: number, end: number]> = []; + pattern.lastIndex = 0; + for (let match = pattern.exec(input); match; match = pattern.exec(input)) { + const start = match.index; + const end = start + match[0].length; + const last = spans[spans.length - 1]; + if (last && start < last[1]) last[1] = Math.max(last[1], end); + else spans.push([start, end]); + // The next search starts one place on, not after this match, so a + // match that begins inside this one is found too. + pattern.lastIndex = start + 1; + } + let redacted = ''; + let from = 0; + for (const [start, end] of spans) { + redacted += `${input.slice(from, start)}redacted`; + from = end; + } + return redacted + input.slice(from); + }; +} + +/** Any value as text: a value that can't be printed reads as a placeholder. */ +export function asText(text: unknown): string { + if (typeof text === 'string') return text; + try { return String(text); } catch { return '[a value that cannot be printed]'; } +} + +/** A part of an environment variable's name that ends with a credential word, in the singular or plural, with any digits after it. */ +const CREDENTIAL_NAME_PART = /(?:tokens?|keys?|secrets?|passwords?|passwd|pass|passphrases?|passcodes?|pat|auth|authorization|authentication|bearer|cookies?|creds|signatures?|credentials?)\d*$/i; + +/** + * A credential provider's failure as a log line may show it: the error's + * class and its message's length, never the text, which may quote the + * credential it failed to refresh. + */ +export function describeCredentialError(error: unknown): string { + if (!(error instanceof Error)) return `a thrown ${typeof error}`; + const length = error.message.length; + return `${error.constructor?.name || error.name || 'Error'}, ${length} character${length === 1 ? '' : 's'}`; +} + +function urlScheme(config: BindingFields): string { + let parsed: URL; + try { + parsed = new URL(config.url!); + } catch { + // Not echoed: a url that doesn't parse can't be redacted. + throw new Error(`MCP server "${config.id}": invalid url (it isn't shown, since it may carry a credential)`); + } + return parsed.protocol; +} + +/** + * Resolve a server's family and transport. Throws on a configuration that + * names no usable transport (for example an `ftp://` URL, or + * `transport: 'http'` with a WebSocket URL), or names two without saying + * which: `command` and `url` together need a `transport`, since one of them + * goes unused. (A host may layer a url and its transport over a file's + * command, so both with a `transport` stay valid.) + */ +export function resolveServerBinding(config: BindingFields): ServerBinding { + if (config.command && config.url && config.transport === undefined) { + throw new Error( + `MCP server "${config.id}": "command" and "url" are both set, and no "transport" says which one to use: ` + + `a server is either launched (command) or dialed (url)`, + ); + } + const networkKind = (): McpTransportKind => { + const scheme = urlScheme(config); + if (scheme === 'ws:' || scheme === 'wss:') return 'websocket'; + if (scheme === 'http:' || scheme === 'https:') return 'http'; + throw new Error( + `MCP server "${config.id}": url must be ws:// or wss:// (legacy MCPL) or http:// or https:// (modern MCP), got "${scheme}"`, + ); + }; + + let transport: McpTransportKind; + if (config.transport === 'stdio' || (config.transport === undefined && config.command)) { + if (!config.command) { + throw new Error(`MCP server "${config.id}": the stdio transport requires "command"`); + } + transport = 'stdio'; + } else if (config.url) { + transport = networkKind(); + if (config.transport !== undefined && config.transport !== transport) { + throw new Error( + `MCP server "${config.id}": transport "${config.transport}" does not match url ${redactUrl(config.url)}`, + ); + } + } else if (config.transport !== undefined) { + throw new Error(`MCP server "${config.id}": transport "${config.transport}" requires "url"`); + } else { + throw new Error(`MCP server "${config.id}": needs "command" (stdio) or "url" (WebSocket or HTTP)`); + } + + if (transport === 'stdio') return { family: config.protocol ?? 'legacy', transport }; + return { family: transport === 'http' ? 'modern' : 'legacy', transport }; +} + +/** A configured value that would take effect, as opposed to an empty default. */ +function isEffective(value: unknown): boolean { + if (value === undefined || value === null || value === false) return false; + if (Array.isArray(value)) return value.length > 0; + if (typeof value === 'object') return Object.keys(value as object).length > 0; + return true; +} + +function isPlainRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +/** + * The parts of a configuration's MCPL policy whose values grant something, + * by path, such as `allowHostCommands`, `scopes..whitelist`, + * `enabledCapabilities: toolLifecycle.observe`, `toolLifecycle.maxInputBytes` + * or `autofetch.maxBytes`: each + * asks for behaviour only an MCPL server can have. A modern server has none, + * so such a value can't apply there, and is refused rather than ignored, so + * that an operator who chose it learns so. + * + * A value that only takes something away, or repeats a default, is already + * satisfied on a modern server and isn't listed: `enabledFeatureSets`, an + * allow-list over every declared set (`feature-set-manager.ts`); + * `enabledCapabilities` patterns that match no path denied by default, since + * elsewhere it is an allow-list too; `disabledFeatureSets`, + * `disabledCapabilities`, a scope's `blacklist`, `channelSubscription: + * 'manual'` (the default), `toolLifecycle` with only a `maxInputBytes` at + * or below its default or with an `observe` or `inputs` that isn't a policy + * object (which grants nothing), an `autofetch` bound at or below its + * default, and + * `shouldTriggerInference`, which only narrows which channel messages + * trigger inference. So a host that stamps a deny-list on every server it + * deploys can deploy a modern one. Empty values grant nothing either. + */ +export function mcplPolicyGrants(config: McplServerConfig): string[] { + const grants: string[] = []; + if (isEffective(config.allowHostCommands)) grants.push('allowHostCommands'); + // An allow-list, except that a pattern matching a path denied by default + // grants it (computeGrant's explicitlyEnabled), named by the path. + const enabled = config.enabledCapabilities as unknown; + if (Array.isArray(enabled)) { + for (const path of DENY_BY_DEFAULT) { + if (enabled.some((pattern) => typeof pattern !== 'string' || capabilityPatternMatches(pattern, path))) { + grants.push(`enabledCapabilities: ${path}`); + } + } + } else if (isEffective(enabled)) { + grants.push('enabledCapabilities'); + } + if (config.channelSubscription !== 'manual' && isEffective(config.channelSubscription)) grants.push('channelSubscription'); + if (isPlainRecord(config.scopes)) { + for (const [set, scope] of Object.entries(config.scopes)) { + if (isEffective((scope as ScopeConfig | null | undefined)?.whitelist)) grants.push(`scopes.${set}.whitelist`); + } + } else if (isEffective(config.scopes)) { + grants.push('scopes'); + } + for (const key of ['observe', 'inputs'] as const) { + if (isPlainRecord((config.toolLifecycle as Record | undefined)?.[key])) grants.push(`toolLifecycle.${key}`); + } + // A larger input bound than the default asks for more than MCPL gives + // by default, as an autofetch bound above its default does. + const maxInputBytes = (config.toolLifecycle as Record | undefined)?.maxInputBytes; + if (maxInputBytes !== undefined && maxInputBytes !== null && !(typeof maxInputBytes === 'number' && maxInputBytes <= DEFAULT_MAX_INPUT_BYTES)) { + grants.push('toolLifecycle.maxInputBytes'); + } + const bounds = [['maxBytes', DEFAULT_EAGER_FETCH_MAX_BYTES], ['maxTotalBytes', DEFAULT_MAX_TOTAL_BYTES]] as const; + for (const [key, ceiling] of bounds) { + const value = (config.autofetch as Record | undefined)?.[key]; + if (value !== undefined && value !== null && !(typeof value === 'number' && value <= ceiling)) grants.push(`autofetch.${key}`); + } + return grants; +} + +/** + * Problems with a server configuration's protocol settings, as messages; empty + * when it is usable. Legacy configurations are checked only for what this + * design added, so every configuration that worked before still passes but + * one: `command` and `url` with no `transport`, which was launched as stdio + * with its url ignored, and is now refused as naming no transport to use. + */ +export function serverConfigProblems(config: McplServerConfig): string[] { + let binding: ServerBinding; + try { + binding = resolveServerBinding(config); + } catch (error) { + return [(error as Error).message]; + } + const problems: string[] = []; + if (binding.transport === 'http') { + // fetch refuses a url with userinfo, and its error quotes the whole url, + // password included, into the connect failure's log line, trace and + // ops alert. A setting nothing could send is refused, not ignored. + const url = new URL(config.url!); + if (url.username || url.password) { + problems.push( + `MCP server "${config.id}": an http(s):// url can't carry userinfo (user:password@), since the HTTP client refuses such a url; give the credential as "token" or "accessProvider"`, + ); + } + } + if (config.protocol !== undefined) { + if (config.protocol !== 'legacy' && config.protocol !== 'modern') { + problems.push(`MCP server "${config.id}": protocol must be 'legacy' or 'modern', got ${JSON.stringify(config.protocol)}`); + } else if (binding.transport !== 'stdio') { + problems.push( + `MCP server "${config.id}": "protocol" applies only to stdio (command) servers; the url's scheme decides a network server's family`, + ); + } + } + const timeout = config.requestTimeoutMs; + if (binding.family === 'modern') { + if (timeout !== undefined && !(Number.isInteger(timeout) && timeout >= 1 && timeout <= MAX_TIMER_MS)) { + problems.push( + `MCP server "${config.id}": requestTimeoutMs must be an integer from 1 to ${MAX_TIMER_MS} for a modern server (got ${String(timeout)}); every modern call has a deadline`, + ); + } + for (const path of mcplPolicyGrants(config)) { + problems.push(`MCP server "${config.id}": "${path}" asks for MCPL behaviour, which a modern MCP server cannot negotiate`); + } + // A modern stdio server takes no credential: nothing would carry it. + if (binding.transport === 'stdio') { + for (const field of ['token', 'accessProvider'] as const) { + if (config[field]) { + problems.push(`MCP server "${config.id}": "${field}" applies only to a network (url) server; a modern stdio server would ignore it`); + } + } + } + } else if (typeof timeout === 'number' && timeout > MAX_TIMER_MS) { + // Never usable: Node runs a longer timer after about 1 ms, so every + // request timed out at once. Every value that worked (0 included) passes. + problems.push( + `MCP server "${config.id}": requestTimeoutMs must be at most ${MAX_TIMER_MS} (got ${String(timeout)}), or 0 to disable the request timeout; a larger value times every request out at once`, + ); + } + return problems; +} + +/** A host a connection to which never leaves this machine, as the WHATWG URL + * parser gives it. The unspecified addresses count: a connection to 0.0.0.0 + * or [::] reaches this host. So do 127.0.0.0/8 written as an IPv4-mapped + * IPv6 address ([::ffff:127.0.0.1] parses as [::ffff:7f00:1]) and a name + * with its root's trailing dot (localhost.). */ +function isLoopbackHost(hostname: string): boolean { + const host = hostname.toLowerCase().replace(/\.$/, ''); + return host === 'localhost' || host.endsWith('.localhost') || host === '[::1]' || /^127(\.\d{1,3}){3}$/.test(host) || + /^\[::ffff:7f[0-9a-f]{2}:[0-9a-f]{1,4}\]$/.test(host) || host === '0.0.0.0' || host === '[::]'; +} + +/** + * What is usable in a server configuration but worth an operator's attention, + * as messages; empty when there is nothing. A credential for an `http://` or + * `ws://` url whose host isn't loopback crosses the network in cleartext: + * `token` or `accessProvider` (in the `Authorization` header, or the + * WebSocket `?token=` query), and a secret the url carries itself (a ws:// + * url's userinfo, a query parameter whose name says it's a credential, or + * whose value is shaped like one under a plain name, a query parameter name + * that may itself be one, or a path segment that looks like one, as + * {@link redactUrl} judges it). A name that may itself be a credential is + * counted, never quoted. That can be deliberate, on a private network or + * behind a TLS-terminating proxy, so it is a warning and never a refusal. + */ +export function serverConfigWarnings(config: McplServerConfig): string[] { + let transport: McpTransportKind; + try { + transport = resolveServerBinding(config).transport; + } catch { + return []; // an unusable configuration is serverConfigProblems' to report + } + if (transport === 'stdio') return []; + const url = new URL(config.url!); + if ((url.protocol !== 'http:' && url.protocol !== 'ws:') || isLoopbackHost(url.hostname)) return []; + const carried: string[] = []; + if (config.token || config.accessProvider) carried.push('its credential'); + // An http:// url's userinfo is refused (serverConfigProblems); a ws:// + // url sends it. + if (url.protocol === 'ws:' && (url.username || url.password)) carried.push("its url's userinfo"); + if (url.pathname.split('/').some(credentialLikeSegment)) carried.push("a credential-like segment of its url's path"); + const entries = [...url.searchParams]; + const hidden = entries.filter(([name, value]) => credentialLikeQueryName(name, value)).length; + // A parameter carries a credential by its name, or by its value's shape, + // as redaction reads it; the name is shown only when it isn't one itself. + const params = [...new Set(entries.filter(([name, value]) => !credentialLikeQueryName(name, value) && (CREDENTIAL_PARAM.test(name) || credentialLikeSegment(value))).map(([name]) => name))]; + if (params.length > 0) { + carried.push(`its url's query ${params.length === 1 ? 'parameter' : 'parameters'} ${params.map((name) => JSON.stringify(name)).join(', ')}`); + } + if (hidden > 0) { + carried.push(hidden === 1 ? 'a query parameter whose name looks like a credential' : `${hidden} query parameters whose names look like credentials`); + } + if (carried.length === 0) return []; + const what = carried.length === 1 ? carried[0] : `${carried.slice(0, -1).join(', ')} and ${carried[carried.length - 1]}`; + const one = carried.length === 1 && params.length <= 1 && hidden <= 1; + const secure = url.protocol === 'http:' ? 'https' : 'wss'; + return [ + `MCP server "${config.id}": ${what} ${one ? 'goes' : 'go'} to ${url.host} unencrypted (${url.protocol}//), ` + + `where anything on the network path can read ${one ? 'it' : 'them'}; use ${secure}:// unless that network is trusted`, + ]; +} + +/** Throw one error listing every problem. Otherwise log each warning and + * return the resolved binding. Both engines admit a configuration here. */ +export function checkServerConfig(config: McplServerConfig): ServerBinding { + const problems = serverConfigProblems(config); + if (problems.length > 0) throw new Error(problems.join('; ')); + for (const warning of serverConfigWarnings(config)) console.warn(`[mcp] ${warning}`); + return resolveServerBinding(config); +} diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index f33879c58..c70b043c1 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -446,3 +446,48 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI case 'invalid': return c.shown; } } + +/** + * RFC-005 treatment for one MCPL wire block, for a script's structured + * result, which carries blocks rather than text. It makes the decision + * referenceStubOrNull makes for the model's lanes, since what a script prints + * comes back to the model. A reference becomes the stub's facts as data, + * without its URI: the host's reference id, what the server said about it in + * this block (its labels stripped and bounded as in the stub), and `savedTo`, + * the workspace path its payload was fetched to, or null when it wasn't + * (`fetch_reference` takes the id as `ref_id`). An unsaved reference that + * counts as expired also carries `expired`, the reason the stub gives in + * place of advising fetch_reference. The reference is registered bound to + * `serverId`, the server whose tool returned it, as the model's lanes bind + * a tool result's references, so the fetch its id offers can be made. A + * block the classifier + * rejects becomes the notice the model's lanes show, and inline data that + * also claims a bulk disposition the withheld notice, each as a text block. + * A text block, or inline data that claims nothing, is returned as sent. + */ +export function referenceBlockForScript(block: unknown, serverId: string): unknown { + const c = classifyBlock(block); + switch (c.kind) { + case 'text': return block; + case 'inline': + return c.contradiction ? { type: 'text', text: INLINE_WITHHELD_TEXT } : block; + case 'invalid': return { type: 'text', text: c.shown }; + case 'reference': { + const t = c.testimony; + const record = referenceRegistry.register(t, serverId); + // As the stub judges it: a saved copy is still served. + const expiry = record.fetchedPath ? null : referenceExpiry(record.testimony); + return { + type: (block as { type: string }).type, + refId: record.refId, + ...(t.name !== undefined ? { name: sanitizeLabel(t.name, STUB_FIELD_CHARS) } : {}), + ...(t.mimeType !== undefined ? { mimeType: sanitizeLabel(t.mimeType, STUB_FIELD_CHARS) } : {}), + ...(t.sizeBytes !== undefined ? { sizeBytes: t.sizeBytes } : {}), + ...(t.digest !== undefined ? { digest: t.digest } : {}), + ...(t.expiresAt !== undefined ? { expiresAt: sanitizeLabel(t.expiresAt, STUB_FIELD_CHARS) } : {}), + savedTo: record.fetchedPath ?? null, + ...(expiry ? { expired: EXPIRY_REASON[expiry] } : {}), + }; + } + } +} diff --git a/src/mcpl/server-connection.ts b/src/mcpl/server-connection.ts index 0ef6bf244..89d5dbf55 100644 --- a/src/mcpl/server-connection.ts +++ b/src/mcpl/server-connection.ts @@ -16,6 +16,12 @@ import { maskNegotiatedCapabilities } from './capability-mask.js'; import { CapabilityGrant, expandAdvertisementShorthand } from './capability-grant.js'; import { CAPABILITY_DISABLED, boundedForLog, describeCause } from './errors.js'; import { logDetail, logValue } from './log-value.js'; +import { + LEGACY_MCP_PROTOCOL_VERSION, + MODERN_MCP_PROTOCOL_VERSION, + checkServerConfig, + redactServerText, +} from './protocol-family.js'; import type { McplServerConfig, @@ -86,13 +92,14 @@ export function stderrTailOf(error: unknown): string | undefined { } /** For the host's own log line about a failed connect: the server's stderr - * that the error keeps as `stderrTail`, one marked line per line of it, - * each through log-value, since it's the server's text. '' when there's + * that the error keeps as `stderrTail`, with the server's credentials + * taken out by `redact` (its redactServerText), one marked line per line of + * it, each through log-value, since it's the server's text. '' when there's * none. */ -export function withStderr(error: unknown): string { +export function withStderr(error: unknown, redact: (text: string) => string): string { const tail = stderrTailOf(error); if (tail === undefined) return ''; - return '. The last of its stderr:' + tail.split('\n').map((line) => `\n stderr: ${logDetail(line)}`).join(''); + return '. The last of its stderr:' + redact(tail).split('\n').map((line) => `\n stderr: ${logDetail(line)}`).join(''); } /** A transport close's reason, as the host's log and the handshake's error @@ -117,8 +124,113 @@ function closeReasonText(info: TransportCloseInfo): string { * tool work while still bounding the hang. */ const DEFAULT_REQUEST_TIMEOUT_MS = 60_000; -/** MCP protocol version used in the initialize handshake. */ -const MCP_PROTOCOL_VERSION = '2024-11-05'; +/** MCP protocol version used in the initialize handshake: the only revision + * this legacy engine offers or accepts. */ +const MCP_PROTOCOL_VERSION = LEGACY_MCP_PROTOCOL_VERSION; + +/** JSON-RPC code for an unsupported protocol version (MCP 2026-07-28 §basic, + * the code a modern-only server answers our `initialize` with). */ +const UNSUPPORTED_PROTOCOL_VERSION_CODE = -32022; + +/** Upper bound on `tools/list` pages, against a server whose cursors never + * end. Generous: real inventories fit in a handful of pages. */ +const MAX_TOOL_LIST_PAGES = 100; + +/** + * The server and this engine have no protocol revision in common, so the + * connection was never established. This is a protocol-negotiation fact, not + * a request outcome. Retrying cannot change it: each engine offers one + * revision, and only configuration can fix the mismatch. A connection that + * gets this error does not schedule reconnects. The legacy engine and the + * modern one (ModernMcpConnection, whose pinned revision the server refused) + * both throw it, so its readers treat them alike. + * + * - `rejected`: the server refused the offered revision: the legacy engine's + * `initialize` answered with error -32022, or a modern server's discovery + * or error naming no common revision. `supported` is the server's own list, + * when it gave one. + * - `mismatch`: the server answered the legacy engine's `initialize` with a + * revision other than 2024-11-05, or with none. + */ +export class McplProtocolVersionError extends Error { + constructor( + message: string, + readonly kind: 'rejected' | 'mismatch', + /** Revision the server answered with (`mismatch`; null when it gave none). */ + readonly returned: string | null, + /** Revisions the server says it supports (`rejected`, when stated). */ + readonly supported: readonly string[] | null, + /** The JSON-RPC error's code and data, verbatim (`rejected`). */ + readonly code?: number, + readonly data?: unknown, + /** The revision this client offered: the legacy engine's, unless a + * modern connection's. */ + readonly offered: string = MCP_PROTOCOL_VERSION, + ) { + super(message); + this.name = 'McplProtocolVersionError'; + } +} + +function protocolVersionRejection(id: string, error: { code: number; message: string; data?: unknown }): McplProtocolVersionError { + const data = error.data as { supported?: unknown } | undefined; + const supported = Array.isArray(data?.supported) && data.supported.every((v) => typeof v === 'string') + ? (data.supported as string[]) + : null; + const modernHint = supported?.includes(MODERN_MCP_PROTOCOL_VERSION) + ? ` It speaks modern MCP: configure it with protocol: 'modern' (stdio) or reach it over http(s).` + : ''; + return new McplProtocolVersionError( + `MCPL server "${id}" does not support MCP ${MCP_PROTOCOL_VERSION}` + + // The server's message can be any JSON value, so it's read without + // throwing (describeCause) and bounded, as the initialize error is. + (supported ? ` (it supports: ${supported.join(', ') || 'none listed'}).` : ` (${boundedForLog(describeCause(error.message))}).`) + + modernHint, + 'rejected', + null, + supported, + error.code, + error.data, + ); +} + +const errorOf = (value: unknown): Error => (value instanceof Error ? value : new Error(String(value))); + +/** + * A handshake that failed, and whose cleanup couldn't confirm that the child + * exited. The transport goes to whoever owns the launch. + */ +class HandshakeCleanupError extends Error { + constructor( + readonly failure: Error, + readonly cleanup: Error, + readonly transport: McplTransport, + ) { + super(`${failure.message}; its cleanup also failed: ${cleanup.message}`, { cause: failure }); + this.name = 'HandshakeCleanupError'; + } +} + +/** + * A connect that failed, and whose cleanup couldn't confirm that the child + * exited: it may still be running. `connection` owns that launch, closed and + * not reconnecting. Its close() retries the reap and resolves once the exit + * is confirmed, so the caller can still clean up, and should start nothing + * else for this server until then. connect() and connectWithReconnect() + * throw this, and the modern engine's ModernMcpConnection.connect() too. + * `cause` is the connect's own failure. + */ +export class McplUnreapedLaunchError } = McplServerConnection> extends Error { + constructor( + readonly connection: Connection, + failure: Error, + /** The cleanup's own failure: the child it couldn't confirm exited. */ + readonly cleanup: Error, + ) { + super(`${failure.message}; its cleanup also failed: ${cleanup.message}`, { cause: failure }); + this.name = 'McplUnreapedLaunchError'; + } +} /** * Why a request to an MCPL server produced no result, as far as the host can @@ -213,6 +325,10 @@ export class McplServerConnection extends EventEmitter { * the sole source of the `tools` capability path). */ mcpToolsAdvertised = false; + /** The MCP revision established by the last successful handshake (always + * 2024-11-05 for this engine), or null before the first one. */ + protocolVersion: string | null = null; + /** Host-owned per-server authority for host/command (config * allowHostCommands, default false). No capability path exists and the * grant cannot confer it — server params never decide admin authority. */ @@ -316,9 +432,12 @@ export class McplServerConnection extends EventEmitter { * a restart both do, so a request after it can't say whether the server * comes back. */ private closedByThisHost = false; + /** The host's close attempt: in flight, or done. A failed one is cleared, + * so the next close() tries again. */ private closePromise: Promise | null = null; + /** The host closed this connection while it was live, and its teardown + * hasn't completed yet: 'close' is emitted once it has. */ private closeIncomplete = false; - private reconnectPromise: Promise | null = null; /** Per-request timeout in ms (0 disables). See McplServerConfig.requestTimeoutMs. */ private requestTimeoutMs = DEFAULT_REQUEST_TIMEOUT_MS; @@ -362,13 +481,30 @@ export class McplServerConnection extends EventEmitter { /** Consecutive failed connect attempts since the last successful handshake. * Drives the exponential backoff and is reported on reconnect events. */ private reconnectAttempts = 0; + /** The reconnect attempt in flight, so close() can abort it and wait for + * its cleanup. */ + private reconnecting: { abort: AbortController; done: Promise } | null = null; + /** + * Launches whose cleanup couldn't confirm that the child exited. While any + * is held, reconnecting is halted, since another launch could run beside + * one that's still alive. close() retries their reap. + */ + private unreaped: Array<{ transport: McplTransport; failure: Error }> = []; /** Whether a background reconnect loop will revive this connection after a - * drop. False once close() has been called (explicit close stops retrying). */ + * drop. False once close() has been called (explicit close stops retrying), + * after a protocol-version verdict, and while a launch is unreaped. */ get willReconnect(): boolean { return this.reconnectEnabled; } + /** Whether a child this connection launched couldn't be confirmed exited. + * While it's true the connection owns that child, so whatever registered + * it keeps its id held; close() retries the reap. */ + get holdsUnreapedChild(): boolean { + return this.unreaped.length > 0; + } + /** Closed with no reconnect to come: after close(), whether or not its * transport close succeeded, or a drop with reconnect off. A connection * between retries keeps reconnectEnabled, so it isn't closed for good. */ @@ -404,16 +540,19 @@ export class McplServerConnection extends EventEmitter { * construction (when setupMessageRouting starts emitting) and now (when * the caller has attached listeners via wireMcplEvents). * - * Does nothing once the connection is closed for good. Its policy was reset - * when it closed, so what it held that needs a grant would be refused, and a - * closed connection sends no response, so nothing it held could be answered. - * Only what needs no grant would pass, to listeners of a server that won't - * be served again. A connection between retries isn't closed for good: what - * it held, such as a reconnect stub's connect-failed, still reaches its - * listeners. + * Once the connection is closed for good it readies nothing. Its policy was + * reset when it closed, so what the server sent that needs a grant would be + * refused, and a closed connection sends no response, so nothing it held + * could be answered. It delivers only its own reports of how it ended (see + * {@link deliverOwnReports}). A connection between retries isn't closed for + * good: what it held, such as a reconnect stub's connect-failed, reaches + * its listeners as it does any connection's. */ ready(): void { - if (this.closedForGood) return; + if (this.closedForGood) { + this.deliverOwnReports(); + return; + } this.controlPlaneReady = true; this.dataPlaneReady = true; const pending = this.bufferedEvents; @@ -443,10 +582,14 @@ export class McplServerConnection extends EventEmitter { * to host requests are never event-buffered, so tools/list and tools/call * responses remain live as well. Inference-bearing requests stay queued in * their original relative order until {@link ready} is called. Like - * {@link ready}, it does nothing once the connection is closed for good. + * {@link ready}, once the connection is closed for good it delivers only + * its own reports of how it ended. */ readyControlPlane(): void { - if (this.closedForGood) return; + if (this.closedForGood) { + this.deliverOwnReports(); + return; + } this.controlPlaneReady = true; const pending = this.bufferedEvents; this.bufferedEvents = []; @@ -461,6 +604,22 @@ export class McplServerConnection extends EventEmitter { } } + /** + * A connection closed for good still says how it ended. Its own buffered + * connect-failed and reconnect-failed, such as the permanent failure of a + * launch held as unreaped before the first gate released it, are the + * framework's records of it: they need no grant, and its listeners act on + * them (a trace, and for a permanent failure an alert and its tools leaving + * the list). What the server sent stays held. + */ + private deliverOwnReports(): void { + const own = (item: { event: string }) => item.event === 'connect-failed' || item.event === 'reconnect-failed'; + const reports = this.bufferedEvents.filter(own); + if (reports.length === 0) return; + this.bufferedEvents = this.bufferedEvents.filter((item) => !own(item)); + for (const { event, args } of reports) super.emit(event, ...args); + } + /** * Close the data plane synchronously while leaving an already-released * control plane live. Used before reconnect adoption and list-change @@ -641,18 +800,47 @@ export class McplServerConnection extends EventEmitter { * capability negotiation, and return a ready-to-use connection. * * Works over stdio (spawned child) or WebSocket, selected by `config` (see - * {@link openTransport}). Throws if the transport can't be opened, the server - * closes/errors before the handshake, or the handshake times out. + * {@link openTransport}). Throws if the configuration can't be used (before + * anything is spawned or dialed), the transport can't be opened, the server + * closes/errors before the handshake, or the handshake times out. If the + * failed launch's cleanup can't confirm its child exited, it throws + * {@link McplUnreapedLaunchError}, whose connection owns that launch. */ static async connect( config: McplServerConfig, hostCapabilities: McplHostCapabilities, + signal?: AbortSignal, + ): Promise { + McplServerConnection.checkConfig(config); + return McplServerConnection.connectChecked(config, hostCapabilities, signal); + } + + /** {@link connect}, for a configuration its caller has already checked + * (and warned about) once. `signal` ends the connect at once, its launch + * reaped as for any failed handshake: its caller's stop() or a + * disconnect of its id. */ + private static async connectChecked( + config: McplServerConfig, + hostCapabilities: McplHostCapabilities, + signal?: AbortSignal, ): Promise { - const { transport, capabilities, droppedCapabilities, mcpToolsAdvertised } = await McplServerConnection.handshake(config, hostCapabilities); + let established: Awaited>; + try { + established = await McplServerConnection.handshake(config, hostCapabilities, signal); + } catch (error) { + if (!(error instanceof HandshakeCleanupError)) throw error; + // The launch may still be running. A connection owns it, closed and + // halted, so the caller can retry the reap by closing it. + const owner = McplServerConnection.createDisconnectedStub(config, hostCapabilities); + owner.holdUnreaped(error.transport, error.cleanup); + throw new McplUnreapedLaunchError(owner, error.failure, error.cleanup); + } + const { transport, capabilities, droppedCapabilities, mcpToolsAdvertised, protocolVersion } = established; const connection = new McplServerConnection(config.id, capabilities, transport); connection.droppedCapabilities = droppedCapabilities; connection.mcpToolsAdvertised = mcpToolsAdvertised; + connection.protocolVersion = protocolVersion; connection.allowHostCommands = config.allowHostCommands === true; connection.requestTimeoutMs = config.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS; @@ -669,18 +857,49 @@ export class McplServerConnection extends EventEmitter { return connection; } + /** + * This engine's configuration check, made before anything is spawned or + * dialed: every problem {@link checkServerConfig} finds, the same check the + * framework applies, and the modern family, which belongs to + * ModernMcpConnection (an http(s) url is never dialed as a WebSocket). + * Every configuration that worked before still passes but the one + * serverConfigProblems names. + */ + private static checkConfig(config: McplServerConfig): void { + const { family } = checkServerConfig(config); + if (family === 'modern') { + throw new Error( + `MCP server "${config.id}" is configured for modern MCP (${MODERN_MCP_PROTOCOL_VERSION}); ` + + `McplServerConnection speaks the legacy family (${MCP_PROTOCOL_VERSION} + MCPL) only`, + ); + } + } + /** * Open a fresh transport and run the MCP/MCPL initialize handshake over it, * returning the connected transport plus the negotiated capabilities. Shared * by {@link connect} and {@link attemptReconnect} so both transports handshake * identically. On any failure the transport is closed so a spawned child / - * open socket never leaks. + * open socket never leaks. If that close can't confirm the child exited, + * the failure is a {@link HandshakeCleanupError} carrying the transport, so + * the caller keeps owning it. `signal` ends a handshake whose connection is + * closed meanwhile: it cuts a WebSocket dial short, is checked once the + * transport is open, and races the initialize answer. */ private static async handshake( config: McplServerConfig, hostCapabilities: McplHostCapabilities, - ): Promise<{ transport: McplTransport; capabilities: McplCapabilities | null; droppedCapabilities: ReadonlySet; mcpToolsAdvertised: boolean }> { - const transport = await openTransport(config); + signal?: AbortSignal, + ): Promise<{ transport: McplTransport; capabilities: McplCapabilities | null; droppedCapabilities: ReadonlySet; mcpToolsAdvertised: boolean; protocolVersion: string }> { + const closedWhileConnecting = () => new Error(`MCPL server "${config.id}" was closed while connecting`); + if (signal?.aborted) throw closedWhileConnecting(); + let transport: McplTransport; + try { + transport = await openTransport(config, signal); + } catch (error) { + if (signal?.aborted) throw closedWhileConnecting(); + throw error; + } // Once the handshake stops listening for 'error', this logs what the // transport reports, until the handshake hands the transport on, or for // good if the handshake fails. EventEmitter throws an 'error' that has no @@ -688,10 +907,11 @@ export class McplServerConnection extends EventEmitter { // WebSocket frame that arrives with the initialize response, or while a // failed handshake's close() runs, is such an error. const logAfterSettled = (err: Error) => { - console.error(`[mcpl] ${config.id}: transport error after the handshake settled: ${err.message}`); + console.error(`[mcpl] ${config.id}: transport error after the handshake settled: ${redactServerText(err.message, config)}`); }; try { + if (signal?.aborted) throw closedWhileConnecting(); const initId = 0; // use id=0 for the handshake const initRequest: JsonRpcRequest = { jsonrpc: '2.0', @@ -704,9 +924,9 @@ export class McplServerConnection extends EventEmitter { }, }; - // Await the initialize response, racing an early transport close/error and - // a timeout. All three paths clean up their listeners. - const { mcpl: rawCapabilities, mcpToolsAdvertised } = await new Promise<{ mcpl: McplCapabilities | null; mcpToolsAdvertised: boolean }>((resolve, reject) => { + // Await the initialize response, racing an early transport close/error, + // a timeout and the owner's close. Every path cleans up its listeners. + const { mcpl: rawCapabilities, mcpToolsAdvertised, protocolVersion } = await new Promise<{ mcpl: McplCapabilities | null; mcpToolsAdvertised: boolean; protocolVersion: string }>((resolve, reject) => { // When the transport fails, the end of the server's stderr often says // why, such as a missing token or a crash at startup. It goes on the // error as `stderrTail`, never into the message: an error's message @@ -720,9 +940,14 @@ export class McplServerConnection extends EventEmitter { transport.off('line', onLine); transport.off('close', onClose); transport.off('error', onError); + signal?.removeEventListener('abort', onAbort); transport.on('error', logAfterSettled); clearTimeout(timer); }; + const onAbort = () => { + cleanup(); + reject(closedWhileConnecting()); + }; const onLine = (line: string) => { let msg: JsonRpcResponse; try { @@ -753,10 +978,26 @@ export class McplServerConnection extends EventEmitter { // exception, which ends a host that doesn't catch them, and a // handshake left to wait forever. It's the server's text in an // error's message, so it's bounded as errors.ts says. - reject(new Error(`MCPL server "${config.id}" initialize error: ${boundedForLog(describeCause(msg.error.message))}`)); + reject(msg.error.code === UNSUPPORTED_PROTOCOL_VERSION_CODE + ? protocolVersionRejection(config.id, msg.error) + : new Error(`MCPL server "${config.id}" initialize error: ${boundedForLog(describeCause(msg.error.message))}`)); return; } const result = msg.result as Record | undefined; + // The server's answer is the established revision. This engine + // implements exactly one, so any other answer (or none) means the + // two sides never agreed on what they're speaking. + const returned = typeof result?.protocolVersion === 'string' ? result.protocolVersion : null; + if (returned !== MCP_PROTOCOL_VERSION) { + reject(new McplProtocolVersionError( + `MCPL server "${config.id}" answered initialize with protocolVersion ` + + `${returned === null ? '(none)' : JSON.stringify(returned)}; this client's legacy engine speaks only ${MCP_PROTOCOL_VERSION}`, + 'mismatch', + returned, + null, + )); + return; + } const caps = result?.capabilities as Record | undefined; const experimental = caps?.experimental as Record | undefined; // §5.1: `tools` capability is the OUTER standard MCP member — the @@ -765,6 +1006,7 @@ export class McplServerConnection extends EventEmitter { resolve({ mcpl: (experimental?.mcpl as McplCapabilities) ?? null, mcpToolsAdvertised: caps?.tools !== undefined, + protocolVersion: returned, }); }; const onClose = (info: TransportCloseInfo) => { @@ -787,6 +1029,7 @@ export class McplServerConnection extends EventEmitter { reject(failed(`MCPL server "${config.id}" initialize handshake timed out`)); }, INITIALIZE_TIMEOUT_MS); + signal?.addEventListener('abort', onAbort, { once: true }); transport.on('line', onLine); transport.on('close', onClose); transport.on('error', onError); @@ -813,10 +1056,16 @@ export class McplServerConnection extends EventEmitter { // listener of its own, in the continuation of this return, before any // I/O callback can emit. transport.off('error', logAfterSettled); - return { transport, capabilities: scoped, droppedCapabilities: new Set(dropped), mcpToolsAdvertised }; + return { transport, capabilities: scoped, droppedCapabilities: new Set(dropped), mcpToolsAdvertised, protocolVersion }; } catch (err) { - // Never leak the child / socket if the handshake fails. - await transport.close().catch(() => { /* best effort */ }); + // Never leak the child / socket if the handshake fails. A cleanup that + // can't confirm the child exited is part of the failure, never + // swallowed: the launch may still be running, so the caller keeps it. + try { + await transport.close(); + } catch (cleanup) { + throw new HandshakeCleanupError(errorOf(err), errorOf(cleanup), transport); + } throw err; } } @@ -825,34 +1074,48 @@ export class McplServerConnection extends EventEmitter { * Connect with reconnect support. * When `config.reconnect` is true and the initial connection fails, * resolves immediately with null capabilities and retries in the background. + * A configuration this engine can't use, or a protocol-version verdict, + * still throws: retrying can't change either. So does a failed launch that + * can't be reaped, as in {@link connect}: no retry starts beside a child + * that may still be running, and the {@link McplUnreapedLaunchError} + * carries the connection that owns it, whose close() retries the reap. * Adapted from Anarchid/agent-framework@mcpl-module-proto. */ static async connectWithReconnect( config: McplServerConfig, hostCapabilities: McplHostCapabilities, + signal?: AbortSignal, ): Promise { + // Outside the retry path below: a configuration error is never retried. + McplServerConnection.checkConfig(config); try { - return await McplServerConnection.connect(config, hostCapabilities); + return await McplServerConnection.connectChecked(config, hostCapabilities, signal); } catch (error) { - if (config.reconnect !== true) { + // No common protocol revision is a configuration fact, not an outage: + // a retry stub would restart the server forever for nothing. A connect + // its caller ended (stop(), or a disconnect of its id) isn't retried + // either, nor is a launch that couldn't be reaped. + if (config.reconnect !== true || error instanceof McplProtocolVersionError || error instanceof McplUnreapedLaunchError || signal?.aborted) { throw error; } - // Non-blocking start: create a disconnected stub that will reconnect in background - console.error(`MCPL server "${config.id}" initial connect failed, will retry: ${(error as Error).message}${withStderr(error)}`); - const stub = McplServerConnection.createDisconnectedStub(config, hostCapabilities); // Surface the initial failure. The event is buffered by the emit() // override until the host wires listeners and calls ready(), so it is // not lost in the window before wireMcplEvents runs. + // Non-blocking start: create a disconnected stub that will reconnect in background + console.error(`MCPL server "${config.id}" initial connect failed, will retry: ${redactServerText((error as Error).message, config)}${withStderr(error, (text) => redactServerText(text, config))}`); + const stub = McplServerConnection.createDisconnectedStub(config, hostCapabilities); const stderrTail = stderrTailOf(error); stub.emit('connect-failed', { error: (error as Error).message, ...(stderrTail ? { stderrTail } : {}), attempt: 0 }); + stub.scheduleReconnect(); return stub; } } /** - * Create a disconnected stub connection that will reconnect in the background. - * Used when initial connect fails and reconnect is enabled. + * Create a disconnected stub connection, for a failed initial connect: one + * that will reconnect in the background (its caller schedules that), or one + * that owns a launch that couldn't be reaped. * @internal */ private static createDisconnectedStub( @@ -865,15 +1128,31 @@ export class McplServerConnection extends EventEmitter { stub.requestTimeoutMs = config.requestTimeoutMs ?? DEFAULT_REQUEST_TIMEOUT_MS; stub.config = config; stub.hostCapabilities = hostCapabilities; - stub.reconnectEnabled = true; + stub.reconnectEnabled = config.reconnect === true; stub.reconnectIntervalMs = config.reconnectIntervalMs ?? 5000; stub.reconnectMaxIntervalMs = config.reconnectMaxIntervalMs ?? 300_000; stub.reconnectAttempts = 1; // the initial connect already failed + return stub; + } - // Schedule background reconnect - stub.scheduleReconnect(); + /** + * Keep a launch whose cleanup couldn't confirm its child exited, and halt + * reconnecting: another launch could run beside one that's still alive. + * close() retries the reap. + */ + private holdUnreaped(transport: McplTransport, failure: Error): void { + this.reconnectEnabled = false; + if (this.reconnectTimer) { + clearTimeout(this.reconnectTimer); + this.reconnectTimer = null; + } + if (!this.unreaped.some((held) => held.transport === transport)) this.unreaped.push({ transport, failure }); + } - return stub; + /** The halt, as a reconnect event's failure text. */ + private unreapedFailure(): string { + const held = this.unreaped[0]; + return `the previous launch could not be reaped (${held?.failure.message ?? 'unknown'}); reconnecting is halted`; } // ========================================================================== @@ -1027,9 +1306,46 @@ export class McplServerConnection extends EventEmitter { // Standard MCP methods // ========================================================================== - /** Send `tools/list` and return the server's tool definitions. */ - sendToolsList(): Promise<{ tools: McpToolDefinition[] }> { - return this.sendRequest('tools/list', {}) as Promise<{ tools: McpToolDefinition[] }>; + /** + * Send `tools/list` and return the server's complete tool inventory, + * following `nextCursor` across pages (2024-11-05 pagination). Cursors are + * opaque strings, `''` included: only an absent or null `nextCursor` ends + * the inventory (a null one is how some servers say "no more", and main + * read it as a complete page). A page that isn't well formed (no `tools` array, a cursor + * that isn't a string), or cursors that don't terminate (a repeat, or more + * than {@link MAX_TOOL_LIST_PAGES} pages), are errors, never a partial or + * empty inventory published as complete. + */ + async sendToolsList(): Promise<{ tools: McpToolDefinition[] }> { + const tools: McpToolDefinition[] = []; + const seen = new Set(); + let cursor: string | undefined; + let first: Record | undefined; + for (let page = 0; page < MAX_TOOL_LIST_PAGES; page++) { + const result = await this.sendRequest('tools/list', cursor === undefined ? {} : { cursor }) as Record | null; + const malformed = (why: string) => + new Error(`MCPL server "${this.id}" sent a malformed tools/list page ${page + 1}: ${why}`); + if (!result || typeof result !== 'object' || !Array.isArray(result.tools)) { + throw malformed('no tools array'); + } + first ??= result; + tools.push(...(result.tools as McpToolDefinition[])); + if (result.nextCursor === undefined || result.nextCursor === null) { + // The first page's other fields are kept as they came, so a + // single-page answer is returned exactly as before. + const { nextCursor: _done, ...rest } = first as { nextCursor?: unknown }; + return { ...rest, tools }; + } + if (typeof result.nextCursor !== 'string') { + throw malformed(`nextCursor must be a string, got ${JSON.stringify(result.nextCursor)}`); + } + if (seen.has(result.nextCursor)) { + throw new Error(`MCPL server "${this.id}" repeated tools/list cursor ${JSON.stringify(result.nextCursor)}`); + } + seen.add(result.nextCursor); + cursor = result.nextCursor; + } + throw new Error(`MCPL server "${this.id}" tools/list did not finish within ${MAX_TOOL_LIST_PAGES} pages`); } /** Send `tools/call` and return the result. Optionally includes state/checkpoint for stateful tools. */ @@ -1075,8 +1391,22 @@ export class McplServerConnection extends EventEmitter { return `${this.pendingRequests.size} request(s) unanswered: ${list}`; } - /** Close the connection: disable reconnect, kill the child process, and clean up resources. */ - async close(): Promise { + /** + * Close the connection for good: disable reconnect, kill the child + * process, and clean up resources. At once, as before, reconnecting stops + * and pending requests fail; a reconnect handshake in flight is aborted. + * + * Calls share one attempt: while it is pending, and once it has + * succeeded, close() returns it. The attempt waits for the aborted + * reconnect's own cleanup, closes the transport (its close is the reaping + * verdict) and retries each launch an earlier cleanup couldn't reap. It + * rejects while any child can't be confirmed exited, with that failure + * itself, or an AggregateError when there are several. A failed attempt is + * cleared, so the next close() retries what it couldn't reap. 'close' is + * emitted once, when the teardown of a connection the host closed while + * live completes. + */ + close(): Promise { // Disable reconnect before closing — explicit close means stop retrying this.reconnectEnabled = false; this.closedByThisHost = true; @@ -1084,46 +1414,36 @@ export class McplServerConnection extends EventEmitter { clearTimeout(this.reconnectTimer); this.reconnectTimer = null; } - + this.reconnecting?.abort.abort(); if (this.closePromise) return this.closePromise; - if (this.closed && !this.closeIncomplete && !this.reconnectPromise) return; - this.closed = true; - this.closeIncomplete = true; - this.resetPolicyForTransportBoundary(); - // Reject all pending requests. Each message is the whole of what an - // agent reads for a tools/call, and what follows a channel tool's own - // words ("Failed to publish to channel: …") for the requests a channel - // tool makes, so it names no method; the log line does. It doesn't say - // whether this host reconnects: a restart closes the connection and then - // connects again. - if (this.pendingRequests.size > 0) { - console.error(`MCPL server "${this.id}": this host closed its connection with ${this.unansweredRequests()}`); - } - for (const pending of this.pendingRequests.values()) { - if (pending.timer) clearTimeout(pending.timer); - pending.reject(new McplRequestError( - `The "${this.id}" server didn't answer before this host closed its connection, so whether the request took effect is unknown. ` + - 'Check its effect before trying again.', - 'no-response', - )); + if (!this.closed) { + this.closed = true; + this.closeIncomplete = true; + this.resetPolicyForTransportBoundary(); + + // Reject all pending requests. Each message is the whole of what an + // agent reads for a tools/call, and what follows a channel tool's own + // words ("Failed to publish to channel: …") for the requests a channel + // tool makes, so it names no method; the log line does. It doesn't say + // whether this host reconnects: a restart closes the connection and then + // connects again. + if (this.pendingRequests.size > 0) { + console.error(`MCPL server "${this.id}": this host closed its connection with ${this.unansweredRequests()}`); + } + for (const pending of this.pendingRequests.values()) { + if (pending.timer) clearTimeout(pending.timer); + pending.reject(new McplRequestError( + `The "${this.id}" server didn't answer before this host closed its connection, so whether the request took effect is unknown. ` + + 'Check its effect before trying again.', + 'no-response', + )); + } + this.pendingRequests.clear(); + this.orphanedRequests.clear(); } - this.pendingRequests.clear(); - this.orphanedRequests.clear(); - - // Tear down the transport (kills the child / closes the socket). The - // transport's own 'close' event is short-circuited by `this.closed` in - // setupLifecycle, so this is the single source of the connection 'close'. - const attempt = Promise.resolve().then(async () => { - // An already opened handshake owns a transport too. It hands that - // transport back without admission once reconnect has been disabled. - // Only the registry's closeAll bounds its caller's wait for this; retry - // reuses this real close attempt. - if (this.reconnectPromise) await this.reconnectPromise; - if (this.transport) await this.transport.close(); - this.closeIncomplete = false; - this.emit('close', null, null, this.closeInfoByHost()); - }); + + const attempt = this.closeStep(this.transport, this.unreaped.map((held) => held.transport)); this.closePromise = attempt; void attempt.catch(() => { if (this.closePromise === attempt) this.closePromise = null; @@ -1131,14 +1451,57 @@ export class McplServerConnection extends EventEmitter { return attempt; } + /** One close attempt. `heldBefore` are the launches earlier cleanups + * couldn't reap, which it retries. */ + private async closeStep(transport: McplTransport | null, heldBefore: McplTransport[]): Promise { + // An aborted reconnect cleans up its own launch, holding it if it can't. + // That cleanup was this attempt's try at it, so its failure is this + // attempt's, not retried here. + await this.reconnecting?.done; + const heldMeanwhile = this.unreaped.filter((held) => !heldBefore.includes(held.transport)); + const failures: Error[] = heldMeanwhile.map((held) => held.failure); + const targets: McplTransport[] = []; + for (const target of [transport, ...heldBefore]) { + if (target && !targets.includes(target) && !heldMeanwhile.some((held) => held.transport === target)) targets.push(target); + } + for (const target of targets) { + try { + // Kills the child / closes the socket. + await target.close(); + this.unreaped = this.unreaped.filter((held) => held.transport !== target); + } catch (error) { + const failure = errorOf(error); + failures.push(failure); + if (!this.unreaped.some((held) => held.transport === target)) this.unreaped.push({ transport: target, failure }); + } + } + if (failures.length === 1) throw failures[0]; + if (failures.length > 1) { + throw new AggregateError(failures, `MCPL server "${this.id}" did not close cleanly: ${failures.map((f) => f.message).join('; ')}`); + } + // The transport's own 'close' event is short-circuited by `this.closed` + // in setupLifecycle, so this is the single source of the connection + // 'close' when the host tears down a live transport. + if (this.closeIncomplete) { + this.closeIncomplete = false; + this.emit('close', null, null, this.closeInfoByHost()); + } + } + /** * Tear down a live transport after an awareness-accounting failure without * disabling configured auto-reconnect. The normal transport close handler * rejects requests, emits lifecycle state, and schedules the next bounded - * reconnect attempt. With reconnect disabled this leaves the server closed. + * reconnect attempt. With reconnect disabled the recycle ends the + * connection for good, as close() does: 'close' comes once the child's + * exit is confirmed, and a child it can't reap stays owned, holding the id. */ async reconnectAfterFailure(): Promise { this.pauseDataPlane(); + if (!this.reconnectEnabled) { + await this.close(); + return; + } if (this.closed) { this.scheduleReconnect(); return; @@ -1192,73 +1555,113 @@ export class McplServerConnection extends EventEmitter { * Attempt to reconnect by opening a fresh transport and performing the * handshake, then adopting that transport in place. Unlike the previous * approach this builds no throwaway connection instance, so there are no - * leaked listeners on a dead object. + * leaked listeners on a dead object. The attempt is close()'s to abort, + * and it settles only once its own launch is adopted, failed or cleaned up. + * One attempt at a time: a request that arrives while one is in flight (a + * recycle, say) starts nothing, since that attempt either adopts its + * launch or schedules the next retry itself. A second attempt would take + * close()'s ownership of the first (Iris-1827, room-293 #55410). */ - private attemptReconnect(): Promise { - if (this.reconnectPromise) return this.reconnectPromise; - const attempt = this.reconnectOnce(); - this.reconnectPromise = attempt; - const clear = () => { if (this.reconnectPromise === attempt) this.reconnectPromise = null; }; - void attempt.then(clear, clear); - return attempt; + private async attemptReconnect(): Promise { + // A queued recovery timer may outlive an earlier successful handshake: + // only a connection that is down reconnects, so it never replaces a live + // transport. + if (this.reconnecting || !this.closed || !this.reconnectEnabled || !this.config || !this.hostCapabilities) return; + const abort = new AbortController(); + const done = this.reconnectOnce(this.config, this.hostCapabilities, abort.signal); + this.reconnecting = { abort, done }; + await done; + if (this.reconnecting?.done === done) this.reconnecting = null; } - private async reconnectOnce(): Promise { - // A queued recovery timer may outlive an earlier successful handshake. - // Recheck disconnection before opening or replacing any owned transport. - if (!this.closed || !this.reconnectEnabled || !this.config || !this.hostCapabilities) return; - + /** One reconnect attempt. It never rejects: each outcome is an event, a + * scheduled retry, or a launch held as unreaped. */ + private async reconnectOnce( + config: McplServerConfig, + hostCapabilities: McplHostCapabilities, + signal: AbortSignal, + ): Promise { // Ordinal of this attempt: 0 was the initial connect, so the Nth retry // reports attempt N (reconnectAttempts counts failures so far). const attempt = Math.max(1, this.reconnectAttempts); - try { - const { transport, capabilities, droppedCapabilities, mcpToolsAdvertised } = await McplServerConnection.handshake( - this.config, - this.hostCapabilities, - ); - - if (!this.reconnectEnabled) { - // The prior transport was disconnected before this retry began. Keep - // the fresh, unadmitted transport owned by close(), including on a - // failed transport cleanup, instead of reviving a stopped connection. - // It isn't wired, and the handshake has removed its own listeners, so - // an error it reports from here on, such as a malformed frame from a - // WebSocket server while close() closes it, is logged here rather than - // thrown where nothing catches it. - transport.on('error', (err: Error) => { - console.error(`[mcpl] ${this.id}: a transport kept for close() reported an error: ${err.message}`); - }); - this.transport = transport; + // The previous launch must be gone before another starts. A transport + // the host closed itself (reconnectAfterFailure) may still be reaping its + // child; one whose child already exited resolves at once. + const previous = this.transport; + if (previous) { + try { + await previous.close(); + } catch (error) { + this.holdUnreaped(previous, errorOf(error)); + if (!signal.aborted) this.emit('reconnect-failed', { error: this.unreapedFailure(), attempt, permanent: true }); return; } + } + if (signal.aborted) return; - // Close the data plane before the new transport can emit. The reconnect - // lifecycle event bypasses buffering; its framework listener installs the - // awareness barrier synchronously, then releases control traffic needed - // to establish the server while data remains held. - this.pauseDataPlane(); - this.resetPolicyForTransportBoundary(); - this.transport = transport; - this.capabilities = capabilities; - this.droppedCapabilities = droppedCapabilities; - this.mcpToolsAdvertised = mcpToolsAdvertised; - this.allowHostCommands = this.config?.allowHostCommands === true; - this.closed = false; - this.nextRequestId = 1; - this.pendingRequests.clear(); - this.wireTransport(transport); - - console.error(`MCPL server "${this.id}" reconnected successfully`); - this.reconnectAttempts = 0; - this.emit('reconnect', { attempts: attempt }); + let established: Awaited>; + try { + established = await McplServerConnection.handshake(config, hostCapabilities, signal); } catch (error) { - console.error(`MCPL server "${this.id}" reconnect failed: ${(error as Error).message}${withStderr(error)}`); + const unreaped = error instanceof HandshakeCleanupError; + if (unreaped) this.holdUnreaped(error.transport, error.cleanup); + // Closed by the host meanwhile: not a failed attempt. + if (signal.aborted) return; + const failure = unreaped ? error.failure : errorOf(error); + const redact = (text: string) => redactServerText(text, this.config ?? {}); + console.error(`MCPL server "${this.id}" reconnect failed: ${redact(failure.message)}${withStderr(failure, redact)}`); this.reconnectAttempts = attempt + 1; - const stderrTail = stderrTailOf(error); - this.emit('reconnect-failed', { error: (error as Error).message, ...(stderrTail ? { stderrTail } : {}), attempt }); - this.scheduleReconnect(); + // No common protocol revision: every further attempt would restart the + // server to hear the same answer. Stop retrying; the configuration is + // what has to change. A launch that couldn't be reaped halts retrying + // for good: close() can confirm its exit, but close() is terminal, so + // nothing resumes this connection. Every other failure keeps the + // backoff loop. + if (error instanceof McplProtocolVersionError) this.reconnectEnabled = false; + const permanent = unreaped || error instanceof McplProtocolVersionError; + const message = unreaped ? `${failure.message}; ${this.unreapedFailure()}` : failure.message; + const stderrTail = stderrTailOf(failure); + this.emit('reconnect-failed', { error: message, ...(stderrTail ? { stderrTail } : {}), attempt, ...(permanent ? { permanent: true } : {}) }); + if (!permanent) this.scheduleReconnect(); + return; + } + const { transport, capabilities, droppedCapabilities, mcpToolsAdvertised, protocolVersion } = established; + + if (signal.aborted) { + // Closed by the host while the handshake finished: this launch never + // serves. close() is waiting on this attempt, so it hears the verdict. + // The transport isn't wired, and the handshake has taken its own + // listeners off, so an error it reports while it closes, such as a + // malformed frame from a WebSocket server, is logged here rather than + // thrown where nothing catches it. + transport.on('error', (err: Error) => { + console.error(`[mcpl] ${this.id}: a transport closed before it served reported an error: ${redactServerText(err.message, config)}`); + }); + await transport.close().catch((error: unknown) => this.holdUnreaped(transport, errorOf(error))); + return; } + + // Close the data plane before the new transport can emit. The reconnect + // lifecycle event bypasses buffering; its framework listener installs the + // awareness barrier synchronously, then releases control traffic needed + // to establish the server while data remains held. + this.pauseDataPlane(); + this.resetPolicyForTransportBoundary(); + this.transport = transport; + this.capabilities = capabilities; + this.droppedCapabilities = droppedCapabilities; + this.mcpToolsAdvertised = mcpToolsAdvertised; + this.protocolVersion = protocolVersion; + this.allowHostCommands = this.config?.allowHostCommands === true; + this.closed = false; + this.nextRequestId = 1; + this.pendingRequests.clear(); + this.wireTransport(transport); + + console.error(`MCPL server "${this.id}" reconnected successfully`); + this.reconnectAttempts = 0; + this.emit('reconnect', { attempts: attempt }); } // ========================================================================== @@ -1296,6 +1699,21 @@ export class McplServerConnection extends EventEmitter { 'not-sent', )); } + // Once the child's stdin or stdout has failed, the transport is ending + // it and writes nothing more to it, so a request is refused here as not + // sent rather than left to wait out its timeout. The method and the + // pipe's error are the operator's, so they go to the log. + const pipeFailure = this.transport?.pipeFailure; + if (pipeFailure) { + console.error(`MCPL server "${this.id}": ${method} not sent, since its stdio pipe failed: ${logDetail(pipeFailure.message)}`); + return Promise.reject(new McplRequestError( + `The request wasn't sent: this host can no longer write to the "${this.id}" server, so nothing ran. ` + + (this.willReconnect + ? 'This host is reconnecting to it; try again later.' + : "This host won't reconnect to it on its own. If you need it, ask whoever runs this host."), + 'not-sent', + )); + } const id = this.nextRequestId++; const request: JsonRpcRequest = { @@ -1728,7 +2146,7 @@ export class McplServerConnection extends EventEmitter { if (!this.emitToErrorListeners(error)) console.error(`[mcpl] ${this.id}: transport error: ${logDetail(err.message)}`); }); - transport.on('close', (info: TransportCloseInfo) => { + const lost = (info: TransportCloseInfo) => { if (!this.closed) { this.closed = true; this.resetPolicyForTransportBoundary(); @@ -1780,6 +2198,21 @@ export class McplServerConnection extends EventEmitter { this.scheduleReconnect(); } } + }; + transport.on('close', lost); + // A stdio child whose stdin or stdout failed, and that outlived SIGKILL + // too: the transport stays open, owning the child. The connection holds + // it as a launch it can't reap, as a reconnect does, which halts + // reconnecting, and is lost as on a close, with the permanent failure + // that raises the alert and takes its tools off the list. Its id stays + // held until close() confirms the exit. A close the host has begun + // reports the failure itself. During a handshake nothing listens yet: + // the handshake's own cleanup holds the launch, once the handshake fails. + transport.on('unreaped', (failure: Error) => { + if (this.closed || this.transport !== transport) return; + this.holdUnreaped(transport, failure); + lost({ reason: `its stdio pipe failed with ${transport.pipeFailure?.message ?? 'an error'}, and its child outlived SIGKILL` }); + this.emit('reconnect-failed', { error: this.unreapedFailure(), attempt: Math.max(1, this.reconnectAttempts), permanent: true }); }); } } diff --git a/src/mcpl/server-registry.ts b/src/mcpl/server-registry.ts index f4dd246f7..7c22fc1ef 100644 --- a/src/mcpl/server-registry.ts +++ b/src/mcpl/server-registry.ts @@ -5,7 +5,7 @@ */ import type { McplServerConfig, McplHostCapabilities } from './types.js'; -import { McplServerConnection } from './server-connection.js'; +import { McplServerConnection, McplUnreapedLaunchError } from './server-connection.js'; import { DEFAULT_SHUTDOWN_TIMEOUT_MS, validateShutdownTimeout, waitForShutdown } from '../shutdown.js'; /** @@ -26,32 +26,56 @@ export type McplCapabilityQuery = */ export class McplServerRegistry { private servers = new Map(); + /** Ids whose addServer is still connecting. A second add for one is + * refused, so a failed launch's closed connection can always be registered + * as the owner of a child it couldn't reap. */ + private adding = new Set(); private closing = new Map>(); /** * Connect to an MCPL server and register it. * * Throws if a server with the same id is already registered, or if - * the connection/handshake fails. + * the connection/handshake fails. A failed launch whose child couldn't be + * confirmed exited stays registered all the same, as the closed connection + * that owns it ({@link McplUnreapedLaunchError}), so the id stays held + * and removeServer can retry the reap. */ async addServer( config: McplServerConfig, hostCapabilities: McplHostCapabilities, + signal?: AbortSignal, ): Promise { - if (this.servers.has(config.id)) { + if (this.servers.has(config.id) || this.adding.has(config.id)) { throw new Error(`MCPL server "${config.id}" is already registered`); } - const connection = config.reconnect === true - ? await McplServerConnection.connectWithReconnect(config, hostCapabilities) - : await McplServerConnection.connect(config, hostCapabilities); + this.adding.add(config.id); + let connection: McplServerConnection; + try { + connection = config.reconnect === true + ? await McplServerConnection.connectWithReconnect(config, hostCapabilities, signal) + : await McplServerConnection.connect(config, hostCapabilities, signal); + } catch (error) { + if (error instanceof McplUnreapedLaunchError) this.register(config, error.connection); + throw error; + } finally { + this.adding.delete(config.id); + } + this.register(config, connection); + return connection; + } + + private register(config: McplServerConfig, connection: McplServerConnection): void { this.servers.set(config.id, connection); // Auto-remove on close unless the connection will reconnect. That was // decided once, from `reconnect === true` at connect, and a close() the // host asks for disarms it first, so a server closed for good goes too. + // A connection that still owns a child it couldn't reap keeps its id, so + // nothing launches beside that child; removeServer retries the reap. connection.on('close', () => { - if (!connection.willReconnect) { + if (!connection.willReconnect && this.servers.get(config.id) === connection && !connection.holdsUnreapedChild) { this.servers.delete(config.id); } }); @@ -60,20 +84,22 @@ export class McplServerRegistry { connection.on('reconnect', () => { // Connection already in the map — capabilities may have changed }); - - return connection; } /** * Disconnect and remove a server by id. * - * No-op if the server is not registered. + * No-op if the server is not registered. If its close can't confirm the + * child exited, it stays registered and the failure is thrown: removing it + * again retries the reap. */ async removeServer(id: string): Promise { const connection = this.servers.get(id); if (!connection) { return; } + // The id stays held until the close settles, so nothing launches beside + // a child still being reaped; it's freed only once the exit is confirmed. await this.closeServer(id, connection); } @@ -138,8 +164,11 @@ export class McplServerRegistry { } /** - * Close all server connections, retaining failed or pending attempts. - * An explicit timeout bounds the caller's wait without cancelling cleanup. + * Close every server connection. Each one whose close confirms its exit + * is removed. One whose close fails or is still pending stays registered, + * holding its id, and every failure is thrown; removeServer or a later + * closeAll retries it. An explicit timeout bounds the caller's wait + * without cancelling cleanup. */ async closeAll(timeoutMs = DEFAULT_SHUTDOWN_TIMEOUT_MS): Promise { validateShutdownTimeout(timeoutMs); diff --git a/src/mcpl/tool-result-normalize.ts b/src/mcpl/tool-result-normalize.ts new file mode 100644 index 000000000..e3acd4710 --- /dev/null +++ b/src/mcpl/tool-result-normalize.ts @@ -0,0 +1,294 @@ +/** + * What a standard MCP tool result becomes for the framework. Standard means + * the server did not negotiate MCPL: a legacy MCP-only peer or a modern + * server. MCPL peers keep their RFC-005 reading (flat `resource` and + * uri-form media as references), which is not this module's job. + * + * An MCP result has two views, and each consumer gets its own: + * - **The content array is the server's model-facing view.** Text stays text, + * and an inline image stays native when a provider can take it: one of the + * types providers accept, within their per-image size. Anything else the + * provider format can't carry (audio, binary blobs, other images) is saved + * to the workspace, and a bounded stub says where it went. A stub tells the + * reader where the payload is; nothing is dropped silently. If a payload + * can't be saved, the result is a failure to materialize it, not a + * success. A `resource_link` is shown as a link, with its name and URI, + * and never fetched. An embedded resource shows its text, or is saved like + * a blob. + * - **`structuredContent` is the machine-readable view.** It is kept by + * presence on `ToolResult.structured`. The model sees it rendered as JSON + * only when the server's content has no text block of its own, because + * otherwise the server already chose what the model reads. + * + * A failed result is read by the model only as its error text, so there an + * inline image is saved and stubbed too, like any other payload. + * + * This is the model's reading. The direct path returns the server's raw + * content instead. A script receives the server's result whole only when it + * carries structuredContent; an unstructured one reaches a script as this + * reading, its payloads saved to the workspace and stubbed (framework.ts). + */ +import type { ToolResult } from '../types/index.js'; + +/** What a result whose content is present but isn't a list of blocks reads + * as, on the model's path and a script's alike. */ +export const MALFORMED_CONTENT_NOTICE = + '[malformed result: its content is not a list of blocks; the tool may already have completed]'; + +/** Saves a payload under a host-chosen name; resolves to its workspace path, + * or null when there is nowhere writable to put it. */ +export type SaveToolPayload = (fileName: string, bytes: Buffer, mimeType: string) => Promise; + +/** Stub fields are bounded independently of anything the server sends. */ +const STUB_FIELD_POINTS = 200; +/** A URI is the reference itself, so it's bounded far more loosely than a + * stub field: a real one survives, and a pathological one can't flood the + * model's view. */ +const URI_POINTS = 2048; +/** The image types providers take inline. Any other image is saved. */ +const NATIVE_IMAGE_TYPES = new Set(['image/png', 'image/jpeg', 'image/gif', 'image/webp']); +/** The decoded size providers take per inline image: the same 5 MB ceiling + * the framework's read_image tool guards. A larger image is saved. */ +const NATIVE_IMAGE_MAX_BYTES = 5 * 1024 * 1024; + +/** Why a payload isn't shown inline, unless said otherwise. */ +const UNSUPPORTED = "the model can't take this type"; + +type Block = { type: 'text'; text: string } | { type: 'image'; data: string; mimeType: string }; + +/** The first `max` code points of `text`, never splitting a surrogate pair, + * and how many code points were left out. */ +function headPoints(text: string, max: number): { head: string; omitted: number } { + let end = 0; + let points = 0; + while (end < text.length && points < max) { + end += text.codePointAt(end)! > 0xffff ? 2 : 1; + points++; + } + let omitted = 0; + for (let i = end; i < text.length; omitted++) i += text.codePointAt(i)! > 0xffff ? 2 : 1; + return { head: text.slice(0, end), omitted }; +} + +/** Any value as JSON (a string quoted too). Never throws. */ +function asJson(value: unknown): string { + try { + return JSON.stringify(value) ?? 'null'; + } catch { + return Object.prototype.toString.call(value); + } +} + +/** Any value as text: a string as itself, anything else as JSON. Never throws. */ +function asText(value: unknown): string { + return typeof value === 'string' ? value : asJson(value); +} + +function bounded(value: unknown, max = STUB_FIELD_POINTS): string { + const { head, omitted } = headPoints(asText(value), max); + return omitted > 0 ? `${head}…` : head; +} + +/** A payload's decoded size, its base64 read without the whitespace a + * wrapped encoding carries. */ +function decodedSize(base64: string): number { + const compact = base64.replace(/\s+/g, ''); + const padding = compact.endsWith('==') ? 2 : compact.endsWith('=') ? 1 : 0; + return Math.max(0, Math.floor((compact.length * 3) / 4) - padding); +} + +function humanSize(bytes: number): string { + if (bytes < 1024) return `${bytes} B`; + if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`; + return `${(bytes / (1024 * 1024)).toFixed(1)} MB`; +} + +const KNOWN_EXTENSIONS: Record = { 'octet-stream': 'bin', 'mpeg': 'mp3', 'plain': 'txt', 'x-wav': 'wav' }; + +function extensionFor(mimeType: string): string { + const subtype = mimeType.split('/')[1]?.split(';')[0]?.trim().toLowerCase() ?? ''; + // The table's own keys only: a subtype such as 'constructor' must not find + // what Object.prototype holds. + const ext = Object.hasOwn(KNOWN_EXTENSIONS, subtype) ? KNOWN_EXTENSIONS[subtype]! : subtype.replace(/[^a-z0-9]/g, ''); + return ext.length > 0 && ext.length <= 10 ? ext : 'bin'; +} + +/** The image type a provider takes inline, in its canonical spelling, or + * null when it takes none of these. */ +function nativeImageType(mimeType: unknown): string | null { + if (typeof mimeType !== 'string') return null; + const base = mimeType.split(';')[0]!.trim().toLowerCase(); + return NATIVE_IMAGE_TYPES.has(base) ? base : null; +} + +export interface StandardToolResult { + content?: unknown; + isError?: boolean; + structuredContent?: unknown; +} + +/** + * Normalize a standard MCP tool result into the framework's `ToolResult`. + * `label` names saved payloads (one per call; payloads get an index suffix). + * A payload that can't be retained (no `save`, or saving failed) makes the + * result a failure to materialize, never a success with part of it lost. + * Content that is present but isn't a list of blocks makes it a malformed + * result, never an empty success. A tool's error result with no words, or + * only whitespace, reads as `noWords`, which the framework words for its + * server and tool. + */ +export async function normalizeStandardToolResult( + result: StandardToolResult, + label: string, + save: SaveToolPayload | null, + noWords = 'Tool call failed', +): Promise { + const malformed = result.content !== undefined && !Array.isArray(result.content); + const raw: unknown[] = Array.isArray(result.content) ? result.content : []; + const blocks: Block[] = []; + let saved = 0; + /** Payloads that could be neither shown nor retained. */ + let lost = 0; + + const payloadStub = async ( + kind: string, + base64: string, + mimeType: string, + opts: { origin?: string; why?: string } = {}, + ): Promise => { + const why = opts.why ?? UNSUPPORTED; + const size = humanSize(decodedSize(base64)); + const what = `[${kind}${opts.origin ? ` ${bounded(opts.origin, URI_POINTS)}` : ''}: ${bounded(mimeType)}, ${size}`; + if (!save) { + lost++; + return `${what}. Not retained: ${why}, and no workspace is mounted to save it]`; + } + const fileName = `${label}-${++saved}.${extensionFor(mimeType)}`; + const path = await save(fileName, Buffer.from(base64, 'base64'), mimeType).catch((error: unknown) => { + // The stub says only that saving failed; why goes to the host's log. + console.error(`[mcp] saving tool payload ${fileName} (${kind}, ${JSON.stringify(bounded(mimeType))}, ${size}) failed:`, error); + return null; + }); + if (path) return `${what}, saved to workspace file ${path}${why === UNSUPPORTED ? '' : `; not shown inline: ${why}`}]`; + lost++; + return `${what}. Not retained: ${why}, and saving it to the workspace failed]`; + }; + + for (const entry of raw) { + if (!entry || typeof entry !== 'object' || Array.isArray(entry)) { + blocks.push({ type: 'text', text: `[malformed content entry, left out: ${bounded(entry)}]` }); + continue; + } + const block = entry as Record; + switch (block.type) { + case 'text': + blocks.push(typeof block.text === 'string' + ? { type: 'text', text: block.text } + : { type: 'text', text: '[malformed text block, left out: its text is not a string]' }); + break; + case 'image': + case 'audio': { + const kind = block.type; + if (typeof block.data !== 'string') { + blocks.push({ type: 'text', text: `[${kind} without inline data, not fetched${typeof block.uri === 'string' ? `: ${bounded(block.uri, URI_POINTS)}` : ''}]` }); + break; + } + // Inline data with no type is still a payload, of no known type. + const mimeType = typeof block.mimeType === 'string' ? block.mimeType : 'application/octet-stream'; + const native = kind === 'image' ? nativeImageType(block.mimeType) : null; + if (native && decodedSize(block.data) <= NATIVE_IMAGE_MAX_BYTES) { + // Passed on without a wrapped encoding's whitespace, which a + // provider may refuse, and with it the whole request. + blocks.push({ type: 'image', data: block.data.replace(/\s+/g, ''), mimeType: native }); + } else { + const why = native ? `over the ${humanSize(NATIVE_IMAGE_MAX_BYTES)} a model takes per image` : UNSUPPORTED; + blocks.push({ type: 'text', text: await payloadStub(kind, block.data, mimeType, { why }) }); + } + break; + } + case 'resource_link': { + const name = typeof block.name === 'string' ? block.name : undefined; + const mime = typeof block.mimeType === 'string' ? `, ${bounded(block.mimeType)}` : ''; + blocks.push({ type: 'text', text: `[resource link${name ? ` "${bounded(name)}"` : ''}: ${bounded(block.uri, URI_POINTS)}${mime}; not fetched]` }); + break; + } + case 'resource': { + const resource = (block.resource && typeof block.resource === 'object' ? block.resource : {}) as Record; + const uri = typeof resource.uri === 'string' ? resource.uri : undefined; + const mime = typeof resource.mimeType === 'string' ? resource.mimeType : undefined; + if (typeof resource.text === 'string') { + blocks.push({ type: 'text', text: `[resource ${uri === undefined ? '(no uri)' : bounded(uri, URI_POINTS)}${mime ? `, ${bounded(mime)}` : ''}]\n${resource.text}` }); + } else if (typeof resource.blob === 'string') { + blocks.push({ type: 'text', text: await payloadStub('resource', resource.blob, mime ?? 'application/octet-stream', { origin: uri }) }); + } else { + blocks.push({ type: 'text', text: `[resource ${uri === undefined ? '(no uri)' : bounded(uri, URI_POINTS)} with no contents]` }); + } + break; + } + default: { + // Unknown to this client: shown as data rather than dropped, bounded + // like a stub field, so a large or binary payload can't flood the + // model's view. + const { head, omitted } = headPoints(asText(entry), STUB_FIELD_POINTS); + blocks.push({ type: 'text', text: omitted > 0 ? `${head}… [${omitted} more characters of this block left out]` : head }); + } + } + } + + const failing = !!result.isError || lost > 0 || malformed; + if (failing) { + // A failed result is read only as its text: an inline image becomes a + // payload like any other, saved and named, or marked not retained. + for (let i = 0; i < blocks.length; i++) { + const block = blocks[i]!; + if (block.type === 'image') { + blocks[i] = { type: 'text', text: await payloadStub('image', block.data, block.mimeType, { why: 'a failed result is read as its text alone' }) }; + } + } + } + + const hasStructured = Object.prototype.hasOwnProperty.call(result, 'structuredContent'); + // Only the server's own text blocks count as its presentation for the + // model. Stubs this module wrote are notices, not the server's text. + const serverText = raw.some((entry) => { + const block = entry as { type?: unknown; text?: unknown } | null; + return !!block && typeof block === 'object' && block.type === 'text' && typeof block.text === 'string' && block.text.trim().length > 0; + }); + if (hasStructured && !serverText) { + blocks.push({ type: 'text', text: asJson(result.structuredContent) }); + } + + const text = blocks.filter((b): b is Extract => b.type === 'text').map((b) => b.text).join('\n'); + const structured = hasStructured ? { structured: result.structuredContent } : {}; + if (result.isError) { + return { success: false, error: text.trim() ? text : noWords, isError: true, ...structured }; + } + if (malformed) { + return { + success: false, + error: `${MALFORMED_CONTENT_NOTICE}${text ? `\n${text}` : ''}`, + isError: true, + ...structured, + }; + } + if (lost > 0) { + // The server answered, but part of its answer is gone: report that as a + // failure to materialize the result, never as a success. + return { + success: false, + error: `[result incomplete: ${lost} payload(s) could not be retained, so this result is not complete; ` + + `the tool may already have completed]\n${text}`, + isError: true, + ...structured, + }; + } + const hasImage = blocks.some((b) => b.type === 'image'); + return { + success: true, + // Text-only results keep the joined-string convention callers expect. + // An image keeps the array, so the model sees it natively. + data: hasImage ? blocks : (text || undefined), + isError: false, + ...structured, + }; +} diff --git a/src/mcpl/transport.ts b/src/mcpl/transport.ts index fa80cb028..3ccd837f6 100644 --- a/src/mcpl/transport.ts +++ b/src/mcpl/transport.ts @@ -23,6 +23,7 @@ import { StringDecoder } from 'node:string_decoder'; import WebSocket from 'ws'; import type { McplServerConfig } from './types.js'; +import { describeCredentialError, noteDialSent, redactUrl, resolveServerBinding, urlAsDialed } from './protocol-family.js'; import { assertBooleanSetting } from '../config-checks.js'; import { logDetail } from './log-value.js'; @@ -45,6 +46,9 @@ export interface TransportCloseInfo { * - `'stderr'` `(line: string)` diagnostic line (stdio child stderr; never WS) * - `'error'` `(err: Error)` transport-level error * - `'close'` `(info: TransportCloseInfo)` transport closed (once) + * - `'unreaped'` `(failure: Error)` a stdio child whose pipe failed + * outlived SIGKILL too: the transport stays open, owning the child, and + * close() retries the reap (never WS) * * Inbound `'line'`/`'stderr'` events are buffered until the first matching * listener attaches (and re-buffered whenever there are none), so no message is @@ -90,6 +94,17 @@ export abstract class McplTransport extends EventEmitter { return ''; } + /** + * The error a stdio child's stdin or stdout failed with, once one has. The + * transport is ending the child then, and stays open until it exits, but + * writes nothing more to it: writeLine drops each line, and both engines + * refuse a request as not sent. Undefined while writes go out, and always + * for a transport that has no pipes. + */ + get pipeFailure(): Error | undefined { + return undefined; + } + /** * Resolves once the server's diagnostic stream has ended or closed, so that * {@link stderrTail} holds what arrived on it, or once `ms` have passed while @@ -245,6 +260,10 @@ export class StdioTransport extends McplTransport { return (this.stderrCut ? '…' : '') + this.stderrEnd; } + override get pipeFailure(): Error | undefined { + return this.pipeError; + } + override stderrEnded(ms: number): Promise { const stderr = this.child.stderr; if (!stderr || this.stderrDone) return Promise.resolve(); @@ -267,6 +286,8 @@ export class StdioTransport extends McplTransport { private constructor( private readonly child: ChildProcess, private readonly rl: ReadlineInterface, + /** The server this child serves, named in its stderr lines and in what + * its reaping reports. */ serverId: string, ) { super(serverId); @@ -302,16 +323,26 @@ export class StdioTransport extends McplTransport { this.child.on('error', (err) => this.reportError(err)); // The child's stdin and stdout carry the protocol, so after an error on // either, a write's EPIPE or a read error, this transport can't reach the - // child even if it's still running. The handler sends a running child - // SIGTERM, and the exit's close names the error, so the connection handles - // it as any other exit. The close is the report: no 'error' is emitted, - // nothing is thrown. Nothing here escalates to SIGKILL, so a child that - // ignores SIGTERM keeps running, unreported. A child that stops reading - // without closing its stdin raises no error: the writes are buffered, and - // only request timeouts bound it (McplServerConfig.requestTimeoutMs). + // child even if it's still running. The handler ends the child as close() + // does, SIGTERM and then SIGKILL, in the one attempt close() shares, and + // the exit's close names the error, so the connection handles it as any + // other exit. The close is the report: no 'error' is emitted, nothing is + // thrown, and nothing more is written meanwhile (pipeFailure). The first + // error starts that one attempt, and a later one, from either pipe, adds + // nothing to it. A child that outlives SIGKILL too stays this + // transport's: 'unreaped' says so, unless the host is closing it, whose + // close() reports the failure itself. A child that stops reading without + // closing its stdin raises no error: the writes are buffered, and only + // request timeouts bound it (McplServerConfig.requestTimeoutMs). Under + // Bun a child that closes its stdin isn't reported either: Bun emits no + // 'error' for the writes to it (checked on 1.3.10, 1.4.3 and 1.4.4's + // canary), so request timeouts are its only bound too. const endOnPipeError = (err: Error) => { - this.pipeError ??= err; - if (this.child.exitCode === null && this.child.signalCode === null) this.child.kill(); + if (this.pipeError) return; + this.pipeError = err; + void this.terminate().catch((failure: unknown) => { + if (!this.closed) this.emit('unreaped', failure instanceof Error ? failure : new Error(String(failure))); + }); }; this.child.stdin?.on('error', endOnPipeError); // On the stream itself, since close() closes readline, which takes its own @@ -368,31 +399,89 @@ export class StdioTransport extends McplTransport { } writeLine(json: string): void { + // Nothing more goes to a child whose stdin or stdout has failed: this + // transport is ending it. + if (this.pipeError) return; // stdin may be gone if the child died between the closed-check and here. this.child.stdin?.write(json + '\n'); } - async close(): Promise { + private closing: Promise | null = null; + + /** + * Close and reap: resolves once the child has actually exited, not merely + * been signalled (`child.killed` is true as soon as a signal is sent). A + * child that outlives SIGTERM by {@link STDIO_EXIT_GRACE_MS} gets SIGKILL. + * If it still hasn't exited after that bound again, close rejects: an + * explicit cleanup failure, rather than waiting forever or claiming an + * exit that wasn't seen. Calls share one attempt, and so does a pipe + * failure that is already ending the child; a later call succeeds if the + * child has exited since. + */ + close(): Promise { + if (!this.closing) { + this.closing = this.reap(); + return this.closing; + } + // A later call re-checks a failed verdict: a child that has exited since + // is reaped now. No new signals are sent. + return this.closing.catch((error: unknown) => { + if (this.exited()) return; + throw error; + }); + } + + private async reap(): Promise { this.markClosed({ reason: 'closed by host' }); this.rl.close(); - if (this.child && !this.child.killed) { - this.child.kill(); - } - await new Promise((resolve) => { - // Once Node has seen a child exit, it has its exitCode set, or its - // signalCode when a signal ended it, and its 'exit' event has fired. A - // signal that didn't come through kill() (an OOM kill, a crash) leaves - // `killed` false. A child that has exited but isn't reaped yet still - // takes the kill() above, which sets `killed`. - if (!this.child || this.child.exitCode !== null || this.child.signalCode !== null || this.child.killed) { - resolve(); - } else { - this.child.once('exit', () => resolve()); - } + await this.terminate(); + } + + /** Whether the child has exited, or never started (a spawn error). */ + private exited(): boolean { + return this.child.pid === undefined || this.child.exitCode !== null || this.child.signalCode !== null; + } + + private terminating: Promise | null = null; + + /** + * End the child: SIGTERM, then SIGKILL once it has outlived SIGTERM by + * {@link STDIO_EXIT_GRACE_MS}. Resolves once it has exited; rejects if it + * still hasn't that long after SIGKILL. close() and a pipe failure share + * one attempt, so no second signal or timer goes out. A failed attempt is + * re-checked, not repeated: a child that has exited since resolves it. + */ + private terminate(): Promise { + this.terminating ??= this.escalate(); + return this.terminating.catch((error: unknown) => { + if (this.exited()) return; + throw error; + }); + } + + private async escalate(): Promise { + const child = this.child; + if (this.exited()) return; + const exited = new Promise((resolve) => child.once('exit', () => resolve())); + const exitWithin = (ms: number) => new Promise((resolve) => { + const timer = setTimeout(() => resolve(false), ms); + void exited.then(() => { clearTimeout(timer); resolve(true); }); }); + child.kill('SIGTERM'); + if (await exitWithin(STDIO_EXIT_GRACE_MS)) return; + const named = `stdio child ${child.pid} of MCP server "${this.serverId}"`; + console.error(`[mcpl] ${named} ignored SIGTERM for ${STDIO_EXIT_GRACE_MS}ms — sending SIGKILL`); + child.kill('SIGKILL'); + if (await exitWithin(STDIO_EXIT_GRACE_MS)) return; + throw new Error( + `${named} did not exit within ${STDIO_EXIT_GRACE_MS}ms of SIGKILL; it could not be reaped`, + ); } } +/** How long a stdio child gets to exit after SIGTERM, and again after SIGKILL. */ +const STDIO_EXIT_GRACE_MS = 2_000; + // --------------------------------------------------------------------------- // WebSocket // --------------------------------------------------------------------------- @@ -400,6 +489,18 @@ export class StdioTransport extends McplTransport { /** Timeout for the WebSocket to reach OPEN before we give up (ms). */ const WS_OPEN_TIMEOUT_MS = 15_000; +/** `work`, or `signal`'s reason once it aborts, whichever comes first. The + * listener comes off the signal when `work` settles, so a long-lived signal, + * such as a reconnect loop's, doesn't gather one per dial. */ +function raceAbort(work: Promise, signal: AbortSignal): Promise { + if (signal.aborted) return Promise.reject(signal.reason); + return new Promise((resolve, reject) => { + const onAbort = () => reject(signal.reason); + signal.addEventListener('abort', onAbort, { once: true }); + work.then(resolve, reject).finally(() => signal.removeEventListener('abort', onAbort)); + }); +} + export class WebSocketTransport extends McplTransport { readonly kind = 'websocket' as const; @@ -426,28 +527,43 @@ export class WebSocketTransport extends McplTransport { /** * Dial `config.url` and resolve once the socket is OPEN. Appends `?token=` * when `config.token` is set. Rejects on connect error / timeout (and tears - * the socket down) so a failed dial never leaks an open handle. + * the socket down) so a failed dial never leaks an open handle. `signal` + * ends a dial whose connection is closed meanwhile, at once, rather than + * at the open timeout. */ - static async open(config: McplServerConfig): Promise { + static async open(config: McplServerConfig, signal?: AbortSignal): Promise { // Resolve the per-dial credential first: the provider runs on every // connect AND reconnect, so rotation happens at the transport layer and // nothing above it ever holds a credential. let token: string | null = config.token ?? null; + const abandoned = (): Error => + new Error(`MCPL server "${config.id}" WebSocket connect abandoned: the connection was closed (${redactUrl(config.url)})`); if (config.accessProvider) { + if (signal?.aborted) throw abandoned(); try { - token = (await config.accessProvider()) ?? token; + // A dial whose connection is closed meanwhile stops waiting for the + // provider; whatever the provider does later is ignored. + const provided = Promise.resolve(config.accessProvider()); + provided.catch(() => { /* a late failure after the dial was abandoned */ }); + token = (signal ? await raceAbort(provided, signal) : await provided) ?? token; } catch (err) { + if (signal?.aborted) throw abandoned(); // Fall back to the static token (if any) rather than failing the dial // outright — the server will refuse a stale credential loudly anyway. + // The provider's own text may quote the credential, so it isn't logged. console.error( - `[mcpl] server "${config.id}": access provider failed (${err instanceof Error ? err.message : err}) — dialing with configured fallback`, + `[mcpl] server "${config.id}": access provider failed (${describeCredentialError(err)}) — dialing with configured fallback`, ); } } const url = buildWebSocketUrl(config, token); + // A provider's token is in this url, and the query may be re-encoded: + // the server's text is recorded with what the dial sent, the token in + // each form a token is replaced in. + noteDialSent(config, { url, token: token ?? undefined }); // Error strings travel: traces, logs, tool results the model reads. They // get the endpoint, never the credential. - const redacted = buildWebSocketUrl({ ...config, token: undefined }, null); + const redacted = redactUrl(config.url); const ws = new WebSocket(url); // A failed dial can emit 'error' MORE THAN ONCE on the raw socket (seen @@ -461,31 +577,36 @@ export class WebSocketTransport extends McplTransport { return new Promise((resolve, reject) => { let settled = false; - const timer = setTimeout(() => { + const abandon = (error: Error): void => { if (settled) return; settled = true; + clearTimeout(timer); + signal?.removeEventListener('abort', onAbort); ws.removeAllListeners(); ws.on('error', swallowLateErrors); try { ws.terminate(); } catch { /* already gone */ } - reject(new Error(`MCPL server "${config.id}" WebSocket connect timed out after ${WS_OPEN_TIMEOUT_MS}ms (${redacted})`)); + reject(error); + }; + const timer = setTimeout(() => { + abandon(new Error(`MCPL server "${config.id}" WebSocket connect timed out after ${WS_OPEN_TIMEOUT_MS}ms (${redacted})`)); }, WS_OPEN_TIMEOUT_MS); + const onAbort = (): void => { + abandon(new Error(`MCPL server "${config.id}" WebSocket connect abandoned: the connection was closed (${redacted})`)); + }; + if (signal?.aborted) onAbort(); + else signal?.addEventListener('abort', onAbort, { once: true }); ws.once('open', () => { if (settled) return; settled = true; clearTimeout(timer); + signal?.removeEventListener('abort', onAbort); ws.removeAllListeners('error'); // the instance re-attaches its own handlers resolve(new WebSocketTransport(ws, config.id)); }); ws.once('error', (err: Error) => { - if (settled) return; - settled = true; - clearTimeout(timer); - ws.removeAllListeners(); - ws.on('error', swallowLateErrors); - try { ws.terminate(); } catch { /* already gone */ } - reject(new Error(`MCPL server "${config.id}" WebSocket connect failed: ${err.message} (${redacted})`)); + abandon(new Error(`MCPL server "${config.id}" WebSocket connect failed: ${err.message} (${redacted})`)); }); }); } @@ -527,7 +648,8 @@ export function buildWebSocketUrl(config: McplServerConfig, tokenOverride?: stri try { parsed = new URL(config.url); } catch { - throw new Error(`MCPL server "${config.id}": invalid WebSocket url "${config.url}"`); + // Not echoed: a url that doesn't parse can't be redacted. + throw new Error(`MCPL server "${config.id}": invalid WebSocket url (it isn't shown, since it may carry a credential)`); } if (parsed.protocol !== 'ws:' && parsed.protocol !== 'wss:') { throw new Error( @@ -538,22 +660,33 @@ export function buildWebSocketUrl(config: McplServerConfig, tokenOverride?: stri throw new Error(`MCPL server "${config.id}": websocket url must not have a fragment, got "${parsed.hash}"`); } const token = tokenOverride !== undefined ? tokenOverride : config.token; - if (token) { - parsed.searchParams.set('token', token); - } - return parsed.toString(); + return urlAsDialed(parsed, token).toString(); } // --------------------------------------------------------------------------- // Factory // --------------------------------------------------------------------------- -/** True when the config selects the WebSocket transport (explicit `transport`, - * or a `url` with no `command`). */ +/** True when the config selects the WebSocket transport, as + * {@link resolveServerBinding} decides it; false for any other transport and + * for a configuration it refuses. */ export function isWebSocketTransport(config: McplServerConfig): boolean { - if (config.transport === 'websocket') return true; - if (config.transport === 'stdio') return false; - return Boolean(config.url) && !config.command; + try { + return resolveServerBinding(config).transport === 'websocket'; + } catch { + return false; + } +} + +/** Whether a server's binding dials it, over WebSocket or HTTP. False for a + * stdio launch, and for a configuration whose binding can't be resolved, + * which the family check refuses. */ +function dialsNetwork(config: McplServerConfig): boolean { + try { + return resolveServerBinding(config).transport !== 'stdio'; + } catch { + return false; + } } /** The settings of McplServerConfig whose type is boolean. */ @@ -577,25 +710,29 @@ const SERVER_CONFIG_BOOLEANS: Record = { * only `true` as on, so the quotes would hide what the value does. null counts * as not set, so a generated config that writes an unset setting as null * keeps working. `inheritEnv` is checked only for a stdio server, since a - * WebSocket server never reads it. The error names the server, the setting - * and the value's kind, not the value, which could be a secret pasted into - * the wrong field. + * network server, WebSocket or HTTP, never reads it. The error names the + * server, the setting and the value's kind, not the value, which could be a + * secret pasted into the wrong field. */ export function assertServerConfigBooleans(config: McplServerConfig): void { for (const [setting, whatTrueDoes] of Object.entries(SERVER_CONFIG_BOOLEANS) as Array<[BooleanSetting, string]>) { - if (setting === 'inheritEnv' && isWebSocketTransport(config)) continue; + if (setting === 'inheritEnv' && dialsNetwork(config)) continue; assertBooleanSetting(`MCPL server "${config.id}" ${setting}`, config[setting], whatTrueDoes); } } /** - * Open the transport selected by `config` (WebSocket when `transport` is - * 'websocket' or a bare `url` is given; stdio otherwise). The returned transport - * is connected at the byte level but has NOT performed the MCPL handshake. + * Open the legacy transport `config` selects, as {@link resolveServerBinding} + * decides it: stdio for a `command`, MCPL's WebSocket binding for a ws(s):// + * url. A configuration it refuses, or one for the modern family (an http(s) + * url, or `protocol: 'modern'`), throws before anything is spawned or dialed. + * The returned transport is connected at the byte level but has NOT + * performed the MCPL handshake. */ -export async function openTransport(config: McplServerConfig): Promise { - if (isWebSocketTransport(config)) { - return WebSocketTransport.open(config); +export async function openTransport(config: McplServerConfig, signal?: AbortSignal): Promise { + const binding = resolveServerBinding(config); + if (binding.family !== 'legacy') { + throw new Error(`MCP server "${config.id}" is configured for modern MCP; the legacy transports can't open it`); } - return StdioTransport.spawn(config); + return binding.transport === 'websocket' ? WebSocketTransport.open(config, signal) : StdioTransport.spawn(config); } diff --git a/src/mcpl/types.ts b/src/mcpl/types.ts index d84c655ed..fcc6ab6fc 100644 --- a/src/mcpl/types.ts +++ b/src/mcpl/types.ts @@ -211,15 +211,22 @@ export interface McplChannelCapabilities { /** * Configuration for connecting to a single MCPL server. * Provided in FrameworkConfig.mcplServers[]. + * + * Treat a submitted configuration and its nested values as immutable for as + * long as the server uses them. Apply changed settings with + * `restartMcplServer(id, newConfig)`; use `accessProvider` for a network + * server's rotating credentials (a stdio server's changed `env` goes through + * a restart with the new configuration). */ export interface McplServerConfig { /** Unique server identifier */ id: string; /** - * Command to spawn the server process (stdio transport). Mutually exclusive - * with `url`. Required for the stdio transport; omit it (and set `url`) for - * the WebSocket transport. + * Command to spawn the server process (stdio transport). Required for the + * stdio transport; omit it (and set `url`) for a network server, over + * WebSocket or HTTP. A configuration with both needs `transport` to say + * which one is used, and is refused without it. */ command?: string; @@ -246,22 +253,41 @@ export interface McplServerConfig { inheritEnv?: boolean; /** - * WebSocket URL for the network transport (`ws://` or `wss://`). Mutually - * exclusive with `command`. When set (or `transport: 'websocket'`), the host - * dials this endpoint instead of spawning a process — newline-delimited - * JSON-RPC MCPL runs over the socket identically to stdio. + * URL for a network server, used instead of `command`. A configuration + * with both needs `transport` to say which one is used, and is refused + * without it. Its scheme decides the protocol family: + * - `ws://` or `wss://` is a legacy MCPL server over MCPL's WebSocket + * binding. The host dials it and runs the same newline-delimited JSON-RPC + * as over stdio. + * - `http://` or `https://` is a modern MCP (2026-07-28) server over + * Streamable HTTP. */ url?: string; /** - * Transport selector. Defaults to `'websocket'` when a bare `url` is given - * (no `command`), `'stdio'` otherwise. Set explicitly to disambiguate. + * Transport selector. When it is absent, a `command` means stdio and a + * `url` means the transport its scheme names; with both, it is required. + * When it is set, it decides which of the two is used and must agree with + * it: `'stdio'` needs a `command`, `'websocket'` a ws(s):// url and + * `'http'` an http(s):// url. */ - transport?: 'stdio' | 'websocket'; + transport?: 'stdio' | 'websocket' | 'http'; /** - * Bearer token for WebSocket auth. Appended to `url` as a `token` query - * parameter (`?token=…`) on connect. Ignored by the stdio transport. + * Protocol family for a stdio (`command`) server. Defaults to `'legacy'`, + * which is MCP 2024-11-05 + MCPL and today's behavior. `'modern'` speaks + * MCP 2026-07-28 through the SDK-backed modern engine. A network server's + * family comes from its url's scheme, so setting this on one is a + * configuration error. There is no probing between families. + */ + protocol?: 'legacy' | 'modern'; + + /** + * Bearer token for a network server. Where it travels depends on the + * transport: on a ws(s):// url it is appended as a `token` query parameter + * (`?token=…`) on connect; to an http(s):// server it is sent as an + * `Authorization: Bearer` header and never put in the url. A legacy stdio + * server ignores it, and a modern stdio server refuses it. */ token?: string; @@ -384,12 +410,20 @@ export interface McplServerConfig { * descriptive error, which the framework surfaces as a normal isError * tool_result. Request timeouts are the only bound for a stdio server that * stops reading its stdin without closing it: the host's writes are - * buffered, no error is raised, and the connection stays up. This is the - * timeout of every request that names none; the handshake's initialize has - * its own. Set 0 to disable, except for channels/publish and the - * channels/open a delivery may send first, which keep the 60 s default: a - * publish's outcome is reported to the resident and recorded for its - * draft, so it must end. Default: 60000 (60 seconds). + * buffered, no error is raised, and the connection stays up. Under Bun they + * are also the only bound for one that closes its stdin, since Bun emits no + * 'error' event for the writes to it. requestTimeoutMs is the timeout of + * every request that names none; the handshake's initialize has its own. + * Set 0 to disable, except for channels/publish and the channels/open a + * delivery may send first, which keep the 60 s default: a publish's outcome + * is reported to the resident and recorded for its draft, so it must end. + * Default: 60000 (60 seconds). A value above 2^31−1 is refused: Node runs a + * longer timer after about 1 ms, so every request would time out at once. + * + * For a modern server it is one deadline per tool call, across every leg + * of the call, and must be an integer from 1 to 2^31−1. It can't be + * disabled: the SDK reads 0 as "time out now". At the deadline + * cancellation is requested and the outcome is unknown. */ requestTimeoutMs?: number; @@ -1333,6 +1367,8 @@ export interface McpToolDefinition { name: string; description?: string; inputSchema: Record; + /** Server metadata for the tool (e.g. RFC-008 `mcpl/class` hints). */ + _meta?: Record; } /** diff --git a/src/types/events.ts b/src/types/events.ts index 976731ff9..1a7142320 100644 --- a/src/types/events.ts +++ b/src/types/events.ts @@ -192,6 +192,14 @@ export interface ToolResult { error?: string; /** Whether this was an error (for LLM) */ isError?: boolean; + /** + * An MCP tool's `structuredContent`: its machine-readable result, beside + * `data`, which is the content view. Present exactly when the server sent + * one, so `false`, `0` and `null` are values here, not absence. Programs + * read it (scripts get it in their result object); the model sees content, + * and sees this rendered as JSON only when the content carries no text. + */ + structured?: unknown; /** * When true, the framework saves tool_use + tool_result messages to context, * cancels the active stream, and resets the agent to idle. diff --git a/test/checkpoint-attribution.test.ts b/test/checkpoint-attribution.test.ts index 3f9581105..db30b80b7 100644 --- a/test/checkpoint-attribution.test.ts +++ b/test/checkpoint-attribution.test.ts @@ -44,6 +44,7 @@ function makeFrameworkHarness( }> = []; const server = { id: 'srv', + isConnected: true, async sendToolsList() { return { tools }; }, async sendToolsCall( name: string, diff --git a/test/code-execution-structured-results.test.ts b/test/code-execution-structured-results.test.ts new file mode 100644 index 000000000..c80d710c6 --- /dev/null +++ b/test/code-execution-structured-results.test.ts @@ -0,0 +1,180 @@ +/** + * A tool's structured result end to end: a real MCPL connection, the + * framework's dispatch, and a real Python script. Adapted from + * anima-research/agent-framework#181's test of the same name, which first + * found scripts losing structuredContent. Beside what a script reads, it + * checks that the model's own calls and the stored history keep the readable + * text, and that a result an in-process module returns but JSON can't hold + * reaches the script as an error naming the tool. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { AddressInfo } from 'node:net'; +import { WebSocketServer } from 'ws'; + +import { AgentFramework } from '../src/index.js'; +import type { EventResponse, Module, ModuleContext, ToolCall, ToolDefinition, ToolResult } from '../src/index.js'; +import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; + +const envelope = { + runId: 'r41', state: 'running', exit: null, nextCursor: 17, truncated: false, + output: 'café 🌱', details: { retries: 0, lines: ['one', 'two'] }, +}; +const transcript = 'Readable transcript: work continues.'; +const text = (t: string) => [{ type: 'text', text: t }]; + +/** tools/call results by variant, as an MCPL peer sends them. */ +const fixtures: Record = { + dual: { content: text(transcript), structuredContent: envelope }, + empty: { content: text('not the empty object'), structuredContent: {} }, + only: { content: [], structuredContent: envelope }, + plain: { content: text('plain text') }, + absent: { content: [] }, + error: { content: text('run refused'), structuredContent: { state: 'rejected' }, isError: true }, + large: { content: text('short rendering'), structuredContent: { output: 'x'.repeat(5_000_000) } }, +}; + +/** An in-process module whose results JSON can't hold. */ +class UnconvertibleModule implements Module { + readonly name = 'loop'; + async start(_ctx: ModuleContext): Promise {} + async stop(): Promise {} + async onProcess(): Promise { return {}; } + getTools(): ToolDefinition[] { + return ['cyclic', 'bigint', 'cyclic_data', 'cyclic_error', 'bare_throw'].map((name) => ({ name, description: name, inputSchema: { type: 'object' } })); + } + async handleToolCall(call: ToolCall): Promise { + const name = call.name.includes('--') ? call.name.slice(call.name.lastIndexOf('--') + 2) : call.name; + const cyclic: Record = { kind: 'loop' }; + cyclic.self = cyclic; + if (name === 'cyclic') return { success: true, data: text('a loop'), structured: cyclic }; + if (name === 'bigint') return { success: true, data: text('a big number'), structured: { n: 1n } }; + if (name === 'cyclic_error') return { success: false, isError: true, error: 'db timeout', data: text('failed'), structured: cyclic }; + // A toJSON that throws a value with no prototype, so String() can't describe it either. + if (name === 'bare_throw') return { success: true, data: text('bare'), structured: { toJSON() { throw Object.create(null); } } }; + return { success: true, data: cyclic }; + } +} + +test("a script reads a structured result whole, the model's calls and history keep the readable text, and a result JSON can't hold is an error naming the tool", async () => { + const calls: string[] = []; + const wss = new WebSocketServer({ port: 0, host: '127.0.0.1' }); + wss.on('connection', (ws) => ws.on('message', (data) => { + const msg = JSON.parse(String(data)); + const reply = (result: unknown) => ws.send(JSON.stringify({ jsonrpc: '2.0', id: msg.id, result })); + if (msg.method === 'initialize') { + reply({ + protocolVersion: '2024-11-05', serverInfo: { name: 'fixture', version: '1.0.0' }, + capabilities: { tools: {}, experimental: { mcpl: { version: '0.4', pushEvents: true } } }, + }); + } else if (msg.method === 'featureSets/update') { + reply({}); + } else if (msg.method === 'tools/list') { + reply({ tools: [{ name: 'inspect', description: 'Read work status', inputSchema: { type: 'object' } }] }); + } else if (msg.method === 'tools/call') { + calls.push(msg.params.arguments.variant); + reply(fixtures[msg.params.arguments.variant]); + } + })); + if (!wss.address()) await new Promise((resolve) => wss.once('listening', resolve)); + const dir = mkdtempSync(join(tmpdir(), 'pytc-structured-')); + const membrane = new MockMembrane(); + let framework: AgentFramework | undefined; + try { + framework = await AgentFramework.create({ + storePath: join(dir, 'store.chronicle'), membrane: membrane.asMembrane(), + agents: [], modules: [new UnconvertibleModule()], syncIntervalMs: 0, codeExecution: { enabled: true }, + mcplServers: [{ id: 'fixture', url: `ws://127.0.0.1:${(wss.address() as AddressInfo).port}`, toolPrefix: 'fixture' }], + }); + await framework.start(); + for (let i = 0; i < 100 && !framework.getAllTools().some((t) => t.name === 'fixture--inspect'); i++) { + await new Promise((resolve) => setTimeout(resolve, 50)); + } + + // The promise-based path, which modules and outside callers use, carries + // both views. + const direct = await framework.executeToolCall({ id: 'direct-api', name: 'fixture--inspect', input: { variant: 'dual' } }); + assert.deepEqual(direct.data, text(transcript)); + assert.deepEqual(direct.structured, envelope); + + const code = [ + 'import json', + `ENVELOPE = json.loads(${JSON.stringify(JSON.stringify(envelope))})`, + `TRANSCRIPT = ${JSON.stringify(transcript)}`, + 'r = json.loads(await fixture__inspect({"variant": "dual"}))', + 'assert r == {"content": [{"type": "text", "text": TRANSCRIPT}], "structuredContent": ENVELOPE, "isError": False}, r', + 'empty = json.loads(await fixture__inspect({"variant": "empty"}))', + 'assert empty["structuredContent"] == {} and empty["content"] == [{"type": "text", "text": "not the empty object"}], empty', + 'only = json.loads(await fixture__inspect({"variant": "only"}))', + 'assert only == {"content": [], "structuredContent": ENVELOPE, "isError": False}, only', + // Without structured content, a result is the text it always was. + 'plain = await fixture__inspect({"variant": "plain"})', + 'assert plain == json.dumps("plain text"), plain', + 'absent = await fixture__inspect({"variant": "absent"})', + 'assert absent == "", absent', + 'error = json.loads(await fixture__inspect({"variant": "error"}))', + 'assert error == {"content": [{"type": "text", "text": "run refused"}], "structuredContent": {"state": "rejected"}, "isError": True, "error": "run refused"}, error', + // Over the cap, with no workspace to save it to: a failure, never a success missing its result. + 'large = json.loads(await fixture__inspect({"variant": "large"}))', + 'assert large["isError"] is True and large["oversized"]["savedTo"] is None and large["oversized"]["chars"] > 5000000, large', + 'assert "could not be delivered" in large["error"], large', + // Results JSON can't hold: the dispatch makes each an error naming the + // tool (a module's result has to be stored), which a script reads as + // its error text, structured or not. + 'cyclic = await loop__cyclic({})', + 'assert cyclic.startswith("Error: Tool \'loop--cyclic\' ") and "circular" in cyclic, cyclic', + 'bigint = await loop__bigint({})', + 'assert bigint.startswith("Error: Tool \'loop--bigint\' ") and "BigInt" in bigint, bigint', + 'cyclic_data = await loop__cyclic_data({})', + 'assert cyclic_data.startswith("Error: Tool \'loop--cyclic_data\' ") and "circular" in cyclic_data, cyclic_data', + 'assert all(e.endswith("The call itself completed.") for e in [cyclic, bigint, cyclic_data]), "completed"', + // An error result that can't be converted keeps the tool's own error. + 'cyclic_error = await loop__cyclic_error({})', + 'assert cyclic_error.endswith("The call itself completed, with this error: db timeout"), cyclic_error', + 'bare_throw = await loop__bare_throw({})', + 'assert "(a value with no text form)" in bare_throw, bare_throw', + 'print(r["structuredContent"]["runId"], r["structuredContent"]["state"], r["structuredContent"]["output"])', + ].join('\n'); + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'model-direct', name: 'fixture--inspect', input: { variant: 'dual' } }, + { type: 'tool_use', id: 'model-only', name: 'fixture--inspect', input: { variant: 'only' } }, + { type: 'tool_use', id: 'model-python', name: 'code_execution', input: { code } }, + ], 'tool_use')); + membrane.pushResponse(createMockResponse([])); + const created = await framework.createEphemeralAgent({ name: 'worker', model: 'test-model', systemPrompt: 'Inspect work.', allowedTools: 'all' }); + created.contextManager.addMessage('user', [{ type: 'text', text: 'Go.' }]); + await framework.runEphemeralToCompletion(created.agent, created.contextManager); + + const results = membrane.lastStream!.receivedToolResults.flat() as Array<{ toolUseId: string; content: string }>; + const resultOf = (id: string) => results.find((r) => r.toolUseId === id)!.content; + // The model reads the content: the text when there is some, and the + // structured JSON only when the content has none. + assert.equal(resultOf('model-direct'), JSON.stringify(transcript)); + assert.match(resultOf('model-only'), /\\"runId\\":\\"r41\\"/); + const python = JSON.parse(resultOf('model-python')); + assert.equal(python.stderr, ''); + assert.equal(python.return_code, 0); + assert.equal(python.stdout.trim(), 'r41 running café 🌱'); + assert.deepEqual(calls, ['dual', 'dual', 'only', 'dual', 'empty', 'only', 'plain', 'absent', 'error', 'large']); + + // The stored history holds what the model read: the readable direct + // result and the script's chosen stdout, never the script's intermediate + // results. + const stored = created.contextManager.getAllMessages().flatMap((m) => m.content) + .filter((b) => b.type === 'tool_result') as unknown as Array<{ toolUseId: string; content: string }>; + const storedOf = (id: string) => JSON.stringify(stored.find((b) => b.toolUseId === id)!.content); + assert.ok(storedOf('model-direct').includes('Readable transcript')); + assert.ok(!storedOf('model-direct').includes('nextCursor')); + assert.ok(!storedOf('model-python').includes('nextCursor')); + assert.ok(!storedOf('model-python').includes('short rendering')); + assert.ok(!storedOf('model-python').includes('structuredContent')); + } finally { + await framework?.stop(); + for (const client of wss.clients) client.terminate(); + await new Promise((resolve) => wss.close(() => resolve())); + rmSync(dir, { recursive: true, force: true }); + } +}); diff --git a/test/failed-create-releases.test.ts b/test/failed-create-releases.test.ts index 7d063defc..9d82eeb62 100644 --- a/test/failed-create-releases.test.ts +++ b/test/failed-create-releases.test.ts @@ -83,7 +83,7 @@ test('a create() whose module fails to start stops every module it started, that await retried.stop(); }); -test('a create() that fails after its MCPL server connected closes the server and frees its store', async t => { +test('a create() that fails after its MCP servers connected closes them, in both engines, and frees its store', async t => { const dir = tempDir(t); const outboxPath = join(dir, 'awareness.json'); // A pending Discord awareness batch makes startup run the awareness barrier @@ -99,7 +99,12 @@ test('a create() that fails after its MCPL server connected closes the server an agents: [{ name: 'assistant', model: 'test', systemPrompt: 'test' }], discordAwarenessOutboxPath: outboxPath, discordAwarenessDeadlineMs: 1_000, + // The modern server connects first, so both engines hold a server when + // the failure comes. mcplServers: [{ + id: 'modern', command: process.execPath, protocol: 'modern' as const, + args: [fileURLToPath(new URL('./fixtures/modern-raw-server.mjs', import.meta.url)), '', join(dir, 'modern.jsonl')], + }, { id: 'discord', command: process.execPath, args: [fileURLToPath(new URL('./fixtures/awareness-barrier-mcpl-server.mjs', import.meta.url))], requestTimeoutMs: 1_000, enabledFeatureSets: ['chat'], @@ -120,14 +125,21 @@ test('a create() that fails after its MCPL server connected closes the server an .map(line => JSON.parse(line) as { event: string; pid?: number }) .find(record => record.event === 'initialized'); assert.ok(initialized?.pid, 'the MCPL server had finished its handshake before the failure'); - // Wait on that child itself, with room for a loaded machine: only a child - // still running after 10 s fails this. + const modern = readFileSync(join(dir, 'modern.jsonl'), 'utf8').split('\n').filter(Boolean) + .map(line => JSON.parse(line) as { event: string; method?: string; pid?: number }); + const modernPid = modern.find(record => record.event === 'start')?.pid; + assert.ok(modernPid, 'the modern server was launched'); const running = (pid: number) => { try { process.kill(pid, 0); return true; } catch (error) { return (error as NodeJS.ErrnoException).code === 'EPERM'; } }; - for (let waits = 0; running(initialized.pid); waits++) { - assert.ok(waits < 1000, 'the MCPL server is still running after create() rejected'); + // Whatever the assertions find, no child outlives the test. + t.after(() => { if (running(modernPid)) process.kill(modernPid, 'SIGKILL'); }); + assert.ok(modern.some(record => record.method === 'server/discover'), 'the modern server had its handshake before the failure'); + // Wait on those children themselves, with room for a loaded machine: only + // a child still running after 10 s fails this. + for (let waits = 0; running(initialized.pid) || running(modernPid); waits++) { + assert.ok(waits < 1000, `still running after create() rejected: ${running(initialized.pid) ? 'the MCPL server' : 'the modern server'}`); await new Promise(done => setTimeout(done, 10)); } const retried = await AgentFramework.create(config); diff --git a/test/fixtures/awareness-barrier-mcpl-server.mjs b/test/fixtures/awareness-barrier-mcpl-server.mjs index b83fff1a5..e247a8031 100644 --- a/test/fixtures/awareness-barrier-mcpl-server.mjs +++ b/test/fixtures/awareness-barrier-mcpl-server.mjs @@ -171,6 +171,7 @@ function beginServerTraffic() { function handle(message) { if (message.method === 'initialize') { reply(message.id, { + protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { diff --git a/test/fixtures/legacy-version-server.mjs b/test/fixtures/legacy-version-server.mjs new file mode 100644 index 000000000..7a12ee592 --- /dev/null +++ b/test/fixtures/legacy-version-server.mjs @@ -0,0 +1,90 @@ +// Stdio MCP server for the legacy engine's protocol-version and pagination +// tests. Usage: node legacy-version-server.mjs [startLog] +// +// Modes (how `initialize` is answered): +// ok protocolVersion 2024-11-05 +// reject error -32022 with data { supported: ['2026-07-28'], requested } +// reject-old error -32022 with data { supported: ['2025-03-26'], requested } +// reject-nodata error -32022 without data +// mismatch protocolVersion 2025-06-18 +// missing no protocolVersion at all +// error a JSON-RPC error (-32603), not a version verdict +// hang never answers +// slow `ok`, after SLOW_INIT_MS (default 600) +// ok-then- `ok` on the first start recorded in startLog, after +// +// IGNORE_SIGTERM=1 makes the process survive SIGTERM (only SIGKILL ends it). +// +// tools/list pages: by default three pages (cursors p2, p3). MODE_LIST picks +// an edge case: loop (always nextCursor "same"), empty (nextCursor "" then a +// second page), notools (no tools array), badcursor (nextCursor 5), +// nullcursor (nextCursor null), endless (a new cursor on every page, so the +// list never ends), badlater (a well-formed first page, then a page with no +// tools array). Tool `die` exits the process. +import { appendFileSync, existsSync, readFileSync } from 'node:fs'; + +let mode = process.argv[2] ?? 'ok'; +const startLog = process.argv[3]; +if (startLog) { + const priorStarts = existsSync(startLog) ? readFileSync(startLog, 'utf8').split('\n').filter(Boolean).length : 0; + appendFileSync(startLog, `start ${process.pid}\n`); + if (mode.startsWith('ok-then-')) mode = priorStarts === 0 ? 'ok' : mode.slice('ok-then-'.length); +} +if (process.env.IGNORE_SIGTERM === '1') process.on('SIGTERM', () => {}); +const listMode = process.env.MODE_LIST ?? 'pages'; + +const send = (m) => process.stdout.write(JSON.stringify(m) + '\n'); +const reply = (id, result) => send({ jsonrpc: '2.0', id, result }); +const error = (id, code, message, data) => send({ jsonrpc: '2.0', id, error: { code, message, ...(data === undefined ? {} : { data }) } }); + +const tool = (name) => ({ name, description: name, inputSchema: { type: 'object' } }); + +let buf = ''; +process.stdin.on('data', (chunk) => { + buf += chunk.toString('utf8'); + let i; + while ((i = buf.indexOf('\n')) >= 0) { + const line = buf.slice(0, i); + buf = buf.slice(i + 1); + if (!line.trim()) continue; + let m; + try { m = JSON.parse(line); } catch { continue; } + if (m.method === 'initialize') { + const requested = m.params?.protocolVersion; + if (mode === 'reject') error(m.id, -32022, `Unsupported protocol version: ${requested}`, { supported: ['2026-07-28'], requested }); + else if (mode === 'reject-old') error(m.id, -32022, `Unsupported protocol version: ${requested}`, { supported: ['2025-03-26'], requested }); + else if (mode === 'reject-nodata') error(m.id, -32022, `Unsupported protocol version: ${requested}`); + else if (mode === 'mismatch') reply(m.id, { protocolVersion: '2025-06-18', capabilities: { tools: {} } }); + else if (mode === 'missing') reply(m.id, { capabilities: { tools: {} } }); + else if (mode === 'error') error(m.id, -32603, 'initialize failed on purpose'); + else if (mode === 'hang') { /* never answers */ } + else if (mode === 'slow') setTimeout(() => reply(m.id, { protocolVersion: '2024-11-05', capabilities: { tools: {} } }), Number(process.env.SLOW_INIT_MS ?? 600)); + else reply(m.id, { protocolVersion: '2024-11-05', capabilities: { tools: {} } }); + } else if (m.method === 'tools/list') { + const cursor = m.params?.cursor; + if (listMode === 'loop') reply(m.id, { tools: [tool(`loop-${cursor ?? 'first'}`)], nextCursor: 'same' }); + // '' is an opaque cursor like any other: it continues to a second page. + else if (listMode === 'empty' && cursor === undefined) reply(m.id, { tools: [tool('first')], nextCursor: '' }); + else if (listMode === 'empty' && cursor === '') reply(m.id, { tools: [tool('second')] }); + else if (listMode === 'notools') reply(m.id, {}); + else if (listMode === 'badcursor') reply(m.id, { tools: [tool('x')], nextCursor: 5 }); + else if (listMode === 'nullcursor') reply(m.id, { tools: [tool('x')], nextCursor: null }); + else if (listMode === 'endless') { + const n = cursor === undefined ? 1 : Number(String(cursor).slice(1)) + 1; + reply(m.id, { tools: [tool(`t${n}`)], nextCursor: `c${n}` }); + } + else if (listMode === 'badlater' && cursor === undefined) reply(m.id, { tools: [tool('first')], nextCursor: 'p2' }); + else if (listMode === 'badlater') reply(m.id, {}); + else if (cursor === undefined) reply(m.id, { tools: [tool('a1'), tool('a2')], nextCursor: 'p2' }); + else if (cursor === 'p2') reply(m.id, { tools: [tool('b1')], nextCursor: 'p3' }); + else if (cursor === 'p3') reply(m.id, { tools: [tool('c1'), tool('die')] }); + else error(m.id, -32602, `unknown cursor ${cursor}`); + } else if (m.method === 'tools/call') { + if (m.params?.name === 'die') process.exit(3); + reply(m.id, { content: [{ type: 'text', text: 'ok' }] }); + } else if (m.id !== undefined) { + error(m.id, -32601, `Method not found: ${m.method}`); + } + } +}); +process.stdin.on('end', () => process.exit(0)); diff --git a/test/fixtures/mcpl-inbound-host-probe.mjs b/test/fixtures/mcpl-inbound-host-probe.mjs index 9906d7615..a51ea233d 100644 --- a/test/fixtures/mcpl-inbound-host-probe.mjs +++ b/test/fixtures/mcpl-inbound-host-probe.mjs @@ -17,7 +17,7 @@ process.stdin.on('data', (chunk) => { if (process.argv[1] === 'handshake') { for (const value of values) process.stdout.write(value + '\n'); } - send({jsonrpc:'2.0', id:request.id, result:{capabilities:{}}}); + send({jsonrpc:'2.0', id:request.id, result:{protocolVersion:'2024-11-05', capabilities:{}}}); } else if (request.method === 'tools/list') { if (process.argv[1] === 'connected') { for (const value of values) process.stdout.write(value + '\n'); diff --git a/test/fixtures/modern-mcp-server.mjs b/test/fixtures/modern-mcp-server.mjs new file mode 100644 index 000000000..21c324d38 --- /dev/null +++ b/test/fixtures/modern-mcp-server.mjs @@ -0,0 +1,13 @@ +// Stdio modern-MCP fixture: SDK 2.x serveStdio pinned to 2026-07-28 with +// legacy openings refused (the house services' posture). +// Usage: node modern-mcp-server.mjs [startLog] [eventsFile] +// startLog receives one line per process start, to count launches. +import { appendFileSync } from 'node:fs'; +import { serveStdio } from '@modelcontextprotocol/server/stdio'; +import { createServer } from './modern-tools.mjs'; + +const [startLog, eventsFile] = process.argv.slice(2); +if (startLog) appendFileSync(startLog, `start ${process.pid}\n`); +process.stderr.write(`modern fixture ${process.pid} up\n`); + +serveStdio(() => createServer({ eventsFile }), { legacy: 'reject' }); diff --git a/test/fixtures/modern-raw-server.mjs b/test/fixtures/modern-raw-server.mjs new file mode 100644 index 000000000..9724a4fbf --- /dev/null +++ b/test/fixtures/modern-raw-server.mjs @@ -0,0 +1,171 @@ +// A hand-written modern-MCP (2026-07-28) stdio server, for the edge cases an +// SDK server won't produce on request. It logs every launch and request to +// a JSONL file, so tests can check what reached the wire against what the +// client reports. +// Usage: node modern-raw-server.mjs +// Flags, comma-separated: +// invalid-schema tools/list advertises an outputSchema that doesn't compile +// listen-fail-first the first subscriptions/listen of each launch is refused +// exit-on-listen-first the first launch exits when its subscriptions/listen arrives +// hang-discover-later launches after the first never answer server/discover +// hang-discover no launch ever answers server/discover +// reject-discover-later launches after the first answer server/discover with an error +// reject-discover every launch answers server/discover with an error +// ignore-sigterm the process survives SIGTERM (only SIGKILL ends it) +// other-revision every launch's server/discover lists only a revision the client doesn't speak +// other-revision-later launches after the first do +// Tools: op (outputSchema {value: integer}, answers with a string: invalid +// structured content), plain (text), err (a JSON-RPC error), touch (announces +// a tool list change on the open subscription), nullframe (writes JSON lines +// that aren't JSON-RPC messages, then answers), cont (a state-only +// input_required first leg; the second leg does what arguments.leg2 says: +// 'error', 'hang', 'ok', or 'again' for another state-only round every time. +// arguments.leg1 'exit' exits just after the first leg's answer, and +// 'elicit' asks for elicitation input instead), task (a result type this +// revision doesn't define), bare (a result with no resultType), die (exits), +// vanish (exits without answering), deafen (answers, then closes its stdin and +// stays up, so the next write to it fails with EPIPE). +import { createInterface } from 'node:readline'; +import { appendFileSync, closeSync, existsSync, readFileSync } from 'node:fs'; + +const [flagList = '', log] = process.argv.slice(2); +const flags = new Set(flagList.split(',').filter(Boolean)); +const prior = existsSync(log) ? readFileSync(log, 'utf8').split('\n').filter(Boolean).map((l) => JSON.parse(l)) : []; +const launch = prior.filter((x) => x.event === 'start').length + 1; +const note = (x) => appendFileSync(log, `${JSON.stringify({ ...x, launch })}\n`); +note({ event: 'start', pid: process.pid }); +if (flags.has('ignore-sigterm')) process.on('SIGTERM', () => note({ event: 'sigterm-ignored' })); + +const send = (obj) => process.stdout.write(`${JSON.stringify(obj)}\n`); +const result = (id, body) => send({ jsonrpc: '2.0', id, result: { resultType: 'complete', ...body } }); +const error = (id, code, message, data) => send({ jsonrpc: '2.0', id, error: { code, message, ...(data === undefined ? {} : { data }) } }); +const SUB = 'io.modelcontextprotocol/subscriptionId'; + +let listens = 0; +let subscription = null; + +const lines = createInterface({ input: process.stdin }); +lines.on('line', (line) => { + let m; + try { m = JSON.parse(line); } catch { return; } + note({ event: m.id === undefined ? 'notification' : 'request', method: m.method, id: m.id ?? null, ...(m.method === 'tools/call' ? { tool: m.params?.name } : {}) }); + switch (m.method) { + case 'server/discover': + if (flags.has('other-revision') || (flags.has('other-revision-later') && launch > 1)) { + return result(m.id, { + supportedVersions: ['2099-01-01'], + capabilities: { tools: {} }, + ttlMs: 60000, + cacheScope: 'private', + _meta: { 'io.modelcontextprotocol/serverInfo': { name: 'raw-fixture', version: '1' } }, + }); + } + if (flags.has('hang-discover-later') && launch > 1) return; + if (flags.has('hang-discover')) return; + if (flags.has('reject-discover-later') && launch > 1) return error(m.id, -32603, 'not today'); + if (flags.has('reject-discover')) return error(m.id, -32603, 'not today'); + return result(m.id, { + supportedVersions: ['2026-07-28'], + capabilities: { tools: { listChanged: true } }, + ttlMs: 60000, + cacheScope: 'private', + _meta: { 'io.modelcontextprotocol/serverInfo': { name: 'raw-fixture', version: '1' } }, + }); + case 'subscriptions/listen': + if (flags.has('exit-on-listen-first') && launch === 1) process.exit(0); + listens++; + if (flags.has('listen-fail-first') && listens === 1) return error(m.id, -32603, 'first listen unavailable'); + subscription = m.id; + return send({ + jsonrpc: '2.0', + method: 'notifications/subscriptions/acknowledged', + params: { notifications: { toolsListChanged: true }, _meta: { [SUB]: m.id } }, + }); + case 'tools/list': + // 2026-07-28 list results carry cache hints. + return result(m.id, { + ttlMs: 60000, + cacheScope: 'private', + tools: [ + { + name: 'op', + inputSchema: { type: 'object' }, + outputSchema: { + type: 'object', + properties: { value: { type: flags.has('invalid-schema') ? 'not-a-json-schema-type' : 'integer' } }, + required: ['value'], + }, + }, + { name: 'plain', inputSchema: { type: 'object' } }, + { name: 'err', inputSchema: { type: 'object' } }, + { name: 'touch', inputSchema: { type: 'object' } }, + { name: 'nullframe', inputSchema: { type: 'object' } }, + { name: 'cont', inputSchema: { type: 'object' } }, + { name: 'task', inputSchema: { type: 'object' } }, + { name: 'bare', inputSchema: { type: 'object' } }, + { name: 'die', inputSchema: { type: 'object' } }, + { name: 'vanish', inputSchema: { type: 'object' } }, + { name: 'deafen', inputSchema: { type: 'object' } }, + ], + }); + case 'tools/call': { + const name = m.params?.name; + if (name === 'op') return result(m.id, { content: [], structuredContent: { value: 'not an integer' } }); + if (name === 'err') return error(m.id, -32603, 'boom', { why: 'on purpose' }); + if (name === 'touch') { + if (subscription !== null) { + send({ jsonrpc: '2.0', method: 'notifications/tools/list_changed', params: { _meta: { [SUB]: subscription } } }); + } + return result(m.id, { content: [{ type: 'text', text: subscription === null ? 'no subscription' : 'touched' }] }); + } + if (name === 'die') { setTimeout(() => process.exit(5), 10); return result(m.id, { content: [{ type: 'text', text: 'bye' }] }); } + if (name === 'vanish') process.exit(6); + if (name === 'deafen') { + result(m.id, { content: [{ type: 'text', text: 'deaf now' }] }); + // Node keeps a stdio descriptor open when its stream is destroyed, so + // the read end is closed by hand once the stream is done with it. + process.stdin.once('close', () => { + try { closeSync(0); } catch { /* already closed */ } + note({ event: 'deaf' }); + }); + lines.close(); + process.stdin.destroy(); + // Nothing else keeps the process up once its stdin is gone. + setInterval(() => {}, 1 << 30); + return; + } + if (name === 'task') return send({ jsonrpc: '2.0', id: m.id, result: { resultType: 'task', taskId: 'task-1' } }); + if (name === 'bare') return send({ jsonrpc: '2.0', id: m.id, result: { content: [{ type: 'text', text: 'no resultType' }] } }); + if (name === 'nullframe') { + process.stdout.write('null\n[1,2]\n"just a string"\n{"no":"jsonrpc"}\n'); + return result(m.id, { content: [{ type: 'text', text: 'after the noise' }] }); + } + if (name === 'cont') { + const { leg1, leg2 } = m.params?.arguments ?? {}; + if (m.params?.requestState === undefined) { + if (leg1 === 'elicit') { + return send({ + jsonrpc: '2.0', + id: m.id, + result: { + resultType: 'input_required', + inputRequests: { confirm: { method: 'elicitation/create', params: { message: 'Proceed?', requestedSchema: { type: 'object', properties: {} } } } }, + }, + }); + } + send({ jsonrpc: '2.0', id: m.id, result: { resultType: 'input_required', requestState: `state-for-${leg2}` } }); + if (leg1 === 'exit') setTimeout(() => process.exit(5), 10); + return; + } + if (leg2 === 'error') return error(m.id, -32603, 'second leg failed', { leg: 2, state: m.params.requestState }); + if (leg2 === 'hang') return; + if (leg2 === 'again') return send({ jsonrpc: '2.0', id: m.id, result: { resultType: 'input_required', requestState: `${m.params.requestState}+` } }); + return result(m.id, { content: [{ type: 'text', text: `leg 2 done (${m.params.requestState})` }] }); + } + return result(m.id, { content: [{ type: 'text', text: 'plain' }] }); + } + default: + if (m.id !== undefined) error(m.id, -32601, `Method not found: ${m.method}`); + } +}); +process.stdin.on('end', () => process.exit(0)); diff --git a/test/fixtures/modern-tools.mjs b/test/fixtures/modern-tools.mjs new file mode 100644 index 000000000..242f7b9a6 --- /dev/null +++ b/test/fixtures/modern-tools.mjs @@ -0,0 +1,87 @@ +// Tools for the modern-MCP fixture servers (SDK 2.x McpServer), shared by the +// stdio fixture (modern-mcp-server.mjs) and the in-process HTTP test server. +import { appendFileSync } from 'node:fs'; +import { fromJsonSchema, McpServer } from '@modelcontextprotocol/server'; + +const obj = (properties = {}) => fromJsonSchema({ type: 'object', properties }); + +/** + * @param {{ log?: (line: string) => void, eventsFile?: string }} [opts] + * `eventsFile` (optional) receives one line per notable server-side event + * (`slow-start`, `slow-aborted`, `slow-finished`) so a test can observe what + * the server saw, independently of what the client reports. + */ +export function createServer(opts = {}) { + const note = (line) => { + opts.log?.(line); + if (opts.eventsFile) appendFileSync(opts.eventsFile, `${line}\n`); + }; + const server = new McpServer( + { name: 'modern-fixture', version: '1.0.0' }, + { capabilities: { tools: { listChanged: true } } }, + ); + server.registerTool('echo', { description: 'Echo text back', inputSchema: obj({ text: { type: 'string' } }) }, + async (args) => ({ content: [{ type: 'text', text: `echo:${args.text}` }] })); + server.registerTool('structured_only', { description: 'Only structured content', inputSchema: obj() }, + async () => ({ content: [], structuredContent: { answer: 42, ok: false, nothing: null } })); + server.registerTool('structured_and_text', { description: 'Summary text plus structured data', inputSchema: obj() }, + async () => ({ content: [{ type: 'text', text: 'Found 2 rows.' }], structuredContent: { rows: [{ id: 1 }, { id: 2 }] } })); + server.registerTool('structured_zero', { description: 'Falsy structured value', inputSchema: obj() }, + async () => ({ content: [], structuredContent: 0 })); + server.registerTool('sized', { + description: 'Structured content padded to `pad` characters, optionally a tool error', + inputSchema: obj({ pad: { type: 'number' }, isError: { type: 'boolean' } }), + }, async (args) => ({ + content: [], + structuredContent: { pad: 'x'.repeat(Number(args.pad ?? 0)) }, + ...(args.isError ? { isError: true } : {}), + })); + server.registerTool('scored_image', { description: 'An image with machine-readable scores', inputSchema: obj() }, + async () => ({ + content: [{ type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }], + structuredContent: { score: 0.9, label: 'cat' }, + })); + const media = [ + { type: 'text', text: 'before' }, + { type: 'resource_link', uri: 'file:///srv/report.pdf', name: 'report.pdf', mimeType: 'application/pdf' }, + { type: 'resource', resource: { uri: 'memo://note', mimeType: 'text/plain', text: 'embedded note text' } }, + { type: 'resource', resource: { uri: 'memo://bin', mimeType: 'application/octet-stream', blob: Buffer.from('binary!').toString('base64') } }, + { type: 'audio', data: Buffer.from('RIFF0000WAVEfmt ').toString('base64'), mimeType: 'audio/wav' }, + { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }, + ]; + server.registerTool('media', { description: 'Mixed standard content', inputSchema: obj() }, + async () => ({ content: media })); + server.registerTool('structured_media', { description: 'Mixed standard content with structured data', inputSchema: obj() }, + async () => ({ content: media, structuredContent: { blocks: media.length } })); + server.registerTool('fail', { description: 'Reports a tool error', inputSchema: obj() }, + async () => ({ content: [{ type: 'text', text: 'the tool failed on purpose' }], isError: true })); + server.registerTool('slow', { description: 'Sleeps, then answers', inputSchema: obj({ ms: { type: 'number' } }) }, + async (args, ctx) => { + const ms = Number(args.ms ?? 1000); + const signal = ctx?.mcpReq?.signal; + note('slow-start'); + signal?.addEventListener('abort', () => note('slow-aborted')); + await new Promise((resolve) => setTimeout(resolve, ms)); + note('slow-finished'); + return { content: [{ type: 'text', text: `slept ${ms}` }] }; + }); + server.registerTool('env', { description: 'Reports selected environment variables', inputSchema: obj() }, + async () => ({ content: [{ type: 'text', text: JSON.stringify({ + declared: process.env.FIXTURE_DECLARED ?? null, + hostOnly: process.env.FIXTURE_HOST_ONLY ?? null, + path: Boolean(process.env.PATH), + }) }] })); + let added = 0; + server.registerTool('add_tool', { description: 'Registers another tool (list change)', inputSchema: obj() }, + async () => { + const name = `added_${++added}`; + server.registerTool(name, { description: 'added at runtime', inputSchema: obj() }, + async () => ({ content: [{ type: 'text', text: name }] })); + return { content: [{ type: 'text', text: `registered ${name}` }] }; + }); + server.registerTool('shout', { description: 'Writes a line to stderr', inputSchema: obj({ text: { type: 'string' } }) }, + async (args) => { process.stderr.write(`shout: ${args.text}\n`); return { content: [{ type: 'text', text: 'shouted' }] }; }); + server.registerTool('die', { description: 'Exits the server process', inputSchema: obj() }, + async () => { setTimeout(() => process.exit(7), 10); return { content: [{ type: 'text', text: 'bye' }] }; }); + return server; +} diff --git a/test/fixtures/surgery-marks-mcpl-server.mjs b/test/fixtures/surgery-marks-mcpl-server.mjs index 4d2a0a117..1eb76c2ed 100644 --- a/test/fixtures/surgery-marks-mcpl-server.mjs +++ b/test/fixtures/surgery-marks-mcpl-server.mjs @@ -47,6 +47,7 @@ function answer(message) { function handle(message) { if (message.method === 'initialize') { reply(message.id, { + protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { diff --git a/test/host-quiesce.test.ts b/test/host-quiesce.test.ts index 0144d53c0..f095ca04b 100644 --- a/test/host-quiesce.test.ts +++ b/test/host-quiesce.test.ts @@ -767,7 +767,7 @@ process.stdin.on('data', (c) => { if (!line.trim()) continue; let m; try { m = JSON.parse(line); } catch { continue; } const reply = (result) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: m.id, result }) + '\\n'); - if (m.method === 'initialize') reply({ capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); + if (m.method === 'initialize') reply({ protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); else if (m.method === 'tools/list') reply({ tools: [] }); else if (m.id === 900) log('push-response', { error: m.error ?? null }); else if (m.id === 901) log('resume-response', { result: m.result ?? null, error: m.error ?? null }); diff --git a/test/mcp-modern-connection.test.ts b/test/mcp-modern-connection.test.ts new file mode 100644 index 000000000..615d807fe --- /dev/null +++ b/test/mcp-modern-connection.test.ts @@ -0,0 +1,214 @@ +/** + * The modern engine (room-284, shelf-487) over stdio, against an SDK 2.x + * fixture that refuses legacy openings, as the house services do. What + * Connectome owns here, rather than the SDK: + * - one launch per connect, through Connectome's own spawner (env allowlist, + * stderr), instead of the SDK's sibling-probe launch; + * - one deadline per call, which ends as `no-response` with cancellation + * requested, and is never replayed; + * - results kept by presence (structured `0` is a value); + * - reconnect with the legacy backoff settings, and list-change awareness; + * - the request-outcome contract on failures. + */ +import { test, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, readFileSync, existsSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { ModernMcpConnection } from '../src/mcpl/modern-connection.js'; +import { McplRequestError } from '../src/mcpl/server-connection.js'; +import type { McplServerConfig } from '../src/mcpl/types.js'; + +const here = dirname(fileURLToPath(import.meta.url)); +const SERVER = join(here, 'fixtures', 'modern-mcp-server.mjs'); + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +function scratch(): string { + const dir = mkdtempSync(join(tmpdir(), 'modern-mcp-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + return dir; +} + +const lines = (file: string) => (existsSync(file) ? readFileSync(file, 'utf8').split('\n').filter(Boolean) : []); +const until = async (cond: () => boolean, ms = 5000) => { + const deadline = Date.now() + ms; + while (!cond() && Date.now() < deadline) await new Promise((r) => setTimeout(r, 20)); + assert.ok(cond(), 'condition reached in time'); +}; + +function config(dir: string, extra: Partial = {}): McplServerConfig { + return { + id: 'modern', + command: process.execPath, + args: [SERVER, join(dir, 'starts.log'), join(dir, 'events.log')], + protocol: 'modern', + ...extra, + }; +} + +async function connect(c: McplServerConfig): Promise { + const connection = await ModernMcpConnection.connect(c); + cleanups.push(() => connection.close()); + return connection; +} + +test('connects once over Connectome\'s spawner and speaks 2026-07-28', async () => { + const dir = scratch(); + const stderr: string[] = []; + const connection = await connect(config(dir)); + connection.on('stderr', ({ line }: { line: string }) => stderr.push(line)); + assert.equal(connection.isConnected, true); + assert.equal(connection.protocolVersion, '2026-07-28'); + assert.equal(connection.transportKind, 'stdio'); + const names = (await connection.listTools()).map((t) => t.name); + assert.ok(names.includes('echo') && names.includes('structured_only'), names.join()); + const echo = await connection.callTool('echo', { text: 'hi' }); + assert.deepEqual(echo.content, [{ type: 'text', text: 'echo:hi' }]); + // In-place discovery: the SDK's sibling probe would have made this 2. + assert.equal(lines(join(dir, 'starts.log')).length, 1); + // The child's stderr surfaces as lines, as a legacy child's does, + // startup lines included: they came before anyone could listen. + await until(() => stderr.some((l) => /^modern fixture \d+ up$/.test(l))); + await connection.callTool('shout', { text: 'hello' }); + await until(() => stderr.includes('shout: hello')); +}); + +test('the child gets the env allowlist plus declared env, as legacy children do', async () => { + const dir = scratch(); + process.env.FIXTURE_HOST_ONLY = 'leaked'; + cleanups.push(() => { delete process.env.FIXTURE_HOST_ONLY; }); + const connection = await connect(config(dir, { env: { FIXTURE_DECLARED: 'yes' } })); + const result = await connection.callTool('env', {}); + const seen = JSON.parse((result.content[0] as { text: string }).text); + assert.deepEqual(seen, { declared: 'yes', hostOnly: null, path: true }); +}); + +test('results are kept by presence: structured-only, falsy structured, tool errors', async () => { + const connection = await connect(config(scratch())); + const only = await connection.callTool('structured_only', {}); + assert.deepEqual(only.content, []); + assert.deepEqual(only.structuredContent, { answer: 42, ok: false, nothing: null }); + const zero = await connection.callTool('structured_zero', {}); + assert.ok('structuredContent' in zero); + assert.equal(zero.structuredContent, 0); + const plain = await connection.callTool('echo', { text: 'x' }); + assert.ok(!('structuredContent' in plain)); + const fail = await connection.callTool('fail', {}); + assert.equal(fail.isError, true); +}); + +test('a deadline requests cancellation and reports no-response, outcome unknown', async () => { + const dir = scratch(); + const connection = await connect(config(dir, { requestTimeoutMs: 300 })); + await connection.callTool('echo', { text: 'warm' }); // the slow call is not the connection's first request + // The tool would run far past the deadline, so the call rejecting while + // the tool hasn't finished shows it ended at the deadline, whatever the + // runner's load did to the wall clock. + await assert.rejects(connection.callTool('slow', { ms: 20_000 }), (err: unknown) => { + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /did not answer tools\/call "slow" within 300ms/); + assert.match(err.message, /Cancellation was requested; the outcome is unknown/); + return true; + }); + assert.ok(!lines(join(dir, 'events.log')).includes('slow-finished'), 'rejected at the deadline, not at the tool\'s end'); + // The server saw the cancellation: the request was cancelled, not merely abandoned. + await until(() => lines(join(dir, 'events.log')).includes('slow-aborted')); +}); + +test('a list change is announced, and the next listing has the new tool', async () => { + const connection = await connect(config(scratch())); + let changes = 0; + connection.on('tools-list-changed', () => changes++); + await connection.callTool('add_tool', {}); + await until(() => changes > 0); + assert.ok((await connection.listTools()).some((t) => t.name === 'added_1')); +}); + +test('a lost child reconnects with backoff, relaunching once, and lists again', async () => { + const dir = scratch(); + const connection = await connect(config(dir, { reconnect: true, reconnectIntervalMs: 50, reconnectMaxIntervalMs: 100 })); + const events: string[] = []; + connection.on('close', () => events.push('close')); + connection.on('reconnect', () => events.push('reconnect')); + await connection.callTool('die', {}); + await until(() => events.includes('reconnect')); + assert.deepEqual(events, ['close', 'reconnect']); + assert.equal(connection.isConnected, true); + assert.ok((await connection.listTools()).some((t) => t.name === 'echo')); + assert.equal(lines(join(dir, 'starts.log')).length, 2); +}); + +test('without reconnect a lost child stays down, and calls are refused before sending', async () => { + const connection = await connect(config(scratch())); + let closed = false; + connection.on('close', () => { closed = true; }); + await connection.callTool('die', {}); + await until(() => closed); + assert.equal(connection.isConnected, false); + assert.equal(connection.willReconnect, false); + await assert.rejects(connection.callTool('echo', { text: 'x' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'not-sent'); + return true; + }); +}); + +test('a first connect that fails keeps retrying with reconnect, and close stops it', async () => { + const dir = scratch(); + const failures: unknown[] = []; + const connection = await connect({ + id: 'nothing', + command: process.execPath, + args: ['-e', 'process.exit(1)'], + protocol: 'modern', + reconnect: true, + reconnectIntervalMs: 30, + reconnectMaxIntervalMs: 60, + }); + connection.on('connect-failed', (e) => failures.push(e)); + connection.on('reconnect-failed', (e) => failures.push(e)); + assert.equal(connection.isConnected, false); + await until(() => failures.length >= 2); + await connection.close(); + const count = failures.length; + await new Promise((r) => setTimeout(r, 200)); + assert.equal(failures.length, count, 'no attempts after close'); + void dir; +}); + +test('a modern stdio server whose command can\'t be spawned fails its connect at once, with its spawn error (R1-4)', async () => { + const began = Date.now(); + await assert.rejects( + ModernMcpConnection.connect({ id: 'typo', command: 'no-such-command-r1-4', protocol: 'modern' }), + /^Error: MCPL server "typo" could not be started: spawn no-such-command-r1-4 ENOENT$/, + ); + const tookMs = Date.now() - began; + assert.ok(tookMs < 5000, `failed after ${tookMs} ms, not at the discover probe's 30 s timeout`); +}); + +test('configuration mistakes throw before anything is launched', async () => { + // x.invalid never resolves, so a dial would fail with a connect error instead. + await assert.rejects(ModernMcpConnection.connect({ id: 'x', url: 'https://x.invalid/mcp', protocol: 'modern' }), /applies only to stdio/); + // Stdio refusals use the real fixture, whose start log records any launch. + const dir = scratch(); + const refused: Array<[Partial, RegExp]> = [ + [{ requestTimeoutMs: 0 }, /requestTimeoutMs must be an integer/], + [{ protocol: undefined }, /not configured for modern MCP/], + [{ token: 't' }, /"token" applies only to a network \(url\) server/], + [{ url: 'https://x.invalid/mcp' }, /"command" and "url" are both set/], + ]; + for (const [extra, pattern] of refused) { + await assert.rejects(ModernMcpConnection.connect(config(dir, extra)), pattern, JSON.stringify(extra)); + } + assert.deepEqual(lines(join(dir, 'starts.log')), [], 'nothing was launched'); + // The same fixture, configured right, does launch: the log above can tell. + await connect(config(dir)); + assert.equal(lines(join(dir, 'starts.log')).length, 1); +}); diff --git a/test/mcp-modern-framework.test.ts b/test/mcp-modern-framework.test.ts new file mode 100644 index 000000000..acf85fcfe --- /dev/null +++ b/test/mcp-modern-framework.test.ts @@ -0,0 +1,570 @@ +/** + * A modern-MCP server inside a real framework: tool listing and status, both + * dispatch paths, the script view of structured results, an MCP-only legacy + * peer read the same standard way, and the host lifetime contract: + * - a modern call already in flight completes while the host quiesces; + * - a list change while quiesced refreshes the inventory, and its wake parks + * until resume; + * - a modern child lost while quiesced reconnects and relists without + * starting a turn. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, readFileSync, existsSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { AgentFramework, WorkspaceModule } from '../src/index.js'; +import type { EventResponse, Module, ModuleContext, ProcessEvent, ProcessState, ToolDefinition, ToolResult } from '../src/index.js'; +import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; + +const here = dirname(fileURLToPath(import.meta.url)); +const MODERN = join(here, 'fixtures', 'modern-mcp-server.mjs'); + +/** Turns every external message into a wake, so tool-list diffs wake agents. */ +class Waker implements Module { + readonly name = 'waker'; + async start(_ctx: ModuleContext): Promise {} + async stop(): Promise {} + getTools(): ToolDefinition[] { return []; } + async handleToolCall(): Promise { return { success: false, error: 'no tools', isError: true }; } + async onProcess(event: ProcessEvent, _state: ProcessState): Promise { + if (event.type === 'external-message') { + return { addMessages: [{ participant: 'User', content: [{ type: 'text', text: 'go' }] }], requestInference: true }; + } + return {}; + } +} + +async function waitFor(what: string, cond: () => boolean, timeoutMs = 8000): Promise { + const deadline = Date.now() + timeoutMs; + while (!cond()) { + if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`); + await new Promise((r) => setTimeout(r, 25)); + } +} + +const lines = (file: string) => (existsSync(file) ? readFileSync(file, 'utf8').split('\n').filter(Boolean) : []); + +interface Harness { framework: AgentFramework; membrane: MockMembrane; dir: string } + +async function withModern( + fn: (h: Harness) => Promise, + extraServers: Record[] = [], + extraModules: Module[] = [], +): Promise { + const dir = mkdtempSync(join(tmpdir(), 'modern-framework-')); + const membrane = new MockMembrane(); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: membrane.asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [new Waker(), ...extraModules], + mcplServers: [ + { + id: 'modern', + command: process.execPath, + args: [MODERN, join(dir, 'starts.log'), join(dir, 'events.log')], + protocol: 'modern', + reconnect: true, + reconnectIntervalMs: 50, + reconnectMaxIntervalMs: 100, + }, + ...(extraServers as never[]), + ], + }); + try { + await fn({ framework, membrane, dir }); + } finally { + await framework.stop(); + rmSync(dir, { recursive: true, force: true }); + } +} + +const call = (name: string, input: Record = {}) => ({ id: `t-${name}-${Math.random()}`, name, input }); + +test('modern tools are listed under the prefix, and status names family, revision and transport', async () => { + await withModern(async ({ framework }) => { + const names = framework.getAllTools().map((t) => t.name); + assert.ok(names.includes('mcpl--modern--echo'), names.join()); + const [status] = framework.listMcplServers(); + assert.equal(status!.family, 'modern'); + assert.equal(status!.protocolVersion, '2026-07-28'); + assert.equal(status!.transport, 'stdio'); + assert.equal(status!.connected, true); + assert.ok(status!.toolCount >= 5); + assert.deepEqual(status!.effectiveGrant, []); + }); +}); + +test('direct path: data stays the raw content; tool errors fail; structured by presence', async () => { + await withModern(async ({ framework }) => { + const only = await framework.executeToolCall(call('mcpl--modern--structured_only')); + assert.equal(only.success, true); + assert.deepEqual(only.data, [], 'the raw content array, as module callers have always received it'); + assert.deepEqual(only.structured, { answer: 42, ok: false, nothing: null }); + + const zero = await framework.executeToolCall(call('mcpl--modern--structured_zero')); + assert.ok('structured' in zero); + assert.equal(zero.structured, 0); + + const echo = await framework.executeToolCall(call('mcpl--modern--echo', { text: 'x' })); + assert.deepEqual(echo, { success: true, data: [{ type: 'text', text: 'echo:x' }] }); + + const fail = await framework.executeToolCall(call('mcpl--modern--fail')); + assert.equal(fail.success, false); + assert.equal(fail.isError, true); + assert.equal(fail.error, 'the tool failed on purpose'); + + const media = await framework.executeToolCall(call('mcpl--modern--media')); + assert.deepEqual((media.data as Array<{ type: string }>).map((b) => b.type), + ['text', 'resource_link', 'resource', 'resource', 'audio', 'image'], 'every block as sent'); + }); +}); + +test('model path: a payload the host cannot retain is reported as an incomplete result', async () => { + await withModern(async ({ framework, membrane }) => { + framework.start(); + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'toolu_01MODERNMEDIA00000000000', name: 'mcpl--modern--media', input: {} }, + ] as never, 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Done.' }])); + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as never); + await waitFor('the tool round', () => (membrane.lastStream?.receivedToolResults.length ?? 0) >= 1); + const [delivered] = membrane.lastStream!.receivedToolResults[0] as Array<{ content: unknown; isError: boolean }>; + // No workspace here: audio and the blob can be neither shown nor kept. + // That makes the result a failure, read as its text alone, so the inline + // image is a payload too, and can't be kept either. + assert.equal(delivered!.isError, true); + const text = JSON.stringify(delivered!.content); + assert.match(text, /result incomplete: 3 payload\(s\) could not be retained/); + assert.match(text, /\[image: image\/png, 8 B\. Not retained: a failed result is read as its text alone, and no workspace is mounted to save it\]/); + assert.match(text, /resource link \\"report\.pdf\\": file:\/\/\/srv\/report\.pdf, application\/pdf; not fetched/); + assert.match(text, /embedded note text/); + }); +}); + +test('model path: the result reaches the model, structured-only as JSON', async () => { + await withModern(async ({ framework, membrane }) => { + framework.start(); + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'toolu_01MODERNSTRUCTURED000000', name: 'mcpl--modern--structured_only', input: {} }, + ] as never, 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Done.' }])); + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as never); + await waitFor('the tool round', () => (membrane.lastStream?.receivedToolResults.length ?? 0) >= 1); + // The framework renders string data JSON-encoded, as it does for every + // string tool result; decoded, it is the structured value's JSON. + const [delivered] = membrane.lastStream!.receivedToolResults[0] as Array<{ content: string }>; + assert.deepEqual(JSON.parse(JSON.parse(delivered!.content)), { answer: 42, ok: false, nothing: null }); + }); +}); + +test('scripts get structured results whole, as one JSON object', async () => { + await withModern(async ({ framework }) => { + framework.start(); + const handle = (framework as unknown as { + handleScriptToolCall: (agent: string, tool: string, args: Record) => Promise; + }).handleScriptToolCall.bind(framework); + const out = JSON.parse(await handle('agent', 'mcpl--modern--structured_and_text', {})); + assert.deepEqual(out, { + content: [{ type: 'text', text: 'Found 2 rows.' }], + structuredContent: { rows: [{ id: 1 }, { id: 2 }] }, + isError: false, + }); + // Image plus scores: both views reach the script whole, the image's + // payload included (no placeholder stands in for it). + const scored = JSON.parse(await handle('agent', 'mcpl--modern--scored_image', {})); + assert.deepEqual(scored, { + content: [{ type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }], + structuredContent: { score: 0.9, label: 'cat' }, + isError: false, + }); + // A resource link, embedded resources and audio are ordinary MCP blocks + // from a modern server, not RFC-005 references: the script reads them as + // the server sent them. + const media = JSON.parse(await handle('agent', 'mcpl--modern--structured_media', {})); + assert.deepEqual(media, { + content: [ + { type: 'text', text: 'before' }, + { type: 'resource_link', uri: 'file:///srv/report.pdf', name: 'report.pdf', mimeType: 'application/pdf' }, + { type: 'resource', resource: { uri: 'memo://note', mimeType: 'text/plain', text: 'embedded note text' } }, + { type: 'resource', resource: { uri: 'memo://bin', mimeType: 'application/octet-stream', blob: Buffer.from('binary!').toString('base64') } }, + { type: 'audio', data: Buffer.from('RIFF0000WAVEfmt ').toString('base64'), mimeType: 'audio/wav' }, + { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }, + ], + structuredContent: { blocks: 6 }, + isError: false, + }); + // Without structured content the script contract is unchanged (pinned): + // the history rendering, which JSON-encodes a text result. + assert.equal(await handle('agent', 'mcpl--modern--echo', { text: 'x' }), JSON.stringify('echo:x')); + }); +}); + +// The script cap measures the serialized object's length: `sized` pads +// structuredContent so that length is exact. +const SCRIPT_CAP = 5_000_000; +const sizedObject = (pad: number, isError = false) => ({ + content: [], + structuredContent: { pad: 'x'.repeat(pad) }, + isError, + // A tool error with no text carries the framework's own error text, as + // the model's dispatch words it. + ...(isError ? { + error: 'The "modern" server reported an error for mcpl--modern--sized without saying what went wrong. ' + + 'Whether it did anything is unknown: check before calling it again.', + } : {}), +}); +/** The pad that makes the script's JSON exactly `chars` long. */ +const padFor = (chars: number, isError = false) => chars - JSON.stringify(sizedObject(0, isError)).length; +const scriptHandle = (framework: AgentFramework) => (framework as unknown as { + handleScriptToolCall: (agent: string, tool: string, args: Record) => Promise; +}).handleScriptToolCall.bind(framework); + +async function withWorkspace(fn: (h: Harness & { workspace: WorkspaceModule }) => Promise): Promise { + const board = mkdtempSync(join(tmpdir(), 'modern-board-')); + const workspace = new WorkspaceModule({ mounts: [{ name: 'board', path: board, mode: 'read-write', watch: 'never' }] }); + try { + await withModern((h) => { + // The host wires the store after create(), as conhost does. + workspace.initStore(h.framework.getStore()); + return fn({ ...h, workspace }); + }, [], [workspace]); + } finally { + rmSync(board, { recursive: true, force: true }); + } +} + +/** A saved file's bytes, read back through the workspace as a resident would. */ +async function readSaved(workspace: WorkspaceModule, path: string): Promise { + const read = await workspace.readBinary(path); + assert.ok('data' in read, `readable: ${path} (${'error' in read ? read.error : ''})`); + return read.data.toString('utf8'); +} + +async function savedObject(workspace: WorkspaceModule, savedTo: string): Promise { + assert.match(savedTo, /^board\/tool-results\/\d{4}-\d{2}-\d{2}-script-[0-9a-f-]{36}\.json$/); + return JSON.parse(await readSaved(workspace, savedTo)); +} + +test('script results: exactly at the cap arrive inline; one past it are saved whole, a tool error included', async () => { + await withWorkspace(async ({ framework, workspace }) => { + framework.start(); + const handle = scriptHandle(framework); + + const at = await handle('agent', 'mcpl--modern--sized', { pad: padFor(SCRIPT_CAP) }); + assert.equal(at.length, SCRIPT_CAP); + assert.deepEqual(JSON.parse(at), sizedObject(padFor(SCRIPT_CAP))); + + const over = JSON.parse(await handle('agent', 'mcpl--modern--sized', { pad: padFor(SCRIPT_CAP + 1) })); + assert.deepEqual(Object.keys(over).sort(), ['isError', 'oversized']); + assert.equal(over.isError, false); + assert.equal(over.oversized.chars, SCRIPT_CAP + 1); + assert.deepEqual(await savedObject(workspace, over.oversized.savedTo), sizedObject(padFor(SCRIPT_CAP + 1)), 'the whole object, recoverable'); + + const failed = JSON.parse(await handle('agent', 'mcpl--modern--sized', { pad: padFor(SCRIPT_CAP + 1, true), isError: true })); + assert.equal(failed.isError, true, 'a tool error stays visible without reading the file'); + assert.equal(failed.oversized.chars, SCRIPT_CAP + 1); + assert.notEqual(failed.oversized.savedTo, over.oversized.savedTo); + assert.deepEqual(await savedObject(workspace, failed.oversized.savedTo), sizedObject(padFor(SCRIPT_CAP + 1, true), true)); + }); +}); + +test('script results past the cap: no workspace, or a failed write, is a failure to deliver, never a success', async () => { + const assertUndelivered = (out: { isError: boolean; error: string; oversized: unknown }) => { + assert.equal(out.isError, true); + assert.match(out.error, /exceeds the 5000000-char script result cap and could not be saved to the workspace, so it could not be delivered; the tool may already have completed/); + assert.deepEqual(out.oversized, { chars: SCRIPT_CAP + 1, savedTo: null }); + }; + await withModern(async ({ framework }) => { + framework.start(); + assertUndelivered(JSON.parse(await scriptHandle(framework)('agent', 'mcpl--modern--sized', { pad: padFor(SCRIPT_CAP + 1) }))); + }); + await withWorkspace(async ({ framework, workspace }) => { + framework.start(); + workspace.writeBinary = async () => ({ success: false, error: 'disk full', isError: true }); + assertUndelivered(JSON.parse(await scriptHandle(framework)('agent', 'mcpl--modern--sized', { pad: padFor(SCRIPT_CAP + 1) }))); + }); +}); + +test('saved payloads: call ids that sanitize to one label still get distinct files', async () => { + await withWorkspace(async ({ framework, workspace }) => { + const read = (framework as unknown as { + mcpToolResult: (result: unknown, callId: string, mcplPeer: boolean) => Promise; + }).mcpToolResult.bind(framework); + const audio = (bytes: string) => ({ content: [{ type: 'audio', data: Buffer.from(bytes).toString('base64'), mimeType: 'audio/wav' }] }); + const savedTo = (result: ToolResult) => /saved to workspace file (\S+)\]/.exec(String(result.data))?.[1]; + // `call:a` and `call/a` sanitize alike; so do ids that differ only past the kept prefix. + const long = 'x'.repeat(90); + const results = [ + await read(audio('first'), 'call:a', false), + await read(audio('second'), 'call/a', false), + await read(audio('third'), `${long}-1`, false), + await read(audio('fourth'), `${long}-2`, false), + ]; + const paths = results.map(savedTo); + assert.ok(paths.every((p) => typeof p === 'string'), JSON.stringify(results)); + assert.equal(new Set(paths).size, 4, `distinct paths: ${paths.join(', ')}`); + assert.deepEqual( + await Promise.all(paths.map((p) => readSaved(workspace, p!))), + ['first', 'second', 'third', 'fourth'], + 'each file keeps its own call\'s bytes', + ); + }); +}); + +test("a payload whose workspace write rejects is lost from a completed call's result, with why logged, not a dispatch failure", async (t) => { + await withWorkspace(async ({ framework, workspace }) => { + const read = (framework as unknown as { + mcpToolResult: (result: unknown, callId: string, mcplPeer: boolean) => Promise; + }).mcpToolResult.bind(framework); + const logged: string[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => { + logged.push(args.map((a) => (a instanceof Error ? a.message : String(a))).join(' ')); + }); + const clip = { content: [{ type: 'audio', data: Buffer.from('clip').toString('base64'), mimeType: 'audio/wav' }] }; + + workspace.writeBinary = async () => { throw new Error('disk gone'); }; + const rejected = await read(clip, 'call-1', false); + assert.equal(rejected.success, false); + assert.match(rejected.error!, /saving it to the workspace failed\]/); + + workspace.writeBinary = async () => ({ success: false, error: 'disk full', isError: true }); + const refused = await read(clip, 'call-2', false); + assert.equal(refused.success, false); + assert.match(refused.error!, /saving it to the workspace failed\]/); + + const saves = logged.filter((line) => line.startsWith('[mcp] saving tool payload')); + assert.equal(saves.length, 2, logged.join('\n')); + assert.match(saves[0]!, /^\[mcp\] saving tool payload board\/tool-results\/\S+\.wav \("audio\/wav"\) failed: disk gone$/); + assert.match(saves[1]!, /^\[mcp\] saving tool payload board\/tool-results\/\S+\.wav \("audio\/wav"\) failed: disk full$/); + }); +}); + +test('a modern server deployed with a host\'s MCPL deny-list connects, since what only narrows is already satisfied (R1-3)', async () => { + await withModern(async ({ framework, dir }) => { + await framework.connectMcplServer({ + id: 'overlay', + command: process.execPath, + args: [MODERN, join(dir, 'overlay-starts.log'), join(dir, 'overlay-events.log')], + protocol: 'modern', + // connectome-host's overlay route stamps this on every server it deploys. + disabledCapabilities: ['contextHooks', 'inferenceLifecycle', 'inferenceRequest', 'toolLifecycle'], + disabledFeatureSets: ['*'], + }); + const status = framework.listMcplServers().find((s) => s.id === 'overlay'); + assert.equal(status?.connected, true); + assert.equal(status?.family, 'modern'); + assert.ok(framework.getAllTools().some((t) => t.name === 'mcpl--overlay--echo')); + }); +}); + +test('MCPL policy on a modern server is a configuration error at connect', async () => { + await withModern(async ({ framework }) => { + await assert.rejects( + framework.connectMcplServer({ id: 'bad', url: 'https://example.invalid/mcp', allowHostCommands: true }), + /"allowHostCommands" asks for MCPL behaviour/, + ); + await assert.rejects( + framework.connectMcplServer({ id: 'bad2', url: 'https://example.invalid/mcp', protocol: 'modern' }), + /applies only to stdio/, + ); + // Selector combinations that name no usable transport are errors too, + // not a fall-through to some other opener. + await assert.rejects( + framework.connectMcplServer({ id: 'bad3', url: 'https://example.invalid/mcp', transport: 'websocket' }), + /does not match url/, + ); + await assert.rejects( + framework.connectMcplServer({ id: 'bad4', url: 'ws://127.0.0.1:1/mcpl', transport: 'http' }), + /does not match url/, + ); + await assert.rejects( + framework.connectMcplServer({ id: 'bad5', command: process.execPath, transport: 'http' }), + /transport "http" requires "url"/, + ); + assert.ok(!framework.listMcplServers().some((s) => s.id.startsWith('bad')), 'nothing registered'); + }); +}); + +test('an MCP-only legacy peer gets the same standard reading', async () => { + const LEGACY_MCP_ONLY = ` +let buf = ''; +process.stdin.on('data', (c) => { + buf += c.toString('utf8'); + let i; + while ((i = buf.indexOf('\\n')) >= 0) { + const line = buf.slice(0, i); buf = buf.slice(i + 1); + if (!line.trim()) continue; + let m; try { m = JSON.parse(line); } catch { continue; } + const reply = (result) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: m.id, result }) + '\\n'); + if (m.method === 'initialize') reply({ protocolVersion: '2024-11-05', capabilities: { tools: {} } }); + else if (m.method === 'tools/list') reply({ tools: [{ name: 'docs', inputSchema: { type: 'object' } }] }); + else if (m.method === 'tools/call') reply({ + content: [ + { type: 'resource_link', uri: 'https://example.invalid/a.md', name: 'a.md' }, + { type: 'resource', resource: { uri: 'memo://n', mimeType: 'text/plain', text: 'note body' } }, + ], + structuredContent: { count: 2 }, + }); + } +}); +`; + await withModern(async ({ framework, membrane }) => { + framework.start(); + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'toolu_01LEGACYDOCS000000000000', name: 'mcpl--plain--docs', input: {} }, + ] as never, 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Done.' }])); + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as never); + await waitFor('the tool round', () => (membrane.lastStream?.receivedToolResults.length ?? 0) >= 1); + const [delivered] = membrane.lastStream!.receivedToolResults[0] as Array<{ content: string; isError: boolean }>; + assert.equal(delivered!.isError, false); + // No server text block: the structured value is shown beside the stubs. + assert.equal(JSON.parse(delivered!.content), + '[resource link "a.md": https://example.invalid/a.md; not fetched]\n[resource memo://n, text/plain]\nnote body\n{"count":2}'); + const status = framework.listMcplServers().find((s) => s.id === 'plain')!; + assert.equal(status.family, 'legacy'); + assert.equal(status.protocolVersion, '2024-11-05'); + }, [{ id: 'plain', command: process.execPath, args: ['-e', LEGACY_MCP_ONLY] }]); +}); + +test('quiesce: a modern call already in flight completes, and its result reaches the turn', async () => { + await withModern(async ({ framework, membrane, dir }) => { + framework.start(); + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'toolu_01MODERNSLOW0000000000000', name: 'mcpl--modern--slow', input: { ms: 1500 } }, + ] as never, 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Done.' }])); + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as never); + await waitFor('the slow call to start server-side', () => lines(join(dir, 'events.log')).includes('slow-start')); + + // Quiesce begins while the call is provably still running server-side, + // so its resolving after 'slow-finished' shows it waited for the call. + const quiescing = framework.quiesce({ reason: 'maintenance', timeoutMs: 10_000 }); + assert.ok(!lines(join(dir, 'events.log')).includes('slow-finished'), 'the call is still in flight as quiesce begins'); + const status = await quiescing; + assert.equal(status.quiesced, true); + assert.equal(status.drained, true, 'the turn drained: its modern call completed'); + assert.ok(lines(join(dir, 'events.log')).includes('slow-finished')); + assert.ok(!lines(join(dir, 'events.log')).includes('slow-aborted'), 'quiesce did not cancel it'); + assert.match(JSON.stringify(membrane.lastStream!.receivedToolResults[0]), /slept 1500/); + await framework.resume(); + }); +}); + +test('quiesce: a list change refreshes the inventory, and its wake parks until resume', async () => { + await withModern(async ({ framework, membrane }) => { + framework.start(); + await framework.quiesce({ reason: 'maintenance' }); + const callsBefore = membrane.calls.length; + const gatedBefore = framework.getHostModeStatus().gatedRequests; + + await framework.executeToolCall(call('mcpl--modern--add_tool')); + await waitFor('the refreshed inventory', () => framework.getAllTools().some((t) => t.name === 'mcpl--modern--added_1')); + await waitFor('the wake to park', () => framework.getHostModeStatus().gatedRequests > gatedBefore); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(membrane.calls.length, callsBefore, 'no inference while quiesced'); + + await framework.resume(); + await waitFor('the parked wake to run', () => membrane.calls.length > callsBefore); + }); +}); + +test('quiesce: a lost modern child reconnects and relists without starting a turn', async () => { + await withModern(async ({ framework, membrane, dir }) => { + framework.start(); + await framework.quiesce({ reason: 'maintenance' }); + const callsBefore = membrane.calls.length; + const gatedBefore = framework.getHostModeStatus().gatedRequests; + + await framework.executeToolCall(call('mcpl--modern--die')); + await waitFor('a second launch', () => lines(join(dir, 'starts.log')).length === 2); + await waitFor('reconnected', () => framework.listMcplServers()[0]!.connected); + await waitFor('the relisted inventory', () => framework.getAllTools().some((t) => t.name === 'mcpl--modern--echo')); + await new Promise((r) => setTimeout(r, 200)); + + assert.equal(membrane.calls.length, callsBefore, 'no turn started'); + assert.equal(framework.getHostModeStatus().gatedRequests, gatedBefore, 'no wake parked: the inventory is unchanged'); + assert.deepEqual((await framework.executeToolCall(call('mcpl--modern--echo', { text: 'back' }))).data, [{ type: 'text', text: 'echo:back' }]); + await framework.resume(); + }); +}); + +test('the framework owns a modern connect in flight: stop() ends it, and a second connect is refused', async () => { + const RAW = join(here, 'fixtures', 'modern-raw-server.mjs'); + const dir = mkdtempSync(join(tmpdir(), 'modern-framework-')); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [], + }); + try { + const log = join(dir, 'hang.jsonl'); + const config = { id: 'hang', command: process.execPath, args: [RAW, 'hang-discover', log], protocol: 'modern' as const }; + const pending = framework.connectMcplServer(config); + await waitFor('the launch waiting on discover', () => lines(log).some((l) => l.includes('server/discover'))); + await assert.rejects(framework.connectMcplServer(config), /already registered/); + const pid = (JSON.parse(lines(log)[0]!) as { pid: number }).pid; + // stop() ends the connect in flight rather than waiting for its handshake. + const stopping = Date.now(); + await framework.stop(); + assert.ok(Date.now() - stopping < 10_000, `stop() took ${Date.now() - stopping} ms, as if it waited out the connect`); + const alive = (p: number) => { try { process.kill(p, 0); return true; } catch { return false; } }; + assert.equal(alive(pid), false, 'the launch in flight was reaped by stop()'); + await pending.catch(() => {}); + assert.ok(!framework.listMcplServers().some((s) => s.id === 'hang' && s.connected), 'nothing installed after stop'); + // A connect that arrives after stop() is refused before anything spawns. + const before = lines(log).length; + await assert.rejects( + framework.connectMcplServer({ ...config, id: 'late' }), + /connect MCPL server refused: the framework is stopping/, + ); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(lines(log).length, before, 'no launch after stop'); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +test('a lost modern child is relisted from the new launch: what only the old one listed is gone', async () => { + await withModern(async ({ framework, dir }) => { + framework.start(); + // The first launch registers a tool at runtime; a fresh launch won't have it. + await framework.executeToolCall(call('mcpl--modern--add_tool')); + await waitFor('the first launch\'s added tool', () => framework.getAllTools().some((t) => t.name === 'mcpl--modern--added_1')); + await framework.executeToolCall(call('mcpl--modern--die')); + await waitFor('a second launch', () => lines(join(dir, 'starts.log')).length === 2); + await waitFor('the relist from the new launch', () => + framework.listMcplServers()[0]!.connected + && framework.getAllTools().some((t) => t.name === 'mcpl--modern--echo') + && !framework.getAllTools().some((t) => t.name === 'mcpl--modern--added_1')); + }); +}); + +test("a modern tool's error result is traced as tool:failed, not tool:completed", async () => { + const traces: Array<{ type: string; tool?: string; error?: string }> = []; + const watcher: Module = { + name: 'trace-watcher', + async start(ctx: ModuleContext) { ctx.onTrace((event) => traces.push(event as never)); }, + async stop() {}, + getTools: (): ToolDefinition[] => [], + handleToolCall: async (): Promise => ({ success: false, error: 'no tools', isError: true }), + onProcess: async () => ({}), + }; + await withModern(async ({ framework, membrane }) => { + framework.start(); + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'toolu_01MODERNFAIL0000000000000', name: 'mcpl--modern--fail', input: {} }, + ] as never, 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'Noted.' }])); + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as never); + await waitFor('the tool trace', () => traces.some((t) => (t.type === 'tool:failed' || t.type === 'tool:completed') && t.tool === 'fail')); + const outcome = traces.find((t) => (t.type === 'tool:failed' || t.type === 'tool:completed') && t.tool === 'fail')!; + assert.equal(outcome.type, 'tool:failed'); + assert.match(outcome.error ?? '', /the tool failed on purpose/); + }, [], [watcher]); +}); diff --git a/test/mcp-modern-http.test.ts b/test/mcp-modern-http.test.ts new file mode 100644 index 000000000..6e7a07448 --- /dev/null +++ b/test/mcp-modern-http.test.ts @@ -0,0 +1,283 @@ +/** + * The modern engine over Streamable HTTP (room-284, shelf-487), against an + * in-process SDK 2.x `createMcpHandler` that refuses legacy openings. This is + * the one HTTP smoke path. It covers Connectome's own seams, and leaves the + * SDK's conformance to the SDK: + * - credentials: a cached bearer from `accessProvider`, refreshed once on 401; + * - the required headers reach the server; + * - a deadline aborts the request stream, which the server sees as cancellation; + * - an auth wall is an `error-response` carrying the HTTP status, unless an + * earlier round of the call was answered, when it is a `no-response`. + */ +import { test, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { createServer as createHttpServer, type Server, type ServerResponse } from 'node:http'; +import { Readable } from 'node:stream'; +import { join, dirname } from 'node:path'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { AgentFramework } from '../src/framework.js'; +import { MockMembrane } from './helpers/mock-membrane.js'; +import { fileURLToPath, pathToFileURL } from 'node:url'; +import type { AddressInfo } from 'node:net'; + +import { createMcpHandler } from '@modelcontextprotocol/server'; +import { SdkErrorCode, SdkHttpError } from '@modelcontextprotocol/client'; +import { ModernMcpConnection } from '../src/mcpl/modern-connection.js'; +import { McplRequestError } from '../src/mcpl/server-connection.js'; + +const here = dirname(fileURLToPath(import.meta.url)); + +interface Seen { method: string; auth: string | null; version: string | null; mcpMethod: string | null; mcpName: string | null } + +/** A JSON-RPC request a test answers itself, at the HTTP layer, where an SDK + * server can't be made to answer as the case needs. True once answered. */ +type HandAnswer = (message: { method?: unknown; id?: unknown; params?: Record }, res: ServerResponse) => boolean; + +async function startServer(accepted: (token: string | null) => boolean, answer?: HandAnswer): Promise<{ url: string; seen: Seen[]; events: string[]; server: Server }> { + const { createServer } = await import(pathToFileURL(join(here, 'fixtures', 'modern-tools.mjs')).href) as { + createServer: (opts: { log: (line: string) => void }) => unknown; + }; + const seen: Seen[] = []; + const events: string[] = []; + const handler = createMcpHandler(() => createServer({ log: (line) => events.push(line) }) as never, { legacy: 'reject' }); + const server = createHttpServer(async (req, res) => { + const auth = req.headers.authorization ?? null; + seen.push({ + method: req.method ?? '', + auth, + version: (req.headers['mcp-protocol-version'] as string | undefined) ?? null, + mcpMethod: (req.headers['mcp-method'] as string | undefined) ?? null, + mcpName: (req.headers['mcp-name'] as string | undefined) ?? null, + }); + if (!accepted(auth?.replace(/^Bearer /, '') ?? null)) { + res.writeHead(401, { 'content-type': 'text/plain' }).end('unauthorized'); + return; + } + const chunks: Buffer[] = []; + for await (const chunk of req) chunks.push(chunk as Buffer); + if (answer && req.method === 'POST') { + let message: unknown; + try { message = JSON.parse(Buffer.concat(chunks).toString('utf8')); } catch { message = undefined; } + if (message !== null && typeof message === 'object' && !Array.isArray(message) && answer(message as never, res)) return; + } + const headers = new Headers(); + for (const [key, value] of Object.entries(req.headers)) { + if (typeof value === 'string') headers.set(key, value); + else if (Array.isArray(value)) headers.set(key, value.join(', ')); + } + // The client closing its request stream is the HTTP binding's + // cancellation: carry it into the handler as an abort. + const abort = new AbortController(); + res.on('close', () => { if (!res.writableEnded) abort.abort(); }); + const response = await handler.fetch(new Request(`http://127.0.0.1${req.url}`, { + method: req.method, + headers, + body: req.method === 'GET' || req.method === 'HEAD' ? undefined : Buffer.concat(chunks), + signal: abort.signal, + })); + res.writeHead(response.status, Object.fromEntries(response.headers)); + if (response.body) Readable.fromWeb(response.body as never).pipe(res); + else res.end(); + }); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + const { port } = server.address() as AddressInfo; + return { url: `http://127.0.0.1:${port}/mcp`, seen, events, server }; +} + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +test('HTTP: bearer from accessProvider, one refresh on 401, required headers, calls', async () => { + const { url, seen, server } = await startServer((token) => token === 'token-2'); + cleanups.push(() => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); })); + let issued = 0; + const connection = await ModernMcpConnection.connect({ + id: 'remote', + url, + accessProvider: async () => `token-${++issued}`, + }); + cleanups.push(() => connection.close()); + + assert.equal(connection.transportKind, 'http'); + assert.equal(connection.protocolVersion, '2026-07-28'); + assert.deepEqual((await connection.callTool('echo', { text: 'over http' })).content, [{ type: 'text', text: 'echo:over http' }]); + const only = await connection.callTool('structured_only', {}); + assert.deepEqual(only.structuredContent, { answer: 42, ok: false, nothing: null }); + + assert.equal(issued, 2, 'token-1 was refused once, token-2 cached thereafter'); + const posts = seen.filter((s) => s.method === 'POST'); + assert.ok(posts.every((s) => s.version === '2026-07-28'), 'every POST names the revision'); + assert.ok(posts.every((s) => s.mcpMethod !== null), 'every POST names its method'); + assert.ok(posts.some((s) => s.mcpMethod === 'tools/call' && s.mcpName === 'echo'), 'tools/call names its tool'); + assert.ok(posts.slice(1).every((s) => s.auth === 'Bearer token-2')); +}); + +test('HTTP: a deadline aborts the request stream, and the server sees the cancellation', async () => { + const { url, events, server } = await startServer(() => true); + cleanups.push(() => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); })); + const connection = await ModernMcpConnection.connect({ id: 'remote', url, token: 't', requestTimeoutMs: 300 }); + cleanups.push(() => connection.close()); + await assert.rejects(connection.callTool('slow', { ms: 1500 }), (err: unknown) => { + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /Cancellation was requested/); + return true; + }); + const deadline = Date.now() + 3000; + while (!events.includes('slow-aborted') && Date.now() < deadline) await new Promise((r) => setTimeout(r, 20)); + assert.ok(events.includes('slow-aborted'), `server events: ${events.join(', ')}`); +}); + +test('HTTP: a credential the server keeps refusing fails the connect, naming 401', async () => { + const { url, server } = await startServer(() => false); + cleanups.push(() => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); })); + await assert.rejects(ModernMcpConnection.connect({ id: 'remote', url, token: 'stale' }), /401/); +}); + +test('HTTP: an auth wall on a call is an error-response with the status in data', async () => { + let allow = true; + const { url, server } = await startServer(() => allow); + cleanups.push(() => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); })); + const connection = await ModernMcpConnection.connect({ id: 'remote', url, token: 't' }); + cleanups.push(() => connection.close()); + allow = false; + await assert.rejects(connection.callTool('echo', { text: 'x' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'error-response'); + // The status is HTTP's, not a JSON-RPC code: it rides in data. + assert.equal((err.data as { httpStatus?: number }).httpStatus, 401); + return true; + }); +}); + +test('HTTP: an auth wall on a later round of a call is a no-response, since the first round may have acted', async () => { + // The SDK server can't be made to ask for another round, so `cont`, which + // it doesn't list, is answered here: a state-only input_required for the + // first round, then 401 for every later one. + const states: unknown[] = []; + const { url, server } = await startServer(() => true, (message, res) => { + if (message.method !== 'tools/call' || message.params?.name !== 'cont') return false; + const state = message.params.requestState; + states.push(state); + if (state === undefined) { + res.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({ + jsonrpc: '2.0', id: message.id, result: { resultType: 'input_required', requestState: 'state-1' }, + })); + } else { + res.writeHead(401, { 'content-type': 'text/plain' }).end('unauthorized'); + } + return true; + }); + cleanups.push(() => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); })); + const connection = await ModernMcpConnection.connect({ id: 'remote', url, token: 't' }); + cleanups.push(() => connection.close()); + await assert.rejects(connection.callTool('cont', {}), (err: unknown) => { + assert.ok(err instanceof McplRequestError); + // Not the error-response a refused first round gets: the server took + // the first round, and may have acted on it. + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /asked for another round \(input_required\), and the server refused a later one: HTTP 401/); + return true; + }); + assert.equal(states[0], undefined, 'the first round carried no state'); + assert.ok(states.length >= 2 && states.slice(1).every((s) => s === 'state-1'), `each later round carried it: ${JSON.stringify(states)}`); +}); + +test('HTTP: an echoed path in a tool transport failure is redacted, but a JSON-RPC error is the server answer', async (t) => { + const consoleLines: string[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => consoleLines.push(args.map(String).join(' '))); + const secret = 'pathSecret0123456789'; + let rpc = true; + const { url, server } = await startServer(() => true, (message, res) => { + if (message.method !== 'tools/call') return false; + if (rpc) { + res.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({ jsonrpc: '2.0', id: message.id, error: { code: -32603, message: secret } })); + } else { + res.writeHead(500, { 'content-type': 'text/plain' }).end(`Cannot POST /api/mcp/s/${secret}/mcp`); + } + return true; + }); + cleanups.push(() => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); })); + const connection = await ModernMcpConnection.connect({ id: 'remote', url: url.replace('/mcp', `/api/mcp/s/${secret}/mcp`) }); + cleanups.push(() => connection.close()); + await assert.rejects(connection.callTool('echo', {}), (error: unknown) => { + assert.ok(error instanceof McplRequestError); + assert.match(error.message, new RegExp(secret)); + assert.equal(error.outcome, 'error-response'); + return true; + }); + rpc = false; + await assert.rejects(connection.callTool('echo', {}), (error: unknown) => { + assert.ok(error instanceof McplRequestError); + assert.match(error.message, /Cannot POST \/api\/mcp\/s\/redacted\/mcp/); + assert.ok(!error.message.includes(secret)); + assert.equal(error.outcome, 'no-response'); + return true; + }); + assert.ok(consoleLines.some(line => line.includes('Cannot POST /api/mcp/s/redacted/mcp'))); + assert.ok(!consoleLines.join('\n').includes(secret)); + const dir = mkdtempSync(join(tmpdir(), 'mcp-tool-transport-')); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], + modules: [], + mcplServers: [{ id: 'remote', url: url.replace('/mcp', `/api/mcp/s/${secret}/mcp`) }], + }); + try { + const result = await framework.executeToolCall({ id: 'echo-failed', name: 'mcpl--remote--echo', input: {} }); + assert.equal(result.success, false); + assert.match(result.error!, /Cannot POST \/api\/mcp\/s\/redacted\/mcp/); + assert.ok(!JSON.stringify(result).includes(secret), 'the model-facing tool result contains no path credential'); + } finally { + await framework.stop(); + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("HTTP: a 403 refusal's response body, which can echo the url, stays out of the request error's data", () => { + // The SDK attaches the body as `text` to its step-up 403; staging step-up + // takes an OAuth provider, so the refusal is handed to the connection's + // reading of a failed request directly. + const connection = Object.create(ModernMcpConnection.prototype) as { + id: string; config: unknown; + requestFailure(what: string, record: unknown, error: unknown): Error; + }; + connection.id = 'remote'; + connection.config = { id: 'remote', url: 'https://h/api/mcp/s/pathSecret0123456789/mcp' }; + const refusal = new SdkHttpError(SdkErrorCode.ClientHttpForbidden, 'Server returned 403 insufficient_scope', { + status: 403, statusText: 'Forbidden', text: 'denied for /api/mcp/s/pathSecret0123456789/mcp', + }); + const err = connection.requestFailure('tools/call "echo"', { method: 'tools/call', handedOff: true, answer: 'none' }, refusal); + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'error-response'); + assert.equal((err.data as { httpStatus?: number }).httpStatus, 403); + assert.equal((err.data as { text?: unknown }).text, undefined); + assert.ok(!JSON.stringify(err.data).includes('pathSecret'), JSON.stringify(err.data)); +}); + +for (const [what, headers] of [ + ['a plain 403', {}], + ['an insufficient_scope 403', { 'www-authenticate': 'Bearer error="insufficient_scope", scope="tools"' }], +] as const) { + test(`HTTP: ${what} on a first-round call is an error-response with the status in data, over the wire`, async () => { + const { url, server } = await startServer(() => true, (message, res) => { + if (message.method !== 'tools/call') return false; + res.writeHead(403, { 'content-type': 'text/plain', ...headers }).end('forbidden here'); + return true; + }); + cleanups.push(() => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); })); + const connection = await ModernMcpConnection.connect({ id: 'remote', url, token: 't' }); + cleanups.push(() => connection.close()); + await assert.rejects(connection.callTool('echo', { text: 'x' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'error-response', err.message); + assert.equal((err.data as { httpStatus?: number }).httpStatus, 403); + assert.equal((err.data as { text?: unknown }).text, undefined); + return true; + }); + }); +} + diff --git a/test/mcp-modern-outcomes.test.ts b/test/mcp-modern-outcomes.test.ts new file mode 100644 index 000000000..b948f3697 --- /dev/null +++ b/test/mcp-modern-outcomes.test.ts @@ -0,0 +1,840 @@ +/** + * The modern engine's request outcomes and lifetime against what actually + * crossed the wire (room-284 review, Theo #53300 / Petra #53269). A + * hand-written server logs every request, and each case checks the reported + * outcome against that log: + * - an SDK-local rejection before dispatch is `not-sent` (0 calls on the wire); + * - a result that fails validation is an answer that can't be used (1 call, + * answered), never a claimed server error; + * - a JSON-RPC error on the wire is `error-response` with its code and data; + * - a call that ends after an `input_required` answer, before a final one, is + * `no-response`: at the deadline (with nothing in flight to cancel), with + * the connection gone, or when this client can't go on (PR #262 review); + * so is one whose answer has a result type this revision doesn't define, + * or none; + * - a list-change subscription refused at connect is reopened; + * - `close()` during a reconnect handshake ends the launch it was waiting on. + */ +import { test, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, readFileSync, existsSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { SdkError, SdkErrorCode, SdkHttpError, UnsupportedProtocolVersionError } from '@modelcontextprotocol/client'; +import { ModernMcpConnection } from '../src/mcpl/modern-connection.js'; +import { StdioTransport } from '../src/mcpl/transport.js'; +import { McplProtocolVersionError, McplRequestError } from '../src/mcpl/server-connection.js'; +import { MAX_TIMER_MS } from '../src/mcpl/protocol-family.js'; +import type { McplServerConfig } from '../src/mcpl/types.js'; + +const here = dirname(fileURLToPath(import.meta.url)); +const SERVER = join(here, 'fixtures', 'modern-raw-server.mjs'); + +interface LogEntry { event: string; method?: string; id?: unknown; tool?: string; pid?: number; launch: number } + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +function scratchLog(): string { + const dir = mkdtempSync(join(tmpdir(), 'modern-raw-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + return join(dir, 'wire.jsonl'); +} + +const wire = (log: string): LogEntry[] => + existsSync(log) ? readFileSync(log, 'utf8').split('\n').filter(Boolean).map((l) => JSON.parse(l) as LogEntry) : []; +const calls = (log: string, tool: string) => wire(log).filter((e) => e.method === 'tools/call' && e.tool === tool).length; + +async function until(cond: () => boolean, what: string, ms = 5000): Promise { + const deadline = Date.now() + ms; + while (!cond() && Date.now() < deadline) await new Promise((r) => setTimeout(r, 20)); + assert.ok(cond(), `in time: ${what}`); +} + +async function connect(flags: string, log: string, extra: Partial = {}): Promise { + const connection = await ModernMcpConnection.connect({ + id: 'raw', + command: process.execPath, + args: [SERVER, flags, log], + protocol: 'modern', + ...extra, + }); + cleanups.push(() => connection.close()); + return connection; +} + +test('an outputSchema the SDK cannot compile rejects before dispatch: not-sent, nothing on the wire', async () => { + const log = scratchLog(); + const connection = await connect('invalid-schema', log); + await connection.listTools(); + await assert.rejects(connection.callTool('op', {}), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'not-sent'); + assert.match(err.message, /tools\/call "op" was not sent: .*invalid outputSchema/); + return true; + }); + assert.equal(calls(log, 'op'), 0); +}); + +test('a result that fails its outputSchema was answered: a plain error, not a server error', async () => { + const log = scratchLog(); + const connection = await connect('', log); + await connection.listTools(); + await assert.rejects(connection.callTool('op', {}), (err: unknown) => { + assert.ok(!(err instanceof McplRequestError), 'no request outcome is claimed'); + assert.match((err as Error).message, /answered tools\/call "op", but the result can't be used: .*output schema/); + return true; + }); + assert.equal(calls(log, 'op'), 1); +}); + +test('a JSON-RPC error on the wire is error-response with its data, and its code in the message', async () => { + const log = scratchLog(); + const connection = await connect('', log); + await connection.listTools(); + await assert.rejects(connection.callTool('err', {}), (err: unknown) => { + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'error-response'); + assert.deepEqual(err.data, { why: 'on purpose' }); + assert.match(err.message, /returned error for tools\/call "err": \[-32603\] boom/); + return true; + }); + // The plain call on the same connection is unaffected. + assert.deepEqual((await connection.callTool('plain', {})).content, [{ type: 'text', text: 'plain' }]); +}); + +test('a subscription refused at connect is reopened, and change awareness returns', async () => { + const log = scratchLog(); + const connection = await connect('listen-fail-first', log, { reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + let changes = 0; + connection.on('tools-list-changed', () => changes++); + await until(() => wire(log).filter((e) => e.method === 'subscriptions/listen').length >= 2, 'a second listen'); + // Reopening refreshes once, since changes may have been missed meanwhile. + await until(() => changes >= 1, 'the refresh after reopening'); + const before = changes; + const touched = await connection.callTool('touch', {}); + assert.deepEqual(touched.content, [{ type: 'text', text: 'touched' }]); + await until(() => changes > before, 'the announced change'); +}); + +test('awareness survives a reconnect whose first listen is refused again', async () => { + const log = scratchLog(); + const connection = await connect('listen-fail-first', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + let changes = 0; + connection.on('tools-list-changed', () => changes++); + let atReconnect: number | null = null; + connection.on('reconnect', () => { atReconnect = changes; }); + await connection.callTool('die', {}); + await until(() => atReconnect !== null, 'reconnect'); + // Launch 2 refuses its first listen too; the newer generation reopens it. + // A reopened subscription is answered before the connection announces the + // refresh it makes for changes it may have missed, so that announcement + // marks the subscription accepted. + await until(() => changes > atReconnect!, 'the reopened subscription answered on launch 2'); + assert.ok(wire(log).filter((e) => e.method === 'subscriptions/listen' && e.launch === 2).length >= 2, 'a second listen on launch 2'); + const before = changes; + await connection.callTool('touch', {}); + await until(() => changes > before, 'the announced change on launch 2'); +}); + +test('a child that exits during the connect-time subscribe fails the connect, rather than being adopted as connected (R1-2)', async () => { + const log = scratchLog(); + await assert.rejects(connect('exit-on-listen-first', log), /closed its connection while connecting/); + assert.ok(wire(log).some((e) => e.method === 'subscriptions/listen'), 'it exited at the subscribe'); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 1, 'one launch, and no reconnect'); +}); + +test('with reconnect, a child that exits during the connect-time subscribe is a failed connect, and the next launch serves (R1-2)', async () => { + const log = scratchLog(); + const connection = await connect('exit-on-listen-first', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + await until(() => connection.isConnected, 'connected on the next launch'); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 2, 'launch 1 failed its connect, and launch 2 serves'); + const plain = await connection.callTool('plain', {}); + assert.ok(plain.content.length > 0); + assert.equal(calls(log, 'plain'), 1); + assert.equal(wire(log).find((e) => e.method === 'tools/call')!.launch, 2); +}); + +/** + * The stdio line under a connection's live session, reached from inside, to + * stand for a loss read from a failed pipe while the child still runs: the + * transport closes before the child has exited (R1-1). + */ +function liveLine(connection: ModernMcpConnection): { markClosed: (info: { reason: string }) => void; child: { pid: number; kill: (signal?: string) => boolean } } { + const session = (connection as unknown as { session: { wire: { inner: { line: unknown } } } | null }).session; + assert.ok(session, 'a live session'); + return session.wire.inner.line as never; +} +const alive = (pid: number) => { try { process.kill(pid, 0); return true; } catch { return false; } }; + +test('a transport lost while its child still runs is reaped before the reconnect, so no second child runs beside the first (R1-1)', async () => { + const log = scratchLog(); + const connection = await connect('', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + const first = wire(log).find((e) => e.event === 'start')!.pid!; + liveLine(connection).markClosed({ reason: 'stdio pipe failed' }); + await until(() => wire(log).filter((e) => e.event === 'start').length === 2, 'the next launch'); + assert.equal(alive(first), false, 'the first child was gone when the second started'); + await until(() => connection.isConnected, 'connected on the next launch'); +}); + +test('close() during a lost transport\'s reap waits for its child to exit (R1-1)', async () => { + const log = scratchLog(); + const connection = await connect('ignore-sigterm', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + const first = wire(log).find((e) => e.event === 'start')!.pid!; + liveLine(connection).markClosed({ reason: 'stdio pipe failed' }); + await connection.close(); + assert.equal(alive(first), false, 'dead when close() resolved'); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 1, 'no launch after close'); +}); + +test('a lost transport whose child can\'t be reaped halts reconnecting (R1-1)', async () => { + const log = scratchLog(); + const connection = await connect('', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + const errors: string[] = []; + connection.on('error', (error: Error) => errors.push(error.message)); + const failures: Array<{ error: string; permanent?: boolean }> = []; + connection.on('reconnect-failed', (failure: { error: string; permanent?: boolean }) => failures.push(failure)); + const line = liveLine(connection); + const realKill = line.child.kill.bind(line.child); + line.child.kill = () => true; // signals never arrive: the child outlives both bounds + // Before the connection's close(), which then reaps an exited child. + cleanups.push(async () => { + realKill('SIGKILL'); + await until(() => !alive(line.child.pid), 'the held child killed'); + }); + line.markClosed({ reason: 'stdio pipe failed' }); + await until(() => errors.some((m) => /Not starting another launch while it may still be running/.test(m)), 'the launch held as unreaped', 8000); + assert.equal(connection.willReconnect, false); + // No later attempt will come, so the halt is a permanent failure, which the + // framework raises as an alert. + assert.equal(failures.length, 1); + assert.equal(failures[0]!.permanent, true); + assert.match(failures[0]!.error, /reconnecting is halted/); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 1, 'no second launch'); +}); + +test("without reconnect, a lost transport whose child can't be reaped is a permanent failure that says nothing of reconnecting (R1-1)", async () => { + const log = scratchLog(); + const connection = await connect('', log); + connection.on('error', () => { /* the held launch, reported */ }); + const failures: Array<{ error: string; permanent?: boolean }> = []; + connection.on('reconnect-failed', (failure: { error: string; permanent?: boolean }) => failures.push(failure)); + const line = liveLine(connection); + const realKill = line.child.kill.bind(line.child); + line.child.kill = () => true; // signals never arrive: the child outlives both bounds + cleanups.push(async () => { + realKill('SIGKILL'); + await until(() => !alive(line.child.pid), 'the held child killed'); + }); + line.markClosed({ reason: 'stdio pipe failed' }); + await until(() => failures.length > 0, 'the permanent failure', 8000); + assert.equal(failures[0]!.permanent, true); + assert.match(failures[0]!.error, /the previous launch could not be reaped/); + assert.doesNotMatch(failures[0]!.error, /reconnecting/); +}); + +test("a close() during a lost transport's failing reap reports the failure itself, with no reconnect failure beside it (R1-1)", async () => { + const log = scratchLog(); + const connection = await connect('', log); + const failures: unknown[] = []; + connection.on('reconnect-failed', (failure: unknown) => failures.push(failure)); + connection.on('error', () => { /* the held launch, reported */ }); + const line = liveLine(connection); + const realKill = line.child.kill.bind(line.child); + line.child.kill = () => true; // signals never arrive: the child outlives both bounds + cleanups.push(async () => { + realKill('SIGKILL'); + await until(() => !alive(line.child.pid), 'the held child killed'); + }); + line.markClosed({ reason: 'stdio pipe failed' }); + // The host closes it while the lost child's reap runs: the close's + // rejection is the report, so no permanent failure goes out beside it. + await assert.rejects(connection.close()); + assert.deepEqual(failures, []); +}); + +test("a 'close' listener that throws doesn't skip a lost transport's reap (R1-1)", async () => { + const log = scratchLog(); + const connection = await connect('ignore-sigterm', log); + const first = wire(log).find((e) => e.event === 'start')!.pid!; + cleanups.push(() => { try { process.kill(first, 'SIGKILL'); } catch { /* reaped already */ } }); + connection.on('close', () => { throw new Error('a listener failed'); }); + try { + liveLine(connection).markClosed({ reason: 'stdio pipe failed' }); + } catch { /* the listener's own error, wherever it surfaces */ } + // The child ignores SIGTERM, so its reap ends with SIGKILL after 2 s. + await until(() => !alive(first), "the lost child's reap", 8000); +}); + +test('close() marks a connection closed at once, so a start() in the same turn launches nothing', async () => { + const log = scratchLog(); + const connection = ModernMcpConnection.create({ id: 'raw', command: process.execPath, args: [SERVER, '', log], protocol: 'modern' }); + const closing = connection.close(); + assert.equal(connection.isClosed, true, 'closed from the call on'); + await connection.start(); + await closing; + assert.equal(connection.isConnected, false); + assert.equal((connection as unknown as { generation: number }).generation, 0, 'no launch began'); +}); + +/** The pid of each child StdioTransport.spawn starts from here on, from the spawn itself. */ +function spawnedPids(): number[] { + const realSpawn = StdioTransport.spawn; + const pids: number[] = []; + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = async (config) => { + const transport = await realSpawn.call(StdioTransport, config); + pids.push((transport as unknown as { child: { pid: number } }).child.pid); + return transport; + }; + cleanups.push(() => { (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = realSpawn; }); + return pids; +} + +test("a read error on a stdio child's stderr is reported through the transport's 'error', and one on stdout ends the child; neither is thrown with nothing listening", async () => { + const transport = await StdioTransport.spawn({ id: 'pipes', command: process.execPath, args: ['-e', 'setInterval(() => {}, 1 << 30)'] }); + cleanups.push(() => transport.close()); + const child = (transport as unknown as { child: import('node:child_process').ChildProcess }).child; + const closed = new Promise<{ reason?: string }>((resolve) => { transport.once('close', resolve); }); + assert.doesNotThrow(() => child.stderr!.emit('error', new Error('stderr read failed')), 'stderr, nothing listening'); + const errors: string[] = []; + transport.on('error', (error: Error) => errors.push(error.message)); + child.stderr!.emit('error', new Error('stderr read failed')); + assert.deepEqual(errors, ['stderr read failed']); + // stdout carries the protocol: the child is ended, and its close names + // the error instead of an 'error'. + assert.doesNotThrow(() => child.stdout!.emit('error', new Error('stdout read failed')), 'stdout'); + const info = await closed; + assert.match(info.reason ?? '', /its pipe failed with stdout read failed/); + assert.deepEqual(errors, ['stderr read failed']); +}); + +test('close() right after start() ends a modern stdio connect still spawning, without waiting out its discover probe', async () => { + const log = scratchLog(); + const pids = spawnedPids(); + const connection = ModernMcpConnection.create({ id: 'raw', command: process.execPath, args: [SERVER, 'hang-discover', log], protocol: 'modern' }); + const began = Date.now(); + const started = connection.start().then(() => null, (error: unknown) => error); + await connection.close(); + const tookMs = Date.now() - began; + assert.ok(tookMs < 15_000, `close() took ${tookMs} ms, not the discover probe's 30 s`); + // The child may be ended before it can log its start, so its pid comes + // from the spawn. + assert.equal(pids.length, 1, 'one launch'); + assert.equal(alive(pids[0]!), false, 'the child is gone once close() resolved'); + await started; + const endedMs = Date.now() - began; + assert.ok(endedMs < 15_000, `the connect ended after ${endedMs} ms, not the discover probe's 30 s`); + assert.equal(connection.isConnected, false); +}); + +test('close() during a reconnect handshake ends the launch it was waiting on, and stops', async () => { + const log = scratchLog(); + const connection = await connect('hang-discover-later', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + await connection.callTool('die', {}); + await until(() => wire(log).some((e) => e.launch === 2 && e.method === 'server/discover'), 'launch 2 waiting on discover'); + const second = wire(log).find((e) => e.launch === 2 && e.event === 'start')!; + await connection.close(); + const alive = (pid: number) => { try { process.kill(pid, 0); return true; } catch { return false; } }; + await until(() => !alive(second.pid!), 'the hanging launch has exited', 3000); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 2, 'no launch after close'); + assert.equal(connection.isConnected, false); +}); + +test('an input_required continuation keeps its evidence: the second leg decides the outcome', async () => { + const log = scratchLog(); + const connection = await connect('', log, { requestTimeoutMs: 600 }); + await connection.listTools(); + await assert.rejects(connection.callTool('cont', { leg2: 'error' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'error-response'); + assert.match(err.message, /\[-32603\]/); + assert.deepEqual(err.data, { leg: 2, state: 'state-for-error' }); + return true; + }); + await assert.rejects(connection.callTool('cont', { leg2: 'hang' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /Cancellation was requested/); + return true; + }); + assert.equal(calls(log, 'cont'), 4, 'two legs each'); +}); + +test('a deadline between continuation rounds is no-response, and claims no cancellation', async () => { + const log = scratchLog(); + const connection = await connect('', log, { requestTimeoutMs: 1500 }); + await connection.listTools(); + // Hold the round between leg 1's answer and leg 2 explicitly, so the + // deadline passes with nothing in flight by construction. Leg 1 asks for + // elicitation input, and an input handler registered on the session's + // client holds until the deadline aborts it, ending as the SDK's own pause + // does. Don't replace this with a deadline timed against the SDK's internal + // pause before a state-only round: that races leg 1 under load. + let held = false; + const client = (connection as unknown as { session: { client: { _requestHandlers: Map } } }).session.client; + client._requestHandlers.set('elicitation/create', (_request: unknown, ctx: { mcpReq: { signal: AbortSignal } }) => { + held = true; + const signal = ctx.mcpReq.signal; + return new Promise((_, reject) => { + const stop = () => reject(signal.reason instanceof SdkError ? signal.reason : new SdkError(SdkErrorCode.RequestTimeout, String(signal.reason))); + if (signal.aborted) stop(); + else signal.addEventListener('abort', stop, { once: true }); + }); + }); + await assert.rejects(connection.callTool('cont', { leg1: 'elicit' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match( + err.message, + /did not finish tools\/call "cont" within 1500ms: it asked for another round \(input_required\), and the deadline passed before that round was sent, so nothing was in flight to cancel\. The outcome is unknown/, + ); + assert.doesNotMatch(err.message, /Cancellation was requested|answered/); + return true; + }); + assert.ok(held, 'the round was held'); + // The wire is ordered: once a later call has reached the server, any + // second leg sent before it would have too. + await connection.callTool('plain', {}); + assert.equal(calls(log, 'plain'), 1); + assert.equal(calls(log, 'cont'), 1, 'only leg 1 reached the wire'); + assert.equal(wire(log).filter((e) => e.method === 'notifications/cancelled').length, 0, 'nothing was cancelled'); +}); + +test('a connection lost between continuation rounds is no-response, not an answer', async () => { + const log = scratchLog(); + const connection = await connect('', log); + await connection.listTools(); + await assert.rejects(connection.callTool('cont', { leg1: 'exit', leg2: 'ok' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match( + err.message, + /did not finish tools\/call "cont": it asked for another round \(input_required\), and the call ended before that round was sent: Not connected\. The outcome is unknown/, + ); + return true; + }); + assert.equal(calls(log, 'cont'), 1); +}); + +test('a continuation this client cannot go on with is no-response: the server may have acted', async () => { + const log = scratchLog(); + const connection = await connect('', log); + await connection.listTools(); + // Input this client can't supply: it advertises no elicitation. + await assert.rejects(connection.callTool('cont', { leg1: 'elicit' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /it asked for another round \(input_required\), and the call ended before that round was sent: Cannot fulfil input request 'confirm'/); + return true; + }); + assert.equal(calls(log, 'cont'), 1); + // A server that sheds load on every round meets the rounds cap: the first + // leg and ten retries. + await assert.rejects(connection.callTool('cont', { leg2: 'again' }), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /the call ended before that round was sent: Multi-round-trip request 'tools\/call' still required input after 10 rounds/); + return true; + }); + assert.equal(calls(log, 'cont'), 1 + 11); +}); + +test('a result type this revision does not define, or none, is not a final answer: no-response', async () => { + const log = scratchLog(); + const connection = await connect('', log); + await connection.listTools(); + await assert.rejects(connection.callTool('task', {}), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /did not finish tools\/call "task": its answer was not a complete result: Unsupported result type 'task'.*The outcome is unknown/); + return true; + }); + await assert.rejects(connection.callTool('bare', {}), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /did not finish tools\/call "bare": its answer was not a complete result: .*missing required resultType.*The outcome is unknown/); + return true; + }); + assert.equal(calls(log, 'task'), 1); + assert.equal(calls(log, 'bare'), 1); +}); + +test('concurrent continuations never cross their evidence', async () => { + const log = scratchLog(); + const connection = await connect('', log, { requestTimeoutMs: 2000 }); + await connection.listTools(); + const [failing, passing, failingToo] = await Promise.allSettled([ + connection.callTool('cont', { leg2: 'error' }), + connection.callTool('cont', { leg2: 'ok' }), + connection.callTool('cont', { leg2: 'error' }), + ]); + for (const settled of [failing, failingToo]) { + assert.equal(settled.status, 'rejected'); + const err = (settled as PromiseRejectedResult).reason; + assert.ok(err instanceof McplRequestError); + assert.equal(err.outcome, 'error-response'); + assert.deepEqual(err.data, { leg: 2, state: 'state-for-error' }); + } + assert.equal(passing.status, 'fulfilled'); + assert.deepEqual((passing as PromiseFulfilledResult<{ content: unknown }>).value.content, [{ type: 'text', text: 'leg 2 done (state-for-ok)' }]); +}); + +test('a JSON line that is not a JSON-RPC message is a diagnostic, never a crash', async () => { + const log = scratchLog(); + const connection = await connect('', log); + const diagnostics: string[] = []; + connection.on('error', (e: Error) => diagnostics.push(e.message)); + await connection.listTools(); + assert.deepEqual((await connection.callTool('nullframe', {})).content, [{ type: 'text', text: 'after the noise' }]); + await until(() => diagnostics.length >= 4, 'four diagnostics'); + assert.ok(diagnostics.every((d) => /sent a malformed JSON-RPC message/.test(d)), diagnostics.join(' | ')); + assert.equal(connection.isConnected, true); + assert.deepEqual((await connection.callTool('plain', {})).content, [{ type: 'text', text: 'plain' }]); +}); + +test('close() reaps: a child that ignores SIGTERM is killed before close resolves', async () => { + const log = scratchLog(); + const connection = await connect('ignore-sigterm', log); + const pid = wire(log).find((e) => e.event === 'start')!.pid!; + await connection.close(); + const alive = (p: number) => { try { process.kill(p, 0); return true; } catch { return false; } }; + assert.equal(alive(pid), false, 'dead when close() resolved'); + assert.ok(wire(log).some((e) => e.event === 'sigterm-ignored'), 'it really did ignore SIGTERM'); +}); + +test('a child that cannot be reaped fails close() explicitly, never a claimed exit', async () => { + const transport = await StdioTransport.spawn({ id: 'stuck', command: process.execPath, args: ['-e', 'setInterval(() => {}, 1 << 30)'] }); + const child = (transport as unknown as { child: { pid: number; kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + child.kill = () => true; // signals never arrive: the child outlives both bounds + cleanups.push(() => { realKill('SIGKILL'); }); + await assert.rejects(transport.close(), /^Error: stdio child \d+ of MCP server "stuck" did not exit within 2000ms of SIGKILL; it could not be reaped$/); + // Calls share the one attempt and its verdict. + await assert.rejects(transport.close(), /could not be reaped/); +}); + +test('a failed launch that cannot be reaped halts reconnecting; close() settles it', async () => { + const log = scratchLog(); + const realSpawn = StdioTransport.spawn; + const stuck: Array<{ pid: number; kill: (signal?: string) => boolean }> = []; + let launches = 0; + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = async (config) => { + const transport = await realSpawn.call(StdioTransport, config); + if (++launches > 1) { + // From the second launch on, signals never arrive. + const child = (transport as unknown as { child: { pid: number; kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + child.kill = () => true; + stuck.push({ pid: child.pid, kill: realKill }); + } + return transport; + }; + cleanups.push(() => { + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = realSpawn; + for (const s of stuck) s.kill('SIGKILL'); + }); + const connection = await connect('reject-discover-later', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + const failures: Array<{ error: string; permanent?: boolean }> = []; + connection.on('reconnect-failed', (f) => failures.push(f)); + connection.on('error', () => {}); + await connection.callTool('die', {}); + await until(() => failures.some((f) => f.permanent), 'a permanent reconnect failure', 9000); + const halted = failures.find((f) => f.permanent)!; + assert.match(halted.error, /the previous launch could not be reaped .*reconnecting is halted/); + assert.equal(connection.willReconnect, false); + await new Promise((r) => setTimeout(r, 300)); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 2, 'no third launch while the second is unreaped'); + // The stuck child goes away; close() re-checks it and settles cleanly. + stuck[0]!.kill('SIGKILL'); + await until(() => { try { process.kill(stuck[0]!.pid, 0); return false; } catch { return true; } }, 'the stuck child gone'); + await connection.close(); +}); + +test('concurrent close() calls share one teardown and its verdict; a later close re-checks', async () => { + const log = scratchLog(); + const realSpawn = StdioTransport.spawn; + let stuck: { pid: number; kill: (signal?: string) => boolean } | null = null; + // Every signal teardown sends: one teardown escalates once, SIGTERM then SIGKILL. + const signals: Array = []; + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = async (config) => { + const transport = await realSpawn.call(StdioTransport, config); + const child = (transport as unknown as { child: { pid: number; kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + child.kill = (signal?: string) => { signals.push(signal); return true; }; // teardown can't reap it + stuck = { pid: child.pid, kill: realKill }; + return transport; + }; + cleanups.push(() => { + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = realSpawn; + stuck?.kill('SIGKILL'); + }); + const connection = await connect('', log); + const settled: string[] = []; + const first = connection.close().then(() => settled.push('first:ok'), () => settled.push('first:failed')); + const second = connection.close().then(() => settled.push('second:ok'), () => settled.push('second:failed')); + await new Promise((r) => setTimeout(r, 1000)); + assert.deepEqual(settled, [], 'neither resolves while teardown is pending'); + await Promise.all([first, second]); + assert.deepEqual(settled, ['first:failed', 'second:failed'], 'one verdict, in order'); + assert.deepEqual(signals, ['SIGTERM', 'SIGKILL'], 'one teardown, shared: the second close sent no signals of its own'); + // Once the child is gone, a later close re-checks and settles. + stuck!.kill('SIGKILL'); + await until(() => { try { process.kill(stuck!.pid, 0); return false; } catch { return true; } }, 'the child gone'); + await connection.close(); +}); + +test('start() runs once: a second call shares the first connect, and no second session opens beside it', async () => { + const log = scratchLog(); + const connection = ModernMcpConnection.create({ id: 'raw', command: process.execPath, args: [SERVER, '', log], protocol: 'modern' }); + cleanups.push(() => connection.close()); + const first = connection.start(); + assert.equal(connection.start(), first, 'the same promise while connecting'); + await first; + assert.equal(connection.start(), first, 'and once connected'); + await connection.start(); + assert.equal(connection.isConnected, true); + const launches = wire(log).filter((e) => e.event === 'start'); + assert.equal(launches.length, 1, 'one launch'); + await connection.close(); + const alive = (p: number) => { try { process.kill(p, 0); return true; } catch { return false; } }; + assert.equal(alive(launches[0]!.pid!), false, 'close() reaped the only child'); +}); + +test('a send the stdio line refused before writing anything is not-sent', async () => { + const log = scratchLog(); + const connection = await connect('', log); + await connection.listTools(); + // The line is closed, while the session still stands: the transport + // refuses the write, so nothing reached the server. + const line = (connection as unknown as { session: { wire: { inner: { line: { _closed: boolean } } } } }).session.wire.inner.line; + line._closed = true; + try { + await assert.rejects(connection.callTool('plain', {}), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'not-sent'); + assert.match(err.message, /tools\/call "plain" was not sent: .*stdio transport is closed/); + return true; + }); + } finally { + line._closed = false; + } + assert.equal(calls(log, 'plain'), 0); +}); + +test('a plain call whose child dies before answering is no-response', async () => { + const log = scratchLog(); + const connection = await connect('', log); + connection.on('error', () => {}); + await connection.listTools(); + await assert.rejects(connection.callTool('vanish', {}), (err: unknown) => { + assert.ok(err instanceof McplRequestError, String(err)); + assert.equal(err.outcome, 'no-response'); + assert.match(err.message, /MCP server "raw" tools\/call "vanish" failed: /); + return true; + }); + assert.equal(calls(log, 'vanish'), 1, 'the call reached the server'); +}); + +test('failures are classified in order: nothing sent first, then the answers, then an HTTP refusal', () => { + const connection = ModernMcpConnection.create({ id: 'raw', command: process.execPath, args: [SERVER, '', scratchLog()], protocol: 'modern' }); + const classify = (record: Record, error: unknown, method = 'tools/call') => + (connection as unknown as { + requestFailure: (what: string, record: unknown, error: unknown, timeoutMs?: number) => Error; + }).requestFailure(method === 'tools/list' ? 'tools/list' : 'tools/call "op"', { method, handedOff: true, answer: 'none', continued: false, ...record }, error, 1000); + const refused = () => new SdkHttpError(SdkErrorCode.ClientHttpAuthentication, 'Unauthorized', { status: 401 } as never); + const outcome = (error: Error) => (error instanceof McplRequestError ? error.outcome : 'plain'); + + const unsent = classify({ handedOff: false }, refused()); + assert.equal(outcome(unsent), 'not-sent', 'a refusal before anything was handed off'); + + const first = classify({}, refused()); + assert.equal(outcome(first), 'error-response'); + assert.equal(((first as McplRequestError).data as { httpStatus?: number }).httpStatus, 401); + + const later = classify({ continued: true }, refused()); + assert.equal(outcome(later), 'no-response', 'after an earlier leg asked for another round'); + assert.match(later.message, /asked for another round \(input_required\), and the server refused a later one: HTTP 401/); + + const listing = classify({}, new SdkError(SdkErrorCode.RequestTimeout, 'Request timed out'), 'tools/list'); + assert.equal(outcome(listing), 'no-response'); + assert.match(listing.message, /did not answer tools\/list within 1000ms\. Cancellation was requested; the list did not arrive in time\. Listing changes nothing on the server, so it is safe to retry\./); + assert.doesNotMatch(listing.message, /side-effecting/); +}); + +test('a reconnect backoff never exceeds the timer limit', () => { + const connection = ModernMcpConnection.create({ + id: 'raw', command: process.execPath, args: [SERVER, '', scratchLog()], protocol: 'modern', + reconnect: true, reconnectIntervalMs: MAX_TIMER_MS, reconnectMaxIntervalMs: MAX_TIMER_MS, + }); + const backoff = (connection as unknown as { backoffDelay: (failures: number) => number }).backoffDelay.bind(connection); + const random = Math.random; + Math.random = () => 0.999; // the jitter's top end + try { + assert.equal(backoff(1), MAX_TIMER_MS); + assert.equal(backoff(30), MAX_TIMER_MS); + } finally { + Math.random = random; + } +}); + +test('a new generation backs off its first reopen from the base interval', async () => { + const log = scratchLog(); + const connection = await connect('listen-fail-first', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60_000 }); + const internals = connection as unknown as { + relistenAttempts: number; + generation: number; + scheduleRelisten: (generation: number) => void; + }; + // The previous generation had failed to reopen many times. + internals.relistenAttempts = 9; + const seen: Array<{ generation: number; attempts: number }> = []; + const schedule = internals.scheduleRelisten.bind(connection); + internals.scheduleRelisten = (generation: number) => { + seen.push({ generation, attempts: internals.relistenAttempts }); + schedule(generation); + }; + let reconnected = false; + connection.on('reconnect', () => { reconnected = true; }); + await connection.callTool('die', {}); + await until(() => reconnected, 'reconnect'); + // Launch 2 refuses its first listen, so the new generation schedules a reopen. + await until(() => seen.some((s) => s.generation === internals.generation), 'the new generation scheduling its reopen'); + assert.equal(seen.find((s) => s.generation === internals.generation)!.attempts, 0); +}); + +test('a server that shares no revision with this client is a configuration verdict: no retry', async () => { + const log = scratchLog(); + const connection = ModernMcpConnection.create({ + id: 'raw', command: process.execPath, args: [SERVER, 'other-revision', log], protocol: 'modern', + reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60, + }); + cleanups.push(() => connection.close()); + await assert.rejects(connection.start(), (err: unknown) => { + assert.ok(err instanceof McplProtocolVersionError, String(err)); + assert.equal(err.kind, 'rejected'); + assert.equal(err.offered, '2026-07-28'); + assert.deepEqual(err.supported, ['2099-01-01']); + assert.match(err.message, /MCP server "raw" does not support MCP 2026-07-28 \(it supports: 2099-01-01\)\./); + return true; + }); + assert.equal(connection.willReconnect, false, 'nothing will retry it'); + assert.equal((connection as unknown as { reconnectTimer: unknown }).reconnectTimer, null); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 1); +}); + +test('a legacy-only server is pointed to what reaches it: its configuration over stdio, stdio or WebSocket for an http url', () => { + const refusal = new UnsupportedProtocolVersionError({ supported: ['2024-11-05'], requested: '2026-07-28' }); + const verdictFor = (config: Parameters[0]) => + (ModernMcpConnection.create(config) as unknown as { versionVerdict: (error: unknown) => McplProtocolVersionError | null }).versionVerdict(refusal)!; + const stdio = verdictFor({ id: 'raw', command: process.execPath, args: [SERVER, '', scratchLog()], protocol: 'modern' }); + assert.match(stdio.message, /\(it supports: 2024-11-05\)\. It speaks legacy MCP: configure it without protocol: 'modern'\.$/); + const http = verdictFor({ id: 'remote', url: 'http://127.0.0.1:1/mcp' }); + assert.match(http.message, /\(it supports: 2024-11-05\)\. It speaks only legacy MCP, which this client reaches over stdio or WebSocket \(ws:\/\/, wss:\/\/\), not http\(s\)\.$/); + assert.doesNotMatch(http.message, /protocol: 'modern'/); +}); + +test('a reconnect that meets a revision the client does not share stops for good', async () => { + const log = scratchLog(); + const connection = await connect('other-revision-later', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }); + const failures: Array<{ error: string; permanent?: boolean }> = []; + connection.on('reconnect-failed', (f) => failures.push(f)); + await connection.callTool('die', {}); + await until(() => failures.length > 0, 'the reconnect failing'); + assert.equal(failures.length, 1); + assert.equal(failures[0]!.permanent, true); + assert.match(failures[0]!.error, /does not support MCP 2026-07-28 \(it supports: 2099-01-01\)/); + assert.equal(connection.willReconnect, false); + assert.equal((connection as unknown as { reconnectTimer: unknown }).reconnectTimer, null, 'no further attempt scheduled'); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 2); +}); + +test('close() during a connect waits for that connect\'s own cleanup, and reports what it could not reap', async () => { + const log = scratchLog(); + const connection = ModernMcpConnection.create({ id: 'raw', command: process.execPath, args: [SERVER, 'hang-discover', log], protocol: 'modern' }); + connection.on('error', () => {}); + const statics = ModernMcpConnection as unknown as { closeSession: (session: unknown) => Promise }; + const real = statics.closeSession; + let closes = 0; + const unconfirmed = new Error('the child could not be confirmed gone (simulated)'); + // close() closes the generation first; the connect's own cleanup closes it + // again, and that second close is the one that can't confirm the exit. + statics.closeSession = async (session: unknown) => { + const verdict = await real.call(ModernMcpConnection, session); + return ++closes === 2 ? unconfirmed : verdict; + }; + cleanups.push(() => { statics.closeSession = real; }); + const starting = connection.start(); + await until(() => wire(log).some((e) => e.method === 'server/discover'), 'the launch waiting on discover'); + await assert.rejects(connection.close(), (error) => error === unconfirmed, 'the failure itself, as the MCPL engine rethrows one'); + assert.equal(closes, 2, 'close() resolved only after the connect\'s own cleanup'); + await starting; + statics.closeSession = real; + await connection.close(); +}); + +test('a server that closes its stdin and ignores SIGTERM is killed and reconnected, and a call made after the failure is refused as not sent', { timeout: 40_000 }, async () => { + const log = scratchLog(); + const connection = await connect('ignore-sigterm', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60, requestTimeoutMs: 30_000 }); + connection.on('error', () => { /* out-of-band SDK errors */ }); + const closes: string[] = []; + connection.on('close', (info: { reason?: string }) => closes.push(info.reason ?? '')); + await connection.callTool('deafen', {}); + await until(() => wire(log).some((e) => e.event === 'deaf'), 'the server to close its stdin'); + // The first write after that fails with EPIPE. It went out, so its call + // isn't answered and isn't refused: it fails with the close. + const first = connection.callTool('plain', {}).then(() => 'answered', (error: unknown) => error); + await until(() => wire(log).some((e) => e.event === 'sigterm-ignored'), 'the transport to send SIGTERM'); + // From here on nothing is written: a call is refused at once, as not sent. + await assert.rejects(connection.callTool('plain', {}), (error: unknown) => { + assert.ok(error instanceof McplRequestError, String(error)); + assert.equal(error.outcome, 'not-sent'); + assert.match(error.message, /stdio pipe failed with write EPIPE/); + return true; + }); + // SIGKILL ends the child, and the connection reconnects as after any loss. + await until(() => wire(log).filter((e) => e.event === 'start').length === 2 && connection.isConnected, 'the next launch, connected', 15_000); + assert.equal(closes.length, 1, closes.join('; ')); + const outcome = await first; + assert.ok(outcome instanceof McplRequestError && outcome.outcome !== 'not-sent', `the written call failed with the close: ${String(outcome)}`); + assert.deepEqual((await connection.callTool('plain', {})).content, [{ type: 'text', text: 'plain' }], 'the next launch answers'); +}); + +test("a server that closes its stdin and outlives SIGKILL too is held as a launch that can't be reaped: a permanent failure, no second launch, and close() retries", { timeout: 40_000 }, async () => { + const log = scratchLog(); + const connection = await connect('', log, { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60, requestTimeoutMs: 30_000 }); + connection.on('error', () => { /* the held launch, reported */ }); + const failures: Array<{ error: string; permanent?: boolean }> = []; + connection.on('reconnect-failed', (failure: { error: string; permanent?: boolean }) => failures.push(failure)); + const line = liveLine(connection); + const realKill = line.child.kill.bind(line.child); + line.child.kill = () => true; // signals never arrive: the child outlives both bounds + // Before the connection's close(), which then reaps an exited child. + cleanups.push(async () => { + realKill('SIGKILL'); + await until(() => !alive(line.child.pid), 'the held child killed'); + }); + await connection.callTool('deafen', {}); + await until(() => wire(log).some((e) => e.event === 'deaf'), 'the server to close its stdin'); + void connection.callTool('plain', {}).catch(() => { /* its write meets EPIPE */ }); + await until(() => failures.length > 0, 'the permanent failure', 12_000); + assert.equal(failures.length, 1); + assert.equal(failures[0]!.permanent, true); + assert.match(failures[0]!.error, /^its stdio pipe failed with write EPIPE; the previous launch could not be reaped \(stdio child \d+ of MCP server "raw" did not exit within 2000ms of SIGKILL; it could not be reaped\); reconnecting is halted$/); + assert.equal(connection.isConnected, false); + assert.equal(connection.willReconnect, false); + assert.equal(connection.holdsUnreapedChild, true); + await new Promise((r) => setTimeout(r, 200)); + assert.equal(wire(log).filter((e) => e.event === 'start').length, 1, 'no second launch'); + // close() retries the reap: refused while the child runs, then done. + await assert.rejects(connection.close(), /could not be reaped/); + realKill('SIGKILL'); + await until(() => !alive(line.child.pid), 'the child to be killed'); + await connection.close(); + assert.equal(connection.holdsUnreapedChild, false); +}); diff --git a/test/mcp-protocol-family.test.ts b/test/mcp-protocol-family.test.ts new file mode 100644 index 000000000..6f23e44f9 --- /dev/null +++ b/test/mcp-protocol-family.test.ts @@ -0,0 +1,584 @@ +/** + * Protocol family and transport selection (room-284 design, shelf-487). + * A URL's scheme decides a network server's family; only a stdio server has + * a choice, and it defaults to legacy. Validation adds checks only for what + * the modern family brought, so every legacy configuration still passes. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; + +import { + resolveServerBinding, + serverConfigProblems, + serverConfigWarnings, + checkServerConfig, + redactUrl, + redactServerText, + credentialRedactor, + serverCredentialPatterns, + serverTextRedactor, + noteDialSent, + describeCredentialError, + mcplPolicyGrants, + MAX_TIMER_MS, +} from '../src/mcpl/protocol-family.js'; +import { getEventListeners } from 'node:events'; +import { DEFAULT_EAGER_FETCH_MAX_BYTES, DEFAULT_MAX_TOTAL_BYTES } from '../src/mcpl/reference-fetcher.js'; +import { DEFAULT_MAX_INPUT_BYTES } from '../src/mcpl/tool-lifecycle.js'; +import { ModernMcpConnection } from '../src/mcpl/modern-connection.js'; +import { WebSocketTransport, buildWebSocketUrl } from '../src/mcpl/transport.js'; +import type { McplServerConfig } from '../src/mcpl/types.js'; + +const cfg = (c: Partial): McplServerConfig => ({ id: 's', ...c }); + +test('family and transport follow the configuration', () => { + assert.deepEqual(resolveServerBinding(cfg({ command: 'srv' })), { family: 'legacy', transport: 'stdio' }); + assert.deepEqual(resolveServerBinding(cfg({ command: 'srv', protocol: 'legacy' })), { family: 'legacy', transport: 'stdio' }); + assert.deepEqual(resolveServerBinding(cfg({ command: 'srv', protocol: 'modern' })), { family: 'modern', transport: 'stdio' }); + assert.deepEqual(resolveServerBinding(cfg({ url: 'ws://localhost:3000/mcpl' })), { family: 'legacy', transport: 'websocket' }); + assert.deepEqual(resolveServerBinding(cfg({ url: 'wss://example.com/mcpl' })), { family: 'legacy', transport: 'websocket' }); + assert.deepEqual(resolveServerBinding(cfg({ url: 'http://localhost:8080/mcp' })), { family: 'modern', transport: 'http' }); + assert.deepEqual(resolveServerBinding(cfg({ url: 'https://example.com/mcp' })), { family: 'modern', transport: 'http' }); + // With both a command and a url, an explicit transport says which is used + // (a host layers a recipe's url and transport over a file's command). + assert.deepEqual( + resolveServerBinding(cfg({ command: 'srv', url: 'ws://x/mcpl', transport: 'websocket' })), + { family: 'legacy', transport: 'websocket' }, + ); + assert.deepEqual( + resolveServerBinding(cfg({ command: 'srv', url: 'ws://x/mcpl', transport: 'stdio' })), + { family: 'legacy', transport: 'stdio' }, + ); + assert.deepEqual( + resolveServerBinding(cfg({ command: 'srv', url: 'https://x/mcp', transport: 'http' })), + { family: 'modern', transport: 'http' }, + ); + assert.deepEqual( + resolveServerBinding(cfg({ url: 'https://x/mcp', transport: 'http' })), + { family: 'modern', transport: 'http' }, + ); +}); + +test('a configuration that names no usable transport throws', () => { + assert.throws(() => resolveServerBinding(cfg({})), /needs "command"/); + assert.throws(() => resolveServerBinding(cfg({ url: 'ftp://x' })), /ws:\/\/ or wss:\/\/.*http:\/\/ or https:\/\//); + assert.throws(() => resolveServerBinding(cfg({ url: 'not a url' })), /invalid url/); + assert.throws(() => resolveServerBinding(cfg({ url: 'ws://x', transport: 'http' })), /does not match url/); + assert.throws(() => resolveServerBinding(cfg({ url: 'https://x', transport: 'websocket' })), /does not match url/); + assert.throws(() => resolveServerBinding(cfg({ url: 'ws://x', transport: 'stdio' })), /requires "command"/); + assert.throws(() => resolveServerBinding(cfg({ command: 'srv', transport: 'http' })), /transport "http" requires "url"/); +}); + +test('command and url without a transport are refused, rather than one of them silently ignored', () => { + for (const url of ['ws://x/mcpl', 'https://x/mcp']) { + for (const extra of [{}, { protocol: 'legacy' as const }, { protocol: 'modern' as const }]) { + const c = cfg({ command: 'srv', url, ...extra }); + assert.throws(() => resolveServerBinding(c), /"command" and "url" are both set, and no "transport" says which one to use/, JSON.stringify(c)); + assert.match(serverConfigProblems(c).join(), /"command" and "url" are both set/, JSON.stringify(c)); + } + } + // A transport that disagrees with the url it would use is still refused. + assert.throws(() => resolveServerBinding(cfg({ command: 'srv', url: 'https://x/mcp', transport: 'websocket' })), /does not match url/); + assert.throws(() => resolveServerBinding(cfg({ command: 'srv', url: 'ws://x/mcpl', transport: 'http' })), /does not match url/); +}); + +test('errors and the status view show a url redacted: no userinfo, no query values', () => { + assert.equal(redactUrl('https://user:pw@mcp.example:8443/v1/mcp?token=abc®ion=eu&token=def#frag'), + 'https://mcp.example:8443/v1/mcp?token=redacted®ion=redacted&token=redacted'); + assert.equal(redactUrl('wss://example.test/mcpl'), 'wss://example.test/mcpl'); + // A path segment that looks like a credential goes too: a hosted server's + // per-user secret, a uuid, anything very long. Short and plain-word ones stay. + assert.equal(redactUrl('https://mcp.zapier.com/api/mcp/s/Zt4kQ9xW2vLm8pRs7YbN/mcp'), 'https://mcp.zapier.com/api/mcp/s/redacted/mcp'); + assert.equal(redactUrl('https://mcp.example/server/123e4567-e89b-12d3-a456-426614174000/mcp'), 'https://mcp.example/server/redacted/mcp'); + assert.equal(redactUrl(`https://mcp.example/${'a'.repeat(32)}/mcp`), 'https://mcp.example/redacted/mcp'); + assert.equal(redactUrl('https://mcp.example/v2/2026-07-28/organization-settings/mcp'), 'https://mcp.example/v2/2026-07-28/organization-settings/mcp'); + // A bare token rides as a valueless name: one shaped like a token, or one + // whose name says it's a credential, goes. A name with a value is a label + // and stays, and so does a short plain name. + assert.equal(redactUrl('https://h/mcp?sk-live-abcdef0123456789'), 'https://h/mcp?redacted=redacted'); + assert.equal(redactUrl('https://h/mcp?apikey_x9'), 'https://h/mcp?redacted=redacted'); + assert.equal(redactUrl('https://h/mcp?token=abc'), 'https://h/mcp?token=redacted'); + assert.equal(redactUrl('https://h/mcp?verbose'), 'https://h/mcp?verbose=redacted'); + assert.equal(redactUrl('not a url ?token=abc'), '(an invalid url, not shown)'); + assert.equal(redactUrl(undefined), '(an invalid url, not shown)'); + // The configuration errors that name a url give it redacted, or not at all. + const mismatch = serverConfigProblems(cfg({ url: 'https://u:pw@x/mcp?key=sekret', transport: 'websocket' })).join(); + assert.match(mismatch, /does not match url https:\/\/x\/mcp\?key=redacted/); + const invalid = serverConfigProblems(cfg({ url: 'sekret token not a url' })).join(); + assert.match(invalid, /invalid url/); + for (const text of [mismatch, invalid]) assert.ok(!/sekret|pw@/.test(text), text); +}); + +test('every legacy configuration shape passes validation unchanged', () => { + const legacy: McplServerConfig[] = [ + cfg({ command: 'srv', requestTimeoutMs: 0 }), // zero still disables the legacy watchdog + cfg({ command: 'srv', enabledFeatureSets: ['a.*'], allowHostCommands: true, channelSubscription: 'auto' }), + cfg({ url: 'wss://x/mcpl', token: 't', toolLifecycle: { observe: { tools: ['*'] } } as never, autofetch: { maxBytes: 1 } }), + cfg({ command: 'srv', protocol: 'legacy', disabledCapabilities: ['contextHooks.*'] }), + ]; + for (const c of legacy) assert.deepEqual(serverConfigProblems(c), [], JSON.stringify(c)); +}); + +test('protocol is a stdio-only choice', () => { + assert.match(serverConfigProblems(cfg({ url: 'https://x/mcp', protocol: 'modern' })).join(), /applies only to stdio/); + assert.match(serverConfigProblems(cfg({ url: 'ws://x/mcpl', protocol: 'legacy' })).join(), /applies only to stdio/); + assert.match(serverConfigProblems(cfg({ command: 'srv', protocol: 'new' as never })).join(), /must be 'legacy' or 'modern'/); +}); + +test('a modern server needs a real deadline', () => { + for (const bad of [0, -1, 1.5, Number.NaN, Number.POSITIVE_INFINITY, MAX_TIMER_MS + 1]) { + const problems = serverConfigProblems(cfg({ command: 'srv', protocol: 'modern', requestTimeoutMs: bad })); + assert.match(problems.join(), /requestTimeoutMs must be an integer from 1/, String(bad)); + } + for (const good of [undefined, 1, 60_000, MAX_TIMER_MS]) { + assert.deepEqual(serverConfigProblems(cfg({ url: 'https://x/mcp', requestTimeoutMs: good })), [], String(good)); + } +}); + +test('a legacy deadline above the timer limit is refused; every value that worked passes', () => { + for (const bad of [MAX_TIMER_MS + 1, Number.POSITIVE_INFINITY]) { + for (const c of [cfg({ command: 'srv', requestTimeoutMs: bad }), cfg({ url: 'wss://x/mcpl', requestTimeoutMs: bad })]) { + assert.match(serverConfigProblems(c).join(), /requestTimeoutMs must be at most 2147483647/, String(bad)); + } + } + for (const good of [undefined, 0, 1, 60_000, MAX_TIMER_MS]) { + assert.deepEqual(serverConfigProblems(cfg({ command: 'srv', requestTimeoutMs: good })), [], String(good)); + } +}); + +test('MCPL policy on a modern server is refused where its value grants, and accepted where it only narrows or repeats a default (R1-3)', () => { + const modern = (c: Partial) => cfg({ url: 'https://x/mcp', ...c }); + // Each value that asks for MCPL behaviour is refused, naming the part that grants. + const granting: Array<[Partial, string]> = [ + [{ enabledCapabilities: ['toolLifecycle.observe'] }, 'enabledCapabilities: toolLifecycle.observe'], + [{ enabledCapabilities: ['contextHooks.beforeInference.inject.*'] }, 'enabledCapabilities: contextHooks.beforeInference.inject.system'], + [{ allowHostCommands: true }, 'allowHostCommands'], + [{ channelSubscription: 'auto' }, 'channelSubscription'], + [{ channelSubscription: ['general'] }, 'channelSubscription'], + [{ scopes: { files: { whitelist: ['read:*'], blacklist: ['write:*'] } } }, 'scopes.files.whitelist'], + [{ toolLifecycle: { observe: {} } }, 'toolLifecycle.observe'], + [{ toolLifecycle: { inputs: { tools: ['x--*'] } } }, 'toolLifecycle.inputs'], + [{ autofetch: { maxBytes: DEFAULT_EAGER_FETCH_MAX_BYTES + 1 } }, 'autofetch.maxBytes'], + [{ autofetch: { maxTotalBytes: DEFAULT_MAX_TOTAL_BYTES + 1 } }, 'autofetch.maxTotalBytes'], + [{ toolLifecycle: { maxInputBytes: DEFAULT_MAX_INPUT_BYTES + 1 } }, 'toolLifecycle.maxInputBytes'], + ]; + // Patterns reaching every path denied by default name each. A `*` matches + // one segment, and counts must agree (capabilityPatternMatches). + assert.deepEqual(mcplPolicyGrants(modern({ enabledCapabilities: ['*.*', '*.*.*.*'] })), [ + 'enabledCapabilities: contextHooks.beforeInference.inject.system', + 'enabledCapabilities: toolLifecycle.observe', + 'enabledCapabilities: toolLifecycle.inputs', + ]); + // An entry that isn't a string can't be matched, so it's read as asking + // for every path denied by default. + assert.deepEqual(mcplPolicyGrants(modern({ enabledCapabilities: [42 as never] })), [ + 'enabledCapabilities: contextHooks.beforeInference.inject.system', + 'enabledCapabilities: toolLifecycle.observe', + 'enabledCapabilities: toolLifecycle.inputs', + ]); + for (const [config, path] of granting) { + assert.deepEqual(mcplPolicyGrants(modern(config)), [path], path); + const problems = serverConfigProblems(modern(config)); + assert.equal(problems.length, 1, path); + assert.ok(problems[0]!.includes(`"${path}" asks for MCPL behaviour, which a modern MCP server cannot negotiate`), problems[0]); + } + // What only takes something away, or repeats a default, is already + // satisfied: a modern server has no MCPL surface. + const narrowing: Partial[] = [ + // connectome-host's overlay route stamps this on every server it deploys. + { disabledCapabilities: ['contextHooks', 'inferenceLifecycle', 'inferenceRequest', 'toolLifecycle'] }, + { disabledFeatureSets: ['b'] }, + // Allow-lists: absent, every declared set or advertised capability is a + // candidate, so a list can only narrow, except where it names a path + // denied by default. + { enabledFeatureSets: ['a'] }, + { enabledCapabilities: ['pushEvents', 'contextHooks.afterInference'] }, + // A one-segment `*` reaches no path denied by default. + { enabledCapabilities: ['*'] }, + { scopes: { files: { blacklist: ['write:*'] } } }, + { scopes: { files: { whitelist: [] } } }, + { channelSubscription: 'manual' }, + { toolLifecycle: { maxInputBytes: 1024 } }, + { toolLifecycle: { maxInputBytes: DEFAULT_MAX_INPUT_BYTES } }, + { toolLifecycle: { observe: false } as never }, + { autofetch: { maxBytes: DEFAULT_EAGER_FETCH_MAX_BYTES, maxTotalBytes: 1 } }, + { shouldTriggerInference: () => false }, + ]; + for (const config of narrowing) { + assert.deepEqual(serverConfigProblems(modern(config)), [], JSON.stringify(config)); + // Nor a warning: a host that stamps its deny-list on every server would + // hear it at every start, with nothing to do about it. + assert.deepEqual(serverConfigWarnings(modern(config)), [], JSON.stringify(config)); + } + // Empty values grant nothing, on a modern stdio server too. + const empty = cfg({ + command: 'srv', + protocol: 'modern', + enabledFeatureSets: [], + disabledCapabilities: [], + scopes: {}, + allowHostCommands: false, + toolLifecycle: {} as never, + channelSubscription: [], + }); + assert.deepEqual(serverConfigProblems(empty), []); +}); + +test('a credential on a modern stdio server is refused; a legacy stdio server still ignores it', () => { + const provider = async () => 'fresh'; + assert.match(serverConfigProblems(cfg({ command: 'srv', protocol: 'modern', token: 't' })).join(), /"token" applies only to a network \(url\) server/); + assert.match(serverConfigProblems(cfg({ command: 'srv', protocol: 'modern', accessProvider: provider })).join(), /"accessProvider" applies only to a network/); + assert.deepEqual(serverConfigProblems(cfg({ command: 'srv', token: 't', accessProvider: provider })), []); + assert.deepEqual(serverConfigProblems(cfg({ url: 'https://x/mcp', token: 't', accessProvider: provider })), []); +}); + +test('a credential sent in cleartext to a host that is not loopback is warned about, never refused', () => { + const provider = async () => 'fresh'; + const warned = (c: Partial) => serverConfigWarnings(cfg(c)); + // The Authorization header over http, and the ?token= query over ws. + assert.match(warned({ url: 'http://10.0.0.5:8080/mcp', token: 't' })[0]!, /credential goes to 10\.0\.0\.5:8080 unencrypted \(http:\/\/\).*use https:\/\//); + assert.match(warned({ url: 'ws://mcp.lan/mcpl', accessProvider: provider })[0]!, /credential goes to mcp\.lan unencrypted \(ws:\/\/\).*use wss:\/\//); + assert.equal(serverConfigProblems(cfg({ url: 'http://10.0.0.5:8080/mcp', token: 't' })).length, 0, 'still usable'); + // Nothing to warn about: encrypted, loopback, no credential, or not dialed. + for (const quiet of [ + { url: 'https://10.0.0.5/mcp', token: 't' }, + { url: 'wss://mcp.lan/mcpl', token: 't' }, + { url: 'http://localhost:8080/mcp', token: 't' }, + { url: 'http://127.0.0.1:8080/mcp', token: 't' }, + { url: 'http://127.0.0.2:8080/mcp', token: 't' }, + { url: 'ws://0.0.0.0:8080/mcpl', token: 't' }, + { url: 'http://[::1]:8080/mcp', token: 't' }, + { url: 'ws://app.localhost/mcpl', token: 't' }, + { url: 'http://[::]:8080/mcp', token: 't' }, + { url: 'http://[::ffff:127.0.0.1]:8080/mcp', token: 't' }, + { url: 'http://[::ffff:127.9.8.7]/mcp', token: 't' }, + { url: 'http://localhost.:8080/mcp', token: 't' }, + { url: 'ws://app.localhost./mcpl', token: 't' }, + { url: 'http://localhost:8080/mcp?token=abc' }, + { url: 'http://10.0.0.5:8080/mcp' }, + { url: 'http://10.0.0.5:8080/mcp?region=eu&page=2' }, + { command: 'srv', url: 'http://10.0.0.5/mcp', token: 't' }, + { url: 'ftp://10.0.0.5/mcp', token: 't' }, + ]) assert.deepEqual(warned(quiet), [], JSON.stringify(quiet)); + // Not loopback: a mapped address outside 127/8, and a name that only ends in "localhost". + for (const loud of ['http://[::ffff:10.0.0.5]/mcp', 'http://notlocalhost/mcp']) { + assert.equal(warned({ url: loud, token: 't' }).length, 1, loud); + } +}); + +test('a secret the url carries itself is warned about too, and the warning never quotes it', () => { + const warned = (url: string, extra: Partial = {}) => serverConfigWarnings(cfg({ url, ...extra })); + assert.match(warned('ws://user:pw@10.0.0.5/mcpl')[0]!, /its url's userinfo goes to 10\.0\.0\.5 unencrypted \(ws:\/\/\).*read it;/); + assert.match(warned('ws://mcp.lan/mcpl?api_key=k1')[0]!, /its url's query parameter "api_key" goes to mcp\.lan unencrypted.*use wss:\/\//); + for (const name of ['token', 'Access_Token', 'apikey', 'client_secret', 'password', 'auth', 'X-Amz-Signature']) { + assert.equal(warned(`http://10.0.0.5/mcp?${name}=v`).length, 1, name); + } + const both = warned('ws://u:p@10.0.0.5/mcpl?token=a&sig_key=b®ion=eu', { token: 't' })[0]!; + assert.match(both, /its credential, its url's userinfo and its url's query parameters "token", "sig_key" go to 10\.0\.0\.5 unencrypted.*read them;/); + for (const text of [both, ...warned('ws://user:pw@10.0.0.5/mcpl?token=sekret')]) assert.ok(!/sekret|pw|u:p/.test(text), text); + // A name that may itself be a credential is warned about and counted, never quoted. + assert.deepEqual(warned('ws://mcp.lan/mcpl?sk-live-abcdef0123456789'), [ + 'MCP server "s": a query parameter whose name looks like a credential goes to mcp.lan unencrypted (ws://), where anything on the network path can read it; use wss:// unless that network is trusted', + ]); + const named = warned('ws://mcp.lan/mcpl?token=a&apikey_x9&x7y8z9w0v1u2t3s4r5')[0]!; + assert.match(named, /its url's query parameter "token" and 2 query parameters whose names look like credentials go to mcp\.lan unencrypted.*read them;/); + assert.ok(!/apikey_x9|x7y8z9/.test(named), named); + const path = warned('http://10.0.0.5/api/mcp/s/Zt4kQ9xW2vLm8pRs7YbN/mcp')[0]!; + assert.match(path, /a credential-like segment of its url's path goes to 10\.0\.0\.5 unencrypted.*read it;/); + assert.ok(!path.includes('Zt4kQ9xW2vLm8pRs7YbN'), path); + assert.deepEqual(warned('http://10.0.0.5/v2/2026-07-28/mcp'), [], 'a short segment with digits is a name'); + // Encrypted or loopback: nothing. + assert.deepEqual(warned('https://user:pw@10.0.0.5/mcp?token=a'), []); + assert.deepEqual(warned('http://user:pw@127.0.0.1/mcp?token=a'), []); +}); + +test('an access provider that throws is never quoted in the log, on either dial', async () => { + const message = 'refresh token sekret-123 expired'; + const accessProvider = async (): Promise => { throw new Error(message); }; + const logged: string[] = []; + const realError = console.error; + console.error = (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }; + try { + // Nothing listens on port 1: each dial fails after asking the provider. + await assert.rejects(WebSocketTransport.open(cfg({ id: 'ws', url: 'ws://127.0.0.1:1/mcpl', accessProvider }))); + await assert.rejects(ModernMcpConnection.connect(cfg({ id: 'http', url: 'http://127.0.0.1:1/mcp', accessProvider }))); + } finally { + console.error = realError; + } + const fallbacks = logged.filter((line) => line.includes('access provider failed')); + assert.equal(fallbacks.length, 2, logged.join('\n')); + for (const line of fallbacks) assert.ok(line.includes(`access provider failed (Error, ${message.length} characters)`), line); + assert.ok(!logged.some((line) => line.includes('sekret')), logged.join('\n')); +}); + +test('an access provider that throws is logged by its error class and length, not its text', () => { + assert.equal(describeCredentialError(new TypeError('refresh failed for sekret-abc')), 'TypeError, 29 characters'); + class ProviderError extends Error {} + assert.equal(describeCredentialError(new ProviderError('x')), 'ProviderError, 1 character'); + assert.equal(describeCredentialError('sekret'), 'a thrown string'); + assert.equal(describeCredentialError(undefined), 'a thrown undefined'); +}); + +test("an http(s) url with userinfo is refused, since the HTTP client can't send it and its error would quote it", () => { + for (const url of ['http://user:S3CRET@127.0.0.1:9/mcp', 'http://user:S3CRET@10.0.0.5/mcp', 'https://user:S3CRET@mcp.example/mcp', 'http://justauser@127.0.0.1:9/mcp']) { + const problems = serverConfigProblems(cfg({ url })); + assert.deepEqual(problems, [ + 'MCP server "s": an http(s):// url can\'t carry userinfo (user:password@), since the HTTP client refuses such a url; give the credential as "token" or "accessProvider"', + ], url); + // The modern engine checks at create(), so nothing is dialed with it. + assert.throws(() => ModernMcpConnection.create(cfg({ id: 's', url })), (error: Error) => error.message === problems[0]); + assert.deepEqual(serverConfigWarnings(cfg({ url })), [], 'a refusal, not also a warning'); + } + assert.deepEqual(serverConfigProblems(cfg({ url: 'ws://user:pw@127.0.0.1:9/mcpl' })), [], 'a WebSocket url sends its userinfo'); +}); + +test('both engines admit a configuration through checkServerConfig, which logs its warnings', () => { + const lines: string[] = []; + const realWarn = console.warn; + console.warn = (...args: unknown[]) => { lines.push(args.join(' ')); }; + try { + checkServerConfig(cfg({ url: 'ws://mcp.lan/mcpl', token: 't' })); + assert.equal(lines.length, 1); + assert.match(lines[0]!, /^\[mcp\] MCP server "s": its credential goes to mcp\.lan unencrypted/); + // The modern engine checks at create(), before anything is dialed. + ModernMcpConnection.create(cfg({ id: 'm', url: 'http://10.0.0.5:8080/mcp', token: 't' })); + assert.equal(lines.length, 2); + assert.match(lines[1]!, /MCP server "m": its credential goes to 10\.0\.0\.5:8080 unencrypted/); + checkServerConfig(cfg({ url: 'https://10.0.0.5/mcp', token: 't' })); + assert.equal(lines.length, 2, 'nothing for https'); + } finally { + console.warn = realWarn; + } +}); + +test("a query credential is removed as the WebSocket dial sends it, the query re-encoded once the token is set", () => { + const config = cfg({ url: 'wss://host.example/mcp?password=secret%20key123&q=a~b', token: 'tok-abcdefgh' }); + const sent = buildWebSocketUrl(config); + // The form the server receives, and can echo, as URLSearchParams writes it. + assert.ok(sent.includes('password=secret+key123') && sent.includes('q=a%7Eb'), sent); + const shown = redactServerText(`closed by the server: ${sent}`, config); + assert.ok(!shown.includes('key123'), shown); + assert.ok(!shown.includes('tok-abcdefgh'), shown); + assert.ok(!shown.includes('q=a%7Eb'), shown); +}); + +test("an access provider's token is removed once a dial has sent it, in a WebSocket url or an HTTP header", async () => { + const ws = cfg({ id: 'ws', url: 'ws://127.0.0.1:1/mcpl?password=secret%20key123', accessProvider: async () => 'prov/ws-0123456789' }); + const http = cfg({ id: 'http', url: 'http://127.0.0.1:1/mcp', accessProvider: async () => 'prov-http-0123456789' }); + // Made before the dial, as the framework makes a connection's. + const recorded = serverTextRedactor(ws); + assert.ok(recorded('token=prov%2Fws-0123456789').includes('ws-0123456789'), 'no dial has sent it yet'); + // Nothing listens on port 1: each dial notes its credential before it fails. + await assert.rejects(WebSocketTransport.open(ws)); + await assert.rejects(ModernMcpConnection.connect(http)); + const shown = recorded('closed: ws://127.0.0.1:1/mcpl?password=secret+key123&token=prov%2Fws-0123456789'); + assert.ok(!shown.includes('ws-0123456789') && !shown.includes('key123'), shown); + // The token in the forms a token takes, as JSON with its slash escaped. + const json = recorded('{"token":"prov\\/ws-0123456789"}'); + assert.ok(!json.includes('ws-0123456789'), json); + const header = redactServerText('refused: Authorization: Bearer prov-http-0123456789', http); + assert.ok(!header.includes('prov-http'), header); +}); + +test("a dial with an access provider takes its abort listener off the signal once the provider has answered", async () => { + // A reconnect loop's signal lives across many dials. + const controller = new AbortController(); + for (let n = 1; n <= 3; n++) { + await assert.rejects(WebSocketTransport.open(cfg({ id: 'ws', url: 'ws://127.0.0.1:1/mcpl', accessProvider: async () => `prov-loop-${n}-0123456789` }), controller.signal)); + assert.equal(getEventListeners(controller.signal, 'abort').length, 0, `after dial ${n}`); + } +}); + +test("a dial whose signal aborts while its access provider is called ends then, not when the provider answers", { timeout: 10_000 }, async () => { + const controller = new AbortController(); + await assert.rejects( + WebSocketTransport.open(cfg({ + id: 'ws', + url: 'ws://127.0.0.1:1/mcpl', + // The connection is closed while the provider runs, and the provider + // never answers. + accessProvider: () => { controller.abort(); return new Promise(() => {}); }, + }), controller.signal), + /WebSocket connect abandoned: the connection was closed/, + ); +}); + +test("a token a dial sends again is the newest, so the oldest of the others goes first", () => { + const config = cfg({ id: 'r', url: 'http://127.0.0.1:1/mcp' }); + noteDialSent(config, { token: 'prov-again-0000' }); + for (let n = 1; n <= 15; n++) noteDialSent(config, { token: `prov-other-${String(n).padStart(4, '0')}` }); + noteDialSent(config, { token: 'prov-again-0000' }); + noteDialSent(config, { token: 'prov-other-0016' }); + assert.equal(redactServerText('prov-again-0000', config), 'redacted'); + assert.equal(redactServerText('prov-other-0001', config), 'prov-other-0001', 'the oldest of the others is let go'); +}); + +test("a configuration keeps only its dials' newest provider tokens", () => { + const config = cfg({ id: 'p', url: 'http://127.0.0.1:1/mcp' }); + for (let n = 0; n <= 16; n++) noteDialSent(config, { token: `prov-token-${String(n).padStart(4, '0')}` }); + assert.ok(redactServerText('prov-token-0000', config).includes('prov-token-0000'), 'the oldest of 17 is let go'); + assert.equal(redactServerText('prov-token-0001 prov-token-0016', config), 'redacted redacted'); +}); + +test('recorded server text removes encoded and decoded URL credentials as exact strings', () => { + const config = cfg({ url: 'https://h/api/s/secret%2B0123456789/mcp?password=p%2Bassw0rd%21&bare%2Bsecret0123456789&plain=value0123456789abcdef®ion=eu&label=a%20b' }); + const secrets = ['secret%2B0123456789', 'secret+0123456789', 'p%2Bassw0rd%21', 'p+assw0rd!', 'bare%2Bsecret0123456789', 'bare+secret0123456789', 'value0123456789abcdef', 'region=eu', 'label=a%20b', 'label=a b']; + for (const secret of secrets) assert.equal(redactServerText(`queue full: ${secret}; eu`, config), 'queue full: redacted; eu', secret); +}); + +test('recorded server text removes WebSocket userinfo and the static token, escaping regex characters', () => { + const config = cfg({ url: 'wss://user%2B12345678:pass%2B12345678@h/mcp', token: 'token.[1234]' }); + for (const secret of ['user%2B12345678', 'user+12345678', 'pass%2B12345678', 'pass+12345678', config.token!]) { + assert.equal(redactServerText(`error: ${secret}`, config), 'error: redacted'); + } + assert.equal(redactServerText('tokenX11234]', config), 'tokenX11234]'); + assert.equal(redactServerText('static-token123', cfg({ token: 'static-token123' })), 'redacted'); + assert.equal(redactServerText('static-token123', cfg({ url: 'invalid', token: 'static-token123' })), 'redacted'); +}); + +test('recorded server text keeps short standalone values and unrelated words, but removes whole query pairs', () => { + const config = cfg({ url: 'https://h/mcp?auth=x&token=auth®ion=eu&debug', token: 'x' }); + assert.equal(redactServerText('auth x eu queue full debug', config), 'auth x eu queue full debug'); + assert.equal(redactServerText('auth=x token=auth region=eu', config), 'redacted redacted redacted'); + assert.equal(redactServerText('1234567 12345678', cfg({token:'1234567'})), '1234567 12345678'); + assert.equal(redactServerText('12345678', cfg({token:'12345678'})), 'redacted'); +}); + +test('recorded server text: the token as a url carries it, credential-named env values, only where nothing adjoins, and any value', () => { + // The static token, raw and as a url carries it, percent- or form-encoded. + const token = 'tok en+x/123='; + const tokenConfig = cfg({ token }); + // Percent-encoded (%20 for the space) and form-encoded (+ for it) differ here. + for (const form of [token, 'tok%20en%2Bx%2F123%3D', 'tok+en%2Bx%2F123%3D']) { + assert.equal(redactServerText(`refused: ${form}`, tokenConfig), 'refused: redacted', form); + } + // A stdio server's env: a value whose name has a part ending in a credential + // word (plural, or with digits after it, too), and no other + // (GIT_AUTHOR_NAME's part AUTHOR doesn't end in 'auth'). + const env = cfg({ command: 'srv', env: { + GITHUB_TOKEN: 'ghp_abcdef123456', API_KEY: 'k3y-abcdefgh', PGPASSWORD: 'pgpass-abcdefgh', ACCESSTOKEN: 'acc-abcdefgh12', + DB_PASS: 'dbpass-abcdefgh', SSH_KEY_PASSPHRASE: 'phrase-abcdefgh', + AUTHORIZATION: 'Bearer-abcdefgh', API_KEYS: 'k1-abcdefgh9', TOKEN2: 'tok2-abcdefgh', API_KEY_ID: 'keyid-abcdefgh', GITHUB_PAT: 'pat-abcdefgh12', + SESSION_COOKIE: 'cookie-abcdefgh', CLOUD_CREDS: 'creds-abcdefgh', + PATH: '/usr/local/bin', GIT_AUTHOR_NAME: 'Somebody Longname', KEYBOARD_LAYOUT: 'dvorak-programmer', TOKENIZERS_PARALLELISM: 'parallel-false', + } }); + assert.equal( + redactServerText('auth ghp_abcdef123456 k3y-abcdefgh pgpass-abcdefgh acc-abcdefgh12 dbpass-abcdefgh phrase-abcdefgh Bearer-abcdefgh k1-abcdefgh9 tok2-abcdefgh keyid-abcdefgh pat-abcdefgh12 cookie-abcdefgh creds-abcdefgh /usr/local/bin Somebody Longname dvorak-programmer parallel-false', env), + 'auth redacted redacted redacted redacted redacted redacted redacted redacted redacted redacted redacted redacted redacted /usr/local/bin Somebody Longname dvorak-programmer parallel-false'); + // A credential env value as a url would carry it, percent-encoded. + assert.equal(redactServerText('GET /x?pw=p%40ss%2Fw0rd!x', cfg({ command: 'srv', env: { DB_PASSWORD: 'p@ss/w0rd!x' } })), 'GET /x?pw=redacted'); + // A value that isn't a string (a JavaScript host can pass one) is skipped, not thrown on. + assert.equal(redactServerText('fine', cfg({ command: 'srv', env: { API_TOKEN: null as unknown as string, OTHER_KEY: 7 as unknown as string } })), 'fine'); + // A short pair only where no letter or digit adjoins it; any other string + // wherever it occurs, after a percent escape, an ANSI code or a JSON escape too. + const pair = cfg({ url: 'https://h/api/s/pathSecret0123456789/mcp?v=2' }); + assert.equal(redactServerText('dev=2 ready; v=2 sent', pair), 'dev=2 ready; redacted sent'); + // A long pair too: only as its own name=value. + const longPair = cfg({ url: 'https://h/mcp?v=12345678' }); + assert.equal(redactServerText('dev=12345678 ready; v=12345678 sent', longPair), 'dev=12345678 ready; redacted sent'); + assert.equal(redactServerText('a%2FpathSecret0123456789 \u001b[31mpathSecret0123456789 "\\npathSecret0123456789"', pair), + 'a%2Fredacted \u001b[31mredacted "\\nredacted"'); + // A ws:// url's userinfo, as for wss://. + const ws = cfg({ url: 'ws://wsuser1234:wspass5678@h/mcp' }); + assert.equal(redactServerText('connect wsuser1234 wspass5678', ws), 'connect redacted redacted'); + // A url with a stray percent sign: its value is kept as written. + const stray = cfg({ url: 'https://h/mcp?note=100%&key=abcdefgh1234' }); + assert.equal(redactServerText('note=100% key=abcdefgh1234', stray), 'redacted redacted'); + // Any value reads as text: a rejection that isn't an Error doesn't throw, + // and neither does one that can't be printed. + assert.equal(redactServerText(undefined, tokenConfig), 'undefined'); + assert.equal(redactServerText({ toString: () => `obj ${token}` }, tokenConfig), 'obj redacted'); + assert.equal(redactServerText(Object.create(null), tokenConfig), '[a value that cannot be printed]'); + assert.equal(redactServerText({ toString() { throw new Error('no'); } }, tokenConfig), '[a value that cannot be printed]'); + // A token that can't be url-encoded (an unpaired surrogate) is still replaced as written. + assert.equal(redactServerText('x abcdefgh\ud800ij', cfg({ token: 'abcdefgh\ud800ij' })), 'x redacted'); +}); + +test('the cleartext warning counts a credential-shaped value under a plain name, as redaction does', () => { + const warned = serverConfigWarnings(cfg({ url: 'http://remote.example/mcp?session=Zx81kQ0pL2mN7vB4tY6c' })); + assert.equal(warned.length, 1); + assert.match(warned[0], /its url's query parameter "session" goes to remote\.example unencrypted/); +}); + +test('recorded server text: a slash escaped as JSON may write it, and percent-escapes in lowercase hex', () => { + const config = cfg({ token: 'tok/en+x0123', env: { API_TOKEN: 'env/secret+0123' } } as Partial); + for (const form of ['tok/en+x0123', 'tok\\/en+x0123', 'tok%2Fen%2Bx0123', 'tok%2fen%2bx0123', 'env\\/secret+0123', 'env%2fsecret%2b0123']) { + assert.equal(redactServerText(`refused: ${form}`, config), 'refused: redacted', form); + } + // A token whose form encoding differs from its percent encoding: a space is + there, %20 here. + const spaced = cfg({ token: 'tok en/x0123' }); + for (const form of ['tok+en%2Fx0123', 'tok+en%2fx0123', 'tok%20en%2Fx0123', 'tok%20en%2fx0123']) { + assert.equal(redactServerText(`refused: ${form}`, spaced), 'refused: redacted', form); + } +}); + +test("recorded server text: a ws:// url's userinfo as one unit before its @, and the Basic credential sent for it", () => { + // Neither part is long enough to replace on its own, nor is the unit. + const config = cfg({ url: 'wss://bot:pw@h/mcp' }); + assert.equal(redactServerText('dial wss://bot:pw@h/mcp failed', config), 'dial wss://redacted@h/mcp failed'); + assert.equal(redactServerText('Authorization: Basic Ym90OnB3', config), 'Authorization: Basic redacted'); + // The @ and a boundary before it anchor the unit. + assert.equal(redactServerText('bot:pw is not followed by its @ here', config), 'bot:pw is not followed by its @ here'); + assert.equal(redactServerText('robot:pw@h', config), 'robot:pw@h'); + assert.equal(redactServerText('dial wss://alice:pw@h/mcp failed', cfg({ url: 'wss://alice:pw@h/mcp' })), 'dial wss://redacted@h/mcp failed'); + // ws:// as wss://. + const plain = cfg({ url: 'ws://bot:pw@h/mcp' }); + assert.equal(redactServerText('dial ws://bot:pw@h/mcp failed', plain), 'dial ws://redacted@h/mcp failed'); + assert.equal(redactServerText('Authorization: Basic Ym90OnB3', plain), 'Authorization: Basic redacted'); + // A username alone, as a token written as the username is: the unit is the + // username, and the Basic credential is that of "name:". + for (const url of ['ws://k3y@h/mcp', 'wss://k3y@h/mcp']) { + const tokenOnly = cfg({ url }); + assert.equal(redactServerText(`dial ${url} failed`, tokenOnly), `dial ${url.replace('k3y', 'redacted')} failed`, url); + assert.equal(redactServerText(`Authorization: Basic ${Buffer.from('k3y:').toString('base64')}`, tokenOnly), 'Authorization: Basic redacted', url); + assert.equal(redactServerText('monk3y@h', tokenOnly), 'monk3y@h', url); + } + // Percent-encoded, with every part too short to replace on its own: the + // unit as written and as decoded, and the Basic credential sent for each. + const encoded = cfg({ url: 'wss://b%40t:p%40ss@h/mcp' }); + assert.equal(redactServerText('dial wss://b%40t:p%40ss@h/mcp failed', encoded), 'dial wss://redacted@h/mcp failed'); + assert.equal(redactServerText('as user b@t:p@ss@h', encoded), 'as user redacted@h'); + for (const userinfo of ['b%40t:p%40ss', 'b@t:p@ss']) { + assert.equal(redactServerText(`Authorization: Basic ${Buffer.from(userinfo).toString('base64')}`, encoded), 'Authorization: Basic redacted', userinfo); + } +}); + +test('recorded server text: a credential as JSON.stringify writes it, its quotes and backslashes escaped', () => { + const password = 'pa"ss\\w0rd/x1'; + const config = cfg({ command: 'srv', token: 'to"ken\\0123', env: { DB_PASSWORD: password } } as Partial); + for (const secret of [password, config.token!]) { + const json = JSON.stringify(secret).slice(1, -1); + for (const form of [secret, json, json.replace(/\//g, '\\/')]) { + assert.equal(redactServerText(`config {"secret":"${form}"} refused`, config), 'config {"secret":"redacted"} refused', form); + } + } +}); + +test("one redactor over several configurations' strings takes the longest secret among them all", () => { + const short = cfg({ id: 'a', token: 'ghp_SHAREDPREFIX123' }); + const long = cfg({ id: 'b', token: 'ghp_SHAREDPREFIX123_but_longer_TAIL9' }); + const redact = credentialRedactor([...serverCredentialPatterns(short), ...serverCredentialPatterns(long)]); + assert.equal(redact('auth failed for ghp_SHAREDPREFIX123_but_longer_TAIL9'), 'auth failed for redacted'); + assert.equal(redact('auth failed for ghp_SHAREDPREFIX123'), 'auth failed for redacted'); + assert.equal(credentialRedactor([])('nothing to replace'), 'nothing to replace'); +}); + +test("one redactor over several configurations' strings replaces strings that overlap, both whole", () => { + // The first's end is the second's start. + const head = cfg({ id: 'a', token: 'XXXXXXXXYYYY' }); + const tail = cfg({ id: 'b', token: 'YYYYZZZZZZZZ' }); + const redact = credentialRedactor([...serverCredentialPatterns(head), ...serverCredentialPatterns(tail)]); + assert.equal(redact('auth failed for XXXXXXXXYYYYZZZZZZZZ.'), 'auth failed for redacted.'); + assert.equal(redact('XXXXXXXXYYYY and YYYYZZZZZZZZ'), 'redacted and redacted'); + // Strings next to each other, not overlapping, are each replaced, as before. + assert.equal(redact('XXXXXXXXYYYYYYYYZZZZZZZZ'), 'redactedredacted'); +}); + +test('recorded server text uses longest-first, one-pass replacements and is idempotent', () => { + const config = cfg({ url: 'https://h/mcp?secret=token0123456789-tail&redacted=eu', token: 'token0123456789' }); + const original = 'secret=token0123456789-tail token0123456789 redacted=eu queue full'; + const clean = redactServerText(original, config); + assert.equal(clean, 'redacted redacted redacted queue full'); + assert.equal(redactServerText('token0123456789-tail', config), 'redacted'); + assert.equal(redactServerText(clean, config), clean); +}); diff --git a/test/mcp-recorded-text.test.ts b/test/mcp-recorded-text.test.ts new file mode 100644 index 000000000..aceff1e8f --- /dev/null +++ b/test/mcp-recorded-text.test.ts @@ -0,0 +1,347 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { createServer } from 'node:http'; +import type { AddressInfo } from 'node:net'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { Client, type Transport } from '@modelcontextprotocol/client'; +import { EventEmitter } from 'node:events'; +import type { ModuleContext } from '../src/index.js'; +import { AgentFramework } from '../src/framework.js'; +import { McplProtocolVersionError, McplServerConnection } from '../src/mcpl/server-connection.js'; +import { ModernMcpConnection, malformedLineDiagnostic } from '../src/mcpl/modern-connection.js'; +import { noteDialSent } from '../src/mcpl/protocol-family.js'; +import { MockMembrane } from './helpers/mock-membrane.js'; + +/** Discovery succeeds; the inventory request gets a normal echoed-path 404. */ +function echo404Server() { + return createServer(async (req, res) => { + const chunks: Buffer[] = []; + for await (const chunk of req) chunks.push(chunk as Buffer); + const message = JSON.parse(Buffer.concat(chunks).toString('utf8')); + if (message.method === 'server/discover') { + res.writeHead(200, { 'content-type': 'application/json', 'mcp-protocol-version': '2026-07-28' }).end(JSON.stringify({ + jsonrpc: '2.0', id: message.id, result: { resultType: 'complete', supportedVersions: ['2026-07-28'], capabilities: { tools: {} }, ttlMs: 60000, cacheScope: 'private', _meta: { 'io.modelcontextprotocol/serverInfo': { name: 'echo', version: '1' } } }, + })); + } else { + res.writeHead(404, { 'content-type': 'text/plain' }).end(`Cannot POST ${req.url}`); + } + }); +} + +/** The SDK's pin-mode discovery rewrites an HTTP 404 into a version error. + * Send tools/list through the real transport at the connect seam so this test + * isolates how connection recordings handle an SDK error with the real body. */ +function failConnectWithHttpBody(t: import('node:test').TestContext): void { + t.mock.method(Client.prototype, 'connect', async (transport: Transport) => { + await transport.start(); + try { await transport.send({ jsonrpc: '2.0', id: 1, method: 'tools/list' }); } + finally { await transport.close(); } + }); +} + +const secret = 'pathSecret0123456789'; +const config = { id: 'remote', url: `https://h/api/mcp/s/${secret}/mcp`, token: 'staticToken012345' }; + +test('both framework engines scrub failure, error and stderr records before tracing and alerting', () => { + for (const method of ['wireMcplEvents', 'wireModernEvents']) { + const traces: unknown[] = []; + const alerts: string[] = []; + const framework = Object.create(AgentFramework.prototype) as any; + framework.mcplServerConfigs = new Map([[config.id, config]]); + framework.emitTrace = (event: unknown) => traces.push(event); + framework.opsAlert = (_kind: string, _id: string, text: string) => alerts.push(text); + // A permanent failure also refreshes the tool list. + framework.handleToolsListChanged = () => {}; + const connection = Object.assign(new EventEmitter(), { id: config.id, willReconnect: true }); + framework[method](connection, config); + connection.emit('connect-failed', { error: `Cannot POST ${secret}`, attempt: 0 }); + connection.emit('reconnect-failed', { error: `Cannot POST ${secret}`, attempt: 1, permanent: true }); + connection.emit('reconnect-failed', { error: `Cannot POST ${secret}`, attempt: 5 }); + connection.emit('error', new Error(`transport ${secret}`)); + connection.emit('stderr', { line: `server ${config.token}` }); + assert.equal(traces.length, 5); + assert.equal(alerts.length, 2); + assert.ok(!JSON.stringify(traces).includes(secret), method); + assert.ok(!JSON.stringify(traces).includes(config.token), method); + assert.ok(alerts.every(text => text.includes('Cannot POST redacted')), method); + } +}); + +test("both engines' wiring redacts a stdio server's credential env value in its stderr records", () => { + const stdio = { id: 'local', command: 'srv', env: { API_TOKEN: 'envToken0123456789', PATH: '/usr/local/bin' } }; + for (const method of ['wireMcplEvents', 'wireModernEvents']) { + const traces: unknown[] = []; + const framework = Object.create(AgentFramework.prototype) as any; + framework.mcplServerConfigs = new Map([[stdio.id, stdio]]); + framework.emitTrace = (event: unknown) => traces.push(event); + framework.opsAlert = () => {}; + const connection = Object.assign(new EventEmitter(), { id: stdio.id, willReconnect: false }); + framework[method](connection, stdio); + connection.emit('stderr', { line: 'auth with envToken0123456789 from /usr/local/bin' }); + assert.equal(traces.length, 1, method); + assert.match(JSON.stringify(traces[0]), /auth with redacted from \/usr\/local\/bin/, method); + } +}); + +test('an SDK HTTP 404 error keeps its path but hides the secret in connection console, traces and ops alerts', async (t) => { + failConnectWithHttpBody(t); + const dir = mkdtempSync(join(tmpdir(), 'mcp-recorded-text-')); + const consoleLines: string[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => consoleLines.push(args.map(String).join(' '))); + const traces: Array<{ type: string; error?: string; message?: string; attempt?: number }> = []; + const server = echo404Server(); + await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)); + const port = (server.address() as AddressInfo).port; + let framework: AgentFramework | undefined; + try { + framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], + modules: [{ name: 'watcher', async start(ctx: ModuleContext) { ctx.onTrace(event => traces.push(event as never)); }, async stop() {}, getTools: () => [], async handleToolCall() { return { success: false }; }, async onProcess() { return {}; } }], + mcplServers: [{ id: 'echo', url: `http://127.0.0.1:${port}/api/mcp/s/${secret}/mcp`, reconnect: true, reconnectIntervalMs: 10, reconnectMaxIntervalMs: 20 }], + }); + // The loop ends as soon as the record appears; the ceiling only bounds a hang. + const deadline = Date.now() + 30_000; + while (!traces.some(e => e.type === 'ops:alert') && Date.now() < deadline) await new Promise(resolve => setTimeout(resolve, 10)); + const failed = traces.filter(e => e.type === 'mcpl:server-connect-failed'); + assert.ok(failed.some(e => e.attempt === 0)); + assert.ok(failed.some(e => (e.attempt ?? 0) >= 5)); + assert.ok(failed.every(e => e.error?.includes('Cannot POST /api/mcp/s/redacted/mcp')), JSON.stringify(failed)); + assert.ok(traces.some(e => e.type === 'ops:alert' && e.message?.includes('Cannot POST /api/mcp/s/redacted/mcp'))); + assert.ok(consoleLines.some(line => line.includes('Cannot POST /api/mcp/s/redacted/mcp'))); + assert.ok(!JSON.stringify(traces).includes(secret)); + assert.ok(!consoleLines.join('\n').includes(secret)); + } finally { + await framework?.stop(); + server.closeAllConnections(); + await new Promise(resolve => server.close(() => resolve())); + rmSync(dir, { recursive: true, force: true }); + } +}); + + +test('startup records redact an echoed path while the direct host connect error stays as raised', async (t) => { + failConnectWithHttpBody(t); + const dir = mkdtempSync(join(tmpdir(), 'mcp-startup-text-')); + const consoleLines: string[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => consoleLines.push(args.map(String).join(' '))); + const traces: Array<{ type: string; error?: string }> = []; + const server = echo404Server(); + await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)); + const url = `http://127.0.0.1:${(server.address() as AddressInfo).port}/api/mcp/s/${secret}/mcp`; + let framework: AgentFramework | undefined; + try { + await assert.rejects(ModernMcpConnection.connect({ id: 'raw', url }), error => { + assert.ok(error instanceof Error && error.message.includes(secret)); + return true; + }); + framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], + modules: [{ name: 'watcher', async start(ctx: ModuleContext) { ctx.onTrace(event => traces.push(event as never)); }, async stop() {}, getTools: () => [], async handleToolCall() { return { success: false }; }, async onProcess() { return {}; } }], + mcplServers: [{ id: 'echo', url }], + }); + assert.ok(traces.some(e => e.type === 'mcpl:server-connect-failed' && e.error?.includes('Cannot POST /api/mcp/s/redacted/mcp'))); + assert.ok(consoleLines.some(line => line.includes('Failed to connect MCPL server') && line.includes('Cannot POST /api/mcp/s/redacted/mcp'))); + assert.ok(!JSON.stringify(traces).includes(secret)); + assert.ok(!consoleLines.join('\n').includes(secret)); + } finally { + await framework?.stop(); + server.closeAllConnections(); + await new Promise(resolve => server.close(() => resolve())); + rmSync(dir, { recursive: true, force: true }); + } +}); + + +test('legacy initial and reconnect console records redact known credentials', async (t) => { + const lines: string[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => lines.push(args.map(String).join(' '))); + t.mock.method(McplServerConnection as any, 'handshake', async () => { throw new Error(`transport ${secret}`); }); + const legacyConfig = { id: 'legacy', url: `ws://127.0.0.1/s/${secret}`, reconnect: true, reconnectIntervalMs: 10, reconnectMaxIntervalMs: 20 }; + const capabilities = { version: '0.5', pushEvents: true }; + const connection = await McplServerConnection.connectWithReconnect(legacyConfig, capabilities); + try { + // The loop ends as soon as the record appears; the ceiling only bounds a hang. + const deadline = Date.now() + 30_000; + while (!lines.some(line => line.includes('reconnect failed:')) && Date.now() < deadline) await new Promise(resolve => setTimeout(resolve, 10)); + assert.ok(lines.some(line => line.includes('initial connect failed, will retry: transport redacted'))); + assert.ok(lines.some(line => line.includes('reconnect failed: transport redacted'))); + assert.ok(!lines.join('\n').includes(secret)); + } finally { await connection.close(); } +}); + +test('a real discover-then-tools/list 404 is redacted in the framework error trace', async (t) => { + t.mock.method(console, 'error', () => {}); + const dir = mkdtempSync(join(tmpdir(), 'mcp-list-text-')); + const traces: Array<{ type: string; error?: string }> = []; + const server = echo404Server(); + await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)); + let framework: AgentFramework | undefined; + try { + framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], + modules: [{ name: 'watcher', async start(ctx: ModuleContext) { ctx.onTrace(event => traces.push(event as never)); }, async stop() {}, getTools: () => [], async handleToolCall() { return { success: false }; }, async onProcess() { return {}; } }], + mcplServers: [{ id: 'echo', url: `http://127.0.0.1:${(server.address() as AddressInfo).port}/api/mcp/s/${secret}/mcp` }], + }); + assert.ok(traces.some(e => e.type === 'mcpl:server-error' && e.error?.includes('Cannot POST /api/mcp/s/redacted/mcp')), JSON.stringify(traces)); + assert.ok(!JSON.stringify(traces).includes(secret)); + } finally { + await framework?.stop(); + server.closeAllConnections(); + await new Promise(resolve => server.close(() => resolve())); + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("a tool call's failure, from either engine, reaches its caller and its trace without the server's credentials", async () => { + // A legacy MCPL server's call rejected with a close reason that quotes the + // url; the modern engine's transport text is requestFailure's, tested over HTTP. + const traces: Array<{ type: string; error?: string; stack?: string }> = []; + const framework = Object.create(AgentFramework.prototype) as any; + framework.mcplServerConfigs = new Map([[config.id, config]]); + framework.emitTrace = (event: { type: string }) => traces.push(event); + const server = { id: config.id, sendToolsCall: async () => { throw new Error(`socket closed: 1008 /api/mcp/s/${secret}/mcp`); } }; + framework.mcplServerRegistry = { getServer: (id: string) => (id === config.id ? server : null) }; + framework.modernServers = () => new Map(); + const result = await framework.executeMcplToolCall({ id: 'c1', name: 'mcpl--remote--echo', input: {} }, config.id, 'mcpl--remote'); + assert.equal(result.success, false); + assert.match(String(result.error), /socket closed: 1008 \/api\/mcp\/s\/redacted\/mcp/); + // A redactor for any server's text is made before an operation waits, so + // a server removed meanwhile still has its credentials replaced. + const other = { id: 'other', url: 'https://g/api/s/otherSecret0123456789/mcp' }; + framework.mcplServerConfigs = new Map([[config.id, config], [other.id, other]]); + const everyServer = framework.everyServerRedactor(); + framework.mcplServerConfigs = new Map(); + assert.equal(everyServer(`closed ${secret} and otherSecret0123456789`), 'closed redacted and redacted'); + // A server connected after it was made is covered too. + framework.mcplServerConfigs = new Map([['late', { id: 'late', url: 'https://k/api/s/lateSecret0123456789/mcp' }]]); + assert.equal(everyServer('closed lateSecret0123456789'), 'closed redacted'); + // What a server's dial sent after the redactor was made is covered, though + // the server has gone by the time the text is recorded. + const dialing = { id: 'dialing', url: 'https://d/mcp' }; + framework.mcplServerConfigs = new Map([[dialing.id, dialing]]); + const beforeDial = framework.everyServerRedactor(); + noteDialSent(dialing, { token: 'prov-late-0123456789' }); + framework.mcplServerConfigs = new Map(); + assert.equal(beforeDial('refused prov-late-0123456789'), 'refused redacted'); + // A value that can't be printed reads as a placeholder. + assert.equal(everyServer(Object.create(null)), '[a value that cannot be printed]'); + // One pass over every server's strings: a secret that begins another + // server's longer one doesn't cut that one short. + framework.mcplServerConfigs = new Map([ + ['a', { id: 'a', token: 'ghp_SHAREDPREFIX123' }], + ['b', { id: 'b', token: 'ghp_SHAREDPREFIX123_but_longer_TAIL9' }], + ]); + assert.equal(framework.everyServerRedactor()('auth failed for ghp_SHAREDPREFIX123_but_longer_TAIL9'), 'auth failed for redacted'); + // The same across the two sets it reads: the secret of a server present + // when it was made begins the longer secret of a server connected since. + framework.mcplServerConfigs = new Map([['a', { id: 'a', token: 'ghp_SHAREDPREFIX123' }]]); + const across = framework.everyServerRedactor(); + framework.mcplServerConfigs = new Map([['b', { id: 'b', token: 'ghp_SHAREDPREFIX123_but_longer_TAIL9' }]]); + assert.equal(across('auth failed for ghp_SHAREDPREFIX123_but_longer_TAIL9'), 'auth failed for redacted'); + framework.mcplServerConfigs = new Map([[config.id, config]]); + + // The model's dispatch: the tool:failed trace (message and stack) and the + // result the model gets. + const events: Array<{ type: string; result?: { error?: string } }> = []; + framework.pushEvent = (event: { type: string }) => events.push(event); + framework.toolLifecycleEmitter = null; + const modern = { callTool: async () => { throw new Error(`HTTP 500 at /api/mcp/s/${secret}/mcp with ${config.token}`); } }; + framework.modernServers = () => new Map([[config.id, modern]]); + framework.dispatchMcplToolCall('assistant', { id: 'c2', name: 'mcpl--remote--echo', input: {} }, config.id, 'mcpl--remote'); + // The server is disconnected while the call is in flight: its failure is + // still recorded with the configuration it was dispatched under. + framework.mcplServerConfigs.delete(config.id); + const until = Date.now() + 30_000; + while (!(traces.some((event) => event.type === 'tool:failed') && events.length > 0) && Date.now() < until) { + await new Promise((resolve) => setTimeout(resolve, 5)); + } + const failed = traces.find((event) => event.type === 'tool:failed'); + assert.ok(failed, 'the failure is traced'); + for (const text of [failed!.error, failed!.stack, events[0]?.result?.error]) { + assert.ok(text && !text.includes(secret) && !text.includes(config.token), String(text)); + } + assert.match(String(events[0]?.result?.error), /HTTP 500 at \/api\/mcp\/s\/redacted\/mcp with redacted/); +}); + +test("a malformed line's diagnostic is redacted before it is cut, so no part of a credential across the cut is kept", () => { + const envSecret = 'envSecretValue0123456789'; + const line = JSON.stringify({ note: `${'p'.repeat(185)} ${envSecret}` }); + // The credential starts this far into the line; a cut at 200 would keep + // the first (200 - start) of its characters. + const start = line.indexOf(envSecret); + const kept = envSecret.slice(0, 200 - start); + assert.ok(kept.length > 0 && kept.length < envSecret.length, `the cut falls inside the credential (${kept.length})`); + const diagnostic = malformedLineDiagnostic({ id: 'stdio-env', command: 'srv', env: { API_TOKEN: envSecret } } as never, line, new Error('not a message')); + assert.match(diagnostic, /sent a malformed JSON-RPC message/); + assert.ok(!diagnostic.includes(kept), diagnostic); + // The parser's own message can quote the line; it's redacted too. + const quoting = malformedLineDiagnostic({ id: 'stdio-env', command: 'srv', env: { API_TOKEN: envSecret } } as never, '{}', new Error(`unexpected ${envSecret}`)); + assert.ok(!quoting.includes(envSecret), quoting); + assert.match(diagnostic, / redac/, 'the replacement, cut where the line is cut'); +}); + +test("the startup alert for a server with no common protocol revision carries the server's text without its credentials", async (t) => { + // The version verdict quotes the SDK's error, which is server text. + t.mock.method(ModernMcpConnection.prototype as never, 'open' as never, (async () => { + throw new McplProtocolVersionError(`MCP server "echo" does not support MCP 2025-11-25 (refused at /api/mcp/s/${secret}/mcp).`, 'rejected', null, null); + }) as never); + const dir = mkdtempSync(join(tmpdir(), 'mcp-startup-alert-')); + t.mock.method(console, 'error', () => {}); + const traces: Array<{ type: string; message?: string; error?: string }> = []; + let framework: AgentFramework | undefined; + try { + framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], + modules: [{ name: 'watcher', async start(ctx: ModuleContext) { ctx.onTrace(event => traces.push(event as never)); }, async stop() {}, getTools: () => [], async handleToolCall() { return { success: false }; }, async onProcess() { return {}; } }], + mcplServers: [{ id: 'echo', url: `https://127.0.0.1:9/api/mcp/s/${secret}/mcp` }], + }); + const alert = traces.find(e => e.type === 'ops:alert'); + assert.ok(alert, JSON.stringify(traces.map(e => e.type))); + assert.match(String(alert!.message), /MCPL server unreachable at startup: .*refused at \/api\/mcp\/s\/redacted\/mcp/); + assert.ok(!JSON.stringify(traces).includes(secret)); + } finally { + await framework?.stop(); + rmSync(dir, { recursive: true, force: true }); + } +}); + +test("a runtime connect's failed cleanup is traced without the server's credentials, and the host's error stays as raised", async (t) => { + const prototype = ModernMcpConnection.prototype as unknown as { open(): Promise }; + const open = t.mock.method(prototype, 'open', async () => { + throw new Error('connect refused'); + }); + const close = t.mock.method(ModernMcpConnection.prototype, 'close', async () => { + throw new Error(`reap failed for /api/mcp/s/${secret}/mcp`); + }); + const dir = mkdtempSync(join(tmpdir(), 'mcp-cleanup-trace-')); + t.mock.method(console, 'error', () => {}); + const traces: Array<{ type: string; error?: string }> = []; + let framework: AgentFramework | undefined; + try { + framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], + modules: [{ name: 'watcher', async start(ctx: ModuleContext) { ctx.onTrace(event => traces.push(event as never)); }, async stop() {}, getTools: () => [], async handleToolCall() { return { success: false }; }, async onProcess() { return {}; } }], + }); + await assert.rejects(framework.connectMcplServer({ id: 'echo', url: `https://127.0.0.1:9/api/mcp/s/${secret}/mcp` }), (error: unknown) => { + assert.ok(error instanceof Error && error.message.includes(secret), 'the host gets the error as raised'); + return true; + }); + const traced = traces.find(e => e.type === 'mcpl:server-error'); + assert.ok(traced, JSON.stringify(traces.map(e => e.type))); + assert.match(String(traced!.error), /reap failed for \/api\/mcp\/s\/redacted\/mcp/); + assert.ok(!JSON.stringify(traces).includes(secret)); + } finally { + open.mock.restore(); + close.mock.restore(); + await framework?.stop().catch(() => {}); + rmSync(dir, { recursive: true, force: true }); + } +}); + diff --git a/test/mcp-server-ownership.test.ts b/test/mcp-server-ownership.test.ts new file mode 100644 index 000000000..b8da064c7 --- /dev/null +++ b/test/mcp-server-ownership.test.ts @@ -0,0 +1,1060 @@ +/** + * One owner per MCP server id across both engines: from a connect's + * admission until its teardown is done, exactly one owner holds an id. + * - a connect is refused while a connect or disconnect for the id is in + * flight, a legacy handshake included; + * - a disconnect waits for what held the id before it, so nothing it removes + * belongs to a later connection, and each release frees only its own claim; + * - a teardown that can't confirm a child exited isn't finished: the server + * stays registered, closed, a connect is refused, and a retried disconnect + * completes once the exit is confirmed. + */ +import { test, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, readFileSync, existsSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { AgentFramework, ModernMcpConnection, McplUnreapedLaunchError, ShutdownTimeoutError } from '../src/index.js'; +import type { McplServerConfig } from '../src/mcpl/types.js'; +import { StdioTransport } from '../src/mcpl/transport.js'; +import type { McplServerConnection } from '../src/mcpl/server-connection.js'; +import { MockMembrane } from './helpers/mock-membrane.js'; + +const here = dirname(fileURLToPath(import.meta.url)); +const RAW = join(here, 'fixtures', 'modern-raw-server.mjs'); +const LEGACY = join(here, 'fixtures', 'legacy-version-server.mjs'); + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +const lines = (file: string) => (existsSync(file) ? readFileSync(file, 'utf8').split('\n').filter(Boolean) : []); +/** Launch pids from a raw modern server's JSONL log, or a legacy start log. */ +const launches = (file: string): number[] => lines(file).flatMap((l) => { + if (l.startsWith('start ')) return [Number(l.slice('start '.length))]; + const entry = JSON.parse(l) as { event?: string; pid?: number }; + return entry.event === 'start' ? [entry.pid!] : []; +}); +const alive = (pid: number) => { try { process.kill(pid, 0); return true; } catch { return false; } }; + +async function until(cond: () => boolean, what: string, ms = 10_000): Promise { + const deadline = Date.now() + ms; + while (!cond() && Date.now() < deadline) await new Promise((r) => setTimeout(r, 20)); + assert.ok(cond(), `in time: ${what}`); +} + +async function withFramework(fn: (h: { framework: AgentFramework; dir: string }) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), 'mcp-ownership-')); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [], + }); + try { + await fn({ framework, dir }); + } finally { + await framework.stop(); + rmSync(dir, { recursive: true, force: true }); + } +} + +const modern = (id: string, flags: string, log: string): McplServerConfig => + ({ id, command: process.execPath, args: [RAW, flags, log], protocol: 'modern' }); +const legacy = (id: string, mode: string, log: string, env: Record = {}): McplServerConfig => + ({ id, command: process.execPath, args: [LEGACY, mode, log], env }); +const status = (framework: AgentFramework, id: string) => framework.listMcplServers().find((s) => s.id === id); +const toolsOf = (framework: AgentFramework, id: string) => framework.getAllTools().map((t) => t.name).filter((n) => n.startsWith(`mcpl--${id}--`)); +const call = (name: string) => ({ id: `t-${Math.random()}`, name, input: {} }); +const RECONNECT = { reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 } as const; +/** A legacy server's connection, from the framework's registry. */ +const legacyServer = (framework: AgentFramework, id: string) => + (framework as unknown as { mcplServerRegistry: { getServer(id: string): McplServerConnection | null } }).mcplServerRegistry.getServer(id)!; + +/** + * Stdio launches can't be signalled while `stuck.active` is set, so their + * cleanup can't reap them; `kill` really ends one (Olive-1804's pattern). + */ +function unreapable(): { active: boolean; stuck: Array<{ pid: number; kill: () => void }> } { + const realSpawn = StdioTransport.spawn; + const state = { active: true, stuck: [] as Array<{ pid: number; kill: () => void }> }; + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = async (c) => { + const transport = await realSpawn.call(StdioTransport, c); + if (state.active) { + const child = (transport as unknown as { child: { pid: number; kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + child.kill = () => true; + state.stuck.push({ pid: child.pid, kill: () => { realKill('SIGKILL'); } }); + } + return transport; + }; + cleanups.push(() => { + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = realSpawn; + for (const s of state.stuck) s.kill(); + }); + return state; +} + +async function killAndWait(stuck: { pid: number; kill: () => void }): Promise { + stuck.kill(); + await until(() => !alive(stuck.pid), `child ${stuck.pid} gone`); +} + +test('a connect during a disconnect is refused; once the teardown is done, the new server is listed and routed', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + // A child that ignores SIGTERM keeps the teardown in flight for ~2 s. + const config = modern('m', 'ignore-sigterm', log); + await framework.connectMcplServer(config); + const disconnecting = framework.disconnectMcplServer('m'); + await assert.rejects(framework.connectMcplServer(config), /MCP server "m" is still being disconnected/); + assert.equal(launches(log).length, 1, 'nothing launched beside the closing child'); + await disconnecting; + assert.equal(status(framework, 'm'), undefined); + + await framework.connectMcplServer(config); + assert.equal(status(framework, 'm')?.connected, true, 'listed: its routing was not erased'); + assert.ok(toolsOf(framework, 'm').includes('mcpl--m--plain')); + assert.deepEqual((await framework.executeToolCall(call('mcpl--m--plain'))).data, [{ type: 'text', text: 'plain' }]); + }); +}); + +test('two simultaneous disconnects both settle, in order, and leave the id free', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + const config = modern('m', 'ignore-sigterm', log); + await framework.connectMcplServer(config); + const order: string[] = []; + const first = framework.disconnectMcplServer('m').then(() => order.push('first')); + const second = framework.disconnectMcplServer('m').then(() => order.push('second')); + await Promise.all([first, second]); + assert.deepEqual(order, ['first', 'second']); + assert.equal(status(framework, 'm'), undefined); + assert.equal(alive(launches(log)[0]!), false); + await framework.connectMcplServer(config); + assert.equal(status(framework, 'm')?.connected, true); + }); +}); + +test('a hold releases only its own claim: a disconnect queued behind another still holds the id', async () => { + await withFramework(async ({ framework, dir }) => { + const hold = (framework as unknown as { + holdMcpServerId: (id: string, kind: 'connect' | 'disconnect') => { previous: Promise | undefined; release: () => void }; + }).holdMcpServerId.bind(framework); + const first = hold('x', 'disconnect'); + const second = hold('x', 'disconnect'); + assert.equal(first.previous, undefined); + let secondMayRun = false; + void second.previous!.then(() => { secondMayRun = true; }); + first.release(); + await new Promise((r) => setImmediate(r)); + assert.equal(secondMayRun, true, 'the second runs once the first has settled'); + const config = modern('x', '', join(dir, 'raw.jsonl')); + await assert.rejects(framework.connectMcplServer(config), /"x" is still being disconnected/, 'the first release left the second claim'); + second.release(); + await framework.connectMcplServer(config); + assert.equal(status(framework, 'x')?.connected, true); + }); +}); + +test('a modern connect is refused while a legacy connect for the same id is still handshaking', async () => { + await withFramework(async ({ framework, dir }) => { + const legacyLog = join(dir, 'legacy.log'); + const modernLog = join(dir, 'raw.jsonl'); + const pending = framework.connectMcplServer(legacy('same', 'slow', legacyLog, { SLOW_INIT_MS: '800' })); + await until(() => launches(legacyLog).length === 1, 'the legacy launch, handshaking'); + await assert.rejects(framework.connectMcplServer(modern('same', '', modernLog)), /MCPL server "same" has a connect in flight; try again once it settles/); + assert.equal(launches(modernLog).length, 0, 'the modern server was never launched'); + await pending; + const served = status(framework, 'same')!; + assert.equal(served.family, 'legacy'); + assert.equal(served.connected, true); + assert.deepEqual((await framework.executeToolCall(call('mcpl--same--a1'))).data, [{ type: 'text', text: 'ok' }], 'its tools reach the legacy server'); + }); +}); + +test('a disconnect during a legacy handshake ends that connect at once, and removes it whole', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'legacy.log'); + // The handshake never finishes on its own, so only the disconnect can + // end the connect. + const pending = framework.connectMcplServer(legacy('same', 'slow', log, { SLOW_INIT_MS: '3600000' })); + await until(() => launches(log).length === 1, 'the legacy launch, handshaking'); + const disconnecting = framework.disconnectMcplServer('same'); + await assert.rejects(framework.connectMcplServer(legacy('same', 'ok', log)), /"same" is still being disconnected/); + await assert.rejects(pending, /MCP server "same" was not connected: it was disconnected while connecting/); + await disconnecting; + assert.equal(status(framework, 'same'), undefined); + assert.deepEqual(toolsOf(framework, 'same'), []); + assert.equal(alive(launches(log)[0]!), false, 'its child was reaped'); + assert.equal(launches(log).length, 1, 'and nothing else launched'); + }); +}); + +test('a disconnect during a modern connect ends it at once', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + const pending = framework.connectMcplServer(modern('m', 'hang-discover', log)); + await until(() => lines(log).some((l) => l.includes('server/discover')), 'the launch waiting on discover'); + const began = Date.now(); + await framework.disconnectMcplServer('m'); + await assert.rejects(pending, /MCP server "m" was not connected: it was disconnected while connecting/); + // A generous bound, well below the 30 s connect timeout it rules out. + assert.ok(Date.now() - began < 20_000, 'not after the 30 s connect timeout'); + assert.equal(status(framework, 'm'), undefined); + assert.equal(alive(launches(log)[0]!), false, 'its child was reaped'); + }); +}); + +test('modern: a disconnect that cannot reap keeps the server registered; a connect is refused until a retried disconnect confirms the exit', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + const config = modern('m', '', log); + const reaping = unreapable(); + await framework.connectMcplServer(config); + assert.ok(toolsOf(framework, 'm').length > 0); + await assert.rejects( + framework.disconnectMcplServer('m'), + /could not be reaped\. MCP server "m" stays registered, closed, until its exit is confirmed: disconnect it again to retry the reap/, + ); + assert.equal(status(framework, 'm')?.connected, false, 'still listed, closed'); + assert.equal(toolsOf(framework, 'm').length, 0, 'closed either way, so its tools left the list'); + await assert.rejects(framework.connectMcplServer(config), /"m" is already registered \(not connected and not reconnecting: disconnect it first\)/); + assert.equal(launches(log).length, 1, 'no launch beside the unreaped child'); + + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + await framework.disconnectMcplServer('m'); + assert.equal(status(framework, 'm'), undefined, 'the retried disconnect completed'); + await framework.connectMcplServer(config); + assert.equal(status(framework, 'm')?.connected, true, 'the id is free'); + }); +}); + +test('legacy: a disconnect that cannot reap keeps the server registered; a connect is refused until a retried disconnect confirms the exit', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'legacy.log'); + const config = legacy('l', 'ok', log); + const reaping = unreapable(); + await framework.connectMcplServer(config); + assert.ok(toolsOf(framework, 'l').length > 0); + await assert.rejects( + framework.disconnectMcplServer('l'), + /could not be reaped\. MCP server "l" stays registered, closed, until its exit is confirmed: disconnect it again to retry the reap/, + ); + assert.equal(status(framework, 'l')?.connected, false, 'still listed, closed'); + assert.equal(toolsOf(framework, 'l').length, 0, 'closed either way, so its tools left the list'); + await assert.rejects(framework.connectMcplServer(config), /"l" is already registered \(not connected and not reconnecting: disconnect it first\)/); + assert.equal(launches(log).length, 1, 'no launch beside the unreaped child'); + + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + await framework.disconnectMcplServer('l'); + assert.equal(status(framework, 'l'), undefined, 'the retried disconnect completed'); + await framework.connectMcplServer(config); + assert.equal(status(framework, 'l')?.connected, true, 'the id is free'); + }); +}); + +test('modern: a failed connect whose cleanup cannot reap stays registered until a disconnect confirms the exit', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + const reaping = unreapable(); + // The SDK reports a refused server/discover as a failed version negotiation. + await assert.rejects( + framework.connectMcplServer(modern('m', 'reject-discover', log)), + /server\/discover .*; its cleanup also failed: .*could not be reaped\. MCP server "m" stays registered, closed, until its exit is confirmed: disconnect it to retry the reap/, + ); + await assert.rejects(framework.connectMcplServer(modern('m', '', log)), /"m" is already registered \(not connected and not reconnecting/); + assert.equal(launches(log).length, 1); + + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + await framework.disconnectMcplServer('m'); + await framework.connectMcplServer(modern('m', '', log)); + assert.equal(status(framework, 'm')?.connected, true); + }); +}); + +test('modern, with reconnect: a failed first connect whose launch cannot be reaped fails, and stays registered until a disconnect confirms the exit', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + const reaping = unreapable(); + // No retry can come beside a launch that may still run, so the connect + // fails rather than reporting a server that will never serve. + await assert.rejects( + framework.connectMcplServer({ ...modern('m', 'reject-discover', log), reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }), + /server\/discover .*; its cleanup also failed: .*could not be reaped\. MCP server "m" stays registered, closed, until its exit is confirmed: disconnect it to retry the reap/, + ); + await assert.rejects(framework.connectMcplServer(modern('m', '', log)), /"m" is already registered \(not connected and not reconnecting/); + await new Promise((resolve) => setTimeout(resolve, 200)); + assert.equal(launches(log).length, 1, 'nothing launched beside it'); + + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + await framework.disconnectMcplServer('m'); + await framework.connectMcplServer(modern('m', '', log)); + assert.equal(status(framework, 'm')?.connected, true); + }); +}); + +test("at startup, a server with reconnect whose first launch can't be reaped is traced as not retrying, and alerts, on both engines", async () => { + const dir = mkdtempSync(join(tmpdir(), 'mcp-ownership-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const log = join(dir, 'raw.jsonl'); + const reaping = unreapable(); + const traces: Array<{ type: string; serverId?: string; willRetry?: boolean; kind?: string; agentName?: string }> = []; + const watcher = { + name: 'trace-watcher', + async start(ctx: { onTrace(listener: (event: unknown) => void): void }) { ctx.onTrace((event) => traces.push(event as never)); }, + async stop() {}, + getTools: () => [], + handleToolCall: async () => ({ success: false, error: 'no tools', isError: true }), + onProcess: async () => ({}), + }; + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [watcher as never], + mcplServers: [ + { ...modern('m', 'reject-discover', log), reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }, + { ...legacy('l', 'error', join(dir, 'legacy.log')), reconnect: true, reconnectIntervalMs: 30, reconnectMaxIntervalMs: 60 }, + ], + }); + cleanups.push(async () => { + for (const stuck of reaping.stuck) await killAndWait(stuck); + await framework.stop().catch(() => {}); + }); + for (const id of ['m', 'l']) { + const failed = traces.filter((t) => t.type === 'mcpl:server-connect-failed' && t.serverId === id); + // No retry can come beside a launch that may still run, and only a + // disconnect retries its reap: the operator is told. + assert.equal(failed.length, 1, id); + assert.equal(failed[0]!.willRetry, false, id); + assert.ok(traces.some((t) => t.type === 'ops:alert' && t.kind === 'mcpl-down' && t.agentName === id), id); + } +}); + +/** The stdio line under a modern server's live session, reached from inside, + * to stand for a loss read from a failed pipe while the child still runs. */ +function liveLineOf(framework: AgentFramework, id: string): { markClosed: (info: { reason: string }) => void; child: { pid: number; kill: (signal?: string) => boolean } } { + const connection = (framework as unknown as { modernServers(): Map }).modernServers().get(id); + assert.ok(connection?.session, 'a live session'); + return connection.session.wire.inner.line as never; +} + +test('modern, without reconnect: a server lost on its own holds its id until its child is reaped, then leaves', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + await framework.connectMcplServer(modern('m', 'ignore-sigterm', log)); + assert.ok(toolsOf(framework, 'm').length > 0); + const servers = (framework as unknown as { modernServers(): Map }).modernServers(); + const connection = servers.get('m'); + const line = liveLineOf(framework, 'm'); + line.markClosed({ reason: 'stdio pipe failed' }); + // The child ignores SIGTERM, so its reap takes about 2 s: meanwhile the + // id is held, and nothing launches beside the child. + assert.equal(servers.get('m'), connection, 'still registered while its reap runs'); + await assert.rejects(framework.connectMcplServer(modern('m', '', log)), /"m" is already registered/); + await until(() => !servers.has('m'), 'gone once its reap is confirmed', 10_000); + assert.equal(alive(line.child.pid), false, 'its child had exited'); + await until(() => toolsOf(framework, 'm').length === 0, 'its tools left the list'); + assert.equal(launches(log).length, 1, 'nothing launched beside it'); + }); +}); + +for (const reconnect of [false, true]) { +test(`modern, ${reconnect ? 'with' : 'without'} reconnect: a server lost on its own whose child can't be reaped stays registered, alerts, and its tools leave`, async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + const alerts: string[] = []; + framework.onTrace((event) => { + const traced = event as { type: string; kind?: string; agentName?: string }; + if (traced.type === 'ops:alert') alerts.push(`${traced.kind} ${traced.agentName}`); + }); + await framework.connectMcplServer({ ...modern('m', '', log), ...(reconnect ? RECONNECT : {}) }); + assert.ok(toolsOf(framework, 'm').length > 0); + const line = liveLineOf(framework, 'm'); + const realKill = line.child.kill.bind(line.child); + line.child.kill = () => true; // signals never arrive: the child outlives both bounds + cleanups.push(() => { realKill('SIGKILL'); }); + line.markClosed({ reason: 'stdio pipe failed' }); + await until(() => alerts.includes('mcpl-down m'), 'the halt raised as an alert', 10_000); + await until(() => toolsOf(framework, 'm').length === 0, 'its tools left the list'); + assert.equal(status(framework, 'm')?.connected, false, 'still registered, not connected'); + await assert.rejects(framework.connectMcplServer(modern('m', '', log)), /"m" is already registered \(not connected and not reconnecting/); + assert.equal(launches(log).length, 1, 'nothing launched beside it'); + + realKill('SIGKILL'); + await until(() => !alive(line.child.pid), 'the held child killed'); + await framework.disconnectMcplServer('m'); + assert.equal(status(framework, 'm'), undefined); + }); +}); +} + +test('legacy, with reconnect: a reconnect that meets a protocol-version verdict halts for good, alerts, and its tools leave', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'legacy.log'); + const alerts: string[] = []; + framework.onTrace((event) => { + const traced = event as { type: string; kind?: string; agentName?: string }; + if (traced.type === 'ops:alert') alerts.push(`${traced.kind} ${traced.agentName}`); + }); + // The first launch serves; every later one refuses the revision. + await framework.connectMcplServer({ ...legacy('l', 'ok-then-reject', log), ...RECONNECT }); + assert.ok(toolsOf(framework, 'l').length > 0); + await legacyServer(framework, 'l').sendToolsCall('die', {}).catch(() => { /* the session ends */ }); + await until(() => alerts.includes('mcpl-down l'), 'the halt raised as an alert', 10_000); + await until(() => toolsOf(framework, 'l').length === 0, 'its tools left the list'); + assert.equal(launches(log).length, 2, 'no attempt after the verdict'); + }); +}); + +test("legacy, without reconnect: a recycle whose child can't be reaped keeps the server registered, holding its id, until a disconnect confirms the exit, and its tools leave", async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'legacy.log'); + const config = legacy('l', 'ok', log); + const reaping = unreapable(); + await framework.connectMcplServer(config); + assert.ok(toolsOf(framework, 'l').length > 0); + // The awareness accounting fails, and the gate recycles every server. + const gate = framework as unknown as { + installMcplDataPlaneGate(): unknown; + failMcplDataPlaneGate(barrier: unknown, context: string, error: unknown): Promise; + }; + await gate.failMcplDataPlaneGate(gate.installMcplDataPlaneGate(), 'a test', new Error('accounting failed on purpose')); + assert.equal(status(framework, 'l')?.connected, false, 'still listed, closed'); + await until(() => toolsOf(framework, 'l').length === 0, 'its tools left the list'); + await assert.rejects(framework.connectMcplServer(config), /"l" is already registered \(not connected and not reconnecting/); + assert.equal(launches(log).length, 1, 'no launch beside the unreaped child'); + + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + await framework.disconnectMcplServer('l'); + assert.equal(status(framework, 'l'), undefined, 'the disconnect completed'); + }); +}); + +test("legacy, with reconnect: a recycle whose close can't reap leaves the server's tools listed for its reconnect loop to settle", async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'legacy.log'); + // Its reconnect is a minute away, so what the recycle itself did is seen + // before the reconnect loop settles anything. + const config = { ...legacy('l', 'ok', log), reconnect: true, reconnectIntervalMs: 60_000, reconnectMaxIntervalMs: 60_000 }; + const reaping = unreapable(); + await framework.connectMcplServer(config); + assert.ok(toolsOf(framework, 'l').length > 0); + const internals = framework as unknown as { handleToolsListChanged(id: string): void }; + const refresh = internals.handleToolsListChanged.bind(framework); + const refreshes: string[] = []; + internals.handleToolsListChanged = (id: string) => { refreshes.push(id); refresh(id); }; + // The awareness accounting fails, and the gate recycles every server. + const gate = framework as unknown as { + installMcplDataPlaneGate(): unknown; + failMcplDataPlaneGate(barrier: unknown, context: string, error: unknown): Promise; + }; + await gate.failMcplDataPlaneGate(gate.installMcplDataPlaneGate(), 'a test', new Error('accounting failed on purpose')); + assert.equal(legacyServer(framework, 'l').willReconnect, true, 'its reconnect is still to come'); + assert.deepEqual(refreshes, [], 'the failed recycle refreshed nothing'); + assert.ok(toolsOf(framework, 'l').length > 0, 'its tools are still listed'); + + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + await framework.disconnectMcplServer('l'); + assert.equal(status(framework, 'l'), undefined, 'the disconnect completed'); + }); +}); + +for (const engine of ['legacy', 'modern'] as const) { + test(`${engine}: a connect that stop() ends, whose cleanup can't reap its child, says the framework stopped and offers no disconnect`, async () => { + const dir = mkdtempSync(join(tmpdir(), 'mcp-ownership-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [], + }); + const reaping = unreapable(); + const log = join(dir, engine === 'legacy' ? 'legacy.log' : 'raw.jsonl'); + // Neither connect finishes on its own: the legacy handshake waits an + // hour, and the modern server never answers discover. + const pending = framework.connectMcplServer(engine === 'legacy' + ? legacy('s', 'slow', log, { SLOW_INIT_MS: '3600000' }) + : modern('s', 'hang-discover', log)).then(() => null, (error: unknown) => error as Error); + await until( + () => (engine === 'legacy' ? launches(log).length === 1 : lines(log).some((l) => l.includes('server/discover'))), + 'the connect in flight', + ); + await assert.rejects(framework.stop(), /could not be reaped/); + const refused = await pending; + assert.match( + refused?.message ?? 'connected', + /^MCP server "s" was not connected: the framework stopped while it was connecting; its cleanup also failed: .*could not be reaped\. MCP server "s" stays registered, closed, until its exit is confirmed$/, + ); + for (const stuck of reaping.stuck) await killAndWait(stuck); + reaping.active = false; + await framework.stop(); + }); +} + +test("modern: ModernMcpConnection.connect hands back the connection that owns a launch its cleanup couldn't reap", async () => { + const dir = mkdtempSync(join(tmpdir(), 'mcp-ownership-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const log = join(dir, 'raw.jsonl'); + const reaping = unreapable(); + const failed = await ModernMcpConnection.connect(modern('m', 'reject-discover', log)).then( + () => null, + (error: unknown) => error, + ); + assert.ok(failed instanceof McplUnreapedLaunchError, String(failed)); + assert.ok(failed.connection instanceof ModernMcpConnection, 'the owner of the launch'); + assert.match(failed.message, /its cleanup also failed: .*could not be reaped/); + assert.equal(launches(log).length, 1); + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + // Its close() retries the reap, and resolves once the exit is confirmed. + await failed.connection.close(); +}); + +for (const reconnect of [false, true]) { +test(`legacy${reconnect ? ', with reconnect' : ''}: a failed connect whose cleanup cannot reap stays registered until a disconnect confirms the exit`, async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'legacy.log'); + const reaping = unreapable(); + await assert.rejects( + framework.connectMcplServer({ ...legacy('l', 'error', log), ...(reconnect ? RECONNECT : {}) }), + /initialize failed on purpose; its cleanup also failed: .*could not be reaped\. MCP server "l" stays registered, closed, until its exit is confirmed: disconnect it to retry the reap/, + ); + await assert.rejects(framework.connectMcplServer(legacy('l', 'ok', log)), /"l" is already registered \(not connected and not reconnecting/); + assert.equal(launches(log).length, 1); + + await killAndWait(reaping.stuck[0]!); + reaping.active = false; + await framework.disconnectMcplServer('l'); + await framework.connectMcplServer(legacy('l', 'ok', log)); + assert.equal(status(framework, 'l')?.connected, true); + }); +}); +} + +test('stop() closes every connection, keeps the ones it cannot reap registered, and rejects with each failure; once they have exited, a retry completes', async () => { + const dir = mkdtempSync(join(tmpdir(), 'mcp-ownership-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [], + }); + const reaping = unreapable(); + await framework.connectMcplServer(modern('m', '', join(dir, 'raw.jsonl'))); + await framework.connectMcplServer(legacy('l', 'ok', join(dir, 'legacy.log'))); + await assert.rejects(framework.stop(), (error) => { + assert.ok(error instanceof AggregateError, String(error)); + assert.equal(error.errors.length, 2, 'one failure from each engine'); + assert.match(error.message, /stdio child \d+ of MCP server "m" did not exit within \d+ms of SIGKILL; it could not be reaped/); + assert.match(error.message, /stdio child \d+ of MCP server "l" did not exit within \d+ms of SIGKILL; it could not be reaped/); + return true; + }); + const internals = framework as unknown as { + modernServers(): Map; + mcplServerRegistry: { getServer(id: string): unknown }; + }; + assert.ok(internals.modernServers().has('m'), 'the modern one it could not reap keeps its id'); + assert.ok(internals.mcplServerRegistry.getServer('l'), 'and so does the legacy one'); + for (const stuck of reaping.stuck) await killAndWait(stuck); + // The failed stop left its store open, so it can be retried: the retry + // confirms both exits and frees the ids. + await framework.stop(); + assert.equal(internals.modernServers().has('m'), false); + assert.equal(internals.mcplServerRegistry.getServer('l'), null); +}); + +test('a teardown that cannot confirm the exit traces no close; the retry that confirms it traces one, on both engines', async () => { + await withFramework(async ({ framework, dir }) => { + const closed: string[] = []; + framework.onTrace((event) => { if (event.type === 'mcpl:server-closed') closed.push(event.serverId); }); + const reaping = unreapable(); + await framework.connectMcplServer(modern('m', '', join(dir, 'raw.jsonl'))); + await framework.connectMcplServer(legacy('l', 'ok', join(dir, 'legacy.log'))); + await Promise.all(['m', 'l'].map((id) => assert.rejects( + framework.disconnectMcplServer(id), + new RegExp(`stdio child \\d+ of MCP server "${id}" did not exit .*stays registered, closed`), + ))); + assert.deepEqual(closed, [], 'neither is traced closed while its child may still be running'); + for (const stuck of reaping.stuck) await killAndWait(stuck); + await framework.disconnectMcplServer('m'); + await framework.disconnectMcplServer('l'); + assert.deepEqual(closed.sort(), ['l', 'm'], 'each is traced closed once, by the retry that confirms its exit'); + }); +}); + +test("stop()'s teardown budget bounds a modern server's close too, and the server stays registered while its close is pending", async () => { + const dir = mkdtempSync(join(tmpdir(), 'mcp-ownership-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [], + }); + let release!: () => void; + const released = new Promise((resolve) => { release = resolve; }); + const pending = { close: async () => { await released; } }; + const modernServers = (framework as unknown as { modernServers(): Map }).modernServers(); + modernServers.set('m', pending); + await assert.rejects(framework.stop(50), (error) => + error instanceof ShutdownTimeoutError && error.pending.includes('mcp:m')); + assert.equal(modernServers.get('m'), pending, 'still registered while its close is pending'); + release(); + await framework.stop(); + assert.equal(modernServers.has('m'), false, 'removed once its close completed'); +}); + +test("a modern server's tool prefix meets another server's under #469's rule, at a runtime connect and a restart, whichever family is first", async () => { + await withFramework(async ({ framework, dir }) => { + const modernLog = join(dir, 'raw.jsonl'); + // A modern server first, under `foo`. + await framework.connectMcplServer({ ...modern('m', '', modernLog), toolPrefix: 'foo' }); + await assert.rejects(framework.connectMcplServer({ ...legacy('l', 'ok', join(dir, 'legacy.log')), toolPrefix: 'foo--bar' }), + /toolPrefix "foo--bar" meets server "m"'s toolPrefix "foo"/, 'a legacy server whose prefix meets it'); + await assert.rejects(framework.connectMcplServer({ ...modern('m2', '', join(dir, 'raw2.jsonl')), toolPrefix: 'foo-' }), + /toolPrefix "foo-" meets server "m"'s toolPrefix "foo"/, 'a modern server whose prefix meets it'); + // A legacy server first, under `bar`. + await framework.connectMcplServer({ ...legacy('l2', 'ok', join(dir, 'legacy2.log')), toolPrefix: 'bar' }); + await assert.rejects(framework.connectMcplServer({ ...modern('m3', '', join(dir, 'raw3.jsonl')), toolPrefix: 'bar--baz' }), + /toolPrefix "bar--baz" meets server "l2"'s toolPrefix "bar"/, 'a modern server whose prefix meets a legacy one'); + // A restart that would move the modern server to a prefix meeting the legacy one's is refused before its teardown. + await assert.rejects(framework.restartMcplServer('m', { ...modern('m', '', modernLog), toolPrefix: 'bar-' }), + /toolPrefix "bar-" meets server "l2"'s toolPrefix "bar"/, 'a restart into a prefix that meets another'); + assert.equal(status(framework, 'm')?.connected, true, 'the refused restart left the server up'); + for (const id of ['l', 'm2', 'm3']) assert.equal(status(framework, id), undefined, `${id} was never connected`); + }); +}); + +test('a restart whose new configuration is refused leaves the running server as it was', async () => { + await withFramework(async ({ framework, dir }) => { + const log = join(dir, 'raw.jsonl'); + await framework.connectMcplServer(modern('m', '', log)); + await assert.rejects( + framework.restartMcplServer('m', { id: 'm', url: 'ftp://127.0.0.1/x' }), + /MCP server "m" configuration: .*url must be/, + ); + assert.equal(status(framework, 'm')?.connected, true, 'still connected'); + assert.equal(launches(log).length, 1, 'never torn down or relaunched'); + assert.ok(toolsOf(framework, 'm').length > 0, 'its tools still listed'); + }); +}); + +test("a refresh in flight when a server is disconnected doesn't bring its tools back", async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('m', '', join(dir, 'raw.jsonl'))); + assert.ok(toolsOf(framework, 'm').length > 0); + const internals = framework as unknown as { + modernServers(): Map }>; + refreshMcplTools(): Promise; + }; + const connection = internals.modernServers().get('m')!; + const realList = connection.listTools.bind(connection); + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let listed = false; + connection.listTools = async () => { const tools = await realList(); listed = true; await gate; return tools; }; + const refreshing = internals.refreshMcplTools(); + await until(() => listed, 'the refresh holds the server\'s list'); + await framework.disconnectMcplServer('m'); + release(); + await refreshing; + assert.deepEqual(toolsOf(framework, 'm'), [], 'the earlier refresh, finishing last, published none of its tools'); + }); +}); + +test("a server connected while a refresh waits keeps its own tool prefix, and the held refresh publishes nothing", async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('m', '', join(dir, 'raw.jsonl'))); + const internals = framework as unknown as { + modernServers(): Map }>; + refreshMcplTools(): Promise; + }; + const connection = internals.modernServers().get('m')!; + const realList = connection.listTools.bind(connection); + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let listed = false; + connection.listTools = async () => { const tools = await realList(); listed = true; await gate; return tools; }; + const refreshing = internals.refreshMcplTools(); + await until(() => listed, "the refresh holds the first server's list"); + let lists = 0; + connection.listTools = async () => { lists += 1; return realList(); }; + // Connected while the first refresh waits, under a prefix of its own. + await framework.connectMcplServer({ ...modern('n', '', join(dir, 'raw-n.jsonl')), toolPrefix: 'custom' }); + // The held refresh finishes last, and publishes nothing: the connect's + // own refresh, started after it, listed both servers. So it starts no + // other refresh either, since that one has a listing of every server it + // listed (a refresh begins listing at once). + const listsBefore = lists; + release(); + await refreshing; + assert.equal(lists, listsBefore, 'the dropped refresh started no other'); + const names = framework.getAllTools().map((t) => t.name); + assert.ok(names.some((n) => n.startsWith('custom--')), names.join(', ')); + assert.ok(!names.some((n) => n.startsWith('mcpl--n--')), names.join(', ')); + }); +}); + +test("a refresh in flight when a server restarts under the same id publishes none of the old connection's listing", async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('m', '', join(dir, 'raw.jsonl'))); + assert.ok(toolsOf(framework, 'm').length > 0); + const internals = framework as unknown as { + modernServers(): Map }>; + refreshMcplTools(): Promise; + }; + const connection = internals.modernServers().get('m')!; + const realList = connection.listTools.bind(connection); + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let listed = false; + connection.listTools = async () => { const tools = await realList(); listed = true; await gate; return tools; }; + const refreshing = internals.refreshMcplTools(); + await until(() => listed, "the refresh holds the old connection's list"); + // Restarted under the same id, its tools now under another prefix. + await framework.restartMcplServer('m', { ...modern('m', '', join(dir, 'raw-2.jsonl')), toolPrefix: 'second' }); + assert.notEqual(internals.modernServers().get('m'), connection, 'a new connection'); + release(); + await refreshing; + const names = framework.getAllTools().map((t) => t.name); + assert.deepEqual(toolsOf(framework, 'm'), [], `the old connection's listing isn't published: ${names.join(', ')}`); + assert.ok(names.some((n) => n.startsWith('second--')), names.join(', ')); + }); +}); + +test('a server whose listing a later refresh failed is listed again when the earlier listing is dropped', async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('m', '', join(dir, 'raw.jsonl'))); + assert.ok(toolsOf(framework, 'm').length > 0); + const internals = framework as unknown as { + modernServers(): Map }>; + refreshMcplTools(): Promise; + }; + const connection = internals.modernServers().get('m')!; + const realList = connection.listTools.bind(connection); + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let mode: 'hold' | 'refuse' | 'answer' = 'hold'; + let held = false; + connection.listTools = async () => { + if (mode === 'refuse') throw new Error('refused while another listing is open'); + const tools = await realList(); + if (mode === 'hold') { held = true; await gate; } + return tools; + }; + // The first refresh lists m and holds the answer. + const first = internals.refreshMcplTools(); + await until(() => held, 'the first refresh holds its listing of m'); + // A later refresh, whose request m refuses, publishes first, without m. + mode = 'refuse'; + await internals.refreshMcplTools(); + assert.deepEqual(toolsOf(framework, 'm'), [], "the later refresh published no listing of m"); + // The first refresh's listing is older, so it's dropped, and one more + // refresh lists m again. + mode = 'answer'; + release(); + await first; + await until(() => toolsOf(framework, 'm').length > 0, "one more refresh brings m's tools back"); + }); +}); + +for (const engine of ['legacy', 'modern'] as const) { + test(`${engine}: a listing begun before its server's session closed doesn't bring back the class hints the close dropped`, async () => { + await withFramework(async ({ framework, dir }) => { + const base = engine === 'legacy' ? legacy('s', 'ok', join(dir, 'legacy.log')) : modern('s', '', join(dir, 'raw.jsonl')); + // It reconnects, but not before the test is over. + await framework.connectMcplServer({ ...base, reconnect: true, reconnectIntervalMs: 60_000, reconnectMaxIntervalMs: 60_000 }); + type Tool = { name: string; _meta?: Record }; + type Server = { on(event: 'close', listener: () => void): void; sendToolsList(): Promise<{ tools: Tool[] }>; listTools(): Promise }; + const internals = framework as unknown as { + mcplServerRegistry: { getServer(id: string): Server }; + modernServers(): Map; + refreshMcplTools(): Promise; + }; + const server = engine === 'legacy' ? internals.mcplServerRegistry.getServer('s') : internals.modernServers().get('s')!; + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let held = false; + // The session about to close declares its tools `files`. + const classed = (tools: Tool[]): Tool[] => tools.map((tool) => ({ ...tool, _meta: { ...tool._meta, 'mcpl/class': ['files'] } })); + if (engine === 'legacy') { + const real = server.sendToolsList.bind(server); + server.sendToolsList = async () => { const result = await real(); held = true; await gate; return { ...result, tools: classed(result.tools) }; }; + } else { + const real = server.listTools.bind(server); + server.listTools = async () => { const tools = classed(await real()); held = true; await gate; return tools; }; + } + const refreshing = internals.refreshMcplTools(); + await until(() => held, 'the refresh holds its listing'); + let closed = false; + server.on('close', () => { closed = true; }); + await framework.executeToolCall(call('mcpl--s--die')).catch(() => {}); + await until(() => closed, 'the session closes, and the server waits to reconnect'); + release(); + await refreshing; + const kept = framework.listToolClasses().filter((tool) => tool.serverId === 's' && tool.class.includes('files')); + assert.deepEqual(kept, [], "no tool keeps the closed session's class"); + assert.deepEqual(framework.getAllTools().filter((tool) => tool.name.startsWith('mcpl--s--')), [], "nor are the closed session's tools published"); + }); + }); +} + +/** Hold modern server `id`'s listings once they've answered, until + * release(): every one, or only the first `first`. */ +function holdListings(framework: AgentFramework, id: string, first = Infinity): { held: () => number; release: () => void } { + const connection = (framework as unknown as { modernServers(): Map }> }).modernServers().get(id)!; + const realList = connection.listTools.bind(connection); + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let held = 0; + connection.listTools = async () => { + const tools = await realList(); + if (held >= first) return tools; + held += 1; + await gate; + return tools; + }; + return { held: () => held, release }; +} + +test("a disconnect during a connect's tool listing ends the connect", async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('a', '', join(dir, 'raw-a.jsonl'))); + const a = holdListings(framework, 'a'); + const servers = (framework as unknown as { modernServers(): Map }).modernServers(); + const connecting = framework.connectMcplServer(modern('m', '', join(dir, 'raw-m.jsonl'))); + await until(() => a.held() > 0, "m's connect, listing tools, waits on a"); + // The disconnect removes m whole while that listing waits. Its own + // refresh then waits on a too, so it's awaited once a answers. + const disconnecting = framework.disconnectMcplServer('m'); + await until(() => !servers.has('m'), 'm is removed'); + a.release(); + await assert.rejects(connecting, /MCP server "m" was not connected: it was disconnected or lost while its tools were being listed/); + await disconnecting; + }); +}); + +test("a disconnect during a restart's tool listing ends the restart, which says the server is down", async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('a', '', join(dir, 'raw-a.jsonl'))); + await framework.connectMcplServer(modern('m', '', join(dir, 'raw-m.jsonl'))); + const servers = (framework as unknown as { modernServers(): Map }).modernServers(); + const before = servers.get('m'); + // Hold a's listings made once the restart has registered the new m: its + // connect's. The teardown's refresh, and the disconnect's below, come + // while no m is registered, and pass. + const a = servers.get('a') as { listTools(): Promise }; + const realList = a.listTools.bind(a); + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let held = 0; + a.listTools = async () => { + const tools = await realList(); + if (!servers.has('m') || servers.get('m') === before) return tools; + held += 1; + await gate; + return tools; + }; + const restarting = framework.restartMcplServer('m'); + // Handled here too, so an assertion that fails first leaves no + // unhandled rejection behind. + void restarting.catch(() => {}); + let disconnecting: Promise | undefined; + try { + await until(() => held > 0, "the restart's connect, listing tools, waits on a"); + disconnecting = framework.disconnectMcplServer('m'); + await until(() => !servers.has('m'), 'the new m is removed'); + } finally { + release(); + } + await assert.rejects(restarting, /^Error: MCP server "m" is down: it was disconnected for its restart, and the restart's connect failed: MCP server "m" was not connected: it was disconnected or lost while its tools were being listed$/); + await disconnecting; + }); +}); + +test("a disconnect still under way when a connect's tool listing ends fails the connect", async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('a', '', join(dir, 'raw-a.jsonl'))); + const a = holdListings(framework, 'a'); + const servers = (framework as unknown as { modernServers(): Map; isConnected: boolean }> }).modernServers(); + const connecting = framework.connectMcplServer(modern('m', '', join(dir, 'raw-m.jsonl'))); + await until(() => a.held() > 0, "m's connect, listing tools, waits on a"); + // m's close waits for the test, so its disconnect holds the id while m + // is still registered and still connected. + const m = servers.get('m')!; + const realClose = m.close.bind(m); + let openClose!: () => void; + const closeGate = new Promise((resolve) => { openClose = resolve; }); + m.close = async () => { await closeGate; return realClose(); }; + let disconnected = false; + const disconnecting = framework.disconnectMcplServer('m').then(() => { disconnected = true; }); + try { + a.release(); + await assert.rejects(connecting, /MCP server "m" was not connected: it was disconnected or lost while its tools were being listed/); + assert.equal(disconnected, false, 'the disconnect was still under way'); + assert.equal(servers.get('m'), m, 'with m still registered'); + assert.equal(m.isConnected, true, 'and still connected'); + } finally { + openClose(); + await disconnecting; + } + }); +}); + +test("a disconnect that fails to reap during a connect's tool listing fails the connect", async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(modern('a', '', join(dir, 'raw-a.jsonl'))); + // Only the connect's listing: the failed disconnect's own refresh lists + // a too. + const a = holdListings(framework, 'a', 1); + const reaping = unreapable(); + const connecting = framework.connectMcplServer(modern('m', '', join(dir, 'raw-m.jsonl'))); + await until(() => a.held() > 0, "m's connect, listing tools, waits on a"); + reaping.active = false; + try { + // The disconnect closes m and can't reap its child: m stays + // registered, closed, and the disconnect has let go of the id. + await assert.rejects(framework.disconnectMcplServer('m'), /could not be reaped/); + assert.equal(status(framework, 'm')?.connected, false, 'still listed, closed'); + a.release(); + await assert.rejects(connecting, /MCP server "m" was not connected: it was disconnected or lost while its tools were being listed/); + } finally { + a.release(); + for (const stuck of reaping.stuck) await killAndWait(stuck); + } + await framework.disconnectMcplServer('m'); + }); +}); + +test("a listing of a server whose disconnect failed to reap isn't published", async () => { + await withFramework(async ({ framework, dir }) => { + const reaping = unreapable(); + await framework.connectMcplServer(modern('m', '', join(dir, 'raw-m.jsonl'))); + reaping.active = false; + assert.ok(toolsOf(framework, 'm').length > 0); + type Tool = { name: string; _meta?: Record }; + const internals = framework as unknown as { + modernServers(): Map }>; + refreshMcplTools(): Promise; + }; + const connection = internals.modernServers().get('m')!; + const realList = connection.listTools.bind(connection); + let release!: () => void; + const gate = new Promise((resolve) => { release = resolve; }); + let held = false; + connection.listTools = async () => { + const tools = (await realList()).map((tool) => ({ ...tool, _meta: { ...tool._meta, 'mcpl/class': ['files'] } })); + held = true; + await gate; + return tools; + }; + const refreshing = internals.refreshMcplTools(); + await until(() => held, 'the refresh holds its listing of m'); + let disconnecting: Promise | undefined; + try { + // The disconnect closes m, and its reap then fails, which takes + // seconds; the refresh publishes meanwhile. A teardown that hasn't + // confirmed the exit has emitted no 'close', so only m's state says + // it's over. + disconnecting = framework.disconnectMcplServer('m'); + await until(() => (connection as unknown as { isClosed: boolean }).isClosed, 'm closed by the disconnect'); + release(); + await refreshing; + assert.deepEqual(toolsOf(framework, 'm'), [], "the closed connection's listing isn't published"); + assert.deepEqual(framework.listToolClasses().filter((tool) => tool.serverId === 'm' && tool.class.includes('files')), []); + await assert.rejects(disconnecting, /could not be reaped/); + } finally { + release(); + await disconnecting?.catch(() => { /* its verdict is asserted above */ }); + for (const stuck of reaping.stuck) await killAndWait(stuck); + } + await framework.disconnectMcplServer('m'); + }); +}); + +test('a listing begun before its server closed and reconnected is left out, and the new session is listed', async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer({ ...modern('m', '', join(dir, 'raw.jsonl')), reconnect: true, reconnectIntervalMs: 50, reconnectMaxIntervalMs: 100 }); + type Tool = { name: string; _meta?: Record }; + const internals = framework as unknown as { + modernServers(): Map }>; + refreshMcplTools(): Promise; + }; + const connection = internals.modernServers().get('m')!; + const realList = connection.listTools.bind(connection); + // The first listing is the old session's, which declares `files`. The + // second is the reconnect's own refresh, of the new session. + const gates = [0, 1].map(() => { let open!: () => void; const shut = new Promise((resolve) => { open = resolve; }); return { open, shut }; }); + let calls = 0; + const answered: boolean[] = []; + connection.listTools = async () => { + const call = calls++; + const tools = await realList(); + answered[call] = true; + if (call > 1) return tools; + await gates[call]!.shut; + return call === 0 ? tools.map((tool) => ({ ...tool, _meta: { ...tool._meta, 'mcpl/class': ['files'] } })) : tools; + }; + const refreshing = internals.refreshMcplTools(); + try { + await until(() => answered[0] === true, "the refresh holds the old session's listing of m"); + await framework.executeToolCall(call('mcpl--m--die')).catch(() => {}); + // m's session closes and a new one starts: its reconnect lists it. + await until(() => calls > 1, "m's reconnect lists its new session"); + gates[0]!.open(); + await refreshing; + assert.deepEqual(toolsOf(framework, 'm'), [], "the old session's listing isn't published, though m is connected again"); + assert.deepEqual(framework.listToolClasses().filter((tool) => tool.serverId === 'm' && tool.class.includes('files')), []); + gates[1]!.open(); + await until(() => toolsOf(framework, 'm').length > 0, "the new session's listing is published"); + assert.deepEqual(framework.listToolClasses().filter((tool) => tool.serverId === 'm' && tool.class.includes('files')), []); + } finally { + for (const gate of gates) gate.open(); + } + }); +}); + +test('a server lost for good, legacy or modern, leaves the tool list', async () => { + await withFramework(async ({ framework, dir }) => { + await framework.connectMcplServer(legacy('l', 'ok', join(dir, 'legacy.log'))); + await framework.connectMcplServer(modern('m', '', join(dir, 'raw.jsonl'))); + assert.ok(toolsOf(framework, 'l').length > 0 && toolsOf(framework, 'm').length > 0); + await framework.executeToolCall(call('mcpl--l--die')).catch(() => {}); + await framework.executeToolCall(call('mcpl--m--die')).catch(() => {}); + await until(() => toolsOf(framework, 'l').length === 0, 'the legacy server\'s tools gone'); + await until(() => toolsOf(framework, 'm').length === 0, 'the modern server\'s tools gone'); + }); +}); + +test("a stdio server switched to the modern family is a new coalescing binding; a legacy configuration's binding is unchanged", async () => { + await withFramework(async ({ framework }) => { + const internals = framework as unknown as { + mcplServerConfigs: Map; + coalescingBinding(id: string): string; + }; + internals.mcplServerConfigs.set('s', { id: 's', command: 'srv', args: ['a'] }); + const legacyBinding = internals.coalescingBinding('s'); + const { createHash } = await import('node:crypto'); + assert.equal( + legacyBinding, + createHash('sha256').update(JSON.stringify(['s', null, 'srv', ['a']])).digest('hex').slice(0, 16), + 'the hash a stored legacy binding was made with', + ); + internals.mcplServerConfigs.set('s', { id: 's', command: 'srv', args: ['a'], protocol: 'modern' }); + assert.notEqual(internals.coalescingBinding('s'), legacyBinding); + internals.mcplServerConfigs.delete('s'); + }); +}); diff --git a/test/mcp-tool-result-normalize.test.ts b/test/mcp-tool-result-normalize.test.ts new file mode 100644 index 000000000..b864c5cce --- /dev/null +++ b/test/mcp-tool-result-normalize.test.ts @@ -0,0 +1,276 @@ +/** + * The standard reading of an MCP tool result: the content array is the + * model's view, and `structuredContent` the programmatic one, kept by + * presence. A payload the model can't take is saved and described by a + * bounded stub; nothing is fetched, and nothing is dropped silently. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; + +import { normalizeStandardToolResult, type SaveToolPayload } from '../src/mcpl/tool-result-normalize.js'; + +const b64 = (s: string) => Buffer.from(s).toString('base64'); + +function recordingSave(): { save: SaveToolPayload; saved: Array<{ fileName: string; bytes: string; mimeType: string }> } { + const saved: Array<{ fileName: string; bytes: string; mimeType: string }> = []; + return { + saved, + save: async (fileName, bytes, mimeType) => { + saved.push({ fileName, bytes: bytes.toString('utf8'), mimeType }); + return `home/tool-results/${fileName}`; + }, + }; +} + +test('text only: the joined string, as before, and no structured field', async () => { + const r = await normalizeStandardToolResult({ content: [{ type: 'text', text: 'a' }, { type: 'text', text: 'b' }] }, 'L', null); + assert.deepEqual(r, { success: true, data: 'a\nb', isError: false }); + assert.ok(!('structured' in r)); +}); + +test('structured only: the model sees it as JSON, programs get the value', async () => { + const r = await normalizeStandardToolResult({ content: [], structuredContent: { answer: 42, ok: false, nothing: null } }, 'L', null); + assert.equal(r.success, true); + assert.equal(r.data, '{"answer":42,"ok":false,"nothing":null}'); + assert.deepEqual(r.structured, { answer: 42, ok: false, nothing: null }); +}); + +test('falsy structured values are values: 0, false and null are kept and shown', async () => { + for (const value of [0, false, null, '']) { + const r = await normalizeStandardToolResult({ content: [], structuredContent: value }, 'L', null); + assert.ok('structured' in r, String(value)); + assert.equal(r.structured, value); + assert.equal(r.data, JSON.stringify(value)); + } +}); + +test('text plus structured: the server chose the model\'s text; structured rides beside it', async () => { + const r = await normalizeStandardToolResult({ + content: [{ type: 'text', text: 'Found 2 rows.' }], + structuredContent: { rows: [{ id: 1 }, { id: 2 }] }, + }, 'L', null); + assert.equal(r.data, 'Found 2 rows.'); + assert.deepEqual(r.structured, { rows: [{ id: 1 }, { id: 2 }] }); +}); + +test('an inline image stays native: the array is kept', async () => { + const r = await normalizeStandardToolResult({ + content: [{ type: 'text', text: 'see' }, { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }], + }, 'L', null); + assert.deepEqual(r.data, [{ type: 'text', text: 'see' }, { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }]); +}); + +test('audio and binary resources are saved to the workspace and described', async () => { + const { save, saved } = recordingSave(); + const r = await normalizeStandardToolResult({ + content: [ + { type: 'audio', data: b64('RIFFdata'), mimeType: 'audio/wav' }, + { type: 'resource', resource: { uri: 'memo://bin', mimeType: 'application/octet-stream', blob: b64('binary!') } }, + ], + }, '2026-10-08-call1', save); + assert.deepEqual(saved, [ + { fileName: '2026-10-08-call1-1.wav', bytes: 'RIFFdata', mimeType: 'audio/wav' }, + { fileName: '2026-10-08-call1-2.bin', bytes: 'binary!', mimeType: 'application/octet-stream' }, + ]); + assert.equal(r.data, + '[audio: audio/wav, 8 B, saved to workspace file home/tool-results/2026-10-08-call1-1.wav]\n' + + '[resource memo://bin: application/octet-stream, 7 B, saved to workspace file home/tool-results/2026-10-08-call1-2.bin]'); +}); + +test('a payload that cannot be retained makes the result a failure to materialize, not a success', async () => { + const r = await normalizeStandardToolResult({ + content: [{ type: 'text', text: 'here is the clip' }, { type: 'audio', data: b64('x'), mimeType: 'audio/mpeg' }], + }, 'L', null); + assert.equal(r.success, false); + assert.equal(r.isError, true); + assert.equal(r.error, + '[result incomplete: 1 payload(s) could not be retained, so this result is not complete; the tool may already have completed]\n' + + 'here is the clip\n' + + "[audio: audio/mpeg, 1 B. Not retained: the model can't take this type, and no workspace is mounted to save it]"); + const failedSave = await normalizeStandardToolResult( + { content: [{ type: 'resource', resource: { uri: 'memo://b', blob: b64('zz') } }] }, + 'L', + async () => null, + ); + assert.equal(failedSave.success, false); + assert.match(failedSave.error!, /saving it to the workspace failed/); +}); + +test('generated stubs are not the server\'s text: structured still shows beside them', async () => { + const r = await normalizeStandardToolResult({ + content: [{ type: 'resource_link', uri: 'file:///r.csv', name: 'r.csv' }], + structuredContent: { rows: 3 }, + }, 'L', null); + assert.equal(r.data, '[resource link "r.csv": file:///r.csv; not fetched]\n{"rows":3}'); +}); + +test('a resource link is shown as a reference and never fetched; embedded text is shown', async () => { + const r = await normalizeStandardToolResult({ + content: [ + { type: 'resource_link', uri: 'file:///srv/report.pdf', name: 'report.pdf', mimeType: 'application/pdf' }, + { type: 'resource', resource: { uri: 'memo://note', mimeType: 'text/plain', text: 'embedded note text' } }, + ], + }, 'L', null); + assert.equal(r.data, + '[resource link "report.pdf": file:///srv/report.pdf, application/pdf; not fetched]\n' + + '[resource memo://note, text/plain]\nembedded note text'); +}); + +test('a tool error is a failure with its text; structured survives', async () => { + const r = await normalizeStandardToolResult({ content: [{ type: 'text', text: 'nope' }], isError: true, structuredContent: { code: 7 } }, 'L', null); + assert.deepEqual(r, { success: false, error: 'nope', isError: true, structured: { code: 7 } }); +}); + +test('an unknown block is shown as data, and stub fields are bounded', async () => { + const r = await normalizeStandardToolResult({ + content: [{ type: 'hologram', depth: 3 }, { type: 'resource_link', uri: 'memo://r', name: 'n'.repeat(1000) }], + }, 'L', null); + const [unknown, link] = String(r.data).split('\n'); + assert.equal(unknown, '{"type":"hologram","depth":3}'); + assert.equal(link, `[resource link "${'n'.repeat(200)}…": memo://r; not fetched]`); +}); + +test('a large unknown block is bounded like a stub field, saying how much was left out', async () => { + const r = await normalizeStandardToolResult({ content: [{ type: 'hologram', blob: 'x'.repeat(10_000) }] }, 'L', null); + const json = JSON.stringify({ type: 'hologram', blob: 'x'.repeat(10_000) }); + assert.equal(r.data, `${json.slice(0, 200)}… [${json.length - 200} more characters of this block left out]`); +}); + +test('a URI is bounded loosely: a real one survives whole, a pathological one is cut', async () => { + const real = `https://example.com/report?${'q'.repeat(1500)}`; + const huge = `memo://${'x'.repeat(5000)}`; + const r = await normalizeStandardToolResult({ + content: [{ type: 'resource_link', uri: real }, { type: 'resource_link', uri: huge }], + }, 'L', null); + const [whole, cut] = String(r.data).split('\n'); + assert.equal(whole, `[resource link: ${real}; not fetched]`); + assert.equal(cut, `[resource link: ${huge.slice(0, 2048)}…; not fetched]`); +}); + +test('in a failed result an inline image is saved and named, or marked not retained, never dropped', async () => { + const { save, saved } = recordingSave(); + const r = await normalizeStandardToolResult({ + content: [{ type: 'text', text: 'render failed' }, { type: 'image', data: b64('partial'), mimeType: 'image/png' }], + isError: true, + }, 'L', save); + assert.equal(r.success, false); + assert.deepEqual(saved, [{ fileName: 'L-1.png', bytes: 'partial', mimeType: 'image/png' }]); + assert.equal(r.error, + 'render failed\n' + + '[image: image/png, 7 B, saved to workspace file home/tool-results/L-1.png; not shown inline: a failed result is read as its text alone]'); + + const unsaved = await normalizeStandardToolResult({ + content: [{ type: 'image', data: b64('png'), mimeType: 'image/png' }], + isError: true, + }, 'L', null); + assert.equal(unsaved.error, + '[image: image/png, 3 B. Not retained: a failed result is read as its text alone, and no workspace is mounted to save it]'); + + // A payload lost beside it makes the result a failure too: the image is + // then a payload like the audio, and both are counted. + const lost = await normalizeStandardToolResult({ + content: [{ type: 'image', data: b64('png'), mimeType: 'image/png' }, { type: 'audio', data: b64('wav'), mimeType: 'audio/wav' }], + }, 'L', null); + assert.equal(lost.success, false); + assert.match(lost.error!, /^\[result incomplete: 2 payload\(s\) could not be retained/); + assert.match(lost.error!, /\[image: image\/png, 3 B\. Not retained: a failed result is read as its text alone/); +}); + +test('image and audio with inline data but no mimeType are payloads of no known type, saved', async () => { + const { save, saved } = recordingSave(); + const r = await normalizeStandardToolResult({ content: [{ type: 'image', data: b64('img') }, { type: 'audio', data: b64('aud') }] }, 'L', save); + assert.deepEqual(saved.map((p) => [p.fileName, p.bytes, p.mimeType]), [ + ['L-1.bin', 'img', 'application/octet-stream'], + ['L-2.bin', 'aud', 'application/octet-stream'], + ]); + assert.equal(r.success, true); + assert.equal(r.data, + '[image: application/octet-stream, 3 B, saved to workspace file home/tool-results/L-1.bin]\n' + + '[audio: application/octet-stream, 3 B, saved to workspace file home/tool-results/L-2.bin]'); +}); + +test("an image a provider can't take inline is saved: another type, or over the per-image size", async () => { + const { save, saved } = recordingSave(); + const atLimit = Buffer.alloc(5 * 1024 * 1024, 1).toString('base64'); + const overLimit = Buffer.alloc(5 * 1024 * 1024 + 1, 1).toString('base64'); + const r = await normalizeStandardToolResult({ + content: [ + { type: 'image', data: b64('tiff'), mimeType: 'image/tiff' }, + { type: 'image', data: overLimit, mimeType: 'image/png' }, + { type: 'image', data: atLimit, mimeType: 'image/png' }, + { type: 'image', data: b64('ok'), mimeType: 'IMAGE/JPEG; q=1' }, + ], + }, 'L', save); + assert.deepEqual(saved.map((p) => [p.fileName, p.mimeType]), [['L-1.tiff', 'image/tiff'], ['L-2.png', 'image/png']]); + const data = r.data as Array<{ type: string; text?: string; data?: string; mimeType?: string }>; + assert.equal(data[0]!.text, '[image: image/tiff, 4 B, saved to workspace file home/tool-results/L-1.tiff]'); + assert.equal(data[1]!.text, + '[image: image/png, 5.0 MB, saved to workspace file home/tool-results/L-2.png; not shown inline: over the 5.0 MB a model takes per image]'); + assert.deepEqual(data[2], { type: 'image', data: atLimit, mimeType: 'image/png' }, 'exactly at the limit stays native'); + assert.deepEqual(data[3], { type: 'image', data: b64('ok'), mimeType: 'image/jpeg' }, 'a type a provider takes, in its canonical spelling'); +}); + +test("a native image is passed without its encoding's line wrapping", async () => { + const flat = b64('a small png, wrapped'); + const wrapped = flat.replace(/(.{8})/g, '$1\r\n'); + assert.notEqual(wrapped, flat); + const r = await normalizeStandardToolResult({ content: [{ type: 'image', data: wrapped, mimeType: 'image/png' }] }, 'L', null); + assert.deepEqual(r.data, [{ type: 'image', data: flat, mimeType: 'image/png' }]); +}); + +test('a save that rejects is a lost payload, and why it failed is logged', async (t) => { + const logged: string[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => { logged.push(args.map((a) => (a instanceof Error ? a.message : String(a))).join(' ')); }); + const r = await normalizeStandardToolResult( + { content: [{ type: 'audio', data: b64('clip'), mimeType: 'audio/wav' }] }, + 'L', + async () => { throw new Error('disk gone'); }, + ); + assert.equal(r.success, false); + assert.match(r.error!, /\[audio: audio\/wav, 4 B\. Not retained: the model can't take this type, and saving it to the workspace failed\]/); + const saves = logged.filter((line) => line.startsWith('[mcp] saving tool payload')); + assert.equal(saves.length, 1, logged.join('\n')); + assert.match(saves[0]!, /saving tool payload L-1\.wav \(audio, "audio\/wav", 4 B\) failed: disk gone/); +}); + +test('malformed content: a bad entry is named and left out; content that is not a list is a malformed result', async () => { + const r = await normalizeStandardToolResult({ content: [{ type: 'text', text: 'ok' }, null, 'stray', { type: 'text', text: 42 }] }, 'L', null); + assert.equal(r.success, true); + assert.equal(r.data, + 'ok\n[malformed content entry, left out: null]\n[malformed content entry, left out: stray]\n' + + '[malformed text block, left out: its text is not a string]'); + + const notList = await normalizeStandardToolResult({ content: { type: 'text', text: 'a block, not a list' }, structuredContent: { n: 1 } }, 'L', null); + assert.deepEqual(notList, { + success: false, + error: '[malformed result: its content is not a list of blocks; the tool may already have completed]\n{"n":1}', + isError: true, + structured: { n: 1 }, + }); + + // Absent content is no content, as a structured-only result may send it. + const absent = await normalizeStandardToolResult({ structuredContent: { n: 1 } }, 'L', null); + assert.equal(absent.success, true); + assert.equal(absent.data, '{"n":1}'); +}); + +test("stub text never splits a surrogate pair, and a wrapped payload's size counts only its bytes", async () => { + // One ASCII character first, so a cut by UTF-16 units would land inside a pair. + const r = await normalizeStandardToolResult({ content: [{ type: 'resource_link', uri: 'memo://x', name: `a${'😀'.repeat(300)}` }] }, 'L', null); + assert.equal(r.data, `[resource link "a${'😀'.repeat(199)}…": memo://x; not fetched]`); + + const wrapped = b64('hello world').replace(/(.{4})/g, '$1\r\n'); + const sized = await normalizeStandardToolResult({ content: [{ type: 'audio', data: wrapped, mimeType: 'audio/wav' }] }, 'L', null); + assert.match(sized.error!, /\[audio: audio\/wav, 11 B\. Not retained/); +}); + +test("a subtype named like one of Object.prototype's members is just a name", async () => { + const { save, saved } = recordingSave(); + await normalizeStandardToolResult({ + content: [ + { type: 'audio', data: b64('a'), mimeType: 'audio/constructor' }, + { type: 'audio', data: b64('b'), mimeType: 'audio/__proto__' }, + ], + }, 'L', save); + assert.deepEqual(saved.map((p) => p.fileName), ['L-1.bin', 'L-2.proto']); +}); diff --git a/test/mcpl-agent-words.test.ts b/test/mcpl-agent-words.test.ts index e2aa88d72..30fb9d7cf 100644 --- a/test/mcpl-agent-words.test.ts +++ b/test/mcpl-agent-words.test.ts @@ -86,7 +86,7 @@ process.stdin.on('data', (c) => { if (!line.trim()) continue; let m; try { m = JSON.parse(line); } catch { continue; } const send = (o) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: m.id, ...o }) + '\\n'); - if (m.method === 'initialize') send({ result: { capabilities: {} } }); + if (m.method === 'initialize') send({ result: { protocolVersion: '2024-11-05', capabilities: {} } }); else if (m.method === 'tools/list') send({ result: { tools: [] } }); else if (m.method === 'tools/call' && m.params.name === 'refuse') send({ error: { code: -32000, message: 'Missing Access' } }); else if (m.method === 'tools/call' && m.params.name === 'mute') send({ error: { code: -32000, message: ' ' } }); @@ -241,7 +241,6 @@ function frameworkWith(server: unknown, config: Record = {}) { }; framework.mcplServerRegistry = { getServer: () => server }; framework.mcplServerConfigs = new Map([['words', { id: 'words', ...config }]]); - framework.mcplConnecting = new Set(); framework.emitTrace = () => {}; framework.pushEvent = (event: ToolResultEvent) => { events.push(event); if (event.type === 'tool-result') resolve(); }; return { framework, events, delivered }; @@ -329,9 +328,12 @@ test("a call while this host connects to the tool's server says so, and after a assert.equal((await framework.executeMcplToolCall(call('c6'), 'words', 'words')).error, goneText, "a script's call reads the same"); }); -test('a second connect, a restart or a disconnect of a server whose connect is in flight is refused before it writes anything', async () => { +test('a second connect, or a restart into the id, of a server whose connect is in flight is refused in these words before it writes anything', async () => { // The registry takes the server once its handshake is done, so until then - // only the connecting mark says the id is taken. Each handshake here waits for the test. + // the connect's hold on the id (mcpServerIdTaken) says the id is taken. Each + // handshake here waits for the test. A disconnect isn't refused: it ends the + // connect, as mcp-server-ownership's tests of a disconnect during a legacy + // handshake and during a modern connect pin. const servers = new Map(); const handshakes: Array<() => void> = []; const { framework } = frameworkWith(null); @@ -340,7 +342,7 @@ test('a second connect, a restart or a disconnect of a server whose connect is i getServer: (id: string) => servers.get(id) ?? null, addServer: (cfg: { id: string }) => new Promise((resolve) => { handshakes.push(() => { - const connection = { on: () => {} }; + const connection = { on: () => {}, isConnected: true, willReconnect: false }; servers.set(cfg.id, connection); resolve(connection); }); @@ -357,30 +359,22 @@ test('a second connect, a restart or a disconnect of a server whose connect is i registerMcplServerFeatures: async () => {}, }); const fw = framework as unknown as { - connectMcplServerInternal(config: McplServerConfig, deferAwareness?: boolean): Promise; - refuseMcplConnect(config: McplServerConfig, replacing?: string): void; - runDisconnectMcplServer(id: string): Promise; + connectMcplServerInternal(config: McplServerConfig, deferAwareness?: boolean): Promise; + mcpServerIdTaken(id: string, claim?: unknown, restarting?: string): string | null; }; const words = { id: 'words', url: 'ws://127.0.0.1:9', toolPrefix: 'words' } as McplServerConfig; - const other = { id: 'other', url: 'ws://127.0.0.1:9', toolPrefix: 'other' } as McplServerConfig; const connecting = fw.connectMcplServerInternal(words, true); await new Promise((r) => setImmediate(r)); assert.equal(handshakes.length, 1, 'the handshake has begun'); - const inFlight = { message: 'MCPL server "words" has a connect in flight; try again once it settles' }; - assert.throws(() => fw.refuseMcplConnect(words), inFlight, 'a second connect of it'); - // A restart's disconnect can't stop the connect in flight, which would register the server after it. - assert.throws(() => fw.refuseMcplConnect(words, 'words'), inFlight, 'a restart of it'); - assert.throws(() => fw.refuseMcplConnect(other, 'words'), inFlight, 'a restart of it into another id'); - assert.throws(() => fw.refuseMcplConnect(words, 'other'), inFlight, 'a restart of another server into its id'); - // A disconnect can't stop it either: the server would register after it, its config and prefix gone. - await assert.rejects(fw.runDisconnectMcplServer('words'), inFlight, 'a disconnect of it'); - assert.equal((framework.mcplPrefixMap as Map).get('words'), 'words', 'the refused disconnect left its prefix'); + const inFlight = 'MCPL server "words" has a connect in flight; try again once it settles'; + await assert.rejects(fw.connectMcplServerInternal(words, true), { message: inFlight }, 'a second connect of it'); + assert.equal(handshakes.length, 1, 'the refused connect began no handshake'); + assert.equal(fw.mcpServerIdTaken('words', undefined, 'other'), `MCP server "other" was not restarted: ${inFlight}`, + 'a restart of another server into its id'); handshakes[0]!(); await connecting; - // Once the connect settles, the mark is gone: the registry's check refuses a - // second connect, and a restart, which disconnects the server first, may go ahead. - assert.throws(() => fw.refuseMcplConnect(words), { message: 'MCPL server "words" is already registered' }); - assert.doesNotThrow(() => fw.refuseMcplConnect(words, 'words')); + // Once the connect settles, the registered server holds the id. + assert.equal(fw.mcpServerIdTaken('words'), 'MCPL server "words" is already registered'); }); test("an MCPL tool's error result with no words says so, and that its effect is unknown", async () => { diff --git a/test/mcpl-awareness-barrier.test.ts b/test/mcpl-awareness-barrier.test.ts index 52c657152..7ea978dba 100644 --- a/test/mcpl-awareness-barrier.test.ts +++ b/test/mcpl-awareness-barrier.test.ts @@ -858,6 +858,11 @@ async function failPolicyRegistrationWhileServing(dir: string, closeFails: boole try { const traces: Array<{ type: string; serverId?: string; error?: string }> = []; framework.onTrace((event) => traces.push(event as never)); + // Which servers' tools the framework asks to relist. + const lists = framework as unknown as { handleToolsListChanged(id: string): void }; + const relist = lists.handleToolsListChanged.bind(framework); + const refreshed: string[] = []; + lists.handleToolsListChanged = (id: string) => { refreshed.push(id); relist(id); }; const features = FeatureSetManager.prototype as any; const initializeServer = features.initializeServer; features.initializeServer = function (this: unknown, serverId: string, ...rest: unknown[]) { @@ -900,7 +905,7 @@ async function failPolicyRegistrationWhileServing(dir: string, closeFails: boole false, 'no gate failed', ); - return { framework, discord, registry: internals.mcplServerRegistry }; + return { framework, discord, registry: internals.mcplServerRegistry, refreshed }; } catch (error) { await framework.stop(); throw error; @@ -943,6 +948,7 @@ test('a runtime connect whose policy registration throws and whose close then fa assert.ok(refused, 'the connection whose close failed is still registered'); assert.equal(refused.willReconnect, false, 'its close had disarmed it before the transport close failed'); assert.equal(refused.dataPlaneReady, false, 'the gate\'s completion left the refused connection paused'); + assert.ok(failed.refreshed.includes(failed.discord.id), 'a refresh of the tool list was asked for though its close failed'); await failed.framework.syncDiscordAwarenessMarkers(); assert.equal(refused.dataPlaneReady, false, 'a later gate\'s completion leaves it paused too'); assert.ok( @@ -1158,7 +1164,7 @@ test('inbound handlers wait out a generation installed while they wait', async ( framework.handleHostCommand = async () => { handled.push('host-command'); return { ok: true }; }; const connection = new EventEmitter() as any; connection.id = 'discord'; - framework.wireMcplEvents(connection); + framework.wireMcplEvents(connection, { id: 'discord' }); const responder = { id: 1, respond: () => {}, respondError: () => {} }; for (const event of ['push-event', 'inference-request', 'channels-incoming', 'host-command']) { connection.emit(event, {}, responder); @@ -1194,7 +1200,7 @@ test('an inbound handler waiting on a generation that fails refuses its request framework.handleHostCommand = async () => { handled.push('host-command'); return { ok: true }; }; const connection = new EventEmitter() as any; connection.id = 'discord'; - framework.wireMcplEvents(connection); + framework.wireMcplEvents(connection, { id: 'discord' }); const answers: Array<{ event: string; code?: number; result?: unknown }> = []; for (const event of ['push-event', 'inference-request', 'channels-incoming', 'host-command']) { connection.emit(event, {}, { @@ -1226,7 +1232,7 @@ test('with no generation current, an inbound event is handled in the turn it arr framework.handleHostCommand = async () => { handled.push('host-command'); return { ok: true }; }; const connection = new EventEmitter() as any; connection.id = 'discord'; - framework.wireMcplEvents(connection); + framework.wireMcplEvents(connection, { id: 'discord' }); const responder = { id: 1, respond: () => {}, respondError: () => {} }; for (const event of ['push-event', 'inference-request', 'channels-incoming', 'host-command']) { connection.emit(event, {}, responder); diff --git a/test/mcpl-capability-scoping.test.ts b/test/mcpl-capability-scoping.test.ts index e0c7777ef..29f223e16 100644 --- a/test/mcpl-capability-scoping.test.ts +++ b/test/mcpl-capability-scoping.test.ts @@ -179,7 +179,7 @@ process.stdin.on('data', (c) => { if (!line.trim()) continue; let m; try { m = JSON.parse(line); } catch { continue; } if (m.method === 'initialize') { - send({ jsonrpc: '2.0', id: m.id, result: { capabilities: { experimental: { mcpl: { + send({ jsonrpc: '2.0', id: m.id, result: { protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true, contextHooks: { beforeInference: true, afterInference: { blocking: true } }, diff --git a/test/mcpl-config-booleans.test.ts b/test/mcpl-config-booleans.test.ts index 42fccef1a..28e027400 100644 --- a/test/mcpl-config-booleans.test.ts +++ b/test/mcpl-config-booleans.test.ts @@ -62,11 +62,15 @@ test('any other value is refused, naming the server, the setting and the kind of } }); -test('inheritEnv is checked only for a stdio server, since a WebSocket server never reads it', () => { +test('inheritEnv is checked only for a stdio server, since a network server never reads it', () => { // A url alone or transport: 'websocket' dials, as openTransport chooses. check({ ...WEBSOCKET, inheritEnv: 'false' }); check({ id: 'remote', url: 'ws://127.0.0.1:1', command: 'unused', transport: 'websocket', inheritEnv: 'no' }); - // A url beside a command, with no transport, spawns the command. + // So does an http(s) url, over the modern engine. + check({ id: 'web', url: 'https://127.0.0.1:1/mcp', inheritEnv: 'false' }); + // A url beside a command, with no transport, names no transport to use, so + // its family check refuses it; its booleans are still checked as a stdio + // server's. assert.throws(() => check({ id: 'both', url: 'ws://127.0.0.1:1', command: 'discord-mcpl', inheritEnv: 'false' }), /MCPL server "both" inheritEnv must be true or false, unquoted, not a string: /); // The other two apply to both transports. diff --git a/test/mcpl-config-diagnostics.test.ts b/test/mcpl-config-diagnostics.test.ts index a820d86dc..c0c085ec6 100644 --- a/test/mcpl-config-diagnostics.test.ts +++ b/test/mcpl-config-diagnostics.test.ts @@ -334,6 +334,78 @@ test('FeatureSetManager: each §6.4 disablement is traced beside its console lin assert.equal(quiet.lines.length, 3, 'the console line stays'); }); +test("a refused initial policy's reason is recorded without the server's configured credentials", async () => { + const fw = Object.create(AgentFramework.prototype) as any; + fw.traceListeners = []; + fw.featureSetManager = new FeatureSetManager(); + fw.checkpointManager = null; + const traces: Traced[] = []; + fw.onTrace((e: Traced) => traces.push(e)); + const secret = 'srvSecret0123456789'; + const connection = { + id: 'srv', + capabilities: { version: '0.5', pushEvents: true } as unknown as McplCapabilities, + mcpToolsAdvertised: false, + establishGrant: () => {}, + sendFeatureSetsUpdateRequest: async () => ({ accepted: false, fallback: 'mcp-only', reason: `policy for ${secret} too narrow` }), + close: async () => {}, + }; + const capture = captureConsoleError(); + try { + await fw.registerMcplServerFeatures({ id: 'srv', url: `wss://h/mcp/${secret}` } as McplServerConfig, connection); + } finally { + capture.stop(); + } + const refused = traces.filter((t) => t.type === 'mcpl:policy-refused'); + assert.equal(refused.length, 1); + assert.equal((refused[0] as { reason?: string }).reason, 'policy for redacted too narrow'); + assert.ok(capture.lines.some((l) => l.includes('srv refused initial policy (policy for redacted too narrow)')), capture.lines.join('\n')); + assert.ok(!capture.lines.join('\n').includes(secret)); +}); + +test("the initial policy's other records keep the credentials out too", async () => { + const secret = 'srvSecret0123456789'; + const run = async (answer: () => Promise) => { + const fw = Object.create(AgentFramework.prototype) as any; + fw.traceListeners = []; + fw.featureSetManager = new FeatureSetManager(); + fw.checkpointManager = null; + const traces: Traced[] = []; + fw.onTrace((e: Traced) => traces.push(e)); + const config = { id: 'srv', url: `wss://h/mcp/${secret}` } as McplServerConfig; + const connection = { + id: 'srv', + capabilities: { version: '0.5', pushEvents: true } as unknown as McplCapabilities, + mcpToolsAdvertised: false, + establishGrant: () => {}, + sendFeatureSetsUpdateRequest: async () => answer(), + close: async () => {}, + }; + const capture = captureConsoleError(); + try { await fw.registerMcplServerFeatures(config, connection); } finally { capture.stop(); } + const text = capture.lines.join('\n') + JSON.stringify(traces); + assert.ok(!text.includes(secret), text); + return { lines: capture.lines, traces }; + }; + // An unanswered request whose error quotes the url. + const unanswered = await run(async () => { throw new Error(`timed out talking to wss://h/mcp/${secret}`); }); + assert.ok(unanswered.lines.some((l) => l.includes('did not answer the initial policy Request') && l.includes('timed out talking to wss://h/mcp/redacted')), unanswered.lines.join('\n')); + // A refusal whose fallback isn't one the host knows: it logs what it does. + const odd = await run(async () => ({ accepted: false, fallback: `close-${secret}`, reason: 'no' })); + assert.ok(odd.lines.some((l) => l.includes('refused initial policy (no) — fallback: mcp-only')), odd.lines.join('\n')); + // A degraded acceptance whose effect quotes the url, and an entry with no + // capability list, which is logged rather than read as an unanswered request. + const degraded = await run(async () => ({ accepted: true, mode: 'degraded', unavailableFeatures: [ + { featureSet: 'fs', effect: `see ${secret}`, missingCapabilities: ['x'] }, + { featureSet: 'odd', effect: 'none' }, + ] })); + assert.ok(degraded.lines.some((l) => l.includes('srv degraded: fs (see redacted; missing x)')), degraded.lines.join('\n')); + assert.ok(degraded.lines.some((l) => l.includes('srv degraded: odd (none; missing unknown)')), degraded.lines.join('\n')); + assert.ok(!degraded.lines.some((l) => l.includes('did not answer')), degraded.lines.join('\n')); + // A refusal's reason, recorded with the derivation the operation began with. + await run(async () => ({ accepted: false, fallback: 'mcp-only', reason: `bad ${secret}` })); +}); + test('registerMcplServerFeatures: a refused initial policy is traced, and fallback close closes the server and unregisters it', async () => { for (const fallback of ['close', 'mcp-only'] as const) { const fw = Object.create(AgentFramework.prototype) as any; @@ -464,6 +536,83 @@ test('manifest change: a refused policy is traced with phase manifest-change, fo ]); }); +test("a policy refusal's 'close' fallback whose close fails reports that failure, not an unanswered policy, and asks for the tool list to be refreshed", async () => { + const unreaped = () => new Error('stdio child 7 of MCP server "srv" did not exit within 2000ms of SIGKILL; it could not be reaped'); + // At the initial policy. + const fw = Object.create(AgentFramework.prototype) as any; + fw.traceListeners = []; + fw.featureSetManager = new FeatureSetManager(); + fw.checkpointManager = null; + const refreshed: string[] = []; + fw.handleToolsListChanged = (id: string) => { refreshed.push(id); }; + const connection = { + id: 'srv', + capabilities: { version: '0.5', pushEvents: true } as unknown as McplCapabilities, + mcpToolsAdvertised: false, + establishGrant: () => {}, + sendFeatureSetsUpdateRequest: async () => ({ accepted: false, fallback: 'close', reason: 'policy too narrow' }), + close: async () => { throw unreaped(); }, + }; + const registry = new McplServerRegistry(); + (registry as any).servers.set('srv', connection); + fw.mcplServerRegistry = registry; + const initial = captureConsoleError(); + try { + await assert.rejects(fw.registerMcplServerFeatures({ id: 'srv', command: 'unused' } as McplServerConfig, connection), /it could not be reaped$/); + } finally { + initial.stop(); + } + assert.deepEqual(refreshed, ['srv'], 'a refresh of the tool list was asked for'); + assert.equal(registry.getServer('srv'), connection, 'it stays registered, holding its id'); + assert.ok(!initial.lines.some((l) => l.includes('did not answer')), initial.lines.join('\n')); + + // At a manifest change. + const h = manifestHarness({ accepted: false, fallback: 'close', reason: 'too narrow' }); + h.connection.close = () => Promise.reject(unreaped()); + const changed: string[] = []; + h.fw.handleToolsListChanged = (id: string) => { changed.push(id); }; + const later = captureConsoleError(); + try { + await assert.rejects(h.fw.handleManifestChanged(h.connection, { revision: 'r2', domains: ['capabilities'] }), /it could not be reaped$/); + } finally { + later.stop(); + } + assert.deepEqual(changed, ['srv'], 'a refresh of the tool list was asked for'); + assert.ok(!later.lines.some((l) => l.includes('did not answer')), later.lines.join('\n')); +}); + +for (const queuedBehind of ['the fetch floor', 'a run in flight'] as const) { + test(`a manifest refresh queued behind ${queuedBehind}, whose 'close' fallback fails to close, logs that failure`, async () => { + const h = manifestHarness({ accepted: false, fallback: 'close', reason: 'too narrow' }); + h.connection.close = () => Promise.reject(new Error('stdio child 7 of MCP server "srv" did not exit within 2000ms of SIGKILL; it could not be reaped')); + const announce = { revision: 'r2', domains: ['capabilities'] }; + const capture = captureConsoleError(); + try { + if (queuedBehind === 'the fetch floor') { + // The last refresh finished just inside the floor, so this one is + // queued, and runs on its own a few milliseconds later. + const floor = (AgentFramework as unknown as { MANIFEST_FETCH_FLOOR_MS: number }).MANIFEST_FETCH_FLOOR_MS; + h.fw.manifestRefreshState.set('srv', { inFlight: false, queued: false, lastCompletedAt: Date.now() - floor + 20 }); + await h.fw.handleManifestChanged(h.connection, announce); + assert.equal(h.fw.manifestRefreshState.get('srv').queued, true, 'queued, not run yet'); + } else { + // The second announcement comes while the first run is in flight, so + // it's queued, and runs once the fetch floor has passed after it. + const first = h.fw.handleManifestChanged(h.connection, announce).then(() => null, (error: unknown) => error); + await h.fw.handleManifestChanged(h.connection, announce); + assert.equal(h.fw.manifestRefreshState.get('srv').queued, true, 'queued behind the first'); + assert.match(String(await first), /it could not be reaped$/, 'the first run rejects with its failed close'); + } + await waitFor( + () => capture.lines.some((l) => l.includes('srv manifest-changed handler failed') && l.includes('it could not be reaped')), + 'the queued run to log its failed close', + ); + } finally { + capture.stop(); + } + }); +} + test('policy-refused traces report the branch the host took, not raw receipt fields', async () => { // Anything but `close` takes the mcp-only branch; a non-string reason is dropped. const odd = await runManifestChange({ accepted: false, fallback: 'other', reason: 42 }); diff --git a/test/mcpl-late-render.test.ts b/test/mcpl-late-render.test.ts index 1a3289070..7e1135ee6 100644 --- a/test/mcpl-late-render.test.ts +++ b/test/mcpl-late-render.test.ts @@ -38,7 +38,7 @@ async function timedOutRender(t: TestContext, params: PushRenderParams = RENDER) const framework = Object.create(AgentFramework.prototype) as any; framework.traceListeners = []; framework.onTrace((event: never) => traces.push(event)); - framework.wireMcplEvents(connection); + framework.wireMcplEvents(connection, { id: 'srv' }); const orphans: unknown[] = []; connection.on('orphaned-response', (info: unknown) => orphans.push(info)); connection.ready(); diff --git a/test/mcpl-legacy-lifetime.test.ts b/test/mcpl-legacy-lifetime.test.ts new file mode 100644 index 000000000..678abb461 --- /dev/null +++ b/test/mcpl-legacy-lifetime.test.ts @@ -0,0 +1,426 @@ +/** + * The legacy engine's lifetime contract: one owner for every launch, the + * rule the modern engine keeps. + * - a configuration it can't use is refused before anything is dialed, and + * never turned into a retry stub; + * - concurrent close() calls share one teardown and its verdict; + * - a failed launch that can't be reaped is held, never orphaned: connect() + * hands its owner to the caller, connectWithReconnect() and the reconnect + * loop halt, and close() retries the reap; + * - close() during a reconnect handshake ends that launch before resolving; + * - no reconnect launches while the previous child may still be running. + */ +import { test, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, readFileSync, existsSync, rmSync } from 'node:fs'; +import type { AddressInfo } from 'node:net'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { WebSocketServer } from 'ws'; + +import { McplServerConnection, McplUnreapedLaunchError } from '../src/mcpl/server-connection.js'; +import { McplServerRegistry } from '../src/mcpl/server-registry.js'; +import { StdioTransport, openTransport } from '../src/mcpl/transport.js'; +import { createServer, type Socket } from 'node:net'; +import { createServer as createHttpServer } from 'node:http'; +import { EventEmitter } from 'node:events'; +import type { McplHostCapabilities, McplServerConfig } from '../src/mcpl/types.js'; + +const here = dirname(fileURLToPath(import.meta.url)); +const SERVER = join(here, 'fixtures', 'legacy-version-server.mjs'); +const HOST_CAPS: McplHostCapabilities = { version: '0.5', pushEvents: true }; + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +function scratchLog(): string { + const dir = mkdtempSync(join(tmpdir(), 'legacy-lifetime-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + return join(dir, 'starts.log'); +} + +/** The pid of each launch, in order. */ +const starts = (log: string): number[] => + existsSync(log) ? readFileSync(log, 'utf8').split('\n').filter(Boolean).map((l) => Number(l.split(' ')[1])) : []; +const alive = (pid: number) => { try { process.kill(pid, 0); return true; } catch { return false; } }; + +async function until(cond: () => boolean, what: string, ms = 10_000): Promise { + const deadline = Date.now() + ms; + while (!cond() && Date.now() < deadline) await new Promise((r) => setTimeout(r, 20)); + assert.ok(cond(), `in time: ${what}`); +} + +function config(mode: string, log: string, extra: Partial = {}): McplServerConfig { + return { id: 'legacy', command: process.execPath, args: [SERVER, mode, log], ...extra }; +} +const RECONNECT = { reconnect: true, reconnectIntervalMs: 20, reconnectMaxIntervalMs: 40 } as const; + +/** + * Launches from the `from`-th on can't be signalled, so their cleanup can't + * reap them (Olive-1804's pattern in mcp-modern-outcomes). `kill` really ends + * one; cleanup restores spawning and ends any left. + */ +function unreapableFrom(from: number): Array<{ pid: number; kill: () => void }> { + const realSpawn = StdioTransport.spawn; + const stuck: Array<{ pid: number; kill: () => void }> = []; + let launches = 0; + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = async (c) => { + const transport = await realSpawn.call(StdioTransport, c); + if (++launches >= from) { + const child = (transport as unknown as { child: { pid: number; kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + child.kill = () => true; + stuck.push({ pid: child.pid, kill: () => { realKill('SIGKILL'); } }); + } + return transport; + }; + cleanups.push(() => { + (StdioTransport as unknown as { spawn: typeof StdioTransport.spawn }).spawn = realSpawn; + for (const s of stuck) s.kill(); + }); + return stuck; +} + +async function killAndWait(stuck: { pid: number; kill: () => void }): Promise { + stuck.kill(); + await until(() => !alive(stuck.pid), `child ${stuck.pid} gone`); +} + +test('a configuration the engine cannot use is refused before anything is dialed, and never retried', async () => { + const wss = new WebSocketServer({ host: '127.0.0.1', port: 0 }); + await new Promise((r) => wss.once('listening', r)); + cleanups.push(() => new Promise((r) => wss.close(() => r()))); + let dials = 0; + wss.on('connection', (socket) => { dials++; socket.close(); }); + const url = `ws://127.0.0.1:${(wss.address() as AddressInfo).port}/mcpl`; + + const modernOnWs = { id: 'ws', url, protocol: 'modern' as const }; + await assert.rejects(McplServerConnection.connect(modernOnWs, HOST_CAPS), /"protocol" applies only to stdio/); + // With reconnect too: a configuration error is thrown, not turned into a + // stub that retries it forever. + await assert.rejects(McplServerConnection.connectWithReconnect({ ...modernOnWs, ...RECONNECT }, HOST_CAPS), /"protocol" applies only to stdio/); + await assert.rejects( + McplServerConnection.connectWithReconnect({ id: 'ftp', url: 'ftp://127.0.0.1/x', ...RECONNECT }, HOST_CAPS), + /url must be ws:\/\/ or wss:\/\//, + ); + // A refusal is synchronous with the connect call; 200 ms covers several + // reconnect intervals (20–40 ms) in which a stub would have dialed. + await new Promise((r) => setTimeout(r, 200)); + assert.equal(dials, 0, 'nothing was dialed'); +}); + +test('concurrent close() calls share the teardown: none resolves while the child still runs', async () => { + const log = scratchLog(); + const connection = await McplServerConnection.connect(config('ok', log, { env: { IGNORE_SIGTERM: '1' } }), HOST_CAPS); + connection.ready(); + const [pid] = starts(log); + const order: string[] = []; + const first = connection.close().then(() => order.push(`first:${alive(pid!) ? 'alive' : 'gone'}`)); + const second = connection.close().then(() => order.push(`second:${alive(pid!) ? 'alive' : 'gone'}`)); + await Promise.all([first, second]); + assert.deepEqual(order, ['first:gone', 'second:gone']); +}); + +test('a close that cannot reap fails, and so do concurrent ones; a later close re-checks and settles', async () => { + const log = scratchLog(); + const stuck = unreapableFrom(1); + const connection = await McplServerConnection.connect(config('ok', log), HOST_CAPS); + connection.ready(); + const settled: string[] = []; + const outcome = (name: string) => [() => settled.push(`${name}:ok`), (e: Error) => settled.push(`${name}:${e.message}`)] as const; + const first = connection.close().then(...outcome('first')); + const second = connection.close().then(...outcome('second')); + // The teardown sends SIGTERM, waits 2 s, sends SIGKILL and waits 2 s more + // before it gives up, so at 1 s it is still pending. + await new Promise((r) => setTimeout(r, 1000)); + assert.deepEqual(settled, [], 'neither resolves while the teardown is pending'); + await Promise.all([first, second]); + assert.equal(settled.length, 2); + assert.match(settled[0]!, /^first:stdio child \d+ of MCP server "legacy" did not exit within \d+ms of SIGKILL; it could not be reaped/); + assert.match(settled[1]!, /^second:.*could not be reaped/, 'the second shares the attempt and its verdict, rather than claiming success'); + await killAndWait(stuck[0]!); + await connection.close(); +}); + +test('a failed connect that cannot be reaped hands its owner to the caller, who can retry the reap', async () => { + const log = scratchLog(); + const stuck = unreapableFrom(1); + const error = await McplServerConnection.connect(config('error', log), HOST_CAPS).then(() => null, (e: unknown) => e); + assert.ok(error instanceof McplUnreapedLaunchError, String(error)); + assert.match(error.message, /initialize failed on purpose; its cleanup also failed: .*could not be reaped/); + assert.match((error.cause as Error).message, /initialize failed on purpose/); + const owner = error.connection; + assert.equal(owner.isConnected, false); + assert.equal(owner.willReconnect, false); + await assert.rejects(owner.close(), /could not be reaped/, 'still running: the reap fails again'); + await killAndWait(stuck[0]!); + await owner.close(); +}); + +test('with reconnect too, a failed first launch that cannot be reaped throws, handing over its owner, as without', async () => { + const log = scratchLog(); + const stuck = unreapableFrom(1); + const error = await McplServerConnection.connectWithReconnect(config('error', log, RECONNECT), HOST_CAPS).then(() => null, (e: unknown) => e); + // No stub: no retry starts beside a child that may still be running. + assert.ok(error instanceof McplUnreapedLaunchError, String(error)); + assert.match(error.message, /initialize failed on purpose; its cleanup also failed: .*could not be reaped/); + const owner = error.connection; + cleanups.push(() => owner.close().catch(() => {})); + assert.equal(owner.willReconnect, false); + await assert.rejects(owner.close(), /could not be reaped/, 'still running: the reap fails again'); + await killAndWait(stuck[0]!); + await owner.close(); + assert.equal(starts(log).length, 1, 'no launch beside the unreaped one'); +}); + +test('a reconnect launch that cannot be reaped halts reconnecting; close() settles it once it has exited', async () => { + const log = scratchLog(); + const stuck = unreapableFrom(2); + const connection = await McplServerConnection.connectWithReconnect(config('ok-then-error', log, RECONNECT), HOST_CAPS); + cleanups.push(() => connection.close().catch(() => {})); + connection.ready(); + const failures: Array<{ error: string; attempt: number; permanent?: boolean }> = []; + connection.on('reconnect-failed', (f) => failures.push(f)); + await assert.rejects(connection.sendToolsCall('die', {})); + await until(() => failures.some((f) => f.permanent), 'a permanent reconnect failure'); + assert.match(failures.find((f) => f.permanent)!.error, /initialize failed on purpose; the previous launch could not be reaped .*reconnecting is halted/); + assert.equal(connection.willReconnect, false); + // 300 ms is several reconnect intervals (20–40 ms): a retry would have launched. + await new Promise((r) => setTimeout(r, 300)); + assert.equal(starts(log).length, 2, 'no third launch while the second is unreaped'); + await assert.rejects(connection.close(), /could not be reaped/); + await killAndWait(stuck[0]!); + await connection.close(); +}); + +test("a reconnect stub held as unreaped before its first ready() still delivers its connect-failed and its permanent failure there", async () => { + const log = scratchLog(); + // The first launch fails its handshake and is reaped, so a stub reconnects; + // the second fails too, and its cleanup can't reap it. + const stuck = unreapableFrom(2); + const connection = await McplServerConnection.connectWithReconnect(config('error', log, RECONNECT), HOST_CAPS); + cleanups.push(() => connection.close().catch(() => {})); + const reports: Array<{ event: string; attempt: number; permanent?: boolean }> = []; + connection.on('connect-failed', (f: { attempt: number }) => reports.push({ event: 'connect-failed', ...f })); + connection.on('reconnect-failed', (f: { attempt: number; permanent?: boolean }) => reports.push({ event: 'reconnect-failed', ...f })); + await until(() => connection.holdsUnreapedChild, 'the reconnect launch held'); + assert.equal(connection.willReconnect, false); + assert.equal(reports.length, 0, 'held until the first release'); + connection.ready(); + assert.deepEqual( + reports.map((r) => [r.event, r.event === 'connect-failed' ? r.attempt : r.permanent === true]), + [['connect-failed', 0], ['reconnect-failed', true]], + ); + await assert.rejects(connection.close(), /could not be reaped/); + await killAndWait(stuck[0]!); + await connection.close(); +}); + +test('close() during a reconnect handshake ends that launch before resolving, and nothing is adopted', async () => { + const log = scratchLog(); + const connection = await McplServerConnection.connectWithReconnect(config('ok-then-hang', log, RECONNECT), HOST_CAPS); + connection.ready(); + let reconnected = false; + connection.on('reconnect', () => { reconnected = true; }); + await assert.rejects(connection.sendToolsCall('die', {})); + await until(() => starts(log).length === 2, 'the reconnect launch has started (its start line)'); + const second = starts(log)[1]!; + const began = Date.now(); + await connection.close(); + // A generous bound, well below the 30 s handshake timeout it rules out. + assert.ok(Date.now() - began < 20_000, 'promptly: the handshake was ended, not waited out (30 s)'); + assert.equal(alive(second), false, 'the launch in flight was reaped before close() resolved'); + assert.equal(reconnected, false); + assert.equal(connection.isConnected, false); + // Several reconnect intervals (20–40 ms) later, still nothing. + await new Promise((r) => setTimeout(r, 200)); + assert.equal(starts(log).length, 2, 'and no later launch'); +}); + +test('a recycle during a reconnect in flight starts no second attempt, and close() still ends the one in flight', async () => { + // Iris-1827's interleaving (room-293 #55410): before, the recycle's attempt + // took close()'s ownership of the first, which outlived a successful close. + const log = scratchLog(); + const connection = await McplServerConnection.connectWithReconnect(config('ok-then-hang', log, RECONNECT), HOST_CAPS); + connection.ready(); + await assert.rejects(connection.sendToolsCall('die', {})); + await until(() => starts(log).length === 2, 'the reconnect launch has started (its start line)'); + await connection.reconnectAfterFailure(); + // Several reconnect intervals (20–40 ms) later, still only the one in flight. + await new Promise((r) => setTimeout(r, 400)); + assert.equal(starts(log).length, 2, 'no second attempt beside the one in flight'); + const began = Date.now(); + await connection.close(); + assert.ok(Date.now() - began < 20_000, 'promptly: well before the 30 s handshake timeout'); + assert.deepEqual(starts(log).filter(alive), [], 'every launch reaped when close() resolved'); +}); + +test('a reconnect never launches while the previous child may still be running', async () => { + const log = scratchLog(); + const connection = await McplServerConnection.connectWithReconnect( + config('ok', log, { ...RECONNECT, env: { IGNORE_SIGTERM: '1' } }), + HOST_CAPS, + ); + cleanups.push(() => connection.close().catch(() => {})); + connection.ready(); + const [first] = starts(log); + let reconnected = false; + connection.on('reconnect', () => { reconnected = true; }); + // Recycle it as the framework does after an awareness failure: the host + // closes the transport itself, and the child ignores SIGTERM for 2 s. + const recycled = connection.reconnectAfterFailure(); + await until(() => starts(log).length === 2, 'the next launch'); + assert.equal(alive(first!), false, 'it started only once the first child was gone'); + await recycled; + await until(() => reconnected, 'reconnected'); +}); + +test('registry: an id stays held through its reap, and a second add is refused while one is connecting', async () => { + const log = scratchLog(); + const registry = new McplServerRegistry(); + // A child that ignores SIGTERM keeps its reap going for about 2 s. + const removing = (async () => { + await registry.addServer(config('ok', log, { env: { IGNORE_SIGTERM: '1' } }), HOST_CAPS); + return registry.removeServer('legacy'); + })(); + await until(() => registry.getServer('legacy') !== null, 'registered'); + await until(() => starts(log).length === 1, 'launched'); + await new Promise((r) => setTimeout(r, 100)); + await assert.rejects(registry.addServer(config('ok', log), HOST_CAPS), /already registered/, 'held while its reap runs'); + await removing; + assert.equal(registry.getServer('legacy'), null, 'freed once the exit is confirmed'); + assert.equal(starts(log).length, 1, 'nothing launched beside it'); + + // Two adds at once: the second is refused while the first is connecting. + const slow = registry.addServer(config('slow', log, { env: { SLOW_INIT_MS: '300' } }), HOST_CAPS); + await assert.rejects(registry.addServer(config('ok', log), HOST_CAPS), /already registered/); + await slow; + await registry.removeServer('legacy'); +}); + +test('registry: a connection whose reap fails stays registered, through close() and closeAll(), until a retry confirms the exit', async () => { + const log = scratchLog(); + const registry = new McplServerRegistry(); + const stuck = unreapableFrom(1); + const connection = await registry.addServer(config('ok', log), HOST_CAPS); + // Its own close() can't reap: the registry keeps it, holding the id. + await assert.rejects(connection.close(), /could not be reaped/); + assert.equal(registry.getServer('legacy'), connection, 'still registered after its own close'); + // closeAll() reports it, and keeps it. + await assert.rejects(registry.closeAll(), /could not be reaped/); + assert.equal(registry.getServer('legacy'), connection, 'still registered after closeAll'); + await assert.rejects(registry.addServer(config('ok', log), HOST_CAPS), /already registered/); + // removeServer retries the reap; once the child is gone it succeeds. + await assert.rejects(registry.removeServer('legacy'), /could not be reaped/); + assert.equal(registry.getServer('legacy'), connection); + await killAndWait(stuck[0]!); + await registry.removeServer('legacy'); + assert.equal(registry.getServer('legacy'), null); +}); + +test('registry: closeAll closes every connection, removes the ones it reaped, and keeps and reports the one it could not', async () => { + const log = scratchLog(); + const registry = new McplServerRegistry(); + const stuck = unreapableFrom(2); + const fine = await registry.addServer({ ...config('ok', log), id: 'fine' }, HOST_CAPS); + const held = await registry.addServer({ ...config('ok', log), id: 'held' }, HOST_CAPS); + await assert.rejects(registry.closeAll(), /could not be reaped/); + assert.equal(registry.getServer('fine'), null, 'the one it reaped is removed'); + assert.equal(fine.isConnected, false); + assert.equal(registry.getServer('held'), held, 'the one it could not reap stays'); + await killAndWait(stuck[0]!); + await registry.removeServer('held'); +}); + +test('a reconnect whose previous child cannot be reaped holds it and halts reconnecting', async () => { + const log = scratchLog(); + const stuck = unreapableFrom(1); + const connection = await McplServerConnection.connectWithReconnect(config('ok', log, RECONNECT), HOST_CAPS); + cleanups.push(() => connection.close().catch(() => {})); + connection.ready(); + const failures: Array<{ error: string; attempt: number; permanent?: boolean }> = []; + connection.on('reconnect-failed', (f) => failures.push(f)); + // Recycle it as the framework does: the host closes the transport, whose + // child can't be reaped, and the loop then finds that child still held. + await connection.reconnectAfterFailure().catch(() => {}); + await until(() => failures.some((f) => f.permanent), 'a permanent reconnect failure'); + assert.match(failures.find((f) => f.permanent)!.error, /could not be reaped/); + assert.equal(connection.willReconnect, false); + // Several reconnect intervals (20–40 ms) later, nothing launched beside it. + await new Promise((r) => setTimeout(r, 300)); + assert.equal(starts(log).length, 1); + await killAndWait(stuck[0]!); + await connection.close(); +}); + +test('a WebSocket dial whose connection is closed meanwhile ends at once, not at the open timeout', async () => { + // A listener that accepts and never answers the upgrade. + const sockets: Socket[] = []; + const server = createServer((socket) => { sockets.push(socket); }); + await new Promise((r) => server.listen(0, '127.0.0.1', () => r())); + cleanups.push(() => new Promise((r) => { for (const s of sockets) s.destroy(); server.close(() => r()); })); + const url = `ws://127.0.0.1:${(server.address() as AddressInfo).port}/mcpl`; + const abort = new AbortController(); + const began = Date.now(); + const dial = openTransport({ id: 'ws', url }, abort.signal); + await until(() => sockets.length === 1, 'the dial reached the listener'); + abort.abort(); + await assert.rejects(dial, /WebSocket connect abandoned: the connection was closed/); + assert.ok(Date.now() - began < 5_000, 'well before the 15 s open timeout'); +}); + +test("a write to a stdio child that has just exited is never thrown, and isn't reported as an error: the exit's close names it", (t) => { + const child = Object.assign(new EventEmitter(), { stdin: new EventEmitter(), stderr: new EventEmitter(), killed: false, exitCode: null }); + const rl = new EventEmitter(); + const transport = new (StdioTransport as unknown as new (c: unknown, r: unknown, id: string) => StdioTransport)(child, rl, 'legacy'); + const logged: unknown[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => { logged.push(args); }); + const epipe = Object.assign(new Error('write EPIPE'), { code: 'EPIPE' }); + assert.doesNotThrow(() => child.stdin.emit('error', epipe), 'with nothing listening'); + const reported: unknown[] = []; + transport.on('error', (error) => reported.push(error)); + child.stdin.emit('error', epipe); + assert.deepEqual(reported, [], "the stdin carries the protocol, so the exit's close reports it"); + const closed: Array<{ reason?: string }> = []; + transport.on('close', (info: { reason?: string }) => closed.push(info)); + child.emit('exit', 0, null); + assert.match(closed[0]?.reason ?? '', /its pipe failed with write EPIPE/); +}); + +test('close() during a WebSocket reconnect dial returns promptly, ending the dial', async () => { + // The first upgrade gets a minimal MCPL server; every later one hangs, so + // the reconnect's dial would wait out its 15 s open timeout. + const http = createHttpServer(); + const wss = new WebSocketServer({ noServer: true }); + const hung: Socket[] = []; + let upgrades = 0; + http.on('upgrade', (request, socket, head) => { + if (++upgrades === 1) wss.handleUpgrade(request, socket as Socket, head, (ws) => wss.emit('connection', ws, request)); + else hung.push(socket as Socket); + }); + let first: import('ws').WebSocket | undefined; + wss.on('connection', (ws) => { + first = ws; + ws.on('message', (data) => { + const message = JSON.parse(String(data)) as { id?: number; method?: string }; + if (message.id === undefined) return; + const result = message.method === 'initialize' ? { protocolVersion: '2024-11-05', capabilities: {} } : {}; + ws.send(JSON.stringify({ jsonrpc: '2.0', id: message.id, result })); + }); + }); + await new Promise((r) => http.listen(0, '127.0.0.1', () => r())); + cleanups.push(() => new Promise((r) => { for (const s of hung) s.destroy(); wss.close(); http.close(() => r()); })); + const url = `ws://127.0.0.1:${(http.address() as AddressInfo).port}/mcpl`; + + const connection = await McplServerConnection.connectWithReconnect({ id: 'ws', url, ...RECONNECT }, HOST_CAPS); + connection.ready(); + first!.close(); + await until(() => upgrades === 2, 'the reconnect is dialing'); + const began = Date.now(); + await connection.close(); + assert.ok(Date.now() - began < 5_000, 'promptly: the dial was ended, not waited out (15 s)'); + assert.equal(connection.isConnected, false); +}); diff --git a/test/mcpl-legacy-version.test.ts b/test/mcpl-legacy-version.test.ts new file mode 100644 index 000000000..c5f51cfcf --- /dev/null +++ b/test/mcpl-legacy-version.test.ts @@ -0,0 +1,327 @@ +/** + * The legacy engine's protocol boundary and inventory: + * - it offers and accepts exactly MCP 2024-11-05; anything else is a typed + * McplProtocolVersionError carrying what each side said; + * - such a verdict is never retried (no reconnect stub, no backoff loop), and + * at startup it raises the ops alert at once, as it does on reconnect; + * - tools/list follows nextCursor to the complete inventory, and refuses a + * server whose cursors don't terminate. + */ +import { test, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, readFileSync, existsSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +import { McplServerConnection, McplProtocolVersionError, McplRequestError } from '../src/mcpl/server-connection.js'; +import type { McplHostCapabilities, McplServerConfig } from '../src/mcpl/types.js'; +import { AgentFramework } from '../src/framework.js'; +import type { Module, ModuleContext, ToolDefinition, ToolResult, TraceEvent } from '../src/index.js'; + +const here = dirname(fileURLToPath(import.meta.url)); +const SERVER = join(here, 'fixtures', 'legacy-version-server.mjs'); +const HOST_CAPS: McplHostCapabilities = { version: '0.5', pushEvents: true }; + +function config(mode: string, extra: Partial = {}, startLog?: string): McplServerConfig { + return { + id: 'legacy', + command: process.execPath, + args: [SERVER, mode, ...(startLog ? [startLog] : [])], + ...extra, + }; +} + +const cleanups: Array<() => Promise | void> = []; +afterEach(async () => { + while (cleanups.length) await cleanups.pop()!(); +}); + +async function connect(c: McplServerConfig): Promise { + const connection = await McplServerConnection.connect(c, HOST_CAPS); + cleanups.push(() => connection.close()); + connection.ready(); + return connection; +} + +test('a 2024-11-05 answer establishes the connection and records its revision', async () => { + const connection = await connect(config('ok')); + assert.equal(connection.protocolVersion, '2024-11-05'); +}); + +test('-32022 with a supported list is a typed rejection naming the modern remedy', async () => { + await assert.rejects(McplServerConnection.connect(config('reject'), HOST_CAPS), (err: unknown) => { + assert.ok(err instanceof McplProtocolVersionError); + assert.equal(err.kind, 'rejected'); + assert.equal(err.offered, '2024-11-05'); + assert.deepEqual(err.supported, ['2026-07-28']); + assert.equal(err.code, -32022); + assert.deepEqual(err.data, { supported: ['2026-07-28'], requested: '2024-11-05' }); + assert.match(err.message, /does not support MCP 2024-11-05 \(it supports: 2026-07-28\)/); + assert.match(err.message, /protocol: 'modern'/); + // A negotiation verdict, not a request outcome. + assert.ok(!(err instanceof McplRequestError)); + return true; + }); +}); + +test('-32022 without data still rejects, with the server\'s own message', async () => { + await assert.rejects(McplServerConnection.connect(config('reject-nodata'), HOST_CAPS), (err: unknown) => { + assert.ok(err instanceof McplProtocolVersionError); + assert.equal(err.kind, 'rejected'); + assert.equal(err.supported, null); + assert.match(err.message, /Unsupported protocol version: 2024-11-05/); + assert.doesNotMatch(err.message, /protocol: 'modern'/); + return true; + }); +}); + +test('another legacy revision, or none, is a mismatch rather than a silent connection', async () => { + await assert.rejects(McplServerConnection.connect(config('mismatch'), HOST_CAPS), (err: unknown) => { + assert.ok(err instanceof McplProtocolVersionError); + assert.equal(err.kind, 'mismatch'); + assert.equal(err.returned, '2025-06-18'); + assert.match(err.message, /protocolVersion "2025-06-18"; this client's legacy engine speaks only 2024-11-05/); + return true; + }); + await assert.rejects(McplServerConnection.connect(config('missing'), HOST_CAPS), (err: unknown) => { + assert.ok(err instanceof McplProtocolVersionError); + assert.equal(err.kind, 'mismatch'); + assert.equal(err.returned, null); + assert.match(err.message, /protocolVersion \(none\)/); + return true; + }); +}); + +test('a version verdict gets no reconnect stub even with reconnect: true', async () => { + await assert.rejects( + McplServerConnection.connectWithReconnect(config('reject', { reconnect: true, reconnectIntervalMs: 20 }), HOST_CAPS), + (err: unknown) => err instanceof McplProtocolVersionError, + ); +}); + +test('a version verdict on reconnect stops the backoff loop', async () => { + const dir = mkdtempSync(join(tmpdir(), 'legacy-version-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const startLog = join(dir, 'starts.log'); + const connection = await McplServerConnection.connectWithReconnect( + config('ok-then-reject', { reconnect: true, reconnectIntervalMs: 20, reconnectMaxIntervalMs: 40 }, startLog), + HOST_CAPS, + ); + cleanups.push(() => connection.close()); + connection.ready(); + const failures: Array<{ error: string; attempt: number; permanent?: boolean }> = []; + connection.on('reconnect-failed', (info) => failures.push(info)); + + // The first process answers 2024-11-05; make it exit so the loop runs. + await assert.rejects(connection.sendToolsCall('die', {})); + const deadline = Date.now() + 5_000; + while (failures.length === 0 && Date.now() < deadline) await new Promise((r) => setTimeout(r, 20)); + assert.equal(failures.length, 1, 'one failed reconnect'); + assert.equal(failures[0]!.permanent, true); + assert.match(failures[0]!.error, /does not support MCP 2024-11-05/); + assert.equal(connection.willReconnect, false); + + // No further attempts: the start count stays at 2 (original + one retry). + await new Promise((r) => setTimeout(r, 300)); + const starts = existsSync(startLog) ? readFileSync(startLog, 'utf8').split('\n').filter(Boolean).length : 0; + assert.equal(starts, 2); + assert.equal(failures.length, 1); +}); + +test('a version verdict at startup raises the ops alert at once, as on reconnect', async () => { + const dir = mkdtempSync(join(tmpdir(), 'legacy-version-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + // Modules start before the MCP servers connect, so this one sees startup. + const traces: TraceEvent[] = []; + const watcher: Module = { + name: 'trace-watcher', + async start(ctx: ModuleContext) { ctx.onTrace((event) => traces.push(event)); }, + async stop() {}, + getTools: (): ToolDefinition[] => [], + handleToolCall: async (): Promise => ({ success: false, error: 'no tools', isError: true }), + onProcess: async () => ({}), + }; + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: { complete: async () => { throw new Error('not used'); } } as unknown as import('@animalabs/membrane').Membrane, + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [watcher], + mcplServers: [config('missing', { reconnect: true })], + }); + cleanups.push(() => framework.stop()); + const failed = traces.find((t) => t.type === 'mcpl:server-connect-failed') as { serverId?: string; willRetry?: boolean } | undefined; + assert.equal(failed?.serverId, 'legacy'); + assert.equal(failed?.willRetry, false); + const alert = traces.find((t) => t.type === 'ops:alert') as { kind?: string; agentName?: string; message?: string } | undefined; + assert.equal(alert?.kind, 'mcpl-down'); + assert.equal(alert?.agentName, 'legacy'); + assert.match(alert?.message ?? '', /unreachable at startup: .*protocolVersion \(none\)/); +}); + +test('tools/list follows nextCursor to the complete inventory', async () => { + const connection = await connect(config('ok')); + const { tools } = await connection.sendToolsList(); + assert.deepEqual(tools.map((t) => t.name), ['a1', 'a2', 'b1', 'c1', 'die']); +}); + +test('an empty-string cursor is a cursor: only an absent nextCursor ends the inventory', async () => { + const connection = await connect(config('ok', { env: { MODE_LIST: 'empty' } })); + const { tools } = await connection.sendToolsList(); + assert.deepEqual(tools.map((t) => t.name), ['first', 'second']); +}); + +test('a malformed page is an error, never a partial or empty inventory', async () => { + for (const [mode, pattern] of [ + ['notools', /malformed tools\/list page 1: no tools array/], + ['badcursor', /malformed tools\/list page 1: nextCursor must be a string, got 5/], + ] as const) { + const connection = await connect(config('ok', { env: { MODE_LIST: mode } })); + await assert.rejects(connection.sendToolsList(), pattern, mode); + } +}); + +test('a null nextCursor ends the inventory, as an absent one does', async () => { + const connection = await connect(config('ok', { env: { MODE_LIST: 'nullcursor' } })); + const { tools } = await connection.sendToolsList(); + assert.deepEqual(tools.map((t) => t.name), ['x']); +}); + +test('a repeating cursor is an error, not a silently partial inventory', async () => { + const connection = await connect(config('ok', { env: { MODE_LIST: 'loop' } })); + await assert.rejects(connection.sendToolsList(), /repeated tools\/list cursor "same"/); +}); + +test('a modern configuration is refused by the legacy engine before anything is dialed or spawned', async () => { + await assert.rejects( + McplServerConnection.connect({ id: 'modern', url: 'https://example.invalid/mcp' }, HOST_CAPS), + /configured for modern MCP \(2026-07-28\); McplServerConnection speaks the legacy family/, + ); + const dir = mkdtempSync(join(tmpdir(), 'legacy-version-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const startLog = join(dir, 'starts.log'); + await assert.rejects( + McplServerConnection.connect(config('ok', { protocol: 'modern' }, startLog), HOST_CAPS), + /configured for modern MCP/, + ); + await assert.rejects( + McplServerConnection.connectWithReconnect(config('ok', { protocol: 'modern', reconnect: true }, startLog), HOST_CAPS), + /configured for modern MCP/, + ); + assert.equal(existsSync(startLog), false, 'the fixture was never spawned'); + // The same fixture, configured as legacy, does start: the log above can tell. + await connect(config('ok', {}, startLog)); + assert.equal(readFileSync(startLog, 'utf8').split('\n').filter(Boolean).length, 1); +}); + +test('a -32022 whose supported list lacks the modern revision names no modern remedy', async () => { + await assert.rejects(McplServerConnection.connect(config('reject-old'), HOST_CAPS), (err: unknown) => { + assert.ok(err instanceof McplProtocolVersionError); + assert.deepEqual(err.supported, ['2025-03-26']); + assert.match(err.message, /it supports: 2025-03-26/); + assert.doesNotMatch(err.message, /protocol: 'modern'/); + return true; + }); +}); + +function startsIn(startLog: string): string[] { + return existsSync(startLog) ? readFileSync(startLog, 'utf8').split('\n').filter(Boolean) : []; +} + +function alive(pid: number): boolean { + try { process.kill(pid, 0); return true; } catch { return false; } +} + +test('a version verdict is never retried: one launch, and that child is reaped', async () => { + const dir = mkdtempSync(join(tmpdir(), 'legacy-version-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + for (const mode of ['reject', 'mismatch']) { + const startLog = join(dir, `${mode}.log`); + await assert.rejects( + McplServerConnection.connectWithReconnect(config(mode, { reconnect: true, reconnectIntervalMs: 20, reconnectMaxIntervalMs: 40 }, startLog), HOST_CAPS), + (err: unknown) => err instanceof McplProtocolVersionError, + ); + const starts = startsIn(startLog); + assert.equal(starts.length, 1, `${mode}: launched once`); + // The rejection waits for the handshake's own cleanup, so its child is gone. + const pid = Number(starts[0]!.split(' ')[1]); + assert.equal(alive(pid), false, `${mode}: its child was reaped`); + // Past several reconnect intervals, nothing has launched again. + await new Promise((r) => setTimeout(r, 200)); + assert.equal(startsIn(startLog).length, 1, `${mode}: never retried`); + } +}); + +test('an initialize error that is not a version verdict stays a plain, retryable error', async () => { + await assert.rejects(McplServerConnection.connect(config('error'), HOST_CAPS), (err: unknown) => { + assert.ok(err instanceof Error); + assert.ok(!(err instanceof McplProtocolVersionError)); + assert.match(err.message, /initialize failed on purpose/); + return true; + }); + // With reconnect, it leaves a stub that keeps trying, as any transient failure does. + const stub = await McplServerConnection.connectWithReconnect(config('error', { reconnect: true, reconnectIntervalMs: 20 }), HOST_CAPS); + cleanups.push(() => stub.close()); + assert.equal(stub.willReconnect, true); +}); + +test('a modern stdio configuration is refused before anything is spawned', async () => { + const dir = mkdtempSync(join(tmpdir(), 'legacy-version-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const startLog = join(dir, 'starts.log'); + await assert.rejects( + McplServerConnection.connect(config('ok', { protocol: 'modern' }, startLog), HOST_CAPS), + /configured for modern MCP/, + ); + assert.deepEqual(startsIn(startLog), [], 'nothing was spawned'); +}); + +test('a list that never ends is refused after its page bound', async () => { + const connection = await connect(config('ok', { env: { MODE_LIST: 'endless' } })); + await assert.rejects(connection.sendToolsList(), /did not finish within 100 pages/); +}); + +test('a page after the first is checked as the first is', async () => { + const connection = await connect(config('ok', { env: { MODE_LIST: 'badlater' } })); + await assert.rejects(connection.sendToolsList(), /malformed tools\/list page 2: no tools array/); +}); + +async function startupTraces(dir: string, server: McplServerConfig): Promise { + const traces: TraceEvent[] = []; + const watcher: Module = { + name: 'trace-watcher', + async start(ctx: ModuleContext) { ctx.onTrace((event) => traces.push(event)); }, + async stop() {}, + getTools: (): ToolDefinition[] => [], + handleToolCall: async (): Promise => ({ success: false, error: 'no tools', isError: true }), + onProcess: async () => ({}), + }; + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: { complete: async () => { throw new Error('not used'); } } as unknown as import('@animalabs/membrane').Membrane, + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], + modules: [watcher], + mcplServers: [server], + }); + cleanups.push(() => framework.stop().catch(() => {})); + return traces; +} + +test('at startup, willRetry follows reconnect and the verdict, and only a verdict raises the alert', async () => { + const cases: Array<[string, McplServerConfig, boolean, boolean]> = [ + // [label, config, willRetry, alert] + ['verdict without reconnect', config('missing'), false, true], + ['failure with reconnect', config('error', { reconnect: true }), true, false], + ['failure without reconnect', config('error'), false, false], + // Refused before anything launches: nothing will retry it, reconnect or not. + ['configuration refused, with reconnect', { id: 'legacy', url: 'ftp://127.0.0.1/x', reconnect: true }, false, false], + ]; + for (const [label, server, willRetry, alert] of cases) { + const dir = mkdtempSync(join(tmpdir(), 'legacy-version-')); + cleanups.push(() => rmSync(dir, { recursive: true, force: true })); + const traces = await startupTraces(dir, server); + const failed = traces.find((t) => t.type === 'mcpl:server-connect-failed') as { willRetry?: boolean } | undefined; + assert.equal(failed?.willRetry, willRetry, label); + assert.equal(traces.some((t) => t.type === 'ops:alert'), alert, `${label}: alert`); + } +}); diff --git a/test/mcpl-list-status.test.ts b/test/mcpl-list-status.test.ts index 348275257..b55eb57c5 100644 --- a/test/mcpl-list-status.test.ts +++ b/test/mcpl-list-status.test.ts @@ -42,6 +42,11 @@ test('listMcplServers exposes the live grant layers and host-owned authority', ( id: 'discord', connected: true, retrying: false, + family: 'legacy', + // The stub never handshook, so no revision is established. + protocolVersion: null, + transport: 'stdio', + refused: null, toolPrefix: 'mcpl--discord', toolCount: 1, policyEstablished: true, @@ -88,3 +93,41 @@ test('listMcplServers distinguishes a disconnected retrying stub from connected' lastNegotiatedAt: null, }); }); + +test('listMcplServers lists a refused configuration as refused, shows urls redacted, and gives a modern server no MCPL policy', () => { + const framework = Object.create(AgentFramework.prototype) as any; + framework.mcplServerConfigs = new Map>([ + ['both', { id: 'both', command: 'node', url: 'wss://u:pw@x.example/mcpl?token=sekret' }], + ['slow', { id: 'slow', url: 'https://x.example/mcp', requestTimeoutMs: 0 }], + ['docs', { id: 'docs', url: 'https://mcp.example/mcp?api_key=sekret®ion=eu' }], + ['chat', { id: 'chat', url: 'wss://user:pw@chat.example/mcpl' }], + ['zap', { id: 'zap', url: 'https://mcp.zapier.com/api/mcp/s/sekretZt4kQ9xW2vLm8/mcp' }], + ]); + framework.mcplServerRegistry = { getServer: () => null }; + framework.mcplTools = []; + framework.mcplPrefixMap = new Map(); + const listed = (framework as AgentFramework).listMcplServers(); + const byId = new Map(listed.map((s) => [s.id, s])); + + // No usable transport: no family or transport is guessed. + const both = byId.get('both')!; + assert.equal(both.family, null); + assert.equal(both.transport, null); + assert.match(both.refused!, /"command" and "url" are both set/); + assert.equal(both.url, 'wss://x.example/mcpl?token=redacted'); + // A binding that resolves, refused for another reason. + const slow = byId.get('slow')!; + assert.equal(slow.family, 'modern'); + assert.equal(slow.transport, 'http'); + assert.match(slow.refused!, /requestTimeoutMs must be an integer from 1/); + // Usable: nothing refused. MCPL policy doesn't apply to a modern server. + const docs = byId.get('docs')!; + assert.deepEqual([docs.family, docs.transport, docs.refused, docs.policyEstablished], ['modern', 'http', null, null]); + assert.equal(docs.url, 'https://mcp.example/mcp?api_key=redacted®ion=redacted'); + const chat = byId.get('chat')!; + assert.deepEqual([chat.family, chat.transport, chat.refused, chat.policyEstablished], ['legacy', 'websocket', null, false]); + assert.equal(chat.url, 'wss://chat.example/mcpl'); + // A per-user secret in the path goes too. + assert.equal(byId.get('zap')!.url, 'https://mcp.zapier.com/api/mcp/s/redacted/mcp'); + assert.ok(!/sekret|pw@/.test(JSON.stringify(listed)), 'no credential in the status'); +}); diff --git a/test/mcpl-manifest-changed.test.ts b/test/mcpl-manifest-changed.test.ts index eeffc5383..c3fc78c3f 100644 --- a/test/mcpl-manifest-changed.test.ts +++ b/test/mcpl-manifest-changed.test.ts @@ -103,6 +103,13 @@ function makeHarness() { const settle = () => new Promise((r) => setImmediate(r)); +test('a manifest change from a server with no configuration (removed) fetches nothing', async () => { + const { fw, connection } = makeHarness(); + (fw as unknown as { mcplServerConfigs: Map }).mcplServerConfigs = new Map(); + await fw.handleManifestChanged(connection, { revision: 'sha256:gone', domains: ['capabilities'] }); + assert.deepEqual(connection.events.map((e) => e.kind), [], 'no fetch, grant or policy request'); +}); + test('reduction applies BEFORE the policy Request; receipt names it applied', async () => { const { fw, connection, traces } = makeHarness(); // New manifest drops channels.publish (and the chat set with it, since diff --git a/test/mcpl-reconnect-reregistration.test.ts b/test/mcpl-reconnect-reregistration.test.ts index 6a03af945..912479b28 100644 --- a/test/mcpl-reconnect-reregistration.test.ts +++ b/test/mcpl-reconnect-reregistration.test.ts @@ -107,8 +107,6 @@ async function makeHarness() { fw.mcplTools = []; fw.mcplToolRefreshInFlight = false; fw.mcplToolRefreshPending = false; - // A disconnect reads the servers whose connect is in flight. - fw.mcplConnecting = new Set(); fw.channelRegistry = null; fw.inferenceRouter = null; fw.eventGate = null; @@ -137,7 +135,7 @@ async function makeHarness() { }; fw.discordAwarenessBarrier = null; fw.discordAwarenessBarrierGeneration = 0; - fw.wireMcplEvents(connection); + fw.wireMcplEvents(connection, config); await fw.registerMcplServerFeatures(config, connection); // The push handler responds synchronously when no awareness barrier is diff --git a/test/mcpl-reference-fetcher.test.ts b/test/mcpl-reference-fetcher.test.ts index dbbb930f4..952b2e45e 100644 --- a/test/mcpl-reference-fetcher.test.ts +++ b/test/mcpl-reference-fetcher.test.ts @@ -355,10 +355,11 @@ test('a copy saved earlier is still served once its expiry has passed or reads a async function eagerFetch(content: unknown[], fetcher: ReferenceFetcher = makeFetcher({ srv: { url: ORIGIN } })): Promise { const framework = Object.create(AgentFramework.prototype) as { getReferenceFetcher(): ReferenceFetcher; - autoFetchReferences(content: unknown[], serverId: string, eager: boolean): Promise; + autoFetchReferences(content: unknown[], serverId: string, forScript: boolean): Promise; }; framework.getReferenceFetcher = () => fetcher; - await framework.autoFetchReferences(content, 'srv', true); + // The eager pass alone, as for the model's call: a script's demand would fetch the rest. + await framework.autoFetchReferences(content, 'srv', false); } test('the eager default skips expired references, so the live ones after them get its slots', async (t) => { diff --git a/test/mcpl-reference-fields.test.ts b/test/mcpl-reference-fields.test.ts index 265962122..73a11bcab 100644 --- a/test/mcpl-reference-fields.test.ts +++ b/test/mcpl-reference-fields.test.ts @@ -1,8 +1,8 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; import { - buildReferenceStub, classifyBlock, isReferenceExpired, sanitizeLabel, REFERENCE_LIMITS, - referenceRegistry, referenceStubOrNull, UNBOUND_REASON, unfetchableReason, + buildReferenceStub, classifyBlock, isReferenceExpired, sanitizeLabel, EXPIRY_REASON, REFERENCE_LIMITS, + referenceBlockForScript, referenceRegistry, referenceStubOrNull, UNBOUND_REASON, unfetchableReason, } from '../src/mcpl/references.js'; const fields = ['name', 'mimeType', 'expiresAt'] as const; @@ -116,6 +116,29 @@ test('the stub leaves out a type, or a provenance, that sanitizes to nothing', ( assert.equal(originStub, `[${referenceRegistry.findByUri(originGone.uri)!.refId}] image/png — fetch with fetch_reference`); }); +test("an expired reference reaches a script with the reason its stub gives, and a live or saved one without it", () => { + // Each bound to its server, as a script's tool result binds them. + const cases = [['2001-01-01T00:00:00Z', EXPIRY_REASON.passed], ['not-a-date', EXPIRY_REASON.unparseable], + ['x'.repeat(REFERENCE_LIMITS.expiresAt + 1), EXPIRY_REASON.rejected]] as const; + for (const [i, [expiresAt, why]] of cases.entries()) { + const { block, testimony } = reference('resource', { uri: `https://example.test/script-expired-${i}`, expiresAt }); + const view = referenceBlockForScript(block, 'srv') as { savedTo: unknown; expired?: string }; + assert.equal(view.savedTo, null); + assert.equal(view.expired, why); + assert.ok(buildReferenceStub(testimony, undefined, 'srv').includes(why), 'the stub gives the same reason'); + } + const live = reference('resource', { uri: 'https://example.test/script-live', expiresAt: '2999-01-01T00:00:00Z' }).block; + assert.equal((referenceBlockForScript(live, 'srv') as { expired?: string }).expired, undefined); + // Bound to its server, so the fetch its id offers can be made. + assert.deepEqual(referenceRegistry.findByUri(live.uri)?.serverIds, ['srv']); + // A copy saved before its expiry passed is still served, and the script gets its path. + const saved = reference('resource', { uri: 'https://example.test/script-expired-0', expiresAt: '2001-01-01T00:00:00Z' }).block; + referenceRegistry.findByUri(saved.uri)!.fetchedPath = 'scratch/refs/kept.png'; + const view = referenceBlockForScript(saved, 'srv') as { savedTo: unknown; expired?: string }; + assert.equal(view.savedTo, 'scratch/refs/kept.png'); + assert.equal(view.expired, undefined); +}); + test("an expired reference's stub says why, where a live one's advises fetch_reference", () => { // Each bound to its server, as a tool result's references are, so the expiry is what's left to say. const cases = [['2001-01-01T00:00:00Z', /[\]—] expired$/], ['not-a-date', /[\]—] treated as expired, since its expiry time can't be read$/], diff --git a/test/mcpl-request-handler-errors.test.ts b/test/mcpl-request-handler-errors.test.ts index a7c3e0693..dfcb72fd2 100644 --- a/test/mcpl-request-handler-errors.test.ts +++ b/test/mcpl-request-handler-errors.test.ts @@ -19,7 +19,7 @@ function harness(connection = Object.assign(new EventEmitter(), { id: 'srv' })) // Main's trace listeners are registrations (#318), which onTrace makes. framework.traceListeners = []; framework.onTrace((event: unknown) => traces.push(event)); - framework.wireMcplEvents(connection); + framework.wireMcplEvents(connection, { id: connection.id }); return { framework, connection, traces }; } @@ -678,7 +678,16 @@ test('a manifest refresh deferred by the fetch floor reports a failure other tha t.mock.method(console, 'error', (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }); // The last refresh completed just now, so this announcement waits out the floor. framework.manifestRefreshState = new Map([['srv', { inFlight: false, queued: false, lastCompletedAt: 100_000 }]]); - framework.mcplServerConfigs = { get() { throw new Error('the deferred refresh failed before its fetch'); } }; + // Each run reads the server's configuration before anything else: the + // announcement's own run finds it, and the deferred run fails there. + let lookups = 0; + framework.mcplServerConfigs = { + get() { + lookups++; + if (lookups > 1) throw new Error('the deferred refresh failed before its fetch'); + return { id: 'srv' }; + }, + }; connection.emit('manifest-changed', { revision: 'r2' }); await settle(); assert.deepEqual(failureTraces(traces), [], 'nothing has run yet'); @@ -698,11 +707,13 @@ test('a manifest refresh queued behind a fetch in flight reports a failure other const lines: string[] = []; t.mock.method(console, 'error', (...args: unknown[]) => { lines.push(args.map(String).join(' ')); }); framework.manifestRefreshState = new Map(); + // Each run reads the server's configuration before anything else: both + // announcements' own runs find it, and the queued re-run fails there. let lookups = 0; framework.mcplServerConfigs = { get() { lookups++; - if (lookups > 1) throw new Error('the queued refresh failed before its fetch'); + if (lookups > 2) throw new Error('the queued refresh failed before its fetch'); return { id: 'srv' }; }, }; @@ -721,7 +732,7 @@ test('a manifest refresh queued behind a fetch in flight reports a failure other assert.equal(failures.length, 1); assert.equal(failures[0].event, 'manifest-changed'); assert.match(failures[0].error, /the queued refresh failed before its fetch/); - assert.equal(lookups, 2); + assert.equal(lookups, 3); assert.ok(lines.includes('[mcpl] srv manifest-changed handler failed: "the queued refresh failed before its fetch"'), lines.join('\n')); }); diff --git a/test/mcpl-request-outcome.test.ts b/test/mcpl-request-outcome.test.ts index 27b2a45fd..43be23748 100644 --- a/test/mcpl-request-outcome.test.ts +++ b/test/mcpl-request-outcome.test.ts @@ -27,7 +27,7 @@ process.stdin.on('data', (c) => { if (!line.trim()) continue; let m; try { m = JSON.parse(line); } catch { continue; } const out = (o) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: m.id, ...o }) + '\\n'); - if (m.method === 'initialize') out({ result: { capabilities: {} } }); + if (m.method === 'initialize') out({ result: { protocolVersion: '2024-11-05', capabilities: {} } }); else if (m.method === 'tools/call' && m.params.name === 'fail') { out({ error: { code: -32001, message: 'POSTED: part 1 of 2', data: { posted: ['111'] } } }); } else if (m.method === 'tools/call' && m.params.name === 'exit') { @@ -79,8 +79,11 @@ test('a connection lost while awaiting is no-response', async () => { test('closing the connection while a request awaits its answer rejects it as no-response', async () => { connection = await McplServerConnection.connect(config({ requestTimeoutMs: 10_000 }), HOST_CAPS); const pending = connection.sendToolsCall('stuck', {}); + // The request fails as the close begins, and the close resolves once the + // child has exited, so the rejection is observed before the close is awaited. + const rejected = rejectsWith(pending, 'no-response'); await connection.close(); - await rejectsWith(pending, 'no-response'); + await rejected; connection = null; }); diff --git a/test/mcpl-request-timeout.test.ts b/test/mcpl-request-timeout.test.ts index 0bd581bfc..f308aab77 100644 --- a/test/mcpl-request-timeout.test.ts +++ b/test/mcpl-request-timeout.test.ts @@ -29,7 +29,7 @@ process.stdin.on('data', (c) => { if (!line.trim()) continue; let m; try { m = JSON.parse(line); } catch { continue; } const reply = (result, id = m.id) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id, result }) + '\\n'); - if (m.method === 'initialize') reply({ capabilities: {} }); + if (m.method === 'initialize') reply({ protocolVersion: '2024-11-05', capabilities: {} }); else if (m.method === 'tools/list') { const released = held !== null; if (held) { reply(held.result, held.id); held = null; } diff --git a/test/mcpl-runtime-lifecycle.test.ts b/test/mcpl-runtime-lifecycle.test.ts index f3ad6c9c4..9f0f5cb6f 100644 --- a/test/mcpl-runtime-lifecycle.test.ts +++ b/test/mcpl-runtime-lifecycle.test.ts @@ -14,12 +14,24 @@ import { WebSocketServer } from 'ws'; import type { AddressInfo } from 'node:net'; import { AgentFramework } from '../src/index.js'; +import type { EventResponse, Module, ModuleContext, ToolDefinition, ToolResult } from '../src/index.js'; import { MockMembrane } from './helpers/mock-membrane.js'; +/** A module with no tools: only a name, for a server's tool prefix to collide with. */ +class NamedModule implements Module { + constructor(readonly name: string) {} + async start(_ctx: ModuleContext): Promise {} + async stop(): Promise {} + async onProcess(): Promise { return {}; } + getTools(): ToolDefinition[] { return []; } + async handleToolCall(): Promise { return { success: true }; } +} + // A tiny stdio MCPL server (same shape as mcpl-ws-transport.test.ts): reads // NDJSON on stdin, answers initialize / tools/list / tools/call on stdout. const STDIO_SERVER = ` let buf = ''; +const answered = {}; process.stdin.on('data', (c) => { buf += c.toString('utf8'); let i; @@ -27,14 +39,39 @@ process.stdin.on('data', (c) => { const line = buf.slice(0, i); buf = buf.slice(i + 1); if (!line.trim()) continue; let m; try { m = JSON.parse(line); } catch { continue; } + // Requests a test's server never answers, named in SILENT_ON, and one it + // answers a number of times and then never again (SILENT_AFTER=method:n). + if ((process.env.SILENT_ON || '').split(',').includes(m.method)) continue; + const [afterMethod, afterCount] = (process.env.SILENT_AFTER || ':').split(':'); + if (m.method === afterMethod) { + answered[m.method] = (answered[m.method] || 0) + 1; + if (answered[m.method] > Number(afterCount)) continue; + } const reply = (result) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: m.id, result }) + '\\n'); - if (m.method === 'initialize') reply({ capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); + if (m.method === 'initialize') reply({ protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); else if (m.method === 'tools/list') reply({ tools: [{ name: 'echo', description: 'e', inputSchema: { type: 'object' } }] }); else if (m.method === 'tools/call') reply({ content: [{ type: 'text', text: String((m.params && m.params.arguments && m.params.arguments.text) || '') }] }); + // Any other request is answered as unknown, so the host's wait for it ends + // at once rather than at its timeout. + else if (m.id !== undefined) process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: m.id, error: { code: -32601, message: 'method not found' } }) + '\\n'); } }); `; +/** The framework's inner disconnect, which a restart's own teardown calls + * (the public disconnectMcplServer is an operator operation around it). */ +type Inner = { runDisconnectMcplServer(id: string): Promise }; +type Internals = { + legacyConnectAborts(): Map; + mcplServerRegistry: { getServer(id: string): { sendToolsList(): Promise } | null } | null; +}; + +async function until(cond: () => boolean, what: string, ms = 10_000): Promise { + const deadline = Date.now() + ms; + while (!cond() && Date.now() < deadline) await new Promise((resolve) => setTimeout(resolve, 20)); + assert.ok(cond(), `in time: ${what}`); +} + function serverConfig(id: string, extra?: Record) { return { id, command: process.execPath, args: ['-e', STDIO_SERVER], ...extra }; } @@ -102,7 +139,7 @@ test('restartMcplServer respawns the server and keeps tools available', async () assert.equal(servers.length, 1); assert.equal(servers[0].connected, true); assert.equal(servers[0].toolCount, 1); - assert.ok(framework.getAllTools().some(t => t.name === 'mcpl--first--echo')); + assert.ok(framework.getAllTools().some(tool => tool.name === 'mcpl--first--echo')); }); }); @@ -116,6 +153,429 @@ test('restartMcplServer accepts an updated config (new toolPrefix)', async () => }); }); +test("restartMcplServer refuses what the connect would refuse before tearing anything down: another server's prefix or id, or a module's name", async () => { + await withFramework(true, async (framework) => { + await framework.connectMcplServer(serverConfig('second') as never); + await framework.addModule(new NamedModule('notes')); + const untouched = () => { + const live = framework.listMcplServers().filter(s => s.connected).map(s => s.id).sort(); + assert.deepEqual(live, ['first', 'second'], 'both servers still connected'); + const tools = framework.getAllTools().map(t => t.name); + assert.ok(tools.includes('mcpl--first--echo') && tools.includes('mcpl--second--echo'), 'and their tools still listed'); + }; + await assert.rejects( + framework.restartMcplServer('first', serverConfig('first', { toolPrefix: 'mcpl--second' }) as never), + /collides with server "second"/, + ); + untouched(); + await assert.rejects( + framework.restartMcplServer('first', serverConfig('first', { toolPrefix: 'notes' }) as never), + /collides with module "notes"/, + ); + untouched(); + // Under another live server's id, whose prefix is that server's own, so + // only the id is taken. + await assert.rejects( + framework.restartMcplServer('first', serverConfig('second') as never), + /MCPL server "second" is already registered/, + ); + untouched(); + }); +}); + +test('restartMcplServer under a new id may keep the tool prefix the restarted server frees', async () => { + await withFramework(true, async (framework) => { + await framework.restartMcplServer('first', serverConfig('renamed', { toolPrefix: 'mcpl--first' }) as never); + const servers = framework.listMcplServers(); + assert.deepEqual(servers.map(s => [s.id, s.connected]), [['renamed', true]], 'first was replaced'); + assert.ok(framework.getAllTools().some(t => t.name === 'mcpl--first--echo'), 'and the prefix it freed serves the new id'); + }); +}); + +test('restartMcplServer refuses once stop() has begun, before it disconnects anything', async (t) => { + await withFramework(true, async (framework) => { + const disconnect = t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer'); + const stopping = framework.stop(); + await assert.rejects( + framework.restartMcplServer('first'), + /restart MCPL server refused: the framework is stopping/, + ); + await stopping; + assert.equal(disconnect.mock.callCount(), 0, 'the restart disconnected nothing'); + }); +}); + +test('restartMcplServer of an id whose disconnect is in flight is refused before it tears anything down', async (t) => { + await withFramework(true, async (framework) => { + const disconnecting = framework.disconnectMcplServer('first'); + const disconnect = t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer'); + await assert.rejects( + framework.restartMcplServer('first'), + /MCP server "first" was not restarted: it is being disconnected; restart it once that has finished/, + ); + assert.equal(disconnect.mock.callCount(), 0, 'the restart started no disconnect of its own'); + disconnect.mock.restore(); + await disconnecting; + }); +}); + +test("a disconnect removes its server's prefix claim, and leaves one another server holds", async () => { + await withFramework(true, async (framework) => { + const internals = framework as unknown as { mcplPrefixMap: Map }; + assert.equal(internals.mcplPrefixMap.get('mcpl--first'), 'first'); + await framework.disconnectMcplServer('first'); + assert.equal(internals.mcplPrefixMap.has('mcpl--first'), false, 'its own claim is released'); + // As after a restart under a new id: `second` holds the prefix `first` had. + await framework.connectMcplServer(serverConfig('first') as never); + internals.mcplPrefixMap.set('mcpl--first', 'second'); + await framework.disconnectMcplServer('first'); + assert.equal(internals.mcplPrefixMap.get('mcpl--first'), 'second', "the other server's claim stands"); + }); +}); + +test('a disconnect releases the prefix its server claimed at connect, though its configuration changed since', async () => { + await withFramework(false, async (framework) => { + const config = serverConfig('dyn', { toolPrefix: 'p1' }); + await framework.connectMcplServer(config as never); + // Against the documented rule, the host changes the configuration it submitted. + (config as unknown as { toolPrefix: string }).toolPrefix = 'p2'; + await framework.disconnectMcplServer('dyn'); + await framework.connectMcplServer(serverConfig('other', { toolPrefix: 'p1' }) as never); + assert.deepEqual(framework.listMcplServers().map(s => s.id), ['other']); + }); +}); + +test('restartMcplServer of an id whose connect is in flight is refused before it tears anything down', async (t) => { + // MCPL is up, so the connect holds its id before its first await. + await withFramework(true, async (framework) => { + const connecting = framework.connectMcplServer(serverConfig('later') as never); + const disconnect = t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer'); + await assert.rejects( + framework.restartMcplServer('later', serverConfig('renamed') as never), + /MCP server "later" was not restarted: it is being connected; restart it once that has finished/, + ); + assert.equal(disconnect.mock.callCount(), 0, 'the restart started no disconnect of its own'); + disconnect.mock.restore(); + await connecting; + }); +}); + +test('a connect started during a restart is refused the tool prefix the restart moves to, and the restart completes', async () => { + await withFramework(true, async (framework) => { + const restarting = framework.restartMcplServer('first', serverConfig('first', { toolPrefix: 'xx' }) as never); + await assert.rejects( + framework.connectMcplServer(serverConfig('third', { toolPrefix: 'xx' }) as never), + /MCPL server "third" toolPrefix "xx" is held by the restart of "first"/, + ); + await restarting; + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['first', true]]); + assert.ok(framework.getAllTools().some(t => t.name === 'xx--echo'), 'the restarted server serves the prefix'); + }); +}); + +test('a connect started during a restart is refused the id the restart moves to, and the restart completes', async () => { + await withFramework(true, async (framework) => { + const restarting = framework.restartMcplServer('first', serverConfig('second') as never); + // Another prefix, so only the id is contested. + await assert.rejects( + framework.connectMcplServer(serverConfig('second', { toolPrefix: 'other' }) as never), + /MCP server "second" is held by the restart of "first"/, + ); + await restarting; + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['second', true]]); + assert.ok(framework.getAllTools().some(t => t.name === 'mcpl--second--echo')); + }); +}); + +test("a connect between a restart's disconnect and its connect is refused the id the restart keeps", async (t) => { + await withFramework(true, async (framework) => { + const disconnect = (framework as unknown as Inner).runDisconnectMcplServer.bind(framework); + let racing: Promise | undefined; + t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer', async (id: string) => { + await disconnect(id); + // The old server is gone and its disconnect holds nothing now. + racing = framework.connectMcplServer(serverConfig('first', { toolPrefix: 'other' }) as never) + .then(() => 'connected', (error: unknown) => error); + }); + await framework.restartMcplServer('first'); + const outcome = await racing; + assert.ok(outcome instanceof Error, `the racing connect was refused, not ${String(outcome)}`); + assert.match(outcome.message, /MCP server "first" is held by the restart of "first"/); + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['first', true]]); + assert.ok(framework.getAllTools().some(tool => tool.name === 'mcpl--first--echo')); + }); +}); + +test('a restart of an id another restart holds is refused before it tears anything down', async (t) => { + await withFramework(true, async (framework) => { + await framework.connectMcplServer(serverConfig('second') as never); + const disconnect = (framework as unknown as Inner).runDisconnectMcplServer.bind(framework); + let during: (() => Promise) | undefined; + const refusals: string[] = []; + t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer', async (id: string) => { + await disconnect(id); + // Between the restart's disconnect and its connect, only its claims hold these ids. + const attempt = during; + during = undefined; + if (attempt) { + await attempt().then(() => refusals.push('restarted'), (error: unknown) => refusals.push((error as Error).message)); + } + }); + // A restart under the same id holds it. + during = () => framework.restartMcplServer('first', serverConfig('first') as never); + await framework.restartMcplServer('first'); + // A restart under a new id holds the new one. + during = () => framework.restartMcplServer('renamed', serverConfig('renamed') as never); + await framework.restartMcplServer('second', serverConfig('renamed') as never); + assert.deepEqual(refusals, [ + 'MCP server "first" was not restarted: it is being restarted; restart it once that has finished', + 'MCP server "renamed" was not restarted: the restart of "second" holds its id; restart it once that has finished', + ]); + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]).sort(), [['first', true], ['renamed', true]]); + }); +}); + +test('a module is refused a name a server holds as its tool prefix, or a restart is moving its server to', async () => { + await withFramework(true, async (framework) => { + await assert.rejects( + framework.addModule(new NamedModule('mcpl--first')), + /Module "mcpl--first" collides with MCPL server "first"'s toolPrefix "mcpl--first"/, + ); + const restarting = framework.restartMcplServer('first', serverConfig('first', { toolPrefix: 'xx' }) as never); + await assert.rejects( + framework.addModule(new NamedModule('xx')), + /Module "xx" collides with the toolPrefix "xx" the restart of "first" holds/, + ); + await restarting; + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['first', true]]); + assert.ok(framework.getAllTools().some(t => t.name === 'xx--echo'), 'the restarted server serves the prefix'); + }); +}); + +test('a restart that fails releases the id and the tool prefix it held', async (t) => { + await withFramework(true, async (framework) => { + t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer', async () => { + throw new Error('disconnect failed for the test'); + }); + await assert.rejects( + framework.restartMcplServer('first', serverConfig('renamed', { toolPrefix: 'xx' }) as never), + /disconnect failed for the test/, + ); + t.mock.restoreAll(); + await framework.connectMcplServer(serverConfig('renamed', { toolPrefix: 'xx' }) as never); + assert.deepEqual( + framework.listMcplServers().map(s => [s.id, s.connected]).sort(), + [['first', true], ['renamed', true]], + ); + }); +}); + +test('a restart moving to an id or a tool prefix another restart holds is refused before it tears anything down', async (t) => { + await withFramework(true, async (framework) => { + await framework.connectMcplServer(serverConfig('second') as never); + const disconnect = (framework as unknown as Inner).runDisconnectMcplServer.bind(framework); + const refusals: string[] = []; + t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer', async (id: string) => { + await disconnect(id); + if (id !== 'first') return; + // Between first's disconnect and its connect, only its claims hold "moved" and "xx". + for (const attempt of [ + () => framework.restartMcplServer('second', serverConfig('moved', { toolPrefix: 'other' }) as never), + () => framework.restartMcplServer('second', serverConfig('second', { toolPrefix: 'xx' }) as never), + ]) { + await attempt().then(() => refusals.push('restarted'), (error: unknown) => refusals.push((error as Error).message)); + } + }); + await framework.restartMcplServer('first', serverConfig('moved', { toolPrefix: 'xx' }) as never); + assert.deepEqual(refusals, [ + 'MCP server "second" was not restarted: MCP server "moved" is held by the restart of "first"; restart "second" once that has finished', + 'MCPL server "second" toolPrefix "xx" is held by the restart of "first"', + ]); + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]).sort(), [['moved', true], ['second', true]]); + const tools = framework.getAllTools().map(tool => tool.name); + assert.ok(tools.includes('xx--echo') && tools.includes('mcpl--second--echo'), 'each serves its own prefix'); + }); +}); + +test('a restart whose connect fails leaves the server down, configured and not connected, and releases its holds', async () => { + await withFramework(true, async (framework) => { + const exits = { id: 'renamed', command: process.execPath, args: ['-e', 'process.exit(3)'], toolPrefix: 'xx' }; + await assert.rejects( + framework.restartMcplServer('first', exits as never), + /^Error: MCP server "first" is down: it was disconnected for its restart, and the restart's connect failed: /, + ); + // As any failed connect leaves its server: listed as not connected, holding its prefix. + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['renamed', false]]); + await assert.rejects( + framework.addModule(new NamedModule('xx')), + /Module "xx" collides with MCPL server "renamed"'s toolPrefix "xx"/, + ); + // The restart's holds are released, so the id and the prefix take a working configuration. + await framework.restartMcplServer('renamed', serverConfig('renamed', { toolPrefix: 'xx' }) as never); + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['renamed', true]]); + assert.ok(framework.getAllTools().some(tool => tool.name === 'xx--echo')); + }); +}); + +for (const when of ['teardown', 'gap', 'connect'] as const) { + test(`a disconnect that begins during a restart's ${when} ends it: the server stays disconnected and the holds are released`, async (t) => { + await withFramework(true, async (framework) => { + const disconnect = (framework as unknown as Inner).runDisconnectMcplServer.bind(framework); + let host: Promise | undefined; + // The host's disconnect, an operator operation of its own. + const hostDisconnect = () => { + host = framework.disconnectMcplServer('first').then(() => 'disconnected', (error: unknown) => error); + }; + let restarts = true; + t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer', async (id: string) => { + // The host's disconnect goes through as it is. + if (!restarts) return disconnect(id); + restarts = false; + // The restart's own disconnect. + const own = disconnect(id); + if (when === 'teardown') hostDisconnect(); + await own; + if (when === 'gap') hostDisconnect(); + // The restart's connect takes the id within this turn, before an immediate runs. + if (when === 'connect') setImmediate(hostDisconnect); + }); + const refused = await framework.restartMcplServer('first').then(() => null, (error: unknown) => error as Error); + assert.match(refused?.message ?? 'restarted', /^MCP server "first" is down: it was disconnected for its restart, and the restart's connect failed: /); + if (when !== 'connect') { + // Its connect is refused at once, naming the disconnect. + assert.match(refused!.message, /: MCP server "first" is being disconnected, which ends the restart of "first"$/); + } else { + // Its connect, under way, is ended by the disconnect, in whichever + // phase the disconnect finds it. + assert.match(refused!.message, /: MCP server "first" (was not connected: it was disconnected (while connecting|or lost while its tools were being listed)|is being disconnected, which ends the restart of "first")$/); + } + assert.equal(await host, 'disconnected', 'the disconnect completed'); + assert.deepEqual(framework.listMcplServers(), [], 'nothing is left of the server'); + t.mock.restoreAll(); + // Nothing holds the id or its prefix now. + await framework.connectMcplServer(serverConfig('first') as never); + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['first', true]]); + }); + }); +} + +test("a stop() that begins in a restart's gap ends it, and the restart says the server is down", async (t) => { + await withFramework(true, async (framework) => { + const disconnect = (framework as unknown as Inner).runDisconnectMcplServer.bind(framework); + let stopping: Promise | undefined; + t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer', async (id: string) => { + await disconnect(id); + stopping = framework.stop(); + }); + await assert.rejects( + framework.restartMcplServer('first'), + /^Error: MCP server "first" is down: it was disconnected for its restart, and the restart's connect failed: MCP server "first" was not connected: the framework is stopping or stopped$/, + ); + await stopping; + }); +}); + +test("a disconnect of the id a restart moves to ends the restart, even one that finishes before the restart's connect", async (t) => { + await withFramework(true, async (framework) => { + const disconnect = (framework as unknown as Inner).runDisconnectMcplServer.bind(framework); + t.mock.method(framework as unknown as Inner, 'runDisconnectMcplServer', async (id: string) => { + await disconnect(id); + // In the restart's gap, the id it moves to is disconnected, and that + // disconnect is over before the restart's connect begins. + if (id === 'first') await framework.disconnectMcplServer('second'); + }); + await assert.rejects( + framework.restartMcplServer('first', serverConfig('second') as never), + /^Error: MCP server "first" is down: it was disconnected for its restart, and the restart's connect failed: MCP server "second" was disconnected, which ends the restart of "first"$/, + ); + assert.deepEqual(framework.listMcplServers(), [], 'nothing was connected under the new id'); + assert.ok(!framework.getAllTools().some(tool => tool.name.startsWith('mcpl--second--'))); + // The id and the prefix are free again. + await framework.connectMcplServer(serverConfig('second') as never); + assert.deepEqual(framework.listMcplServers().map(s => [s.id, s.connected]), [['second', true]]); + }); +}); + +/** A stdio server that reads its input and never answers, so a connect waits in its handshake. */ +const SILENT = { id: 'silent', command: process.execPath, args: ['-e', 'process.stdin.resume()'] }; + +for (const ending of ['stop()', 'a disconnect'] as const) { + test(`${ending} ends a legacy connect stuck in its handshake at once`, async () => { + await withFramework(true, async (framework) => { + const connecting = framework.connectMcplServer(SILENT as never).then(() => null, (error: unknown) => error as Error); + // The connect is in its handshake, which never finishes. + await until(() => (framework as unknown as Internals).legacyConnectAborts().has('silent'), 'the connect in its handshake'); + const started = Date.now(); + if (ending === 'stop()') await framework.stop(); + else await framework.disconnectMcplServer('silent'); + const took = Date.now() - started; + assert.ok(took < 10_000, `${ending} took ${took} ms, as if it waited out the handshake`); + const refused = await connecting; + assert.match(refused?.message ?? 'connected', ending === 'stop()' + ? /^MCP server "silent" was not connected: the framework stopped while it was connecting$/ + : /^MCP server "silent" was not connected: it was disconnected while connecting$/); + if (ending === 'a disconnect') assert.ok(!framework.listMcplServers().some(s => s.id === 'silent'), 'nothing is left of it'); + }); + }); +} + +for (const silentOn of ['featureSets/update', 'tools/list'] as const) { + test(`stop() ends a legacy connect past its handshake whose server never answers ${silentOn}, with no request timeout`, async () => { + await withFramework(true, async (framework) => { + const config = serverConfig('late', { env: { SILENT_ON: silentOn }, requestTimeoutMs: 0 }); + const connecting = framework.connectMcplServer(config as never).then(() => null, (error: unknown) => error as Error); + // Past the handshake, registered, waiting for the answer that never comes. + await until(() => !!(framework as unknown as Internals).mcplServerRegistry?.getServer('late'), 'the connect past its handshake'); + const started = Date.now(); + await framework.stop(); + const took = Date.now() - started; + assert.ok(took < 10_000, `stop() took ${took} ms, as if it waited on the server`); + const refused = await connecting; + assert.match(refused?.message ?? 'connected', /^MCP server "late" was not connected: the framework stopped while it was connecting$/); + }); + }); +} + +test("stop() ends a connect whose tool listing waits on another server that stopped answering", async () => { + await withFramework(false, async (framework) => { + // Lists its tools once, at its own connect, and never again. + await framework.connectMcplServer(serverConfig('quiet', { env: { SILENT_AFTER: 'tools/list:1' }, requestTimeoutMs: 0 }) as never); + const quiet = (framework as unknown as Internals).mcplServerRegistry!.getServer('quiet')!; + const realList = quiet.sendToolsList.bind(quiet); + let asked = false; + quiet.sendToolsList = () => { asked = true; return realList(); }; + const connecting = framework.connectMcplServer(serverConfig('late') as never).then(() => null, (error: unknown) => error as Error); + // The new connect is listing every server's tools, and waits on "quiet". + await until(() => asked, "the new connect's listing asked quiet"); + const started = Date.now(); + await framework.stop(); + const took = Date.now() - started; + assert.ok(took < 10_000, `stop() took ${took} ms, as if it waited on the other server`); + const refused = await connecting; + assert.match(refused?.message ?? 'connected', /^MCP server "late" was not connected: the framework stopped while it was connecting$/); + }); +}); + +test("stop() ends a WebSocket dial still waiting for its access provider", async () => { + await withFramework(true, async (framework) => { + let asked = false; + const waiting = { + id: 'dial', + url: 'ws://127.0.0.1:9/mcp', + // A provider that never settles. + accessProvider: () => { asked = true; return new Promise(() => {}); }, + }; + const connecting = framework.connectMcplServer(waiting as never).then(() => null, (error: unknown) => error as Error); + for (let i = 0; i < 50 && !asked; i++) await new Promise((resolve) => setTimeout(resolve, 10)); + assert.ok(asked, 'the dial is waiting for its provider'); + const started = Date.now(); + await framework.stop(); + assert.ok(Date.now() - started < 10_000, 'stop() did not wait for the provider'); + const refused = await connecting; + assert.match(refused?.message ?? 'connected', /^MCP server "dial" was not connected: the framework stopped while it was connecting$/); + }); +}); + test('connectMcplServer rejects a duplicate id and a colliding prefix', async () => { await withFramework(true, async (framework) => { await assert.rejects( @@ -220,7 +680,7 @@ test('connectMcplServer hot-connects a network-only (WebSocket) MCPL server', as let msg: { id?: unknown; method?: string; params?: { arguments?: Record } }; try { msg = JSON.parse(String(data)); } catch { return; } const reply = (result: unknown) => ws.send(JSON.stringify({ jsonrpc: '2.0', id: msg.id, result })); - if (msg.method === 'initialize') reply({ capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); + if (msg.method === 'initialize') reply({ protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); else if (msg.method === 'tools/list') reply({ tools: [{ name: 'echo', description: 'e', inputSchema: { type: 'object' } }] }); }); }); diff --git a/test/mcpl-stderr-tail.test.ts b/test/mcpl-stderr-tail.test.ts index fb129aaa8..94d6d56b9 100644 --- a/test/mcpl-stderr-tail.test.ts +++ b/test/mcpl-stderr-tail.test.ts @@ -17,11 +17,15 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; import { EventEmitter } from 'node:events'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; import type { ChildProcess } from 'node:child_process'; import { STDERR_TAIL_CHARS, StdioTransport } from '../src/mcpl/transport.js'; import { McplServerConnection } from '../src/mcpl/server-connection.js'; import { AgentFramework } from '../src/framework.js'; +import { MockMembrane } from './helpers/mock-membrane.js'; /** `promise`, or `fallback` once `ms` (15 s) have passed, so a wait can't hang * the file, and a slow runner isn't mistaken for a defect. */ @@ -54,7 +58,7 @@ function standIn(): { stderr: EventEmitter; transport: StdioTransport; lines: st const child = Object.assign(new EventEmitter(), { stdin: new EventEmitter(), stdout: new EventEmitter(), stderr, killed: false, exitCode: null, signalCode: null, }); - const transport = new (StdioTransport as unknown as new (child: unknown, rl: unknown) => StdioTransport)(child, new EventEmitter()); + const transport = new (StdioTransport as unknown as new (child: unknown, rl: unknown, serverId: string) => StdioTransport)(child, new EventEmitter(), 'stand-in'); const lines: string[] = []; transport.on('stderr', (line: string) => lines.push(line)); return { stderr, transport, lines }; @@ -169,7 +173,8 @@ test("a connect and a reconnect that fail their handshakes report the server's s { id: 'needs-token', command: process.execPath, - args: ['-e', `console.error('fatal: DISCORD_TOKEN is not set'); ${EXIT_ON_INITIALIZE}`], + args: ['-e', `console.error('fatal: DISCORD_TOKEN ' + process.env.DISCORD_TOKEN + ' was refused'); ${EXIT_ON_INITIALIZE}`], + env: { DISCORD_TOKEN: 'tok-0123456789abcdef' }, reconnect: true, reconnectIntervalMs: 10, reconnectMaxIntervalMs: 10, @@ -183,15 +188,17 @@ test("a connect and a reconnect that fail their handshakes report the server's s // The stub holds its lifecycle events until the host says it's ready. connection.ready(); const failure = 'MCPL server "needs-token" transport closed before handshake completed (child process exited)'; - const stderr = '. The last of its stderr:\n stderr: "fatal: DISCORD_TOKEN is not set"'; + // The log lines are records, so the server's credentials are redacted + // there; the event carries the tail as the server wrote it. + const stderr = '. The last of its stderr:\n stderr: "fatal: DISCORD_TOKEN redacted was refused"'; const first = await within(connectFailed, null); assert.ok(first, 'the first connect failed and was reported within 15 s'); - assert.deepEqual(first, { error: failure, stderrTail: 'fatal: DISCORD_TOKEN is not set', attempt: 0 }); + assert.deepEqual(first, { error: failure, stderrTail: 'fatal: DISCORD_TOKEN tok-0123456789abcdef was refused', attempt: 0 }); assert.ok(logged.includes(`MCPL server "needs-token" initial connect failed, will retry: ${failure}${stderr}`), logged.join('\n')); const event = await within(reconnectFailed, null); assert.ok(event, 'a reconnect attempt failed within 15 s'); assert.equal(event.error, failure); - assert.equal(event.stderrTail, 'fatal: DISCORD_TOKEN is not set'); + assert.equal(event.stderrTail, 'fatal: DISCORD_TOKEN tok-0123456789abcdef was refused'); assert.ok(logged.includes(`MCPL server "needs-token" reconnect failed: ${failure}${stderr}`), logged.join('\n')); }); @@ -206,7 +213,7 @@ function alertingFramework() { const traces: Array> = []; framework.onTrace((event: Record) => traces.push(event)); const connection = Object.assign(new EventEmitter(), { id: 'needs-token', willReconnect: true }); - framework.wireMcplEvents(connection); + framework.wireMcplEvents(connection, { id: 'needs-token', command: 'needs-token', env: { API_TOKEN: 'sk-0123456789abcdef' } }); return { connection, traces }; } @@ -227,7 +234,8 @@ test("the mcpl-down ops alert names a failed handshake without the server's stde const trace = traces.find((event) => event.type === 'mcpl:server-connect-failed'); assert.equal(trace?.error, error); - assert.equal(trace?.stderrTail, stderrTail); + // The trace is a record, so the server's credential is redacted in it. + assert.equal(trace?.stderrTail, 'fatal: token redacted was refused'); assert.equal(traces.find((event) => event.type === 'ops:alert')?.message, `MCPL server unreachable (attempt 5): ${error}`); // What the operator's webhook is sent, which posts outside the process. assert.equal(posted.length, 1); @@ -235,6 +243,13 @@ test("the mcpl-down ops alert names a failed handshake without the server's stde assert.ok(!posted[0]!.includes('sk-0123456789abcdef')); }); +test("a failed initial connect's trace keeps the server's stderr with its credentials redacted, as a reconnect's does", () => { + const { connection, traces } = alertingFramework(); + const error = 'MCPL server "needs-token" transport closed before handshake completed (child process exited)'; + connection.emit('connect-failed', { error, stderrTail: 'fatal: token sk-0123456789abcdef was refused', attempt: 0 }); + assert.equal(traces.find((event) => event.type === 'mcpl:server-connect-failed')?.stderrTail, 'fatal: token redacted was refused'); +}); + test('a reconnect failure with no stderr, such as a spawn error, is traced with no `stderrTail` and alerted by its error', () => { const { connection, traces } = alertingFramework(); const error = 'MCPL server "needs-token" could not be started: spawn not-a-command ENOENT'; @@ -243,6 +258,43 @@ test('a reconnect failure with no stderr, such as a spawn error, is traced with assert.equal(traces.find((event) => event.type === 'ops:alert')?.message, `MCPL server unreachable (attempt 5): ${error}`); }); +test("at startup, a failed connect's trace and log line carry the server's stderr with its credentials redacted", { timeout: 30_000 }, async (t) => { + const logged: string[] = []; + t.mock.method(console, 'error', (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }); + const dir = mkdtempSync(join(tmpdir(), 'mcpl-stderr-tail-')); + const traces: Array> = []; + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test', systemPrompt: 'test' }], + modules: [{ + name: 'watcher', + async start(ctx: { onTrace(listener: (event: unknown) => void): void }) { ctx.onTrace((event) => traces.push(event as never)); }, + async stop() {}, + getTools: () => [], + async handleToolCall() { return { success: false }; }, + async onProcess() { return {}; }, + } as never], + mcplServers: [{ + id: 'needs-token', + command: process.execPath, + args: ['-e', `console.error('fatal: DISCORD_TOKEN ' + process.env.DISCORD_TOKEN + ' was refused'); ${EXIT_ON_INITIALIZE}`], + env: { DISCORD_TOKEN: 'tok-0123456789abcdef' }, + }], + }); + t.after(async () => { + await framework.stop(); + rmSync(dir, { recursive: true, force: true }); + }); + const failed = traces.find((event) => event.type === 'mcpl:server-connect-failed'); + assert.equal(failed?.stderrTail, 'fatal: DISCORD_TOKEN redacted was refused'); + assert.ok( + logged.some((line) => line.startsWith('Failed to connect MCPL server "needs-token"') && line.endsWith('stderr: "fatal: DISCORD_TOKEN redacted was refused"')), + logged.join('\n'), + ); + assert.ok(!logged.join('\n').includes('tok-0123456789abcdef')); +}); + test("a refusal waits for the server's stderr to end, so a line that a process sharing it writes after the exit is in its tail", { timeout: 30_000 }, async () => { // The server starts a process that shares its stderr. That process writes its // line only once the host has reaped the server, which the host does as it diff --git a/test/mcpl-transport-close.test.ts b/test/mcpl-transport-close.test.ts index b9dcdf147..b8e3d728d 100644 --- a/test/mcpl-transport-close.test.ts +++ b/test/mcpl-transport-close.test.ts @@ -16,11 +16,14 @@ import { test, type TestContext } from 'node:test'; import assert from 'node:assert/strict'; import { EventEmitter } from 'node:events'; import type { ChildProcess } from 'node:child_process'; +import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs'; import type { AddressInfo, Socket } from 'node:net'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; import { WebSocketServer } from 'ws'; import { McplTransport, StdioTransport, type TransportCloseInfo } from '../src/mcpl/transport.js'; -import { McplServerConnection } from '../src/mcpl/server-connection.js'; +import { McplRequestError, McplServerConnection } from '../src/mcpl/server-connection.js'; import { AgentFramework } from '../src/framework.js'; function deferred() { @@ -47,8 +50,8 @@ function captureConsoleError(t: TestContext): string[] { /** Polls until the condition holds, failing after 5 s, so a regression fails * the test rather than keeping the process alive. */ -async function waitFor(condition: () => boolean, what: string): Promise { - const deadline = Date.now() + 5000; +async function waitFor(condition: () => boolean, what: string, ms = 5000): Promise { + const deadline = Date.now() + ms; while (!condition()) { if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`); await new Promise((done) => { setTimeout(done, 5); }); @@ -116,7 +119,7 @@ test('a connect whose child a signal ends during initialize is refused, and its assert.ok(logged.some((line) => line.includes('initial connect failed, will retry')), logged.join('\n')); }); -test('a transport kept for close() during a reconnect logs the errors it reports, rather than throwing them', { timeout: 10_000 }, async (t) => { +test('a transport a reconnect gets after close() has begun logs the errors it reports while it closes, rather than throwing them', { timeout: 10_000 }, async (t) => { const handshake = deferred(); const closing = deferred(); const transport = new HeldTransport(closing.promise); @@ -137,16 +140,16 @@ test('a transport kept for close() during a reconnect logs the errors it reports internals.config = { id: 'kept', command: 'unused', reconnect: true }; internals.hostCapabilities = { version: '0.5' }; const reconnecting = internals.attemptReconnect(); - // close() disables reconnect, waits for the handshake in flight, and then - // closes the transport it handed back. + // close() ends the reconnect in flight: once its handshake returns, the + // attempt closes the transport it got, and close() waits for that. const closed = connection.close(); handshake.resolve(); - await reconnecting; - await waitFor(() => transport.calls > 0, 'close() to reach the kept transport'); + await waitFor(() => transport.calls > 0, 'the ended reconnect to close the transport it got'); const logged = captureConsoleError(t); assert.doesNotThrow(() => transport.emit('error', new Error('a malformed frame while close() waits'))); closing.resolve(); + await reconnecting; await closed; assert.doesNotThrow(() => transport.emit('error', new Error('another, after close() has finished'))); assert.equal(connection.isConnected, false); @@ -184,6 +187,29 @@ test("a failed handshake's transport logs an error it reports while it closes, r ); }); +test("a failed handshake's transport error is logged without the server's credentials", { timeout: 10_000 }, async (t) => { + const logged = captureConsoleError(t); + const server = new WebSocketServer({ port: 0, host: '127.0.0.1' }); + t.after(() => new Promise((resolve) => { server.close(() => resolve()); })); + await new Promise((resolve) => { server.once('listening', () => resolve()); }); + server.on('connection', (socket) => { + socket.on('message', (data) => { + const request = JSON.parse(String(data)) as { id?: number; method?: string }; + if (request.method !== 'initialize') return; + socket.send(JSON.stringify({ jsonrpc: '2.0', id: request.id, error: { code: -32000, message: 'refused by the test server' } })); + (socket as unknown as { _socket: Socket })._socket.write(Buffer.from([0x83, 0x00])); + }); + }); + const { port } = server.address() as AddressInfo; + // The frame's error text, "invalid opcode 3", quotes this token. + await assert.rejects(McplServerConnection.connect({ id: 'malformed', url: `ws://127.0.0.1:${port}`, token: 'invalid opcode' }, { version: '0.5' })); + await waitFor(() => logged.some((line) => line.includes('after the handshake settled')), 'the frame\'s error to be logged'); + assert.ok( + logged.some((line) => line === '[mcpl] malformed: transport error after the handshake settled: Invalid WebSocket frame: redacted 3'), + logged.join('\n'), + ); +}); + /** Settles with the promise's value, or with `fallback` after `ms` (5 s), so a * test that waits for something that never comes still reaches its cleanup. */ async function within(promise: Promise, fallback: F, ms = 5000): Promise { @@ -349,13 +375,13 @@ test("the framework's mcpl:server-closed trace carries the transport's reason, a const framework = Object.create(AgentFramework.prototype) as { traceListeners: unknown[]; onTrace(listener: (event: Record) => void): void; - wireMcplEvents(connection: unknown): void; + wireMcplEvents(connection: unknown, config: unknown): void; }; framework.traceListeners = []; const closes: Array> = []; framework.onTrace((event) => { if (event.type === 'mcpl:server-closed') closes.push(event); }); const connection = Object.assign(new EventEmitter(), { id: 'srv', willReconnect: true }); - framework.wireMcplEvents(connection); + framework.wireMcplEvents(connection, { id: 'srv' }); connection.emit('close', null, 'SIGTERM', { transport: 'stdio', code: null, signal: 'SIGTERM', byHost: false, reason: 'child process exited, and its pipe failed with write EPIPE', @@ -413,3 +439,289 @@ test("a server that stops reading its stdin without closing it stays connected, assert.equal(closed, false, 'nothing closed the connection'); assert.equal(connection.isConnected, true, 'the connection is still up'); }); + +/** A child that closes its end of stdin and ignores SIGTERM, appending a line + * to `signals` for each SIGTERM it gets, so only SIGKILL ends it. */ +async function spawnDeafStubbornChild(t: TestContext, id: string, signals: string): Promise { + const transport = await StdioTransport.spawn({ + id, + command: process.execPath, + args: ['-e', [ + "const fs = require('fs');", + `process.on('SIGTERM', () => fs.appendFileSync(${JSON.stringify(signals)}, 'SIGTERM\\n'));`, + "fs.closeSync(0); console.log('stdin closed'); setInterval(() => {}, 1 << 30);", + ].join(' ')], + }); + t.after(() => transport.close().catch(() => {})); + const ready = await within(new Promise((resolve) => { transport.once('line', () => resolve(true)); }), false); + assert.ok(ready, 'the child said it had closed its stdin within 5 s'); + return transport; +} + +function signalsFile(t: TestContext): string { + const dir = mkdtempSync(join(tmpdir(), 'mcpl-pipe-signals-')); + t.after(() => rmSync(dir, { recursive: true, force: true })); + return join(dir, 'signals.txt'); +} +const received = (file: string) => (existsSync(file) ? readFileSync(file, 'utf8').split('\n').filter(Boolean).length : 0); + +test("a failed write to a child that ignores SIGTERM ends it with SIGKILL 2 s later, and the close still names the failure", { timeout: 15_000 }, async (t) => { + const signals = signalsFile(t); + const transport = await spawnDeafStubbornChild(t, 'deaf-stubborn', signals); + const logged = captureConsoleError(t); + const closed = new Promise((resolve) => { transport.once('close', resolve); }); + transport.writeLine('{}'); + await waitFor(() => transport.pipeFailure !== undefined, 'the write to fail'); + assert.equal(transport.pipeFailure?.message, 'write EPIPE', 'writes are refused from here on'); + const info = await within(closed, null, 8000); + assert.ok(info, 'the transport closed within 8 s'); + assert.equal(info.signal, 'SIGKILL', 'SIGKILL ended the child'); + assert.equal(info.reason, 'child process exited, and its pipe failed with write EPIPE'); + assert.equal(received(signals), 1, 'one SIGTERM, then SIGKILL'); + assert.equal(logged.filter((line) => line.includes('ignored SIGTERM for 2000ms')).length, 1, logged.join('\n')); +}); + +test("close() during a pipe failure's escalation shares its attempt: one SIGTERM, one SIGKILL", { timeout: 15_000 }, async (t) => { + const signals = signalsFile(t); + const transport = await spawnDeafStubbornChild(t, 'deaf-shared', signals); + const logged = captureConsoleError(t); + transport.writeLine('{}'); + await waitFor(() => received(signals) === 1, 'the pipe failure to send SIGTERM'); + // close() comes inside the 2 s the child has before SIGKILL, and resolves + // once that SIGKILL has ended it, with no SIGTERM of its own. + const child = (transport as unknown as { child: ChildProcess }).child; + assert.ok(child.exitCode === null && child.signalCode === null, 'close() comes while the child still runs'); + await transport.close(); + assert.equal(child.signalCode, 'SIGKILL'); + assert.equal(received(signals), 1, 'close() sent no second SIGTERM'); + assert.equal(logged.filter((line) => line.includes('sending SIGKILL')).length, 1, logged.join('\n')); +}); + +test("a pipe failure whose child outlives SIGKILL too leaves the transport open and says 'unreaped'; a close() once the child has exited reaps it", { timeout: 20_000 }, async (t) => { + const signals = signalsFile(t); + const transport = await spawnDeafStubbornChild(t, 'deaf-unreaped', signals); + captureConsoleError(t); + const child = (transport as unknown as { child: ChildProcess & { kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + child.kill = () => true; // signals never arrive: the child outlives both bounds + t.after(() => { realKill('SIGKILL'); }); + const unreaped = new Promise((resolve) => { transport.once('unreaped', resolve); }); + transport.writeLine('{}'); + const failure = await within(unreaped, null, 10_000); + assert.ok(failure, "'unreaped' within 10 s"); + assert.match(failure.message, /^stdio child \d+ of MCP server "deaf-unreaped" did not exit within 2000ms of SIGKILL; it could not be reaped$/); + assert.equal(transport.closed, false, 'the transport stays open, owning the child'); + // The child exits at last: its exit closes the transport, naming the pipe + // failure, and the first close() after it re-checks the failed attempt + // rather than repeating its verdict. + const closed = new Promise((resolve) => { transport.once('close', resolve); }); + realKill('SIGKILL'); + assert.equal((await within(closed, null))?.reason, 'child process exited, and its pipe failed with write EPIPE'); + await transport.close(); +}); + +test("a host close() during an unreapable child's pipe escalation reports the failure itself, with no 'unreaped'", { timeout: 20_000 }, async (t) => { + const signals = signalsFile(t); + const transport = await spawnDeafStubbornChild(t, 'deaf-host-close', signals); + captureConsoleError(t); + const child = (transport as unknown as { child: ChildProcess & { kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + child.kill = () => true; + t.after(() => { realKill('SIGKILL'); }); + let saidUnreaped = false; + transport.on('unreaped', () => { saidUnreaped = true; }); + transport.writeLine('{}'); + await waitFor(() => transport.pipeFailure !== undefined, 'the write to fail'); + await assert.rejects(transport.close(), /could not be reaped/); + await new Promise((done) => { setTimeout(done, 50); }); + assert.equal(saidUnreaped, false); +}); + +test("after a pipe fails nothing more is written, a second pipe error starts nothing, and a close() while the child runs rejects again without a signal", { timeout: 20_000 }, async (t) => { + // A child that ignores SIGTERM and keeps its stdin open: only its stdout + // fails here, so a write would still reach it. + const transport = await StdioTransport.spawn({ + id: 'read-failed', + command: process.execPath, + args: ['-e', "process.on('SIGTERM', () => {}); console.log('ready'); setInterval(() => {}, 1 << 30);"], + }); + t.after(() => transport.close().catch(() => {})); + captureConsoleError(t); + assert.ok(await within(new Promise((resolve) => { transport.once('line', () => resolve(true)); }), false), 'the child is up'); + const child = (transport as unknown as { child: ChildProcess & { kill: (signal?: string) => boolean } }).child; + const realKill = child.kill.bind(child); + let signalled = 0; + child.kill = () => { signalled++; return true; }; // signals never arrive: the child outlives both bounds + t.after(() => { realKill('SIGKILL'); }); + const stdin = child.stdin!; + const realWrite = stdin.write.bind(stdin); + let written = 0; + (stdin as unknown as { write: (...args: unknown[]) => boolean }).write = (...args: unknown[]) => { written++; return (realWrite as (...a: unknown[]) => boolean)(...args); }; + let unreaped = 0; + const first = new Promise((resolve) => { transport.on('unreaped', () => { unreaped++; resolve(); }); }); + + transport.writeLine('{"before":true}'); + assert.equal(written, 1, 'a write goes out while the pipes hold'); + child.stdout!.emit('error', new Error('read failed')); + assert.equal(transport.pipeFailure?.message, 'read failed'); + transport.writeLine('{"after":true}'); + assert.equal(written, 1, 'nothing more is written once a pipe has failed'); + child.stdin!.emit('error', new Error('write EPIPE')); + assert.equal(transport.pipeFailure?.message, 'read failed', 'the first failure is the one kept'); + + assert.ok(await within(first.then(() => true), false, 10_000), "'unreaped' within 10 s"); + await new Promise((resolve) => { setImmediate(resolve); }); + assert.equal(unreaped, 1, "one 'unreaped' for the one failed attempt"); + assert.equal(signalled, 2, 'SIGTERM, then SIGKILL'); + // While the child runs, close() fails with the same verdict and sends no + // signal of its own. + await assert.rejects(transport.close(), /could not be reaped/); + assert.equal(signalled, 2, 'close() sent no signal'); +}); + +/** A legacy MCPL server that answers initialize, then, once the initialized + * notification is in, closes its stdin, ignoring SIGTERM, and says so on + * stderr. */ +const DEAF_AND_STUBBORN = [ + "process.on('SIGTERM', () => console.error('ignored SIGTERM'));", + "const rl = require('readline').createInterface({ input: process.stdin });", + "rl.on('line', (line) => {", + ' const message = JSON.parse(line);', + " if (message.method === 'initialize') {", + " process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: message.id, result: { protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { version: '0.5' } } }, serverInfo: { name: 'deaf-and-stubborn', version: '0.0.0' } } }) + '\\n');", + " } else if (message.method === 'notifications/initialized') {", + " process.stdin.once('close', () => { try { require('fs').closeSync(0); } catch {} console.error('closed stdin'); });", + ' rl.close(); process.stdin.destroy();', + ' }', + '});', + 'setInterval(() => {}, 1 << 30);', +].join('\n'); + +test('a legacy request made after a stdin failure is refused as not sent, and the server, which ignores SIGTERM, is killed and reconnected', { timeout: 60_000 }, async (t) => { + const connection = await McplServerConnection.connectWithReconnect( + { id: 'deaf-and-stubborn', command: process.execPath, args: ['-e', DEAF_AND_STUBBORN], reconnect: true, reconnectIntervalMs: 10, reconnectMaxIntervalMs: 10 }, + { version: '0.5' }, + ); + t.after(() => connection.close()); + const logged = captureConsoleError(t); + const stderr: string[] = []; + connection.on('stderr', ({ line }: { line: string }) => { stderr.push(line); }); + const closed = new Promise<{ reason?: string; signal?: string | null }>((resolve) => { + connection.once('close', (_code: unknown, _signal: unknown, info: { reason?: string; signal?: string | null }) => resolve(info)); + }); + const reconnected = new Promise((resolve) => { connection.once('reconnect', () => resolve()); }); + connection.ready(); + await waitFor(() => stderr.includes('closed stdin'), 'the server to close its stdin'); + // The first write after that fails with EPIPE. It went out, so its request + // waits for the close, as possibly executed. + const first = connection.sendManifestRequest(30_000).then(() => null, (error: unknown) => error); + await waitFor(() => stderr.includes('ignored SIGTERM'), 'the transport to send SIGTERM'); + // From here on nothing is written: a request is refused at once. + await assert.rejects(connection.sendManifestRequest(30_000), (error: unknown) => { + assert.ok(error instanceof Error && (error as { outcome?: string }).outcome === 'not-sent', String(error)); + assert.equal((error as Error).message, 'The request wasn\'t sent: this host can no longer write to the "deaf-and-stubborn" server, ' + + 'so nothing ran. This host is reconnecting to it; try again later.'); + return true; + }); + // The pipe's error is the operator's, in the log. + assert.ok(logged.some((line) => /^MCPL server "deaf-and-stubborn": \S+ not sent, since its stdio pipe failed: .*write EPIPE/.test(line)), logged.join('\n')); + const info = await within(closed, null, 10_000); + assert.equal(info?.signal, 'SIGKILL', 'SIGKILL ended it'); + assert.equal(info?.reason, 'child process exited, and its pipe failed with write EPIPE'); + const outcome = await within(first, 'still waiting', 10_000); + assert.ok(outcome instanceof McplRequestError, `the written request failed with the close: ${String(outcome)}`); + assert.equal(outcome.outcome, 'no-response', 'it went out, so it may have run'); + assert.equal(outcome.message, 'The connection to the "deaf-and-stubborn" server closed before it answered, so whether the request ' + + 'took effect is unknown. Check its effect before trying again; this host is reconnecting to it.'); + // How it closed, and the request it left unanswered, are the operator's, in the log. + assert.ok(logged.some((line) => /^MCPL server "deaf-and-stubborn" disconnected \(.*signal=SIGKILL, reason=child process exited, and its pipe failed with write EPIPE\) with 1 request\(s\) unanswered: mcpl\/manifest/.test(line)), logged.join('\n')); + assert.equal(await within(reconnected.then(() => 'reconnected'), 'no reconnect within 15 s', 15_000), 'reconnected'); +}); + +test("a legacy server whose stdin fails and whose child outlives SIGKILL is held as a launch it can't reap: a permanent failure, and close() retries", { timeout: 60_000 }, async (t) => { + const connection = await McplServerConnection.connectWithReconnect( + { id: 'deaf-unreapable', command: process.execPath, args: ['-e', DEAF_AND_STUBBORN], reconnect: true, reconnectIntervalMs: 10, reconnectMaxIntervalMs: 10 }, + { version: '0.5' }, + ); + captureConsoleError(t); + const child = (connection as unknown as { transport: { child: ChildProcess & { kill: (signal?: string) => boolean } } }).transport.child; + const realKill = child.kill.bind(child); + child.kill = () => true; // signals never arrive: the child outlives both bounds + t.after(async () => { + realKill('SIGKILL'); + await waitFor(() => child.signalCode !== null, 'the held child killed'); + await connection.close(); + }); + const stderr: string[] = []; + connection.on('stderr', ({ line }: { line: string }) => { stderr.push(line); }); + const order: string[] = []; + const failures: Array<{ error: string; permanent?: boolean }> = []; + connection.on('close', (_code: unknown, _signal: unknown, info: { reason?: string }) => { + order.push(`close: ${info.reason} (holds its child: ${connection.holdsUnreapedChild})`); + }); + connection.on('reconnect-failed', (failure: { error: string; permanent?: boolean }) => { order.push('reconnect-failed'); failures.push(failure); }); + connection.ready(); + await waitFor(() => stderr.includes('closed stdin'), 'the server to close its stdin'); + void connection.sendManifestRequest(30_000).catch(() => {}); // its write meets EPIPE + await within(new Promise((resolve) => { connection.once('reconnect-failed', () => resolve()); }), null, 10_000); + assert.deepEqual(order, [ + 'close: its stdio pipe failed with write EPIPE, and its child outlived SIGKILL (holds its child: true)', + 'reconnect-failed', + ]); + assert.equal(failures[0]!.permanent, true); + assert.match(failures[0]!.error, /^the previous launch could not be reaped \(stdio child \d+ of MCP server "deaf-unreapable" did not exit within 2000ms of SIGKILL; it could not be reaped\); reconnecting is halted$/); + assert.equal(connection.isConnected, false); + assert.equal(connection.willReconnect, false); + // close() retries the reap: refused while the child runs, then done. + await assert.rejects(connection.close(), /could not be reaped/); + realKill('SIGKILL'); + await waitFor(() => child.signalCode !== null, 'the child to be killed'); + await connection.close(); + assert.equal(connection.holdsUnreapedChild, false); +}); + +for (const release of ['ready', 'readyControlPlane'] as const) { + test(`a legacy connection held as unreaped before its first ${release}() still delivers its permanent failure there, and nothing else`, { timeout: 60_000 }, async (t) => { + const connection = await McplServerConnection.connectWithReconnect( + { id: 'held-early', command: process.execPath, args: ['-e', DEAF_AND_STUBBORN], reconnect: true, reconnectIntervalMs: 10, reconnectMaxIntervalMs: 10 }, + { version: '0.5' }, + ); + captureConsoleError(t); + const transport = (connection as unknown as { transport: { pipeFailure: Error | undefined; child: ChildProcess & { kill: (signal?: string) => boolean } } }).transport; + const child = transport.child; + const realKill = child.kill.bind(child); + child.kill = () => true; // signals never arrive: the child outlives both bounds + t.after(async () => { + realKill('SIGKILL'); + await waitFor(() => child.signalCode !== null, 'the held child killed'); + await connection.close(); + }); + const failures: Array<{ error: string; permanent?: boolean }> = []; + connection.on('reconnect-failed', (failure: { error: string; permanent?: boolean }) => { failures.push(failure); }); + const delivered: string[] = []; + connection.on('stderr', ({ line }: { line: string }) => { delivered.push(line); }); + // The server closes its stdin once the handshake is done, so a write after + // that meets EPIPE. (Before ready(), the connection holds its own events, + // the server's stderr among them, so the writes say when.) + await waitFor(() => { + if (transport.pipeFailure === undefined) void connection.sendManifestRequest(30_000).catch(() => {}); + return transport.pipeFailure !== undefined; + }, 'a write to meet EPIPE'); + await waitFor(() => connection.holdsUnreapedChild, 'the launch held', 10_000); + assert.equal(connection.willReconnect, false); + assert.equal(failures.length, 0, 'held until the first release'); + // The server's stderr line came before its pipe failed, and is held too. + const heldStderr = () => (connection as unknown as { bufferedEvents: Array<{ event: string; args: unknown[] }> }).bufferedEvents + .filter((item) => item.event === 'stderr') + .map((item) => (item.args[0] as { line: string }).line); + assert.ok(heldStderr().includes('closed stdin'), heldStderr().join('\n')); + connection[release](); + assert.equal(failures.length, 1, `${release}() delivered the permanent failure`); + assert.equal(failures[0]!.permanent, true); + assert.match(failures[0]!.error, /could not be reaped/); + // The server's own traffic, its stderr among it, stays held. + assert.deepEqual(delivered, [], 'nothing the server sent was delivered'); + assert.ok(heldStderr().includes('closed stdin'), 'its stderr line is still held'); + assert.equal((connection as unknown as { dataPlaneReady: boolean }).dataPlaneReady, false, 'and no plane was readied'); + }); +} diff --git a/test/mcpl-ws-transport.test.ts b/test/mcpl-ws-transport.test.ts index 6a08fe532..3f4c01a30 100644 --- a/test/mcpl-ws-transport.test.ts +++ b/test/mcpl-ws-transport.test.ts @@ -3,7 +3,7 @@ import assert from 'node:assert/strict'; import { WebSocketServer, WebSocket } from 'ws'; import type { AddressInfo } from 'node:net'; import { McplServerConnection, type McplConnectionCloseInfo } from '../src/mcpl/server-connection.js'; -import { buildWebSocketUrl, isWebSocketTransport } from '../src/mcpl/transport.js'; +import { buildWebSocketUrl, isWebSocketTransport, openTransport } from '../src/mcpl/transport.js'; import type { McplHostCapabilities } from '../src/mcpl/types.js'; const HOST_CAPS: McplHostCapabilities = { version: '0.4', pushEvents: true, featureSets: true }; @@ -28,7 +28,7 @@ function startMockMcplWsServer(onSocket?: (ws: WebSocket) => void) { const reply = (result: unknown) => ws.send(JSON.stringify({ jsonrpc: '2.0', id: msg.id, result })); switch (msg.method) { case 'initialize': - reply({ capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); + reply({ protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); break; case 'notifications/initialized': break; // notification, no reply @@ -82,6 +82,9 @@ test('buildWebSocketUrl validates protocol and appends the token as a query para ); assert.throws(() => buildWebSocketUrl({ id: 's', url: 'http://example.com' }), /ws:\/\/ or wss:\/\//); assert.throws(() => buildWebSocketUrl({ id: 's', command: 'x' }), /requires "url"/); + // A url that doesn't parse can't be redacted, so the error doesn't echo it. + assert.throws(() => buildWebSocketUrl({ id: 's', url: 'not a url ?token=sekret' }), + (error: Error) => /invalid WebSocket url/.test(error.message) && !error.message.includes('sekret')); }); test('buildWebSocketUrl refuses a url with a fragment, which the ws library refuses to dial', () => { @@ -95,9 +98,23 @@ test('isWebSocketTransport selects the transport from config shape', () => { assert.equal(isWebSocketTransport({ id: 's', url: 'wss://x/y' }), true); assert.equal(isWebSocketTransport({ id: 's', url: 'wss://x/y', transport: 'websocket' }), true); assert.equal(isWebSocketTransport({ id: 's', command: 'node' }), false); - // Explicit transport wins over shape. assert.equal(isWebSocketTransport({ id: 's', command: 'node', transport: 'stdio' }), false); - assert.equal(isWebSocketTransport({ id: 's', url: 'wss://x', command: 'node' }), false); // url+command → stdio unless transport says otherwise + // The scheme decides, as resolveServerBinding does: an http(s) url is modern MCP, never a WebSocket. + assert.equal(isWebSocketTransport({ id: 's', url: 'https://x/mcp' }), false); + assert.equal(isWebSocketTransport({ id: 's', url: 'http://x/mcp', transport: 'http' }), false); + // With both a command and a url, the explicit transport decides. + assert.equal(isWebSocketTransport({ id: 's', url: 'wss://x', command: 'node', transport: 'websocket' }), true); + assert.equal(isWebSocketTransport({ id: 's', url: 'wss://x', command: 'node', transport: 'stdio' }), false); + // A configuration resolveServerBinding refuses selects nothing. + assert.equal(isWebSocketTransport({ id: 's', url: 'wss://x', command: 'node' }), false); + assert.equal(isWebSocketTransport({ id: 's', url: 'https://x/mcp', transport: 'websocket' }), false); +}); + +test('openTransport refuses a modern or refused configuration before dialing or spawning anything', async () => { + // x.invalid never resolves, so a dial would fail with a connect error, not these. + await assert.rejects(openTransport({ id: 'm', url: 'https://x.invalid/mcp' }), /configured for modern MCP; the legacy transports can't open it/); + await assert.rejects(openTransport({ id: 'm', command: 'srv', protocol: 'modern' }), /configured for modern MCP; the legacy transports can't open it/); + await assert.rejects(openTransport({ id: 'b', command: 'srv', url: 'wss://x.invalid/mcpl' }), /"command" and "url" are both set/); }); test('connects over WebSocket, negotiates capabilities, lists tools, and round-trips a call', async () => { @@ -134,6 +151,17 @@ test('a failed WebSocket dial rejects with a clear error (no reconnect)', async ); }); +test("a failed dial's error names the endpoint, never a credential the url or the token carries", async () => { + await assert.rejects( + McplServerConnection.connect({ id: 'dead', url: 'ws://u:pw@127.0.0.1:1/mcpl/s/sekret4kQ9xW2vLm8pRs/x?api_key=sekret&v=2', token: 'tok-sekret' }, HOST_CAPS), + (error: Error) => { + assert.match(error.message, /WebSocket connect (failed|timed out).*\(ws:\/\/127\.0\.0\.1:1\/mcpl\/s\/redacted\/x\?api_key=redacted&v=redacted\)/); + assert.ok(!/sekret|pw@/.test(error.message), error.message); + return true; + }, + ); +}); + // A tiny stdio MCPL server: reads NDJSON on stdin, answers the same three // methods on stdout, and writes one line to stderr on boot (to prove stderr // still surfaces after the transport refactor). Kept inline so the test is @@ -149,7 +177,7 @@ process.stdin.on('data', (c) => { if (!line.trim()) continue; let m; try { m = JSON.parse(line); } catch { continue; } const reply = (result) => process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: m.id, result }) + '\\n'); - if (m.method === 'initialize') reply({ capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); + if (m.method === 'initialize') reply({ protocolVersion: '2024-11-05', capabilities: { experimental: { mcpl: { version: '0.4', pushEvents: true } } } }); else if (m.method === 'tools/list') reply({ tools: [{ name: 'echo', description: 'e', inputSchema: { type: 'object' } }] }); else if (m.method === 'tools/call') reply({ content: [{ type: 'text', text: String((m.params && m.params.arguments && m.params.arguments.text) || '') }] }); } diff --git a/test/placed-reaction-baseline.test.ts b/test/placed-reaction-baseline.test.ts index 77c7dbee3..ef3a91a52 100644 --- a/test/placed-reaction-baseline.test.ts +++ b/test/placed-reaction-baseline.test.ts @@ -12,6 +12,7 @@ import { MockMembrane } from './helpers/mock-membrane.js'; const ENV_KEY = 'DISCORD_SUPPRESSED_REACTIONS_BASELINE'; const FIXTURE = fileURLToPath(new URL('./fixtures/reaction-baseline-mcpl-server.mjs', import.meta.url)); +const MODERN_FIXTURE = fileURLToPath(new URL('./fixtures/modern-mcp-server.mjs', import.meta.url)); function server(overrides: Partial = {}): McplServerConfig { return { id: 'probe', command: process.execPath, args: [FIXTURE], ...overrides }; @@ -140,13 +141,23 @@ describe('placed-reaction baseline', () => { assert.deepEqual(config.env, { BASELINE_TEST_EXTRA: 'preserved' }, 'caller config stays unchanged'); }); - for (const transport of [undefined, 'stdio'] as const) { - it('injects the baseline with command + URL and ' + (transport ?? 'implicit') + ' transport', async () => { - const fw = await create('🔕', [server({ url: 'ws://127.0.0.1:1', transport })]); - const env = await childEnv(fw); - assert.deepEqual(new Set(env.baseline?.split(',')), new Set([...REFUSAL_REACTION_BASELINE, '💤', '🔕'])); - }); - } + it('injects the baseline with command + URL and stdio transport', async () => { + const fw = await create('🔕', [server({ url: 'ws://127.0.0.1:1', transport: 'stdio' })]); + const env = await childEnv(fw); + assert.deepEqual(new Set(env.baseline?.split(',')), new Set([...REFUSAL_REACTION_BASELINE, '💤', '🔕'])); + }); + + it('refuses command + URL with no transport before spawning anything', async () => { + // Which of the two the configuration means is ambiguous, so its binding + // is refused: nothing is launched, and no baseline is derived for it. + const fw = await create('🔕'); + await assert.rejects( + () => fw.connectMcplServer(server({ url: 'ws://127.0.0.1:1' })), + /"command" and "url" are both set, and no "transport" says which one to use/, + ); + const registry = (fw as unknown as { mcplServerRegistry: { getServer(id: string): unknown } | null }).mcplServerRegistry; + assert.equal(registry?.getServer('probe') ?? null, null, 'nothing is registered or spawned'); + }); it('derives the default on runtime connect, including newly retained ledger markers', async () => { const fw = await create('🔕'); @@ -290,6 +301,22 @@ describe('placed-reaction baseline', () => { }); } + it('a modern server connects and restarts on a corrupt ledger, which it never reads', async () => { + // A modern connection has no awareness barrier and no reaction default, + // so the ledger is no reason of its own to refuse it. + const fw = await create(); + const modern = (id: string): McplServerConfig => ({ + id, command: process.execPath, args: [MODERN_FIXTURE, join(dir, `${id}-starts.log`), join(dir, `${id}-events.log`)], protocol: 'modern', + }); + await fw.connectMcplServer(modern('modern')); + writeFileSync(outboxPath, '{broken'); + await fw.restartMcplServer('modern'); + await fw.connectMcplServer(modern('second')); + assert.deepEqual(fw.listMcplServers().map((s) => [s.id, s.connected]).sort(), [['modern', true], ['second', true]]); + // A legacy connect still refuses on it. + await assert.rejects(() => fw.connectMcplServer(server()), accountingRefusal); + }); + it('passes a URL-only WebSocket config through unchanged', async () => { const fw = await create(); // Bring up the MCPL subsystem as a first runtime connect does. diff --git a/test/reaction-baseline-platform.test.ts b/test/reaction-baseline-platform.test.ts index 38d12dc01..4f6f9e5d3 100644 --- a/test/reaction-baseline-platform.test.ts +++ b/test/reaction-baseline-platform.test.ts @@ -25,11 +25,11 @@ async function connectionEnv( mcplHostCapabilities: {}, mcplPrefixMap: new Map(), mcplServerConfigs: new Map(), - mcplConnecting: new Set(), channelRegistry: null, discordAwarenessOutbox: null, discordAwarenessEmoji: '🔕', mcplServerRegistry: { + getServer: () => undefined, async addServer(config: McplServerConfig) { captured = config; throw stopBeforeSpawn; diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts new file mode 100644 index 000000000..465ca069f --- /dev/null +++ b/test/script-structured-references.test.ts @@ -0,0 +1,279 @@ +/** + * What a script receives from a tool whose result carries structuredContent, + * through the real dispatch. An MCPL peer's content blocks get the treatment + * RFC-005 gives them on every lane the model reads, since what a script prints + * comes back to the model: a reference arrives without its URI, as the host's + * reference id, what the server said about it in that block, and `savedTo`, + * the workspace path the dispatch fetched its payload to for the script; a + * block the classifier rejects arrives as the notice the model's lanes show, + * and inline data that claims a bulk disposition as the withheld notice. A + * standard peer's blocks arrive as the server sent them. + */ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import { mkdirSync, mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { AgentFramework, WorkspaceModule } from '../src/index.js'; +import type { ToolResult } from '../src/index.js'; +import { EXPIRY_REASON, INLINE_WITHHELD_TEXT } from '../src/mcpl/references.js'; +import type { ReferenceRecord } from '../src/mcpl/references.js'; +import { MockMembrane } from './helpers/mock-membrane.js'; + +const SAVED = 'board/references/report.pdf'; +const DIGEST = 'sha256:47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU'; + +/** The URIs of one test's references. The reference registry lives as long + * as the process, so each test names its own. */ +const uris = (test: string) => ({ + report: `https://files.example.invalid/${test}/report.pdf?sig=secret-token`, + big: `https://files.example.invalid/${test}/big.png?sig=other-token`, + old: `https://files.example.invalid/${test}/old.png?sig=old-token`, +}); + +/** A block of each kind the dispatch decides about. */ +const blocks = (test: string) => [ + // A reference whose payload the fetch saves. Its labels carry a bidi + // override, a bidi mark and a control character. + { type: 'resource', uri: uris(test).report, name: 'report\u202E.pdf\u0007', mimeType: 'application/pdf\u200F', sizeBytes: 1234, + digest: DIGEST, expiresAt: '2026-12-31T00:00:00Z' }, + // The same reference again, saying something else about itself. + { type: 'resource', uri: uris(test).report, name: 'report-copy.pdf' }, + // A reference the fetch doesn't save. + { type: 'image', uri: uris(test).big, mimeType: 'image/png', disposition: 'ref', expiresAt: '2027-01-01T00:00:00Z' }, + // Inline data that claims a bulk disposition, and inline data with a uri + // beside it: withheld. + { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png', disposition: 'ref' }, + { type: 'audio', data: 'UklGRg==', mimeType: 'audio/wav', uri: 'https://files.example.invalid/beside?sig=beside-token' }, + // Inline data that claims nothing. + { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }, + // Blocks the classifier rejects: a uri longer than its limit (4096 UTF-16 + // units), an MCP embedded resource (its uri nested), and a type RFC-005 + // doesn't define. + { type: 'resource', uri: `https://files.example.invalid/long?sig=${'s'.repeat(4096)}`, name: 'long' }, + { type: 'resource', resource: { uri: 'https://files.example.invalid/embedded?sig=embedded-token', mimeType: 'text/plain', text: 'embedded' } }, + { type: 'resource_link', uri: 'https://files.example.invalid/link?sig=link-token', name: 'link' }, + { type: 'text', text: 'two attachments' }, + // A reference that counts as expired: with a bidi override after it, its + // expiresAt doesn't parse, so it fails closed. It isn't fetched, and the + // script is told why. + { type: 'image', uri: uris(test).old, mimeType: 'image/png', expiresAt: '2027-01-01T00:00:00Z\u202E' }, +]; + +/** A tools/call result as a server may send it, nonconformant blocks included. */ +interface Answer { content: unknown[]; structuredContent?: unknown; isError?: boolean } + +interface Harness { + /** Sets what every tools/call answers from now on. */ + answer: (result: Answer) => void; + /** A script's call to a tool of the MCPL peer `files`, or of the MCP-only peer `plain`. */ + script: (tool: string) => Promise; + /** A call the model makes to a tool of `files` (or of `plain`), through the same dispatch. */ + modelCall: (tool: string, server?: 'files' | 'plain') => Promise; + /** The URIs the fetcher was asked for, saved records aside. */ + fetched: string[]; + workspace: WorkspaceModule; +} + +async function withPeers(fn: (h: Harness) => Promise, opts: { eager?: boolean } = {}): Promise { + const dir = mkdtempSync(join(tmpdir(), 'script-structured-references-')); + const board = join(dir, 'board'); + mkdirSync(board); + const workspace = new WorkspaceModule({ mounts: [{ name: 'board', path: board, mode: 'read-write', watch: 'never' }] }); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test' }], modules: [workspace], + }); + workspace.initStore(framework.getStore()); + let answer: Answer = { content: [] }; + // Two legacy-engine peers: `files` negotiated MCPL, `plain` is MCP-only. + const peer = (id: string, capabilities: Record | null) => ({ id, capabilities, sendToolsCall: async () => answer }); + const peers = new Map([['files', peer('files', {})], ['plain', peer('plain', null)]]); + const fetched: string[] = []; + const internals = framework as unknown as { + mcplServerRegistry: unknown; + mcplPrefixMap: Map; + getReferenceFetcher: () => unknown; + handleScriptToolCall: (agent: string, tool: string, args: Record) => Promise; + dispatchMcplToolCall: (agent: string, call: { id: string; name: string; input: unknown }, serverId: string, prefix: string) => void; + pushEvent: (event: { type: string; callId?: string; result?: ToolResult }) => void; + }; + internals.mcplServerRegistry = { getServer: (id: string) => peers.get(id) ?? null, getAllServers: () => [], closeAll: async () => {} }; + internals.mcplPrefixMap.set('mcpl--files', 'files'); + internals.mcplPrefixMap.set('mcpl--plain', 'plain'); + // Stands in for the host-mediated fetcher: every reference is eager or + // none is, as the test asks, the report is saved and anything else + // refused, and a saved record returns at once. + internals.getReferenceFetcher = () => ({ + isEagerEligible: () => opts.eager === true, + fetch: async (record: ReferenceRecord) => { + if (record.fetchedPath) return { ok: true, path: record.fetchedPath }; + fetched.push(record.testimony.uri); + if (!record.testimony.uri.includes('report.pdf')) return { ok: false, error: 'refused by the test' }; + record.fetchedPath = SAVED; + return { ok: true, path: SAVED }; + }, + }); + const modelCall = (tool: string, server: 'files' | 'plain' = 'files') => new Promise((resolve) => { + const id = `model-${randomUUID()}`; + const push = internals.pushEvent; + internals.pushEvent = (event) => { + if (event.type !== 'tool-result' || event.callId !== id) return push.call(framework, event); + internals.pushEvent = push; + resolve(event.result!); + }; + internals.dispatchMcplToolCall('agent', { id, name: `mcpl--${server}--${tool}`, input: {} }, server, `mcpl--${server}`); + }); + framework.start(); + try { + await fn({ + answer: (result) => { answer = result; }, + script: (tool) => internals.handleScriptToolCall.call(framework, 'agent', tool, {}), + modelCall, + fetched, + workspace, + }); + } finally { + await framework.stop(); + rmSync(dir, { recursive: true, force: true }); + } +} + +/** What a script reads for those blocks from an MCPL peer, given each reference's id. */ +const treated = (report: string, big: string, old: string) => [ + { type: 'resource', refId: report, name: 'report.pdf', mimeType: 'application/pdf', sizeBytes: 1234, digest: DIGEST, + expiresAt: '2026-12-31T00:00:00Z', savedTo: SAVED }, + { type: 'resource', refId: report, name: 'report-copy.pdf', savedTo: SAVED }, + { type: 'image', refId: big, mimeType: 'image/png', expiresAt: '2027-01-01T00:00:00Z', savedTo: null }, + { type: 'text', text: INLINE_WITHHELD_TEXT }, + { type: 'text', text: INLINE_WITHHELD_TEXT }, + { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }, + { type: 'text', text: '[Reference not shown: its link is longer than this host accepts.]' }, + { type: 'text', text: '[Reference not shown: it arrived without a link.]' }, + { type: 'text', text: `[Content not shown: this host can't display "resource_link" content.]` }, + { type: 'text', text: 'two attachments' }, + { type: 'image', refId: old, mimeType: 'image/png', expiresAt: '2027-01-01T00:00:00Z', savedTo: null, expired: EXPIRY_REASON.unparseable }, +]; + +test("an MCPL peer's structured result reaches a script with each block treated as the model's lanes treat it", async () => { + await withPeers(async ({ answer, script, fetched }) => { + answer({ content: blocks('treated'), structuredContent: { attachments: 2 } }); + const out = await script('mcpl--files--attach'); + assert.ok(!out.includes('example.invalid'), `no URI reaches the script: ${out.slice(0, 2000)}`); + assert.deepEqual(fetched, [uris('treated').report, uris('treated').big], + 'every reference was fetched for the script before it read the result, except the one that counts as expired'); + const object = JSON.parse(out) as { content: Array<{ refId?: string }>; structuredContent: unknown; isError: boolean }; + const [report, , big] = object.content; + const old = object.content[10]; + assert.match(String(report!.refId), /^ref_/); + assert.match(String(big!.refId), /^ref_/); + assert.match(String(old!.refId), /^ref_/); + assert.equal(new Set([report!.refId, big!.refId, old!.refId]).size, 3); + assert.deepEqual(object, { content: treated(report!.refId!, big!.refId!, old!.refId!), structuredContent: { attachments: 2 }, isError: false }); + }); +}); + +test("an MCP-only peer's structured result reaches a script as the server sent it", async () => { + await withPeers(async ({ answer, script, fetched }) => { + answer({ content: blocks('plain'), structuredContent: { attachments: 2 } }); + const object = JSON.parse(await script('mcpl--plain--attach')); + assert.deepEqual(object, { content: blocks('plain'), structuredContent: { attachments: 2 }, isError: false }, + 'resource links, embedded resources, uri-form media and inline data are ordinary MCP blocks there'); + assert.deepEqual(fetched, [], 'nothing is a reference, so nothing is fetched'); + }); +}); + +test("content that isn't a list of blocks reaches a script as the model reads it, for each kind of peer", async () => { + await withPeers(async ({ answer, script, modelCall }) => { + answer({ content: 'not a list' as unknown as unknown[], structuredContent: { a: 1 } }); + const notice = '[malformed result: its content is not a list of blocks; the tool may already have completed]'; + // A standard peer's reading calls it a malformed result, on both paths. + const plainModel = await modelCall('attach', 'plain'); + assert.equal(plainModel.isError, true); + assert.ok(plainModel.error?.startsWith(notice), plainModel.error); + assert.deepEqual(JSON.parse(await script('mcpl--plain--attach')), { content: [], structuredContent: { a: 1 }, isError: true, error: notice }); + // An MCPL peer's reading has no content and reads the structured view, on both paths. + const filesModel = await modelCall('attach', 'files'); + assert.equal(filesModel.isError, false); + assert.deepEqual(JSON.parse(await script('mcpl--files--attach')), { content: [], structuredContent: { a: 1 }, isError: false }); + // Content left out isn't malformed: no content, on either peer. + answer({ structuredContent: { a: 1 } } as unknown as Answer); + assert.equal((await modelCall('attach', 'plain')).isError, false); + for (const tool of ['mcpl--plain--attach', 'mcpl--files--attach']) { + assert.deepEqual(JSON.parse(await script(tool)), { content: [], structuredContent: { a: 1 }, isError: false }, tool); + } + }); +}); + +test('without structured content, an MCPL peer\'s result reaches a script as stubs naming the same reference id and saved path', async () => { + await withPeers(async ({ answer, script }) => { + // Only blocks the text lane recognizes: one it doesn't sends the whole + // array to the JSON fallback, which is older than this script path. + const all = blocks('unstructured'); + const content = [all[0], all[2], all[9]]; + answer({ content, structuredContent: {} }); + const structuredId = String((JSON.parse(await script('mcpl--files--attach')) as { content: Array<{ refId?: string }> }).content[0]!.refId); + answer({ content }); + const out = await script('mcpl--files--attach'); + assert.ok(!out.includes('example.invalid'), out); + assert.ok(out.includes(`[${structuredId}]`), `the same reference id: ${out}`); + assert.ok(out.includes(`saved: ${SAVED}`), `the same saved path: ${out}`); + }); +}); + +test("an MCPL peer's structured result past the script cap is saved with its references as their ids, no URI in the file", async () => { + await withPeers(async ({ answer, script, workspace }) => { + answer({ content: blocks('oversized'), structuredContent: { pad: 'x'.repeat(5_000_000) } }); + const out = JSON.parse(await script('mcpl--files--attach')) as { isError: boolean; oversized: { chars: number; savedTo: string } }; + assert.equal(out.isError, false); + const read = await workspace.readBinary(out.oversized.savedTo); + assert.ok('data' in read, `the saved result is readable: ${out.oversized.savedTo}`); + const saved = read.data.toString('utf8'); + assert.ok(!saved.includes('example.invalid'), 'no URI in the saved file'); + const object = JSON.parse(saved) as { content: Array<{ refId?: string }> }; + assert.deepEqual(object.content, treated(object.content[0]!.refId!, object.content[2]!.refId!, object.content[10]!.refId!)); + }); +}); + +test("only a script that reads the blocks gets the script-demand fetch: not the model's call, nor an error a script reads as text", async () => { + await withPeers(async ({ answer, script, modelCall, fetched }) => { + answer({ content: blocks('model'), structuredContent: { attachments: 2 } }); + const model = await modelCall('attach'); + assert.equal(model.success, true); + assert.deepEqual(fetched, [], 'nothing here is eager-eligible, and the model\'s call gets no more than the eager pass'); + + answer({ content: [blocks('error')[0], { type: 'text', text: 'the attach failed' }], isError: true }); + assert.equal(await script('mcpl--files--attach'), 'Error: the attach failed'); + assert.deepEqual(fetched, [], 'a script that reads an error as its text alone gets no more than the eager pass'); + }); +}); + +test("the eager pass still runs for the model's call and for an error a script reads as text", async () => { + await withPeers(async ({ answer, script, modelCall, fetched }) => { + answer({ content: [blocks('eager-model')[0]], structuredContent: { attachments: 1 } }); + assert.equal((await modelCall('attach')).success, true); + assert.deepEqual(fetched, [uris('eager-model').report]); + + answer({ content: [blocks('eager-error')[0], { type: 'text', text: 'the attach failed' }], isError: true }); + assert.equal(await script('mcpl--files--attach'), 'Error: the attach failed'); + assert.deepEqual(fetched, [uris('eager-model').report, uris('eager-error').report]); + }, { eager: true }); +}); + +test('an error with structured content is fetched for the script, and reaches it treated, with its error text and structured content', async () => { + await withPeers(async ({ answer, script, fetched }) => { + answer({ content: [blocks('structured-error')[0], { type: 'text', text: 'the attach failed' }], structuredContent: { partial: true }, isError: true }); + const out = await script('mcpl--files--attach'); + assert.ok(!out.includes('example.invalid'), out); + const object = JSON.parse(out) as { content: Array<{ refId?: string }>; structuredContent: unknown; isError: boolean; error: string }; + assert.deepEqual(object, { + content: [treated(object.content[0]!.refId!, '', '')[0], { type: 'text', text: 'the attach failed' }], + structuredContent: { partial: true }, + isError: true, + error: 'the attach failed', + }); + assert.deepEqual(fetched, [uris('structured-error').report], 'the script reads the blocks, so its reference was fetched first'); + }); +}); diff --git a/test/stop-with-host-operations.test.ts b/test/stop-with-host-operations.test.ts index 1132961b2..7d31a52ec 100644 --- a/test/stop-with-host-operations.test.ts +++ b/test/stop-with-host-operations.test.ts @@ -54,7 +54,7 @@ interface Internals { maintenancePass: Promise | null; mcplServerRegistry: unknown; beginDiscordAwarenessBarrier: () => { requiresBarrier: boolean; promise: Promise }; - wireMcplEvents(connection: unknown): void; + wireMcplEvents(connection: unknown, config: { id: string }): void; operatorOperations: Set<{ label: string; promise: Promise }>; moduleRegistry: { getModule(name: string): unknown }; saveTurnCheckpoints(name: string, checkpoints: unknown[]): void; @@ -259,7 +259,7 @@ describe('stop() with an operator operation in flight', () => { it("stop() waits for a host/command, so its answer goes out before the connections close", async () => { const framework = await create(); const connection = Object.assign(new EventEmitter(), { id: 'srv' }); - internals(framework).wireMcplEvents(connection); + internals(framework).wireMcplEvents(connection, { id: 'srv' }); const cm = framework.getAgent('agent')!.getContextManager(); for (const n of [1, 2]) { cm.addMessage('user', [{ type: 'text', text: `m${n}` }], { serverId: 'srv', channelId: 'discord:g1:c1', messageId: `d${n}` }); @@ -358,7 +358,7 @@ describe('stop() with an operator operation in flight', () => { record('agent'); framework.getAgent('agent')!.getContextManager().addMessage('user', [{ type: 'text', text: 'a turn' }]); const connection = Object.assign(new EventEmitter(), { id: 'srv' }); - internals(framework).wireMcplEvents(connection); + internals(framework).wireMcplEvents(connection, { id: 'srv' }); let barriers = 0; internals(framework).beginDiscordAwarenessBarrier = () => { barriers++; @@ -814,7 +814,7 @@ describe('stop() with an operator operation in flight', () => { && error.code === 'agent-busy' && error.message === 'Cannot redo: the store is under live switch branch for every agent'); // A host/command undo in turns mode answers the refusal with its code. const connection = Object.assign(new EventEmitter(), { id: 'srv' }); - fw.wireMcplEvents(connection); + fw.wireMcplEvents(connection, { id: 'srv' }); let answer: unknown; connection.emit('host-command', { command: 'undo', agentName: 'agent', turns: 1 }, { id: 11, @@ -961,7 +961,15 @@ describe('stop() with an operator operation in flight', () => { await until(() => entered, 'MCPL operation'); const stopping = framework.stop(); await releaseWhenStopWaits(release); - await running; + // A disconnect finishes. A connect, or a restart's connect, that stop() + // overtook says so rather than reporting a connection the teardown is + // about to close. + if (action === 'disconnect') await running; + else { + await assert.rejects(running, action === 'connect' + ? /^Error: MCP server "srv" was not connected: the framework stopped while it was connecting$/ + : /^Error: MCP server "srv" is down: it was disconnected for its restart, and the restart's connect failed: MCP server "srv" was not connected: the framework stopped while it was connecting$/); + } await stopping; assert.equal(connected, false); const settled = action === 'disconnect' ? 'connection removed' : 'connection opened'; @@ -1071,7 +1079,7 @@ describe('stop() with an operator operation in flight', () => { if (event.type === 'undo:completed') stopping = framework.stop(); }); const connection = Object.assign(new EventEmitter(), { id: 'srv' }); - fw.wireMcplEvents(connection); + fw.wireMcplEvents(connection, { id: 'srv' }); connection.emit('host-command', { command: 'undo', agentName: 'agent', turns: 1 }, { id: 9, respond: () => { order.push('undo answered'); }, @@ -1129,7 +1137,7 @@ describe('stop() with an operator operation in flight', () => { if (event.type === 'undo:completed' && !stopping) stopping = framework.stop(); }); const connection = Object.assign(new EventEmitter(), { id: 'srv' }); - fw.wireMcplEvents(connection); + fw.wireMcplEvents(connection, { id: 'srv' }); let answer: { ok?: boolean; undone?: number; requested?: number } | undefined; // Read as the answer goes out: stop() closes the store once the undo is done. let checkpoints: unknown[] | undefined; diff --git a/test/stop-with-work-in-flight.test.ts b/test/stop-with-work-in-flight.test.ts index 3cc458c29..1c9ef70fc 100644 --- a/test/stop-with-work-in-flight.test.ts +++ b/test/stop-with-work-in-flight.test.ts @@ -996,7 +996,7 @@ describe('stop() with work still in flight', () => { modules: [], }); const connection = Object.assign(new EventEmitter(), { id: 'srv' }); - (framework as unknown as { wireMcplEvents(c: unknown): void }).wireMcplEvents(connection); + (framework as unknown as { wireMcplEvents(c: unknown, config: unknown): void }).wireMcplEvents(connection, { id: 'srv', url: 'wss://h/mcp?token=wireToken0123456789' }); return { framework, connection }; } @@ -1219,6 +1219,32 @@ describe('stop() with work still in flight', () => { } }); + it("a handler's error quoting the server's configured credential is recorded and answered without it, also across the cut", async () => { + const { framework, connection } = await wiredFramework(); + try { + // The credential starts just before the 500-character cut, so a cut + // made before redacting would keep its first characters. + // Once whole, and once with the credential across the 500-character + // cut, where a cut made before redacting keeps its first nine. + const errors = ['refused wireToken0123456789', `${'x'.repeat(490)} wireToken0123456789 and more`]; + for (const [i, error] of errors.entries()) { + (framework as unknown as { pushHandler: unknown }).pushHandler = { + handlePushEvent: async () => { throw new Error(error); }, + }; + const push = responder(); + connection.emit('push-event', { featureSet: 'chat', eventId: `e-cred-${i}`, payload: { content: [{ type: 'text', text: 'x' }] } }, push); + await answered(push); + const message = push.answers[0]!.error!.message; + await until(() => logged.filter((line) => line.includes('srv push-event handler failed')).length > i, 'the log line'); + for (const text of [message, ...logged]) { + assert.ok(!text.includes('wireToken'), text.slice(-120)); + } + } + } finally { + await framework.stop(); + } + }); + it('a request the handler already answered is not answered again when the handler then throws', async () => { const { framework, connection } = await wiredFramework(); try { diff --git a/test/tools-observe-parser-conformance.test.ts b/test/tools-observe-parser-conformance.test.ts index c493ed8f6..efdac0e66 100644 --- a/test/tools-observe-parser-conformance.test.ts +++ b/test/tools-observe-parser-conformance.test.ts @@ -5,6 +5,7 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; import { AgentFramework } from '../src/framework.js'; +import type { McplServerConfig } from '../src/mcpl/types.js'; import { McplServerConnection } from '../src/mcpl/server-connection.js'; import { McplTransport } from '../src/mcpl/transport.js'; import { CapabilityGrant } from '../src/mcpl/capability-grant.js'; @@ -79,9 +80,11 @@ function wireHarness() { ) => McplServerConnection; const connection = new Connection('observer', null, transport); const framework = Object.create(AgentFramework.prototype) as { - wireMcplEvents(connection: McplServerConnection): void; + mcplServerConfigs: Map; + wireMcplEvents(connection: McplServerConnection, config: McplServerConfig): void; }; - framework.wireMcplEvents(connection); + framework.mcplServerConfigs = new Map([['observer', { id: 'observer', command: 'unused' }]]); + framework.wireMcplEvents(connection, { id: 'observer', command: 'unused' }); connection.establishGrant(new CapabilityGrant(new Set(['toolLifecycle.observe']), [])); connection.ready(); let nextId = 1;