From 44543aea1026b1d81bcea5124bbc5dfd1d0c3a85 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 09:06:35 -0400 Subject: [PATCH 01/40] fix(tool-results): show each block of a content array, never the array's JSON MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A tool result whose content array held one block the tool-result lane didn't recognize went to history, and to the model, as the whole array's JSON. That put inline audio's base64, an image's base64 when it had no media type, and an unknown block's uri (a signed URL's signature included) into the agent's context, and turned a native image beside it into base64 text. The push lane showed each of those blocks as a marker. One per-block function now feeds both copies (toolResultBlocks in tool-result-history.ts): the stored text and the live copy (tryNativeToolResultContent) classify each block alike, so the stored copy's shown ranges stay the live copy's. Each block is itself (text, or an image with data and a media type), a reference's stub, inline data withheld for a contradiction, or a marker saying what it was and why it isn't shown: the push lane's markers, and for inline audio, which this host's tool results can't carry, "[Audio not shown: this host's tool results carry only text and images (, ).]". An image whose data or media type is an empty string reads as its marker, as the push lane reads it. Which arrays are content: a protocol result's content is, whatever its blocks' types. ToolResult.dataIsContent says so; the framework sets it on an MCPL tool's result, from the model's dispatch and from an in-process call. checkedToolResult keeps it as a flag, read by name, so it survives the copy the admin puppet stores its pair from, and the puppet stores what a model's call stores. Readers take it for its truth, as a result's other flags. Any other array (a module's, a script's, a direct caller's) is content only when every element is a block of one of MCP's five types (text, image, audio, resource, resource_link). Any other element makes it data, written as JSON as before. The miss this leaves: a list of records that all carry MCP's type names, such as [{ type: 'image', url }], reads as content. MCP's resource_link is a reference (RFC-005 §8.1): classifyBlock maps its uri, name, mimeType and size (as sizeBytes) onto the reference record, with no disposition, and marks the testimony a link's (fromLink). No uri reaches context that was kept out of it before: - A link the host can fetch shows as a stub on every lane, convertBlock's included, and at any depth in data (governedByRfc005); showsAsItself lets a render result carry one. - A tool result's link the host can't fetch, whose JSON showed its uri before, shows its name, uri and type, "[resource link "": , ; not fetched]", as #262 shows a standard server's link: the uri is the agent's only handle on it, and showing it is §5's MAY for a reference the host classifies as non-capability-bearing. The tool-result lane asks for this (StubOptions.unfetchedLinkUri) at both its sites. - Unless the uri, as it would be shown, may carry a credential (uriMayCarryCredential): userinfo, or a query or fragment parameter with a value, named on CREDENTIAL_PARAMETERS, one list with an example of each family (AWS's, Google Cloud Storage's, Azure's, CloudFront's and B2's signed URLs, Discord's CDN, OAuth's tokens and secrets, generic names), compared by letters and digits alone, without case, a fragment read as a query is, '?' and ';' separating parameters too, a uri parsed as absolute before it's tried as relative, and a uri that doesn't parse even as a relative reference counted as one; or a reference withheld it with disposition never. Such a link gets the stub that says why it can't be fetched, with no uri. Plain links with a query, such as watch?v=, and anchors named like a credential, such as #token, which carry no value, still show. - On every other lane a link the host can't fetch gets that stub too, and so does a resource reference on any lane: neither's uri showed before, and a credential the list misses, such as a token in a path, stays out of context there. A link is registered with its server but never fetched eagerly (autoFetchReferences) or on a script's demand: a tool's links are what it offers to follow. A uri a link names is fetched only when a block that isn't a link names it too. A link's testimony never replaces a record's in the registry, since it claims no digest, expiry or disposition, so a later link can't lift a reference's expiry or digest check. A stub's promise is the fetch's own check. #518's pre-dial check of the server and expiry becomes dialServer (servers in order, the origin) and dialCheck (dialServer, then the expiry) in references.ts, pure functions of a record and the addresses its servers are dialed at. The fetcher runs them on its own framework's configs, through its resolver, so the origin it checks is the url of the config whose token it sends, as #518 checked it against the binding the token came from; #518's eager check joins a fetch under way with dialServer alone. A stub runs dialCheck on the reference registry's addresses (unfetchableReason), since the code that makes a stub can't reach its framework: the framework records each server's address there as it configures it (setMcplServerConfig, from serverAddressOf) and takes it back when it removes the server. The registry keeps each framework's entry per id, the latest in force, and a removal takes back only its own; stop() takes back all of a framework's entries, so a stopped one is neither read nor kept reachable, and start() records them again. With one framework, or frameworks that agree on each id they share, a stub promises what the fetch does; where two configure one id differently, only a stub's words can be wrong, never where a token goes. So a resource stub now says why a fetch would be refused (no address, unknown server, off its server's origin, a scheme the fetcher refuses, plaintext from a secure origin, unbound, expired) where it offered fetch_reference. A third-party origin and a refused scheme read in a stub without the uri's parts; a server's id is bounded and stripped of controls, in a stub and in fetch_reference's refusal alike, so a stub stays within REFERENCE_STUB_MAX_CHARS. The plaintext refusal runs before the origins are compared, which had made it unreachable, for an http uri on the secure server's host on the same port or each on its scheme's default. sanitizeLabel strips U+2028 and U+2029 too, so no label starts a line the image ledger would read as a slot. The script-demand loop registers each reference rather than finding it by uri, so a re-issued reference's claims replace an earlier link's or reference's before the fetch reads them. The empty-link rule is made whole, as #506's merge note leaves open: inline data beside uri '' is plain inline data on every lane. A link that isn't empty still withholds it. Tests: tool-result-blocks.test.ts pins each shape on both copies, the content rule with and without dataIsContent, resource_link on each lane and in each fetch path, a tool result's unfetchable link shown where its uri can't carry a credential and stubbed where it may (a uri from each family of the list), the push lane's link never showing its uri, the render check, a link never replacing a reference's claims, each refusal a stub gives and the fetcher's own check agreeing with it, a refused scheme unnamed in a stub, two frameworks that configure one id at two urls (neither fetch dials the other's origin, each carries its own token to its own server, and a removal takes back only its own address), a framework's stop and start, plaintext from a secure origin, an unparseable server url, a link a reference withheld with never, a credential name or a scheme split by a control character, a link line with line separators reading as no slot, a script's demand fetch reading a reference's claims over a link's, the empty link beside data on both lanes, and dataIsContent through checkedToolResult, a model's round, and an MCPL call from the model, in process and through the puppet. Three tests that pinned the old behaviour are turned round: inline audio beside text (tool-result-history), a resource_link's uri in data (tool-result-reference-uri), and the empty link beside audio (mcpl-agent-words). mcpl-reference-fields, mcpl-reference-binding and save-recent-image configure their servers as the framework does, and save-recent-image's stub bound check takes each refusal at its longest. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.added.md | 1 + changelog.d/tool-result-blocks.changed.md | 1 + changelog.d/tool-result-blocks.fixed.md | 8 + src/framework.ts | 185 ++--- src/mcpl/convert-block.ts | 5 + src/mcpl/push-coalescer.ts | 2 +- src/mcpl/reference-fetcher.ts | 113 ++- src/mcpl/references.ts | 379 ++++++++-- src/module-registry.ts | 4 +- src/tool-result-history.ts | 196 ++++-- src/types/events.ts | 12 + test/mcpl-agent-words.test.ts | 12 +- test/mcpl-reference-binding.test.ts | 26 +- test/mcpl-reference-fields.test.ts | 5 + test/save-recent-image.test.ts | 25 +- test/tool-result-blocks.test.ts | 811 ++++++++++++++++++++++ test/tool-result-history.test.ts | 10 +- test/tool-result-reference-uri.test.ts | 21 +- 18 files changed, 1527 insertions(+), 289 deletions(-) create mode 100644 changelog.d/tool-result-blocks.added.md create mode 100644 changelog.d/tool-result-blocks.changed.md create mode 100644 changelog.d/tool-result-blocks.fixed.md create mode 100644 test/tool-result-blocks.test.ts diff --git a/changelog.d/tool-result-blocks.added.md b/changelog.d/tool-result-blocks.added.md new file mode 100644 index 000000000..60d208237 --- /dev/null +++ b/changelog.d/tool-result-blocks.added.md @@ -0,0 +1 @@ +- `ToolResult.dataIsContent` says a result's array `data` is a protocol result's content, in which every element is a block, of a type the host can show or not. The model then reads each element as itself, as a reference's stub, or as a marker saying what it was, never as the array's JSON. agent-framework sets it on an MCPL tool's result, and a module that returns a content array can set it too. diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md new file mode 100644 index 000000000..7f9fb9e28 --- /dev/null +++ b/changelog.d/tool-result-blocks.changed.md @@ -0,0 +1 @@ +- In a tool result, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md new file mode 100644 index 000000000..291af8225 --- /dev/null +++ b/changelog.d/tool-result-blocks.fixed.md @@ -0,0 +1,8 @@ +- A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. +- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. +- A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it says why the fetch would be refused, in the words the refusal gives. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, a uri off its server's origin, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. +- An `http` reference on the host and port of a server dialed over a secure connection is refused as `plaintext reference from a secure origin`, where the origin check called it a third-party origin. +- A label in a stub (a name, a type, a server's id, a provenance or a saved path) loses any line or paragraph separator (U+2028, U+2029), as it loses control characters. +- A script's demand fetch of a reference not yet fetched reads that reference's own digest and expiry, where it read whatever an earlier reference to the same uri had claimed. +- Inline data beside an empty `uri` is shown as any inline data is, on every lane, where it was withheld as data that also claimed a link. +- On the tool-result lane, an image whose data or media type is an empty string shows its marker, as on the push lane, where it was sent to the model as an image with that field empty. diff --git a/src/framework.ts b/src/framework.ts index cb39c3006..b8d214605 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,6 +1,6 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { dirname, join } from 'node:path'; -import { INLINE_WITHHELD_TEXT, REFERENCE_LIMITS, classifyBlock, isInlineContradiction, isReferenceExpired, referenceRegistry, referenceStubOrNull, sanitizeLabel } from './mcpl/references.js'; +import { REFERENCE_LIMITS, classifyBlock, isReferenceExpired, referenceRegistry, sanitizeLabel, type ServerAddress } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, @@ -113,6 +113,7 @@ import type { WorkspaceModule } from './modules/workspace/index.js'; import type { ToolResultGuard, WithheldSpill } from './tool-result-guard.js'; import { nonErrorResultText, + toolResultBlocks, toolResultDataToHistoryString, truncateForHistory, DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS, @@ -360,6 +361,23 @@ function extractChannelIdFromToolResult( } } +/** + * How the host reaches an MCPL server, for fetching its references: the url + * it dials the server at, as the transport decides. None for a server the + * host spawns, as a stdio server, nor for a WebSocket server whose url the + * transport won't dial, which buildWebSocketUrl refuses: one that isn't a + * ws:// or wss:// url, or has a fragment. + */ +function serverAddressOf(cfg: McplServerConfig): ServerAddress { + if (!isWebSocketTransport(cfg) || !cfg.url) return 'no-address'; + try { + buildWebSocketUrl(cfg, null); + } catch { + return 'no-address'; + } + return { url: cfg.url }; +} + /** * The text of an MCPL tools/call result: its text blocks' text, joined by * line breaks, with empty ones left out. A text block whose text isn't a @@ -3100,6 +3118,11 @@ export class AgentFramework { // and withheld spills, closed for the generation a stop ended, are open // for this one. this.lifecycleGeneration++; + // Its servers' addresses, which an earlier stop() took out of the + // reference registry, for its stubs. + for (const config of this.mcplServerConfigs.values()) { + referenceRegistry.setServerAddress(config.id, serverAddressOf(config), this); + } this.loopPromise = this.runLoop(); // Start periodic sync timer (if enabled) @@ -3458,6 +3481,10 @@ export class AgentFramework { if (this.mcplServerRegistry) { shutdownPromises.push(this.mcplServerRegistry.closeAll(teardownTimeoutMs)); } + // A stopped framework's servers' addresses leave the reference registry, + // so no other framework's stub reads them and the registry doesn't keep + // this one reachable. start() records them again. + step(() => { for (const id of this.mcplServerConfigs.keys()) referenceRegistry.forgetServerAddress(id, this); }); // Timed-out or failed teardown retains its context and may still write. // Sync available state best-effort, but leave its store open for a retry. for (const result of await Promise.allSettled(shutdownPromises)) { @@ -4907,18 +4934,8 @@ export class AgentFramework { (serverId) => { const cfg = this.mcplServerConfigs.get(serverId); if (!cfg) return undefined; - // A server's address is the url the host dials it at, as the - // transport decides: none for a server the host spawns, as a stdio - // server, nor for a WebSocket server whose url the transport won't - // dial, which buildWebSocketUrl refuses: one that isn't a ws:// or - // wss:// url, or has a fragment. - if (!isWebSocketTransport(cfg) || !cfg.url) return 'no-address'; - try { - buildWebSocketUrl(cfg, null); - } catch { - return 'no-address'; - } - return { url: cfg.url, token: cfg.token, autofetch: cfg.autofetch }; + const address = serverAddressOf(cfg); + return address === 'no-address' ? address : { url: address.url, token: cfg.token, autofetch: cfg.autofetch }; }, async (fileName, data, mimeType) => { const workspace = this.getWorkspaceModule(); @@ -4941,6 +4958,14 @@ export class AgentFramework { return this.referenceFetcherInstance; } + /** Configure an MCPL server: its config, for tool routing and the fetcher, + * which reads a server's address and credential from it, and its address + * in the reference registry, where a reference's stub reads it. */ + private setMcplServerConfig(config: McplServerConfig): void { + this.mcplServerConfigs.set(config.id, config); + referenceRegistry.setServerAddress(config.id, serverAddressOf(config), this); + } + /** Register every RFC-005 reference in a tool-result content array against * its server, then eagerly fetch the ones inside the autofetch envelope * (below the eager ceiling, connection origin — never worse than the @@ -4956,18 +4981,27 @@ export class AgentFramework { // record here, hence the key by uri. A uri that any of this result's // blocks calls expired takes no slot, whichever block registered last. A // later result's fresh expiry still supersedes, through register(). - const byUri = new Map(); + // A link (MCP's resource_link) is registered, so the agent can fetch it, + // but never fetched here: a tool's links are what it offers to follow, + // and fetching them would spend the server's fetch budget on bytes + // nobody asked for. A uri a link names is fetched only when a block of + // the result that isn't a link names it too. + const byUri = new Map(); for (const block of content) { const c = classifyBlock(block); if (c.kind !== 'reference') continue; const record = referenceRegistry.register(c.testimony, serverId); - const expiredHere = byUri.get(c.testimony.uri)?.expiredHere === true || isReferenceExpired(c.testimony); - byUri.set(c.testimony.uri, { record, expiredHere }); + const prior = byUri.get(c.testimony.uri); + const expiredHere = prior?.expiredHere === true || isReferenceExpired(c.testimony); + const linkOnly = (prior?.linkOnly ?? true) && c.testimony.fromLink === true; + byUri.set(c.testimony.uri, { record, expiredHere, linkOnly }); } if (!eager) return; const fetcher = this.getReferenceFetcher(); const eligible = [...byUri.values()] - .filter(({ record, expiredHere }) => !expiredHere && fetcher.isEagerEligible(record)) + .filter(({ record, expiredHere, linkOnly }) => !expiredHere && !linkOnly && fetcher.isEagerEligible(record)) .map(({ record }) => record); // Parallel with a count cap: N small fetches cost one round-trip of // wall-clock, and anything past the cap stays lazy (stub says so). @@ -17848,6 +17882,7 @@ export class AgentFramework { return { success: true, data: result.content, + dataIsContent: true, }; } catch (err) { return { @@ -18367,9 +18402,9 @@ export class AgentFramework { * * The live copy is the stored text itself unless the result is a content * array carrying an inline image, which tryNativeToolResultContent sends - * as native blocks. It takes the same arrays the serializer takes block by - * block, and shows each image and each reference block's stub whole and - * each text block up to where it cuts that block at the cap (`maxChars`, + * as native blocks. It takes the blocks the serializer takes + * (toolResultBlocks), and shows each image, stub and marker whole and each + * text block up to where it cuts that block at the cap (`maxChars`, * toolResultCutAt), so those are the ranges of the text it showed. */ private storedToolResultText( @@ -18644,21 +18679,27 @@ export class AgentFramework { if (result.data === undefined || result.data === null) return nonErrorResultText(result, 5_000_000); // RFC-005 "eagerly lazy": a script receiving a result is the strongest // available signal that the bytes are wanted (scripts process, they do - // not browse) — materialize the references of a content array now, so - // the stub the script receives carries a workspace path it can open + // not browse) — materialize the references at the top level of an array + // now, whether or not the array is shown as content, so the stub the + // script receives carries a workspace path it can open // directly. Only a reference registered against its server can be // fetched: an MCPL server's result content is, on arrival, including // content a module got from a call of its own (ctx.callTool) and returns. // A reference a module makes itself has no server, so its fetch is // refused (and logged here) and the script gets its stub, as it does for - // a reference nested in other data, which isn't fetched. + // a reference nested in other data, which isn't fetched. A link (MCP's + // resource_link) isn't fetched either: a tool's links are what it offers + // to follow, and the script can fetch the ones it wants with + // fetch_reference. if (Array.isArray(result.data)) { const fetcher = this.getReferenceFetcher(); for (const block of result.data) { const c = classifyBlock(block); - if (c.kind !== 'reference') continue; - const record = referenceRegistry.findByUri(c.testimony.uri) - ?? referenceRegistry.register(c.testimony); + if (c.kind !== 'reference' || c.testimony.fromLink) continue; + // Registered, not only found, so a re-issued reference's claims + // replace a link's, or a stale reference's, before the fetch reads + // them, as on any other path, while the record isn't fetched yet. + const record = referenceRegistry.register(c.testimony); const outcome = await fetcher.fetch(record); if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); } @@ -18668,7 +18709,7 @@ export class AgentFramework { // reference is its stub at any depth and inline data claiming a uri or a // disposition is withheld (RFC-005). 5MB protocol safety cap only; no // context-cap truncation, no file spill. - return toolResultDataToHistoryString(result.data, 5_000_000); + return toolResultDataToHistoryString(result.data, 5_000_000, { dataIsContent: Boolean(result.dataIsContent) }); } /** @@ -18769,7 +18810,8 @@ export class AgentFramework { // blocks natively rather than JSON-stringifying them away. Anything else // goes as text. The error path was handled above, so isError is always // false on these return paths. - const blocks = this.tryNativeToolResultContent(afResult.data, maxChars, precomputed?.filePath ?? null); + const blocks = this.tryNativeToolResultContent( + afResult.data, Boolean(afResult.dataIsContent), maxChars, precomputed?.filePath ?? null); if (blocks) { return { toolUseId: callId, content: blocks, isError: false }; } @@ -18784,9 +18826,10 @@ export class AgentFramework { } /** - * If `data` is an MCP tool-result content array carrying at least one image, - * convert to Membrane's native ToolResultContentBlock[]. Returns null when - * the array is text-only (let JSON path handle it; saves a code path). + * If `data` is a content array (toolResultBlocks) carrying at least one + * image, its blocks as Membrane's native ToolResultContentBlock[]: each + * image as itself, and every other block as the text the stored copy shows + * for it. Returns null otherwise, and the live copy is the stored text. * `maxChars`, when provided, caps each accompanying text block so an image * inlined alongside an enormous text payload can't blow the context. A * block is cut as the stored copy is (toolResultCutAt), never inside a @@ -18794,51 +18837,27 @@ export class AgentFramework { */ private tryNativeToolResultContent( data: unknown, + dataIsContent: boolean, maxChars?: number, /** Spill file written by the history path — referenced from truncation * notices so an image-adjacent giant text block stays recoverable. */ spillPath: string | null = null, ): ToolResultContentBlock[] | null { - if (!Array.isArray(data)) return null; - let hasImage = false; - const blocks: ToolResultContentBlock[] = []; - for (const raw of data) { - if (!raw || typeof raw !== 'object') return null; - const b = raw as { type?: unknown; text?: unknown; data?: unknown; mimeType?: unknown }; - if (isInlineContradiction(b)) { - // RFC-005 vector 2: withhold inline data claiming bulk disposition. - blocks.push({ type: 'text', text: INLINE_WITHHELD_TEXT }); - continue; - } - if (b.type === 'text') { - // A text that isn't a string shows as the converters' placeholder, - // rather than sending the whole array to the JSON fallback. - let text = typeof b.text === 'string' ? b.text : NON_STRING_TEXT_PLACEHOLDER; - if (maxChars && text.length > maxChars) { - text = text.slice(0, toolResultCutAt(text, maxChars)) - + '\n\n[truncated — original was ' + text.length + ' chars' - + (spillPath ? `; full serialized result: workspace file ${spillPath}` : '') - + ']'; - } - blocks.push({ type: 'text', text }); - } else if (b.type === 'image' && typeof b.data === 'string' && typeof b.mimeType === 'string') { - hasImage = true; - blocks.push({ - type: 'image', - source: { type: 'base64', data: b.data, mediaType: b.mimeType }, - }); - } else if (b.type === 'resource' || ((b.type === 'image' || b.type === 'audio') && typeof (b as { uri?: unknown }).uri === 'string')) { - // RFC-005 reference block → bounded stub. Previously this bailed the - // whole array to JSON, which both inlined the reference verbatim and - // demoted any adjacent image block to stringified base64. - const stub = referenceStubOrNull(b, 'from tool result'); - if (stub === null) return null; - blocks.push({ type: 'text', text: stub }); - } else { - return null; // unknown shape: the stored copy's text, which shows any reference in it as its stub - } - } - return hasImage ? blocks : null; + const blocks = toolResultBlocks(data, dataIsContent); + if (!blocks?.some((block) => block.kind === 'image')) return null; + return blocks.map((block): ToolResultContentBlock => { + if (block.kind === 'image') { + return { type: 'image', source: { type: 'base64', data: block.data, mediaType: block.mimeType } }; + } + let text = block.text; + if (block.kind === 'text' && maxChars && text.length > maxChars) { + text = text.slice(0, toolResultCutAt(text, maxChars)) + + '\n\n[truncated — original was ' + text.length + ' chars' + + (spillPath ? `; full serialized result: workspace file ${spillPath}` : '') + + ']'; + } + return { type: 'text', text }; + }); } private toolImageLedgerFor(agentName: string): ToolImageLedger { @@ -20719,7 +20738,7 @@ export class AgentFramework { for (const config of serverConfigs) { const prefix = config.toolPrefix ?? `mcpl--${config.id}`; this.mcplPrefixMap.set(prefix, config.id); - this.mcplServerConfigs.set(config.id, config); + this.setMcplServerConfig(config); } // Find shouldTriggerInference callback: @@ -21417,7 +21436,7 @@ export class AgentFramework { if (owner === config.id && other !== prefix) this.mcplPrefixMap.delete(other); } this.mcplPrefixMap.set(prefix, config.id); - this.mcplServerConfigs.set(config.id, config); + this.setMcplServerConfig(config); // Record per-server channel subscription policy before the server // registers channels — handleRegister fires during the handshake. @@ -22086,6 +22105,7 @@ export class AgentFramework { const prefix = config?.toolPrefix ?? `mcpl--${id}`; if (this.mcplPrefixMap.get(prefix) === id) this.mcplPrefixMap.delete(prefix); this.mcplServerConfigs.delete(id); + referenceRegistry.forgetServerAddress(id, this); await this.refreshMcplTools(); this.emitMcplToolDiff(oldToolNames, id); @@ -23147,19 +23167,16 @@ export class AgentFramework { // is a failure (mcplToolFailure, which an in-process caller's error // takes too). When a success contains non-text blocks (e.g. images // from an MCP tool like zulip-mcp's fetch_attachment), pass the full - // content array through so toMembraneToolResult can preserve image - // blocks natively. Text-only results still collapse to a joined - // string for backward compatibility with callers that expect data to - // be string-ish. + // content array through, marked as content (dataIsContent), so + // toMembraneToolResult can preserve image blocks natively and show + // every other block as its stub or marker. Text-only results still + // collapse to a joined string for backward compatibility with + // callers that expect data to be string-ish. const converted: ToolResult = result.isError ? mcplToolFailure(result.content, serverId, call.name) - : { - success: true, - data: result.content?.some((c) => c.type !== 'text') - ? result.content - : (mcplResultText(result.content) || undefined), - isError: false, - }; + : result.content?.some((c) => c.type !== 'text') + ? { success: true, data: result.content, dataIsContent: true, isError: false } + : { success: true, data: mcplResultText(result.content) || undefined, isError: false }; this.emitTrace({ type: converted.isError ? 'tool:failed' : 'tool:completed', module: `mcpl:${serverId}`, tool: toolName, callId: call.id, durationMs, diff --git a/src/mcpl/convert-block.ts b/src/mcpl/convert-block.ts index 8c3924afc..67a9b2ea2 100644 --- a/src/mcpl/convert-block.ts +++ b/src/mcpl/convert-block.ts @@ -87,6 +87,11 @@ export function convertBlock(block: McplContentBlock, provenance: string, server return { type: 'text', text: referenceStubOrNull(block, provenance, serverId) ?? '[reference]' }; default: { + // MCP's link to a resource isn't one of MCPL's own blocks, but a server + // can send one, and RFC-005 §8.1 makes it a reference, shown as one. + if ((block as { type?: unknown }).type === 'resource_link') { + return { type: 'text', text: referenceStubOrNull(block, provenance, serverId) ?? '[reference]' }; + } // Unknown wire block types previously fell off the exhaustive switch // and propagated `undefined` into ContentBlock[]. Fail visibly. A type // that isn't a string is no type, and is never converted to one: an diff --git a/src/mcpl/push-coalescer.ts b/src/mcpl/push-coalescer.ts index 0080f48a2..0dcdb2af1 100644 --- a/src/mcpl/push-coalescer.ts +++ b/src/mcpl/push-coalescer.ts @@ -323,7 +323,7 @@ export function validateCoalescedContent( */ function showsAsItself(b: Record): boolean { if (b.type === 'text') return typeof b.text === 'string'; - if (b.type === 'resource') return classifyBlock(b).kind === 'reference'; + if (b.type === 'resource' || b.type === 'resource_link') return classifyBlock(b).kind === 'reference'; if (b.type !== 'image' && b.type !== 'audio') return false; if (isInlineContradiction(b)) return false; if (isFilledString(b.uri) && b.disposition) return classifyBlock(b).kind === 'reference'; diff --git a/src/mcpl/reference-fetcher.ts b/src/mcpl/reference-fetcher.ts index 3d9a9261e..513ceaaba 100644 --- a/src/mcpl/reference-fetcher.ts +++ b/src/mcpl/reference-fetcher.ts @@ -29,8 +29,10 @@ * that the bytes are wanted). Per-server budgets cap cumulative appetite. */ import { createHash } from 'node:crypto'; -import type { ReferenceRecord } from './references.js'; -import { EXPIRY_REASON, noAddressReason, referenceExpiry, REFERENCE_LIMITS, sanitizeLabel, UNBOUND_REASON } from './references.js'; +import type { ReferenceRecord, Refusal, ServerAddress } from './references.js'; +import { dialCheck, dialServer, noAddressReason, REFERENCE_LIMITS, sanitizeLabel, shownServerId } from './references.js'; + +export { normalizedOrigin, type Refusal } from './references.js'; /** Per-fetch hard ceiling on actual bytes (overridable per server / call). */ export const DEFAULT_FETCH_MAX_BYTES = 256 * 1024 * 1024; @@ -64,15 +66,6 @@ export interface ResolvedServerBinding { * know resolves to undefined. */ export type ServerResolution = ResolvedServerBinding | 'no-address'; -/** Why the fetcher would refuse a reference without dialing. `ofReference` - * marks one of the reference's own phrases that its stub shares (unbound, - * expired), which fetch_reference words as "reference "; it gives - * the others as they are. */ -export interface Refusal { - reason: string; - ofReference: boolean; -} - /** fetch_reference's words for a refusal. */ function refusalText(refusal: Refusal): string { return refusal.ofReference ? `reference ${refusal.reason}` : refusal.reason; @@ -118,19 +111,6 @@ const EXT_BY_MIME = new Map(Object.entries({ 'application/pdf': 'pdf', 'application/zip': 'zip', })); -/** ws→http / wss→https with default ports elided, for origin comparison. */ -export function normalizedOrigin(url: string): string | null { - try { - const u = new URL(url); - const scheme = u.protocol === 'ws:' ? 'http:' : u.protocol === 'wss:' ? 'https:' : u.protocol; - const defaultPort = scheme === 'https:' ? '443' : '80'; - const port = u.port === '' || u.port === defaultPort ? '' : `:${u.port}`; - return `${scheme}//${u.hostname.toLowerCase()}${port}`; - } catch { - return null; - } -} - export class ReferenceFetcher { /** Bytes read through each server in this process, charged to the server a * fetch dials through as each chunk is read, kept or not. Concurrent @@ -140,12 +120,37 @@ export class ReferenceFetcher { private inFlight = new Map>(); constructor( + /** A server's address, credential and limits, by id, from the + * framework's own config: the origin a fetch is checked against and the + * token it carries come from one answer. */ private readonly resolveServer: (serverId: string) => ServerResolution | undefined, /** Persist the payload; returns a mount-prefixed path or null when no * writable workspace exists. File name is host-generated (§7.3). */ private readonly saveFile: (fileName: string, data: Buffer, mimeType: string) => Promise, ) {} + /** How the host reaches a server, as the resolver answers: what dialCheck + * and dialServer read for a fetch. */ + private readonly addressOf = (serverId: string): ServerAddress | undefined => { + const resolution = this.resolveServer(serverId); + return resolution === undefined || resolution === 'no-address' ? resolution : { url: resolution.url }; + }; + + /** A server's credential and limits, for a server dialServer chose. */ + private bindingFor(serverId: string): { serverId: string; binding: ResolvedServerBinding } | { refusal: Refusal } { + const binding = this.resolveServer(serverId); + if (binding === undefined) return { refusal: { reason: `unknown server: ${shownServerId(serverId)}`, ofReference: false } }; + if (binding === 'no-address') return { refusal: { reason: noAddressReason(shownServerId(serverId)), ofReference: false } }; + return { serverId, binding }; + } + + /** The server a fetch of a record dials (§6.2), with its credential and + * limits, or why none can be. */ + private serverFor(record: ReferenceRecord): { serverId: string; binding: ResolvedServerBinding } | { refusal: Refusal } { + const server = dialServer(record, this.addressOf); + return 'refusal' in server ? server : this.bindingFor(server.serverId); + } + /** Would the eager (autofetch) default fetch this record? Not one the fetch * would refuse before dialing, asked as the eager fetch asks it, with the * default ceiling, so such a record takes no eager slot; and only one whose @@ -161,45 +166,22 @@ export class ReferenceFetcher { return record.testimony.sizeBytes !== undefined && record.testimony.sizeBytes <= cap; } - /** The server to dial for a record (§6.2): the first, in the order they - * delivered it, whose address is on the uri's origin. When none is, the - * first server's refusal (unknown to this host, with no address to dial, - * or the origin check's), and for a record no server delivered, that it's - * unbound. */ - private serverFor(record: ReferenceRecord): { serverId: string; binding: ResolvedServerBinding } | { refusal: Refusal } { - let refusal: Refusal | undefined; - for (const serverId of record.serverIds) { - const binding = this.resolveServer(serverId); - if (!binding) { - refusal ??= { reason: `unknown server: ${serverId}`, ofReference: false }; - continue; - } - if (binding === 'no-address') { - refusal ??= { reason: noAddressReason(serverId), ofReference: false }; - continue; - } - const offOrigin = this.originCheck(record, binding); - if (offOrigin === null) return { serverId, binding }; - refusal ??= { reason: offOrigin, ofReference: false }; - } - return { refusal: refusal ?? { reason: UNBOUND_REASON, ofReference: true } }; - } - /** Every check the fetch makes before it dials, in its order: the server - * (none, unknown, no address, off the uri's origin), the expiry, the - * server's spent budget, a claimed size over the ceiling. For a record - * with no saved copy and no fetch under way, the fetch, `refusal()` and - * the eager check all ask this, so what save_recent_image says is the - * reason the fetch would refuse with, and no eager slot goes to a fetch - * that would be refused. */ + * (none, unknown, no address, off the uri's origin) and the expiry, which + * dialCheck makes as a reference's stub does; then the server's spent + * budget, a claimed size over the ceiling. For a record with no saved + * copy and no fetch under way, the fetch, `refusal()` and the eager + * check all ask this, so what save_recent_image says is the reason the + * fetch would refuse with, and no eager slot goes to a fetch that would + * be refused. */ private beforeDial(record: ReferenceRecord, opts?: { maxBytes?: number }): { refusal: Refusal } | { serverId: string; binding: ResolvedServerBinding; ceiling: number; budget: number } { - const server = this.serverFor(record); + const dial = dialCheck(record, this.addressOf); + if ('refusal' in dial) return dial; + const server = this.bindingFor(dial.serverId); if ('refusal' in server) return server; const { serverId, binding } = server; const t = record.testimony; - const expiry = referenceExpiry(t); - if (expiry) return { refusal: { reason: EXPIRY_REASON[expiry], ofReference: true } }; const ceiling = opts?.maxBytes ?? DEFAULT_FETCH_MAX_BYTES; // The budget is checked before the ceiling, here and while reading: a @@ -239,23 +221,6 @@ export class ReferenceFetcher { return 'refusal' in check ? check.refusal : null; } - /** §6.1/§7.1 policy check; returns a refusal reason or null. */ - private originCheck(record: ReferenceRecord, binding: ResolvedServerBinding): string | null { - const refOrigin = normalizedOrigin(record.testimony.uri); - const dialedOrigin = normalizedOrigin(binding.url); - if (!refOrigin) return 'reference uri does not parse'; - if (!dialedOrigin) return 'server url does not parse'; - const scheme = refOrigin.split('//')[0]; - if (scheme !== 'https:' && scheme !== 'http:') return `scheme not allowed: ${scheme}`; - if (refOrigin !== dialedOrigin) { - return `third-party origin (${refOrigin} != dialed ${dialedOrigin}); not fetched by this host`; - } - if (scheme === 'http:' && dialedOrigin.startsWith('https:')) { - return 'plaintext reference from a secure origin'; - } - return null; - } - /** Fetch a reference to workspace storage. Idempotent per record; concurrent * callers share one in-flight fetch. */ fetch(record: ReferenceRecord, opts?: { maxBytes?: number; timeoutMs?: number }): Promise { diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index e37bab3f0..5c1b86839 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -49,6 +49,10 @@ export interface ReferenceTestimony { disposition?: ReferenceDisposition; rejectedFields: string[]; truncatedFields: string[]; + /** Set when the testimony is an MCP resource_link's, which claims no + * digest, expiry or disposition. A link is registered but never fetched + * on the side, and its testimony never replaces a record's (register). */ + fromLink?: true; } export type BlockClassification = @@ -92,29 +96,35 @@ export function classifyBlock(block: unknown): BlockClassification { : { kind: 'invalid', shown: NON_STRING_TEXT_PLACEHOLDER }; case 'image': case 'audio': { - const hasData = typeof block.data === 'string'; - const hasUri = typeof block.uri === 'string'; - if (hasData) { - // Inline form claiming bulk disposition (or smuggling a uri beside + // An empty uri is no link, as the push lane's converter reads it. + const hasLink = typeof block.uri === 'string' && block.uri !== ''; + if (typeof block.data === 'string') { + // Inline form claiming bulk disposition (or smuggling a link beside // the data) is the vector-2 contradiction: fail closed, withhold. - // A uri of '' counts here too, though it's no link: until the - // tool-result lane gives every inline shape a marker of its own, - // showing that data would put some of it into history as raw JSON. - // Once that lane does, an empty link beside data reads as no link. - return { kind: 'inline', contradiction: block.disposition !== undefined || hasUri }; + return { kind: 'inline', contradiction: block.disposition !== undefined || hasLink }; } - // With no data, an empty uri is no link, as the push lane's converter - // reads it. - if (hasUri && block.uri !== '') return parseReferenceFields(block); + if (hasLink) return parseReferenceFields(block); return { kind: 'invalid', shown: block.type === 'image' ? IMAGE_NO_DATA_TEXT : AUDIO_NO_DATA_TEXT }; } case 'resource': return parseReferenceFields(block); + case 'resource_link': + // MCP's link to a resource is a reference too. RFC-005 §8.1 maps it + // onto the reference record field for field, its `size` as `sizeBytes`, + // with no disposition: the host's default policy, as for any reference + // that claims none. Its title, description and annotations aren't + // testimony. + return linkTestimony(parseReferenceFields({ uri: block.uri, name: block.name, mimeType: block.mimeType, sizeBytes: block.size })); default: return { kind: 'invalid', shown: unknownBlockText(block) }; } } +/** A link's classification, its testimony marked as a link's (fromLink). */ +function linkTestimony(c: BlockClassification): BlockClassification { + return c.kind === 'reference' ? { kind: 'reference', testimony: { ...c.testimony, fromLink: true } } : c; +} + function parseReferenceFields(block: Record): BlockClassification { if (typeof block.uri !== 'string' || block.uri.length === 0) { return { kind: 'invalid', shown: REFERENCE_NO_LINK_TEXT }; @@ -222,21 +232,225 @@ export const EXPIRY_REASON = { * only through a server that delivered the reference. */ export const UNBOUND_REASON = "not tied to any server, so there's nowhere to fetch it from"; -/** The fetcher's refusal for a server it has no address to dial, one the - * host doesn't connect to over a network, such as a stdio server; the - * fetcher dials only through a server's address. save_recent_image gives it - * too. A stub can't know it, since only the framework holds its servers' - * configs. */ +/** The refusal for a server the host has no address to dial, one it doesn't + * connect to over a network, such as a stdio server: the fetcher dials only + * through a server's address. A reference's stub, fetch_reference and + * save_recent_image all give it (dialCheck). */ export function noAddressReason(serverId: string): string { return `no address: this host doesn't connect to server ${serverId} over a network, so there's nowhere to fetch it from`; } -/** Strip control characters (C0, DEL, C1) and every bidi control (marks, - * embeddings, overrides, isolates, and the Arabic letter mark), trim the - * whitespace (a BOM included) left at either end, and bound the length. - * Labels are labels — storage naming is host-generated elsewhere (§7.3). */ +/** A server's id as the agent reads it in a stub or a refusal: as a label + * is, bounded and with its controls removed. */ +export function shownServerId(serverId: string): string { + return sanitizeLabel(serverId, STUB_FIELD_CHARS); +} + +/** + * Query and fragment parameters that carry a credential, by name, compared + * by their letters and digits alone, without letter case, so `access_token`, + * `accessToken`, `access-token` and `access_token[]` are one name. RFC-005 §5: "Signed URLs and query capabilities are + * bearer credentials that look like locations", so a uri is shown only where + * the host's policy classifies it as non-capability-bearing, which here is + * one with no userinfo and none of these parameters (uriMayCarryCredential). + * For example: + * - AWS's signed URLs: `https://b.s3.amazonaws.com/k?X-Amz-Algorithm=…&X-Amz-Credential=…&X-Amz-Security-Token=…&X-Amz-Signature=…` + * - Google Cloud Storage's: `https://storage.googleapis.com/b/k?X-Goog-Algorithm=…&X-Goog-Credential=…&X-Goog-Signature=…` + * - Azure's shared access signatures: `https://a.blob.core.windows.net/c/b?sv=…&se=…&sp=r&sig=…` + * - CloudFront's: `https://d1.cloudfront.net/k?Expires=…&Signature=…&Key-Pair-Id=…`, or `?Policy=…&Signature=…&Key-Pair-Id=…` + * - Discord's CDN: `https://cdn.discordapp.com/attachments/1/2/f.png?ex=…&is=…&hm=…` + * - Backblaze B2's: `https://f000.backblazeb2.com/file/b/k?Authorization=…` + * - OAuth's: `#access_token=…&token_type=bearer`, `#id_token=…`, `?refresh_token=…`, `?client_secret=…`, + * and OAuth 1.0a's `?oauth_token=…&oauth_signature=…` + * - the generic: `?token=…`, `?auth=…`, `?key=…`, `?apikey=…`, `?api_key=…`, `?x-api-key=…`, `?private_token=…`, + * `?secret=…`, `?password=…`, `?signature=…` + * A credential named otherwise, or carried in a path, such as a webhook's + * `https://discord.com/api/webhooks//`, isn't caught. + */ +const CREDENTIAL_PARAMETERS: ReadonlySet = new Set([ + 'x-amz-signature', 'x-amz-credential', 'x-amz-security-token', + 'x-goog-signature', 'x-goog-credential', + 'sig', + 'signature', 'key-pair-id', 'policy', + 'hm', + 'authorization', + 'access_token', 'id_token', 'refresh_token', 'client_secret', + 'oauth_token', 'oauth_signature', + 'token', 'auth', 'key', 'api_key', 'x-api-key', 'private_token', 'secret', 'password', +].map(credentialName)); + +/** A parameter's name as CREDENTIAL_PARAMETERS compares it: its letters and + * digits alone, in lower case. */ +function credentialName(name: string): string { + return name.toLowerCase().replace(/[^a-z0-9]/g, ''); +} + +/** + * Whether a uri may carry a credential: userinfo, or a query or fragment + * parameter named on CREDENTIAL_PARAMETERS with a value, a fragment read as + * `name=value&…` as a query is. A name with no value carries nothing, as an + * anchor (`#token`) or `?token=` doesn't. A `?` or `;` inside either + * separates parameters too, as in a route kept in a fragment + * (`#/reset?token=…`). A uri that doesn't parse, even as a relative + * reference, may, since its parts can't be read. + */ +export function uriMayCarryCredential(uri: string): boolean { + let u: URL; + try { + // Absolute first: against an http base, `http:user:pass@host` would read + // as a relative path and hide its userinfo. + u = new URL(uri); + } catch { + try { + u = new URL(uri, 'http://relative.invalid/'); + } catch { + return true; + } + } + if (u.username !== '' || u.password !== '') return true; + const named = (params: string) => + [...new URLSearchParams(params.replace(/[?;]/g, '&')).entries()] + .some(([name, value]) => value !== '' && CREDENTIAL_PARAMETERS.has(credentialName(name))); + return named(u.search) || named(u.hash.slice(1)); +} + +/** How the host reaches a server, as far as fetching its references goes: + * the url it dials the server at, or none, for a server it doesn't connect + * to over a network, such as a stdio server. A framework records each of its + * servers' in the registry (ReferenceRegistry.setServerAddress), where a + * stub reads it; its fetcher reads the same from the framework's own + * configs, beside each server's credential. */ +export type ServerAddress = { url: string } | 'no-address'; + +/** Why a fetch of a reference would be refused before it dials. `ofReference` + * marks one of the reference's own phrases (unbound, expired), which + * fetch_reference words as "reference "; it gives the others as + * they are. `inStub`, where it's set, is how the reference's stub words it, + * without the parts of the uri the stub never shows. */ +export interface Refusal { + reason: string; + ofReference: boolean; + inStub?: string; +} + +/** ws→http / wss→https with default ports elided, for origin comparison. */ +export function normalizedOrigin(url: string): string | null { + try { + const u = new URL(url); + const scheme = u.protocol === 'ws:' ? 'http:' : u.protocol === 'wss:' ? 'https:' : u.protocol; + const defaultPort = scheme === 'https:' ? '443' : '80'; + const port = u.port === '' || u.port === defaultPort ? '' : `:${u.port}`; + return `${scheme}//${u.hostname.toLowerCase()}${port}`; + } catch { + return null; + } +} + +/** Whether two urls that parse name one host, on the same port, explicit or + * each its scheme's default: `http://h` and `wss://h`, or `http://h:443` and + * `wss://h`. */ +function samePlace(a: string, b: string): boolean { + const x = new URL(a); + const y = new URL(b); + const port = (u: URL) => u.port || (u.protocol === 'https:' || u.protocol === 'wss:' ? '443' : '80'); + return x.hostname.toLowerCase() === y.hostname.toLowerCase() && (x.port === y.port || port(x) === port(y)); +} + +/** The §6.1/§7.1 policy check of a uri against the url a server is dialed + * at: null where a fetch through that server may fetch it. */ +function originRefusal(uri: string, serverUrl: string): Refusal | null { + const refOrigin = normalizedOrigin(uri); + const dialedOrigin = normalizedOrigin(serverUrl); + if (!refOrigin) return { reason: 'reference uri does not parse', ofReference: false }; + if (!dialedOrigin) return { reason: 'server url does not parse', ofReference: false }; + const scheme = refOrigin.split('//')[0]; + if (scheme !== 'https:' && scheme !== 'http:') { + return { + reason: `scheme not allowed: ${sanitizeLabel(scheme, STUB_FIELD_CHARS)}`, + ofReference: false, + inStub: 'scheme not allowed: this host fetches only http and https uris', + }; + } + // An http uri on the host of a server dialed over a secure connection, on + // the same port or each on its scheme's default, before the origins are + // compared, which would call it a third party's. + if (scheme === 'http:' && dialedOrigin.startsWith('https:') && samePlace(uri, serverUrl)) { + return { reason: 'plaintext reference from a secure origin', ofReference: false }; + } + if (refOrigin !== dialedOrigin) { + return { + reason: `third-party origin (${refOrigin} != dialed ${dialedOrigin}); not fetched by this host`, + ofReference: false, + inStub: "third-party origin: it isn't on its server's own address, so this host doesn't fetch it", + }; + } + return null; +} + +/** + * The server a fetch of a reference would dial: its servers, in the order + * they delivered it, the first whose address is on the uri's origin (§6.2). + * When none is, the first server's refusal (unknown to this host, with no + * address, or the origin check's), and for a record no server delivered, + * that it's unbound. + */ +export function dialServer( + record: ReferenceRecord, + addressOf: (serverId: string) => ServerAddress | undefined, +): { serverId: string } | { refusal: Refusal } { + let refusal: Refusal | undefined; + let dialed: string | undefined; + for (const serverId of record.serverIds) { + const address = addressOf(serverId); + const shownId = shownServerId(serverId); + if (address === undefined) { + refusal ??= { reason: `unknown server: ${shownId}`, ofReference: false }; + } else if (address === 'no-address') { + refusal ??= { reason: noAddressReason(shownId), ofReference: false }; + } else { + const offOrigin = originRefusal(record.testimony.uri, address.url); + if (offOrigin === null) { + dialed = serverId; + break; + } + refusal ??= offOrigin; + } + } + return dialed === undefined ? { refusal: refusal ?? { reason: UNBOUND_REASON, ofReference: true } } : { serverId: dialed }; +} + +/** + * Where a fetch of a reference would dial, or why it would be refused before + * dialing, whatever the fetch: dialServer's server, then the reference's + * expiry. A pure function of the record and the servers' addresses: a + * reference's stub asks it of the registry's (unfetchableReason), and the + * fetcher's pre-dial check of its own framework's configs, whose credential + * goes with the address it checks. In a process with one framework, or with + * frameworks that agree on each id they share, the two sources hold the same + * addresses, so a stub promises what the fetch does. Where two frameworks + * configure one id differently, only a stub's words can be wrong, never where + * a token goes. A fetch's own limits, its budget and ceiling, come after it. + */ +export function dialCheck( + record: ReferenceRecord, + addressOf: (serverId: string) => ServerAddress | undefined, + nowMs = Date.now(), +): { serverId: string } | { refusal: Refusal } { + const server = dialServer(record, addressOf); + if ('refusal' in server) return server; + const expiry = referenceExpiry(record.testimony, nowMs); + return expiry ? { refusal: { reason: EXPIRY_REASON[expiry], ofReference: true } } : server; +} + +/** Strip control characters (C0, DEL, C1), every bidi control (marks, + * embeddings, overrides, isolates, and the Arabic letter mark) and the line + * and paragraph separators (U+2028, U+2029), which a regex reads as line + * ends, so a label can't start a line of its own, such as one the image + * ledger would read as a slot. Then trim the whitespace (a BOM included) + * left at either end, and bound the length. Labels are labels — storage + * naming is host-generated elsewhere (§7.3). */ export function sanitizeLabel(s: string, maxChars: number): string { - const cleaned = s.replace(/[\p{Cc}\p{Bidi_Control}]/gu, '').trim(); + const cleaned = s.replace(/[\p{Cc}\p{Bidi_Control}\u2028\u2029]/gu, '').trim(); if (cleaned.length <= maxChars) return cleaned; let cut = cleaned.slice(0, Math.max(0, maxChars - 1)); // never split a surrogate pair at the truncation boundary @@ -318,8 +532,9 @@ export class ReferenceRegistry { // Refresh testimony while unfetched: a re-issued reference with a new // digest/expiry supersedes the stale claims. Once fetched, the record // describes verified bytes on disk and keeps the testimony they were - // verified against. - if (!existing.fetchedPath) existing.testimony = testimony; + // verified against. A link claims no digest, expiry or disposition, + // so its testimony never replaces a record's claims. + if (!existing.fetchedPath && !testimony.fromLink) existing.testimony = testimony; // Newest again, so a uri just mentioned is the last to be evicted. this.records.delete(existingId); this.records.set(existingId, existing); @@ -345,6 +560,36 @@ export class ReferenceRegistry { return this.records.get(refId); } + /** How the host reaches each server, by id, as each framework in the process + * configured it, which a reference's stub reads (unfetchableReason). A + * fetch reads its own framework's configs instead, so a server's token only + * ever goes to the origin its own config names. Keyed by id as a record's + * servers are: where two frameworks configure one id, the latest entry is + * in force, and a stub's words can be wrong for the other one's fetch. */ + private addresses = new Map>(); + + /** Record how the host reaches a server, as a framework (`owner`) does for + * each server it configures. Its entry for the id is then in force. */ + setServerAddress(serverId: string, address: ServerAddress, owner: object): void { + const entries = (this.addresses.get(serverId) ?? []).filter((e) => e.owner !== owner); + entries.push({ owner, address }); + this.addresses.set(serverId, entries); + } + + /** Take back a framework's entry for a server, as it does for a server it + * removes. Another framework's entry for the id, if any, is in force + * again; with none, the id's references are refused as an unknown + * server's, as that framework's fetch refuses them. */ + forgetServerAddress(serverId: string, owner: object): void { + const entries = (this.addresses.get(serverId) ?? []).filter((e) => e.owner !== owner); + if (entries.length > 0) this.addresses.set(serverId, entries); + else this.addresses.delete(serverId); + } + + serverAddress(serverId: string): ServerAddress | undefined { + return this.addresses.get(serverId)?.at(-1)?.address; + } + /** Find the record for a uri. */ findByUri(uri: string): ReferenceRecord | undefined { const id = this.byKey.get(uri); @@ -365,13 +610,15 @@ export function referenceTypeLabel(mimeType: string | undefined): string { } /** - * The longest line buildReferenceStub writes, in UTF-16 code units, from its + * The longest stub buildReferenceStub writes, in UTF-16 code units, from its * own bounds: the longest id the registry mints, then the name, the type with * the largest size it shows, and the origin, each at its bound, and the * longest ending, a saved copy's path, with their separators. The other * endings, the fetch hint and the reasons a fetch would be refused, are * shorter. A line shaped like a stub but longer than this is quoted text the - * writer can't have written, and no image slot (tool-image-ledger.ts). + * writer can't have written, and no image slot (tool-image-ledger.ts). A + * tool result's link line (StubOptions.unfetchedLinkUri) isn't shaped like a + * stub, so it's no slot whatever its length. */ export const REFERENCE_STUB_MAX_CHARS = `[ref_${MAX_SAFE.toString(36)}_] `.length + REF_ID_SUFFIX_CHARS @@ -380,14 +627,31 @@ export const REFERENCE_STUB_MAX_CHARS = + ' — '.length + STUB_FIELD_CHARS + ' — '.length + 'saved: '.length + SAVED_PATH_CHARS; +/** How a lane shows a reference beyond its stub. */ +export interface StubOptions { + /** + * Show a link (MCP's resource_link) the host can't fetch as its name, uri + * and type, unless its uri may carry a credential (uriMayCarryCredential). + * For a tool result's links, whose JSON showed every link's uri before + * this host read links as references: there the uri is the agent's only + * handle on a link the host can't fetch, and showing it is §5's MAY for a + * reference the host's policy classifies as non-capability-bearing. No + * other lane showed a link's uri, so none shows one now. + */ + unfetchedLinkUri?: boolean; +} + /** - * Build the model-visible stub for a reference. Output length is bounded by - * host constants and independent of every server-supplied field length. - * Under `disposition:"never"` the uri is omitted unconditionally; under - * `ref`/absent it is omitted too — this host's default policy is the opaque - * id everywhere (the safe default of §5), with retrieval via the record. + * Build the model-visible stub for a reference. Its length is bounded by + * host constants and independent of every server-supplied field's length: + * REFERENCE_STUB_MAX_CHARS. Under `disposition:"never"` the uri is omitted + * unconditionally; under `ref`/absent it is omitted too — this host's default + * policy is the opaque id everywhere (the safe default of §5), with retrieval + * via the record. The one exception is a tool result's link the host can't + * fetch (StubOptions.unfetchedLinkUri), which is no stub but a line of its + * own, its uri bounded by REFERENCE_LIMITS.uri. */ -export function buildReferenceStub(t: ReferenceTestimony, provenance?: string, serverId?: string): string { +export function buildReferenceStub(t: ReferenceTestimony, provenance?: string, serverId?: string, options?: StubOptions): string { const record = referenceRegistry.register(t, serverId); const parts: string[] = []; // A label is shown only when something is left of it once sanitized. @@ -405,29 +669,46 @@ export function buildReferenceStub(t: ReferenceTestimony, provenance?: string, s // workspace path (host-generated, §7.3) so existing tools just work. parts.push(`saved: ${sanitizeLabel(record.fetchedPath, SAVED_PATH_CHARS)}`); } else { - // A reference whose record shows the fetcher would refuse it says why, - // rather than inviting a fetch_reference call. - parts.push(unfetchableReason(record) ?? 'fetch with fetch_reference'); + // A reference the fetcher would refuse says why, rather than inviting a + // fetch_reference call. + const refusal = unfetchableReason(record); + if (refusal !== null && t.fromLink && options?.unfetchedLinkUri && record.testimony.disposition !== 'never') { + // The credential check reads the uri as it would be shown, with what + // sanitizeLabel strips gone, so a uri split by a control character is + // read as it would read: `ht\u0000tps://user:pass@…` parses as a path, + // and shows as userinfo. A reference's disposition:"never" for the uri + // keeps it out whatever the link says. + const uri = sanitizeLabel(t.uri, REFERENCE_LIMITS.uri); + if (!uriMayCarryCredential(uri)) return unfetchedLinkText(t, uri); + } + parts.push(refusal ?? 'fetch with fetch_reference'); } return `[${record.refId}] ${parts.join(' — ') || 'referenced content'}`; } /** - * Why the fetcher would refuse a reference, as far as its record alone shows, - * for its stub: no server registered for it, which the fetcher checks first, - * or its expiry. Null for a saved copy, which the fetcher serves before any - * check, and when the record shows neither. The fetcher's other refusals rest - * on what a record doesn't hold: whether any of its servers is known to the - * host, has an address and is on the uri's origin, the dialed server's fetch - * budget spent, or a claimed size over the ceiling a fetch is given. The - * fetcher gives those in its own words, and save_recent_image, in the - * framework where it can ask the fetcher, says them too. + * What a tool result's link the host can't fetch shows, where its uri can't + * carry a credential: its name, its uri and its type, as a standard MCP + * server's link shows (StubOptions.unfetchedLinkUri). + */ +function unfetchedLinkText(t: ReferenceTestimony, uri: string): string { + const name = t.name ? sanitizeLabel(t.name, STUB_FIELD_CHARS) : ''; + const mimeType = referenceTypeLabel(t.mimeType); + return `[resource link${name ? ` "${name}"` : ''}: ${uri}${mimeType ? `, ${mimeType}` : ''}; not fetched]`; +} + +/** + * Why the fetcher would refuse a reference before it dials, whatever the + * fetch, in its stub's words: the registry's dialCheck, from its record and + * its servers' addresses. Null for a saved copy, which the fetcher serves + * before any check, and where a fetch would dial. A fetch's own limits, the + * server's spent budget or a claimed size over the ceiling it's given, the + * fetcher gives in its own words, and save_recent_image says them too. */ export function unfetchableReason(record: ReferenceRecord, nowMs = Date.now()): string | null { if (record.fetchedPath) return null; - if (record.serverIds.length === 0) return UNBOUND_REASON; - const expiry = referenceExpiry(record.testimony, nowMs); - return expiry ? EXPIRY_REASON[expiry] : null; + const dial = dialCheck(record, (serverId) => referenceRegistry.serverAddress(serverId), nowMs); + return 'refusal' in dial ? dial.refusal.inStub ?? dial.refusal.reason : null; } /** @@ -435,13 +716,13 @@ export function unfetchableReason(record: ReferenceRecord, nowMs = Date.now()): * lanes. Returns a replacement string, or null when the block should keep * its existing (inline/text) handling. */ -export function referenceStubOrNull(block: unknown, provenance?: string, serverId?: string): string | null { +export function referenceStubOrNull(block: unknown, provenance?: string, serverId?: string, options?: StubOptions): string | null { const c = classifyBlock(block); switch (c.kind) { case 'text': return null; case 'inline': return c.contradiction ? INLINE_WITHHELD_TEXT : null; - case 'reference': return buildReferenceStub(c.testimony, provenance, serverId); + case 'reference': return buildReferenceStub(c.testimony, provenance, serverId, options); case 'invalid': return c.shown; } } diff --git a/src/module-registry.ts b/src/module-registry.ts index 6f5019be6..6ed3745ac 100644 --- a/src/module-registry.ts +++ b/src/module-registry.ts @@ -846,7 +846,7 @@ class ProcessStateImpl implements ProcessState { } /** A result's flags, which consumers read for their truth: booleans here. */ -const RESULT_FLAGS = ['success', 'isError', 'endTurn'] as const; +const RESULT_FLAGS = ['success', 'isError', 'endTurn', 'dataIsContent'] as const; /** * A module's tool result as the framework can store, send and trace it. A @@ -916,7 +916,7 @@ export function checkedToolResult(call: ToolCall, result: unknown): { result: To endTurn = Boolean(fields.endTurn); read('error'); ownText = errorText(fields.error); - for (const key of ['success', 'isError', 'data']) read(key); + for (const key of ['success', 'isError', 'data', 'dataIsContent']) read(key); for (const key of Object.keys(candidate)) { if (key !== 'toJSON' && !(key in fields)) fields[key] = candidate[key]; } diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 107e46538..e69564d0f 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -9,28 +9,123 @@ * `maxChars` slice and re-introduces, one turn deferred, exactly the silent * hallucination this feature exists to kill. * - * For MCP-shaped content arrays, this helper keeps text blocks verbatim and - * replaces image blocks with a short `[image: mimeType, ~NKB]` placeholder. - * For anything else it falls back to JSON, written by RFC-005's rules at - * every depth (historyJson): a reference is shown as its stub wherever it - * sits in the data, so its uri never reaches history, and inline data that - * claims a uri or a disposition is withheld. + * A content array (toolResultBlocks) is shown block by block, each block as + * both copies show it: text verbatim, an image as a short + * `[image: mimeType, ~NKB]` placeholder, a reference as its stub, and any + * other block as a marker saying what it was and why it isn't shown. Anything + * else falls back to JSON, written by RFC-005's rules at every depth + * (historyJson): a reference is shown as its stub wherever it sits in the + * data, so its uri never reaches history, and inline data that claims a uri + * or a disposition is withheld. */ -import { INLINE_WITHHELD_TEXT, isInlineContradiction, referenceStubOrNull } from './mcpl/references.js'; -import { NON_STRING_TEXT_PLACEHOLDER } from './mcpl/visible-content.js'; +import { + AUDIO_NO_DATA_TEXT, + AUDIO_NO_MEDIA_TYPE_TEXT, + IMAGE_NO_DATA_TEXT, + IMAGE_NO_MEDIA_TYPE_TEXT, + INLINE_WITHHELD_TEXT, + buildReferenceStub, + classifyBlock, + referenceStubOrNull, + referenceTypeLabel, +} from './mcpl/references.js'; import { formatToolImagePlaceholder, toolResultCutAt } from './tool-image-ledger.js'; /** Where one block of a content array sits in its history text, and what * kind of block it was: text, an inline image (its placeholder), an RFC-005 - * reference (its stub), or inline data withheld for a contradiction. */ + * reference (its stub), inline data withheld for a contradiction, or a + * marker in place of a block the lane can't show. */ export interface HistoryBlockSpan { - kind: 'text' | 'image' | 'reference' | 'withheld'; + kind: ToolResultBlock['kind']; start: number; end: number; } +/** + * One block of a tool result's content array as both of its copies show it + * (toolResultBlocks): a text block's text, an inline image, which the stored + * copy shows as its placeholder and the live copy as itself, or the text that + * stands in its place, the same in both. That text is a reference's stub, + * inline data withheld for a contradiction, or a marker saying what kind of + * block it was and why the lane can't show it. + */ +export type ToolResultBlock = + | { kind: 'text'; text: string } + | { kind: 'image'; data: string; mimeType: string } + | { kind: 'reference' | 'withheld' | 'marker'; text: string }; + +/** The block types MCP defines for a tool result's content. */ +const MCP_BLOCK_TYPES = new Set(['text', 'image', 'audio', 'resource', 'resource_link']); + +/** + * Each element of `data` as both copies of a tool result show it, or null + * when `data` isn't a content array, so the result is data, written as JSON. + * An array is content when the result says so (`dataIsContent`, which the + * framework sets on an MCPL tool's result: every element of a protocol + * result's content is a block, of a type the host shows or not). Otherwise + * it's content only when every element is a block of one of MCP's own types, + * so a list of records of other types stays data. A module's array of + * records that all carry MCP's type names, such as `[{ type: 'image', url }]`, + * is read as content, and shows a marker for each record that isn't a block + * the lane can show. + */ +export function toolResultBlocks(data: unknown, dataIsContent?: boolean): ToolResultBlock[] | null { + if (!Array.isArray(data)) return null; + if (dataIsContent !== true && !data.every((element) => isPlainRecord(element) && MCP_BLOCK_TYPES.has(element.type as string))) { + return null; + } + return data.map(toolResultBlock); +} + +/** One element of a content array, as toolResultBlocks gives it. Only an + * image with data and a media type is shown as itself, as the push lane's + * converter shows one, and only text and images can be: this host's tool + * results carry nothing else, so inline audio is a marker here. */ +function toolResultBlock(raw: unknown): ToolResultBlock { + const c = classifyBlock(raw); + switch (c.kind) { + case 'text': + return { kind: 'text', text: (raw as { text: string }).text }; + case 'reference': + // A bounded stub, never the block's JSON, which would show its uri. A + // stub that shows an image type occupies a slot in save_recent_image's + // inventory (a failing one: its bytes are behind fetch_reference), so + // the index never slides past an image the resident saw. A link the + // host can't fetch, shown by its uri, is no slot, as no link was before + // links were references, when the result's JSON showed it. + return { kind: 'reference', text: buildReferenceStub(c.testimony, 'from tool result', undefined, { unfetchedLinkUri: true }) }; + case 'invalid': + return { kind: 'marker', text: c.shown }; + case 'inline': { + if (c.contradiction) return { kind: 'withheld', text: INLINE_WITHHELD_TEXT }; + const b = raw as { type: 'image' | 'audio'; data: string; mimeType?: unknown }; + const image = b.type === 'image'; + if (b.data === '') return { kind: 'marker', text: image ? IMAGE_NO_DATA_TEXT : AUDIO_NO_DATA_TEXT }; + if (typeof b.mimeType !== 'string' || b.mimeType === '') { + return { kind: 'marker', text: image ? IMAGE_NO_MEDIA_TYPE_TEXT : AUDIO_NO_MEDIA_TYPE_TEXT }; + } + if (image) return { kind: 'image', data: b.data, mimeType: b.mimeType }; + return { kind: 'marker', text: toolResultAudioText(b.mimeType, b.data) }; + } + } +} + +/** What a tool result's inline audio shows as: its type and size, and that + * this host's tool results carry only text and images. */ +function toolResultAudioText(mimeType: string, data: string): string { + const what = [referenceTypeLabel(mimeType), formatSize(Math.floor(data.length * 3 / 4))].filter(Boolean).join(', '); + return `[Audio not shown: this host's tool results carry only text and images (${what}).]`; +} + export interface HistorySerializeOptions { + /** + * Whether the result's data is a protocol result's content + * (ToolResult.dataIsContent), so an array is shown block by block whatever + * its blocks' types (toolResultBlocks). nonErrorResultText takes it from + * the result instead, whatever this says. + */ + dataIsContent?: boolean; /** * Called for every image block the serializer replaces with a placeholder * (issue #104). Returns the retention ref to embed in the placeholder — @@ -86,9 +181,11 @@ export function toolResultDataToHistoryString( * * Each field is read once: the direct Agent API passes its caller's own * result here, unchecked, and a getter needn't answer the same way twice. + * Whether an array is content is the result's own `dataIsContent`, read for + * its truth as a result's flags are, whatever `options` says. */ export function nonErrorResultText( - result: { data?: unknown; error?: string }, + result: { data?: unknown; error?: string; dataIsContent?: boolean }, maxChars?: number, options?: HistorySerializeOptions, ): string { @@ -97,7 +194,7 @@ export function nonErrorResultText( const error = result.error; if (typeof error === 'string' && error) return maxChars ? truncateForHistory(error, maxChars) : error; } - return toolResultDataToHistoryString(data, maxChars, options); + return toolResultDataToHistoryString(data, maxChars, { ...options, dataIsContent: Boolean(result.dataIsContent) }); } /** Bounded copy of an arbitrary string with the standard truncation notice, @@ -113,44 +210,20 @@ export function truncateForHistory(str: string, maxChars: number): string { } /** - * If `data` is an MCP content array whose every block has a shape we - * recognize, return a history-safe string (images → placeholders). Otherwise - * return `null` to defer to the fallback, historyJson, which shows a - * reference anywhere in the data as its stub. + * A content array's history text (toolResultBlocks): each block's text, an + * image as its placeholder, joined by newlines. Null when `data` isn't a + * content array, for the fallback, historyJson. */ function tryHistoryStringFromContentArray(data: unknown, options?: HistorySerializeOptions): string | null { - if (!Array.isArray(data)) return null; - const parts: Array<{ kind: HistoryBlockSpan['kind']; text: string }> = []; - for (const [blockIndex, raw] of data.entries()) { - if (!raw || typeof raw !== 'object') return null; - const b = raw as { type?: unknown; text?: unknown; data?: unknown; mimeType?: unknown }; - if (b.type === 'text') { - // A text that isn't a string shows as the converters' placeholder, - // rather than sending the whole array to the JSON fallback. - parts.push({ kind: 'text', text: typeof b.text === 'string' ? b.text : NON_STRING_TEXT_PLACEHOLDER }); - } else if (isInlineContradiction(b)) { - // RFC-005 vector 2: checked before the data branch so inline data - // claiming bulk disposition is withheld, not rendered. - parts.push({ kind: 'withheld', text: INLINE_WITHHELD_TEXT }); - } else if (b.type === 'image' && typeof b.data === 'string' && typeof b.mimeType === 'string') { - // Decoded byte estimate from base64 length (3/4 ratio, rounded). - const approxBytes = Math.floor(b.data.length * 3 / 4); - const ref = options?.imageRef?.(blockIndex, { data: b.data, mimeType: b.mimeType }) ?? null; - parts.push({ kind: 'image', text: formatToolImagePlaceholder(b.mimeType, formatSize(approxBytes), ref) }); - } else if (b.type === 'resource' || ((b.type === 'image' || b.type === 'audio') && typeof (b as { uri?: unknown }).uri === 'string')) { - // RFC-005 reference block: a bounded stub, never the raw block JSON - // (which used to inline the URI — and the whole array — into history). - // A stub that shows an image type occupies a slot in - // save_recent_image's inventory (a failing one — bytes live behind - // fetch_reference), so the index never slides past an image the - // resident saw. - const stub = referenceStubOrNull(b, 'from tool result'); - if (stub === null) return null; - parts.push({ kind: 'reference', text: stub }); - } else { - return null; - } - } + const blocks = toolResultBlocks(data, options?.dataIsContent); + if (blocks === null) return null; + const parts = blocks.map((block, blockIndex): { kind: HistoryBlockSpan['kind']; text: string } => { + if (block.kind !== 'image') return block; + // Decoded byte estimate from base64 length (3/4 ratio, rounded). + const approxBytes = Math.floor(block.data.length * 3 / 4); + const ref = options?.imageRef?.(blockIndex, { data: block.data, mimeType: block.mimeType }) ?? null; + return { kind: 'image', text: formatToolImagePlaceholder(block.mimeType, formatSize(approxBytes), ref) }; + }); if (options?.onBlocks) { const spans: HistoryBlockSpan[] = []; let start = 0; @@ -166,15 +239,16 @@ function tryHistoryStringFromContentArray(data: unknown, options?: HistorySerial /** * Whether RFC-005 governs a value of a tool result's data, wherever it sits. * A reference is a `resource` with a uri, at its top level or under - * `resource` (where MCP embeds a resource's uri), or an `image` or `audio` - * given by uri; inline data is an `image` or `audio` whose data is a string. - * A uri that is null, absent or empty counts as none, and data that isn't a - * string isn't inline data, so a record that only shares one of those type - * names is data. A uri of any other value, string or not, counts, so a - * credential fails closed. + * `resource` (where MCP embeds a resource's uri), an `image` or `audio` + * given by uri, or MCP's `resource_link` (RFC-005 §8.1); inline data is an + * `image` or `audio` whose data is a string. A uri that is null, absent or + * empty counts as none, and data that isn't a string isn't inline data, so a + * record that only shares one of those type names is data. A uri of any + * other value, string or not, counts, so a credential fails closed. */ function governedByRfc005(b: Record): boolean { if (b.type === 'resource') return hasUri(b.uri) || (isPlainRecord(b.resource) && hasUri(b.resource.uri)); + if (b.type === 'resource_link') return hasUri(b.uri); if (b.type === 'image' || b.type === 'audio') return typeof b.data === 'string' || hasUri(b.uri); return false; } @@ -192,11 +266,11 @@ function isPlainRecord(value: unknown): value is Record { * wherever it sits in the data. A value RFC-005 governs (governedByRfc005) is * shown as the reference lanes show it: a reference as its stub, an invalid * one (such as a uri that isn't a string, or MCP's embedded resource, whose - * uri isn't at its top level) as an unrecognized-block line, and inline data - * that also claims a uri or a disposition as withheld, so no uri is ever - * shown. Inline data that claims neither is itself, as is any other value: - * a block of another type keeps its fields, a `resource_link`'s uri among - * them, since RFC-005 governs references. + * uri isn't at its top level) as the marker saying why it isn't shown, and + * inline data that also claims a uri or a disposition as withheld, so no uri + * is ever shown. Inline data that claims neither is itself, as is any other value: + * a block of another type keeps its fields, since RFC-005 governs + * references. */ function shownInHistory(value: unknown): unknown { if (!value || typeof value !== 'object' || Array.isArray(value)) return value; @@ -208,13 +282,13 @@ function shownInHistory(value: unknown): unknown { // no uri: an object can carry a credential. An empty uri carries none. return hasUri(b.uri) || b.disposition !== undefined ? INLINE_WITHHELD_TEXT : value; } - return referenceStubOrNull(b, 'from tool result') ?? value; + return referenceStubOrNull(b, 'from tool result', undefined, { unfetchedLinkUri: true }) ?? value; } /** * The text RFC-005 shows a value of a tool result's data as, when it shows it * as text rather than as its fields: a reference's stub, an invalid one's - * unrecognized-block line, or inline data withheld. Null for any other value. + * marker, or inline data withheld. Null for any other value. */ export function rfc005TextOf(value: unknown): string | null { const shown = shownInHistory(value); diff --git a/src/types/events.ts b/src/types/events.ts index 976731ff9..10f7e1f33 100644 --- a/src/types/events.ts +++ b/src/types/events.ts @@ -188,6 +188,18 @@ export interface ToolResult { success: boolean; /** Result data (if success) */ data?: unknown; + /** + * Whether `data` is a protocol result's content: an array in which every + * element is a content block, of a type this host can show or not. The + * framework sets it on an MCPL tool's result, and a module that returns a + * content array can set it too. When it's set, each element reaches the + * model as itself, as a reference's stub, or as a marker saying what it was + * and why it isn't shown, never as JSON. When it isn't, an array counts as + * content only when every element is a block of one of MCP's types (text, + * image, audio, resource, resource_link), and any other array is data, + * written as JSON. + */ + dataIsContent?: boolean; /** Error message (if failure) */ error?: string; /** Whether this was an error (for LLM) */ diff --git a/test/mcpl-agent-words.test.ts b/test/mcpl-agent-words.test.ts index e2aa88d72..13bb431e2 100644 --- a/test/mcpl-agent-words.test.ts +++ b/test/mcpl-agent-words.test.ts @@ -59,13 +59,15 @@ test("a reference block that can't be read says which part is missing, and what assert.equal(referenceStubOrNull(block), text); assert.deepEqual(convertBlock(block as never, 'test', 'words'), { type: 'text', text }); } - // Beside inline data, any string uri, an empty one too, stays withheld as - // on main: the tool-result lane can't show every inline shape yet, and - // showing this audio would put its data into history as raw JSON. + // Beside inline data, an empty link is no link either, now that the + // tool-result lane gives every inline shape its marker: this audio shows as + // a tool result's audio does, and its data stays out of history. const audio = { type: 'audio', data: 'aGVsbG8=', mimeType: 'audio/wav', uri: '' }; - assert.deepEqual(classifyBlock(audio), { kind: 'inline', contradiction: true }); + assert.deepEqual(classifyBlock(audio), { kind: 'inline', contradiction: false }); const stored = toolResultDataToHistoryString([{ type: 'text', text: 't' }, audio]); - assert.ok(stored.includes(INLINE_WITHHELD_TEXT) && !stored.includes('aGVsbG8='), stored); + assert.equal(stored, "t\n[Audio not shown: this host's tool results carry only text and images (audio/wav, 6B).]"); + // A link that isn't empty still withholds it. + assert.deepEqual(classifyBlock({ ...audio, uri: 'https://zz.example/a.wav' }), { kind: 'inline', contradiction: true }); }); // --------------------------------------------------------------------------- diff --git a/test/mcpl-reference-binding.test.ts b/test/mcpl-reference-binding.test.ts index c12dc95a6..f7b33d871 100644 --- a/test/mcpl-reference-binding.test.ts +++ b/test/mcpl-reference-binding.test.ts @@ -150,19 +150,22 @@ test("fetch_reference reaches a push event's reference through its server's own dispatchFetchReferenceToolCall(agentName: string, call: { id: string; name: string; input: unknown }): void; }; Object.assign(framework, { - mcplServerConfigs: new Map([ - ['push-srv', { id: 'push-srv', url: wsOrigin }], - ['stdio-srv', { id: 'stdio-srv', command: 'stdio-server' }], - ['spawned-srv', { id: 'spawned-srv', command: 'stdio-server', url: wsOrigin }], - ['ws-command-srv', { id: 'ws-command-srv', transport: 'websocket', command: 'stdio-server', url: wsOrigin }], - ['stdio-url-srv', { id: 'stdio-url-srv', transport: 'stdio', url: wsOrigin }], - ['ws-http-srv', { id: 'ws-http-srv', transport: 'websocket', url: origin }], - ['ws-fragment-srv', { id: 'ws-fragment-srv', url: `${wsOrigin}/mcpl#frag` }], - ]), + mcplServerConfigs: new Map(), getWorkspaceModule: () => ({ writeBinary: async () => ({ success: true }) }), firstWritableMountName: () => 'scratch', emitTrace: () => {}, }); + // Configured as the framework configures a server, its address recorded + // in the reference registry, where the stub and the fetcher read it. + for (const config of [ + { id: 'push-srv', url: wsOrigin }, + { id: 'stdio-srv', command: 'stdio-server' }, + { id: 'spawned-srv', command: 'stdio-server', url: wsOrigin }, + { id: 'ws-command-srv', transport: 'websocket', command: 'stdio-server', url: wsOrigin }, + { id: 'stdio-url-srv', transport: 'stdio', url: wsOrigin }, + { id: 'ws-http-srv', transport: 'websocket', url: origin }, + { id: 'ws-fragment-srv', url: `${wsOrigin}/mcpl#frag` }, + ]) (framework as unknown as { setMcplServerConfig(config: unknown): void }).setMcplServerConfig(config); // Bounded, so a dispatch that never answers fails the test rather than hanging it. const fetchReference = (refId: string): Promise => { let timer: NodeJS.Timeout | undefined; @@ -245,6 +248,11 @@ async function puppetCall(answer: unknown, fetcher: unknown): Promise<{ result: }, getReferenceFetcher: () => fetcher, }); + // eido is dialed on its references' origin, so a stub of one can offer + // the fetch: the reference registry records its address, as the + // framework does for a server it configures. + (framework as unknown as { setMcplServerConfig(config: unknown): void }) + .setMcplServerConfig({ id: 'eido', url: 'wss://editor.example.invalid' }); const quiet = console.log; console.log = () => {}; const { result } = await framework.puppetToolCall('princess', 'mcpl--eido--look', {}) diff --git a/test/mcpl-reference-fields.test.ts b/test/mcpl-reference-fields.test.ts index 265962122..0d7be009d 100644 --- a/test/mcpl-reference-fields.test.ts +++ b/test/mcpl-reference-fields.test.ts @@ -5,6 +5,11 @@ import { referenceRegistry, referenceStubOrNull, UNBOUND_REASON, unfetchableReason, } from '../src/mcpl/references.js'; +// The server these tests bind references to, dialed on their origin, so a +// bound reference's stub can offer the fetch; the framework records each +// server it configures the same way. +referenceRegistry.setServerAddress('srv', { url: 'wss://example.test' }, {}); + const fields = ['name', 'mimeType', 'expiresAt'] as const; const types = ['resource', 'image', 'audio'] as const; function reference(type: string, extra: Record = {}) { diff --git a/test/save-recent-image.test.ts b/test/save-recent-image.test.ts index 3b7eec8b8..04e0d674c 100644 --- a/test/save-recent-image.test.ts +++ b/test/save-recent-image.test.ts @@ -72,8 +72,8 @@ import { createMockResponse, MockMembrane } from './helpers/mock-membrane.js'; * connecting to them: a ws:// or wss:// url for one the host connects to over * a network, a command alone for a stdio server. */ function configureServers(framework: AgentFramework, configs: Array<{ id: string; url?: string; command?: string; autofetch?: { maxTotalBytes?: number } }>): void { - const known = (framework as unknown as { mcplServerConfigs: Map }).mcplServerConfigs; - for (const config of configs) known.set(config.id, config); + const configure = (framework as unknown as { setMcplServerConfig(config: unknown): void }).setMcplServerConfig.bind(framework); + for (const config of configs) configure(config); } const PNG = Buffer.from( @@ -1674,10 +1674,31 @@ describe('ToolImageLedger', () => { ['bound', {}, 'srv'], ['unbound', {}, undefined], ]; + // And the reasons a fetch would be refused before it dials, each at its + // longest: a server's id is bounded as a label is, and a refused scheme + // isn't named. + const longServer = (kind: string) => `${kind}-${'s'.repeat(400)}`; + const host = {}; + referenceRegistry.setServerAddress(longServer('stdio'), 'no-address', host); + referenceRegistry.setServerAddress(longServer('elsewhere'), { url: 'wss://elsewhere.invalid' }, host); + referenceRegistry.setServerAddress(longServer('dialed'), { url: 'wss://example.invalid' }, host); + endings.push( + ['no address', {}, longServer('stdio')], + ['unknown server', {}, longServer('unknown')], + ['third-party origin', {}, longServer('elsewhere')], + ['scheme', { uri: `${'x'.repeat(400)}://example.invalid/sri-longest-scheme.png` }, longServer('dialed')], + ); + const reasons: Record = { + 'no address': '— no address: this host', + 'unknown server': '— unknown server: unknown-s', + 'third-party origin': '— third-party origin:', + scheme: '— scheme not allowed: this host fetches only', + }; for (const [name, extra, serverId] of endings) { const stub = longestId(buildReferenceStub(testimony(name, extra), 'o'.repeat(400), serverId)); assert.ok(stub.length < REFERENCE_STUB_MAX_CHARS, `${name}: ${stub.length}`); assert.strictEqual(parseImagePlaceholders(stub).length, 1, name); + if (reasons[name]) assert.ok(stub.includes(reasons[name]!), `${name}: ${stub}`); } // A placeholder: RFC 6838 bounds a type or subtype name at 127 // characters, so normalizeMediaType keeps the longest essence and refuses diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts new file mode 100644 index 000000000..da4919e49 --- /dev/null +++ b/test/tool-result-blocks.test.ts @@ -0,0 +1,811 @@ +/** + * A tool result's content array reaches the agent block by block, on both of + * its copies, the stored text history keeps and the live copy the model reads + * when the result is answered: each block as itself, as a reference's stub, + * or as a marker saying what kind of block it was and why it isn't shown, + * never as the array's JSON, which would carry inline data's base64 and a + * link's uri. A protocol result's content is content whatever its blocks' + * types (ToolResult.dataIsContent, which an MCPL tool's result carries). Any + * other array is content only when every element is a block of one of MCP's + * types, so a list of records of other types stays data, written as JSON. + * + * Run: node --import tsx --test test/tool-result-blocks.test.ts + */ +import { describe, it, test } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { createServer } from 'node:http'; +import type { AddressInfo } from 'node:net'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { JsStore } from '@animalabs/chronicle'; +import { AgentFramework } from '../src/index.js'; +import type { Module, ModuleContext, ToolCall, ToolResult, ToolDefinition } from '../src/types/index.js'; +import { + AUDIO_NO_DATA_TEXT, + AUDIO_NO_MEDIA_TYPE_TEXT, + IMAGE_NO_DATA_TEXT, + IMAGE_NO_MEDIA_TYPE_TEXT, + INLINE_WITHHELD_TEXT, + REFERENCE_LIMITS, + REFERENCE_LINK_TOO_LONG_TEXT, + REFERENCE_NO_LINK_TEXT, + buildReferenceStub, + classifyBlock, + noAddressReason, + referenceRegistry, + sanitizeLabel, + unknownBlockText, + type BlockClassification, +} from '../src/mcpl/references.js'; +import { convertBlock } from '../src/mcpl/convert-block.js'; +import { CoalesceError, validateCoalescedContent } from '../src/mcpl/push-coalescer.js'; +import { ReferenceFetcher } from '../src/mcpl/reference-fetcher.js'; +import { checkedToolResult } from '../src/module-registry.js'; +import { parseImagePlaceholders } from '../src/tool-image-ledger.js'; +import { nonErrorResultText, toolResultDataToHistoryString } from '../src/tool-result-history.js'; +import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; +import { fixture } from './helpers/coalescing-fixture.js'; + +const SECRET = 'zz-secret-signature'; +/** 3,000 bytes of audio or image data, base64. */ +const BASE64 = Buffer.alloc(3000, 7).toString('base64'); +const PNG = Buffer.from('89504e470d0a1a0a', 'hex').toString('base64'); +const TEXT = { type: 'text', text: 'zz-text' }; +const IMAGE = { type: 'image', data: PNG, mimeType: 'image/png' }; +const AUDIO = { type: 'audio', data: BASE64, mimeType: 'audio/wav' }; +const AUDIO_TEXT = "[Audio not shown: this host's tool results carry only text and images (audio/wav, ~3KB).]"; +const VIDEO = { type: 'video', uri: `https://zz-files.example/clip.mp4?sig=${SECRET}` }; +const LINK_URI = `https://zz-files.example/report.csv?sig=${SECRET}`; +const LINK = { + type: 'resource_link', uri: LINK_URI, name: 'report.csv', mimeType: 'text/csv', size: 1234, + title: 'zz-title', description: 'zz-description', annotations: { audience: ['user'] }, +}; +const STUB = /\[ref_[^\]]+\] report\.csv — text\/csv, ~1KB claimed — from tool result — /; +/** LINK's stub where the host can't fetch it, as where no server delivered it: its uri may carry a credential (sig). */ +const LINK_STUB = new RegExp(`${STUB.source}not tied to any server, so there's nowhere to fetch it from$`); +/** A link whose uri can't carry a credential. */ +const PLAIN_URI = 'https://zz-files.example/report.csv'; +const PLAIN_LINK = { ...LINK, uri: PLAIN_URI }; +/** What PLAIN_LINK shows in a tool result where the host can't fetch it, as where no server delivered it. */ +const PLAIN_LINK_TEXT = `[resource link "report.csv": ${PLAIN_URI}, text/csv; not fetched]`; +/** Record a server dialed on the links' origin, as the framework records a server it configures. Each + * test that needs it asks, so the file loads against a registry that keeps no addresses. */ +const servingFiles = () => referenceRegistry.setServerAddress('zz-files', { url: 'wss://zz-files.example' }, TEST_HOST); +/** The framework these tests record servers' addresses for, as one does. */ +const TEST_HOST = {}; + +/** Each block the lane can't show as itself, and the marker that stands in its place. */ +const MARKED: Array<[string, unknown, string]> = [ + ['inline audio', AUDIO, AUDIO_TEXT], + ['audio data with no media type', { type: 'audio', data: BASE64 }, AUDIO_NO_MEDIA_TYPE_TEXT], + ['image data with no media type', { type: 'image', data: BASE64 }, IMAGE_NO_MEDIA_TYPE_TEXT], + ['image data with an empty media type', { type: 'image', data: BASE64, mimeType: '' }, IMAGE_NO_MEDIA_TYPE_TEXT], + ['an image whose data is empty', { type: 'image', data: '', mimeType: 'image/png' }, IMAGE_NO_DATA_TEXT], + ['audio whose data is empty', { type: 'audio', data: '', mimeType: 'audio/wav' }, AUDIO_NO_DATA_TEXT], + ['an image with neither data nor a link', { type: 'image' }, IMAGE_NO_DATA_TEXT], + ['a block of a type the host can\'t show', VIDEO, unknownBlockText(VIDEO)], + ['an element that isn\'t a block', 42, unknownBlockText(42)], + ['null', null, unknownBlockText(null)], +]; + +type Live = { content: unknown; isError: boolean }; +const framework = () => Object.create(AgentFramework.prototype) as { + toMembraneToolResult(callId: string, result: ToolResult, maxChars?: number): Live; +}; +/** Both copies of a result: the stored text, and the content the model is sent. */ +function copies(result: ToolResult, maxChars?: number): { stored: string; live: unknown } { + return { stored: nonErrorResultText(result, maxChars), live: framework().toMembraneToolResult('zz-call', result, maxChars).content }; +} + +describe("a protocol result's content (dataIsContent)", () => { + it('shows each block the lane can\'t show as its marker on both copies, never as JSON', () => { + assert.equal(AUDIO_TEXT, `[Audio not shown: this host's tool results carry only text and images (audio/wav, ~3KB).]`); + for (const [what, block, marker] of MARKED) { + const { stored, live } = copies({ success: true, data: [TEXT, block], dataIsContent: true }); + assert.equal(stored, `zz-text\n${marker}`, what); + // With no image, the live copy is the stored text. + assert.equal(live, stored, what); + } + const all = copies({ success: true, data: [TEXT, ...MARKED.map(([, block]) => block)], dataIsContent: true }).stored; + assert.ok(!all.includes(BASE64.slice(0, 64)) && !all.includes(SECRET), all); + }); + + it('keeps an image native in the live copy beside them, and its placeholder in the stored text', () => { + for (const [what, block, marker] of MARKED) { + const { stored, live } = copies({ success: true, data: [IMAGE, TEXT, block], dataIsContent: true }); + assert.equal(stored, `[image: image/png, 9B]\nzz-text\n${marker}`, what); + assert.deepEqual(live, [ + { type: 'image', source: { type: 'base64', data: PNG, mediaType: 'image/png' } }, + { type: 'text', text: 'zz-text' }, + { type: 'text', text: marker }, + ], what); + } + }); + + it('shows a reference as its stub, and inline data claiming a link as withheld, as before', () => { + const { stored, live } = copies({ + success: true, + data: [IMAGE, { type: 'resource', uri: `https://zz-files.example/a.pdf?sig=${SECRET}`, name: 'a.pdf' }, { ...AUDIO, uri: 'https://zz-files.example/a.wav' }], + dataIsContent: true, + }); + assert.match(stored, /^\[image: image\/png, 9B\]\n\[ref_[^\]]+\] a\.pdf — from tool result — [^\n]+\n/); + assert.ok(stored.endsWith(`\n${INLINE_WITHHELD_TEXT}`), stored); + assert.ok(!stored.includes(SECRET), stored); + assert.deepEqual((live as Array<{ type: string; text?: string }>).map((b) => b.text ?? b.type), stored.split('\n').map((line, i) => (i === 0 ? 'image' : line))); + }); + + it('cuts a text block at the cap in the live copy, and shows each marker whole', () => { + const long = { type: 'text', text: 'z'.repeat(500) }; + const { live } = copies({ success: true, data: [IMAGE, long, AUDIO], dataIsContent: true }, 40); + const [, text, marker] = live as Array<{ type: string; text: string }>; + assert.match(text.text, /^z{40}\n\n\[truncated — original was 500 chars\]$/); + assert.ok(AUDIO_TEXT.length > 40); + assert.equal(marker.text, AUDIO_TEXT); + }); +}); + +describe('an array without dataIsContent', () => { + it('is content when every element is a block of one of MCP\'s types, each block the lane can\'t show as its marker', () => { + assert.equal(toolResultDataToHistoryString([TEXT, AUDIO]), `zz-text\n${AUDIO_TEXT}`); + assert.equal(toolResultDataToHistoryString([TEXT, PLAIN_LINK]), `zz-text\n${PLAIN_LINK_TEXT}`); + assert.equal(toolResultDataToHistoryString([]), ''); + // A list of records that all carry MCP's type names is read as content + // too: this one, with its url, shows a marker where JSON showed it. + assert.equal(toolResultDataToHistoryString([{ type: 'image', url: 'https://zz-images.example/a.png' }]), IMAGE_NO_DATA_TEXT); + }); + + it('is data, written as JSON, when any element is of another type or isn\'t a record', () => { + for (const data of [ + [TEXT, VIDEO], + [TEXT, { type: 'user', id: 1 }], + [{ name: 'a' }, { name: 'b' }], + ['a', 'b'], + [TEXT, 42], + [TEXT, null], + [TEXT, { type: 42 }], + ]) { + assert.equal(toolResultDataToHistoryString(data), JSON.stringify(data)); + } + // So is data that isn't an array, whatever it holds. + assert.equal(toolResultDataToHistoryString({ content: [TEXT] }), JSON.stringify({ content: [TEXT] })); + }); + + it('is content whatever its elements when the result says it is, and only then', () => { + assert.equal(toolResultDataToHistoryString([TEXT, VIDEO], undefined, { dataIsContent: true }), `zz-text\n${unknownBlockText(VIDEO)}`); + assert.equal(nonErrorResultText({ data: [TEXT, VIDEO], dataIsContent: true }), `zz-text\n${unknownBlockText(VIDEO)}`); + assert.equal(nonErrorResultText({ data: [TEXT, VIDEO], dataIsContent: false }), JSON.stringify([TEXT, VIDEO])); + // Data that isn't an array is data, whatever the result says. + assert.equal(nonErrorResultText({ data: { a: 1 }, dataIsContent: true }), '{"a":1}'); + // The flag is read for its truth, as a result's other flags are, on both copies. + const loose = { success: true, data: [IMAGE, TEXT, VIDEO], dataIsContent: 1 as never }; + const { stored, live } = copies(loose); + assert.equal(stored, `[image: image/png, 9B]\nzz-text\n${unknownBlockText(VIDEO)}`); + assert.equal((live as unknown[]).length, 3); + }); +}); + +/** A block's reference testimony, as classifyBlock reads it. */ +function testimonyOf(block: unknown) { + const c = classifyBlock(block); + assert.equal(c.kind, 'reference'); + return (c as Extract).testimony; +} + +describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { + it('maps onto the reference record field for field, its size as sizeBytes, with no disposition', () => { + const t = testimonyOf({ ...LINK, disposition: 'never', digest: `sha256:${'A'.repeat(43)}`, expiresAt: '2020-01-01T00:00:00Z' }); + assert.deepEqual([t.uri, t.name, t.mimeType, t.sizeBytes], [LINK_URI, 'report.csv', 'text/csv', 1234]); + assert.deepEqual([t.disposition, t.digest, t.expiresAt], [undefined, undefined, undefined]); + assert.deepEqual(t.rejectedFields, []); + // A size that isn't a byte count is dropped as a field, as a sizeBytes is. + const bad = testimonyOf({ ...LINK, size: -1 }); + assert.equal(bad.sizeBytes, undefined); + assert.deepEqual(bad.rejectedFields, ['sizeBytes']); + assert.deepEqual(classifyBlock({ type: 'resource_link', name: 'zz' }), { kind: 'invalid', shown: REFERENCE_NO_LINK_TEXT }); + assert.deepEqual(classifyBlock({ type: 'resource_link', uri: 'u'.repeat(REFERENCE_LIMITS.uri + 1) }), { kind: 'invalid', shown: REFERENCE_LINK_TOO_LONG_TEXT }); + }); + + it('shows as a stub where the host can fetch it, never its uri, in a content array and at any depth in data', () => { + const uri = `https://zz-files.example/fetchable.csv?sig=${SECRET}`; + const fetchable = { ...LINK, uri }; + servingFiles(); + referenceRegistry.register(testimonyOf(fetchable), 'zz-files'); + const { stored, live } = copies({ success: true, data: [TEXT, fetchable], dataIsContent: true }); + assert.match(stored, new RegExp(`^zz-text\\n${STUB.source}fetch with fetch_reference$`)); + assert.equal(live, stored); + const nested = toolResultDataToHistoryString({ found: [fetchable], page: 2 }); + assert.ok(!nested.includes(SECRET), nested); + assert.match((JSON.parse(nested) as { found: string[] }).found[0]!, STUB); + }); + + it("in a tool result, shows its name, uri and type where the host can't fetch it and the uri can't carry a credential", () => { + // PLAIN_LINK has no server, as a module's link hasn't. + const { stored, live } = copies({ success: true, data: [TEXT, PLAIN_LINK], dataIsContent: true }); + assert.equal(stored, `zz-text\n${PLAIN_LINK_TEXT}`); + assert.equal(live, stored); + assert.deepEqual(JSON.parse(toolResultDataToHistoryString({ found: [PLAIN_LINK], page: 2 })), { found: [PLAIN_LINK_TEXT], page: 2 }); + assert.equal(toolResultDataToHistoryString(PLAIN_LINK), PLAIN_LINK_TEXT); + // So too where a server delivered it, with a scheme the fetcher doesn't + // fetch or off its server's origin. A resource there gets the stub that + // says why, with no uri, as a resource always has. + servingFiles(); + for (const [uri, why] of [ + ['file:///zz/report.csv', 'scheme not allowed: this host fetches only http and https uris'], + ['zz-server://reports/report.csv', 'scheme not allowed: this host fetches only http and https uris'], + ['https://zz-elsewhere.example/watch?v=zz', "third-party origin: it isn't on its server's own address, so this host doesn't fetch it"], + ['https://zz-elsewhere.example/d/zz/edit?usp=sharing#heading=h.zz', "third-party origin: it isn't on its server's own address, so this host doesn't fetch it"], + ['https://zz-elsewhere.example/cb?token_type=bearer&key_id=zz', "third-party origin: it isn't on its server's own address, so this host doesn't fetch it"], + // A credential's name with no value, such as an anchor, carries nothing. + ...['#token', '#policy', '#access_token=', '?token', '?token=&page=2'].map((tail) => + [`https://zz-elsewhere.example/docs/auth${tail}`, "third-party origin: it isn't on its server's own address, so this host doesn't fetch it"]), + // A relative one is read as a relative reference is. + ['zz-reports/report.csv?page=2', 'reference uri does not parse'], + ]) { + const link = { type: 'resource_link', uri, name: 'report.csv' }; + referenceRegistry.register(testimonyOf(link), 'zz-files'); + assert.equal(toolResultDataToHistoryString([TEXT, link]), `zz-text\n[resource link "report.csv": ${uri}; not fetched]`); + const resource = { type: 'resource', uri, name: 'report.csv' }; + const stub = toolResultDataToHistoryString([TEXT, resource]); + assert.ok(stub.endsWith(`report.csv — from tool result — ${why}`) && !stub.includes(uri), stub); + } + }); + + it("in a tool result, shows the stub that says why where the host can't fetch it and the uri may carry a credential", () => { + const carriers = [ + // userinfo, whatever the scheme + 'postgres://zz-user:zz-pass@zz-db.example/zz', + 'https://zz-user@zz-elsewhere.example/report.csv', + // AWS's and Google's signed URLs, names compared without letter case + 'https://zz-bucket.s3.amazonaws.com/report.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=zz&X-Amz-Signature=zz', + 'https://zz-bucket.s3.amazonaws.com/report.csv?x-amz-security-token=zz', + 'https://storage.googleapis.com/zz-bucket/report.csv?X-Goog-Credential=zz&X-Goog-Signature=zz', + // Azure's shared access signature, CloudFront's, Discord's CDN, B2's + 'https://zz.blob.core.windows.net/c/report.csv?sv=2024-01-01&se=2030-01-01&sp=r&sig=zz', + 'https://zz.cloudfront.net/report.csv?Expires=1&Signature=zz&Key-Pair-Id=zz', + 'https://zz.cloudfront.net/report.csv?Policy=zz', + 'https://cdn.discordapp.com/attachments/1/2/report.csv?ex=1&is=2&hm=zz', + 'https://f000.backblazeb2.com/file/zz/report.csv?Authorization=zz', + // OAuth's, in a fragment or a query, and the generic names + 'https://zz-elsewhere.example/cb#access_token=zz&token_type=bearer', + 'https://zz-elsewhere.example/cb#id_token=zz', + 'https://zz-elsewhere.example/#/reset?token=zz', + 'https://zz-elsewhere.example/cb?v=1;sig=zz', + 'https://zz-elsewhere.example/cb?v=1&token=zz', + // names compared by their letters and digits alone, without case, and OAuth 1.0a's + ...['accessToken', 'access-token', 'Api-Key', 'x-api-key', 'private-token', 'oauth_token', 'oauth_signature', 'token[]', 'api.key'] + .map((name) => `https://zz-elsewhere.example/cb?${name}=zz`), + // userinfo after a scheme with no slashes, read as an absolute uri + 'http:zz-user:zz-pass@zz-elsewhere.example/report.csv', + // a credential's name, or a scheme before userinfo, split by a + // character the line doesn't show: the check reads the uri as shown + 'https://zz-elsewhere.example/cb?to\u0000ken=zz', + 'ht\u0000tps://zz-user:zz-pass@zz-elsewhere.example/report.csv', + 'https://zz-elsewhere.example/cb?refresh_token=zz', + 'https://zz-elsewhere.example/cb?client_secret=zz', + ...['token', 'auth', 'key', 'apikey', 'api_key', 'secret', 'password', 'signature'].map((name) => `zz-server://reports/report.csv?${name}=zz`), + // a relative one, read as a relative reference is, and one that doesn't + // parse even so, whose parts can't be read + 'zz-reports/report.csv?token=zz', + 'https://zz elsewhere.example/report.csv', + ]; + for (const uri of carriers) { + const link = { type: 'resource_link', uri, name: 'report.csv' }; + const shown = toolResultDataToHistoryString([TEXT, link]); + assert.match(shown, /^zz-text\n\[ref_[^\]]+\] report\.csv — from tool result — not tied to any server, so there's nowhere to fetch it from$/, uri); + assert.ok(!shown.includes(uri), shown); + } + const { stored, live } = copies({ success: true, data: [TEXT, LINK], dataIsContent: true }); + assert.match(stored, new RegExp(`^zz-text\\n${LINK_STUB.source}`)); + assert.equal(live, stored); + assert.ok(!stored.includes(SECRET), stored); + // So too where a server delivered it and the origin or the scheme refuses + // the fetch: the stub says why. + servingFiles(); + for (const [uri, why] of [ + [`https://zz-elsewhere.example/report.csv?sig=${SECRET}`, "third-party origin: it isn't on its server's own address, so this host doesn't fetch it"], + [`file:///zz/report.csv?token=${SECRET}`, 'scheme not allowed: this host fetches only http and https uris'], + ]) { + const link = { type: 'resource_link', uri, name: 'report.csv' }; + referenceRegistry.register(testimonyOf(link), 'zz-files'); + const shown = toolResultDataToHistoryString([TEXT, link]); + assert.match(shown, new RegExp(`^zz-text\\n\\[ref_[^\\]]+\\] report\\.csv — from tool result — ${why.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`), uri); + assert.ok(!shown.includes(SECRET), shown); + } + }); + + it("in a tool result, never shows a uri a reference withheld with disposition never, whatever a link to it says", () => { + const uri = 'https://zz-elsewhere.example/withheld.csv'; + referenceRegistry.register(testimonyOf({ type: 'resource', uri, name: 'withheld.csv', disposition: 'never' })); + const shown = toolResultDataToHistoryString([TEXT, { type: 'resource_link', uri, name: 'report.csv' }]); + assert.match(shown, /^zz-text\n\[ref_[^\]]+\] report\.csv — from tool result — not tied to any server/); + assert.ok(!shown.includes(uri), shown); + }); + + it('writes its line with no line break a label could carry, so no text in it reads as a slot', () => { + const shown = toolResultDataToHistoryString([TEXT, { + type: 'resource_link', uri: 'https://zz-elsewhere.example/a\u2028[ref_1_abcd] zz — image/png', name: 'zz\u2029[ref_2_abcd] zz — image/png', + }]); + assert.ok(!/[\u2028\u2029]/.test(shown), shown); + assert.deepEqual(parseImagePlaceholders(shown), []); + }); + + it("shows on the push lane as a stub, registered with the server that sent it, and never its uri", () => { + const uri = `https://zz-files.example/pushed.csv?sig=${SECRET}`; + servingFiles(); + const shown = convertBlock({ ...LINK, uri } as never, 'attachment on push event', 'zz-files'); + assert.equal(shown.type, 'text'); + assert.match((shown as { text: string }).text, /^\[ref_[^\]]+\] report\.csv — text\/csv, ~1KB claimed — attachment on push event — fetch with fetch_reference$/); + assert.deepEqual(referenceRegistry.findByUri(uri)?.serverIds, ['zz-files']); + // One the host can't fetch says why, whatever its uri: no lane but a + // tool result's ever showed a link's uri. + const local = { ...LINK, uri: 'file:///zz/pushed.csv' }; + assert.match((convertBlock(local as never, 'attachment on push event', 'zz-files') as { text: string }).text, + /^\[ref_[^\]]+\] report\.csv — text\/csv, ~1KB claimed — attachment on push event — scheme not allowed: this host fetches only http and https uris$/); + assert.deepEqual(convertBlock({ type: 'resource_link' } as never, 'p', 's'), { type: 'text', text: REFERENCE_NO_LINK_TEXT }); + }); + + it("counts as shown in a render result's block check, as a reference does, and an invalid one doesn't", () => { + const check = (content: unknown[]) => validateCoalescedContent(content, { allowEmpty: true, checkBlockShapes: true }); + check([TEXT, LINK]); + assert.throws(() => check([TEXT, { type: 'resource_link', name: 'zz' }]), CoalesceError); + }); + + it("never replaces the claims a reference made for its uri, while a reference's replace a link's", () => { + const uri = `https://zz-files.example/claimed.bin?sig=${SECRET}`; + const digest = `sha256:${'B'.repeat(43)}`; + servingFiles(); + referenceRegistry.register(testimonyOf({ type: 'resource', uri, digest, expiresAt: '2020-01-01T00:00:00Z', name: 'claimed.bin' }), 'zz-files'); + referenceRegistry.register(testimonyOf({ type: 'resource_link', uri, name: 'linked.bin' }), 'zz-files'); + const kept = referenceRegistry.findByUri(uri)!.testimony; + assert.deepEqual([kept.digest, kept.expiresAt, kept.name, kept.fromLink], [digest, '2020-01-01T00:00:00Z', 'claimed.bin', undefined]); + // A link's stub reads its record, so it says what the reference's expiry makes of it. + assert.match((convertBlock({ type: 'resource_link', uri, name: 'linked.bin' } as never, 'p', 'zz-files') as { text: string }).text, /^\[ref_[^\]]+\] linked\.bin — p — expired$/); + const later = 'https://zz-files.example/linked-first.bin'; + referenceRegistry.register(testimonyOf({ type: 'resource_link', uri: later, name: 'first' }), 'zz-files'); + referenceRegistry.register(testimonyOf({ type: 'resource', uri: later, digest, name: 'second' }), 'zz-files'); + const replaced = referenceRegistry.findByUri(later)!.testimony; + assert.deepEqual([replaced.digest, replaced.name, replaced.fromLink], [digest, 'second', undefined]); + }); + + it('is registered with its server and never fetched eagerly, unless another block names its uri', async () => { + const fw = Object.create(AgentFramework.prototype) as { + autoFetchReferences(content: unknown[], serverId: string, eager: boolean): Promise; + getReferenceFetcher(): unknown; + }; + const fetched: string[] = []; + fw.getReferenceFetcher = () => ({ + isEagerEligible: () => true, + fetch: async (record: { testimony: { uri: string } }) => { fetched.push(record.testimony.uri); return { ok: true }; }, + }); + const u = (name: string) => `https://zz-files.example/eager/${name}`; + await fw.autoFetchReferences([ + { type: 'resource_link', uri: u('linked') }, + { type: 'resource_link', uri: u('linked-then-referenced') }, + { type: 'resource', uri: u('linked-then-referenced') }, + { type: 'resource', uri: u('referenced-then-linked') }, + { type: 'resource_link', uri: u('referenced-then-linked') }, + { type: 'resource', uri: u('referenced') }, + ], 'zz-eager-server', true); + assert.deepEqual(fetched.sort(), [u('linked-then-referenced'), u('referenced'), u('referenced-then-linked')].sort()); + assert.deepEqual(referenceRegistry.findByUri(u('linked'))?.serverIds, ['zz-eager-server']); + }); + + it("isn't fetched for a script on demand, where a reference is", async () => { + const fw = Object.create(AgentFramework.prototype) as { + dispatchScriptToolCall: (...args: unknown[]) => Promise; + getReferenceFetcher(): unknown; + handleScriptToolCall(agentName: string, toolName: string, args: Record): Promise; + }; + const reference = { + type: 'resource', uri: `https://zz-files.example/wanted.pdf?sig=${SECRET}`, name: 'wanted.pdf', + digest: `sha256:${'C'.repeat(43)}`, expiresAt: '2099-01-01T00:00:00Z', + }; + // A link to the same uri came first, so its record holds the link's + // claims until the reference's replace them. + referenceRegistry.register(testimonyOf({ type: 'resource_link', uri: reference.uri, name: 'linked.pdf' }), 'zz-files'); + fw.dispatchScriptToolCall = async () => ({ success: true, data: [TEXT, LINK, reference, VIDEO], dataIsContent: true }); + const fetched: string[] = []; + const claims: unknown[] = []; + fw.getReferenceFetcher = () => ({ + fetch: async (record: { testimony: { uri: string; name?: string; digest?: string; expiresAt?: string } }) => { + fetched.push(record.testimony.uri); + claims.push([record.testimony.name, record.testimony.digest, record.testimony.expiresAt]); + return { ok: false, error: 'zz-refused' }; + }, + }); + const error = console.error; + console.error = () => {}; + let text: string; + try { + text = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + } finally { + console.error = error; + } + assert.deepEqual(fetched, [reference.uri]); + assert.deepEqual(claims, [['wanted.pdf', reference.digest, reference.expiresAt]]); + // The script reads the text history stores for a protocol result's + // content: the link, which no server delivered, as the stub that says so. + assert.match(text, new RegExp(`^zz-text\\n${LINK_STUB.source.slice(0, -1)}\\n\\[ref_[^\\]]+\\] wanted\\.pdf — from tool result[^\\n]+\\n`)); + assert.ok(text.endsWith(`\n${unknownBlockText(VIDEO)}`), text); + assert.ok(!text.includes(SECRET), text); + }); +}); + +describe('what a stub says of a fetch, which the fetch then does', () => { + // Servers as a framework records them: one dialed on the references' + // origin, one with no address, as a stdio server; zz-gone is unknown. + const serving = () => { + referenceRegistry.setServerAddress('zz-dialed', { url: 'wss://zz-refs.example' }, TEST_HOST); + referenceRegistry.setServerAddress('zz-stdio', 'no-address', TEST_HOST); + }; + /** The fetcher of a framework configured as `serving` records it. */ + const servedFetcher = () => new ReferenceFetcher( + (serverId) => (serverId === 'zz-dialed' ? { url: 'wss://zz-refs.example' } : serverId === 'zz-stdio' ? 'no-address' : undefined), + async () => null, + ); + const record = (name: string, serverIds: string[], extra: Record = {}) => { + const t = testimonyOf({ type: 'resource', uri: `https://zz-refs.example/${name}?sig=${SECRET}`, name, ...extra }); + let r = referenceRegistry.register(t); + for (const id of serverIds) r = referenceRegistry.register(t, id); + return r; + }; + // Made when a test first asks, so the file loads against a registry that + // keeps no addresses. + let made: Array<[string, ReturnType, string | null]> | undefined; + const cases = () => made ??= (serving(), [ + ['dialed on its origin', record('dialed.bin', ['zz-dialed']), null], + ['from no server', record('unbound.bin', []), "not tied to any server, so there's nowhere to fetch it from"], + ['from an unknown server', record('gone.bin', ['zz-gone']), 'unknown server: zz-gone'], + ['from a server with no address', record('stdio.bin', ['zz-stdio']), "no address: this host doesn't connect to server zz-stdio over a network, so there's nowhere to fetch it from"], + ['from a server with no address, and one dialed on its origin', record('both.bin', ['zz-stdio', 'zz-dialed']), null], + ['dialed on its origin, but expired', record('expired.bin', ['zz-dialed'], { expiresAt: '2020-01-01T00:00:00Z' }), 'expired'], + ]); + + it("ends a resource's stub with why a fetch would be refused, or the fetch it offers", () => { + for (const [what, r, why] of cases()) { + const stub = buildReferenceStub(r.testimony, 'zz'); + assert.ok(stub.endsWith(`— zz — ${why ?? 'fetch with fetch_reference'}`) && !stub.includes(SECRET), `${what}: ${stub}`); + } + }); + + it("agrees with the fetcher's own check, which reads its framework's configs", () => { + const fetcher = servedFetcher(); + for (const [what, r, why] of cases()) { + const refusal = fetcher.refusal(r); + assert.equal(refusal === null, why === null, `${what}: ${refusal?.reason}`); + if (refusal && why) assert.equal(refusal.reason, why, what); + } + }); + + it("is the framework's fetcher's check too, and the framework records each server's address for the stub", () => { + const fw = Object.create(AgentFramework.prototype) as { + mcplServerConfigs: Map; + setMcplServerConfig(config: unknown): void; + getReferenceFetcher(): ReferenceFetcher; + }; + fw.mcplServerConfigs = new Map(); + fw.setMcplServerConfig({ id: 'zz-wired', url: 'wss://zz-refs.example' }); + assert.deepEqual(referenceRegistry.serverAddress('zz-wired'), { url: 'wss://zz-refs.example' }); + // A server the host spawns has no address, whatever url its config holds. + fw.setMcplServerConfig({ id: 'zz-spawned', command: 'zz-server', url: 'wss://zz-refs.example' }); + assert.equal(referenceRegistry.serverAddress('zz-spawned'), 'no-address'); + const wired = record('wired.bin', ['zz-wired']); + const spawned = record('spawned.bin', ['zz-spawned']); + const fetcher = fw.getReferenceFetcher(); + assert.equal(fetcher.refusal(wired), null); + assert.ok(buildReferenceStub(wired.testimony, 'zz').endsWith('— zz — fetch with fetch_reference')); + assert.equal(fetcher.refusal(spawned)?.reason, noAddressReason('zz-spawned')); + assert.ok(buildReferenceStub(spawned.testimony, 'zz').endsWith(`— zz — ${noAddressReason('zz-spawned')}`)); + }); + + it("never sends a framework's token to another framework's origin for one id, and a removal takes back only its own address", () => { + const framework = () => { + const fw = Object.create(AgentFramework.prototype) as { + mcplServerConfigs: Map; + setMcplServerConfig(config: unknown): void; + getReferenceFetcher(): ReferenceFetcher; + }; + fw.mcplServerConfigs = new Map(); + return fw; + }; + const earlier = framework(); + const later = framework(); + earlier.setMcplServerConfig({ id: 'zz-shared', url: 'wss://zz-earlier.example', token: 'zz-earlier-token' }); + later.setMcplServerConfig({ id: 'zz-shared', url: 'wss://zz-later.example', token: 'zz-later-token' }); + // The later framework's entry is in force for the stub. + assert.deepEqual(referenceRegistry.serverAddress('zz-shared'), { url: 'wss://zz-later.example' }); + const onLater = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'https://zz-later.example/shared.bin', name: 'shared.bin' }), 'zz-shared'); + const onEarlier = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'https://zz-earlier.example/shared.bin', name: 'shared.bin' }), 'zz-shared'); + // Each fetcher checks a uri against its own config's address, whose token + // it carries: the earlier framework's never dials the later one's origin. + assert.match(earlier.getReferenceFetcher().refusal(onLater)!.reason, /^third-party origin \(https:\/\/zz-later\.example != dialed https:\/\/zz-earlier\.example\)/); + assert.equal(earlier.getReferenceFetcher().refusal(onEarlier), null); + assert.equal(later.getReferenceFetcher().refusal(onLater), null); + assert.match(later.getReferenceFetcher().refusal(onEarlier)!.reason, /^third-party origin/); + // The later framework's removal takes back only its own entry, and the + // earlier one's is in force again. + referenceRegistry.forgetServerAddress('zz-shared', later); + assert.deepEqual(referenceRegistry.serverAddress('zz-shared'), { url: 'wss://zz-earlier.example' }); + assert.ok(buildReferenceStub(onEarlier.testimony, 'zz').endsWith('— zz — fetch with fetch_reference')); + referenceRegistry.forgetServerAddress('zz-shared', earlier); + assert.equal(referenceRegistry.serverAddress('zz-shared'), undefined); + }); + + it("carries each framework's own token to its own server's origin, and to no other, when it fetches", async (t) => { + const heard: Array<[string, string | undefined]> = []; + const serve = async (name: string) => { + const server = createServer((req, res) => { + heard.push([name, req.headers.authorization]); + res.writeHead(200, { 'content-type': 'application/octet-stream' }); + res.end(Buffer.from('zz-bytes')); + }); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + t.after(() => new Promise((resolve) => server.close(() => resolve()))); + return `127.0.0.1:${(server.address() as AddressInfo).port}`; + }; + const [hostA, hostB] = [await serve('a'), await serve('b')]; + const framework = (url: string, token: string) => { + const fw = Object.create(AgentFramework.prototype) as { + setMcplServerConfig(config: unknown): void; + getReferenceFetcher(): ReferenceFetcher; + }; + Object.assign(fw, { + mcplServerConfigs: new Map(), + getWorkspaceModule: () => ({ writeBinary: async () => ({ success: true }) }), + firstWritableMountName: () => 'scratch', + emitTrace: () => {}, + }); + fw.setMcplServerConfig({ id: 'zz-shared-fetch', url, token }); + return fw; + }; + const a = framework(`ws://${hostA}`, 'zz-token-a'); + const b = framework(`ws://${hostB}`, 'zz-token-b'); + const onA = referenceRegistry.register(testimonyOf({ type: 'resource', uri: `http://${hostA}/a.bin`, name: 'a.bin' }), 'zz-shared-fetch'); + const onB = referenceRegistry.register(testimonyOf({ type: 'resource', uri: `http://${hostB}/b.bin`, name: 'b.bin' }), 'zz-shared-fetch'); + // Each framework refuses the other's origin without dialing it, and + // dials its own with its own token. + assert.equal((await b.getReferenceFetcher().fetch(onA)).ok, false); + assert.equal((await a.getReferenceFetcher().fetch(onB)).ok, false); + assert.equal((await a.getReferenceFetcher().fetch(onA)).ok, true); + assert.equal((await b.getReferenceFetcher().fetch(onB)).ok, true); + assert.deepEqual(heard, [['a', 'Bearer zz-token-a'], ['b', 'Bearer zz-token-b']]); + referenceRegistry.forgetServerAddress('zz-shared-fetch', a); + referenceRegistry.forgetServerAddress('zz-shared-fetch', b); + }); + + it("words a server's id as a label in a stub, and a third-party origin and a refused scheme without the uri's parts", () => { + serving(); + const weird = `zz-\u202E${'x'.repeat(300)}`; + const r = record('weird.bin', [weird]); + const stub = buildReferenceStub(r.testimony, 'zz'); + assert.ok(stub.includes('unknown server: zz-xxx') && !stub.includes('\u202E') && stub.length < 400, stub); + referenceRegistry.setServerAddress('zz-other', { url: 'wss://zz-other.example' }, TEST_HOST); + const offOrigin = record('elsewhere.bin', ['zz-other']); + assert.ok(buildReferenceStub(offOrigin.testimony, 'zz').endsWith("— zz — third-party origin: it isn't on its server's own address, so this host doesn't fetch it")); + const fetcher = new ReferenceFetcher(() => ({ url: 'wss://zz-other.example' }), async () => null); + assert.match(fetcher.refusal(offOrigin)!.reason, /^third-party origin \(https:\/\/zz-refs\.example != dialed https:\/\/zz-other\.example\); not fetched by this host$/); + // A scheme the fetcher doesn't fetch: the stub doesn't name it, and the + // fetch's refusal names it bounded as a label is. + const scheme = `zz${'s'.repeat(300)}`; + const t = testimonyOf({ type: 'resource', uri: `${scheme}://zz-refs.example/long.bin`, name: 'long.bin' }); + const long = referenceRegistry.register(t, 'zz-dialed'); + assert.ok(buildReferenceStub(t, 'zz').endsWith('— zz — scheme not allowed: this host fetches only http and https uris')); + assert.equal(new ReferenceFetcher(() => ({ url: 'wss://zz-refs.example' }), async () => null).refusal(long)?.reason, + `scheme not allowed: ${sanitizeLabel(`${scheme}:`, 120)}`); + }); + + it('refuses an http uri on a securely dialed server as plaintext, and a server url that does not parse as such', () => { + const plain = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'http://zz-refs.example/plain.bin', name: 'plain.bin' }), 'zz-secure'); + referenceRegistry.setServerAddress('zz-secure', { url: 'wss://zz-refs.example' }, TEST_HOST); + assert.ok(buildReferenceStub(plain.testimony, 'zz').endsWith('— zz — plaintext reference from a secure origin')); + assert.equal(new ReferenceFetcher(() => ({ url: 'wss://zz-refs.example' }), async () => null).refusal(plain)?.reason, 'plaintext reference from a secure origin'); + // On the secure server's port named outright, it's the same place. + const port = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'http://zz-refs.example:443/port.bin', name: 'port.bin' }), 'zz-secure'); + assert.ok(buildReferenceStub(port.testimony, 'zz').endsWith('— zz — plaintext reference from a secure origin')); + // Another host over plaintext is a third party's, as before. + const elsewhere = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'http://zz-elsewhere.example/plain.bin', name: 'plain.bin' }), 'zz-secure'); + assert.match(buildReferenceStub(elsewhere.testimony, 'zz'), /— zz — third-party origin: /); + referenceRegistry.setServerAddress('zz-garbled', { url: 'zz not a url' }, TEST_HOST); + const garbled = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'https://zz-refs.example/garbled.bin', name: 'garbled.bin' }), 'zz-garbled'); + assert.ok(buildReferenceStub(garbled.testimony, 'zz').endsWith('— zz — server url does not parse')); + }); +}); + +describe('an empty link beside inline data is no link', () => { + const image = { ...IMAGE, uri: '' }; + const audio = { ...AUDIO, uri: '' }; + + it('so the data is shown as any inline data is, on the push lane and the tool-result lane', () => { + assert.deepEqual(classifyBlock(image), { kind: 'inline', contradiction: false }); + assert.deepEqual(classifyBlock(audio), { kind: 'inline', contradiction: false }); + assert.deepEqual(convertBlock(image as never, 'p', 's'), { type: 'image', source: { type: 'base64', data: PNG, mediaType: 'image/png' } }); + assert.deepEqual(convertBlock(audio as never, 'p', 's'), { type: 'audio', source: { type: 'base64', data: BASE64, mediaType: 'audio/wav' } }); + const { stored, live } = copies({ success: true, data: [image, audio], dataIsContent: true }); + assert.equal(stored, `[image: image/png, 9B]\n${AUDIO_TEXT}`); + assert.deepEqual(live, [{ type: 'image', source: { type: 'base64', data: PNG, mediaType: 'image/png' } }, { type: 'text', text: AUDIO_TEXT }]); + }); + + it('while a link that isn\'t empty still withholds it, on both lanes', () => { + const linked = { ...AUDIO, uri: 'https://zz-files.example/a.wav' }; + assert.deepEqual(classifyBlock(linked), { kind: 'inline', contradiction: true }); + assert.deepEqual(convertBlock(linked as never, 'p', 's'), { type: 'text', text: INLINE_WITHHELD_TEXT }); + assert.equal(nonErrorResultText({ data: [TEXT, linked], dataIsContent: true }), `zz-text\n${INLINE_WITHHELD_TEXT}`); + }); +}); + +describe('dataIsContent travels with the result', () => { + it('is kept as its truth when a module\'s result is checked, read once', () => { + const call = { id: 'zz', name: 'zz_tool', input: {} }; + assert.equal(checkedToolResult(call, { success: true, data: [TEXT], dataIsContent: 1 }).result.dataIsContent, true); + assert.equal(checkedToolResult(call, { success: true, data: [TEXT], dataIsContent: 0 }).result.dataIsContent, false); + assert.equal('dataIsContent' in checkedToolResult(call, { success: true, data: [TEXT] }).result, false); + let reads = 0; + class Answer { + success = true; + data = [TEXT, VIDEO]; + get dataIsContent() { reads += 1; return true; } + } + assert.equal(checkedToolResult(call, new Answer()).result.dataIsContent, true); + assert.equal(reads, 1); + }); + + it("through a model's round, from a module's result: both copies show its blocks", async () => { + for (const dataIsContent of [true, undefined]) { + const { wire, stored } = await answerModelCall({ success: true, data: [IMAGE, TEXT, AUDIO, VIDEO], ...(dataIsContent ? { dataIsContent } : {}) }); + if (dataIsContent) { + // The image's placeholder carries the ref the ledger retained it under. + assert.equal(stored.replace(/ref img_[A-Za-z0-9]+_\d+/, 'ref IMG'), JSON.stringify([`[image: image/png, 9B, ref IMG]\nzz-text\n${AUDIO_TEXT}\n${unknownBlockText(VIDEO)}`])); + assert.equal(wire, JSON.stringify([[ + { type: 'image', source: { type: 'base64', data: PNG, mediaType: 'image/png' } }, + { type: 'text', text: 'zz-text' }, + { type: 'text', text: AUDIO_TEXT }, + { type: 'text', text: unknownBlockText(VIDEO) }, + ]])); + } else { + // Without it, the block of a type MCP doesn't define makes the array data. + assert.equal(stored, JSON.stringify([JSON.stringify([IMAGE, TEXT, AUDIO, VIDEO])])); + assert.equal(wire, stored); + } + } + }); + + test("an MCPL tool's result carries it, to the model's call, an in-process caller and the puppet's stored pair", { timeout: 30_000 }, async (t) => { + // The fixture's server is dialed at ws://127.0.0.1:, so a link on + // that origin can be fetched, and one elsewhere can't. + let BLOCKS: unknown[] = []; + const f = await fixture({ + tools: [{ name: 'lookup', description: 'Look something up.', inputSchema: { type: 'object' } }], + toolsCall: (params) => ((params.arguments as { case: string }).case === 'text' + ? { content: [TEXT, { type: 'text', text: 'zz-more' }] } + : { content: BLOCKS }), + }); + t.after(f.close); + const elsewhere = `https://zz-files.example/mcpl.csv?sig=${SECRET}`; + const plain = 'https://zz-files.example/mcpl.csv'; + const own = `http://127.0.0.1:${f.port}/mcpl.csv?sig=${SECRET}`; + BLOCKS = [TEXT, VIDEO, AUDIO, { ...LINK, uri: own }, { ...LINK, uri: elsewhere, name: 'elsewhere.csv' }, { ...LINK, uri: plain, name: 'plain.csv' }]; + assert.deepEqual(referenceRegistry.serverAddress('editor'), { url: `ws://127.0.0.1:${f.port}` }); + const internals = f.framework as unknown as { pushEvent(event: unknown): void; dispatchToolCall(agent: string, call: unknown): void }; + const viaModel = (id: string, c: string) => new Promise((resolve) => { + const original = internals.pushEvent; + internals.pushEvent = (event: unknown) => { + const e = event as { type: string; callId?: string; result?: ToolResult }; + if (e.type === 'tool-result' && e.callId === id) { internals.pushEvent = original; resolve(e.result!); } + else original.call(internals, event); + }; + internals.dispatchToolCall('agent', { id, name: 'mcpl--editor--lookup', input: { case: c } }); + }); + // The puppet stores the pair from a checked copy of the in-process + // result. It runs first, while the agent is idle. + const { toolUseId } = await f.framework.puppetToolCall('agent', 'mcpl--editor--lookup', { case: 'blocks' }); + const stored = f.framework.getAgent('agent')!.getContextManager().getAllMessages() + .flatMap((m) => m.content as unknown as Array>) + .find((b) => b.type === 'tool_result' && b.toolUseId === toolUseId); + const model = await viaModel('zz-model', 'blocks'); + assert.equal(model.dataIsContent, true); + assert.deepEqual(model.data, BLOCKS); + const expected = nonErrorResultText(model); + // The server's own link fetches. Another origin's, which may carry a + // credential, says why it can't be fetched, and the plain one shows its uri. + assert.match(expected, new RegExp( + `^zz-text\\n\\[Content not shown: this host can't display "video" content\\.\\]\\n\\[Audio not shown: [^\\n]+\\n${STUB.source}fetch with fetch_reference\\n` + + `\\[ref_[^\\]]+\\] elsewhere\\.csv — text/csv, ~1KB claimed — from tool result — third-party origin: it isn't on its server's own address, so this host doesn't fetch it\\n` + + `\\[resource link "plain\\.csv": https://zz-files\\.example/mcpl\\.csv, text/csv; not fetched\\]$`)); + assert.ok(!expected.includes(SECRET), expected); + assert.deepEqual(referenceRegistry.findByUri(own)?.serverIds, ['editor']); + // The puppet's stored pair holds what a model's call stores. + assert.equal(stored?.content, expected); + // A text-only result is still its joined text, and carries nothing more. + const text = await viaModel('zz-text', 'text'); + assert.deepEqual([text.data, text.dataIsContent], ['zz-text\nzz-more', undefined]); + const inProcess = await f.framework.executeToolCall({ id: 'zz-x', name: 'mcpl--editor--lookup', input: { case: 'blocks' } }); + assert.deepEqual([inProcess.data, inProcess.dataIsContent], [BLOCKS, true]); + // A server the framework removes is forgotten, and a link of its then + // can't be fetched. + await f.framework.disconnectMcplServer('editor'); + assert.equal(referenceRegistry.serverAddress('editor'), undefined); + assert.match(nonErrorResultText(model), new RegExp(`\\n${STUB.source}unknown server: editor\\n`)); + }); +}); + +test("a framework's stop takes its servers' addresses out of the registry, and its start records them again", async () => { + // On a store of its own, so it can start again after it stops. + const dir = mkdtempSync(join(tmpdir(), 'tool-result-blocks-lifecycle-')); + const store = JsStore.openOrCreate({ path: join(dir, 'store') }); + const framework = await AgentFramework.create({ + store, + membrane: new MockMembrane().asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test', allowedTools: 'all' }], + modules: [], + syncIntervalMs: 0, + maintenanceIntervalMs: 0, + }); + try { + (framework as unknown as { setMcplServerConfig(config: unknown): void }) + .setMcplServerConfig({ id: 'zz-lifecycle', url: 'wss://zz-lifecycle.example' }); + framework.start(); + assert.deepEqual(referenceRegistry.serverAddress('zz-lifecycle'), { url: 'wss://zz-lifecycle.example' }); + await framework.stop(); + assert.equal(referenceRegistry.serverAddress('zz-lifecycle'), undefined); + framework.start(); + assert.deepEqual(referenceRegistry.serverAddress('zz-lifecycle'), { url: 'wss://zz-lifecycle.example' }); + } finally { + await framework.stop(); + store.close(); + rmSync(dir, { recursive: true, force: true }); + } +}); + +/** A module whose one tool answers with the result it was given. */ +class AnswerModule implements Module { + readonly name = 'content'; + constructor(private readonly answer: ToolResult) {} + async start(_ctx: ModuleContext): Promise {} + async stop(): Promise {} + getTools(): ToolDefinition[] { + return [{ name: 'get', description: 'Answer with content.', inputSchema: { type: 'object', properties: {} } }]; + } + async handleToolCall(_call: ToolCall): Promise { + return this.answer; + } + async onProcess(): Promise> { + return {}; + } +} + +/** The content of the tool result the model received for one call, and of what the agent's history stored. */ +async function answerModelCall(answer: ToolResult): Promise<{ wire: string; stored: string }> { + const dir = mkdtempSync(join(tmpdir(), 'tool-result-blocks-')); + const membrane = new MockMembrane(); + const framework = await AgentFramework.create({ + storePath: join(dir, 'store'), + membrane: membrane.asMembrane(), + agents: [{ name: 'agent', model: 'test-model', systemPrompt: 'test', allowedTools: 'all' }], + modules: [new AnswerModule(answer)], + syncIntervalMs: 0, + maintenanceIntervalMs: 0, + }); + try { + membrane.pushResponse(createMockResponse([{ type: 'tool_use', id: 'zz-call', name: 'content--get', input: {} }], 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'zz-done' }])); + framework.start(); + (framework as unknown as { pendingRequests: unknown[] }).pendingRequests.push({ + agentName: 'agent', reason: 'external-message', source: 'test', timestamp: Date.now(), + }); + const stream = () => membrane.lastStream as unknown as { receivedToolResults: unknown[][] } | undefined; + const deadline = Date.now() + 5_000; + while ((stream()?.receivedToolResults.length ?? 0) < 1 && Date.now() < deadline) await new Promise((r) => setTimeout(r, 25)); + const received = stream()?.receivedToolResults[0]; + assert.ok(received, 'the model was answered'); + const stored = framework.getAgent('agent')!.getContextManager().getAllMessages() + .flatMap((message) => message.content) + .filter((block) => (block as { type?: string }).type === 'tool_result'); + assert.equal(stored.length, 1, 'the history holds the tool result'); + const contentOf = (results: unknown[]) => JSON.stringify(results.map((r) => (r as { content?: unknown }).content)); + return { wire: contentOf(received), stored: contentOf(stored) }; + } finally { + await framework.stop(); + rmSync(dir, { recursive: true, force: true }); + } +} diff --git a/test/tool-result-history.test.ts b/test/tool-result-history.test.ts index 7117b820c..faf6393ad 100644 --- a/test/tool-result-history.test.ts +++ b/test/tool-result-history.test.ts @@ -45,11 +45,19 @@ test('unknown block shape falls back to JSON', () => { assert.equal(toolResultDataToHistoryString(data), JSON.stringify(data)); }); -test('mixed known + unknown block shape falls back to JSON (all-or-nothing)', () => { +test('blocks of MCP types the lane cannot show are markers, block by block, never the JSON with their data', () => { const data = [ { type: 'text', text: 'fine' }, { type: 'audio', data: 'XX', mimeType: 'audio/mpeg' }, ]; + assert.equal(toolResultDataToHistoryString(data), "fine\n[Audio not shown: this host's tool results carry only text and images (audio/mpeg, 1B).]"); +}); + +test('a block of a type MCP does not define makes the array data, written as JSON (all-or-nothing)', () => { + const data = [ + { type: 'text', text: 'fine' }, + { type: 'mystery', payload: 42 }, + ]; assert.equal(toolResultDataToHistoryString(data), JSON.stringify(data)); }); diff --git a/test/tool-result-reference-uri.test.ts b/test/tool-result-reference-uri.test.ts index c16402d29..486437efa 100644 --- a/test/tool-result-reference-uri.test.ts +++ b/test/tool-result-reference-uri.test.ts @@ -4,6 +4,9 @@ * content array of blocks the serializer recognizes is written as JSON with * each reference as its stub and inline data claiming a uri or a disposition * withheld. Everything else in it is written as JSON writes it, as before. + * MCP's resource_link is a reference too, and where the host can't fetch + * one, as with no server, it shows its uri, the agent's only handle on it, + * unless the uri may carry a credential. */ import { describe, it } from 'node:test'; import assert from 'node:assert/strict'; @@ -120,11 +123,27 @@ describe('data the serializer doesn\'t read as content blocks', () => { }); it('writes data with nothing RFC-005 governs exactly as JSON does', () => { - const plain = [{ id: 1, url: URI, type: 'zz-record' }, { type: 'resource_link', uri: URI, name: 'zz' }, 'zz', 7, null]; + const plain = [{ id: 1, url: URI, type: 'zz-record' }, 'zz', 7, null]; assert.equal(toolResultDataToHistoryString(plain), JSON.stringify(plain)); assert.equal(toolResultDataToHistoryString({ a: 1 }), '{"a":1}'); }); + it("governs MCP's resource_link at any depth: one the host can't fetch shows its uri only where it can't carry a credential", () => { + // References (RFC-005 §8.1) with no server, as a module's are. A link's + // uri is the agent's only handle on it, so it stays where it can't carry + // a credential, as the JSON showed it; a resource's never shows. + const signed = { type: 'resource_link', uri: URI, name: 'zz.pdf', mimeType: 'application/pdf' }; + const plainUri = 'https://zz-files.example/plain.pdf'; + const plainLink = { ...signed, uri: plainUri }; + const text = toolResultDataToHistoryString([{ id: 1, type: 'zz-record' }, signed, 'zz', RESOURCE, plainLink]); + assert.ok(!text.includes(SIGNATURE), text); + const [record, signedShown, plain, stub, plainShown] = parsed(text) as [unknown, string, string, string, string]; + assert.deepEqual([record, plain], [{ id: 1, type: 'zz-record' }, 'zz']); + assert.match(signedShown, /^\[ref_[^\]]+\] zz\.pdf — application\/pdf — from tool result — not tied to any server/); + assert.match(stub, STUB); + assert.equal(plainShown, `[resource link "zz.pdf": ${plainUri}, application/pdf; not fetched]`); + }); + it('writes a record that only shares a type name, with no uri and no string data, as JSON does', () => { // A uri that is null, absent or empty is none, and data that isn't a // string isn't inline data: nothing here is a reference or inline bytes. From 1dc2c9eaab3b069ccdea7849ae6fb867a5115edb Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 12:45:11 -0400 Subject: [PATCH 02/40] fix(references): read a credential's value both ways; correct two docs uriMayCarryCredential counts a query or fragment parameter named like a credential only when it has a value, and a '?' or ';' separated parameters too, for routes kept in a fragment. So a value that opens with one of them, as in token=;zz or #access_token=?zz, read as empty, and a tool result's link line showed the uri. The parameters are now read both ways, with '&' alone separating them, as a server reading the query as WHATWG does sees them, and with '?' and ';' as well; a credential-named parameter with a value in either reading counts. Both uris join the carriers test. samePlace's doc states the rule as the code has it: the same port, or neither url giving one, or one giving the other's default. forgetServerAddress's doc no longer says a stopped framework's fetch refuses what its stubs do: a stopped framework keeps its configs. Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 34 ++++++++++++++++++++------------- test/tool-result-blocks.test.ts | 3 +++ 2 files changed, 24 insertions(+), 13 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 5c1b86839..c3847b494 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -289,10 +289,12 @@ function credentialName(name: string): string { * Whether a uri may carry a credential: userinfo, or a query or fragment * parameter named on CREDENTIAL_PARAMETERS with a value, a fragment read as * `name=value&…` as a query is. A name with no value carries nothing, as an - * anchor (`#token`) or `?token=` doesn't. A `?` or `;` inside either - * separates parameters too, as in a route kept in a fragment - * (`#/reset?token=…`). A uri that doesn't parse, even as a relative - * reference, may, since its parts can't be read. + * anchor (`#token`) or `?token=` doesn't. A `?` or `;` inside either may + * separate parameters too, as in a route kept in a fragment + * (`#/reset?token=…`), so the parameters are read both with and without + * them as separators, and one with a value in either reading counts. A uri + * that doesn't parse, even as a relative reference, may, since its parts + * can't be read. */ export function uriMayCarryCredential(uri: string): boolean { let u: URL; @@ -308,9 +310,12 @@ export function uriMayCarryCredential(uri: string): boolean { } } if (u.username !== '' || u.password !== '') return true; - const named = (params: string) => - [...new URLSearchParams(params.replace(/[?;]/g, '&')).entries()] - .some(([name, value]) => value !== '' && CREDENTIAL_PARAMETERS.has(credentialName(name))); + // Read both ways: with '&' alone separating parameters, as a server + // reading the query as WHATWG does sees them, and with '?' and ';' too, so + // a value that opens with one (`token=;…`) still counts in the first. + const named = (params: string) => [params, params.replace(/[?;]/g, '&')].some((read) => + [...new URLSearchParams(read).entries()] + .some(([name, value]) => value !== '' && CREDENTIAL_PARAMETERS.has(credentialName(name)))); return named(u.search) || named(u.hash.slice(1)); } @@ -346,9 +351,10 @@ export function normalizedOrigin(url: string): string | null { } } -/** Whether two urls that parse name one host, on the same port, explicit or - * each its scheme's default: `http://h` and `wss://h`, or `http://h:443` and - * `wss://h`. */ +/** Whether two urls that parse name one host and port as written: the same + * port, or neither giving one (`http://h` and `wss://h`, the same site over + * plaintext), or one giving the other's default (`http://h:443` and + * `wss://h`). */ function samePlace(a: string, b: string): boolean { const x = new URL(a); const y = new URL(b); @@ -577,9 +583,11 @@ export class ReferenceRegistry { } /** Take back a framework's entry for a server, as it does for a server it - * removes. Another framework's entry for the id, if any, is in force - * again; with none, the id's references are refused as an unknown - * server's, as that framework's fetch refuses them. */ + * removes, and for each of its servers when it stops. Another framework's + * entry for the id, if any, is in force again; with none, a stub reads the + * id's references as an unknown server's. After a removal, that framework's + * fetch refuses them too. A stopped framework keeps its configs, so its + * fetcher, called before it starts again, can still dial them. */ forgetServerAddress(serverId: string, owner: object): void { const entries = (this.addresses.get(serverId) ?? []).filter((e) => e.owner !== owner); if (entries.length > 0) this.addresses.set(serverId, entries); diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index da4919e49..679b3edcc 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -272,6 +272,9 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { 'https://zz-elsewhere.example/#/reset?token=zz', 'https://zz-elsewhere.example/cb?v=1;sig=zz', 'https://zz-elsewhere.example/cb?v=1&token=zz', + // a value that opens with a character read as a separator + 'https://zz-elsewhere.example/cb?token=;zz', + 'https://zz-elsewhere.example/cb#access_token=?zz', // names compared by their letters and digits alone, without case, and OAuth 1.0a's ...['accessToken', 'access-token', 'Api-Key', 'x-api-key', 'private-token', 'oauth_token', 'oauth_signature', 'token[]', 'api.key'] .map((name) => `https://zz-elsewhere.example/cb?${name}=zz`), From adc597e9d84d50fec92673033a5ff3a532ce298b Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 13:30:54 -0400 Subject: [PATCH 03/40] fix(tool-results): send the stored text when links and markers pass the cap The live copy of a tool result that holds an image sends its blocks natively: the image as itself, each text block cut at the cap, and every other block whole. Links' blocks and markers came onto that path when each block began to show on its own. Before, a result holding a link, inline audio or a block of a type the lane doesn't know went as its stored text, cut at the cap. Whole and with no budget across them, they could carry the live copy far past the cap: a link's line carries its uri of up to 4,096 characters. goesNative (tool-result-history.ts) decides for both copies: the live copy goes native only when the result holds an image and its links' blocks and markers fit the cap together. Otherwise it's the stored text, cut at the cap as it is, with the image as its placeholder and ledger ref. A link's block is told apart by its own kind, `link`, from the testimony's fromLink. A reference's stub and a withheld line still go whole beside an image whatever their length, as #403 designed, with the newest 64 image slots the live copy showed kept past the stored copy's cut, so #403's tests stand unchanged. storedToolResultText's shown ranges ask the same function, so the stored copy keeps only slots the live copy showed. Tests: an image beside six links with 1,000-character uris, against a 2,000-character cap, goes as the stored text, while one such link still goes natively, and so for thirty audio markers against 1,000 characters while two go natively; thirty links through a model's round go as the stored copy, the image's placeholder keeping its ledger ref, and with the image after them its slot isn't kept past the cut. Twenty reference stubs, or five and twelve withheld lines, beside an image still go natively past a 1,000-character cap, and a stub longer than a 40-character cap still goes whole. The marker test's cap is raised so its marker fits. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.fixed.md | 2 +- src/framework.ts | 27 +++++---- src/tool-result-history.ts | 40 +++++++++++-- test/tool-result-blocks.test.ts | 76 ++++++++++++++++++++++++- 4 files changed, 125 insertions(+), 20 deletions(-) diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md index 291af8225..a97165a6c 100644 --- a/changelog.d/tool-result-blocks.fixed.md +++ b/changelog.d/tool-result-blocks.fixed.md @@ -1,4 +1,4 @@ -- A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. +- A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. - MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. - A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it says why the fetch would be refused, in the words the refusal gives. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, a uri off its server's origin, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. - An `http` reference on the host and port of a server dialed over a secure connection is refused as `plaintext reference from a secure origin`, where the origin check called it a third-party origin. diff --git a/src/framework.ts b/src/framework.ts index b8d214605..2f9055081 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -112,6 +112,7 @@ import { ProviderWaits, EVERY_MODEL, isReleaseModel, withoutControls, type Provi import type { WorkspaceModule } from './modules/workspace/index.js'; import type { ToolResultGuard, WithheldSpill } from './tool-result-guard.js'; import { + goesNative, nonErrorResultText, toolResultBlocks, toolResultDataToHistoryString, @@ -18401,11 +18402,12 @@ export class AgentFramework { * the image stays reachable by provenance after the bytes leave the wire. * * The live copy is the stored text itself unless the result is a content - * array carrying an inline image, which tryNativeToolResultContent sends - * as native blocks. It takes the blocks the serializer takes - * (toolResultBlocks), and shows each image, stub and marker whole and each - * text block up to where it cuts that block at the cap (`maxChars`, - * toolResultCutAt), so those are the ranges of the text it showed. + * array carrying an inline image whose links and markers fit the cap + * together (goesNative), which tryNativeToolResultContent sends as native + * blocks. It takes the blocks the serializer takes (toolResultBlocks), and + * shows each image, stub and marker whole and each text block up to where + * it cuts that block at the cap (`maxChars`, toolResultCutAt), so those are + * the ranges of the text it showed. */ private storedToolResultText( agentName: string, @@ -18425,7 +18427,9 @@ export class AgentFramework { }).ref, onBlocks: (spans) => { blocks = spans; }, }); - if (!blocks?.some((block) => block.kind === 'image')) return { text, shown: null }; + if (!blocks || !goesNative(blocks.map(({ kind, start, end }) => ({ kind, length: end - start })), maxChars)) { + return { text, shown: null }; + } return { text, shown: blocks.map(({ kind, start, end }): [number, number] => @@ -18827,9 +18831,10 @@ export class AgentFramework { /** * If `data` is a content array (toolResultBlocks) carrying at least one - * image, its blocks as Membrane's native ToolResultContentBlock[]: each - * image as itself, and every other block as the text the stored copy shows - * for it. Returns null otherwise, and the live copy is the stored text. + * image, and its links and markers fit the cap together (goesNative), + * its blocks as Membrane's native ToolResultContentBlock[]: each image as + * itself, and every other block as the text the stored copy shows for it. + * Returns null otherwise, and the live copy is the stored text. * `maxChars`, when provided, caps each accompanying text block so an image * inlined alongside an enormous text payload can't blow the context. A * block is cut as the stored copy is (toolResultCutAt), never inside a @@ -18844,7 +18849,9 @@ export class AgentFramework { spillPath: string | null = null, ): ToolResultContentBlock[] | null { const blocks = toolResultBlocks(data, dataIsContent); - if (!blocks?.some((block) => block.kind === 'image')) return null; + if (!blocks || !goesNative(blocks.map((block) => ({ kind: block.kind, length: block.kind === 'image' ? 0 : block.text.length })), maxChars)) { + return null; + } return blocks.map((block): ToolResultContentBlock => { if (block.kind === 'image') { return { type: 'image', source: { type: 'base64', data: block.data, mediaType: block.mimeType } }; diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index e69564d0f..f24bdee13 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -42,18 +42,42 @@ export interface HistoryBlockSpan { end: number; } +/** + * Whether a tool result's live copy goes as native blocks, an image among + * them, rather than as its stored text. It does when the result holds an + * image and its links' blocks and markers come to no more than `maxChars` + * together. Past that, the live copy is the stored text, cut at the cap, so + * no number of links and markers carries a result past it. A reference's + * stub and a withheld line go whole beside an image whatever their length, + * and past its cut the stored copy keeps the newest 64 image slots the live + * copy showed (#403). Text blocks don't count either: the native copy cuts + * each at the cap on its own. Both copies decide by this, from each block's + * kind and the length of its text. + */ +export function goesNative( + blocks: ReadonlyArray<{ kind: ToolResultBlock['kind']; length: number }>, + maxChars: number | undefined, +): boolean { + if (!blocks.some((block) => block.kind === 'image')) return false; + if (!maxChars) return true; + let bounded = 0; + for (const block of blocks) if (block.kind === 'link' || block.kind === 'marker') bounded += block.length; + return bounded <= maxChars; +} + /** * One block of a tool result's content array as both of its copies show it * (toolResultBlocks): a text block's text, an inline image, which the stored * copy shows as its placeholder and the live copy as itself, or the text that - * stands in its place, the same in both. That text is a reference's stub, - * inline data withheld for a contradiction, or a marker saying what kind of - * block it was and why the lane can't show it. + * stands in its place, the same in both. That text is a reference's stub, a + * link's stub or line (MCP's resource_link, `link`), inline data withheld for + * a contradiction, or a marker saying what kind of block it was and why the + * lane can't show it. */ export type ToolResultBlock = | { kind: 'text'; text: string } | { kind: 'image'; data: string; mimeType: string } - | { kind: 'reference' | 'withheld' | 'marker'; text: string }; + | { kind: 'reference' | 'link' | 'withheld' | 'marker'; text: string }; /** The block types MCP defines for a tool result's content. */ const MCP_BLOCK_TYPES = new Set(['text', 'image', 'audio', 'resource', 'resource_link']); @@ -93,8 +117,12 @@ function toolResultBlock(raw: unknown): ToolResultBlock { // inventory (a failing one: its bytes are behind fetch_reference), so // the index never slides past an image the resident saw. A link the // host can't fetch, shown by its uri, is no slot, as no link was before - // links were references, when the result's JSON showed it. - return { kind: 'reference', text: buildReferenceStub(c.testimony, 'from tool result', undefined, { unfetchedLinkUri: true }) }; + // links were references, when the result's JSON showed it. A link's + // block, stub or line, is `link`, counted toward the cap (goesNative). + return { + kind: c.testimony.fromLink ? 'link' : 'reference', + text: buildReferenceStub(c.testimony, 'from tool result', undefined, { unfetchedLinkUri: true }), + }; case 'invalid': return { kind: 'marker', text: c.shown }; case 'inline': { diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 679b3edcc..6d6e29b92 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -137,12 +137,58 @@ describe("a protocol result's content (dataIsContent)", () => { it('cuts a text block at the cap in the live copy, and shows each marker whole', () => { const long = { type: 'text', text: 'z'.repeat(500) }; - const { live } = copies({ success: true, data: [IMAGE, long, AUDIO], dataIsContent: true }, 40); + const { live } = copies({ success: true, data: [IMAGE, long, AUDIO], dataIsContent: true }, 100); const [, text, marker] = live as Array<{ type: string; text: string }>; - assert.match(text.text, /^z{40}\n\n\[truncated — original was 500 chars\]$/); - assert.ok(AUDIO_TEXT.length > 40); + assert.match(text.text, /^z{100}\n\n\[truncated — original was 500 chars\]$/); assert.equal(marker.text, AUDIO_TEXT); }); + + it('sends the stored text, cut at the cap, once its links pass the cap together', () => { + // Each link line carries its uri whole. Together past the cap, the live + // copy is the stored text cut at the cap, as main sent a result holding a + // link, so no number of them carries the result past it. + const links = Array.from({ length: 6 }, (_, i) => + ({ type: 'resource_link', uri: `https://zz-files.example/${'p'.repeat(1000)}/${i}.csv`, name: `${i}.csv` })); + const { stored, live } = copies({ success: true, data: [IMAGE, TEXT, ...links], dataIsContent: true }, 2000); + assert.equal(live, stored); + assert.match(stored, /^\[image: image\/png, 9B\]\nzz-text\n\[resource link "0\.csv": /); + assert.match(stored, /\[truncated — original was \d+ chars\]$/); + assert.ok(stored.length < 2100, String(stored.length)); + // Within the cap, they go beside the image as before. + const few = copies({ success: true, data: [IMAGE, TEXT, links[0]], dataIsContent: true }, 2000).live; + assert.ok(Array.isArray(few) && (few as Array<{ type: string }>)[0]!.type === 'image', JSON.stringify(few).slice(0, 200)); + }); + + it('counts markers toward that budget too', () => { + const { stored, live } = copies({ success: true, data: [IMAGE, ...Array.from({ length: 30 }, () => AUDIO)], dataIsContent: true }, 1000); + assert.ok(AUDIO_TEXT.length * 30 > 1000); + assert.equal(live, stored); + assert.match(stored, /\[truncated — original was \d+ chars\]$/); + const few = copies({ success: true, data: [IMAGE, AUDIO, AUDIO], dataIsContent: true }, 1000).live as Array<{ type: string; text?: string }>; + assert.deepEqual(few.map((b) => b.text ?? b.type), ['image', AUDIO_TEXT, AUDIO_TEXT]); + }); + + it("sends references' stubs and withheld lines whole beside an image, whatever their length together, as before", () => { + const refs = Array.from({ length: 20 }, (_, i) => ({ type: 'resource', uri: `https://zz-files.example/r${i}.pdf`, name: `r${i}.pdf` })); + const withheld = { ...AUDIO, uri: 'https://zz-files.example/a.wav' }; + for (const data of [[IMAGE, ...refs], [IMAGE, ...refs.slice(0, 5), ...Array.from({ length: 12 }, () => withheld)]]) { + const result: ToolResult = { success: true, data, dataIsContent: true }; + // The stored text before its cut: each block a line of it. + const whole = nonErrorResultText(result); + assert.ok(whole.length > 1000, String(whole.length)); + const blocks = copies(result, 1000).live as Array<{ type: string; text?: string }>; + assert.ok(Array.isArray(blocks), String(blocks).slice(0, 200)); + assert.deepEqual(blocks.map((b) => b.text ?? b.type), whole.split('\n').map((line, i) => (i === 0 ? 'image' : line))); + assert.match(blocks[1]!.text!, /^\[ref_[^\]]+\] r0\.pdf — from tool result — /); + if (data.includes(withheld)) assert.equal(blocks.at(-1)!.text, INLINE_WITHHELD_TEXT); + } + // At a cap shorter than one stub, the live copy shows the stub whole too: + // it cuts only text blocks (#403). + const stub = nonErrorResultText({ data: [IMAGE, refs[0]], dataIsContent: true }).split('\n')[1]!; + assert.ok(stub.length > 40, stub); + const tiny = copies({ success: true, data: [IMAGE, refs[0]], dataIsContent: true }, 40).live as Array<{ type: string; text?: string }>; + assert.deepEqual(tiny.map((b) => b.text ?? b.type), ['image', stub]); + }); }); describe('an array without dataIsContent', () => { @@ -653,6 +699,30 @@ describe('dataIsContent travels with the result', () => { assert.equal(reads, 1); }); + it("through a model's round, an image past the stored copy's cut keeps no slot there when the live copy is the stored text", async () => { + // With the links before the image passing the cap, the model reads the + // stored text, cut before the image's placeholder: no slot of it is kept + // past the cut, since the model never saw it. + const links = Array.from({ length: 30 }, (_, i) => + ({ type: 'resource_link', uri: `https://zz-files.example/${'p'.repeat(1000)}/${i}.csv`, name: `${i}.csv` })); + const { wire, stored } = await answerModelCall({ success: true, data: [TEXT, ...links, IMAGE], dataIsContent: true }); + assert.equal(wire, stored); + const [text] = JSON.parse(stored) as string[]; + assert.ok(!text!.includes('fell past the cut'), text!.slice(-400)); + assert.ok(!/\[image: image\/png/.test(text!), text!.slice(-400)); + }); + + it("through a model's round, sends the stored copy when its links pass the cap together", async () => { + const links = Array.from({ length: 30 }, (_, i) => + ({ type: 'resource_link', uri: `https://zz-files.example/${'p'.repeat(1000)}/${i}.csv`, name: `${i}.csv` })); + const { wire, stored } = await answerModelCall({ success: true, data: [IMAGE, TEXT, ...links], dataIsContent: true }); + assert.equal(wire, stored); + const [text] = JSON.parse(stored) as string[]; + // The image's placeholder keeps its ledger ref, and the text is cut at the cap. + assert.match(text!, /^\[image: image\/png, 9B, ref img_[a-z0-9]+_\d+\]\nzz-text\n/); + assert.ok(text!.length < 24_000 + 500, String(text!.length)); + }); + it("through a model's round, from a module's result: both copies show its blocks", async () => { for (const dataIsContent of [true, undefined]) { const { wire, stored } = await answerModelCall({ success: true, data: [IMAGE, TEXT, AUDIO, VIDEO], ...(dataIsContent ? { dataIsContent } : {}) }); From 930b5419e688bb40d6be7a7052252ffdbd1439c8 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 14:01:31 -0400 Subject: [PATCH 04/40] docs(changelog): a stub's refusal words after #531; no plaintext refusal The merge of main took #531's refusals into dialServer and dialCheck, and dropped the plaintext-from-secure refusal, as #531 did: an http reference from a server dialed over TLS is refused as off its server's origin, and that clause names both addresses. tool-result-blocks.fixed.md no longer describes the plaintext refusal. It says a stub ends with what fetch_reference would say happened, without what calling again can do, and it adds the refusals #531 brought to what a stub covers: a server whose address doesn't parse, and a uri with a user name or password. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.fixed.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md index a97165a6c..d758af85f 100644 --- a/changelog.d/tool-result-blocks.fixed.md +++ b/changelog.d/tool-result-blocks.fixed.md @@ -1,7 +1,6 @@ - A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. - MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. -- A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it says why the fetch would be refused, in the words the refusal gives. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, a uri off its server's origin, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. -- An `http` reference on the host and port of a server dialed over a secure connection is refused as `plaintext reference from a secure origin`, where the origin check called it a third-party origin. +- A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it ends with what `fetch_reference` would say happened, without what calling again can do. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, one whose address doesn't parse, a uri off its server's origin, one with a user name or password, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. - A label in a stub (a name, a type, a server's id, a provenance or a saved path) loses any line or paragraph separator (U+2028, U+2029), as it loses control characters. - A script's demand fetch of a reference not yet fetched reads that reference's own digest and expiry, where it read whatever an earlier reference to the same uri had claimed. - Inline data beside an empty `uri` is shown as any inline data is, on every lane, where it was withheld as data that also claimed a link. From 59b1bef0c8f9840d5ef75e1ce42e387f0580203d Mon Sep 17 00:00:00 2001 From: Cordelia-2434 Date: Sun, 18 Oct 2026 14:09:02 -0400 Subject: [PATCH 05/40] test(mcpl): the fetcher's own clauses name a server by its bounded id A server's id is bounded and stripped of controls in a stub and in fetch_reference's refusals alike (shownServerId). No test gave a server an id that changes, so the fetcher's clauses after dialing, which name the server through it, and its budget refusals were unpinned. One test now sends a server whose id carries a bidi control and runs past 120 characters through an answer's status, a spent budget and a claim past what's left of one, and checks each clause holds the bounded id. Co-Authored-By: Claude Opus 5.5 --- test/mcpl-reference-fetcher.test.ts | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/test/mcpl-reference-fetcher.test.ts b/test/mcpl-reference-fetcher.test.ts index 773b5ecc2..e814707f4 100644 --- a/test/mcpl-reference-fetcher.test.ts +++ b/test/mcpl-reference-fetcher.test.ts @@ -9,7 +9,7 @@ import assert from 'node:assert/strict'; import { createServer } from 'node:http'; import { createHash } from 'node:crypto'; import { DEFAULT_FETCH_MAX_BYTES, ReferenceFetcher, normalizedOrigin, type ServerResolution } from '../src/mcpl/reference-fetcher.js'; -import { EXPIRY_REASON, noAddressReason, ReferenceRegistry, UNBOUND_REASON, classifyBlock, referenceExpiry, referenceRegistry } from '../src/mcpl/references.js'; +import { EXPIRY_REASON, noAddressReason, ReferenceRegistry, UNBOUND_REASON, classifyBlock, referenceExpiry, referenceRegistry, shownServerId } from '../src/mcpl/references.js'; import { AgentFramework } from '../src/framework.js'; const PAYLOAD = Buffer.from('RIFF-not-really-a-wav-'.repeat(100)); @@ -989,6 +989,25 @@ test('vector 13: magic bytes beat a lying Content-Type header for storage type', assert.ok(rec.fetchedPath!.endsWith('.wav'), rec.fetchedPath); }); +test("the fetcher's own clauses name a server by its id as a stub does, bounded and with its controls removed", async () => { + const reg = new ReferenceRegistry(); + const id = `zz-\u202E${'x'.repeat(300)}`; + const shown = shownServerId(id); + // An answer's status, which every clause after dialing names the server in. + const answered = await makeFetcher({ [id]: { url: `ws://127.0.0.1:${port}` } }).fetch(makeRecord(reg, `${ORIGIN}/status/503`, {}, id)); + assert.equal(answered.error, `the "${shown}" server answered HTTP 503`); + // A spent budget, and a claim past what's left of one. + const spent = await makeFetcher({ [id]: { url: `ws://127.0.0.1:${port}`, autofetch: { maxTotalBytes: 0 } } }) + .fetch(makeRecord(reg, `${ORIGIN}/ok?shown-id-spent`, {}, id)); + assert.equal(spent.error, `this host's fetch budget for the "${shown}" server is used up (0 of 0 bytes)`); + const claimed = await makeFetcher({ [id]: { url: `ws://127.0.0.1:${port}`, autofetch: { maxTotalBytes: 1000 } } }) + .fetch(makeRecord(reg, `${ORIGIN}/ok?shown-id-claim`, { sizeBytes: 2000 }, id), { maxBytes: 1500 }); + assert.equal(claimed.error, `it claims 2000 bytes, more than the 1000 bytes left of this host's fetch budget for the "${shown}" server`); + for (const error of [answered.error, spent.error, claimed.error]) { + assert.ok(!error!.includes('\u202E') && !error!.includes('x'.repeat(200)), error); + } +}); + test("each HTTP status says what calling again can do: later, a fresh reference, the host's credentials, or nothing", async () => { const reg = new ReferenceRegistry(); const f = makeFetcher({ srv: { url: ORIGIN } }); From ec8809b98462c0ff834d3208b34af3297cff4c8d Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 19:11:23 -0400 Subject: [PATCH 06/40] fix(tool-results): one classification for both copies, references registered first, one resolver answer per fetch From a whole-PR review of this change at 34418e2b9. - A tool result's live copy is built from the blocks its stored copy was made from (StoredToolResult.live), never classified again. A link's text reads the reference registry and the clock, and the spill's write lies between the two copies, so two classifications could decide goesNative differently: the live copy could then show an image whose slot the stored copy didn't keep. The path with no stored copy classifies once on its own (tryNativeToolResultContent, nativeContent). - A result's references are registered before any block is shown, in a content array (toolResultBlocks) and at any depth in data (registerReferencesIn, before historyJson), so a disposition:"never" that a later block gives a uri holds for an earlier link to it. - A resource_link at any depth is governed whatever its uri, so one with an empty or non-string uri is its marker there, as on the content lanes, never its fields. - The fetcher reads each server's resolution once per check (chosen), so the address the origin check passes and the token the fetch sends come from one answer; bindingFor, whose refusals that makes unreachable, goes. - An MCPL result with an element that isn't a block, null among them, shows that element's marker: mcplResultText, the model path's non-text scan and the channel-id lookup no longer dereference it. - A script's object for a link gives `expired`, as `unfetchable`, only where the model's line is a stub; a link line that shows its uri gives neither. - CREDENTIAL_PARAMETERS adds auth_token, api_token, session_id and jsessionid (Ada-1017's call). - Docs: shownLinkUri sits above buildReferenceStub's doc, ServerAddress says which servers have no address, and the comments that said no uri reaches the model name the link exception. cutStoredToolResult's doc names goesNative. The fixed fragment says a link at depth reads as in a content array and that a module's array is content by the flag only if it sets it; the changed fragment names MCPL and modules' results and says a standard server's links show their uri. Tests: a later never withholding an earlier link's uri, in a content array and at depth; a link with no usable uri at depth as its marker; the live copy built from the stored pass's blocks; the cap's exact boundary; a result's own dataIsContent over the options'; each credential parameter alone; an audio marker's bounded type; no expiry on a script's object for a shown link; the resolver asked once per check; an MCPL result with a null element; save-recent-image's longest stub reaching every ending; a runtime connect recording its address (mcpl-runtime-lifecycle); the bounded server id pinned literally. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- changelog.d/tool-result-blocks.fixed.md | 4 +- src/framework.ts | 81 ++++++++++------ src/mcpl/reference-fetcher.ts | 45 ++++----- src/mcpl/references.ts | 55 ++++++----- src/tool-image-ledger.ts | 8 +- src/tool-result-history.ts | 65 ++++++++++--- test/mcpl-reference-binding.test.ts | 5 +- test/mcpl-reference-fetcher.test.ts | 2 + test/mcpl-runtime-lifecycle.test.ts | 4 + test/save-recent-image.test.ts | 11 ++- test/script-structured-references.test.ts | 5 +- test/tool-result-blocks.test.ts | 107 +++++++++++++++++++++- 13 files changed, 296 insertions(+), 98 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 7f9fb9e28..267e110a3 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1 +1 @@ -- In a tool result, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md index d758af85f..7d370e33e 100644 --- a/changelog.d/tool-result-blocks.fixed.md +++ b/changelog.d/tool-result-blocks.fixed.md @@ -1,5 +1,5 @@ -- A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. -- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. +- A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's that doesn't set `dataIsContent`, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. +- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link with no usable uri is the marker that says so. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. - A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it ends with what `fetch_reference` would say happened, without what calling again can do. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, one whose address doesn't parse, a uri off its server's origin, one with a user name or password, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. - A label in a stub (a name, a type, a server's id, a provenance or a saved path) loses any line or paragraph separator (U+2028, U+2029), as it loses control characters. - A script's demand fetch of a reference not yet fetched reads that reference's own digest and expiry, where it read whatever an earlier reference to the same uri had claimed. diff --git a/src/framework.ts b/src/framework.ts index 582184f99..b4c03762a 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -120,6 +120,7 @@ import { truncateForHistory, DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS, type HistoryBlockSpan, + type ToolResultBlock, } from './tool-result-history.js'; import { TOOL_IMAGE_REF_RE, @@ -352,7 +353,7 @@ const SEND_ENGAGEMENT_TOOLS = new Set([ function extractChannelIdFromToolResult( content: Array<{ type: string; text?: string }> | undefined, ): string | undefined { - const text = content?.find((c) => c.type === 'text' && c.text)?.text; + const text = content?.find((c) => c?.type === 'text' && c.text)?.text; if (!text) return undefined; try { const parsed = JSON.parse(text) as { channelId?: unknown }; @@ -364,6 +365,16 @@ function extractChannelIdFromToolResult( } } +/** A tool result's stored copy as the history path wrote it: its text, + * spilled or cut at the cap, the spill file, and, where its live copy goes + * as native blocks (goesNative), the blocks the text was made from, which + * the live copy is built from (toMembraneToolResult). */ +interface StoredToolResult { + text: string; + filePath: string | null; + live: ToolResultBlock[] | null; +} + /** * How the host reaches an MCPL server, for fetching its references: the url * it dials the server at, as the transport decides. None for a server the @@ -389,8 +400,10 @@ function serverAddressOf(cfg: McplServerConfig): ServerAddress { * too, has no text. */ function mcplResultText(content: McpToolResultContent[] | undefined): string { + // An element that isn't a block, null among them, has no text: it shows as + // its marker where the content is shown block by block. return (Array.isArray(content) ? content : []) - .filter((c) => c.type === 'text') + .filter((c) => c?.type === 'text') .map((c) => (typeof c.text === 'string' ? c.text : NON_STRING_TEXT_PLACEHOLDER)) .filter((text) => text !== '') .join('\n'); @@ -5152,7 +5165,7 @@ export class AgentFramework { * the eager ceiling, connection origin — never worse than the server * having inlined the bytes). Failures are non-fatal: the stub then says * fetch_reference instead of a path. - * - `script`: as `model`, then fetch every reference ("eagerly lazy"): a + * - `script`: as `model`, then fetch every reference but a link ("eagerly lazy"): a * script receiving a result is the strongest available signal that the * bytes are wanted (scripts process, they do not browse), so the stub it * receives carries a workspace path it can open directly. @@ -18606,13 +18619,13 @@ export class AgentFramework { maxChars: number | undefined, ): Promise<{ blocks: ContentBlock[]; - spilled: Map; + spilled: Map; }> { - const spilled = new Map(); + const spilled = new Map(); const dateLabel = new Date().toISOString().slice(0, 10); for (const tc of toolResults) { - const { text, shown } = this.storedToolResultText(agentName, tc, maxChars); - spilled.set(tc.id, await this.spillOrTruncate(text, maxChars, `${dateLabel}-${tc.id}`, shown)); + const { text, shown, live } = this.storedToolResultText(agentName, tc, maxChars); + spilled.set(tc.id, { ...await this.spillOrTruncate(text, maxChars, `${dateLabel}-${tc.id}`, shown), live }); } const blocks: ContentBlock[] = toolResults.map(tc => ({ type: 'tool_result' as const, @@ -18646,10 +18659,11 @@ export class AgentFramework { agentName: string, tc: CompletedToolCall, maxChars: number | undefined, - ): { text: string; shown: ShownRanges | null } { - if (tc.result.isError) return { text: tc.result.error ?? 'Unknown error', shown: null }; + ): { text: string; shown: ShownRanges | null; live: ToolResultBlock[] | null } { + if (tc.result.isError) return { text: tc.result.error ?? 'Unknown error', shown: null, live: null }; const ledger = this.toolImageLedgerFor(agentName); let blocks = null as HistoryBlockSpan[] | null; + let classified = null as ToolResultBlock[] | null; const text = nonErrorResultText(tc.result, undefined, { imageRef: (blockIndex, image) => ledger.retain({ toolCallId: tc.id, @@ -18658,15 +18672,16 @@ export class AgentFramework { data: image.data, mediaType: image.mimeType, }).ref, - onBlocks: (spans) => { blocks = spans; }, + onBlocks: (spans, result) => { blocks = spans; classified = result; }, }); - if (!blocks || !goesNative(blocks.map(({ kind, start, end }) => ({ kind, length: end - start })), maxChars)) { - return { text, shown: null }; + if (!blocks || !classified || !goesNative(blocks.map(({ kind, start, end }) => ({ kind, length: end - start })), maxChars)) { + return { text, shown: null, live: null }; } return { text, shown: blocks.map(({ kind, start, end }): [number, number] => [start, kind === 'text' && maxChars ? start + toolResultCutAt(text.slice(start, end), maxChars) : end]), + live: classified, }; } @@ -18929,8 +18944,8 @@ export class AgentFramework { // RFC-005 "eagerly lazy": a script receiving a result is the strongest // available signal that the bytes are wanted (scripts process, they do // not browse), so the stub the script receives carries a workspace path - // it can open directly. An MCPL peer's references were fetched for the - // script at dispatch (autoFetchReferences). Any other tool's content + // it can open directly. An MCPL peer's references, but its links, were + // fetched for the script at dispatch (autoFetchReferences). Any other tool's content // array, a module's, is materialized here. Only a reference registered // against its server can be fetched: a module's result can carry content // it got from a call of its own (ctx.callTool), whose references were @@ -19099,9 +19114,11 @@ export class AgentFramework { callId: string, afResult: ToolResult, maxChars?: number, - /** Pre-spilled outcome from the history path — used for the non-image - * path so the live wire copy byte-matches what the window stored. */ - precomputed?: { text: string; filePath: string | null }, + /** The history path's outcome: its stored text, spilled or cut, and the + * blocks it was made from where the live copy goes natively. The live + * copy is built from these, never classified again, so the two copies + * agree on every block and on whether it goes natively (goesNative). */ + precomputed?: StoredToolResult, ): MembraneToolResult { if (afResult.isError) { // Same spill policy as successes: reuse the history copy so a giant @@ -19117,14 +19134,17 @@ export class AgentFramework { // blocks natively rather than JSON-stringifying them away. Anything else // goes as text. The error path was handled above, so isError is always // false on these return paths. - const blocks = this.tryNativeToolResultContent( - afResult.data, Boolean(afResult.dataIsContent), maxChars, precomputed?.filePath ?? null); + if (precomputed !== undefined) { + return { + toolUseId: callId, + content: precomputed.live ? this.nativeContent(precomputed.live, maxChars, precomputed.filePath) : precomputed.text, + isError: false, + }; + } + const blocks = this.tryNativeToolResultContent(afResult.data, Boolean(afResult.dataIsContent), maxChars); if (blocks) { return { toolUseId: callId, content: blocks, isError: false }; } - if (precomputed !== undefined) { - return { toolUseId: callId, content: precomputed.text, isError: false }; - } // Without the history path's copy, the text is made as that copy is // (nonErrorResultText), so a reference's uri stays out of it at any depth // (RFC-005), a result with no data reads its error text, or no text, and @@ -19147,14 +19167,23 @@ export class AgentFramework { data: unknown, dataIsContent: boolean, maxChars?: number, - /** Spill file written by the history path — referenced from truncation - * notices so an image-adjacent giant text block stays recoverable. */ - spillPath: string | null = null, ): ToolResultContentBlock[] | null { const blocks = toolResultBlocks(data, dataIsContent); if (!blocks || !goesNative(blocks.map((block) => ({ kind: block.kind, length: block.kind === 'image' ? 0 : block.text.length })), maxChars)) { return null; } + return this.nativeContent(blocks, maxChars, null); + } + + /** A result's blocks as Membrane's native content: each image as itself, + * each text block cut at the cap, every other block whole. */ + private nativeContent( + blocks: ToolResultBlock[], + maxChars: number | undefined, + /** Spill file written by the history path — referenced from truncation + * notices so an image-adjacent giant text block stays recoverable. */ + spillPath: string | null, + ): ToolResultContentBlock[] { return blocks.map((block): ToolResultContentBlock => { if (block.kind === 'image') { return { type: 'image', source: { type: 'base64', data: block.data, mediaType: block.mimeType } }; @@ -24508,7 +24537,7 @@ export class AgentFramework { // for callers that expect data to be string-ish (mcplResultText). let textContent = mcplResultText(blocks); if (blocks.length === 0 && hasStructured) textContent = JSON.stringify(result.structuredContent) ?? 'null'; - const hasNonText = blocks.some((c) => c.type !== 'text'); + const hasNonText = blocks.some((c) => c?.type !== 'text'); const structured = hasStructured ? { structured: result.structuredContent } : {}; // Blocks passed through are marked as content (dataIsContent), so each // shows as itself, its stub or its marker, never as the array's JSON. diff --git a/src/mcpl/reference-fetcher.ts b/src/mcpl/reference-fetcher.ts index 626225cab..e5750f2ea 100644 --- a/src/mcpl/reference-fetcher.ts +++ b/src/mcpl/reference-fetcher.ts @@ -33,7 +33,7 @@ import { createHash } from 'node:crypto'; import type { ReferenceRecord, Refusal, ServerAddress } from './references.js'; import { boundedForLog, describeCause } from './errors.js'; -import { dialCheck, dialServer, FRESH_REFERENCE, noAddressReason, NOTHING_TO_DO, REFERENCE_LIMITS, sanitizeLabel, SERVER_BACK, shownServerId } from './references.js'; +import { dialCheck, dialServer, FRESH_REFERENCE, NOTHING_TO_DO, REFERENCE_LIMITS, sanitizeLabel, shownServerId } from './references.js'; export { normalizedOrigin, type Refusal } from './references.js'; @@ -53,8 +53,8 @@ const EAGER_FETCH_TIMEOUT_MS = 10_000; // doesn't know where a reference came from (a tool result, a push, a channel // message), so none names a tool, and fetch_reference's max_bytes is named // only where a larger one can fetch it. The refusals before dialing that a -// stub shares take theirs from references.ts (NOTHING_TO_DO, FRESH_REFERENCE, -// SERVER_BACK). +// stub shares come from references.ts (dialServer, dialCheck) with their +// remedies (NOTHING_TO_DO, FRESH_REFERENCE, SERVER_BACK). const TRY_LATER = 'Trying again later may work.'; /** A request that failed before an answer: a refused or reset connection * can clear, and a cause that repeats on every try (a port fetch refuses, a @@ -195,26 +195,31 @@ export class ReferenceFetcher { private readonly saveFile: (fileName: string, data: Buffer, mimeType: string) => Promise, ) {} - /** How the host reaches a server, as the resolver answers: what dialCheck - * and dialServer read for a fetch. */ - private readonly addressOf = (serverId: string): ServerAddress | undefined => { - const resolution = this.resolveServer(serverId); - return resolution === undefined || resolution === 'no-address' ? resolution : { url: resolution.url }; - }; - - /** A server's credential and limits, for a server dialServer chose. */ - private bindingFor(serverId: string): { serverId: string; binding: ResolvedServerBinding } | { refusal: Refusal } { - const binding = this.resolveServer(serverId); - if (binding === undefined) return refused(`this host no longer has the "${shownServerId(serverId)}" server it came from`, SERVER_BACK); - if (binding === 'no-address') return refused(noAddressReason(shownServerId(serverId)), NOTHING_TO_DO); - return { serverId, binding }; + /** The server a check (dialServer or dialCheck) chose, with its credential + * and limits from the same resolver answer whose address the check read: + * each server's resolution is read once here, so the origin the check + * passes and the token the fetch sends come from one answer, whatever a + * resolver answers the next time it's asked. */ + private chosen( + check: (addressOf: (serverId: string) => ServerAddress | undefined) => { serverId: string } | { refusal: Refusal }, + ): { serverId: string; binding: ResolvedServerBinding } | { refusal: Refusal } { + const resolutions = new Map(); + const addressOf = (serverId: string): ServerAddress | undefined => { + if (!resolutions.has(serverId)) resolutions.set(serverId, this.resolveServer(serverId)); + const resolution = resolutions.get(serverId); + return resolution === undefined || resolution === 'no-address' ? resolution : { url: resolution.url }; + }; + const server = check(addressOf); + if ('refusal' in server) return server; + // The check chose only a server whose address it read as a url, so that + // server's resolution is a binding. + return { serverId: server.serverId, binding: resolutions.get(server.serverId) as ResolvedServerBinding }; } /** The server a fetch of a record dials (§6.2), with its credential and * limits, or why none can be. */ private serverFor(record: ReferenceRecord): { serverId: string; binding: ResolvedServerBinding } | { refusal: Refusal } { - const server = dialServer(record, this.addressOf); - return 'refusal' in server ? server : this.bindingFor(server.serverId); + return this.chosen((addressOf) => dialServer(record, addressOf)); } /** Would the eager (autofetch) default fetch this record? Not one the fetch @@ -244,9 +249,7 @@ export class ReferenceFetcher { * refuse with, and no eager slot goes to a fetch that would be refused. */ private beforeDial(record: ReferenceRecord, opts?: { maxBytes?: number }): { refusal: Refusal } | { serverId: string; binding: ResolvedServerBinding; ceiling: number; budget: number } { - const dial = dialCheck(record, this.addressOf); - if ('refusal' in dial) return dial; - const server = this.bindingFor(dial.serverId); + const server = this.chosen((addressOf) => dialCheck(record, addressOf)); if ('refusal' in server) return server; const { serverId, binding } = server; const t = record.testimony; diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index a463ce61d..6d1b0b3f7 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -20,7 +20,8 @@ * retires). * * The ReferenceRegistry is the host-private reference record (§6.2): raw - * URIs and testimony live here and never in model-visible content. Fetching + * URIs and testimony live here, and model-visible content shows a uri only + * as a tool result's link line shows one (shownLinkUri). Fetching * (§6/§7 origin binding, ceilings, redirects) is deliberately not here yet — * the registry is what makes a future host-mediated fetcher and the * code-execution materialization hook possible. @@ -263,8 +264,10 @@ export function shownServerId(serverId: string): string { * - Backblaze B2's: `https://f000.backblazeb2.com/file/b/k?Authorization=…` * - OAuth's: `#access_token=…&token_type=bearer`, `#id_token=…`, `?refresh_token=…`, `?client_secret=…`, * and OAuth 1.0a's `?oauth_token=…&oauth_signature=…` - * - the generic: `?token=…`, `?auth=…`, `?key=…`, `?apikey=…`, `?api_key=…`, `?x-api-key=…`, `?private_token=…`, - * `?secret=…`, `?password=…`, `?signature=…` + * - the generic: `?token=…`, `?auth=…`, `?auth_token=…`, `?key=…`, `?apikey=…`, `?api_key=…`, `?api_token=…`, + * `?x-api-key=…`, `?private_token=…`, `?secret=…`, `?password=…`, `?signature=…` + * - a session's id, a bearer credential too: `?session_id=…`, `?jsessionid=…` (in a path, as + * `;jsessionid=…` usually rides, it isn't read) * A credential named otherwise, or carried in a path, such as a webhook's * `https://discord.com/api/webhooks//`, isn't caught. */ @@ -277,7 +280,8 @@ const CREDENTIAL_PARAMETERS: ReadonlySet = new Set([ 'authorization', 'access_token', 'id_token', 'refresh_token', 'client_secret', 'oauth_token', 'oauth_signature', - 'token', 'auth', 'key', 'api_key', 'x-api-key', 'private_token', 'secret', 'password', + 'token', 'auth', 'auth_token', 'key', 'api_key', 'api_token', 'x-api-key', 'private_token', 'secret', 'password', + 'session_id', 'jsessionid', ].map(credentialName)); /** A parameter's name as CREDENTIAL_PARAMETERS compares it: its letters and @@ -321,8 +325,10 @@ export function uriMayCarryCredential(uri: string): boolean { } /** How the host reaches a server, as far as fetching its references goes: - * the url it dials the server at, or none, for a server it doesn't connect - * to over a network, such as a stdio server. A framework records each of its + * the WebSocket url it dials an MCPL server at, or none: for a server it + * spawns, such as a stdio one, or a modern MCP server over HTTP, whose + * results carry no references (RFC-005 governs MCPL peers' alone), or one + * whose url the transport won't dial. A framework records each of its * servers' in the registry (ReferenceRegistry.setServerAddress), where a * stub reads it; its fetcher reads the same from the framework's own * configs, beside each server's credential. */ @@ -682,16 +688,6 @@ export interface StubOptions { unfetchedLinkUri?: boolean; } -/** - * Build the model-visible stub for a reference. Its length is bounded by - * host constants and independent of every server-supplied field's length: - * REFERENCE_STUB_MAX_CHARS. Under `disposition:"never"` the uri is omitted - * unconditionally; under `ref`/absent it is omitted too — this host's default - * policy is the opaque id everywhere (the safe default of §5), with retrieval - * via the record. The one exception is a tool result's link the host can't - * fetch (StubOptions.unfetchedLinkUri), which is no stub but a line of its - * own, its uri bounded by REFERENCE_LIMITS.uri. - */ /** * The uri a tool result's link shows, or null where it shows none: a link * (fromLink) the host can't fetch, whose uri can't carry a credential, and @@ -714,6 +710,16 @@ export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, now return uriMayCarryCredential(uri) ? null : uri; } +/** + * Build the model-visible stub for a reference. Its length is bounded by + * host constants and independent of every server-supplied field's length: + * REFERENCE_STUB_MAX_CHARS. Under `disposition:"never"` the uri is omitted + * unconditionally; under `ref`/absent it is omitted too — this host's default + * policy is the opaque id everywhere (the safe default of §5), with retrieval + * via the record. The one exception is a tool result's link the host can't + * fetch (StubOptions.unfetchedLinkUri), which is no stub but a line of its + * own, its uri bounded by REFERENCE_LIMITS.uri. + */ export function buildReferenceStub(t: ReferenceTestimony, provenance?: string, serverId?: string, options?: StubOptions): string { const record = referenceRegistry.register(t, serverId); const parts: string[] = []; @@ -792,12 +798,12 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI * the host's reference id, what the server said about it in this block (its * labels stripped and bounded as in the stub), and `savedTo`, the workspace * path its payload was fetched to, or null when it wasn't (`fetch_reference` - * takes the id as `ref_id`). An unsaved reference that counts as expired - * also carries `expired`. Its uri is left out, except that a link (MCP's + * takes the id as `ref_id`). Its uri is left out, except that a link (MCP's * resource_link) carries `uri` where the model's line for it shows the uri - * (shownLinkUri). Where the stub gives a reason the fetcher would refuse, - * in place of advising fetch_reference, the object carries the same words as - * `unfetchable` (unfetchableReason). The reference is registered bound to `serverId`, the + * (shownLinkUri). Where the model's line is a stub, an unsaved reference + * that counts as expired also carries `expired`, and where the stub gives a + * reason the fetcher would refuse, in place of advising fetch_reference, the + * object carries the same words as `unfetchable` (unfetchableReason). The reference is registered bound to `serverId`, the * server whose tool returned it, as the model's lanes bind a tool result's * references, so the fetch its id offers can be made. A block the classifier * rejects becomes the notice the model's lanes show, and inline data that @@ -814,10 +820,11 @@ export function referenceBlockForScript(block: unknown, serverId: string): unkno case 'reference': { const t = c.testimony; const record = referenceRegistry.register(t, serverId); - // As the stub judges it: a saved copy is still served. - const expiry = record.fetchedPath ? null : referenceExpiry(record.testimony); const uri = shownLinkUri(t, record); - // The reason the stub gives, where it gives one, in the same words. + // What the stub says, where the model's line is a stub: a link line + // that shows its uri says neither. As the stub judges it, a saved copy + // is still served. + const expiry = record.fetchedPath || uri !== null ? null : referenceExpiry(record.testimony); const unfetchable = uri === null ? unfetchableReason(record) : null; return { type: (block as { type: string }).type, diff --git a/src/tool-image-ledger.ts b/src/tool-image-ledger.ts index 26489a03a..026540977 100644 --- a/src/tool-image-ledger.ts +++ b/src/tool-image-ledger.ts @@ -364,10 +364,10 @@ export function toolResultCutAt(text: string, cap: number): number { * * `shown` is what the live copy showed beside the stored text: null when the * live copy was the text itself, as for every result that carried no inline - * image. Then what the cut keeps is exactly what the resident saw, and - * nothing past it is kept (#403). A result that carried an inline image went - * to the model as native blocks (tryNativeToolResultContent, framework.ts), - * every image whatever the text's length, so its stored text is the only + * image, or whose links and markers passed the cap together (goesNative). + * Then what the cut keeps is exactly what the resident saw, and nothing past + * it is kept (#403). A result whose live copy went to the model as native + * blocks (framework.ts) sent every image whatever the text's length, so its stored text is the only * place save_recent_image's inventory finds them, and a slot dropped by the * cut would be an image the resident saw that the index slides past (#104). * The image slots past the cut that lie wholly within what the live copy diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index f24bdee13..184c5060e 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -15,8 +15,10 @@ * other block as a marker saying what it was and why it isn't shown. Anything * else falls back to JSON, written by RFC-005's rules at every depth * (historyJson): a reference is shown as its stub wherever it sits in the - * data, so its uri never reaches history, and inline data that claims a uri - * or a disposition is withheld. + * data, so its uri reaches history only as a link's line shows it where the + * host can't fetch the link and its uri can't carry a credential + * (shownLinkUri), and inline data that claims a uri or a disposition is + * withheld. */ import { @@ -27,6 +29,7 @@ import { INLINE_WITHHELD_TEXT, buildReferenceStub, classifyBlock, + referenceRegistry, referenceStubOrNull, referenceTypeLabel, } from './mcpl/references.js'; @@ -99,9 +102,36 @@ export function toolResultBlocks(data: unknown, dataIsContent?: boolean): ToolRe if (dataIsContent !== true && !data.every((element) => isPlainRecord(element) && MCP_BLOCK_TYPES.has(element.type as string))) { return null; } + // Every reference first, so what one says of a uri (a `disposition: never`, + // an expiry) holds for each block that names it, whatever their order. + for (const raw of data) registerReference(raw); return data.map(toolResultBlock); } +/** Register a block that is a reference, before any block is shown. */ +function registerReference(raw: unknown): void { + const c = classifyBlock(raw); + if (c.kind === 'reference') referenceRegistry.register(c.testimony); +} + +/** Register every reference a tool result's data holds, at any depth, as + * historyJson shows them, before any is shown. A value RFC-005 governs is + * shown whole, so what's inside it isn't visited. */ +function registerReferencesIn(value: unknown, seen: Set = new Set()): void { + if (!value || typeof value !== 'object' || seen.has(value)) return; + seen.add(value); + if (Array.isArray(value)) { + for (const element of value) registerReferencesIn(element, seen); + return; + } + const b = value as Record; + if (governedByRfc005(b)) { + registerReference(b); + return; + } + for (const field of Object.values(b)) registerReferencesIn(field, seen); +} + /** One element of a content array, as toolResultBlocks gives it. Only an * image with data and a media type is shown as itself, as the push lane's * converter shows one, and only text and images can be: this host's tool @@ -164,10 +194,12 @@ export interface HistorySerializeOptions { /** * Called once a content array has been serialized block by block, rather * than given up on for the fallback, with where each block sits in the - * joined text, before any cut at `maxChars`. The framework reads from it - * what a result's live copy showed (#403). + * joined text, before any cut at `maxChars`, and the blocks themselves as + * this text was made from them. The framework reads from it what a + * result's live copy shows (#403), and builds that copy from these very + * blocks, so the two copies come from one classification. */ - onBlocks?: (spans: HistoryBlockSpan[]) => void; + onBlocks?: (spans: HistoryBlockSpan[], blocks: ToolResultBlock[]) => void; } /** @@ -259,7 +291,7 @@ function tryHistoryStringFromContentArray(data: unknown, options?: HistorySerial spans.push({ kind: part.kind, start, end: start + part.text.length }); start += part.text.length + 1; // the newline that joins them } - options.onBlocks(spans); + options.onBlocks(spans, blocks); } return parts.map((part) => part.text).join('\n'); } @@ -267,16 +299,19 @@ function tryHistoryStringFromContentArray(data: unknown, options?: HistorySerial /** * Whether RFC-005 governs a value of a tool result's data, wherever it sits. * A reference is a `resource` with a uri, at its top level or under - * `resource` (where MCP embeds a resource's uri), an `image` or `audio` - * given by uri, or MCP's `resource_link` (RFC-005 §8.1); inline data is an - * `image` or `audio` whose data is a string. A uri that is null, absent or - * empty counts as none, and data that isn't a string isn't inline data, so a - * record that only shares one of those type names is data. A uri of any - * other value, string or not, counts, so a credential fails closed. + * `resource` (where MCP embeds a resource's uri), or an `image` or `audio` + * given by uri; inline data is an `image` or `audio` whose data is a string. + * There a uri that is null, absent or empty counts as none, and data that + * isn't a string isn't inline data, so a record that only shares one of + * those type names is data. A uri of any other value, string or not, counts, + * so a credential fails closed. MCP's `resource_link` (RFC-005 §8.1) is + * governed whatever its uri: one without a usable uri is its marker. */ function governedByRfc005(b: Record): boolean { if (b.type === 'resource') return hasUri(b.uri) || (isPlainRecord(b.resource) && hasUri(b.resource.uri)); - if (b.type === 'resource_link') return hasUri(b.uri); + // A link is governed whatever its uri, so one without a usable uri is its + // marker here, as on the content lanes, never its fields. + if (b.type === 'resource_link') return true; if (b.type === 'image' || b.type === 'audio') return typeof b.data === 'string' || hasUri(b.uri); return false; } @@ -296,7 +331,8 @@ function isPlainRecord(value: unknown): value is Record { * one (such as a uri that isn't a string, or MCP's embedded resource, whose * uri isn't at its top level) as the marker saying why it isn't shown, and * inline data that also claims a uri or a disposition as withheld, so no uri - * is ever shown. Inline data that claims neither is itself, as is any other value: + * is shown but a link's where its line shows one (shownLinkUri). Inline data + * that claims neither is itself, as is any other value: * a block of another type keeps its fields, since RFC-005 governs * references. */ @@ -329,6 +365,7 @@ export function rfc005TextOf(value: unknown): string | null { * as a bare reference, is that text. */ function historyJson(data: unknown): string | undefined { + registerReferencesIn(data); const root = shownInHistory(data); if (root !== data && typeof root === 'string') return root; return JSON.stringify(data, (_key, value: unknown) => shownInHistory(value)) as string | undefined; diff --git a/test/mcpl-reference-binding.test.ts b/test/mcpl-reference-binding.test.ts index f130dab13..4cf065ac8 100644 --- a/test/mcpl-reference-binding.test.ts +++ b/test/mcpl-reference-binding.test.ts @@ -155,8 +155,9 @@ test("fetch_reference reaches a push event's reference through its server's own firstWritableMountName: () => 'scratch', emitTrace: () => {}, }); - // Configured as the framework configures a server, its address recorded - // in the reference registry, where the stub and the fetcher read it. + // Configured as the framework configures a server: its config, which the + // fetcher reads, and its address in the reference registry, which the + // stub reads. for (const config of [ { id: 'push-srv', url: wsOrigin }, { id: 'stdio-srv', command: 'stdio-server' }, diff --git a/test/mcpl-reference-fetcher.test.ts b/test/mcpl-reference-fetcher.test.ts index ae78d7716..5141a9d90 100644 --- a/test/mcpl-reference-fetcher.test.ts +++ b/test/mcpl-reference-fetcher.test.ts @@ -994,6 +994,8 @@ test("the fetcher's own clauses name a server by its id as a stub does, bounded const reg = new ReferenceRegistry(); const id = `zz-\u202E${'x'.repeat(300)}`; const shown = shownServerId(id); + // Bounded at 120 characters, its controls gone, cut with an ellipsis. + assert.equal(shown, `zz-${'x'.repeat(116)}…`); // An answer's status, which every clause after dialing names the server in. const answered = await makeFetcher({ [id]: { url: `ws://127.0.0.1:${port}` } }).fetch(makeRecord(reg, `${ORIGIN}/status/503`, {}, id)); assert.equal(answered.error, `the "${shown}" server answered HTTP 503`); diff --git a/test/mcpl-runtime-lifecycle.test.ts b/test/mcpl-runtime-lifecycle.test.ts index 9f0f5cb6f..761a49249 100644 --- a/test/mcpl-runtime-lifecycle.test.ts +++ b/test/mcpl-runtime-lifecycle.test.ts @@ -14,6 +14,7 @@ import { WebSocketServer } from 'ws'; import type { AddressInfo } from 'node:net'; import { AgentFramework } from '../src/index.js'; +import { referenceRegistry } from '../src/mcpl/references.js'; import type { EventResponse, Module, ModuleContext, ToolDefinition, ToolResult } from '../src/index.js'; import { MockMembrane } from './helpers/mock-membrane.js'; @@ -102,6 +103,9 @@ test('connectMcplServer lazily initializes MCPL on a zero-server framework and e assert.deepEqual(framework.listMcplServers(), [], 'no servers configured at start'); await framework.connectMcplServer(serverConfig('dyn')); + // Its address is recorded where a reference's stub reads it: none, for a + // server the host spawns. + assert.equal(referenceRegistry.serverAddress('dyn'), 'no-address'); const servers = framework.listMcplServers(); assert.equal(servers.length, 1); diff --git a/test/save-recent-image.test.ts b/test/save-recent-image.test.ts index 7789079ff..972762270 100644 --- a/test/save-recent-image.test.ts +++ b/test/save-recent-image.test.ts @@ -1686,6 +1686,9 @@ describe('ToolImageLedger', () => { // isn't named. const longServer = (kind: string) => `${kind}-${'s'.repeat(400)}`; const host = {}; + // srv is dialed on the references' own origin, so its rows reach the + // expiry's reasons and the fetch the stub offers. + referenceRegistry.setServerAddress('srv', { url: 'wss://example.invalid' }, host); referenceRegistry.setServerAddress(longServer('stdio'), 'no-address', host); referenceRegistry.setServerAddress(longServer('elsewhere'), { url: 'wss://elsewhere.invalid' }, host); referenceRegistry.setServerAddress(longServer('dialed'), { url: 'wss://example.invalid' }, host); @@ -1699,6 +1702,11 @@ describe('ToolImageLedger', () => { ['user name or password', { uri: 'https://user:pass@example.invalid/sri-longest-userinfo.png' }, longServer('dialed')], ); const reasons: Record = { + passed: '— the reference is expired', + unparseable: '— the reference is treated as expired, since its expiry time can\'t be read', + rejected: '— the reference is treated as expired, since its expiry time isn\'t in a form this host accepts', + bound: '— fetch with fetch_reference', + unbound: '— it\'s not tied to any server', 'no address': '— this host doesn\'t connect to the "stdio-s', 'unknown server': '— this host no longer has the "unknown-s', 'server address': '— the "garbled-s', @@ -1710,8 +1718,9 @@ describe('ToolImageLedger', () => { const stub = longestId(buildReferenceStub(testimony(name, extra), 'o'.repeat(400), serverId)); assert.ok(stub.length < REFERENCE_STUB_MAX_CHARS, `${name}: ${stub.length}`); assert.strictEqual(parseImagePlaceholders(stub).length, 1, name); - if (reasons[name]) assert.ok(stub.includes(reasons[name]!), `${name}: ${stub}`); + assert.ok(stub.includes(reasons[name]!), `${name}: ${stub}`); } + referenceRegistry.forgetServerAddress('srv', host); // A placeholder: RFC 6838 bounds a type or subtype name at 127 // characters, so normalizeMediaType keeps the longest essence and refuses // one character more, and the size and ref take a safe integer's digits. diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index e534560d6..bb25dea33 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -2,7 +2,8 @@ * What a script receives from a tool whose result carries structuredContent, * through the real dispatch. An MCPL peer's content blocks get the treatment * RFC-005 gives them on every lane the model reads, since what a script prints - * comes back to the model: a reference arrives without its URI, as the host's + * comes back to the model: a reference arrives without its URI (a link + * carries one only where the model's line shows it), as the host's * reference id, what the server said about it in that block, and `savedTo`, * the workspace path the dispatch fetched its payload to for the script; a * block the classifier rejects arrives as the notice the model's lanes show, @@ -169,7 +170,7 @@ test("an MCPL peer's structured result reaches a script with each block treated const out = await script('mcpl--files--attach'); assert.ok(!out.includes('example.invalid'), `no URI reaches the script: ${out.slice(0, 2000)}`); assert.deepEqual(fetched, [uris('treated').report, uris('treated').big], - 'every reference was fetched for the script before it read the result, except the one that counts as expired'); + 'every reference was fetched for the script before it read the result, except the one that counts as expired and the link, which is never fetched for a script'); const object = JSON.parse(out) as { content: Array<{ refId?: string }>; structuredContent: unknown; isError: boolean }; const [report, , big] = object.content; const link = object.content[8]; diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index c8a4e406e..828b4a696 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -44,7 +44,7 @@ import { CoalesceError, validateCoalescedContent } from '../src/mcpl/push-coales import { ReferenceFetcher } from '../src/mcpl/reference-fetcher.js'; import { checkedToolResult } from '../src/module-registry.js'; import { parseImagePlaceholders } from '../src/tool-image-ledger.js'; -import { nonErrorResultText, toolResultDataToHistoryString } from '../src/tool-result-history.js'; +import { nonErrorResultText, toolResultBlocks, toolResultDataToHistoryString } from '../src/tool-result-history.js'; import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; import { fixture } from './helpers/coalescing-fixture.js'; @@ -902,3 +902,108 @@ async function answerModelCall(answer: ToolResult): Promise<{ wire: string; stor rmSync(dir, { recursive: true, force: true }); } } + +describe('one answer for every block of a result', () => { + it("withholds a link's uri for a later reference to it with disposition never, whatever the blocks' order", () => { + const uri = 'https://zz-elsewhere.example/order-never.csv'; + const link = { type: 'resource_link', uri, name: 'order.csv' }; + const never = { type: 'resource', uri, name: 'order.csv', disposition: 'never' }; + const shown = toolResultDataToHistoryString([TEXT, link, never]); + assert.ok(!shown.includes(uri), shown); + assert.match(shown.split('\n')[1]!, /^\[ref_[^\]]+\] order\.csv — from tool result — /); + // At any depth in data too. + const uri2 = 'https://zz-elsewhere.example/order-never-deep.csv'; + const deep = toolResultDataToHistoryString({ first: [{ ...link, uri: uri2 }], then: { ref: { ...never, uri: uri2 } } }); + assert.ok(!deep.includes(uri2), deep); + }); + + it('shows a link with no usable uri at any depth as its marker, never its fields', () => { + for (const uri of ['', 42, null]) { + const shown = JSON.parse(toolResultDataToHistoryString({ found: [{ type: 'resource_link', uri, name: 'x', description: 'zz-link-description' }] })); + assert.equal(typeof shown.found[0], 'string', String(uri)); + assert.ok(!JSON.stringify(shown).includes('zz-link-description'), String(uri)); + } + }); + + it("builds the live copy from the stored pass's own blocks, never classifying them again", () => { + const fw = framework() as unknown as { + toMembraneToolResult(callId: string, result: ToolResult, maxChars: number | undefined, precomputed: unknown): Live; + }; + const result: ToolResult = { success: true, data: [IMAGE, PLAIN_LINK], dataIsContent: true }; + // What the stored pass decided, whatever the registry says now. + const live = fw.toMembraneToolResult('zz', result, 1000, { + text: 'zz-stored', filePath: null, + live: [{ kind: 'image', data: PNG, mimeType: 'image/png' }, { kind: 'link', text: 'zz-as-the-stored-pass-wrote-it' }], + }).content as Array<{ type: string; text?: string }>; + assert.deepEqual(live.map((b) => b.text ?? b.type), ['image', 'zz-as-the-stored-pass-wrote-it']); + // And where it decided the stored text, that's the live copy. + assert.equal(fw.toMembraneToolResult('zz', result, 1000, { text: 'zz-stored', filePath: null, live: null }).content, 'zz-stored'); + }); + + it('goes natively with links and markers exactly at the cap, and as the stored text one character past it', () => { + const cap = AUDIO_TEXT.length * 2; + const at = copies({ success: true, data: [IMAGE, AUDIO, AUDIO], dataIsContent: true }, cap).live; + assert.ok(Array.isArray(at), String(at)); + const past = copies({ success: true, data: [IMAGE, AUDIO, AUDIO], dataIsContent: true }, cap - 1); + assert.equal(past.live, past.stored); + }); + + it("reads a result's own dataIsContent, whatever the options say", () => { + const data = [TEXT, { type: 'zz-other' }]; + // The result says data, so the array is JSON, though the options say content. + assert.equal(nonErrorResultText({ data, dataIsContent: false }, undefined, { dataIsContent: true }), JSON.stringify(data)); + // The result says content, so each block shows on its own. + assert.equal(nonErrorResultText({ data, dataIsContent: true }, undefined, { dataIsContent: false }), `zz-text\n${unknownBlockText({ type: 'zz-other' })}`); + }); + + it('hides a link uri with any one credential parameter, each on its own', () => { + for (const name of ['X-Amz-Signature', 'X-Amz-Credential', 'X-Amz-Security-Token', 'X-Goog-Signature', 'X-Goog-Credential', + 'sig', 'signature', 'Key-Pair-Id', 'Policy', 'hm', 'Authorization', 'access_token', 'id_token', 'refresh_token', + 'client_secret', 'oauth_token', 'oauth_signature', 'token', 'auth', 'auth_token', 'key', 'api_key', 'api_token', + 'x-api-key', 'private_token', 'secret', 'password', 'session_id', 'jsessionid']) { + const uri = `https://zz-elsewhere.example/one-${name}.csv?v=1&${name}=zz`; + const line = toolResultDataToHistoryString([TEXT, { type: 'resource_link', uri, name: 'one.csv' }]).split('\n')[1]!; + assert.ok(!line.includes('zz-elsewhere.example'), `${name}: ${line}`); + } + }); + + it("bounds an audio marker's type and strips its controls", () => { + const type = `audio/\u202Ewav\u0007${'x'.repeat(300)}`; + const [marker] = toolResultBlocks([{ type: 'audio', data: BASE64, mimeType: type }], true)! as Array<{ text: string }>; + assert.ok(!marker!.text.includes('\u202E') && !marker!.text.includes('\u0007'), marker!.text); + assert.ok(marker!.text.length < 250, String(marker!.text.length)); + }); + + it("gives a script's object no expiry for a link whose line shows its uri, as the line gives none", () => { + const uri = 'https://zz-elsewhere.example/expired-link.csv'; + referenceRegistry.register(testimonyOf({ type: 'resource', uri, name: 'expired.csv', expiresAt: '2001-01-01T00:00:00Z' })); + const link = { type: 'resource_link', uri, name: 'expired.csv' }; + const object = referenceBlockForScript(link, 'zz-files') as { uri?: string; expired?: string; unfetchable?: string }; + assert.equal(object.uri, uri); + assert.equal(object.expired, undefined); + assert.equal(object.unfetchable, undefined); + assert.ok(toolResultDataToHistoryString([TEXT, link]).endsWith(`[resource link "expired.csv": ${uri}; not fetched]`)); + }); + + it("asks the resolver once for each server a fetch's check reads, so its address and its credential are one answer", () => { + const record = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'https://zz-refs.example/once.bin', name: 'once.bin' }), 'zz-once'); + let asked = 0; + const fetcher = new ReferenceFetcher(() => { + asked += 1; + // A resolver that would answer elsewhere the second time it's asked. + return asked === 1 ? { url: 'wss://zz-refs.example' } : { url: 'wss://zz-other.example' }; + }, async () => null); + assert.equal(fetcher.refusal(record), null); + assert.equal(asked, 1); + }); + + it("makes an MCPL result with an element that isn't a block its marker, not a failure", async () => { + const fw = framework() as unknown as { + mcpToolResult(result: unknown, callId: string, mcplPeer: boolean, serverId: string, callName: string): Promise; + }; + const result = await fw.mcpToolResult({ content: [null, TEXT] }, 'zz-null', true, 'zz-server', 'zz-tool'); + assert.equal(result.success, true); + assert.equal(result.dataIsContent, true); + assert.equal(nonErrorResultText(result), `${unknownBlockText(null)}\nzz-text`); + }); +}); From b98c56a20d2d7e20e74cd053e7af305bcd5909a0 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 19:14:12 -0400 Subject: [PATCH 07/40] docs(changelog): a script's reference object carries the stub's words only where the line is a stub mcp-standard-tool-results.changed.md, unreleased, said an unsaved reference that counts as expired carries `expired`. Since a script's object for a link gives `expired`, as `unfetchable`, only where the model's line for it is a stub, a link whose line shows its uri carries neither. The sentence now says so, and keeps that a reference that counts as expired isn't fetched. Co-Authored-By: Claude Opus 5.5 --- changelog.d/mcp-standard-tool-results.changed.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/changelog.d/mcp-standard-tool-results.changed.md b/changelog.d/mcp-standard-tool-results.changed.md index b1a78672f..9a6139019 100644 --- a/changelog.d/mcp-standard-tool-results.changed.md +++ b/changelog.d/mcp-standard-tool-results.changed.md @@ -43,10 +43,11 @@ the script (`null` when it couldn't; `fetch_reference` takes the id as its `ref_id` argument). It arrives without its URI, except a link (MCP's `resource_link`) that carries `uri` where the model's line for it shows - the uri. An unsaved reference that counts as expired also carries - `expired`, the reason the model's stub gives, and isn't fetched. One the - fetcher would refuse carries `unfetchable`, the words the model's stub - gives in place of advising `fetch_reference`. + the uri. An unsaved reference that counts as expired isn't fetched. + Where the model's line for a reference is a stub, the object also + carries what the stub says in place of advising `fetch_reference`: + `expired`, the reason, for one that counts as expired, and + `unfetchable`, the stub's words, for one the fetcher would refuse. A block the classifier rejects arrives as the notice the model reads in its place, and inline data that also claims a bulk disposition as the withheld notice. #181, by @Meganeuridae, first From 0395acd48fe6e71ff33b7b08668140847e0e326a Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 19:17:12 -0400 Subject: [PATCH 08/40] docs(references): dialCheck's doc says a stopped framework's stubs and fetch differ dialCheck's doc said that with one framework, or frameworks that agree on each id they share, a stub and the fetch read the same addresses. A stopped framework's addresses are out of the reference registry while its configs still resolve them, so that holds only for running frameworks. The description's sentence on the same point was corrected from the whole-PR review (af-f53-sol I11); this is the same claim beside it. Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 6d1b0b3f7..64ba43402 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -470,11 +470,14 @@ export function dialServer( * expiry. A pure function of the record and the servers' addresses: a * reference's stub asks it of the registry's (unfetchableReason), and the * fetcher's pre-dial check of its own framework's configs, whose credential - * goes with the address it checks. In a process with one framework, or with - * frameworks that agree on each id they share, the two sources hold the same - * addresses, so a stub promises what the fetch does. Where two frameworks - * configure one id differently, only a stub's words can be wrong, never where - * a token goes. A fetch's own limits, its budget and ceiling, come after it. + * goes with the address it checks. In a process with one running framework, + * or with running frameworks that agree on each id they share, the two + * sources hold the same addresses, so a stub promises what the fetch does. A + * stopped framework's addresses are out of the registry, so its stubs say its + * servers are unknown while its configs still resolve them. Where two + * frameworks configure one id differently, only a stub's words can be wrong, + * never where a token goes. A fetch's own limits, its budget and ceiling, + * come after it. */ export function dialCheck( record: ReferenceRecord, From 45c782d031710a7a4f1d954079a144b721144df0 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 19:29:26 -0400 Subject: [PATCH 09/40] test(tool-results): read #538's script result as { text, isError } in the on-demand fetch test #538 made handleScriptToolCall return { text, isError } where it returned the text. This branch's test of a link not fetched for a script on demand typed the call as returning a string, so the merge of main 00eb87a8 left it matching a regex against the object (J20564's one failure). It now reads `.text` and checks `isError` is false. Co-Authored-By: Claude Opus 5.5 --- test/tool-result-blocks.test.ts | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 828b4a696..84a8e2603 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -20,6 +20,7 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { JsStore } from '@animalabs/chronicle'; import { AgentFramework } from '../src/index.js'; +import type { ScriptToolResult } from '../src/index.js'; import type { Module, ModuleContext, ToolCall, ToolResult, ToolDefinition } from '../src/types/index.js'; import { AUDIO_NO_DATA_TEXT, @@ -472,7 +473,7 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { const fw = Object.create(AgentFramework.prototype) as { dispatchScriptToolCall: (...args: unknown[]) => Promise; getReferenceFetcher(): unknown; - handleScriptToolCall(agentName: string, toolName: string, args: Record): Promise; + handleScriptToolCall(agentName: string, toolName: string, args: Record): Promise; }; const reference = { type: 'resource', uri: `https://zz-files.example/wanted.pdf?sig=${SECRET}`, name: 'wanted.pdf', @@ -493,12 +494,14 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { }); const error = console.error; console.error = () => {}; - let text: string; + let result: ScriptToolResult; try { - text = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); } finally { console.error = error; } + assert.equal(result.isError, false); + const { text } = result; assert.deepEqual(fetched, [reference.uri]); assert.deepEqual(claims, [['wanted.pdf', reference.digest, reference.expiresAt]]); // The script reads the text history stores for a protocol result's From 2b74db943e0ceb0adaa961b83daf39b95f81323e Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 19:29:34 -0400 Subject: [PATCH 10/40] fix(references): a script's reference object says what its refId meets; one clock reading per object and stub Ada-1017's call (room-203 #148838), on Desmond-2579's read (#148827): a script's reference object always carries a refId, which a model's line for a link doesn't, so its words say what a fetch_reference with that refId meets rather than mirroring the line. An unsaved reference that counts as expired carries `expired` as #262 has it, and one the fetcher would refuse carries `unfetchable`, the words a stub gives, a link whose line shows its uri included: the host can't fetch such a link, so its refId can only be refused. `uri` stays the line's one decision (shownLinkUri). This reverts ec8809b98's gating of `expired` on the line being a stub (af-f53-sol I30), which went past Ada's "`expired` stays as #262 has it" (#148223). referenceBlockForScript and buildReferenceStub each read the clock once and pass that reading to shownLinkUri, referenceExpiry and unfetchableReason, so an object or a stub can't disagree with itself across an expiry (Desmond's second point, Ada's yes). #262's unreleased mcp-standard-tool-results fragment says both words again, and that a link whose line shows its uri carries them. Tests: the five links' objects carry each refusal's exact words, a link whose line shows its uri included; an expired link whose line shows its uri carries `expired` and `unfetchable`; a clock that moves past an expiry after its first reading leaves a reference's and a link's object and stub as of that reading. Co-Authored-By: Claude Opus 5.5 --- .../mcp-standard-tool-results.changed.md | 11 +-- src/mcpl/references.ts | 33 +++++---- test/tool-result-blocks.test.ts | 72 +++++++++++++++---- 3 files changed, 84 insertions(+), 32 deletions(-) diff --git a/changelog.d/mcp-standard-tool-results.changed.md b/changelog.d/mcp-standard-tool-results.changed.md index 9a6139019..c648c3159 100644 --- a/changelog.d/mcp-standard-tool-results.changed.md +++ b/changelog.d/mcp-standard-tool-results.changed.md @@ -43,11 +43,12 @@ the script (`null` when it couldn't; `fetch_reference` takes the id as its `ref_id` argument). It arrives without its URI, except a link (MCP's `resource_link`) that carries `uri` where the model's line for it shows - the uri. An unsaved reference that counts as expired isn't fetched. - Where the model's line for a reference is a stub, the object also - carries what the stub says in place of advising `fetch_reference`: - `expired`, the reason, for one that counts as expired, and - `unfetchable`, the stub's words, for one the fetcher would refuse. + the uri. An unsaved reference that counts as expired also carries + `expired`, the reason the model's stub gives, and isn't fetched. One the + fetcher would refuse carries `unfetchable`, the words a stub gives in + place of advising `fetch_reference`. Both say what a `fetch_reference` + with the object's `refId` meets, so a link whose line shows its uri, + which the host can't fetch, carries them too. A block the classifier rejects arrives as the notice the model reads in its place, and inline data that also claims a bulk disposition as the withheld notice. #181, by @Meganeuridae, first diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 64ba43402..faebf4b24 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -725,6 +725,9 @@ export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, now */ export function buildReferenceStub(t: ReferenceTestimony, provenance?: string, serverId?: string, options?: StubOptions): string { const record = referenceRegistry.register(t, serverId); + // One clock reading for the whole stub, so whether it shows a link's uri + // and the reason it gives can't disagree across an expiry. + const nowMs = Date.now(); const parts: string[] = []; // A label is shown only when something is left of it once sanitized. const name = t.name ? sanitizeLabel(t.name, STUB_FIELD_CHARS) : ''; @@ -744,10 +747,10 @@ export function buildReferenceStub(t: ReferenceTestimony, provenance?: string, s // A reference the fetcher would refuse says why, rather than inviting a // fetch_reference call. if (options?.unfetchedLinkUri) { - const uri = shownLinkUri(t, record); + const uri = shownLinkUri(t, record, nowMs); if (uri !== null) return unfetchedLinkText(t, uri); } - parts.push(unfetchableReason(record) ?? 'fetch with fetch_reference'); + parts.push(unfetchableReason(record, nowMs) ?? 'fetch with fetch_reference'); } return `[${record.refId}] ${parts.join(' — ') || 'referenced content'}`; } @@ -803,10 +806,12 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI * path its payload was fetched to, or null when it wasn't (`fetch_reference` * takes the id as `ref_id`). Its uri is left out, except that a link (MCP's * resource_link) carries `uri` where the model's line for it shows the uri - * (shownLinkUri). Where the model's line is a stub, an unsaved reference - * that counts as expired also carries `expired`, and where the stub gives a - * reason the fetcher would refuse, in place of advising fetch_reference, the - * object carries the same words as `unfetchable` (unfetchableReason). The reference is registered bound to `serverId`, the + * (shownLinkUri). An unsaved reference that counts as expired also carries + * `expired`, and one the fetcher would refuse carries `unfetchable`, the + * words a stub gives in place of advising fetch_reference + * (unfetchableReason). Both say what a fetch_reference with the object's + * `refId` meets, so a link whose line shows its uri, which the host can't + * fetch, carries them too: its line has no id to fetch with. The reference is registered bound to `serverId`, the * server whose tool returned it, as the model's lanes bind a tool result's * references, so the fetch its id offers can be made. A block the classifier * rejects becomes the notice the model's lanes show, and inline data that @@ -823,12 +828,16 @@ export function referenceBlockForScript(block: unknown, serverId: string): unkno case 'reference': { const t = c.testimony; const record = referenceRegistry.register(t, serverId); - const uri = shownLinkUri(t, record); - // What the stub says, where the model's line is a stub: a link line - // that shows its uri says neither. As the stub judges it, a saved copy - // is still served. - const expiry = record.fetchedPath || uri !== null ? null : referenceExpiry(record.testimony); - const unfetchable = uri === null ? unfetchableReason(record) : null; + // One clock reading for the whole object, so its words can't disagree + // with each other across an expiry. + const nowMs = Date.now(); + const uri = shownLinkUri(t, record, nowMs); + // What a fetch_reference with this object's refId meets, in the words + // a stub gives, whether or not the model's line for it is a stub: the + // line has no id to fetch with, and the object does. As the stub + // judges it, a saved copy is still served. + const expiry = record.fetchedPath ? null : referenceExpiry(record.testimony, nowMs); + const unfetchable = unfetchableReason(record, nowMs); return { type: (block as { type: string }).type, refId: record.refId, diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 84a8e2603..aa63a979d 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -25,6 +25,7 @@ import type { Module, ModuleContext, ToolCall, ToolResult, ToolDefinition } from import { AUDIO_NO_DATA_TEXT, AUDIO_NO_MEDIA_TYPE_TEXT, + EXPIRY_REASON, IMAGE_NO_DATA_TEXT, IMAGE_NO_MEDIA_TYPE_TEXT, INLINE_WITHHELD_TEXT, @@ -305,22 +306,29 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { servingFiles(); const withheldUri = 'https://zz-elsewhere.example/withheld-for-script.csv'; referenceRegistry.register(testimonyOf({ type: 'resource', uri: withheldUri, name: 'withheld.csv', disposition: 'never' })); - for (const [what, link, shows] of [ - ["off its server's origin, with no credential", { type: 'resource_link', uri: 'https://zz-elsewhere.example/script-plain.csv', name: 'a.csv' }, true], - ['with a scheme the fetcher refuses', { type: 'resource_link', uri: 'file:///zz/script-plain.csv', name: 'a.csv' }, true], - ['that may carry a credential', { type: 'resource_link', uri: `https://zz-elsewhere.example/script-signed.csv?sig=${SECRET}`, name: 'a.csv' }, false], - ['withheld by a reference with disposition never', { type: 'resource_link', uri: withheldUri, name: 'a.csv' }, false], - ['the host can fetch', { type: 'resource_link', uri: 'https://zz-files.example/script-fetchable.csv', name: 'a.csv' }, false], + const thirdParty = "third-party origin: it isn't on its server's own address, so this host doesn't fetch it"; + for (const [what, link, shows, refused] of [ + ["off its server's origin, with no credential", { type: 'resource_link', uri: 'https://zz-elsewhere.example/script-plain.csv', name: 'a.csv' }, true, thirdParty], + ['with a scheme the fetcher refuses', { type: 'resource_link', uri: 'file:///zz/script-plain.csv', name: 'a.csv' }, true, 'scheme not allowed: this host fetches only http and https uris'], + ['that may carry a credential', { type: 'resource_link', uri: `https://zz-elsewhere.example/script-signed.csv?sig=${SECRET}`, name: 'a.csv' }, false, thirdParty], + ['withheld by a reference with disposition never', { type: 'resource_link', uri: withheldUri, name: 'a.csv' }, false, thirdParty], + ['the host can fetch', { type: 'resource_link', uri: 'https://zz-files.example/script-fetchable.csv', name: 'a.csv' }, false, undefined], ] as const) { // Delivered by the zz-files server, as a script's MCPL call registers it. const object = referenceBlockForScript(link, 'zz-files') as { uri?: string; refId: string; unfetchable?: string }; const line = toolResultDataToHistoryString([TEXT, link], undefined, { dataIsContent: true }).split('\n')[1]!; assert.equal(line.startsWith('[resource link '), shows, `${what}: ${line}`); assert.equal(object.uri, shows ? link.uri : undefined, what); - // Where the stub gives a reason in place of advising fetch_reference, - // the object carries the same words. - const ending = line.slice(line.lastIndexOf(' — ') + 3); - assert.equal(object.unfetchable, shows || ending === 'fetch with fetch_reference' ? undefined : ending, what); + // The object says what a fetch_reference with its refId meets: where + // the line is a stub, the reason it gives in place of advising + // fetch_reference, in the same words, and where the line shows the + // uri, which it does only for a link the host can't fetch, the reason + // that fetch is refused, which the line, having no id, doesn't give. + if (!shows) { + const ending = line.slice(line.lastIndexOf(' — ') + 3); + assert.equal(ending, refused ?? 'fetch with fetch_reference', what); + } + assert.equal(object.unfetchable, refused, what); if (!shows) assert.ok(!line.includes(link.uri) && !JSON.stringify(object).includes(link.uri), what); assert.match(object.refId, /^ref_/, what); } @@ -977,17 +985,51 @@ describe('one answer for every block of a result', () => { assert.ok(marker!.text.length < 250, String(marker!.text.length)); }); - it("gives a script's object no expiry for a link whose line shows its uri, as the line gives none", () => { - const uri = 'https://zz-elsewhere.example/expired-link.csv'; + it("gives a script's object for a link whose line shows its uri what its refId meets, which the line, having no id, doesn't say", () => { + // On its server's own origin, so the expiry is the one refusal. + servingFiles(); + const uri = 'https://zz-files.example/expired-link.csv'; referenceRegistry.register(testimonyOf({ type: 'resource', uri, name: 'expired.csv', expiresAt: '2001-01-01T00:00:00Z' })); const link = { type: 'resource_link', uri, name: 'expired.csv' }; - const object = referenceBlockForScript(link, 'zz-files') as { uri?: string; expired?: string; unfetchable?: string }; + const object = referenceBlockForScript(link, 'zz-files') as { uri?: string; refId: string; expired?: string; unfetchable?: string }; assert.equal(object.uri, uri); - assert.equal(object.expired, undefined); - assert.equal(object.unfetchable, undefined); + assert.match(object.refId, /^ref_/); + assert.equal(object.expired, EXPIRY_REASON.passed); + assert.equal(object.unfetchable, `the reference is ${EXPIRY_REASON.passed}`); assert.ok(toolResultDataToHistoryString([TEXT, link]).endsWith(`[resource link "expired.csv": ${uri}; not fetched]`)); }); + it("reads the clock once for a script's object and once for a stub, so neither disagrees with itself across an expiry", () => { + servingFiles(); + const expiresAt = '2030-01-01T00:00:00Z'; + const at = Date.parse(expiresAt); + for (const [what, block] of [ + ['a reference', { type: 'resource', uri: 'https://zz-files.example/clock.csv', name: 'clock.csv', expiresAt }], + ['a link to a reference', { type: 'resource_link', uri: 'https://zz-files.example/clock-link.csv', name: 'clock.csv' }], + ] as const) { + // Its record holds the expiry, as a reference delivered for the link's + // uri leaves it, before the clock moves. + referenceRegistry.register(testimonyOf({ ...block, type: 'resource', expiresAt }), 'zz-files'); + const now = Date.now; + // A clock that reads just before the expiry the first time it's read, + // and past it every time after. + let reads = 0; + Date.now = () => (reads++ === 0 ? at - 1000 : at + 1000); + let object: { uri?: string; expired?: string; unfetchable?: string }; + let stub: string; + try { + object = referenceBlockForScript(block, 'zz-files') as typeof object; + reads = 0; + stub = buildReferenceStub(testimonyOf(block), 'from tool result', undefined, { unfetchedLinkUri: true }); + } finally { + Date.now = now; + } + // Each says, as of its one reading, that a fetch would dial. + assert.deepEqual([object.uri, object.expired, object.unfetchable], [undefined, undefined, undefined], what); + assert.ok(stub.endsWith('— fetch with fetch_reference'), `${what}: ${stub}`); + } + }); + it("asks the resolver once for each server a fetch's check reads, so its address and its credential are one answer", () => { const record = referenceRegistry.register(testimonyOf({ type: 'resource', uri: 'https://zz-refs.example/once.bin', name: 'once.bin' }), 'zz-once'); let asked = 0; From e30b1c86f8c1f7eb0804adecaa974982c7957509 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 19:31:40 -0400 Subject: [PATCH 11/40] docs(changelog): what agents stop seeing includes a link whose uri a reference marked never MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The changed fragment, which says what agents stop seeing, named only a link whose uri may carry a credential. Since this change's second round, a link to a uri whose reference, as the host last recorded it, said `disposition: never` shows the same stub with no uri too (shownLinkUri), where the result's JSON showed it, as RFC-005 §5 asks. The fragment now says so. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 267e110a3..396ea2d89 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1 +1 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri whose reference, as the host last recorded it, said `disposition: never`, in that result or an earlier one, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. From 9a4068c8354ad3f7f30314996cba79eb1d302789 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 19:47:05 -0400 Subject: [PATCH 12/40] test(tool-results): the audio marker test reads the history text, so the file loads against main for red-first The audio marker test from the whole-PR review's fixes (I19) imported toolResultBlocks, which main's tool-result-history doesn't export. Run against main's src for red-first, the file then failed to load whole and none of its tests ran (J20606's red run: 48 not found). The test now reads the marker through toolResultDataToHistoryString, as the file's other tests do, and checks it is the audio marker. Co-Authored-By: Claude Opus 5.5 --- test/tool-result-blocks.test.ts | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index aa63a979d..c99188907 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -46,7 +46,7 @@ import { CoalesceError, validateCoalescedContent } from '../src/mcpl/push-coales import { ReferenceFetcher } from '../src/mcpl/reference-fetcher.js'; import { checkedToolResult } from '../src/module-registry.js'; import { parseImagePlaceholders } from '../src/tool-image-ledger.js'; -import { nonErrorResultText, toolResultBlocks, toolResultDataToHistoryString } from '../src/tool-result-history.js'; +import { nonErrorResultText, toolResultDataToHistoryString } from '../src/tool-result-history.js'; import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; import { fixture } from './helpers/coalescing-fixture.js'; @@ -980,9 +980,11 @@ describe('one answer for every block of a result', () => { it("bounds an audio marker's type and strips its controls", () => { const type = `audio/\u202Ewav\u0007${'x'.repeat(300)}`; - const [marker] = toolResultBlocks([{ type: 'audio', data: BASE64, mimeType: type }], true)! as Array<{ text: string }>; - assert.ok(!marker!.text.includes('\u202E') && !marker!.text.includes('\u0007'), marker!.text); - assert.ok(marker!.text.length < 250, String(marker!.text.length)); + // The array's one block, so the text is its marker alone. + const marker = toolResultDataToHistoryString([{ type: 'audio', data: BASE64, mimeType: type }], undefined, { dataIsContent: true }); + assert.match(marker, /^\[Audio not shown: /); + assert.ok(!marker.includes('\u202E') && !marker.includes('\u0007'), marker); + assert.ok(marker.length < 250, String(marker.length)); }); it("gives a script's object for a link whose line shows its uri what its refId meets, which the line, having no id, doesn't say", () => { From cddaa5803ef638ada8dab4bb6c2922e049f78f42 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 21:27:08 -0400 Subject: [PATCH 13/40] fix(references): a result's claims are read as one, strictest wins; one classification without a stored copy From the round review of the whole-PR review's fixes (af-f53-sol-r1). - A tool result's references are read as one (ResultClaims, Ada-1017's call): a uri any of them marks `disposition: never` is withheld from every block that names it, and one any of them gives an expiry that counts is expired for each, whatever the order of the blocks and whatever the bounded registry evicted while the result was read. Each block's decision reads the result's claims with its record's own (claimedNever, claimedExpiry): a link's uri (shownLinkUri), a stub's reason (dialCheck through unfetchableReason, StubOptions.claims), a script's object (referenceBlockForScript), and the module demand fetch, which now passes over a claimed expiry as autoFetchReferences does with expiredHere. Across results the registry keeps the latest testimony. A fetch reads the record alone. - A content array's claims come from its elements (contentClaims), for the model's lanes and for an MCPL peer's blocks in a script's result. - At depth they're read by the walk serialization itself takes (walkedInHistory under JSON.stringify, claimsInHistory): through toJSON, into inline data's other fields, and no further than a value serialization can't write. The walk registers nothing. - This replaces registering every reference before showing any, which each block's own registration as it was shown undid. - Without a stored copy, the live copy classifies once: the blocks nonErrorResultText made its text from go natively where goesNative holds (onBlocks), as on the stored path. tryNativeToolResultContent goes. - Comments: the cut's doc says when goesNative says no; the live copy's builder is named as toMembraneToolResult's nativeContent; the fallback and shownInHistory say a link's uri shows only in its line; dialCheck's doc names another running framework's entry for a stopped one's id. - The changed fragment says a never from any reference in the same result, or the latest from an earlier one, withholds a link's uri. Tests: a never withholding a link's uri in five orders, at depth, in a script's object, inside an image's _meta, and past 2,001 references; an expiry read for every block whatever the order; claims read through toJSON beside a throwing getter; nothing registered past a cycle; the module demand fetch passing over a claimed expiry; a fetch's token and the eager join's limit each from one resolver answer; the live copy handed over through buildStoredToolResultContent, and the cap's boundary there; an MCPL send whose result holds a null element before its channel's id. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- src/framework.ts | 77 ++++---- src/mcpl/references.ts | 108 ++++++++--- src/tool-image-ledger.ts | 7 +- src/tool-result-history.ts | 97 ++++++---- test/script-structured-references.test.ts | 19 ++ test/tool-result-blocks.test.ts | 209 +++++++++++++++++++++- 7 files changed, 409 insertions(+), 110 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 396ea2d89..7b933bfbb 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1 +1 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri whose reference, as the host last recorded it, said `disposition: never`, in that result or an earlier one, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. diff --git a/src/framework.ts b/src/framework.ts index 58fe7a8e1..a72fb5aa3 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,6 +1,6 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { dirname, join } from 'node:path'; -import { REFERENCE_LIMITS, classifyBlock, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ServerAddress } from './mcpl/references.js'; +import { REFERENCE_LIMITS, claimedExpiry, classifyBlock, contentClaims, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ServerAddress } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, @@ -115,7 +115,6 @@ import type { ToolResultGuard, WithheldSpill } from './tool-result-guard.js'; import { goesNative, nonErrorResultText, - toolResultBlocks, toolResultDataToHistoryString, truncateForHistory, DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS, @@ -18909,9 +18908,9 @@ export class AgentFramework { * * The live copy is the stored text itself unless the result is a content * array carrying an inline image whose links and markers fit the cap - * together (goesNative), which tryNativeToolResultContent sends as native - * blocks. It takes the blocks the serializer takes (toolResultBlocks), and - * shows each image, stub and marker whole and each text block up to where + * together (goesNative). Then `live` holds the blocks this text was made + * from, which toMembraneToolResult sends as native blocks (nativeContent), + * showing each image, stub and marker whole and each text block up to where * it cuts that block at the cap (`maxChars`, toolResultCutAt), so those are * the ranges of the text it showed. */ @@ -19219,6 +19218,10 @@ export class AgentFramework { // the ones it wants with fetch_reference. if (Array.isArray(result.data) && this.resolveMcplTool(toolName) === null) { const fetcher = this.getReferenceFetcher(); + // The array's claims, read as one: a uri any of its references gives + // an expiry that counts isn't fetched, as autoFetchReferences passes + // over one that any block of an MCPL result marks expired. + const claims = contentClaims(result.data); for (const block of result.data) { const c = classifyBlock(block); if (c.kind !== 'reference' || c.testimony.fromLink) continue; @@ -19226,6 +19229,7 @@ export class AgentFramework { // replace a link's, or a stale reference's, before the fetch reads // them, as on any other path, while the record isn't fetched yet. const record = referenceRegistry.register(c.testimony); + if (!record.fetchedPath && claimedExpiry(record, claims, Date.now()) !== null) continue; const outcome = await fetcher.fetch(record); if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); } @@ -19410,42 +19414,32 @@ export class AgentFramework { isError: false, }; } - const blocks = this.tryNativeToolResultContent(afResult.data, Boolean(afResult.dataIsContent), maxChars); - if (blocks) { - return { toolUseId: callId, content: blocks, isError: false }; + // Without the history path's copy, one classification makes both the + // decision and the copy, as on that path: the text is made as that copy + // is (nonErrorResultText), so a reference shows at any depth as its stub, + // or a link as its line (RFC-005), a result with no data reads its error + // text, or no text, and never the value undefined; and where it's a + // content array whose links and markers fit the cap beside an image + // (goesNative), the very blocks that text was made from go natively. + let spans = null as HistoryBlockSpan[] | null; + let classified = null as ToolResultBlock[] | null; + const text = nonErrorResultText(afResult, maxChars, { onBlocks: (s, b) => { spans = s; classified = b; } }); + if (spans && classified && goesNative((spans as HistoryBlockSpan[]).map(({ kind, start, end }) => ({ kind, length: end - start })), maxChars)) { + return { toolUseId: callId, content: this.nativeContent(classified, maxChars, null), isError: false }; } - // Without the history path's copy, the text is made as that copy is - // (nonErrorResultText), so a reference's uri stays out of it at any depth - // (RFC-005), a result with no data reads its error text, or no text, and - // never the value undefined. - return { toolUseId: callId, content: nonErrorResultText(afResult, maxChars), isError: false }; + return { toolUseId: callId, content: text, isError: false }; } /** - * If `data` is a content array (toolResultBlocks) carrying at least one - * image, and its links and markers fit the cap together (goesNative), - * its blocks as Membrane's native ToolResultContentBlock[]: each image as - * itself, and every other block as the text the stored copy shows for it. - * Returns null otherwise, and the live copy is the stored text. - * `maxChars`, when provided, caps each accompanying text block so an image - * inlined alongside an enormous text payload can't blow the context. A - * block is cut as the stored copy is (toolResultCutAt), never inside a - * slot, so the stored copy can keep exactly the slots this showed (#403). + * A content array's blocks (toolResultBlocks), carrying at least one image + * whose links and markers fit the cap together (goesNative), as Membrane's + * native ToolResultContentBlock[]: each image as itself, each text block + * cut at the cap, and every other block whole, as the text the stored copy + * shows for it. `maxChars`, when provided, caps each text block so an image + * inlined beside an enormous text payload can't blow the context. A block + * is cut as the stored copy is (toolResultCutAt), never inside a slot, so + * the stored copy can keep exactly the slots this showed (#403). */ - private tryNativeToolResultContent( - data: unknown, - dataIsContent: boolean, - maxChars?: number, - ): ToolResultContentBlock[] | null { - const blocks = toolResultBlocks(data, dataIsContent); - if (!blocks || !goesNative(blocks.map((block) => ({ kind: block.kind, length: block.kind === 'image' ? 0 : block.text.length })), maxChars)) { - return null; - } - return this.nativeContent(blocks, maxChars, null); - } - - /** A result's blocks as Membrane's native content: each image as itself, - * each text block cut at the cap, every other block whole. */ private nativeContent( blocks: ToolResultBlock[], maxChars: number | undefined, @@ -24565,7 +24559,12 @@ export class AgentFramework { return this.mcpToolResult(result, call.id, mcpl, serverId, call.name); } const direct = AgentFramework.directToolResult(result, serverId, call.name); - if (mcpl) return Array.isArray(direct.data) ? { ...direct, data: direct.data.map((block: unknown) => referenceBlockForScript(block, serverId)) } : direct; + if (mcpl) { + if (!Array.isArray(direct.data)) return direct; + // Each block reads the array's claims as one, as the model's line does. + const claims = contentClaims(direct.data); + return { ...direct, data: direct.data.map((block: unknown) => referenceBlockForScript(block, serverId, claims)) }; + } // A standard peer's content that isn't a list of blocks is a malformed // result on the model's reading (normalizeStandardToolResult), so the // script reads it that way too, its structured content kept. @@ -25235,8 +25234,8 @@ export class AgentFramework { * Handle the synthesized `read_image` tool: read image bytes from a * workspace mount and return them as a native MCP-shaped content array * (`[{type:'text'…},{type:'image', data, mimeType}]`). The live-turn - * converter (tryNativeToolResultContent → membrane tool_result image - * blocks) delivers the actual image to the model; history storage keeps + * converter (toMembraneToolResult's nativeContent → membrane tool_result + * image blocks) delivers the actual image to the model; history storage keeps * the standard compact `[image: …]` placeholder. */ /** RFC-005 on-demand dereference: [ref id] → workspace file, via the diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index faebf4b24..5390e855f 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -473,8 +473,10 @@ export function dialServer( * goes with the address it checks. In a process with one running framework, * or with running frameworks that agree on each id they share, the two * sources hold the same addresses, so a stub promises what the fetch does. A - * stopped framework's addresses are out of the registry, so its stubs say its - * servers are unknown while its configs still resolve them. Where two + * stopped framework's addresses are out of the registry, so its stubs read + * another running framework's entry for the same id where there is one, and + * otherwise say the server is unknown, while its configs still resolve it. + * Where two * frameworks configure one id differently, only a stub's words can be wrong, * never where a token goes. A fetch's own limits, its budget and ceiling, * come after it. @@ -483,10 +485,14 @@ export function dialCheck( record: ReferenceRecord, addressOf: (serverId: string) => ServerAddress | undefined, nowMs = Date.now(), + /** For a stub or a script's object, the claims of the result it's shown + * in, read with the record's own (claimedExpiry). A fetch reads the + * record alone. */ + claims?: ResultClaims, ): { serverId: string } | { refusal: Refusal } { const server = dialServer(record, addressOf); if ('refusal' in server) return server; - const expiry = referenceExpiry(record.testimony, nowMs); + const expiry = claimedExpiry(record, claims, nowMs); return expiry ? { refusal: { error: `the reference is ${EXPIRY_REASON[expiry]}`, remedy: FRESH_REFERENCE } } : server; } @@ -551,7 +557,7 @@ const MAX_RECORDS = 2000; export class ReferenceRegistry { private records = new Map(); /** uri → refId. The uri ALONE is the identity key: a tool result's stubs - * (tool-result-history.ts's, and framework.ts's tryNativeToolResultContent) + * (tool-result-history.ts's, which both copies of a tool result take) * and a script's demand fetch register with no server context, * while the registrations that know the server carry its id: a tool * result's, before it's shown, whether the agent's dispatch or an @@ -689,6 +695,62 @@ export interface StubOptions { * other lane showed a link's uri, so none shows one now. */ unfetchedLinkUri?: boolean; + /** The claims of the result this block is shown in (resultClaims), read + * with the record's own for what the stub says. */ + claims?: ResultClaims; +} + +/** + * What one tool result's references claim of each uri they name: uri → the + * testimony each of those references gave it. A decision a block of that + * result makes reads them all with its record's own, and the strictest claim + * wins: a uri any of them marks `disposition: never` is withheld, and one + * any of them gives an expiry that counts is expired (claimedExpiry), so a + * result means the same whatever the order of its blocks, and whatever the + * registry evicted while it was read (Ada-1017's call). Across results the + * registry keeps the latest testimony (register). + */ +export type ResultClaims = ReadonlyMap; + +/** A result's claims (ResultClaims), from the testimonies of its references. */ +export function resultClaims(testimonies: Iterable): ResultClaims { + const claims = new Map(); + for (const t of testimonies) { + const list = claims.get(t.uri); + if (list) list.push(t); + else claims.set(t.uri, [t]); + } + return claims; +} + +/** The claims of a content array's references, element by element. */ +export function contentClaims(blocks: readonly unknown[]): ResultClaims { + const testimonies: ReferenceTestimony[] = []; + for (const block of blocks) { + const c = classifyBlock(block); + if (c.kind === 'reference') testimonies.push(c.testimony); + } + return resultClaims(testimonies); +} + +/** Every testimony a decision about a record reads: the record's own, and + * each one the result's references gave its uri. */ +function claimsOf(record: ReferenceRecord, claims?: ResultClaims): ReferenceTestimony[] { + return [record.testimony, ...(claims?.get(record.testimony.uri) ?? [])]; +} + +/** Whether a record's uri is withheld: its own testimony, or any of the + * result's references, marks it `disposition: never`. */ +export function claimedNever(record: ReferenceRecord, claims?: ResultClaims): boolean { + return claimsOf(record, claims).some((t) => t.disposition === 'never'); +} + +/** The expiry a record counts, as of `nowMs`: the first of EXPIRY_REASON's + * cases that its own testimony, or any of the result's references, gives, + * so the order of the blocks doesn't choose the words. Null when none does. */ +export function claimedExpiry(record: ReferenceRecord, claims: ResultClaims | undefined, nowMs: number): keyof typeof EXPIRY_REASON | null { + const found = new Set(claimsOf(record, claims).map((t) => referenceExpiry(t, nowMs))); + return (Object.keys(EXPIRY_REASON) as Array).find((kind) => found.has(kind)) ?? null; } /** @@ -703,12 +765,12 @@ export interface StubOptions { * The credential check reads the uri as it would be shown, with what * sanitizeLabel strips gone, so a uri split by a control character is read * as it would read: `ht\u0000tps://user:pass@…` parses as a path, and shows - * as userinfo. A reference's disposition:"never" for the uri keeps it out - * whatever the link says. + * as userinfo. A reference's disposition:"never" for the uri, the record's + * or any in the same result (claims), keeps it out whatever the link says. */ -export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, nowMs = Date.now()): string | null { - if (!t.fromLink || record.testimony.disposition === 'never') return null; - if (unfetchableReason(record, nowMs) === null) return null; +export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, nowMs = Date.now(), claims?: ResultClaims): string | null { + if (!t.fromLink || claimedNever(record, claims)) return null; + if (unfetchableReason(record, nowMs, claims) === null) return null; const uri = sanitizeLabel(t.uri, REFERENCE_LIMITS.uri); return uriMayCarryCredential(uri) ? null : uri; } @@ -747,10 +809,10 @@ export function buildReferenceStub(t: ReferenceTestimony, provenance?: string, s // A reference the fetcher would refuse says why, rather than inviting a // fetch_reference call. if (options?.unfetchedLinkUri) { - const uri = shownLinkUri(t, record, nowMs); + const uri = shownLinkUri(t, record, nowMs, options.claims); if (uri !== null) return unfetchedLinkText(t, uri); } - parts.push(unfetchableReason(record, nowMs) ?? 'fetch with fetch_reference'); + parts.push(unfetchableReason(record, nowMs, options?.claims) ?? 'fetch with fetch_reference'); } return `[${record.refId}] ${parts.join(' — ') || 'referenced content'}`; } @@ -774,9 +836,9 @@ function unfetchedLinkText(t: ReferenceTestimony, uri: string): string { * server's spent budget or a claimed size over the ceiling it's given, the * fetcher gives in its own words, and save_recent_image says them too. */ -export function unfetchableReason(record: ReferenceRecord, nowMs = Date.now()): string | null { +export function unfetchableReason(record: ReferenceRecord, nowMs = Date.now(), claims?: ResultClaims): string | null { if (record.fetchedPath) return null; - const dial = dialCheck(record, (serverId) => referenceRegistry.serverAddress(serverId), nowMs); + const dial = dialCheck(record, (serverId) => referenceRegistry.serverAddress(serverId), nowMs, claims); return 'refusal' in dial ? dial.refusal.inStub ?? dial.refusal.error : null; } @@ -811,14 +873,17 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI * words a stub gives in place of advising fetch_reference * (unfetchableReason). Both say what a fetch_reference with the object's * `refId` meets, so a link whose line shows its uri, which the host can't - * fetch, carries them too: its line has no id to fetch with. The reference is registered bound to `serverId`, the - * server whose tool returned it, as the model's lanes bind a tool result's - * references, so the fetch its id offers can be made. A block the classifier + * fetch, carries them too: its line has no id to fetch with. `claims` are + * those of the content array the block came in (contentClaims), so its + * other references count as they do for the model's line (ResultClaims). + * The reference is registered bound to `serverId`, the server whose tool + * returned it, as the model's lanes bind a tool result's references, so the + * fetch its id offers can be made. A block the classifier * rejects becomes the notice the model's lanes show, and inline data that * also claims a bulk disposition the withheld notice, each as a text block. * A text block, or inline data that claims nothing, is returned as sent. */ -export function referenceBlockForScript(block: unknown, serverId: string): unknown { +export function referenceBlockForScript(block: unknown, serverId: string, claims?: ResultClaims): unknown { const c = classifyBlock(block); switch (c.kind) { case 'text': return block; @@ -831,13 +896,14 @@ export function referenceBlockForScript(block: unknown, serverId: string): unkno // One clock reading for the whole object, so its words can't disagree // with each other across an expiry. const nowMs = Date.now(); - const uri = shownLinkUri(t, record, nowMs); + const uri = shownLinkUri(t, record, nowMs, claims); // What a fetch_reference with this object's refId meets, in the words // a stub gives, whether or not the model's line for it is a stub: the // line has no id to fetch with, and the object does. As the stub - // judges it, a saved copy is still served. - const expiry = record.fetchedPath ? null : referenceExpiry(record.testimony, nowMs); - const unfetchable = unfetchableReason(record, nowMs); + // judges it, a saved copy is still served, and the result's own claims + // count with the record's. + const expiry = record.fetchedPath ? null : claimedExpiry(record, claims, nowMs); + const unfetchable = unfetchableReason(record, nowMs, claims); return { type: (block as { type: string }).type, refId: record.refId, diff --git a/src/tool-image-ledger.ts b/src/tool-image-ledger.ts index 026540977..8ab117851 100644 --- a/src/tool-image-ledger.ts +++ b/src/tool-image-ledger.ts @@ -326,8 +326,8 @@ export interface StoredCut { * Where a tool result's text longer than `cap` is cut: by its stored copy * (cutStoredToolResult), by history's own truncation (truncateForHistory), * and by its live copy, for each text block beside an image - * (tryNativeToolResultContent, framework.ts), so the stored copy and the live - * copy cut alike. The text's length when it isn't longer than `cap`. + * (toMembraneToolResult's nativeContent, framework.ts), so the stored copy + * and the live copy cut alike. The text's length when it isn't longer than `cap`. * * The cut is made at the cap, or before it when the cap would split a * surrogate pair (safeSlice's rule), and that is where the moves back are @@ -364,7 +364,8 @@ export function toolResultCutAt(text: string, cap: number): number { * * `shown` is what the live copy showed beside the stored text: null when the * live copy was the text itself, as for every result that carried no inline - * image, or whose links and markers passed the cap together (goesNative). + * image, or whose links and markers came to more than the cap together, so + * goesNative said no. * Then what the cut keeps is exactly what the resident saw, and nothing past * it is kept (#403). A result whose live copy went to the model as native * blocks (framework.ts) sent every image whatever the text's length, so its stored text is the only diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 184c5060e..d18aa9894 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -29,9 +29,12 @@ import { INLINE_WITHHELD_TEXT, buildReferenceStub, classifyBlock, - referenceRegistry, + contentClaims, referenceStubOrNull, referenceTypeLabel, + resultClaims, + type ReferenceTestimony, + type ResultClaims, } from './mcpl/references.js'; import { formatToolImagePlaceholder, toolResultCutAt } from './tool-image-ledger.js'; @@ -102,41 +105,18 @@ export function toolResultBlocks(data: unknown, dataIsContent?: boolean): ToolRe if (dataIsContent !== true && !data.every((element) => isPlainRecord(element) && MCP_BLOCK_TYPES.has(element.type as string))) { return null; } - // Every reference first, so what one says of a uri (a `disposition: never`, - // an expiry) holds for each block that names it, whatever their order. - for (const raw of data) registerReference(raw); - return data.map(toolResultBlock); -} - -/** Register a block that is a reference, before any block is shown. */ -function registerReference(raw: unknown): void { - const c = classifyBlock(raw); - if (c.kind === 'reference') referenceRegistry.register(c.testimony); -} - -/** Register every reference a tool result's data holds, at any depth, as - * historyJson shows them, before any is shown. A value RFC-005 governs is - * shown whole, so what's inside it isn't visited. */ -function registerReferencesIn(value: unknown, seen: Set = new Set()): void { - if (!value || typeof value !== 'object' || seen.has(value)) return; - seen.add(value); - if (Array.isArray(value)) { - for (const element of value) registerReferencesIn(element, seen); - return; - } - const b = value as Record; - if (governedByRfc005(b)) { - registerReference(b); - return; - } - for (const field of Object.values(b)) registerReferencesIn(field, seen); + // The result's claims, read as one (ResultClaims): what any of its + // references says of a uri (a `disposition: never`, an expiry) holds for + // each block that names it, whatever their order. + const claims = contentClaims(data); + return data.map((raw) => toolResultBlock(raw, claims)); } /** One element of a content array, as toolResultBlocks gives it. Only an * image with data and a media type is shown as itself, as the push lane's * converter shows one, and only text and images can be: this host's tool * results carry nothing else, so inline audio is a marker here. */ -function toolResultBlock(raw: unknown): ToolResultBlock { +function toolResultBlock(raw: unknown, claims: ResultClaims): ToolResultBlock { const c = classifyBlock(raw); switch (c.kind) { case 'text': @@ -151,7 +131,7 @@ function toolResultBlock(raw: unknown): ToolResultBlock { // block, stub or line, is `link`, counted toward the cap (goesNative). return { kind: c.testimony.fromLink ? 'link' : 'reference', - text: buildReferenceStub(c.testimony, 'from tool result', undefined, { unfetchedLinkUri: true }), + text: buildReferenceStub(c.testimony, 'from tool result', undefined, { unfetchedLinkUri: true, claims }), }; case 'invalid': return { kind: 'marker', text: c.shown }; @@ -330,13 +310,25 @@ function isPlainRecord(value: unknown): value is Record { * shown as the reference lanes show it: a reference as its stub, an invalid * one (such as a uri that isn't a string, or MCP's embedded resource, whose * uri isn't at its top level) as the marker saying why it isn't shown, and - * inline data that also claims a uri or a disposition as withheld, so no uri - * is shown but a link's where its line shows one (shownLinkUri). Inline data - * that claims neither is itself, as is any other value: + * inline data that also claims a uri or a disposition as withheld. So the + * only uri shown is a link's, in its line, where the host can't fetch the + * link and its uri can't carry a credential (shownLinkUri). Inline data that + * claims neither is itself, as is any other value: * a block of another type keeps its fields, since RFC-005 governs * references. */ -function shownInHistory(value: unknown): unknown { +function shownInHistory(value: unknown, claims?: ResultClaims): unknown { + return walkedInHistory(value, (b) => referenceStubOrNull(b, 'from tool result', undefined, { unfetchedLinkUri: true, claims })); +} + +/** + * The one walk of a tool result's data that showing it and reading its + * claims both take: a value RFC-005 governs is text, from `asText`, except + * inline data that claims no uri or disposition, which is itself, as is any + * other value, so serialization descends into exactly the same values each + * time. `asText` gives a governed value's text, or null where it's itself. + */ +function walkedInHistory(value: unknown, asText: (b: Record) => string | null): unknown { if (!value || typeof value !== 'object' || Array.isArray(value)) return value; const b = value as Record; if (!governedByRfc005(b)) return value; @@ -346,7 +338,33 @@ function shownInHistory(value: unknown): unknown { // no uri: an object can carry a credential. An empty uri carries none. return hasUri(b.uri) || b.disposition !== undefined ? INLINE_WITHHELD_TEXT : value; } - return referenceStubOrNull(b, 'from tool result', undefined, { unfetchedLinkUri: true }) ?? value; + return asText(b) ?? value; +} + +/** + * What a tool result's data claims of each uri its references name + * (ResultClaims), read by the walk serialization takes (walkedInHistory), + * through toJSON, the fields of inline data and every depth it shows: so + * every reference whose stub the result shows counts, and nothing it doesn't + * show does. A value serialization can't write, such as a cycle, stops the + * walk where it would stop the result. + */ +function claimsInHistory(data: unknown): ResultClaims { + const testimonies: ReferenceTestimony[] = []; + const read = (b: Record): string | null => { + const c = classifyBlock(b); + if (c.kind === 'reference') testimonies.push(c.testimony); + // Text where referenceStubOrNull gives text, so the walk descends where + // showing descends. + return c.kind === 'text' || (c.kind === 'inline' && !c.contradiction) ? null : ''; + }; + try { + const root = walkedInHistory(data, read); + if (root === data || typeof root !== 'string') JSON.stringify(data, (_key, value: unknown) => walkedInHistory(value, read)); + } catch { + // Showing the result meets the same failure at the same place. + } + return resultClaims(testimonies); } /** @@ -355,6 +373,7 @@ function shownInHistory(value: unknown): unknown { * marker, or inline data withheld. Null for any other value. */ export function rfc005TextOf(value: unknown): string | null { + // A single value claims only what it says itself, which its stub reads. const shown = shownInHistory(value); return shown !== value && typeof shown === 'string' ? shown : null; } @@ -365,10 +384,10 @@ export function rfc005TextOf(value: unknown): string | null { * as a bare reference, is that text. */ function historyJson(data: unknown): string | undefined { - registerReferencesIn(data); - const root = shownInHistory(data); + const claims = claimsInHistory(data); + const root = shownInHistory(data, claims); if (root !== data && typeof root === 'string') return root; - return JSON.stringify(data, (_key, value: unknown) => shownInHistory(value)) as string | undefined; + return JSON.stringify(data, (_key, value: unknown) => shownInHistory(value, claims)) as string | undefined; } function formatSize(bytes: number): string { diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index d06f4228e..e94e1a31b 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -285,3 +285,22 @@ test('an error with structured content is fetched for the script, and reaches it assert.deepEqual(fetched, [uris('structured-error').report], 'the script reads the blocks, so its reference was fetched first'); }); }); + +test("an MCPL peer's link reads every reference in its result, so a never anywhere in it withholds the link's uri from a script", { timeout: 30_000 }, async () => { + await withPeers(async (h) => { + // Off the server's origin, so the host can't fetch it and its line would + // show its uri, but a reference in the same result says never, and a + // later one, registered last, doesn't. + const uri = 'https://elsewhere.example.invalid/never-in-result.csv'; + h.answer({ + content: [ + { type: 'resource', uri, name: 'kept.csv', disposition: 'never' }, + { type: 'resource', uri, name: 'kept.csv' }, + { type: 'resource_link', uri, name: 'kept.csv' }, + ], + structuredContent: {}, + }); + const out = await h.script('mcpl--files--attach'); + assert.ok(!out.includes('elsewhere.example.invalid'), out); + }); +}); diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index c99188907..0f7968fa5 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -915,17 +915,106 @@ async function answerModelCall(answer: ToolResult): Promise<{ wire: string; stor } describe('one answer for every block of a result', () => { - it("withholds a link's uri for a later reference to it with disposition never, whatever the blocks' order", () => { + it("withholds a link's uri wherever a reference in the same result marks it never, whatever the blocks' order", () => { const uri = 'https://zz-elsewhere.example/order-never.csv'; const link = { type: 'resource_link', uri, name: 'order.csv' }; const never = { type: 'resource', uri, name: 'order.csv', disposition: 'never' }; - const shown = toolResultDataToHistoryString([TEXT, link, never]); + const plain = { type: 'resource', uri, name: 'order.csv' }; + for (const order of [[link, never, plain], [never, link, plain], [never, plain, link], [plain, never, link], [plain, link, never]]) { + const what = order.map((b) => (b === link ? 'link' : b === never ? 'never' : 'plain')).join(', '); + const shown = toolResultDataToHistoryString([TEXT, ...order]); + assert.ok(!shown.includes(uri), `${what}: ${shown}`); + // At any depth in data too, and in a script's object. + const deep = toolResultDataToHistoryString({ first: order.slice(0, 1), then: { rest: order.slice(1) } }); + assert.ok(!deep.includes(uri), `${what}: ${deep}`); + // The array's claims (contentClaims), built here so the file loads + // against main's src for red-first. + const claims = new Map([[uri, order.map(testimonyOf)]]); + for (const block of order) { + const object = referenceBlockForScript(block, 'zz-files', claims) as { uri?: string }; + assert.equal(object.uri, undefined, what); + } + } + }); + + it("reads an expiry any reference in the same result gives a uri for every block that names it, whatever their order", () => { + servingFiles(); + const uri = 'https://zz-files.example/order-expired.csv'; + const link = { type: 'resource_link', uri, name: 'order.csv' }; + const expired = { type: 'resource', uri, name: 'order.csv', expiresAt: '2001-01-01T00:00:00Z' }; + const fresh = { type: 'resource', uri, name: 'order.csv', expiresAt: '2099-01-01T00:00:00Z' }; + referenceRegistry.register(testimonyOf(link), 'zz-files'); + const texts = new Set(); + for (const order of [[fresh, link, expired], [expired, link, fresh], [link, fresh, expired]]) { + const lines = toolResultDataToHistoryString(order).split('\n'); + // The link, which the expiry makes unfetchable, shows its line, and + // each reference's stub says the reference is expired. + assert.equal(lines.filter((line) => line === `[resource link "order.csv": ${uri}; not fetched]`).length, 1, lines.join('\n')); + assert.equal(lines.filter((line) => line.endsWith('— the reference is expired')).length, 2, lines.join('\n')); + texts.add([...lines].sort().join('\n').replace(/ref_[^\]]+/g, 'ref')); + // And a script's object for each reference, with the array's claims. + const claims = new Map([[uri, order.map(testimonyOf)]]); + for (const block of [fresh, expired]) { + const object = referenceBlockForScript(block, 'zz-files', claims) as { expired?: string; unfetchable?: string }; + assert.equal(object.expired, EXPIRY_REASON.passed); + assert.equal(object.unfetchable, `the reference is ${EXPIRY_REASON.passed}`); + } + } + assert.equal(texts.size, 1); + }); + + it("finds a never inside inline data's other fields, which showing it descends into, before any block is shown", () => { + const uri = 'https://zz-elsewhere.example/meta-never.csv'; + const data = { + items: [ + { type: 'resource_link', uri, name: 'meta.csv' }, + { type: 'image', data: PNG, mimeType: 'image/png', _meta: { ref: { type: 'resource', uri, name: 'meta.csv', disposition: 'never' } } }, + ], + }; + const shown = toolResultDataToHistoryString(data); assert.ok(!shown.includes(uri), shown); - assert.match(shown.split('\n')[1]!, /^\[ref_[^\]]+\] order\.csv — from tool result — /); - // At any depth in data too. - const uri2 = 'https://zz-elsewhere.example/order-never-deep.csv'; - const deep = toolResultDataToHistoryString({ first: [{ ...link, uri: uri2 }], then: { ref: { ...never, uri: uri2 } } }); - assert.ok(!deep.includes(uri2), deep); + }); + + it("keeps a never for its result when more references than the registry holds come between it and the link", () => { + const uri = 'https://zz-elsewhere.example/evicted-never.csv'; + const many = Array.from({ length: 2001 }, (_, i) => ({ type: 'resource', uri: `https://zz-elsewhere.example/many/${i}.bin`, name: `${i}.bin` })); + const shown = toolResultDataToHistoryString([ + { type: 'resource', uri, name: 'evicted.csv', disposition: 'never' }, ...many, { type: 'resource_link', uri, name: 'evicted.csv' }, + ]); + assert.ok(!shown.includes(uri), shown.slice(-400)); + }); + + it("reads claims through toJSON, as serialization does, so a getter toJSON leaves out isn't read", () => { + const value = { + toJSON: () => ({ zz: 'zz-shown' }), + get broken(): string { throw new Error('zz-getter read'); }, + }; + assert.equal(toolResultDataToHistoryString({ value }), '{"value":{"zz":"zz-shown"}}'); + }); + + it("registers nothing a result's serialization never reaches, such as what lies past a cycle", () => { + const uri = 'https://zz-elsewhere.example/past-cycle.csv'; + const loop: Record = {}; + loop.self = loop; + assert.throws(() => toolResultDataToHistoryString({ loop, later: { type: 'resource', uri, name: 'past.csv', disposition: 'never' } })); + assert.equal(referenceRegistry.findByUri(uri), undefined); + }); + + it("isn't fetched for a script on demand where any reference in the module's array gives its uri an expiry that counts", async () => { + const fw = Object.create(AgentFramework.prototype) as { + dispatchScriptToolCall: (...args: unknown[]) => Promise; + getReferenceFetcher(): unknown; + handleScriptToolCall(agentName: string, toolName: string, args: Record): Promise; + }; + const uri = 'https://zz-files.example/demand-expired.pdf'; + const fresh = { type: 'resource', uri, name: 'demand.pdf', expiresAt: '2099-01-01T00:00:00Z' }; + const expired = { type: 'resource', uri, name: 'demand.pdf', expiresAt: '2001-01-01T00:00:00Z' }; + fw.dispatchScriptToolCall = async () => ({ success: true, data: [fresh, expired], dataIsContent: true }); + const fetched: string[] = []; + fw.getReferenceFetcher = () => ({ fetch: async (record: { testimony: { uri: string } }) => { fetched.push(record.testimony.uri); return { ok: false, error: 'zz-refused' }; } }); + const result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + assert.equal(result.isError, false); + assert.deepEqual(fetched, []); }); it('shows a link with no usable uri at any depth as its marker, never its fields', () => { @@ -1044,6 +1133,112 @@ describe('one answer for every block of a result', () => { assert.equal(asked, 1); }); + it("pairs each check's origin with the same answer's token and eager limit, on a fetch and on the eager check that joins it", async (t) => { + const heard: Array = []; + let arrived!: () => void; + const arrival = new Promise((resolve) => { arrived = resolve; }); + let release!: () => void; + const held = new Promise((resolve) => { release = resolve; }); + const server = createServer((req, res) => { + heard.push(req.headers.authorization); + arrived(); + void held.then(() => { + res.writeHead(200, { 'content-type': 'application/octet-stream' }); + res.end(Buffer.from('zz-bytes')); + }); + }); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + t.after(() => new Promise((resolve) => server.close(() => resolve()))); + const host = `127.0.0.1:${(server.address() as AddressInfo).port}`; + const record = referenceRegistry.register( + testimonyOf({ type: 'resource', uri: `http://${host}/answer.bin`, name: 'answer.bin', sizeBytes: 8 }), 'zz-answer'); + // Every answer is on the server's own origin, each with its own token + // and eager limit, so a second read for one check would mix answers. + const answers = [ + { url: `ws://${host}`, token: 'zz-first' }, + { url: `ws://${host}`, token: 'zz-second', autofetch: { maxBytes: 1 } }, + { url: `ws://${host}`, token: 'zz-third', autofetch: { maxBytes: 100 } }, + ]; + let asked = 0; + const fetcher = new ReferenceFetcher(() => answers[Math.min(asked++, answers.length - 1)], async () => 'scratch/answer.bin'); + const fetching = fetcher.fetch(record); + await arrival; + assert.equal(asked, 1); + // The eager check of a record in flight joins it through dialServer, + // with one answer: the second's eager limit of 1 byte, which 8 bytes pass. + assert.equal(fetcher.isEagerEligible(record), false); + assert.equal(asked, 2); + release(); + assert.equal((await fetching).ok, true); + assert.deepEqual(heard, ['Bearer zz-first']); + }); + + it("hands the live copy the stored pass's own blocks, so a change between the two leaves the live copy as stored", async () => { + servingFiles(); + const link = { type: 'resource_link', uri: 'https://zz-files.example/handed.csv', name: 'handed.csv' }; + // Delivered by the zz-files server, as an MCPL dispatch registers it. + referenceRegistry.register(testimonyOf(link), 'zz-files'); + const result: ToolResult = { success: true, data: [IMAGE, link], dataIsContent: true }; + const fw = Object.create(AgentFramework.prototype) as { + buildStoredToolResultContent(agent: string, calls: unknown[], maxChars: number | undefined): Promise<{ spilled: Map }>; + toMembraneToolResult(callId: string, result: ToolResult, maxChars: number | undefined, precomputed: unknown): Live; + }; + Object.assign(fw, { toolImageLedgers: new Map() }); + const call = { id: 'zz-handed', name: 'zz_tool', input: {}, result }; + const { spilled } = await fw.buildStoredToolResultContent('zz-agent', [call], 1000); + const stored = spilled.get('zz-handed') as { text: string; live: unknown[] | null }; + const [, line] = stored.text.split('\n'); + // Fetchable when stored: the link is a stub. + assert.match(line!, /^\[ref_[^\]]+\] handed\.csv — from tool result — fetch with fetch_reference$/); + // Then its server's address goes, so classified again it would be a line. + referenceRegistry.forgetServerAddress('zz-files', TEST_HOST); + try { + const live = fw.toMembraneToolResult('zz-handed', result, 1000, stored).content as Array<{ type: string; text?: string }>; + assert.deepEqual(live.map((b) => b.text ?? b.type), ['image', line]); + } finally { + servingFiles(); + } + }); + + it("goes natively from the stored pass with links and markers exactly at the cap, and as its stored text one past it", async () => { + const fw = Object.create(AgentFramework.prototype) as { + buildStoredToolResultContent(agent: string, calls: unknown[], maxChars: number | undefined): Promise<{ spilled: Map }>; + toMembraneToolResult(callId: string, result: ToolResult, maxChars: number | undefined, precomputed: unknown): Live; + }; + Object.assign(fw, { toolImageLedgers: new Map(), writeToolResultFile: async () => null }); + const result: ToolResult = { success: true, data: [IMAGE, AUDIO, AUDIO], dataIsContent: true }; + const cap = AUDIO_TEXT.length * 2; + for (const [maxChars, native] of [[cap, true], [cap - 1, false]] as const) { + const call = { id: `zz-cap-${maxChars}`, name: 'zz_tool', input: {}, result }; + const { spilled } = await fw.buildStoredToolResultContent('zz-agent', [call], maxChars); + const stored = spilled.get(call.id) as { text: string; live: unknown[] | null }; + assert.equal(stored.live !== null, native, String(maxChars)); + const live = fw.toMembraneToolResult(call.id, result, maxChars, stored).content; + if (native) assert.ok(Array.isArray(live), String(maxChars)); + else assert.equal(live, stored.text, String(maxChars)); + } + }); + + test("an MCPL send whose result has an element that isn't a block before its channel's id still succeeds", { timeout: 30_000 }, async (t) => { + const f = await fixture({ + tools: [{ name: 'send_message', description: 'Send a message.', inputSchema: { type: 'object' } }], + toolsCall: () => ({ content: [null, { type: 'text', text: JSON.stringify({ channelId: 'zz-chan' }) }] }), + }); + t.after(f.close); + const internals = f.framework as unknown as { pushEvent(event: unknown): void; dispatchToolCall(agent: string, call: unknown): void }; + const result = await new Promise((resolve) => { + const original = internals.pushEvent; + internals.pushEvent = (event: unknown) => { + const e = event as { type: string; callId?: string; result?: ToolResult }; + if (e.type === 'tool-result' && e.callId === 'zz-send') { internals.pushEvent = original; resolve(e.result!); } + else original.call(internals, event); + }; + internals.dispatchToolCall('agent', { id: 'zz-send', name: 'mcpl--editor--send_message', input: { channelId: 'zz-chan' } }); + }); + assert.equal(result.success, true, JSON.stringify(result)); + assert.equal(nonErrorResultText(result), `${unknownBlockText(null)}\n{"channelId":"zz-chan"}`); + }); + it("makes an MCPL result with an element that isn't a block its marker, not a failure", async () => { const fw = framework() as unknown as { mcpToolResult(result: unknown, callId: string, mcplPeer: boolean, serverId: string, callName: string): Promise; From 262f3489dbe6f6c15c393672f494133a780c228b Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 22:23:18 -0400 Subject: [PATCH 14/40] fix(references): a result's claims are one summary per uri, its own first; the demand fetch and kept fields read them as showing does From the round review of the previous commit (af-f53-sol-r2). - A result's claims are one summary per uri (UriClaims: never, rejected, unparseable, earliest expiry), built once from its references, so each decision is constant time where it rescanned every testimony for the uri. - A decision reads its result's claims for a uri where the result's references say anything of it, and the record's own testimony, the latest an earlier result left, only where they don't. A link shown before its result's own reference no longer reads an earlier result's claims. - The module demand fetch reads a result's claims as showing it does (resultClaimsOf: a content array's elements, other data by the walk serialization takes), where it read a data array's top-level elements, and logs a fetch it doesn't make for a claimed expiry. - A withheld result's kept fields are shown with the whole result's claims (HistorySerializeOptions.claims), so a never elsewhere in it withholds a kept link's uri. - The words of a script's object and a stub give why a fetch_reference would be refused, as the result's references claim it; the fetch itself reads the record (Ada-1017's call). - Docs: contentClaims says an element's own fields don't count, the walk's docs hold for values that answer the same each time, claimsInHistory says toJSON and getters run in both walks, nativeContent's doc names its caller's check, shownInHistory's says a value RFC-005 doesn't govern keeps its fields. The claims walk writes a string as 0, its text being thrown away (Desmond-2579's measurement and suggestion). The changed fragment says when an earlier result's never decides. Tests: each block of a never result arrives, as a stub, a script object with its refId, or a script result's content; an expiry at depth; competing expiries worded by one fixed order; a result's own reference deciding over an earlier result's expiry in both orders; a content array not counting an image's _meta; claims read through toJSON, with the getter it leaves out unread; a withheld result's kept link withheld by a never elsewhere; a result's data read once without a stored copy, natively and as text; the demand fetch passing over an expiry at depth in a module's data, and its log line. Co-Authored-By: Claude Opus 5.5 --- .../mcp-standard-tool-results.changed.md | 7 +- changelog.d/tool-result-blocks.changed.md | 2 +- src/framework.ts | 25 ++- src/mcpl/references.ts | 101 +++++++---- src/tool-result-guard.ts | 6 +- src/tool-result-history.ts | 78 +++++--- test/script-structured-references.test.ts | 6 + test/tool-result-blocks.test.ts | 168 +++++++++++++++--- 8 files changed, 298 insertions(+), 95 deletions(-) diff --git a/changelog.d/mcp-standard-tool-results.changed.md b/changelog.d/mcp-standard-tool-results.changed.md index c648c3159..491a081c1 100644 --- a/changelog.d/mcp-standard-tool-results.changed.md +++ b/changelog.d/mcp-standard-tool-results.changed.md @@ -46,9 +46,10 @@ the uri. An unsaved reference that counts as expired also carries `expired`, the reason the model's stub gives, and isn't fetched. One the fetcher would refuse carries `unfetchable`, the words a stub gives in - place of advising `fetch_reference`. Both say what a `fetch_reference` - with the object's `refId` meets, so a link whose line shows its uri, - which the host can't fetch, carries them too. + place of advising `fetch_reference`. Both give why a `fetch_reference` + with the object's `refId` would be refused, as the result's references + claim it, so a link whose line shows its uri, which the host can't + fetch, carries them too. A block the classifier rejects arrives as the notice the model reads in its place, and inline data that also claims a bulk disposition as the withheld notice. #181, by @Meganeuridae, first diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 7b933bfbb..3e2c1ac5f 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1 +1 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or, where none in the result names the uri, whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. diff --git a/src/framework.ts b/src/framework.ts index a72fb5aa3..37cc28603 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,6 +1,6 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { dirname, join } from 'node:path'; -import { REFERENCE_LIMITS, claimedExpiry, classifyBlock, contentClaims, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ServerAddress } from './mcpl/references.js'; +import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, contentClaims, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ServerAddress } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, @@ -115,6 +115,7 @@ import type { ToolResultGuard, WithheldSpill } from './tool-result-guard.js'; import { goesNative, nonErrorResultText, + resultClaimsOf, toolResultDataToHistoryString, truncateForHistory, DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS, @@ -19218,10 +19219,11 @@ export class AgentFramework { // the ones it wants with fetch_reference. if (Array.isArray(result.data) && this.resolveMcplTool(toolName) === null) { const fetcher = this.getReferenceFetcher(); - // The array's claims, read as one: a uri any of its references gives - // an expiry that counts isn't fetched, as autoFetchReferences passes - // over one that any block of an MCPL result marks expired. - const claims = contentClaims(result.data); + // The result's claims, read as showing it reads them (resultClaimsOf): + // a uri any of its references gives an expiry that counts isn't + // fetched, as autoFetchReferences passes over one that any block of an + // MCPL result marks expired. + const claims = resultClaimsOf(result.data, result.dataIsContent); for (const block of result.data) { const c = classifyBlock(block); if (c.kind !== 'reference' || c.testimony.fromLink) continue; @@ -19229,7 +19231,11 @@ export class AgentFramework { // replace a link's, or a stale reference's, before the fetch reads // them, as on any other path, while the record isn't fetched yet. const record = referenceRegistry.register(c.testimony); - if (!record.fetchedPath && claimedExpiry(record, claims, Date.now()) !== null) continue; + const expired = record.fetchedPath ? null : claimedExpiry(record, claims, Date.now()); + if (expired !== null) { + console.error(`[rfc005] script-demand fetch of ${record.refId} not made: by the result's claims, the reference is ${EXPIRY_REASON[expired]}`); + continue; + } const outcome = await fetcher.fetch(record); if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); } @@ -19431,9 +19437,10 @@ export class AgentFramework { } /** - * A content array's blocks (toolResultBlocks), carrying at least one image - * whose links and markers fit the cap together (goesNative), as Membrane's - * native ToolResultContentBlock[]: each image as itself, each text block + * A content array's blocks (toolResultBlocks) as Membrane's native + * ToolResultContentBlock[], for a caller that found they go natively: at + * least one image, with links and markers that fit the cap together + * (goesNative), which this doesn't check. Each image as itself, each text block * cut at the cap, and every other block whole, as the text the stored copy * shows for it. `maxChars`, when provided, caps each text block so an image * inlined beside an enormous text payload can't blow the context. A block diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 5390e855f..7a5b7a5fa 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -701,29 +701,53 @@ export interface StubOptions { } /** - * What one tool result's references claim of each uri they name: uri → the - * testimony each of those references gave it. A decision a block of that - * result makes reads them all with its record's own, and the strictest claim - * wins: a uri any of them marks `disposition: never` is withheld, and one - * any of them gives an expiry that counts is expired (claimedExpiry), so a - * result means the same whatever the order of its blocks, and whatever the - * registry evicted while it was read (Ada-1017's call). Across results the - * registry keeps the latest testimony (register). + * What one tool result's references claim of each uri they name, each uri's + * testimonies read as one (UriClaims): the strictest claim wins, so a uri any + * of them marks `disposition: never` is withheld, and one any of them gives + * an expiry that counts is expired (claimedExpiry), whatever the order of the + * blocks and whatever the registry evicted while it was read (Ada-1017's + * call). A decision about a record reads its result's claims where the + * result's references say anything of its uri, and otherwise the record's + * own testimony, the latest the registry holds from an earlier result. A + * link claims no disposition or expiry, so it adds nothing here. */ -export type ResultClaims = ReadonlyMap; - -/** A result's claims (ResultClaims), from the testimonies of its references. */ +export type ResultClaims = ReadonlyMap; + +/** What a result's references say of one uri, read as one. */ +export interface UriClaims { + /** Whether any of them marks it `disposition: never`. */ + never: boolean; + /** Whether any of them gives an expiry rejected as a field. */ + rejected: boolean; + /** Whether any of them gives an expiry that doesn't parse. */ + unparseable: boolean; + /** The earliest expiry any of them gives, in ms since the epoch, or null. */ + earliestMs: number | null; +} + +/** A result's claims (ResultClaims), from the testimonies of its references, + * each read once, as referenceExpiry reads an expiry. */ export function resultClaims(testimonies: Iterable): ResultClaims { - const claims = new Map(); + const claims = new Map(); for (const t of testimonies) { - const list = claims.get(t.uri); - if (list) list.push(t); - else claims.set(t.uri, [t]); + if (t.fromLink) continue; + let c = claims.get(t.uri); + if (!c) claims.set(t.uri, c = { never: false, rejected: false, unparseable: false, earliestMs: null }); + if (t.disposition === 'never') c.never = true; + if (t.rejectedFields.includes('expiresAt')) { + c.rejected = true; + } else if (t.expiresAt !== undefined) { + const ms = Date.parse(t.expiresAt); + if (Number.isNaN(ms)) c.unparseable = true; + else if (c.earliestMs === null || ms < c.earliestMs) c.earliestMs = ms; + } } return claims; } -/** The claims of a content array's references, element by element. */ +/** The claims of a content array's references, read from its elements as + * the array shows them: an element's own fields, such as an image's + * `_meta`, aren't shown, so a reference inside them doesn't count. */ export function contentClaims(blocks: readonly unknown[]): ResultClaims { const testimonies: ReferenceTestimony[] = []; for (const block of blocks) { @@ -733,24 +757,29 @@ export function contentClaims(blocks: readonly unknown[]): ResultClaims { return resultClaims(testimonies); } -/** Every testimony a decision about a record reads: the record's own, and - * each one the result's references gave its uri. */ -function claimsOf(record: ReferenceRecord, claims?: ResultClaims): ReferenceTestimony[] { - return [record.testimony, ...(claims?.get(record.testimony.uri) ?? [])]; +/** The claims a decision about a record reads: its result's for its uri, + * where the result's references say anything of it, and otherwise the + * record's own testimony. */ +function uriClaims(record: ReferenceRecord, claims?: ResultClaims): UriClaims | undefined { + return claims?.get(record.testimony.uri) ?? resultClaims([record.testimony]).get(record.testimony.uri); } -/** Whether a record's uri is withheld: its own testimony, or any of the - * result's references, marks it `disposition: never`. */ +/** Whether a record's uri is withheld: marked `disposition: never` by its + * result's references, or by its own testimony where they say nothing. */ export function claimedNever(record: ReferenceRecord, claims?: ResultClaims): boolean { - return claimsOf(record, claims).some((t) => t.disposition === 'never'); + return uriClaims(record, claims)?.never === true; } /** The expiry a record counts, as of `nowMs`: the first of EXPIRY_REASON's - * cases that its own testimony, or any of the result's references, gives, - * so the order of the blocks doesn't choose the words. Null when none does. */ + * cases its claims give (uriClaims), so the order of the blocks doesn't + * choose the words. Null when none does. */ export function claimedExpiry(record: ReferenceRecord, claims: ResultClaims | undefined, nowMs: number): keyof typeof EXPIRY_REASON | null { - const found = new Set(claimsOf(record, claims).map((t) => referenceExpiry(t, nowMs))); - return (Object.keys(EXPIRY_REASON) as Array).find((kind) => found.has(kind)) ?? null; + const c = uriClaims(record, claims); + if (!c) return null; + if (c.earliestMs !== null && c.earliestMs <= nowMs) return 'passed'; + if (c.unparseable) return 'unparseable'; + if (c.rejected) return 'rejected'; + return null; } /** @@ -871,9 +900,11 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI * (shownLinkUri). An unsaved reference that counts as expired also carries * `expired`, and one the fetcher would refuse carries `unfetchable`, the * words a stub gives in place of advising fetch_reference - * (unfetchableReason). Both say what a fetch_reference with the object's - * `refId` meets, so a link whose line shows its uri, which the host can't - * fetch, carries them too: its line has no id to fetch with. `claims` are + * (unfetchableReason). Both give why a fetch_reference with the object's + * `refId` would be refused, as the result's references claim it, so a link + * whose line shows its uri, which the host can't fetch, carries them too: its + * line has no id to fetch with. The fetch itself reads the record, which + * keeps the latest testimony registered (Ada-1017's call). `claims` are * those of the content array the block came in (contentClaims), so its * other references count as they do for the model's line (ResultClaims). * The reference is registered bound to `serverId`, the server whose tool @@ -897,11 +928,11 @@ export function referenceBlockForScript(block: unknown, serverId: string, claims // with each other across an expiry. const nowMs = Date.now(); const uri = shownLinkUri(t, record, nowMs, claims); - // What a fetch_reference with this object's refId meets, in the words - // a stub gives, whether or not the model's line for it is a stub: the - // line has no id to fetch with, and the object does. As the stub - // judges it, a saved copy is still served, and the result's own claims - // count with the record's. + // Why a fetch_reference with this object's refId would be refused, in + // the words a stub gives, whether or not the model's line for it is a + // stub: the line has no id to fetch with, and the object does. As the + // stub judges it, a saved copy is still served, and the result's own + // claims count with the record's. const expiry = record.fetchedPath ? null : claimedExpiry(record, claims, nowMs); const unfetchable = unfetchableReason(record, nowMs, claims); return { diff --git a/src/tool-result-guard.ts b/src/tool-result-guard.ts index f4fffebdb..89bce6809 100644 --- a/src/tool-result-guard.ts +++ b/src/tool-result-guard.ts @@ -4,7 +4,7 @@ import type { ContextManager, MessageId, MessageMetadata } from '@animalabs/cont import type { ContentBlock, NormalizedMessage, ToolResult } from '@animalabs/membrane'; import type { CompletedToolCall } from './types/index.js'; import { ensureStateRegistered } from './module-registry.js'; -import { rfc005TextOf, toolResultDataToHistoryString } from './tool-result-history.js'; +import { resultClaimsOf, rfc005TextOf, toolResultDataToHistoryString } from './tool-result-history.js'; import { safeSlice } from './safe-slice.js'; export const TOOL_RESULT_GUARD_NOTICE = 'Tool result withheld by the guard. The tool has already executed.'; @@ -104,7 +104,9 @@ export function keptWhenWithheld(result: CompletedToolCall['result'], maxChars: // Data that RFC-005 shows as text, such as a reference's stub, keeps that // text: picked out of it, a kept uri would lose the type that makes it a // reference, and be written out of the stub that hides it. - const whole = rfc005TextOf(fields) ?? toolResultDataToHistoryString(kept); + // The kept fields are shown as the whole result claims (resultClaimsOf), so + // a never elsewhere in it still withholds a kept link's uri. + const whole = rfc005TextOf(fields) ?? toolResultDataToHistoryString(kept, undefined, { claims: resultClaimsOf(fields) }); if (!maxChars || whole.length <= maxChars) return whole; // Never half a surrogate pair: the stub goes to the provider as text, so // the cut can be one character short of the cap. diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index d18aa9894..fba5dfe15 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -100,16 +100,31 @@ const MCP_BLOCK_TYPES = new Set(['text', 'image', 'audio', 'resource', 'resource * is read as content, and shows a marker for each record that isn't a block * the lane can show. */ -export function toolResultBlocks(data: unknown, dataIsContent?: boolean): ToolResultBlock[] | null { - if (!Array.isArray(data)) return null; - if (dataIsContent !== true && !data.every((element) => isPlainRecord(element) && MCP_BLOCK_TYPES.has(element.type as string))) { - return null; - } +export function toolResultBlocks(data: unknown, dataIsContent?: boolean, claims?: ResultClaims): ToolResultBlock[] | null { + if (!isContentArray(data, dataIsContent)) return null; // The result's claims, read as one (ResultClaims): what any of its // references says of a uri (a `disposition: never`, an expiry) holds for // each block that names it, whatever their order. - const claims = contentClaims(data); - return data.map((raw) => toolResultBlock(raw, claims)); + const read = claims ?? contentClaims(data); + return data.map((raw) => toolResultBlock(raw, read)); +} + +/** Whether a tool result's data is a content array, which toolResultBlocks + * shows block by block. */ +function isContentArray(data: unknown, dataIsContent?: boolean): data is unknown[] { + return Array.isArray(data) + && (dataIsContent === true || data.every((element) => isPlainRecord(element) && MCP_BLOCK_TYPES.has(element.type as string))); +} + +/** + * The claims of a tool result's data (ResultClaims), as showing it reads + * them: a content array's from its elements (contentClaims), and any other + * data's by the walk serialization takes (claimsInHistory). For a decision + * made about the result outside showing it, such as which of its references + * to fetch, or for showing part of it as the rest of it claims. + */ +export function resultClaimsOf(data: unknown, dataIsContent?: boolean): ResultClaims { + return isContentArray(data, dataIsContent) ? contentClaims(data) : claimsInHistory(data); } /** One element of a content array, as toolResultBlocks gives it. Only an @@ -180,6 +195,12 @@ export interface HistorySerializeOptions { * blocks, so the two copies come from one classification. */ onBlocks?: (spans: HistoryBlockSpan[], blocks: ToolResultBlock[]) => void; + /** + * The claims to read (ResultClaims) in place of the data's own, where the + * data shown is part of a larger result whose claims these are + * (resultClaimsOf), such as the fields a withheld result keeps. + */ + claims?: ResultClaims; } /** @@ -209,7 +230,7 @@ export function toolResultDataToHistoryString( const fromArray = tryHistoryStringFromContentArray(data, options); // JSON leaves out undefined, a function and a symbol: a result with no data // is stored as no text, as an empty content array is. - const str = fromArray ?? historyJson(data) ?? ''; + const str = fromArray ?? historyJson(data, options?.claims) ?? ''; return maxChars ? truncateForHistory(str, maxChars) : str; } @@ -255,7 +276,7 @@ export function truncateForHistory(str: string, maxChars: number): string { * content array, for the fallback, historyJson. */ function tryHistoryStringFromContentArray(data: unknown, options?: HistorySerializeOptions): string | null { - const blocks = toolResultBlocks(data, options?.dataIsContent); + const blocks = toolResultBlocks(data, options?.dataIsContent, options?.claims); if (blocks === null) return null; const parts = blocks.map((block, blockIndex): { kind: HistoryBlockSpan['kind']; text: string } => { if (block.kind !== 'image') return block; @@ -311,10 +332,10 @@ function isPlainRecord(value: unknown): value is Record { * one (such as a uri that isn't a string, or MCP's embedded resource, whose * uri isn't at its top level) as the marker saying why it isn't shown, and * inline data that also claims a uri or a disposition as withheld. So the - * only uri shown is a link's, in its line, where the host can't fetch the - * link and its uri can't carry a credential (shownLinkUri). Inline data that - * claims neither is itself, as is any other value: - * a block of another type keeps its fields, since RFC-005 governs + * only reference's uri shown is a link's, in its line, where the host can't + * fetch the link and its uri can't carry a credential (shownLinkUri). Inline + * data that claims neither is itself, as is any other value: a block of + * another type keeps its fields, a url among them, since RFC-005 governs * references. */ function shownInHistory(value: unknown, claims?: ResultClaims): unknown { @@ -325,8 +346,9 @@ function shownInHistory(value: unknown, claims?: ResultClaims): unknown { * The one walk of a tool result's data that showing it and reading its * claims both take: a value RFC-005 governs is text, from `asText`, except * inline data that claims no uri or disposition, which is itself, as is any - * other value, so serialization descends into exactly the same values each - * time. `asText` gives a governed value's text, or null where it's itself. + * other value, so serialization descends into the same values each time, so + * long as each value's toJSON and getters answer the same each time they're + * read. `asText` gives a governed value's text, or null where it's itself. */ function walkedInHistory(value: unknown, asText: (b: Record) => string | null): unknown { if (!value || typeof value !== 'object' || Array.isArray(value)) return value; @@ -347,7 +369,10 @@ function walkedInHistory(value: unknown, asText: (b: Record) => * through toJSON, the fields of inline data and every depth it shows: so * every reference whose stub the result shows counts, and nothing it doesn't * show does. A value serialization can't write, such as a cycle, stops the - * walk where it would stop the result. + * walk where it would stop the result. It's a walk of its own before the one + * that shows the data, so a value's toJSON and getters run in both: one that + * answers differently the second time can claim what the shown text + * doesn't show. */ function claimsInHistory(data: unknown): ResultClaims { const testimonies: ReferenceTestimony[] = []; @@ -355,14 +380,23 @@ function claimsInHistory(data: unknown): ResultClaims { const c = classifyBlock(b); if (c.kind === 'reference') testimonies.push(c.testimony); // Text where referenceStubOrNull gives text, so the walk descends where - // showing descends. - return c.kind === 'text' || (c.kind === 'inline' && !c.contradiction) ? null : ''; + // showing descends. Inline data never reaches here: walkedInHistory + // decides it first. + return c.kind === 'text' ? null : ''; }; try { const root = walkedInHistory(data, read); - if (root === data || typeof root !== 'string') JSON.stringify(data, (_key, value: unknown) => walkedInHistory(value, read)); + // Its text is thrown away, so a string is written as 0, which keeps a + // long one from being copied for nothing; the walk descends as before. + if (root === data || typeof root !== 'string') { + JSON.stringify(data, (_key, value: unknown) => { + const walked = walkedInHistory(value, read); + return typeof walked === 'string' ? 0 : walked; + }); + } } catch { - // Showing the result meets the same failure at the same place. + // Showing the result meets the same failure at the same place, for a + // value that answers the same each time it's read. } return resultClaims(testimonies); } @@ -383,8 +417,8 @@ export function rfc005TextOf(value: unknown): string | null { * it (shownInHistory), at any depth. Data that is itself shown as text, such * as a bare reference, is that text. */ -function historyJson(data: unknown): string | undefined { - const claims = claimsInHistory(data); +function historyJson(data: unknown, given?: ResultClaims): string | undefined { + const claims = given ?? claimsInHistory(data); const root = shownInHistory(data, claims); if (root !== data && typeof root === 'string') return root; return JSON.stringify(data, (_key, value: unknown) => shownInHistory(value, claims)) as string | undefined; diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index e94e1a31b..34aa587e7 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -302,5 +302,11 @@ test("an MCPL peer's link reads every reference in its result, so a never anywhe }); const out = await h.script('mcpl--files--attach'); assert.ok(!out.includes('elsewhere.example.invalid'), out); + // Each block reached the script, the link as a reference object. + const object = JSON.parse(out) as { content: Array<{ type: string; refId?: string; uri?: string }>; isError: boolean }; + assert.equal(object.isError, false); + assert.deepEqual(object.content.map((b) => b.type), ['resource', 'resource', 'resource_link']); + assert.match(String(object.content[2]!.refId), /^ref_/); + assert.equal(object.content[2]!.uri, undefined); }); }); diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 0f7968fa5..8e66dcbc4 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -20,6 +20,8 @@ import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { JsStore } from '@animalabs/chronicle'; import { AgentFramework } from '../src/index.js'; +import * as referencesModule from '../src/mcpl/references.js'; +import { keptWhenWithheld } from '../src/tool-result-guard.js'; import type { ScriptToolResult } from '../src/index.js'; import type { Module, ModuleContext, ToolCall, ToolResult, ToolDefinition } from '../src/types/index.js'; import { @@ -74,6 +76,11 @@ const PLAIN_LINK = { ...LINK, uri: PLAIN_URI }; const PLAIN_LINK_TEXT = `[resource link "report.csv": ${PLAIN_URI}, text/csv; not fetched]`; /** Record a server dialed on the links' origin, as the framework records a server it configures. Each * test that needs it asks, so the file loads against a registry that keeps no addresses. */ +/** A content array's claims, by the module's own contentClaims, read off the + * module's namespace so the file still loads against main's src, which + * lacks it, for red-first. */ +const contentClaimsOf = (blocks: unknown[]) => + (referencesModule as unknown as { contentClaims(blocks: unknown[]): referencesModule.ResultClaims }).contentClaims(blocks); const servingFiles = () => referenceRegistry.setServerAddress('zz-files', { url: 'wss://zz-files.example' }, TEST_HOST); /** The framework these tests record servers' addresses for, as one does. */ const TEST_HOST = {}; @@ -319,8 +326,8 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { const line = toolResultDataToHistoryString([TEXT, link], undefined, { dataIsContent: true }).split('\n')[1]!; assert.equal(line.startsWith('[resource link '), shows, `${what}: ${line}`); assert.equal(object.uri, shows ? link.uri : undefined, what); - // The object says what a fetch_reference with its refId meets: where - // the line is a stub, the reason it gives in place of advising + // The object says why a fetch_reference with its refId would be + // refused: where the line is a stub, the reason it gives in place of advising // fetch_reference, in the same words, and where the line shows the // uri, which it does only for a link the host can't fetch, the reason // that fetch is refused, which the line, having no id, doesn't give. @@ -920,18 +927,23 @@ describe('one answer for every block of a result', () => { const link = { type: 'resource_link', uri, name: 'order.csv' }; const never = { type: 'resource', uri, name: 'order.csv', disposition: 'never' }; const plain = { type: 'resource', uri, name: 'order.csv' }; + const STUB = /^\[ref_[^\]]+\] order\.csv — from tool result — /; for (const order of [[link, never, plain], [never, link, plain], [never, plain, link], [plain, never, link], [plain, link, never]]) { const what = order.map((b) => (b === link ? 'link' : b === never ? 'never' : 'plain')).join(', '); - const shown = toolResultDataToHistoryString([TEXT, ...order]); - assert.ok(!shown.includes(uri), `${what}: ${shown}`); - // At any depth in data too, and in a script's object. - const deep = toolResultDataToHistoryString({ first: order.slice(0, 1), then: { rest: order.slice(1) } }); - assert.ok(!deep.includes(uri), `${what}: ${deep}`); - // The array's claims (contentClaims), built here so the file loads - // against main's src for red-first. - const claims = new Map([[uri, order.map(testimonyOf)]]); + const lines = toolResultDataToHistoryString([TEXT, ...order]).split('\n'); + // Each block is there, the link as a stub with no uri. + assert.equal(lines.length, 4, `${what}: ${lines.join('\n')}`); + assert.ok(lines.slice(1).every((line) => STUB.test(line)), `${what}: ${lines.join('\n')}`); + // At any depth in data too. + const deep = JSON.parse(toolResultDataToHistoryString({ first: order.slice(0, 1), then: { rest: order.slice(1) } })) as + { first: string[]; then: { rest: string[] } }; + assert.ok([...deep.first, ...deep.then.rest].every((stub) => STUB.test(stub)), `${what}: ${JSON.stringify(deep)}`); + // And in a script's object, with the array's claims. + const claims = contentClaimsOf(order); for (const block of order) { - const object = referenceBlockForScript(block, 'zz-files', claims) as { uri?: string }; + const object = referenceBlockForScript(block, 'zz-files', claims) as { type: string; uri?: string; refId?: string }; + assert.equal(object.type, block.type, what); + assert.match(String(object.refId), /^ref_/, what); assert.equal(object.uri, undefined, what); } } @@ -953,7 +965,7 @@ describe('one answer for every block of a result', () => { assert.equal(lines.filter((line) => line.endsWith('— the reference is expired')).length, 2, lines.join('\n')); texts.add([...lines].sort().join('\n').replace(/ref_[^\]]+/g, 'ref')); // And a script's object for each reference, with the array's claims. - const claims = new Map([[uri, order.map(testimonyOf)]]); + const claims = contentClaimsOf(order); for (const block of [fresh, expired]) { const object = referenceBlockForScript(block, 'zz-files', claims) as { expired?: string; unfetchable?: string }; assert.equal(object.expired, EXPIRY_REASON.passed); @@ -961,6 +973,37 @@ describe('one answer for every block of a result', () => { } } assert.equal(texts.size, 1); + // At any depth in data too. + const deep = JSON.parse(toolResultDataToHistoryString({ first: [link], then: { ref: expired } })) as { first: string[]; then: { ref: string } }; + assert.deepEqual(deep.first, [`[resource link "order.csv": ${uri}; not fetched]`]); + assert.ok(deep.then.ref.endsWith('— the reference is expired'), deep.then.ref); + }); + + it("words competing expiries in a result by one fixed order, whatever the order of its blocks", () => { + servingFiles(); + const uri = 'https://zz-files.example/competing.csv'; + const passed = { type: 'resource', uri, name: 'competing.csv', expiresAt: '2001-01-01T00:00:00Z' }; + const unreadable = { type: 'resource', uri, name: 'competing.csv', expiresAt: 'zz-not-a-time' }; + referenceRegistry.register(testimonyOf(passed), 'zz-files'); + for (const order of [[passed, unreadable], [unreadable, passed]]) { + const lines = toolResultDataToHistoryString(order).split('\n'); + assert.ok(lines.every((line) => line.endsWith(`— the reference is ${EXPIRY_REASON.passed}`)), lines.join('\n')); + } + }); + + it("reads the result's own references for a uri before what an earlier result left in the registry, whatever the order", () => { + servingFiles(); + const uri = 'https://zz-files.example/earlier-expired.csv'; + const link = { type: 'resource_link', uri, name: 'earlier.csv' }; + referenceRegistry.register(testimonyOf(link), 'zz-files'); + // An earlier result left the uri expired. + toolResultDataToHistoryString([{ type: 'resource', uri, name: 'earlier.csv', expiresAt: '2001-01-01T00:00:00Z' }]); + const fresh = { type: 'resource', uri, name: 'earlier.csv', expiresAt: '2099-01-01T00:00:00Z' }; + const before = toolResultDataToHistoryString([link, fresh]).split('\n')[0]; + const after = toolResultDataToHistoryString([fresh, link]).split('\n')[1]; + // This result's fresh reference decides: the link is a stub that fetches. + assert.match(before!, /^\[ref_[^\]]+\] earlier\.csv — from tool result — fetch with fetch_reference$/); + assert.equal(after, before); }); it("finds a never inside inline data's other fields, which showing it descends into, before any block is shown", () => { @@ -971,25 +1014,41 @@ describe('one answer for every block of a result', () => { { type: 'image', data: PNG, mimeType: 'image/png', _meta: { ref: { type: 'resource', uri, name: 'meta.csv', disposition: 'never' } } }, ], }; - const shown = toolResultDataToHistoryString(data); - assert.ok(!shown.includes(uri), shown); + const shown = JSON.parse(toolResultDataToHistoryString(data)) as { items: [string, { _meta: { ref: string } }] }; + assert.match(shown.items[0], /^\[ref_[^\]]+\] meta\.csv — from tool result — /); + assert.match(shown.items[1]._meta.ref, /^\[ref_[^\]]+\] meta\.csv — from tool result — /); + // A content array shows an image, not its fields, so there a reference + // inside them doesn't count, and the link shows its uri. + const shownUri = 'https://zz-elsewhere.example/meta-in-array.csv'; + const array = toolResultDataToHistoryString([ + { type: 'resource_link', uri: shownUri, name: 'meta.csv' }, + { type: 'image', data: PNG, mimeType: 'image/png', _meta: { ref: { type: 'resource', uri: shownUri, name: 'meta.csv', disposition: 'never' } } }, + ], undefined, { dataIsContent: true }); + assert.equal(array.split('\n')[0], `[resource link "meta.csv": ${shownUri}; not fetched]`); }); it("keeps a never for its result when more references than the registry holds come between it and the link", () => { const uri = 'https://zz-elsewhere.example/evicted-never.csv'; const many = Array.from({ length: 2001 }, (_, i) => ({ type: 'resource', uri: `https://zz-elsewhere.example/many/${i}.bin`, name: `${i}.bin` })); - const shown = toolResultDataToHistoryString([ + const lines = toolResultDataToHistoryString([ { type: 'resource', uri, name: 'evicted.csv', disposition: 'never' }, ...many, { type: 'resource_link', uri, name: 'evicted.csv' }, - ]); - assert.ok(!shown.includes(uri), shown.slice(-400)); + ]).split('\n'); + assert.equal(lines.length, 2003); + assert.match(lines[2002]!, /^\[ref_[^\]]+\] evicted\.csv — from tool result — /); }); - it("reads claims through toJSON, as serialization does, so a getter toJSON leaves out isn't read", () => { + it("reads claims through toJSON, as serialization does, so a never toJSON gives counts and a getter it leaves out isn't read", () => { + const uri = 'https://zz-elsewhere.example/to-json-never.csv'; + let reads = 0; const value = { - toJSON: () => ({ zz: 'zz-shown' }), - get broken(): string { throw new Error('zz-getter read'); }, + toJSON: () => ({ ref: { type: 'resource', uri, name: 'to-json.csv', disposition: 'never' } }), + get unread(): string { reads += 1; return 'zz-unread'; }, }; - assert.equal(toolResultDataToHistoryString({ value }), '{"value":{"zz":"zz-shown"}}'); + const shown = JSON.parse(toolResultDataToHistoryString({ link: { type: 'resource_link', uri, name: 'to-json.csv' }, value })) as + { link: string; value: { ref: string } }; + assert.match(shown.link, /^\[ref_[^\]]+\] to-json\.csv — from tool result — /); + assert.match(shown.value.ref, /^\[ref_[^\]]+\] to-json\.csv — from tool result — /); + assert.equal(reads, 0); }); it("registers nothing a result's serialization never reaches, such as what lies past a cycle", () => { @@ -1000,6 +1059,59 @@ describe('one answer for every block of a result', () => { assert.equal(referenceRegistry.findByUri(uri), undefined); }); + it("keeps a withheld result's fields as the whole result claims them, so a never elsewhere withholds a kept link's uri", () => { + const uri = 'https://zz-elsewhere.example/kept-never.csv'; + const kept = keptWhenWithheld({ + success: true, + data: { files: [{ type: 'resource_link', uri, name: 'kept.csv' }], other: { type: 'resource', uri, name: 'kept.csv', disposition: 'never' } }, + keepWhenWithheld: ['files'], + }, undefined); + assert.ok(kept !== undefined && /^\{"files":\["\[ref_[^\]]+\] kept\.csv — from tool result — /.test(kept), kept); + }); + + it("reads a result's data once without a stored copy, and sends what that one reading made", () => { + // Natively, beside an image, and as text, with none. + for (const [data, expected] of [ + [[IMAGE, TEXT], ['image', 'text']], + [[TEXT, AUDIO], `zz-text\n${AUDIO_TEXT}`], + ] as const) { + let reads = 0; + const result = { success: true, dataIsContent: true, get data() { reads += 1; return data; } } as unknown as ToolResult; + const live = framework().toMembraneToolResult('zz-once', result, 1000).content; + assert.deepEqual(Array.isArray(live) ? (live as Array<{ type: string }>).map((b) => b.type) : live, expected); + assert.equal(reads, 1, JSON.stringify(expected)); + } + }); + + it("isn't fetched for a script on demand where a reference at depth in the module's data gives its uri an expiry, as the text reads it", async () => { + const fw = Object.create(AgentFramework.prototype) as { + dispatchScriptToolCall: (...args: unknown[]) => Promise; + getReferenceFetcher(): unknown; + handleScriptToolCall(agentName: string, toolName: string, args: Record): Promise; + }; + const uri = 'https://zz-files.example/demand-deep-expired.pdf'; + // Data, not content: its second element isn't a block, so its text reads + // the result's claims at depth, where the expiry is. + const data = [ + { type: 'resource', uri, name: 'deep.pdf', expiresAt: '2099-01-01T00:00:00Z' }, + { nested: { type: 'resource', uri, name: 'deep.pdf', expiresAt: '2001-01-01T00:00:00Z' } }, + ]; + fw.dispatchScriptToolCall = async () => ({ success: true, data }); + const fetched: string[] = []; + fw.getReferenceFetcher = () => ({ fetch: async (record: { testimony: { uri: string } }) => { fetched.push(record.testimony.uri); return { ok: false, error: 'zz-refused' }; } }); + const error = console.error; + console.error = () => {}; + let result: ScriptToolResult; + try { + result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + } finally { + console.error = error; + } + assert.equal(result.isError, false); + assert.deepEqual(fetched, []); + assert.ok(result.text.includes('— the reference is expired'), result.text); + }); + it("isn't fetched for a script on demand where any reference in the module's array gives its uri an expiry that counts", async () => { const fw = Object.create(AgentFramework.prototype) as { dispatchScriptToolCall: (...args: unknown[]) => Promise; @@ -1012,9 +1124,19 @@ describe('one answer for every block of a result', () => { fw.dispatchScriptToolCall = async () => ({ success: true, data: [fresh, expired], dataIsContent: true }); const fetched: string[] = []; fw.getReferenceFetcher = () => ({ fetch: async (record: { testimony: { uri: string } }) => { fetched.push(record.testimony.uri); return { ok: false, error: 'zz-refused' }; } }); - const result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + const logged: string[] = []; + const error = console.error; + console.error = (line: string) => { logged.push(line); }; + let result: ScriptToolResult; + try { + result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + } finally { + console.error = error; + } assert.equal(result.isError, false); assert.deepEqual(fetched, []); + // The fetch not made is logged, as a refused one is. + assert.ok(logged.some((line) => /script-demand fetch of ref_\S+ not made: by the result's claims, the reference is expired/.test(line)), logged.join('\n')); }); it('shows a link with no usable uri at any depth as its marker, never its fields', () => { @@ -1076,7 +1198,7 @@ describe('one answer for every block of a result', () => { assert.ok(marker.length < 250, String(marker.length)); }); - it("gives a script's object for a link whose line shows its uri what its refId meets, which the line, having no id, doesn't say", () => { + it("gives a script's object for a link whose line shows its uri why its refId would be refused, which the line, having no id, doesn't say", () => { // On its server's own origin, so the expiry is the one refusal. servingFiles(); const uri = 'https://zz-files.example/expired-link.csv'; From 6341f5fc0e0c56d0357d91012a0e03e6c702c11c Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 23:29:48 -0400 Subject: [PATCH 15/40] fix(references): EXPIRY_REASON's order words competing expiries; a script's text reads its demand fetch's claims From the round review of the previous commit (af-f53-sol-r3). - claimedExpiry takes the first of EXPIRY_REASON's cases a uri's claims give, by EXPIRY_REASON's key order, as its doc says, where the previous commit wrote the order out by hand. - A module's result shown to a script reads the claims its demand fetch read (HistorySerializeOptions.claims), so its data is walked for claims once, and the fetches and the text read the same claims. - Docs: where a result's references name a uri, a plain one included, they decide it, and the record's testimony decides otherwise: a reference is its result's latest word on the uri, as register() keeps it, so an earlier result's never or expiry gives way to it (Ada-1017's call). The comments that said the record's testimony is read with the result's claims now say so. A stub reads an expiry with its result's claims and the fetch with the record, which keeps the latest testimony until it's fetched; the fixed fragment no longer says a stub ends with what the fetch would say. The changed fragment says a link names a uri without claiming anything, and claimsInHistory says where a withheld result meets a value its walk can't write. - The depth test of the demand fetch, committed in the previous commit without a run, expected the stubs to say the reference is expired; a module's reference has no server, so its stub says that first. It now asserts the demand loop's own log line. Tests: competing expiries of all three kinds in every order; a later result's plain reference lifting an earlier never, link first and reference first, and a link alone leaving it; a module's content array whose unknown block names a uri expired is still fetched; a fresh control for the depth test. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- changelog.d/tool-result-blocks.fixed.md | 2 +- src/framework.ts | 16 +-- src/mcpl/references.ts | 53 +++++---- src/tool-result-history.ts | 25 +++-- test/tool-result-blocks.test.ts | 124 ++++++++++++++-------- 6 files changed, 141 insertions(+), 81 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 3e2c1ac5f..ff4f754cb 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1 +1 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or, where none in the result names the uri, whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md index 7d370e33e..7ebeb247f 100644 --- a/changelog.d/tool-result-blocks.fixed.md +++ b/changelog.d/tool-result-blocks.fixed.md @@ -1,6 +1,6 @@ - A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's that doesn't set `dataIsContent`, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. - MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link with no usable uri is the marker that says so. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. -- A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it ends with what `fetch_reference` would say happened, without what calling again can do. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, one whose address doesn't parse, a uri off its server's origin, one with a user name or password, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. +- A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it ends with why `fetch_reference` would refuse, without what calling again can do. The stub reads an expiry from the references in its result where they name its uri, and the fetch from the reference as the host last recorded it, so where one result names a uri both expired and fresh, the stub says it's expired while a fetch can succeed. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, one whose address doesn't parse, a uri off its server's origin, one with a user name or password, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. - A label in a stub (a name, a type, a server's id, a provenance or a saved path) loses any line or paragraph separator (U+2028, U+2029), as it loses control characters. - A script's demand fetch of a reference not yet fetched reads that reference's own digest and expiry, where it read whatever an earlier reference to the same uri had claimed. - Inline data beside an empty `uri` is shown as any inline data is, on every lane, where it was withheld as data that also claimed a link. diff --git a/src/framework.ts b/src/framework.ts index 37cc28603..cc26932cb 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,6 +1,6 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { dirname, join } from 'node:path'; -import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, contentClaims, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ServerAddress } from './mcpl/references.js'; +import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, contentClaims, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ResultClaims, type ServerAddress } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, @@ -19217,13 +19217,15 @@ export class AgentFramework { // which isn't fetched. A link (MCP's resource_link) isn't fetched either: // a tool's links are what it offers to follow, and the script can fetch // the ones it wants with fetch_reference. + // The result's claims, read as showing it reads them (resultClaimsOf), + // once: the fetches below and the text the script gets read the same. + let claims: ResultClaims | undefined; if (Array.isArray(result.data) && this.resolveMcplTool(toolName) === null) { const fetcher = this.getReferenceFetcher(); - // The result's claims, read as showing it reads them (resultClaimsOf): - // a uri any of its references gives an expiry that counts isn't - // fetched, as autoFetchReferences passes over one that any block of an - // MCPL result marks expired. - const claims = resultClaimsOf(result.data, result.dataIsContent); + // A uri any of the result's references gives an expiry that counts + // isn't fetched, as autoFetchReferences passes over one that any block + // of an MCPL result marks expired. + claims = resultClaimsOf(result.data, result.dataIsContent); for (const block of result.data) { const c = classifyBlock(block); if (c.kind !== 'reference' || c.testimony.fromLink) continue; @@ -19248,7 +19250,7 @@ export class AgentFramework { if (structured) return this.scriptStructuredResult(result, toolName); try { return { - text: toolResultDataToHistoryString(result.data, AgentFramework.SCRIPT_RESULT_MAX_CHARS, { dataIsContent: Boolean(result.dataIsContent) }), + text: toolResultDataToHistoryString(result.data, AgentFramework.SCRIPT_RESULT_MAX_CHARS, { dataIsContent: Boolean(result.dataIsContent), claims }), isError: false, }; } catch (err) { diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 7a5b7a5fa..67ac0f372 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -472,7 +472,10 @@ export function dialServer( * fetcher's pre-dial check of its own framework's configs, whose credential * goes with the address it checks. In a process with one running framework, * or with running frameworks that agree on each id they share, the two - * sources hold the same addresses, so a stub promises what the fetch does. A + * sources hold the same addresses, so a stub promises the server the fetch + * dials. A stub reads an expiry with its result's claims, and the fetch with + * the record's latest testimony, so where one result names a uri both + * expired and fresh, its stub says expired while a fetch can succeed. A * stopped framework's addresses are out of the registry, so its stubs read * another running framework's entry for the same id where there is one, and * otherwise say the server is unknown, while its configs still resolve it. @@ -486,8 +489,9 @@ export function dialCheck( addressOf: (serverId: string) => ServerAddress | undefined, nowMs = Date.now(), /** For a stub or a script's object, the claims of the result it's shown - * in, read with the record's own (claimedExpiry). A fetch reads the - * record alone. */ + * in, which decide where its references name the record's uri, the + * record's own testimony deciding otherwise (claimedExpiry). A fetch + * reads the record alone. */ claims?: ResultClaims, ): { serverId: string } | { refusal: Refusal } { const server = dialServer(record, addressOf); @@ -695,8 +699,9 @@ export interface StubOptions { * other lane showed a link's uri, so none shows one now. */ unfetchedLinkUri?: boolean; - /** The claims of the result this block is shown in (resultClaims), read - * with the record's own for what the stub says. */ + /** The claims of the result this block is shown in (resultClaims), which + * decide what the stub says where its references name the record's uri, + * the record's own testimony deciding otherwise. */ claims?: ResultClaims; } @@ -707,9 +712,14 @@ export interface StubOptions { * an expiry that counts is expired (claimedExpiry), whatever the order of the * blocks and whatever the registry evicted while it was read (Ada-1017's * call). A decision about a record reads its result's claims where the - * result's references say anything of its uri, and otherwise the record's - * own testimony, the latest the registry holds from an earlier result. A - * link claims no disposition or expiry, so it adds nothing here. + * result's references name its uri, a plain one included, and otherwise the + * record's own testimony, from an earlier result. A reference naming the uri + * is that result's word on it, the latest, as the registry keeps it + * (register() replaces a record's testimony with it until the record is + * fetched), so an earlier result's `never` or expiry gives way to it + * (Ada-1017's call). A link claims no disposition or expiry, so it adds + * nothing here, and a result that names a uri only in links leaves its + * record's testimony in force. */ export type ResultClaims = ReadonlyMap; @@ -758,14 +768,15 @@ export function contentClaims(blocks: readonly unknown[]): ResultClaims { } /** The claims a decision about a record reads: its result's for its uri, - * where the result's references say anything of it, and otherwise the - * record's own testimony. */ + * where the result's references name it, and otherwise the record's own + * testimony. */ function uriClaims(record: ReferenceRecord, claims?: ResultClaims): UriClaims | undefined { return claims?.get(record.testimony.uri) ?? resultClaims([record.testimony]).get(record.testimony.uri); } /** Whether a record's uri is withheld: marked `disposition: never` by its - * result's references, or by its own testimony where they say nothing. */ + * result's references, or by its own testimony where none of them names + * it. */ export function claimedNever(record: ReferenceRecord, claims?: ResultClaims): boolean { return uriClaims(record, claims)?.never === true; } @@ -776,10 +787,12 @@ export function claimedNever(record: ReferenceRecord, claims?: ResultClaims): bo export function claimedExpiry(record: ReferenceRecord, claims: ResultClaims | undefined, nowMs: number): keyof typeof EXPIRY_REASON | null { const c = uriClaims(record, claims); if (!c) return null; - if (c.earliestMs !== null && c.earliestMs <= nowMs) return 'passed'; - if (c.unparseable) return 'unparseable'; - if (c.rejected) return 'rejected'; - return null; + const given: Record = { + passed: c.earliestMs !== null && c.earliestMs <= nowMs, + unparseable: c.unparseable, + rejected: c.rejected, + }; + return (Object.keys(EXPIRY_REASON) as Array).find((reason) => given[reason]) ?? null; } /** @@ -794,8 +807,9 @@ export function claimedExpiry(record: ReferenceRecord, claims: ResultClaims | un * The credential check reads the uri as it would be shown, with what * sanitizeLabel strips gone, so a uri split by a control character is read * as it would read: `ht\u0000tps://user:pass@…` parses as a path, and shows - * as userinfo. A reference's disposition:"never" for the uri, the record's - * or any in the same result (claims), keeps it out whatever the link says. + * as userinfo. A `disposition: "never"` for the uri keeps it out whatever + * the link says: any reference's in the same result (claims), or, where no + * reference in the result names the uri, the record's. */ export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, nowMs = Date.now(), claims?: ResultClaims): string | null { if (!t.fromLink || claimedNever(record, claims)) return null; @@ -904,7 +918,8 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI * `refId` would be refused, as the result's references claim it, so a link * whose line shows its uri, which the host can't fetch, carries them too: its * line has no id to fetch with. The fetch itself reads the record, which - * keeps the latest testimony registered (Ada-1017's call). `claims` are + * keeps the latest testimony registered until it's fetched, and then the one + * its saved copy was verified against (Ada-1017's call). `claims` are * those of the content array the block came in (contentClaims), so its * other references count as they do for the model's line (ResultClaims). * The reference is registered bound to `serverId`, the server whose tool @@ -932,7 +947,7 @@ export function referenceBlockForScript(block: unknown, serverId: string, claims // the words a stub gives, whether or not the model's line for it is a // stub: the line has no id to fetch with, and the object does. As the // stub judges it, a saved copy is still served, and the result's own - // claims count with the record's. + // claims decide where its references name the uri (uriClaims). const expiry = record.fetchedPath ? null : claimedExpiry(record, claims, nowMs); const unfetchable = unfetchableReason(record, nowMs, claims); return { diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index fba5dfe15..d186d96f8 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -331,12 +331,12 @@ function isPlainRecord(value: unknown): value is Record { * shown as the reference lanes show it: a reference as its stub, an invalid * one (such as a uri that isn't a string, or MCP's embedded resource, whose * uri isn't at its top level) as the marker saying why it isn't shown, and - * inline data that also claims a uri or a disposition as withheld. So the - * only reference's uri shown is a link's, in its line, where the host can't - * fetch the link and its uri can't carry a credential (shownLinkUri). Inline - * data that claims neither is itself, as is any other value: a block of - * another type keeps its fields, a url among them, since RFC-005 governs - * references. + * inline data that also claims a uri or a disposition as withheld. So of + * the uris RFC-005 governs, the only one shown is a link's, in its line, + * where the host can't fetch the link and its uri can't carry a credential + * (shownLinkUri). Inline data that claims neither is itself, as is any other + * value: a block of another type keeps its fields, a url among them, since + * RFC-005 governs references. */ function shownInHistory(value: unknown, claims?: ResultClaims): unknown { return walkedInHistory(value, (b) => referenceStubOrNull(b, 'from tool result', undefined, { unfetchedLinkUri: true, claims })); @@ -369,10 +369,10 @@ function walkedInHistory(value: unknown, asText: (b: Record) => * through toJSON, the fields of inline data and every depth it shows: so * every reference whose stub the result shows counts, and nothing it doesn't * show does. A value serialization can't write, such as a cycle, stops the - * walk where it would stop the result. It's a walk of its own before the one - * that shows the data, so a value's toJSON and getters run in both: one that - * answers differently the second time can claim what the shown text - * doesn't show. + * walk where it would stop the result's serialization. It's a walk of its + * own before the one that shows the data, so a value's toJSON and getters + * run in both: one that answers differently the second time can claim what + * the shown text doesn't show. */ function claimsInHistory(data: unknown): ResultClaims { const testimonies: ReferenceTestimony[] = []; @@ -396,7 +396,10 @@ function claimsInHistory(data: unknown): ResultClaims { } } catch { // Showing the result meets the same failure at the same place, for a - // value that answers the same each time it's read. + // value that answers the same each time it's read. A withheld result + // shows only the fields it keeps, but the guard serialized the whole + // result for its audit before that (ToolResultGuard.archive), so such a + // value stopped it there. } return resultClaims(testimonies); } diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 8e66dcbc4..813bfff88 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -74,13 +74,13 @@ const PLAIN_URI = 'https://zz-files.example/report.csv'; const PLAIN_LINK = { ...LINK, uri: PLAIN_URI }; /** What PLAIN_LINK shows in a tool result where the host can't fetch it, as where no server delivered it. */ const PLAIN_LINK_TEXT = `[resource link "report.csv": ${PLAIN_URI}, text/csv; not fetched]`; -/** Record a server dialed on the links' origin, as the framework records a server it configures. Each - * test that needs it asks, so the file loads against a registry that keeps no addresses. */ /** A content array's claims, by the module's own contentClaims, read off the * module's namespace so the file still loads against main's src, which * lacks it, for red-first. */ const contentClaimsOf = (blocks: unknown[]) => (referencesModule as unknown as { contentClaims(blocks: unknown[]): referencesModule.ResultClaims }).contentClaims(blocks); +/** Record a server dialed on the links' origin, as the framework records a server it configures. Each + * test that needs it asks, so the file loads against a registry that keeps no addresses. */ const servingFiles = () => referenceRegistry.setServerAddress('zz-files', { url: 'wss://zz-files.example' }, TEST_HOST); /** The framework these tests record servers' addresses for, as one does. */ const TEST_HOST = {}; @@ -979,15 +979,24 @@ describe('one answer for every block of a result', () => { assert.ok(deep.then.ref.endsWith('— the reference is expired'), deep.then.ref); }); - it("words competing expiries in a result by one fixed order, whatever the order of its blocks", () => { + it("words competing expiries in a result by EXPIRY_REASON's order, whatever the order of its blocks", () => { servingFiles(); - const uri = 'https://zz-files.example/competing.csv'; - const passed = { type: 'resource', uri, name: 'competing.csv', expiresAt: '2001-01-01T00:00:00Z' }; - const unreadable = { type: 'resource', uri, name: 'competing.csv', expiresAt: 'zz-not-a-time' }; - referenceRegistry.register(testimonyOf(passed), 'zz-files'); - for (const order of [[passed, unreadable], [unreadable, passed]]) { - const lines = toolResultDataToHistoryString(order).split('\n'); - assert.ok(lines.every((line) => line.endsWith(`— the reference is ${EXPIRY_REASON.passed}`)), lines.join('\n')); + const permutations = (items: T[]): T[][] => items.length <= 1 ? [items] + : items.flatMap((item, i) => permutations([...items.slice(0, i), ...items.slice(i + 1)]).map((rest) => [item, ...rest])); + const cases: Array<[string, Array]> = [ + ['competing-all.csv', ['passed', 'unparseable', 'rejected']], + ['competing-unread.csv', ['unparseable', 'rejected']], + ]; + for (const [name, kinds] of cases) { + const uri = `https://zz-files.example/${name}`; + const expiresAt = { passed: '2001-01-01T00:00:00Z', unparseable: 'zz-not-a-time', rejected: 'x'.repeat(REFERENCE_LIMITS.expiresAt + 1) }; + const blocks = kinds.map((kind) => ({ type: 'resource', uri, name, expiresAt: expiresAt[kind] })); + referenceRegistry.register(testimonyOf(blocks[0]), 'zz-files'); + // The first of EXPIRY_REASON's cases the blocks give. + for (const order of permutations(blocks)) { + const lines = toolResultDataToHistoryString(order).split('\n'); + assert.ok(lines.every((line) => line.endsWith(`— the reference is ${EXPIRY_REASON[kinds[0]!]}`)), lines.join('\n')); + } } }); @@ -1006,6 +1015,27 @@ describe('one answer for every block of a result', () => { assert.equal(after, before); }); + it("lets a later result's plain reference to a uri lift an earlier result's never, whatever the order of its link and reference", () => { + const orders: Array<[string, (link: unknown, plain: unknown) => unknown[]]> = [ + ['link-first', (link, plain) => [link, plain]], + ['reference-first', (link, plain) => [plain, link]], + ]; + for (const [order, blocks] of orders) { + const uri = `https://zz-elsewhere.example/lifted-${order}.csv`; + // An earlier result marks the uri never. + toolResultDataToHistoryString([{ type: 'resource', uri, name: 'lifted.csv', disposition: 'never' }]); + // The later result's reference is its server's latest word on the + // uri, so the link shows it, as if no never had been given. + const lines = toolResultDataToHistoryString(blocks( + { type: 'resource_link', uri, name: 'lifted.csv' }, { type: 'resource', uri, name: 'lifted.csv' })).split('\n'); + assert.ok(lines.includes(`[resource link "lifted.csv": ${uri}; not fetched]`), `${order}: ${lines.join('\n')}`); + } + // A result naming the uri only in a link leaves the earlier never in force. + const uri = 'https://zz-elsewhere.example/lifted-link-only.csv'; + toolResultDataToHistoryString([{ type: 'resource', uri, name: 'kept.csv', disposition: 'never' }]); + assert.match(toolResultDataToHistoryString([{ type: 'resource_link', uri, name: 'kept.csv' }]), /^\[ref_[^\]]+\] kept\.csv — from tool result — /); + }); + it("finds a never inside inline data's other fields, which showing it descends into, before any block is shown", () => { const uri = 'https://zz-elsewhere.example/meta-never.csv'; const data = { @@ -1083,60 +1113,70 @@ describe('one answer for every block of a result', () => { } }); - it("isn't fetched for a script on demand where a reference at depth in the module's data gives its uri an expiry, as the text reads it", async () => { + /** A module's result for a script, its data `data`: what the demand fetch + * tried, what it logged, and what the script got. */ + const demandFetch = async (data: unknown[], dataIsContent?: boolean) => { const fw = Object.create(AgentFramework.prototype) as { dispatchScriptToolCall: (...args: unknown[]) => Promise; getReferenceFetcher(): unknown; handleScriptToolCall(agentName: string, toolName: string, args: Record): Promise; }; - const uri = 'https://zz-files.example/demand-deep-expired.pdf'; - // Data, not content: its second element isn't a block, so its text reads - // the result's claims at depth, where the expiry is. - const data = [ - { type: 'resource', uri, name: 'deep.pdf', expiresAt: '2099-01-01T00:00:00Z' }, - { nested: { type: 'resource', uri, name: 'deep.pdf', expiresAt: '2001-01-01T00:00:00Z' } }, - ]; - fw.dispatchScriptToolCall = async () => ({ success: true, data }); + fw.dispatchScriptToolCall = async () => ({ success: true, data, ...(dataIsContent ? { dataIsContent } : {}) }); const fetched: string[] = []; fw.getReferenceFetcher = () => ({ fetch: async (record: { testimony: { uri: string } }) => { fetched.push(record.testimony.uri); return { ok: false, error: 'zz-refused' }; } }); + const logged: string[] = []; const error = console.error; - console.error = () => {}; - let result: ScriptToolResult; + console.error = (line: string) => { logged.push(line); }; try { - result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + const result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); + return { fetched, logged, result }; } finally { console.error = error; } + }; + const NOT_MADE_EXPIRED = /script-demand fetch of ref_\S+ not made: by the result's claims, the reference is expired/; + + it("isn't fetched for a script on demand where a reference at depth in the module's data gives its uri an expiry, reading the claims as the text does", async () => { + // Data, not content: its second element isn't a block, so its text reads + // the result's claims at depth. + const at = (uri: string, nestedExpiry: string) => [ + { type: 'resource', uri, name: 'deep.pdf', expiresAt: '2099-01-01T00:00:00Z' }, + { nested: { type: 'resource', uri, name: 'deep.pdf', expiresAt: nestedExpiry } }, + ]; + // Where the nested reference is fresh too, the top one is fetched. + const control = 'https://zz-files.example/demand-deep-fresh.pdf'; + const fresh = await demandFetch(at(control, '2099-06-01T00:00:00Z')); + assert.equal(fresh.result.isError, false); + assert.deepEqual(fresh.fetched, [control]); + // Where it has expired, it isn't, for the expiry only a reading at depth sees. + const uri = 'https://zz-files.example/demand-deep-expired.pdf'; + const expired = await demandFetch(at(uri, '2001-01-01T00:00:00Z')); + assert.equal(expired.result.isError, false); + assert.deepEqual(expired.fetched, []); + assert.ok(expired.logged.some((line) => NOT_MADE_EXPIRED.test(line)), expired.logged.join('\n')); + }); + + it("is fetched for a script on demand where only a content array's unshown block names its uri expired, as the text reads it", async () => { + // Content from an MCPL call a module forwards: its elements are blocks, + // and one the lane doesn't know is a marker, its fields unshown. + const uri = 'https://zz-files.example/demand-content-flag.pdf'; + const { fetched, result } = await demandFetch([ + { type: 'resource', uri, name: 'flag.pdf', expiresAt: '2099-01-01T00:00:00Z' }, + { type: 'zz-unknown', inner: { type: 'resource', uri, name: 'flag.pdf', expiresAt: '2001-01-01T00:00:00Z' } }, + ], true); assert.equal(result.isError, false); - assert.deepEqual(fetched, []); - assert.ok(result.text.includes('— the reference is expired'), result.text); + assert.deepEqual(fetched, [uri]); }); it("isn't fetched for a script on demand where any reference in the module's array gives its uri an expiry that counts", async () => { - const fw = Object.create(AgentFramework.prototype) as { - dispatchScriptToolCall: (...args: unknown[]) => Promise; - getReferenceFetcher(): unknown; - handleScriptToolCall(agentName: string, toolName: string, args: Record): Promise; - }; const uri = 'https://zz-files.example/demand-expired.pdf'; const fresh = { type: 'resource', uri, name: 'demand.pdf', expiresAt: '2099-01-01T00:00:00Z' }; const expired = { type: 'resource', uri, name: 'demand.pdf', expiresAt: '2001-01-01T00:00:00Z' }; - fw.dispatchScriptToolCall = async () => ({ success: true, data: [fresh, expired], dataIsContent: true }); - const fetched: string[] = []; - fw.getReferenceFetcher = () => ({ fetch: async (record: { testimony: { uri: string } }) => { fetched.push(record.testimony.uri); return { ok: false, error: 'zz-refused' }; } }); - const logged: string[] = []; - const error = console.error; - console.error = (line: string) => { logged.push(line); }; - let result: ScriptToolResult; - try { - result = await fw.handleScriptToolCall('zz-agent', 'zz_tool', {}); - } finally { - console.error = error; - } + const { fetched, logged, result } = await demandFetch([fresh, expired], true); assert.equal(result.isError, false); assert.deepEqual(fetched, []); // The fetch not made is logged, as a refused one is. - assert.ok(logged.some((line) => /script-demand fetch of ref_\S+ not made: by the result's claims, the reference is expired/.test(line)), logged.join('\n')); + assert.ok(logged.some((line) => NOT_MADE_EXPIRED.test(line)), logged.join('\n')); }); it('shows a link with no usable uri at any depth as its marker, never its fields', () => { From 125db579cc2ad81734b74a1ec804de5702a248bb Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sun, 18 Oct 2026 23:59:52 -0400 Subject: [PATCH 16/40] fix(tool-results): a withheld result keeps nothing where a value stops its claims walk From the round review af-f53-sol-r3 (its I5), which I'd judged not real by reading; Desmond-2579's probe showed it real. A withheld result's kept fields are shown with the whole result's claims, read by a walk of their own. The guard's audit serializes the whole result first, but JSON reads only own enumerable fields and toJSON, while the walk reads a value's `type` as a plain property. So an inherited getter that throws passed the audit and stopped the walk, whose catch kept the claims read so far, and a kept link showed a uri a later reference marked never. - claimsInHistory is null when a value stops its walk, and so is resultClaimsOf: what the rest of the data claims is unknown. - keptWhenWithheld keeps nothing then, as for a failed result. - Showing the whole result reads no claims then (NO_CLAIMS), since it stops at the same value, and the module demand fetch reads the records. Tests: a withheld result whose data holds a value with an inherited getter that throws, which the result's JSON doesn't read, keeps nothing. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 4 +++- src/tool-result-guard.ts | 15 ++++++++++----- src/tool-result-history.ts | 34 +++++++++++++++++++-------------- test/tool-result-blocks.test.ts | 14 ++++++++++++++ 4 files changed, 47 insertions(+), 20 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index cc26932cb..233cc3c9c 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -19225,7 +19225,9 @@ export class AgentFramework { // A uri any of the result's references gives an expiry that counts // isn't fetched, as autoFetchReferences passes over one that any block // of an MCPL result marks expired. - claims = resultClaimsOf(result.data, result.dataIsContent); + // Where a value stops the walk, the records decide: the script gets + // the same stop when its text is written. + claims = resultClaimsOf(result.data, result.dataIsContent) ?? undefined; for (const block of result.data) { const c = classifyBlock(block); if (c.kind !== 'reference' || c.testimony.fromLink) continue; diff --git a/src/tool-result-guard.ts b/src/tool-result-guard.ts index 89bce6809..fd6444d44 100644 --- a/src/tool-result-guard.ts +++ b/src/tool-result-guard.ts @@ -89,9 +89,10 @@ export function withheldResultNotice(spill: WithheldSpill, auditDurable: boolean * the order marked, serialized as a stored result is. Past the inline cap * they're cut, with a note that says so and where the whole result is, since * a later field can then be cut away entirely. Undefined when it keeps - * nothing, as for a failed result (agent-framework #277). A write's file - * list is evidence of what it did, rarely what drew a refusal, and keeping it - * is what shows a clobbered file when the receipt is withheld. + * nothing, as for a failed result (agent-framework #277), or one whose claims + * can't be read whole (resultClaimsOf). A write's file list is evidence of + * what it did, rarely what drew a refusal, and keeping it is what shows a + * clobbered file when the receipt is withheld. */ export function keptWhenWithheld(result: CompletedToolCall['result'], maxChars: number | undefined): string | undefined { const { data, keepWhenWithheld: keys } = result; @@ -105,8 +106,12 @@ export function keptWhenWithheld(result: CompletedToolCall['result'], maxChars: // text: picked out of it, a kept uri would lose the type that makes it a // reference, and be written out of the stub that hides it. // The kept fields are shown as the whole result claims (resultClaimsOf), so - // a never elsewhere in it still withholds a kept link's uri. - const whole = rfc005TextOf(fields) ?? toolResultDataToHistoryString(kept, undefined, { claims: resultClaimsOf(fields) }); + // a never elsewhere in it still withholds a kept link's uri. Where a value + // stops that walk, such as a getter that throws, which the audit's JSON + // doesn't read, what the rest claims is unknown, and nothing is kept. + const claims = resultClaimsOf(fields); + if (claims === null) return undefined; + const whole = rfc005TextOf(fields) ?? toolResultDataToHistoryString(kept, undefined, { claims }); if (!maxChars || whole.length <= maxChars) return whole; // Never half a surrogate pair: the stub goes to the provider as text, so // the cut can be one character short of the cap. diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index d186d96f8..0d621ed73 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -121,9 +121,10 @@ function isContentArray(data: unknown, dataIsContent?: boolean): data is unknown * them: a content array's from its elements (contentClaims), and any other * data's by the walk serialization takes (claimsInHistory). For a decision * made about the result outside showing it, such as which of its references - * to fetch, or for showing part of it as the rest of it claims. + * to fetch, or for showing part of it as the rest of it claims. Null where a + * value stops the walk, so what the data claims is unknown (claimsInHistory). */ -export function resultClaimsOf(data: unknown, dataIsContent?: boolean): ResultClaims { +export function resultClaimsOf(data: unknown, dataIsContent?: boolean): ResultClaims | null { return isContentArray(data, dataIsContent) ? contentClaims(data) : claimsInHistory(data); } @@ -368,13 +369,14 @@ function walkedInHistory(value: unknown, asText: (b: Record) => * (ResultClaims), read by the walk serialization takes (walkedInHistory), * through toJSON, the fields of inline data and every depth it shows: so * every reference whose stub the result shows counts, and nothing it doesn't - * show does. A value serialization can't write, such as a cycle, stops the - * walk where it would stop the result's serialization. It's a walk of its - * own before the one that shows the data, so a value's toJSON and getters - * run in both: one that answers differently the second time can claim what - * the shown text doesn't show. + * show does. A value the walk can't read or write, such as a cycle or a + * getter that throws, stops it, and then it's null, since what the rest of + * the data claims is unknown. It's a walk of its own before the one that + * shows the data, so a value's toJSON and getters run in both: one that + * answers differently the second time can claim what the shown text doesn't + * show. */ -function claimsInHistory(data: unknown): ResultClaims { +function claimsInHistory(data: unknown): ResultClaims | null { const testimonies: ReferenceTestimony[] = []; const read = (b: Record): string | null => { const c = classifyBlock(b); @@ -395,11 +397,11 @@ function claimsInHistory(data: unknown): ResultClaims { }); } } catch { - // Showing the result meets the same failure at the same place, for a - // value that answers the same each time it's read. A withheld result - // shows only the fields it keeps, but the guard serialized the whole - // result for its audit before that (ToolResultGuard.archive), so such a - // value stopped it there. + // Showing the whole result stops at the same value, for a value that + // answers the same each time it's read. A caller that shows part of it, + // such as a withheld result's kept fields, can't know what the rest + // claims. + return null; } return resultClaims(testimonies); } @@ -415,13 +417,17 @@ export function rfc005TextOf(value: unknown): string | null { return shown !== value && typeof shown === 'string' ? shown : null; } +/** The claims of data that claims nothing. */ +const NO_CLAIMS: ResultClaims = new Map(); + /** * JSON of a tool result's data with each value in it shown as history shows * it (shownInHistory), at any depth. Data that is itself shown as text, such * as a bare reference, is that text. */ function historyJson(data: unknown, given?: ResultClaims): string | undefined { - const claims = given ?? claimsInHistory(data); + // A walk that stopped claims nothing: showing stops at the same value. + const claims = given ?? claimsInHistory(data) ?? NO_CLAIMS; const root = shownInHistory(data, claims); if (root !== data && typeof root === 'string') return root; return JSON.stringify(data, (_key, value: unknown) => shownInHistory(value, claims)) as string | undefined; diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 813bfff88..be1dd4086 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1099,6 +1099,20 @@ describe('one answer for every block of a result', () => { assert.ok(kept !== undefined && /^\{"files":\["\[ref_[^\]]+\] kept\.csv — from tool result — /.test(kept), kept); }); + it("keeps nothing of a withheld result whose claims a value stops the walk of, since what the rest claims is unknown", () => { + const uri = 'https://zz-elsewhere.example/kept-unread.csv'; + // A getter on the prototype, which the audit's JSON doesn't read, and + // the claims walk does. + class Status { get type(): string { throw new Error('zz-unreadable'); } } + const result = { + success: true, + data: { status: new Status(), withheld: { type: 'resource', uri, name: 'kept.csv', disposition: 'never' }, files: [{ type: 'resource_link', uri, name: 'kept.csv' }] }, + keepWhenWithheld: ['files'], + }; + assert.doesNotThrow(() => JSON.stringify(result)); + assert.equal(keptWhenWithheld(result, undefined), undefined); + }); + it("reads a result's data once without a stored copy, and sends what that one reading made", () => { // Natively, beside an image, and as text, with none. for (const [data, expected] of [ From d5b60e49a5beeb9c874b99076e4d192ec0c8d17b Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 02:59:09 -0400 Subject: [PATCH 17/40] chore: drop contentClaims from framework.ts's imports, which the merge left unused The MCPL script lane reads the view's claims (scriptViewClaims) since the merge, so framework.ts no longer calls contentClaims. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/framework.ts b/src/framework.ts index a2c8f54ad..c96e9cef6 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,7 +1,7 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { constants as bufferConstants } from 'node:buffer'; import { dirname, join } from 'node:path'; -import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, contentClaims, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ResultClaims, type ServerAddress } from './mcpl/references.js'; +import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ResultClaims, type ServerAddress } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, From e247dd9413e4d99016e53f29f6b73f6b347cc758 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 03:07:42 -0400 Subject: [PATCH 18/40] fix(references): a script's view fails closed where its claims walk stops; tests that separate the view's claims Where the walk of a script view's claims stopped, the treatment still ran, with no claims, on the ground that its own walk would stop at the same value. It needn't: the two walks carry different replacers, so at a stack's edge their depths can differ by a frame, and a module value whose toJSON or getter answers differently on a second read stops one and not the other. With no claims a link's uri falls back to the record's latest testimony, its own, so a never in the view didn't withhold it. The call was to fail closed: - scriptViewClaims throws what stops its walk (claimsInHistoryOrThrow; claimsInHistory still answers null for history's own callers). - The MCPL route reads the view's claims before any treatment: a RangeError takes the unconvertible path (SCRIPT_RESULT_UNCONVERTIBLE), anything else is a fault, which fails the dispatch, as for the round trip. - The module route reads them before the demand fetch, and a stop is the script's failure (failed), the fetch not made. Texts: governedByRfc005's link line says a link naming no uri is data on every lane, as on main, its fields holding no uri RFC-005 governs; the classifier's size line says the object reads the same again where sanitizeLabel left its uri as it was, and that a peer's sizeBytes labels its link on every lane. Tests: a value that stops only the claims walk (a type that throws a RangeError once) fails the view on both routes; a nested never decides over a plain reference at the top, for a link at the top or nested, on both routes, which no reading of the top-level blocks alone, or no claims, gives; a peer's rejected block's fields claim nothing; a result's native live copy survives its turn being stored by shares (variants that survived a run). Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 59 ++++++++++++++------- src/mcpl/references.ts | 4 +- src/tool-result-history.ts | 59 ++++++++++++--------- test/script-structured-references.test.ts | 64 +++++++++++++++++++++++ test/tool-result-blocks.test.ts | 18 +++++++ 5 files changed, 159 insertions(+), 45 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index c96e9cef6..70cfa5d50 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -22556,14 +22556,33 @@ export class AgentFramework { // isn't a list, the structured view carrying the result. A failure with // no blocks reads as its error text. const showsBlocks = structured && (!isError || (Array.isArray(result.data) && result.data.length > 0)); + const failed = (err: unknown): ToolAnswer => { + const error = unconvertibleScriptResult(toolName, err, isError ? result.error : undefined); + return { + ok: true, + json: JSON.stringify({ content: [{ type: 'text', text: error }], structuredContent: null, isError: true }), + isError: true, + brief: error, + }; + }; // The claims a module's result is shown by, read once, so the fetches // below and what the script gets read the same: a structured result's // view's (scriptViewClaims), nested fields included, or the model's text's - // (resultClaimsOf). Where a value stops the walk, the records decide: the - // script gets the same stop when its view is written. + // (resultClaimsOf). A view whose claims walk stops fails closed here, + // before any fetch or treatment, so the treatment never runs without + // them (Ada-1017, room-203 #200250); for the text, where a value stops + // the walk, the records decide, as for the model's own text. let claims: ResultClaims | undefined; if ((!isError || structured) && Array.isArray(result.data) && fromModule) { - claims = (showsBlocks ? scriptViewClaims(result.data, false) : resultClaimsOf(result.data, result.dataIsContent)) ?? undefined; + if (showsBlocks) { + try { + claims = scriptViewClaims(result.data, false); + } catch (err) { + return failed(err); + } + } else { + claims = resultClaimsOf(result.data, result.dataIsContent) ?? undefined; + } const fetcher = this.getReferenceFetcher(); for (const block of result.data) { const c = classifyBlock(block); @@ -22585,15 +22604,6 @@ export class AgentFramework { if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); } } - const failed = (err: unknown): ToolAnswer => { - const error = unconvertibleScriptResult(toolName, err, isError ? result.error : undefined); - return { - ok: true, - json: JSON.stringify({ content: [{ type: 'text', text: error }], structuredContent: null, isError: true }), - isError: true, - brief: error, - }; - }; // An MCPL peer's result whose blocks JSON couldn't hold, so that RFC-005's // treatment couldn't be given them at dispatch. if (SCRIPT_RESULT_UNCONVERTIBLE.has(result)) return failed(SCRIPT_RESULT_UNCONVERTIBLE.get(result)); @@ -29043,18 +29053,27 @@ export class AgentFramework { // dropped, since they are untreated and the result travels on, to the // waiter and to the listeners of its event's trace. Its structured // content is kept, as every result keeps it, since RFC-005's - // treatment doesn't reach it. A walk of the claims that stops, as at - // such a depth or at a fault, claims nothing here: the treatment - // below meets the same value and answers for it. - const claims = scriptViewClaims(direct.data, true) ?? undefined; + // treatment doesn't reach it. The view's claims are read first, by + // a walk of their own: where it stops, the treatment doesn't run + // (Ada-1017, room-203 #200250), a RangeError meaning a result it + // can't convert, as below, and anything else a fault. + const unconvertible = (err: unknown): ToolResult => { + if (!(err instanceof RangeError)) throw err; + const dropped: ToolResult = { ...direct, data: [] }; + SCRIPT_RESULT_UNCONVERTIBLE.set(dropped, err); + return dropped; + }; + let claims: ResultClaims; + try { + claims = scriptViewClaims(direct.data, true); + } catch (err) { + return unconvertible(err); + } const blocks = direct.data.map((block: unknown) => referenceBlockForScript(block, serverId, claims)); try { return { ...direct, data: JSON.parse(JSON.stringify(blocks, (_key, value: unknown) => shownToScriptFromServer(value, serverId, claims))) as unknown[] }; } catch (err) { - if (!(err instanceof RangeError)) throw err; - const unconvertible: ToolResult = { ...direct, data: [] }; - SCRIPT_RESULT_UNCONVERTIBLE.set(unconvertible, err); - return unconvertible; + return unconvertible(err); } } // A standard peer's content that isn't a list of blocks is a malformed diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index c2bedc113..b679e4058 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -116,7 +116,9 @@ export function classifyBlock(block: unknown): BlockClassification { // that claims none. Its title, description and annotations aren't // testimony. // A link's size is MCP's `size`; the script lane's object for a link - // names it `sizeBytes`, so given again it reads the same. + // names it `sizeBytes`, so given again it reads the same, where + // sanitizeLabel left its uri as it was. Read on every lane, so a + // peer's link that names a `sizeBytes` is labelled with it too. return linkTestimony(parseReferenceFields({ uri: block.uri, name: block.name, mimeType: block.mimeType, sizeBytes: block.size ?? block.sizeBytes })); default: return { kind: 'invalid', shown: unknownBlockText(block) }; diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 37078ae1b..17a2f03f4 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -343,10 +343,12 @@ function governedByRfc005(b: Record): boolean { if (b.type === 'resource') return hasUri(b.uri) || (isPlainRecord(b.resource) && hasUri(b.resource.uri)); // A link that names a uri is governed whatever its value, so one without // a usable uri (empty, null, not a string) is its marker here, as on the - // content lanes, never its fields. One that names none carries no uri to - // withhold, as the object the script lane makes of a link whose uri it - // leaves out (referenceBlockForScript), which a script's view, given it - // again, leaves as it is, as it leaves a reference's stub. + // content lanes, never its fields. One that names none is data, on every + // lane, as on main: its fields hold no uri RFC-005 governs, though a value + // in them, such as an untyped object's `uri`, is data as anywhere else. + // That's the object the script lane makes of a link whose uri it leaves + // out (referenceBlockForScript), which a script's view, given it again, + // leaves as it is, as it leaves a reference's stub. if (b.type === 'resource_link') return b.uri !== undefined; if (b.type === 'image' || b.type === 'audio') return typeof b.data === 'string' || hasUri(b.uri); return false; @@ -412,6 +414,19 @@ function walkedInHistory(value: unknown, asText: (b: Record) => * show. */ function claimsInHistory(data: unknown): ResultClaims | null { + try { + return claimsInHistoryOrThrow(data); + } catch { + // Showing the whole result stops at the same value, for a value that + // answers the same each time it's read. A caller that shows part of it, + // such as a withheld result's kept fields, can't know what the rest + // claims. + return null; + } +} + +/** claimsInHistory's walk, which throws what stops it. */ +function claimsInHistoryOrThrow(data: unknown): ResultClaims { const testimonies: ReferenceTestimony[] = []; const read = (b: Record): string | null => { const c = classifyBlock(b); @@ -421,22 +436,14 @@ function claimsInHistory(data: unknown): ResultClaims | null { // decides it first. return c.kind === 'text' ? null : ''; }; - try { - const root = walkedInHistory(data, read); - // Its text is thrown away, so a string is written as 0, which keeps a - // long one from being copied for nothing; the walk descends as before. - if (root === data || typeof root !== 'string') { - JSON.stringify(data, (_key, value: unknown) => { - const walked = walkedInHistory(value, read); - return typeof walked === 'string' ? 0 : walked; - }); - } - } catch { - // Showing the whole result stops at the same value, for a value that - // answers the same each time it's read. A caller that shows part of it, - // such as a withheld result's kept fields, can't know what the rest - // claims. - return null; + const root = walkedInHistory(data, read); + // Its text is thrown away, so a string is written as 0, which keeps a + // long one from being copied for nothing; the walk descends as before. + if (root === data || typeof root !== 'string') { + JSON.stringify(data, (_key, value: unknown) => { + const walked = walkedInHistory(value, read); + return typeof walked === 'string' ? 0 : walked; + }); } return resultClaims(testimonies); } @@ -473,11 +480,15 @@ export function shownToScript(value: unknown, claims?: ResultClaims): unknown { * withholds that uri from every block in the view. Stricter than the * model's line, which shows a block and not its fields (contentClaims), and * never looser. An MCPL peer's top-level block the classifier rejects is - * its notice, its fields unshown, so they don't count (`fromServer`). Null - * where a value stops the walk, which stops the view at the same value. + * its notice, its fields unshown, so they don't count (`fromServer`). It + * throws what stops the walk, such as a value too deep to convert or a + * getter that throws, and the script's view then isn't written: the + * treatment never runs without the claims it reads (Ada-1017, room-203 + * #200250), whether or not its own walk, with other replacers and so other + * stack depths, would stop at the same value. */ -export function scriptViewClaims(content: readonly unknown[], fromServer: boolean): ResultClaims | null { - return claimsInHistory(fromServer ? content.filter((block) => classifyBlock(block).kind !== 'invalid') : content); +export function scriptViewClaims(content: readonly unknown[], fromServer: boolean): ResultClaims { + return claimsInHistoryOrThrow(fromServer ? content.filter((block) => classifyBlock(block).kind !== 'invalid') : content); } /** diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index a1e540952..031799e68 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -637,6 +637,70 @@ test("a RangeError from serializing an MCPL peer's blocks, as a result past the }); }); +test("a nested never decides over a plain reference to its uri at the top, for a link at the top or nested, from a peer or a module", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + // The top-level blocks alone claim nothing that withholds the uri, so + // only a reading of the claims the whole view shows (scriptViewClaims) + // keeps it from the link. Each link comes before the references, so a + // registry that read them as it went wouldn't have them yet. + // A uri of its own for each route, so neither finds the other's + // references in the registry. + const content = (route: string, where: string) => { + const uri = `https://elsewhere.example.invalid/strictest-${route}-${where}/plain.csv`; + const link = { type: 'resource_link', uri, name: 'plain.csv' }; + return [ + where === 'top' ? link : { type: 'text', text: 'a nested link', link }, + { type: 'resource', uri, name: 'plain.csv' }, + { type: 'text', text: 'sealed', attachment: { type: 'resource', uri, name: 'plain.csv', disposition: 'never' } }, + ]; + }; + for (const where of ['top', 'nested']) { + answer({ content: content('peer', where), structuredContent: {} }); + const fromPeer = await script('mcpl--files--attach'); + const fromModule = await exit({ success: true, data: content('module', where), structured: {} }, 'fwd--whole', false); + for (const out of [fromPeer, fromModule]) assert.ok(!out.includes('elsewhere.example.invalid'), `${where}: ${out}`); + } + }); +}); + +test("a peer's top-level block the classifier rejects shows none of its fields, so a never among them claims nothing in a script's view", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script }) => { + const uri = 'https://elsewhere.example.invalid/rejected-fields/plain.csv'; + const content = [ + { type: 'video', uri: 'https://elsewhere.example.invalid/clip.mp4', sealed: { type: 'resource', uri, disposition: 'never' } }, + { type: 'resource', uri, name: 'plain.csv' }, + { type: 'resource_link', uri, name: 'plain.csv' }, + ]; + answer({ content, structuredContent: {} }); + const object = JSON.parse(await script('mcpl--files--attach')) as { content: Array> }; + assert.equal(object.content[0]!.type, 'text', 'the video is its notice'); + assert.equal(object.content[2]!.uri, uri, 'the view shows no never for the uri, so the link carries it, as the model\'s line shows it'); + }); +}); + +test("where the walk of a script view's claims stops, the view isn't written, though the treatment's own walk would pass", async () => { + await withPeers(async ({ answer, script, exit }) => { + // A value that stops the claims walk and not the treatment's: its type + // throws a RangeError the first time it's read, as a stack can run out + // one frame sooner in one walk than in the other, and then reads as data. + const once = () => { + let read = false; + return { get type(): string { if (!read) { read = true; throw new RangeError('Maximum call stack size exceeded'); } return 'note'; } }; + }; + const uri = 'https://elsewhere.example.invalid/stopped-walk/plain.csv'; + const content = () => [{ type: 'text', text: 'one link', deep: once(), attachment: { type: 'resource', uri, disposition: 'never' } }, { type: 'resource_link', uri, name: 'plain.csv' }]; + answer({ content: content(), structuredContent: {} }); + const fromPeer = JSON.parse(await script('mcpl--files--attach')) as { content: [{ text: string }]; isError: boolean }; + assert.equal(fromPeer.isError, true); + assert.match(fromPeer.content[0].text, /^tool 'mcpl--files--attach' returned a result that can't be converted to JSON/); + // A module's result, as the exit receives it. + const fromModule = JSON.parse(await exit({ success: true, data: content(), structured: {} }, 'fwd--whole', false)) as { content: [{ text: string }]; isError: boolean }; + assert.equal(fromModule.isError, true); + assert.match(fromModule.content[0].text, /^tool 'fwd--whole' returned a result that can't be converted to JSON/); + for (const out of [fromPeer, fromModule]) assert.ok(!JSON.stringify(out).includes('elsewhere.example.invalid'), JSON.stringify(out)); + }); +}); + test("a fault in an MCPL peer's round trip other than a RangeError fails the dispatch, as any fault does, with its own message", async () => { await withPeers(async ({ answer, script, observe }) => { const seen = observe(); diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 339df2acd..9651d3be5 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1392,6 +1392,24 @@ describe('one answer for every block of a result', () => { } }); + it("keeps a result's native live copy when its turn's results are stored by their shares of the readers' bound", async () => { + const fw = Object.create(AgentFramework.prototype) as { + buildStoredToolResultContent(agent: string, calls: unknown[], maxChars: number | undefined, bound?: number): Promise<{ spilled: Map }>; + }; + Object.assign(fw, { + toolImageLedgers: new Map(), toolRoundsByShare: new WeakSet(), + writeToolResultFile: async () => null, moduleRegistry: { getAllModules: () => [] }, + }); + // Two results that pass a bound of 600 together, so each is stored by + // its share; the first holds an image, which goes natively at the cap. + const withImage = { id: 'zz-share-image', name: 'zz_tool', input: {}, result: { success: true, data: [IMAGE, { type: 'text', text: 'i'.repeat(400) }], dataIsContent: true } }; + const plain = { id: 'zz-share-text', name: 'zz_tool', input: {}, result: { success: true, data: 't'.repeat(400) } }; + const { spilled } = await fw.buildStoredToolResultContent('zz-agent', [withImage, plain], 24_000, 600); + const stored = spilled.get(withImage.id) as { text: string; live: Array<{ kind: string }> | null }; + assert.ok(stored.text.length < 500, `stored by its share: ${stored.text.length}`); + assert.deepEqual(stored.live?.map((block) => block.kind), ['image', 'text']); + }); + it("goes natively from the stored pass with links and markers exactly at the cap, and as its stored text one past it", async () => { const fw = Object.create(AgentFramework.prototype) as { buildStoredToolResultContent(agent: string, calls: unknown[], maxChars: number | undefined): Promise<{ spilled: Map }>; From b56606eb5ffa25e8684f4ae3e11fe8d6c6cbe37b Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 03:15:27 -0400 Subject: [PATCH 19/40] fix(tool-results): data whose claims walk stops shows a notice of its own on the text lane The premise ruled out for a script's view was still in #527's text lane: historyJson read `claimsInHistory(data) ?? NO_CLAIMS`, so where the claims walk stopped and the render's didn't, a link showed a uri a reference in the result withholds. A throw there, traced: nothing catches it before the event loop, so the round's results would go unstored and unsent with the agent at ready. The call was a notice: with no claims given and none readable, historyJson returns UNCHECKED_RESULT_TEXT, a notice of its own, saying the tool ran and its result couldn't be read through to check it for references, so it's withheld. The round completes; the model and a script's text read the notice; both copies carry it alike. main's own render-throw path stays as it is, the dispatch's conversion check keeping such values from it. The changed fragment says what a caller meets. Test: data with a link beside a never to its uri and a value that stops only the claims walk shows the notice, and a round with it stores and sends the notice alike. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 1 + src/tool-result-history.ts | 25 +++++++++++++++++------ test/tool-result-blocks.test.ts | 24 +++++++++++++++++++++- 3 files changed, 43 insertions(+), 7 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index d04c7de0e..c607b3d4d 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,3 +1,4 @@ - In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. - In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. +- A tool result's data whose references can't be read through to apply RFC-005's rules, such as data deeper than one walk of it can go where the next would get past, shows `[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]`, to the model and to a script reading its text, where it was shown with none of its references' claims. A script's structured view of such a result is one the script can't convert, from an MCPL peer, or a failure naming the tool, from a module. diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 17a2f03f4..70ad3407c 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -519,17 +519,30 @@ export function rfc005TextOf(value: unknown): string | null { return shown !== value && typeof shown === 'string' ? shown : null; } -/** The claims of data that claims nothing. */ -const NO_CLAIMS: ResultClaims = new Map(); +/** + * What a tool result's data shows as where the walk of its claims stops + * (claimsInHistory), so RFC-005's rules can't be read through it: the tool + * ran, and its result isn't shown, rather than shown with no claims, where a + * link could show a uri a reference in the result withholds. The round + * completes, as a withheld result's kept fields keep nothing then + * (keptWhenWithheld). Ada-1017, room-203 #200344. + */ +export const UNCHECKED_RESULT_TEXT = + "[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]"; /** * JSON of a tool result's data with each value in it shown as history shows - * it (shownInHistory), at any depth. Data that is itself shown as text, such - * as a bare reference, is that text. + * it (shownInHistory), at any depth, by the claims `given` or else its own + * (claimsInHistory). Data that is itself shown as text, such as a bare + * reference, is that text, and data whose claims can't be read is + * UNCHECKED_RESULT_TEXT. */ function historyJson(data: unknown, given?: ResultClaims): string | undefined { - // A walk that stopped claims nothing: showing stops at the same value. - const claims = given ?? claimsInHistory(data) ?? NO_CLAIMS; + // A walk that stopped can't say what the data claims, and showing it with + // no claims would let a link show a uri a reference withholds, should the + // render's own walk, with other replacers, get past the same value. + const claims = given ?? claimsInHistory(data); + if (claims === null) return UNCHECKED_RESULT_TEXT; const root = shownInHistory(data, claims); if (root !== data && typeof root === 'string') return root; return JSON.stringify(data, (_key, value: unknown) => shownInHistory(value, claims)) as string | undefined; diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 9651d3be5..94cea2852 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -47,7 +47,7 @@ import { CoalesceError, validateCoalescedContent } from '../src/mcpl/push-coales import { ReferenceFetcher } from '../src/mcpl/reference-fetcher.js'; import { checkedToolResult } from '../src/module-registry.js'; import { parseImagePlaceholders } from '../src/tool-image-ledger.js'; -import { nonErrorResultText, shownToScript, toolResultDataToHistoryString } from '../src/tool-result-history.js'; +import { UNCHECKED_RESULT_TEXT, nonErrorResultText, shownToScript, toolResultDataToHistoryString } from '../src/tool-result-history.js'; import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; import { fixture } from './helpers/coalescing-fixture.js'; @@ -1392,6 +1392,28 @@ describe('one answer for every block of a result', () => { } }); + it('shows data whose claims walk stops as the unchecked notice, never with no claims, and a round with it completes, both copies alike', async () => { + // A value that stops the claims walk and not the render's: its type + // throws a RangeError the first time it's read, as a stack can run out + // one frame sooner in one walk than in the other, and then reads as data. + const once = () => { + let read = false; + return { get type(): string { if (!read) { read = true; throw new RangeError('Maximum call stack size exceeded'); } return 'note'; } }; + }; + // A link beside a never to its uri, which a render with no claims would show. + const data = () => ({ link: { type: 'resource_link', uri: PLAIN_URI, name: 'report.csv' }, sealed: { type: 'resource', uri: PLAIN_URI, disposition: 'never' }, deep: once() }); + assert.equal(toolResultDataToHistoryString(data()), UNCHECKED_RESULT_TEXT); + const fw = Object.create(AgentFramework.prototype) as { + buildStoredToolResultContent(agent: string, calls: unknown[], maxChars: number | undefined): Promise<{ blocks: Array<{ content: unknown }>; spilled: Map }>; + toMembraneToolResult(callId: string, result: ToolResult, maxChars: number | undefined, precomputed: unknown): Live; + }; + Object.assign(fw, { toolImageLedgers: new Map(), writeToolResultFile: async () => null, moduleRegistry: { getAllModules: () => [] } }); + const call = { id: 'zz-unchecked', name: 'zz_tool', input: {}, result: { success: true, data: data() } }; + const { blocks, spilled } = await fw.buildStoredToolResultContent('zz-agent', [call], 24_000); + assert.deepEqual(blocks.map((block) => block.content), [UNCHECKED_RESULT_TEXT]); + assert.equal(fw.toMembraneToolResult(call.id, call.result, 24_000, spilled.get(call.id)).content, UNCHECKED_RESULT_TEXT); + }); + it("keeps a result's native live copy when its turn's results are stored by their shares of the readers' bound", async () => { const fw = Object.create(AgentFramework.prototype) as { buildStoredToolResultContent(agent: string, calls: unknown[], maxChars: number | undefined, bound?: number): Promise<{ spilled: Map }>; From 85b73917e607214ebd0deb3ef3f13de44deaa172 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 03:18:54 -0400 Subject: [PATCH 20/40] fix(references): a script's text route fails where the claims walk stops; texts say what the notice replaces Calls on b56606eb: - A script's text route reads the result's claims before it calls toolResultDataToHistoryString (resultClaimsOrThrow, resultClaimsOf's throwing form) and answers a stop with failed(...) naming the tool, as its structured view does, so `isError` says nothing could be shown on both routes. The read covers whatever the text route renders: a module's array or object, and a peer's data that isn't a list. The model's lane keeps UNCHECKED_RESULT_TEXT, since it has no failure answer that doesn't wedge the round. - b56606eb's message said main's render-throw path stays as it is. It doesn't, by run: the claims walk meets data no JSON can hold, a cycle or a BigInt, or data too deep, before the render does, so the model's lane shows the notice for them where its serializer threw, the direct Agent API's unchecked results included. The notice's doc and the changed fragment say so. Test: a module's object and a module's list that isn't content, through the text route at a claims-only stop, each a failure naming the tool. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- src/framework.ts | 31 +++++++++++++---------- src/tool-result-history.ts | 13 +++++++++- test/script-structured-references.test.ts | 17 +++++++++++++ 4 files changed, 47 insertions(+), 16 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index c607b3d4d..a603ca4de 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,4 +1,4 @@ - In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. - In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. -- A tool result's data whose references can't be read through to apply RFC-005's rules, such as data deeper than one walk of it can go where the next would get past, shows `[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]`, to the model and to a script reading its text, where it was shown with none of its references' claims. A script's structured view of such a result is one the script can't convert, from an MCPL peer, or a failure naming the tool, from a module. +- A tool result's data whose references can't be read through to apply RFC-005's rules shows `[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]` on the model's lane, where it was shown with none of its references' claims, or, for data no JSON can hold, such as a cycle or a BigInt, or data too deep to write, the stored and sent copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. diff --git a/src/framework.ts b/src/framework.ts index 70cfa5d50..0b91dbeb7 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -118,6 +118,7 @@ import { goesNative, nonErrorResultText, resultClaimsOf, + resultClaimsOrThrow, scriptViewClaims, shownToScript, shownToScriptFromServer, @@ -22568,20 +22569,17 @@ export class AgentFramework { // The claims a module's result is shown by, read once, so the fetches // below and what the script gets read the same: a structured result's // view's (scriptViewClaims), nested fields included, or the model's text's - // (resultClaimsOf). A view whose claims walk stops fails closed here, - // before any fetch or treatment, so the treatment never runs without - // them (Ada-1017, room-203 #200250); for the text, where a value stops - // the walk, the records decide, as for the model's own text. + // (resultClaimsOf). Where the walk stops, the script's answer is a + // failure naming the tool, before any fetch or treatment, so neither runs + // without them (Ada-1017, room-203 #200250, #200374): on both of a + // script's routes, `is_error` says nothing could be shown, where the + // model's lane shows UNCHECKED_RESULT_TEXT. let claims: ResultClaims | undefined; if ((!isError || structured) && Array.isArray(result.data) && fromModule) { - if (showsBlocks) { - try { - claims = scriptViewClaims(result.data, false); - } catch (err) { - return failed(err); - } - } else { - claims = resultClaimsOf(result.data, result.dataIsContent) ?? undefined; + try { + claims = showsBlocks ? scriptViewClaims(result.data, false) : resultClaimsOrThrow(result.data, result.dataIsContent); + } catch (err) { + return failed(err); } const fetcher = this.getReferenceFetcher(); for (const block of result.data) { @@ -22627,13 +22625,18 @@ export class AgentFramework { } } else { // As in history, a reference is its stub at any depth and inline data - // claiming a uri or a disposition is withheld (RFC-005). + // claiming a uri or a disposition is withheld (RFC-005), by the + // result's claims, read here where no fetch read them, a stop in their + // walk failing as a stopped render does. try { text = isError ? result.error ?? 'tool call failed' : result.data === undefined || result.data === null ? nonErrorResultText(result) - : toolResultDataToHistoryString(result.data, undefined, { dataIsContent: Boolean(result.dataIsContent), claims }); + : toolResultDataToHistoryString(result.data, undefined, { + dataIsContent: Boolean(result.dataIsContent), + claims: claims ?? resultClaimsOrThrow(result.data, result.dataIsContent), + }); } catch (err) { return failed(err); } diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 70ad3407c..26722bd8f 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -136,6 +136,13 @@ export function resultClaimsOf(data: unknown, dataIsContent?: boolean): ResultCl return isContentArray(data, dataIsContent) ? contentClaims(data) : claimsInHistory(data); } +/** resultClaimsOf, throwing what stops the walk, for a caller that answers + * a stop itself: a script's text route, which fails then, as its view does + * (Ada-1017, room-203 #200374). */ +export function resultClaimsOrThrow(data: unknown, dataIsContent?: boolean): ResultClaims { + return isContentArray(data, dataIsContent) ? contentClaims(data) : claimsInHistoryOrThrow(data); +} + /** One element of a content array, as toolResultBlocks gives it. Only an * image with data and a media type is shown as itself, as the push lane's * converter shows one, and only text and images can be: this host's tool @@ -525,7 +532,11 @@ export function rfc005TextOf(value: unknown): string | null { * ran, and its result isn't shown, rather than shown with no claims, where a * link could show a uri a reference in the result withholds. The round * completes, as a withheld result's kept fields keep nothing then - * (keptWhenWithheld). Ada-1017, room-203 #200344. + * (keptWhenWithheld). The claims walk meets data no JSON can hold, such as + * a cycle or a BigInt, or data too deep to write, before the render does, + * so such data shows this too, where the render threw (Desmond-2579's run). + * A script never gets it: both of its routes read the claims first and + * fail naming the tool (Ada-1017, room-203 #200344, #200374). */ export const UNCHECKED_RESULT_TEXT = "[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]"; diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index 031799e68..14cb97901 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -701,6 +701,23 @@ test("where the walk of a script view's claims stops, the view isn't written, th }); }); +test("where the walk of a result's claims stops, a script's text route fails naming the tool, as its structured view does, for an object or a list", async () => { + await withPeers(async ({ exit }) => { + // A type that throws a RangeError the first time it's read stops the + // claims walk only: the render's walk would read it as data. + const once = () => { + let read = false; + return { get type(): string { if (!read) { read = true; throw new RangeError('Maximum call stack size exceeded'); } return 'note'; } }; + }; + // No structured content, so the text route; data that isn't content. + for (const data of [{ deep: once() }, [{ deep: once() }]]) { + const out = JSON.parse(await exit({ success: true, data }, 'fwd--whole', false)) as { content: [{ text: string }]; isError: boolean }; + assert.equal(out.isError, true, JSON.stringify(out)); + assert.match(out.content[0].text, /^tool 'fwd--whole' returned a result that can't be converted to JSON, so the script can't receive it/); + } + }); +}); + test("a fault in an MCPL peer's round trip other than a RangeError fails the dispatch, as any fault does, with its own message", async () => { await withPeers(async ({ answer, script, observe }) => { const seen = observe(); From 91d67094a199ecd6e739129af188a7bb252d0278 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 03:25:11 -0400 Subject: [PATCH 21/40] test(tool-results): a cycle shows as the unchecked notice, and registers nothing past it cddaa580's test pinned that toolResultDataToHistoryString throws on a cycle and registers nothing past it. Since b56606eb the claims walk meets the cycle first and the result is UNCHECKED_RESULT_TEXT, where main's serializer threw. The test is turned round to that, and still checks nothing past the cycle is registered. A run at 85b73917 failed on it, the only failure of 7,142. Co-Authored-By: Claude Opus 5.5 --- test/tool-result-blocks.test.ts | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 94cea2852..178d2de62 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1085,11 +1085,14 @@ describe('one answer for every block of a result', () => { assert.equal(reads, 0); }); - it("registers nothing a result's serialization never reaches, such as what lies past a cycle", () => { + it("registers nothing a result's serialization never reaches, such as what lies past a cycle, which shows as the unchecked notice", () => { const uri = 'https://zz-elsewhere.example/past-cycle.csv'; const loop: Record = {}; loop.self = loop; - assert.throws(() => toolResultDataToHistoryString({ loop, later: { type: 'resource', uri, name: 'past.csv', disposition: 'never' } })); + // The claims walk meets the cycle before the render does, so the result + // is the notice where main's serializer threw (Ada-1017, room-203 + // #200374), and nothing past the cycle is registered either way. + assert.equal(toolResultDataToHistoryString({ loop, later: { type: 'resource', uri, name: 'past.csv', disposition: 'never' } }), UNCHECKED_RESULT_TEXT); assert.equal(referenceRegistry.findByUri(uri), undefined); }); From 50c2c20cf5d6c6a74e726855c594a1372d5277e8 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 05:06:48 -0400 Subject: [PATCH 22/40] fix(references, tool-results): review fixes: a link's uri by the line's claims, one peer claims walk, the model lane's catch, an inline never The calls on the review's findings: - A script's link object shows its uri only where the model's line for the result shows it, by the claims the line reads (lineClaims: resultClaimsOrThrow, the blocks' own for a content array), and never where the view's claims say never. Reading more claims could loosen: a nested expiry made the link one the host can't fetch, whose uri a link shows, so the script alone saw it. Its expired/unfetchable words still read the view's claims. - An MCPL peer's script-demand fetch reads the view's claims (scriptViewClaims) where it used each block's own expiry, so a nested expiry stops a fetch, and a view whose claims walk stops fetches nothing, its references only registered. The claims are read once, at dispatch, and the script lane's reading of the same content array takes them, or what stopped their walk, from SCRIPT_VIEW_CLAIMS, so the fetch and the view read the same claims. - nonErrorResultText, the model lane's entry, answers anything thrown while reading or showing a result with UNCHECKED_RESULT_TEXT, with a log line naming the tool (options.toolName, which the callers pass) and the error; the blocks reach onBlocks only once the text is made, so a result shown as the notice has no live copy of its own. A script's routes call the serializer directly and keep failing. - The changelog's direct-Agent-API sentence says "with the tool-result guard off"; with it on, the guard's archive still throws, as on main. - A contradictory inline block's disposition never counts for its uri in ResultClaims (inlineNeverUri), in a content array, at depth and in a script's view; its other claims don't, since they could loosen. Also, from the findings: - The peer route's nested pass leaves the top-level blocks to the treatment above it, so a link's object isn't read again as a link to the uri sanitizeLabel cut, which registered a second record without the result's claims. - Framework.ts no longer imports resultClaimsOf; the comment names resultClaimsOrThrow. - Texts: the module route's comment says structured content goes as sent, as on main; claimsInHistory's doc says a value answering differently can also show a reference the claims never read, and only the direct Agent API's results can; a link naming no uri is data at depth and the marker as a content block, in the code and the fixed fragment. Tests: a nested expiry doesn't show a fetchable link's uri to a script, whose line shows none; a peer's demand fetch skips a uri a nested reference calls expired, and fetches nothing for a view it can't read; a top-level link keeps its raw uri's record; a treatment that fails after the claims walk is a result the script can't convert (peer) or a failure naming the tool (module); the nested-never test checks the view was written with the link object; an inline never withholds a link's uri in a content array, at depth and in a script's view, and an inline expiry counts for nothing; the model's lane shows the notice and logs for a block whose text can't be read and for a render that stops after the walk, where the serializer throws; a link's size from size, else sizeBytes, a size of 0 kept. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 5 +- changelog.d/tool-result-blocks.fixed.md | 2 +- src/agent.ts | 2 +- src/framework.ts | 83 ++++++++++++++--- src/mcpl/references.ts | 37 +++++++- src/tool-result-history.ts | 103 ++++++++++++++++----- test/script-structured-references.test.ts | 106 +++++++++++++++++++++- test/tool-result-blocks.test.ts | 66 +++++++++++++- 8 files changed, 358 insertions(+), 46 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index a603ca4de..dfe83ea12 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,4 +1,5 @@ - In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. -- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. +- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads, so a script never sees a uri the model's line withholds. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. +- A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. -- A tool result's data whose references can't be read through to apply RFC-005's rules shows `[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]` on the model's lane, where it was shown with none of its references' claims, or, for data no JSON can hold, such as a cycle or a BigInt, or data too deep to write, the stored and sent copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. +- A tool result's data that can't be read through to apply RFC-005's rules, or shown once it has been, shows `[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]` on the model's lane, in the stored and sent copies alike, with a log line naming the tool and the error. It was shown with none of its references' claims where the walk reading them stopped and the render's didn't, or, for data no JSON can hold, such as a cycle or a BigInt, a getter that throws, or data too deep to write, the copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception, with the tool-result guard off; with it on, its archive of the result still throws, as before. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md index 7ebeb247f..f6522adb6 100644 --- a/changelog.d/tool-result-blocks.fixed.md +++ b/changelog.d/tool-result-blocks.fixed.md @@ -1,5 +1,5 @@ - A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's that doesn't set `dataIsContent`, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. -- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link with no usable uri is the marker that says so. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. +- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link whose uri is empty, null or not a string is the marker that says so. A link that names no uri at all is the marker as a block of a content array, and at depth stays data, as before, since its fields hold no uri this reads. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. - A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it ends with why `fetch_reference` would refuse, without what calling again can do. The stub reads an expiry from the references in its result where they name its uri, and the fetch from the reference as the host last recorded it, so where one result names a uri both expired and fresh, the stub says it's expired while a fetch can succeed. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, one whose address doesn't parse, a uri off its server's origin, one with a user name or password, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. - A label in a stub (a name, a type, a server's id, a provenance or a saved path) loses any line or paragraph separator (U+2028, U+2029), as it loses control characters. - A script's demand fetch of a reference not yet fetched reads that reference's own digest and expiry, where it read whatever an earlier reference to the same uri had claimed. diff --git a/src/agent.ts b/src/agent.ts index c939288b8..db257c9a4 100644 --- a/src/agent.ts +++ b/src/agent.ts @@ -2096,7 +2096,7 @@ export class Agent { toolUseId: r.id, content: isError ? truncateForHistory(typeof error === 'string' ? error.toWellFormed() : error as string, DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS) - : nonErrorResultText(r.result, DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS), + : nonErrorResultText(r.result, DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS, { toolName: r.name }), isError, }; }); diff --git a/src/framework.ts b/src/framework.ts index 0b91dbeb7..6a5a4c7ae 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -117,7 +117,6 @@ import type { ToolResultGuard, WithheldSpill } from './tool-result-guard.js'; import { goesNative, nonErrorResultText, - resultClaimsOf, resultClaimsOrThrow, scriptViewClaims, shownToScript, @@ -1118,6 +1117,15 @@ const SCRIPT_CALL_NO_RESULT = new WeakSet(); */ const SCRIPT_RESULT_UNCONVERTIBLE = new WeakMap(); +/** + * The claims of an MCPL peer's blocks as a script's view reads them + * (scriptViewClaims), or what stopped their walk, read once at dispatch, by + * the content array the result carries: its demand fetch reads them, and the + * script lane's reading of the same blocks takes them from here, so the two + * read the same claims (Ada-1017, room-203 #201013). + */ +const SCRIPT_VIEW_CLAIMS = new WeakMap(); + /** * One logical turn's routing state and receipt tallies * (AgentFramework.turnRouting): everything the turn has decided about where @@ -6261,8 +6269,11 @@ export class AgentFramework { * script receiving a result is the strongest available signal that the * bytes are wanted (scripts process, they do not browse), so the stub it * receives carries a workspace path it can open directly. - * A uri the result calls expired is fetched in no mode. */ - private async autoFetchReferences(content: unknown, serverId: string, mode: 'register' | 'model' | 'script'): Promise { + * A uri the result calls expired is fetched in no mode: by its blocks, or + * by `claims` where given, those of everything a script's view of it + * shows (scriptViewClaims), nested fields included, as the module + * route's demand fetch reads them (Ada-1017, room-203 #201013). */ + private async autoFetchReferences(content: unknown, serverId: string, mode: 'register' | 'model' | 'script', claims?: ResultClaims): Promise { if (!Array.isArray(content)) return; // Register every block first, then judge each uri once, by its latest // record. The registry keys records by uri and evicts the one registered @@ -6291,7 +6302,11 @@ export class AgentFramework { } if (mode === 'register') return; const fetcher = this.getReferenceFetcher(); - const live = [...byUri.values()].filter(({ expiredHere, linkOnly }) => !expiredHere && !linkOnly).map(({ record }) => record); + const nowMs = Date.now(); + const live = [...byUri.values()] + .filter(({ record, expiredHere, linkOnly }) => !expiredHere && !linkOnly + && (claims === undefined || Boolean(record.fetchedPath) || claimedExpiry(record, claims, nowMs) === null)) + .map(({ record }) => record); // Parallel with a count cap: N small fetches cost one round-trip of // wall-clock, and anything past the cap stays lazy (stub says so). const MAX_EAGER_PER_RESULT = 8; @@ -12359,7 +12374,7 @@ export class AgentFramework { this.pendingAssistantBlocks.delete(agent.name); } const membraneResults = currentState.toolResults.map(tc => - this.toMembraneToolResult(tc.id, tc.result, maxChars, spilled.get(tc.id)) + this.toMembraneToolResult(tc.id, tc.result, maxChars, spilled.get(tc.id), tc.name) ); agent.toolResultGuard.storeResults(toolResultContent, membraneResults, currentState.toolResults, turnRowMetadata, keptCap); @@ -20175,7 +20190,7 @@ export class AgentFramework { bound, ); agent.toolResultGuard.storeResults(toolResultContent, readyState.toolResults.map((tc) => - this.toMembraneToolResult(tc.id, tc.result, cap, spilled.get(tc.id))), readyState.toolResults, + this.toMembraneToolResult(tc.id, tc.result, cap, spilled.get(tc.id), tc.name)), readyState.toolResults, this.silentTurnRowMetadata(agent.name), keptCap); // The response is already complete: this batch is never // submitted in this turn, so it cannot be refused. Admit it. @@ -22188,6 +22203,7 @@ export class AgentFramework { let blocks = null as HistoryBlockSpan[] | null; let classified = null as ToolResultBlock[] | null; const text = nonErrorResultText(tc.result, undefined, { + toolName: tc.name, imageRef: (blockIndex, image) => ledger.retain({ toolCallId: tc.id, toolName: tc.name, @@ -22569,15 +22585,20 @@ export class AgentFramework { // The claims a module's result is shown by, read once, so the fetches // below and what the script gets read the same: a structured result's // view's (scriptViewClaims), nested fields included, or the model's text's - // (resultClaimsOf). Where the walk stops, the script's answer is a + // (resultClaimsOrThrow, which throws where resultClaimsOf answers null). + // Where the walk stops, the script's answer is a // failure naming the tool, before any fetch or treatment, so neither runs // without them (Ada-1017, room-203 #200250, #200374): on both of a // script's routes, `is_error` says nothing could be shown, where the // model's lane shows UNCHECKED_RESULT_TEXT. let claims: ResultClaims | undefined; + // The claims the model's line for the result reads, which decide where a + // link's object shows its uri (referenceBlockForScript's lineClaims). + let lineClaims: ResultClaims | undefined; if ((!isError || structured) && Array.isArray(result.data) && fromModule) { try { - claims = showsBlocks ? scriptViewClaims(result.data, false) : resultClaimsOrThrow(result.data, result.dataIsContent); + lineClaims = resultClaimsOrThrow(result.data, result.dataIsContent); + claims = showsBlocks ? scriptViewClaims(result.data, false) : lineClaims; } catch (err) { return failed(err); } @@ -22613,12 +22634,13 @@ export class AgentFramework { // and returns whole, are read as history reads a module's data: what // RFC-005 governs, at any depth, as the script lane shows it, and // anything else as sent. What a script prints comes back to the model, - // so no uri the model's lanes withhold reaches it. What this gives is + // so no uri the model's lanes withhold reaches it through the blocks + // (structured content goes as sent, as on main). What this gives is // a stub, a notice or the value itself, so a module that returns // blocks it already gave has them left as they are. if (fromModule) { try { - content = JSON.parse(JSON.stringify(content, (_key, value: unknown) => shownToScript(value, claims))) as unknown[]; + content = JSON.parse(JSON.stringify(content, (_key, value: unknown) => shownToScript(value, claims, lineClaims))) as unknown[]; } catch (err) { return failed(err); } @@ -22788,6 +22810,8 @@ export class AgentFramework { * copy is built from these, never classified again, so the two copies * agree on every block and on whether it goes natively (goesNative). */ precomputed?: StoredToolResult, + /** The tool's name, for the log line of a result shown as the notice. */ + toolName?: string, ): MembraneToolResult { if (afResult.isError) { // Same spill policy as successes: reuse the history copy so a giant @@ -22819,7 +22843,7 @@ export class AgentFramework { // (goesNative), the very blocks that text was made from go natively. let spans = null as HistoryBlockSpan[] | null; let classified = null as ToolResultBlock[] | null; - const text = nonErrorResultText(afResult, maxChars, { onBlocks: (s, b) => { spans = s; classified = b; } }); + const text = nonErrorResultText(afResult, maxChars, { toolName, onBlocks: (s, b) => { spans = s; classified = b; } }); if (spans && classified && goesNative((spans as HistoryBlockSpan[]).map(({ kind, start, end }) => ({ kind, length: end - start })), maxChars)) { return { toolUseId: callId, content: this.nativeContent(classified, maxChars, null), isError: false }; } @@ -29067,14 +29091,28 @@ export class AgentFramework { return dropped; }; let claims: ResultClaims; + // The model's line for a peer's content reads its blocks' claims + // alone, which decide where a link's object shows its uri. + let lineClaims: ResultClaims; try { - claims = scriptViewClaims(direct.data, true); + // As the dispatch read them for its demand fetch, where it did. + const read = SCRIPT_VIEW_CLAIMS.get(direct.data); + if (read && 'error' in read) throw read.error; + claims = read ? read.claims : scriptViewClaims(direct.data, true); + lineClaims = resultClaimsOrThrow(direct.data, true); } catch (err) { return unconvertible(err); } - const blocks = direct.data.map((block: unknown) => referenceBlockForScript(block, serverId, claims)); + const blocks = direct.data.map((block: unknown) => referenceBlockForScript(block, serverId, claims, lineClaims)); try { - return { ...direct, data: JSON.parse(JSON.stringify(blocks, (_key, value: unknown) => shownToScriptFromServer(value, serverId, claims))) as unknown[] }; + // The top-level blocks were treated just above; the pass treats + // what's nested in them. Reading a treated block again would take + // a link object's uri, as sanitizeLabel cut it, for a new link, + // registered apart from the one the result named. + const nested = function (this: unknown, _key: string, value: unknown): unknown { + return this === blocks ? value : shownToScriptFromServer(value, serverId, claims, lineClaims); + }; + return { ...direct, data: JSON.parse(JSON.stringify(blocks, nested)) as unknown[] }; } catch (err) { return unconvertible(err); } @@ -29162,7 +29200,22 @@ export class AgentFramework { if (mcplPeer) { const scriptReads = forScript && (!result.isError || Object.prototype.hasOwnProperty.call(result, 'structuredContent')); - await this.autoFetchReferences(result.content, serverId, scriptReads ? 'script' : 'model') + // A script's demand fetch reads the claims of everything its view + // shows, as the module route's does; where that walk stops, the view + // isn't written (reading()), so nothing is fetched for it, the + // references only registered (Ada-1017, room-203 #201013). + let mode: 'register' | 'model' | 'script' = scriptReads ? 'script' : 'model'; + let claims: ResultClaims | undefined; + if (scriptReads && Array.isArray(result.content)) { + try { + claims = scriptViewClaims(result.content, true); + SCRIPT_VIEW_CLAIMS.set(result.content, { claims }); + } catch (error) { + SCRIPT_VIEW_CLAIMS.set(result.content, { error }); + mode = 'register'; + } + } + await this.autoFetchReferences(result.content, serverId, mode, claims) .catch((e) => console.error('[rfc005] autofetch error:', (e as Error).message)); } return { result, toolResult: await reading(result, mcplPeer) }; diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index b679e4058..524078caa 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -741,8 +741,17 @@ export interface UriClaims { /** A result's claims (ResultClaims), from the testimonies of its references, * each read once, as referenceExpiry reads an expiry. */ -export function resultClaims(testimonies: Iterable): ResultClaims { +export function resultClaims(testimonies: Iterable, nevers: Iterable = []): ResultClaims { const claims = new Map(); + // A uri inline data marks `disposition: never` beside its data, the + // classifier's contradiction (inlineNeverUri): withheld unconditionally + // (RFC-005 §5), though the block is no reference. Only its never counts, + // since its other claims could loosen (Ada-1017, room-203 #201013). + for (const uri of nevers) { + const c = claims.get(uri); + if (c) c.never = true; + else claims.set(uri, { never: true, rejected: false, unparseable: false, earliestMs: null }); + } for (const t of testimonies) { if (t.fromLink) continue; let c = claims.get(t.uri); @@ -764,11 +773,24 @@ export function resultClaims(testimonies: Iterable): ResultC * `_meta`, aren't shown, so a reference inside them doesn't count. */ export function contentClaims(blocks: readonly unknown[]): ResultClaims { const testimonies: ReferenceTestimony[] = []; + const nevers: string[] = []; for (const block of blocks) { const c = classifyBlock(block); if (c.kind === 'reference') testimonies.push(c.testimony); + const never = inlineNeverUri(block, c); + if (never !== null) nevers.push(never); } - return resultClaims(testimonies); + return resultClaims(testimonies, nevers); +} + +/** The uri inline data marks `disposition: never` beside its data, the + * classifier's contradiction, which counts for that uri's claims + * (resultClaims); null for any other block. A uri that isn't a non-empty + * string names nothing a claim is kept for. */ +export function inlineNeverUri(block: unknown, c: BlockClassification = classifyBlock(block)): string | null { + if (c.kind !== 'inline' || !c.contradiction) return null; + const b = block as { uri?: unknown; disposition?: unknown }; + return b.disposition === 'never' && typeof b.uri === 'string' && b.uri !== '' ? b.uri : null; } /** The claims a decision about a record reads: its result's for its uri, @@ -943,7 +965,7 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI * empty, the withheld notice, each as a text block. * A text block, or inline data that claims nothing, is returned as sent. */ -export function referenceBlockForScript(block: unknown, serverId?: string, claims?: ResultClaims): unknown { +export function referenceBlockForScript(block: unknown, serverId?: string, claims?: ResultClaims, lineClaims?: ResultClaims): unknown { const c = classifyBlock(block); switch (c.kind) { case 'text': return block; @@ -967,7 +989,14 @@ export function referenceBlockForScript(block: unknown, serverId?: string, claim // One clock reading for the whole object, so its words can't disagree // with each other across an expiry. const nowMs = Date.now(); - const uri = shownLinkUri(t, record, nowMs, claims); + // A link's uri where the model's line for the result shows it, by the + // claims the line reads (lineClaims, where they differ from the view's), + // and never where the view's claims withhold it: reading more claims + // can loosen as well as tighten, since an expiry makes a link one the + // host can't fetch, whose uri the line shows (Ada-1017, room-203 + // #201013). A script never sees a uri the model's line withholds. + const lineShows = shownLinkUri(t, record, nowMs, lineClaims ?? claims); + const uri = lineShows !== null && lineClaims !== undefined && claimedNever(record, claims) ? null : lineShows; // Why a fetch_reference with this object's refId would be refused, in // the words a stub gives, whether or not the model's line for it is a // stub: the line has no id to fetch with, and the object does. As the diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 26722bd8f..f23b29fa5 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -31,6 +31,7 @@ import { buildReferenceStub, classifyBlock, contentClaims, + inlineNeverUri, referenceBlockForScript, referenceStubOrNull, referenceTypeLabel, @@ -217,6 +218,11 @@ export interface HistorySerializeOptions { * (resultClaimsOf), such as the fields a withheld result keeps. */ claims?: ResultClaims; + /** + * Whose result this is, as nonErrorResultText's log line names it where it + * shows UNCHECKED_RESULT_TEXT for a result it couldn't read or show. + */ + toolName?: string; } /** @@ -268,6 +274,9 @@ export function toolResultDataToHistoryString( * * Each field is read once: the direct Agent API passes its caller's own * result here, unchecked, and a getter needn't answer the same way twice. + * Whatever stops reading or showing the data is UNCHECKED_RESULT_TEXT, with + * a log line naming the tool (`options.toolName`) and the error: the model's + * lane has no failure answer that doesn't wedge the round. * Whether an array is content is the result's own `dataIsContent`, read for * its truth as a result's flags are, whatever `options` says. * @@ -292,7 +301,38 @@ export function nonErrorResultText( return maxChars ? truncateForHistory(text, maxChars) : text; } } - return toolResultDataToHistoryString(data, maxChars, { ...options, dataIsContent: Boolean(result.dataIsContent) }); + // The blocks go to the caller only once the text is made from them, so a + // result that shows the notice has no live copy of its own: the notice is + // both copies' text. + let blocks: [HistoryBlockSpan[], ToolResultBlock[]] | undefined; + const onBlocks = options?.onBlocks; + try { + const text = toolResultDataToHistoryString(data, maxChars, { + ...options, + dataIsContent: Boolean(result.dataIsContent), + ...(onBlocks ? { onBlocks: (spans: HistoryBlockSpan[], made: ToolResultBlock[]) => { blocks = [spans, made]; } } : {}), + }); + if (onBlocks && blocks) onBlocks(blocks[0], blocks[1]); + return text; + } catch (error) { + // The model's lane has no failure answer that doesn't wedge the round, so + // whatever stops reading or showing a result, such as a getter that + // throws in a content array's block or a render deeper than the claims + // walk got (a script's routes call the serializer directly, and fail), + // is the notice, with a line naming the tool and the error, since it can + // also be a fault of this host's (Ada-1017, room-203 #201013). + console.error(`[tool-results] ${options?.toolName ? `tool '${options.toolName}'` : 'a tool'}: its result couldn't be read or shown, so the model reads that it was withheld: ${thrownLine(error)}`); + return maxChars ? truncateForHistory(UNCHECKED_RESULT_TEXT, maxChars) : UNCHECKED_RESULT_TEXT; + } +} + +/** The first line of what was thrown, for a log line. */ +function thrownLine(error: unknown): string { + try { + return (error instanceof Error ? error.message : String(error)).split('\n')[0] || 'no reason given'; + } catch { + return 'a value that can\'t be read'; + } } /** Bounded copy of an arbitrary string with the standard truncation notice, @@ -344,15 +384,19 @@ function tryHistoryStringFromContentArray(data: unknown, options?: HistorySerial * isn't a string isn't inline data, so a record that only shares one of * those type names is data. A uri of any other value, string or not, counts, * so a credential fails closed. MCP's `resource_link` (RFC-005 §8.1) is - * governed whatever its uri: one without a usable uri is its marker. + * governed whatever value its uri has, so one whose uri isn't usable is its + * marker; one that names no uri is data here, where as a block of a content + * array, which the classifier reads, it's the marker. */ function governedByRfc005(b: Record): boolean { if (b.type === 'resource') return hasUri(b.uri) || (isPlainRecord(b.resource) && hasUri(b.resource.uri)); // A link that names a uri is governed whatever its value, so one without // a usable uri (empty, null, not a string) is its marker here, as on the - // content lanes, never its fields. One that names none is data, on every - // lane, as on main: its fields hold no uri RFC-005 governs, though a value - // in them, such as an untyped object's `uri`, is data as anywhere else. + // content lanes, never its fields. One that names none is data at depth, + // on the model's lanes and a script's, as main has every link at depth: + // its fields hold no uri RFC-005 governs, though a value in them, such as + // an untyped object's `uri`, is data as anywhere else. As a block of a + // content array, the classifier reads it, and there it's the marker. // That's the object the script lane makes of a link whose uri it leaves // out (referenceBlockForScript), which a script's view, given it again, // leaves as it is, as it leaves a reference's stub. @@ -418,16 +462,17 @@ function walkedInHistory(value: unknown, asText: (b: Record) => * the data claims is unknown. It's a walk of its own before the one that * shows the data, so a value's toJSON and getters run in both: one that * answers differently the second time can claim what the shown text doesn't - * show. + * show, or show a reference the claims never read. Only data no check has + * read through JSON can do either: a module's result is parsed JSON from its + * dispatch, and an MCPL peer's blocks are parsed wire JSON, so this is the + * direct Agent API's, whose results a host gives unchecked. */ function claimsInHistory(data: unknown): ResultClaims | null { try { return claimsInHistoryOrThrow(data); } catch { - // Showing the whole result stops at the same value, for a value that - // answers the same each time it's read. A caller that shows part of it, - // such as a withheld result's kept fields, can't know what the rest - // claims. + // What the rest of the data claims is unknown: historyJson shows it as + // UNCHECKED_RESULT_TEXT, and a withheld result's kept fields keep nothing. return null; } } @@ -435,6 +480,14 @@ function claimsInHistory(data: unknown): ResultClaims | null { /** claimsInHistory's walk, which throws what stops it. */ function claimsInHistoryOrThrow(data: unknown): ResultClaims { const testimonies: ReferenceTestimony[] = []; + // Inline data's never, which walkedInHistory withholds before it asks + // `read` (inlineNeverUri). + const nevers: string[] = []; + const noteInline = (value: unknown): void => { + if (!isPlainRecord(value) || (value.type !== 'image' && value.type !== 'audio')) return; + const never = inlineNeverUri(value); + if (never !== null) nevers.push(never); + }; const read = (b: Record): string | null => { const c = classifyBlock(b); if (c.kind === 'reference') testimonies.push(c.testimony); @@ -443,16 +496,18 @@ function claimsInHistoryOrThrow(data: unknown): ResultClaims { // decides it first. return c.kind === 'text' ? null : ''; }; + noteInline(data); const root = walkedInHistory(data, read); // Its text is thrown away, so a string is written as 0, which keeps a // long one from being copied for nothing; the walk descends as before. if (root === data || typeof root !== 'string') { JSON.stringify(data, (_key, value: unknown) => { + if (value !== data) noteInline(value); const walked = walkedInHistory(value, read); return typeof walked === 'string' ? 0 : walked; }); } - return resultClaims(testimonies); + return resultClaims(testimonies, nevers); } /** @@ -465,18 +520,19 @@ function claimsInHistoryOrThrow(data: unknown): ResultClaims { * withheld notice, each as a text block. A null, absent or empty uri claims * none (hasUri), so inline data beside one is itself. Any other value is * itself. `claims` are those of everything the script's view shows - * (scriptViewClaims). What it gives is either a stub or a notice, which + * (scriptViewClaims), and `lineClaims` those the model's line for the + * result reads, which decide where a link's uri is shown. What it gives is either a stub or a notice, which * RFC-005 doesn't govern, or the value itself, which it gives again, so * applying it twice gives what applying it once gave. */ -export function shownToScript(value: unknown, claims?: ResultClaims): unknown { +export function shownToScript(value: unknown, claims?: ResultClaims, lineClaims?: ResultClaims): unknown { if (!value || typeof value !== 'object' || Array.isArray(value)) return value; const b = value as Record; if (!governedByRfc005(b)) return value; if ((b.type === 'image' || b.type === 'audio') && typeof b.data === 'string') { return hasUri(b.uri) || b.disposition !== undefined ? { type: 'text', text: INLINE_WITHHELD_TEXT } : value; } - return referenceBlockForScript(b, undefined, claims); + return referenceBlockForScript(b, undefined, claims, lineClaims); } /** @@ -484,9 +540,12 @@ export function shownToScript(value: unknown, claims?: ResultClaims): unknown { * shows, nested fields included (Ada-1017, room-203 #200094), by the walk * history's JSON takes (claimsInHistory): a script is shown the references * nested in a block's fields, each as its stub, so a `never` among them - * withholds that uri from every block in the view. Stricter than the - * model's line, which shows a block and not its fields (contentClaims), and - * never looser. An MCPL peer's top-level block the classifier rejects is + * withholds that uri from every block in the view. A link's uri is still + * shown only where the model's line shows it, by the claims the line reads + * (contentClaims, which reads a block and not its fields): reading more + * claims can loosen as well as tighten, since an expiry makes a link one the + * host can't fetch (referenceBlockForScript's `lineClaims`). An MCPL peer's + * top-level block the classifier rejects is * its notice, its fields unshown, so they don't count (`fromServer`). It * throws what stops the walk, such as a value too deep to convert or a * getter that throws, and the script's view then isn't written: the @@ -509,10 +568,10 @@ export function scriptViewClaims(content: readonly unknown[], fromServer: boolea * governed, so a block the top-level pass already treated is left as it is. * `claims` are those of the script's whole view (scriptViewClaims). */ -export function shownToScriptFromServer(value: unknown, serverId: string, claims?: ResultClaims): unknown { +export function shownToScriptFromServer(value: unknown, serverId: string, claims?: ResultClaims, lineClaims?: ResultClaims): unknown { if (!value || typeof value !== 'object' || Array.isArray(value)) return value; const b = value as Record; - return governedByRfc005(b) ? referenceBlockForScript(b, serverId, claims) : value; + return governedByRfc005(b) ? referenceBlockForScript(b, serverId, claims, lineClaims) : value; } /** @@ -535,8 +594,10 @@ export function rfc005TextOf(value: unknown): string | null { * (keptWhenWithheld). The claims walk meets data no JSON can hold, such as * a cycle or a BigInt, or data too deep to write, before the render does, * so such data shows this too, where the render threw (Desmond-2579's run). - * A script never gets it: both of its routes read the claims first and - * fail naming the tool (Ada-1017, room-203 #200344, #200374). + * nonErrorResultText, the model lane's entry, shows it as well for anything + * else that stops reading or showing a result (Ada-1017, room-203 #201013). + * A script never gets it: both of its routes read the claims first and call + * the serializer directly, and fail naming the tool (#200344, #200374). */ export const UNCHECKED_RESULT_TEXT = "[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]"; diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index 14cb97901..60dd3311b 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -26,6 +26,7 @@ import { AgentFramework, WorkspaceModule } from '../src/index.js'; import type { EventResponse, Module, ModuleContext, ToolCall, ToolDefinition, ToolResult } from '../src/index.js'; import { EXPIRY_REASON, INLINE_WITHHELD_TEXT, UNBOUND_REASON, referenceRegistry } from '../src/mcpl/references.js'; import type { ReferenceRecord } from '../src/mcpl/references.js'; +import { nonErrorResultText } from '../src/tool-result-history.js'; import { MockMembrane } from './helpers/mock-membrane.js'; const SAVED = 'board/references/report.pdf'; @@ -658,7 +659,110 @@ test("a nested never decides over a plain reference to its uri at the top, for a answer({ content: content('peer', where), structuredContent: {} }); const fromPeer = await script('mcpl--files--attach'); const fromModule = await exit({ success: true, data: content('module', where), structured: {} }, 'fwd--whole', false); - for (const out of [fromPeer, fromModule]) assert.ok(!out.includes('elsewhere.example.invalid'), `${where}: ${out}`); + for (const out of [fromPeer, fromModule]) { + assert.ok(!out.includes('elsewhere.example.invalid'), `${where}: ${out}`); + // The view was written, each block in it, the link as its object. + const object = JSON.parse(out) as { content: Array>; isError: boolean }; + assert.equal(object.isError, false, out); + assert.equal(object.content.length, 3, out); + const link = (where === 'top' ? object.content[0] : object.content[0]!.link) as Record; + assert.equal(link.type, 'resource_link', out); + assert.match(String(link.refId), /^ref_/, out); + assert.equal(link.uri, undefined, out); + } + } + }); +}); + +test("a link's uri reaches a script only where the model's line shows it: a nested expiry, which makes the link unfetchable, doesn't show it", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script, modelCall }) => { + // On the server's origin and with no credential in it, so the host can + // fetch it, and the model's line, reading the top-level blocks, shows no + // uri. A reference nested in another block's field says the uri has + // expired, which a script's view reads: the host couldn't fetch it then, + // and a link it can't fetch shows its uri, so reading more claims would + // show the script what the line withholds (Ada-1017, room-203 #201013). + const uri = 'https://files.example.invalid/line-claims/report.csv'; + const content = () => [ + { type: 'resource_link', uri, name: 'report.csv' }, + { type: 'text', text: 'an old copy', attachment: { type: 'resource', uri, name: 'report.csv', expiresAt: '2000-01-01T00:00:00Z' } }, + ]; + answer({ content: content(), structuredContent: {} }); + // The model's line for the same result, as history stores it. + const line = nonErrorResultText(await modelCall('attach')); + assert.match(line, /report\.csv/); + assert.ok(!line.includes('line-claims'), line); + answer({ content: content(), structuredContent: {} }); + const out = await script('mcpl--files--attach'); + assert.ok(!out.includes('line-claims'), out); + const object = JSON.parse(out) as { content: Array>; isError: boolean }; + assert.equal(object.isError, false); + // Its words still read the view's claims: a fetch with its id would be refused. + assert.equal(object.content[0]!.expired, EXPIRY_REASON.passed); + }); +}); + +test("an MCPL peer's script-demand fetch reads the view's claims: a nested expiry isn't fetched, and a view that can't be read fetches nothing", async () => { + await withPeers(async ({ answer, script, fetched }) => { + // A reference the demand fetch would save, and a reference nested in + // another block that says its uri has expired, which only a reading of + // the whole view's claims sees (Ada-1017, room-203 #201013). + const expiredNested = blocks('nested-expiry')[0] as { uri: string }; + answer({ + content: [expiredNested, { type: 'text', text: 'an old copy', attachment: { type: 'resource', uri: expiredNested.uri, expiresAt: '2000-01-01T00:00:00Z' } }], + structuredContent: {}, + }); + const object = JSON.parse(await script('mcpl--files--attach')) as { content: Array>; isError: boolean }; + assert.equal(object.isError, false); + assert.deepEqual(fetched, [], 'the uri a nested reference calls expired was not fetched'); + assert.equal(object.content[0]!.expired, EXPIRY_REASON.passed); + // A view whose claims walk stops isn't written, and nothing is fetched for it first. + const stuck = { get type(): string { throw new RangeError('Maximum call stack size exceeded'); } }; + answer({ content: [blocks('stopped-walk')[0], { type: 'text', text: 'deep', deep: stuck }], structuredContent: {} }); + const failed = JSON.parse(await script('mcpl--files--attach')) as { content: [{ text: string }]; isError: boolean }; + assert.equal(failed.isError, true); + assert.match(failed.content[0].text, /can't be converted to JSON/); + assert.deepEqual(fetched, [], 'nothing was fetched for a view that was never written'); + }); +}); + +test("an MCPL peer's top-level link, treated once, keeps the record of the uri its result named, whatever sanitizeLabel cut from the uri it shows", async () => { + await withPeers(async ({ answer, script }) => { + // Off the server's origin, so its uri is shown, cut of its bidi override. + // The nested pass leaves the top-level blocks to the treatment above it, + // so the link's object isn't read again as a link to the cut uri. + const raw = 'https://elsewhere.example.invalid/re\u202Eport.csv'; + answer({ content: [{ type: 'resource_link', uri: raw, name: 'report.csv' }], structuredContent: {} }); + const object = JSON.parse(await script('mcpl--files--attach')) as { content: Array> }; + assert.equal(object.content[0]!.uri, 'https://elsewhere.example.invalid/report.csv'); + assert.equal(object.content[0]!.refId, referenceRegistry.findByUri(raw)?.refId); + assert.equal(referenceRegistry.findByUri('https://elsewhere.example.invalid/report.csv'), undefined); + }); +}); + +test("where the treatment fails after the claims walk read the view, a peer's result is one the script can't convert and a module's a failure naming the tool", async () => { + await withPeers(async ({ answer, script, exit }) => { + // The claims walk reads a reference and registers nothing; the treatment + // registers what it shows. A registry that throws a RangeError for one + // nested reference stands in for a treatment that runs out of stack where + // the claims walk, with lighter replacers, didn't. + const uri = 'https://files.example.invalid/treatment-stop/report.csv'; + const real = referenceRegistry.register; + referenceRegistry.register = function (this: typeof referenceRegistry, ...args: Parameters) { + if (args[0].uri === uri) throw new RangeError('Maximum call stack size exceeded'); + return real.apply(this, args); + } as typeof real; + try { + const content = () => [{ type: 'text', text: 'a report', attachment: { type: 'resource', uri, name: 'report.csv' } }]; + answer({ content: content(), structuredContent: {} }); + const fromPeer = JSON.parse(await script('mcpl--files--attach')) as { content: [{ text: string }]; isError: boolean }; + assert.equal(fromPeer.isError, true); + assert.match(fromPeer.content[0].text, /^tool 'mcpl--files--attach' returned a result that can't be converted to JSON/); + const fromModule = JSON.parse(await exit({ success: true, data: content(), structured: {} }, 'fwd--whole', false)) as { content: [{ text: string }]; isError: boolean }; + assert.equal(fromModule.isError, true); + assert.match(fromModule.content[0].text, /^tool 'fwd--whole' returned a result that can't be converted to JSON/); + } finally { + referenceRegistry.register = real; } }); }); diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 178d2de62..09770c0e2 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -47,7 +47,7 @@ import { CoalesceError, validateCoalescedContent } from '../src/mcpl/push-coales import { ReferenceFetcher } from '../src/mcpl/reference-fetcher.js'; import { checkedToolResult } from '../src/module-registry.js'; import { parseImagePlaceholders } from '../src/tool-image-ledger.js'; -import { UNCHECKED_RESULT_TEXT, nonErrorResultText, shownToScript, toolResultDataToHistoryString } from '../src/tool-result-history.js'; +import { UNCHECKED_RESULT_TEXT, nonErrorResultText, shownToScript, toolResultDataToHistoryString, truncateForHistory } from '../src/tool-result-history.js'; import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; import { fixture } from './helpers/coalescing-fixture.js'; @@ -254,6 +254,14 @@ function testimonyOf(block: unknown) { } describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { + it("reads its size from MCP's size, and from sizeBytes only where size is absent or null, a size of 0 included", () => { + const sized = (fields: Record) => testimonyOf({ type: 'resource_link', uri: PLAIN_URI, name: 'report.csv', ...fields }).sizeBytes; + assert.equal(sized({ size: 10, sizeBytes: 20 }), 10, 'size decides where both are named'); + assert.equal(sized({ size: 0, sizeBytes: 20 }), 0, 'a size of 0 is a size'); + assert.equal(sized({ sizeBytes: 2048 }), 2048); + assert.equal(sized({ size: null, sizeBytes: 2048 }), 2048); + }); + it('maps onto the reference record field for field, its size as sizeBytes, with no disposition', () => { const t = testimonyOf({ ...LINK, disposition: 'never', digest: `sha256:${'A'.repeat(43)}`, expiresAt: '2020-01-01T00:00:00Z' }); assert.deepEqual([t.uri, t.name, t.mimeType, t.sizeBytes], [LINK_URI, 'report.csv', 'text/csv', 1234]); @@ -1395,6 +1403,62 @@ describe('one answer for every block of a result', () => { } }); + it("withholds a uri inline data marks never from a link naming it, in a content array, at depth and in a script's view", () => { + // Off the server's origin, so a link to it shows its uri where nothing + // withholds it; the inline block's never does, though it's no reference + // (Ada-1017, room-203 #201013). + const uri = 'https://zz-elsewhere.example/inline-never/report.csv'; + const sealed = { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png', uri, disposition: 'never' }; + const link = { type: 'resource_link', uri, name: 'report.csv' }; + const control = toolResultDataToHistoryString([link], undefined, { dataIsContent: true }); + assert.ok(control.includes('inline-never'), control); + for (const shown of [ + toolResultDataToHistoryString([sealed, link], undefined, { dataIsContent: true }), + toolResultDataToHistoryString({ sealed, link }), + JSON.stringify(shownToScript(link, referencesModule.contentClaims([sealed, link]))), + ]) { + assert.ok(!shown.includes('inline-never'), shown); + } + // Only its never counts: an expiry beside its data claims nothing. + const dated = { ...sealed, disposition: 'ref', expiresAt: '2000-01-01T00:00:00Z' }; + assert.ok(toolResultDataToHistoryString([dated, link], undefined, { dataIsContent: true }).includes('inline-never')); + }); + + it("shows the model the unchecked notice, with a line naming the tool, for whatever stops reading or showing a result, where the serializer throws for a script", () => { + const logged: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }; + try { + // A block of a content array whose text can't be read. + const unreadable = [{ type: 'text', get text(): string { throw new Error('zz-unreadable text'); } }]; + let blocks = false; + const text = nonErrorResultText({ data: unreadable, dataIsContent: true }, undefined, { toolName: 'zz_tool', onBlocks: () => { blocks = true; } }); + assert.equal(text, UNCHECKED_RESULT_TEXT); + assert.equal(blocks, false, 'no live copy of its own: the notice is both copies'); + assert.throws(() => toolResultDataToHistoryString(unreadable, undefined, { dataIsContent: true }), /zz-unreadable text/); + // A render that stops where the claims walk got through: the render + // registers what it shows, and the claims walk registers nothing. + const uri = 'https://zz-elsewhere.example/render-stop/report.csv'; + const real = referencesModule.referenceRegistry.register; + referencesModule.referenceRegistry.register = function (this: unknown, ...args: Parameters) { + if (args[0].uri === uri) throw new RangeError('Maximum call stack size exceeded'); + return real.apply(referencesModule.referenceRegistry, args); + } as typeof real; + try { + const data = { found: { type: 'resource', uri, name: 'report.csv' } }; + assert.equal(nonErrorResultText({ data }, 100, { toolName: 'zz_deep' }), truncateForHistory(UNCHECKED_RESULT_TEXT, 100)); + assert.throws(() => toolResultDataToHistoryString(data), RangeError); + } finally { + referencesModule.referenceRegistry.register = real; + } + } finally { + console.error = error; + } + assert.equal(logged.length, 2, logged.join('\n')); + assert.match(logged[0]!, /tool 'zz_tool'.*zz-unreadable text/); + assert.match(logged[1]!, /tool 'zz_deep'.*Maximum call stack size exceeded/); + }); + it('shows data whose claims walk stops as the unchecked notice, never with no claims, and a round with it completes, both copies alike', async () => { // A value that stops the claims walk and not the render's: its type // throws a RangeError the first time it's read, as a stack can run out From cec3c35829518f8825f6ebeb82b3fcfa1ebd2187 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 05:10:46 -0400 Subject: [PATCH 23/40] fix(tool-results): the model lane's catch covers reading the result's data and error too A precision from a read of 50c2c20c: nonErrorResultText read result.data and result.error before its try, so a direct Agent API caller's result whose `data` is a getter that throws still threw, as on main. Of narrowing the fragment and moving the reads, moving them makes the fragment's "a getter that throws" hold whatever holds it. The test adds such a result, which shows the notice with a line naming the tool. Co-Authored-By: Claude Opus 5.5 --- src/tool-result-history.ts | 18 ++++++++++-------- test/tool-result-blocks.test.ts | 9 +++++++-- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index f23b29fa5..6af165d65 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -293,20 +293,22 @@ export function nonErrorResultText( maxChars?: number, options?: HistorySerializeOptions, ): string { - const data = result.data; - if (data === undefined || data === null) { - const error = result.error; - if (typeof error === 'string' && error) { - const text = error.toWellFormed(); - return maxChars ? truncateForHistory(text, maxChars) : text; - } - } // The blocks go to the caller only once the text is made from them, so a // result that shows the notice has no live copy of its own: the notice is // both copies' text. let blocks: [HistoryBlockSpan[], ToolResultBlock[]] | undefined; const onBlocks = options?.onBlocks; try { + // Read inside the catch too: the direct Agent API's result is its + // caller's own object, whose `data` can be a getter. + const data = result.data; + if (data === undefined || data === null) { + const error = result.error; + if (typeof error === 'string' && error) { + const text = error.toWellFormed(); + return maxChars ? truncateForHistory(text, maxChars) : text; + } + } const text = toolResultDataToHistoryString(data, maxChars, { ...options, dataIsContent: Boolean(result.dataIsContent), diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 09770c0e2..40f0a47f6 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1436,6 +1436,10 @@ describe('one answer for every block of a result', () => { assert.equal(text, UNCHECKED_RESULT_TEXT); assert.equal(blocks, false, 'no live copy of its own: the notice is both copies'); assert.throws(() => toolResultDataToHistoryString(unreadable, undefined, { dataIsContent: true }), /zz-unreadable text/); + // A result whose data itself can't be read, as a host's object to the + // direct Agent API can be. + const result = { get data(): unknown { throw new Error('zz-unreadable data'); } }; + assert.equal(nonErrorResultText(result, undefined, { toolName: 'zz_data' }), UNCHECKED_RESULT_TEXT); // A render that stops where the claims walk got through: the render // registers what it shows, and the claims walk registers nothing. const uri = 'https://zz-elsewhere.example/render-stop/report.csv'; @@ -1454,9 +1458,10 @@ describe('one answer for every block of a result', () => { } finally { console.error = error; } - assert.equal(logged.length, 2, logged.join('\n')); + assert.equal(logged.length, 3, logged.join('\n')); assert.match(logged[0]!, /tool 'zz_tool'.*zz-unreadable text/); - assert.match(logged[1]!, /tool 'zz_deep'.*Maximum call stack size exceeded/); + assert.match(logged[1]!, /tool 'zz_data'.*zz-unreadable data/); + assert.match(logged[2]!, /tool 'zz_deep'.*Maximum call stack size exceeded/); }); it('shows data whose claims walk stops as the unchecked notice, never with no claims, and a round with it completes, both copies alike', async () => { From e067cc808abeb8dea8fceef72657cdeb190bd93a Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 05:14:01 -0400 Subject: [PATCH 24/40] docs(tool-results): the comment says the try, not the catch A word from a read of cec3c358: the reads sit inside the try, which the catch covers. Comment only. Co-Authored-By: Claude Opus 5.5 --- src/tool-result-history.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 6af165d65..9708b9d7b 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -299,7 +299,7 @@ export function nonErrorResultText( let blocks: [HistoryBlockSpan[], ToolResultBlock[]] | undefined; const onBlocks = options?.onBlocks; try { - // Read inside the catch too: the direct Agent API's result is its + // Read inside the try too: the direct Agent API's result is its // caller's own object, whose `data` can be a getter. const data = result.data; if (data === undefined || data === null) { From 7bd4f19ed4acb02357872e7cbf6784b985ae2db0 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 06:37:03 -0400 Subject: [PATCH 25/40] fix(references, tool-results): review fixes: block order, a never by the text a link shows, the model lane's one catch The calls on the review's findings: - A never is found by the text a link shows (shownUri, sanitizeLabel's cut; neverShown, the set of withheld texts read once per ResultClaims), so a never on a uri withholds every link in the result that shows the same text. Every other claim stays keyed by the raw uri, since it's about the record, which the fetch reads by its raw uri: an expiry on one uri says nothing of another that cuts alike. - The notice reads "[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]", true for every way it now arises. - Per link: a link the model's line doesn't show (an error result's, a nested one) shows a script the uri its own link line would, by the claims the model's line reads (as built). - A module's top-level resource_link naming no uri shows its fields to a script, as module data reached a script on main; the code comment, the fixed fragment and the body say so exactly. Also: - lineClaimsBeforeTreatment freezes, before the treatment, the record's claims for each uri the script's view claims and the line doesn't: the line falls back to the record there, and the treatment registers what it shows as it goes, so a reference nested before a link (an expiry, or a plain reference clearing an earlier result's never) changed what the link read, and its uri turned on the order of the blocks. Both routes pass the frozen claims as lineClaims. - The dispatch reads a peer's view claims (SCRIPT_VIEW_CLAIMS) only where reading() writes a view, forScript with structuredContent; a script reading the model lane's text has its fetch read each block's own claims, as the text does. - nonErrorResultTextOrThrow for a script's routes (the null-data route), nonErrorResultText the model lane's one catch; and historyJson throws where its claims walk stops (claimsInHistoryOrThrow), so the depth case gets that catch's log line too, where it returned the notice silently. Texts: claimsInHistory's and the notice's docs; referenceBlockForScript says through its blocks, structured content as sent. Tests: a nested link's uri is the same whichever block comes first, and an earlier result's never stays with the line's reading of the record when a later result's nested reference clears it; a peer's result a script reads as text fetches by each block's own claims; the view reads the claims its dispatch read; inline data's expiry claims nothing for a fetchable link to its uri; an expiry on one uri says nothing of a link to another that shows as the same text, and a never withholds such a link; the model lane's notice for a result whose error or content flag can't be read, and for a value thrown that can't be read as text, which reaches thrownLine's fallback; a stopped claims walk throws for a script and logs the tool through the stored copy's builder; a module's result with no data and an unreadable error fails the script; the inline-never test reads the script's view itself; fixture dates moved from 2026-12-31 and 2027-01-01 to 2099-12-31. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 4 +- changelog.d/tool-result-blocks.fixed.md | 2 +- src/framework.ts | 13 ++- src/mcpl/references.ts | 76 ++++++++++--- src/tool-result-history.ts | 85 +++++++++------ test/script-structured-references.test.ts | 123 ++++++++++++++++++++-- test/tool-result-blocks.test.ts | 64 +++++++++-- 7 files changed, 299 insertions(+), 68 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index dfe83ea12..bee056895 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,5 +1,5 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text once the controls a label drops are cut, or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. - In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads, so a script never sees a uri the model's line withholds. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. - A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. -- A tool result's data that can't be read through to apply RFC-005's rules, or shown once it has been, shows `[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]` on the model's lane, in the stored and sent copies alike, with a log line naming the tool and the error. It was shown with none of its references' claims where the walk reading them stopped and the render's didn't, or, for data no JSON can hold, such as a cycle or a BigInt, a getter that throws, or data too deep to write, the copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception, with the tool-result guard off; with it on, its archive of the result still throws, as before. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. +- A tool result's data that can't be read through to apply RFC-005's rules, or shown once it has been, shows `[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]` on the model's lane, in the stored and sent copies alike, with a log line naming the tool and the error. It was shown with none of its references' claims where the walk reading them stopped and the render's didn't, or, for data no JSON can hold, such as a cycle or a BigInt, a getter that throws, or data too deep to write, the copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception, with the tool-result guard off; with it on, its archive of the result still throws, as before. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md index f6522adb6..12c0c86b3 100644 --- a/changelog.d/tool-result-blocks.fixed.md +++ b/changelog.d/tool-result-blocks.fixed.md @@ -1,5 +1,5 @@ - A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's that doesn't set `dataIsContent`, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. -- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link whose uri is empty, null or not a string is the marker that says so. A link that names no uri at all is the marker as a block of a content array, and at depth stays data, as before, since its fields hold no uri this reads. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. +- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link whose uri is empty, null or not a string is the marker that says so. A link that names no uri at all is the marker as a block of a content array the model reads, and at depth, or in a script's view of a module's result, stays data, as before, since its fields hold no uri this reads. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. - A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it ends with why `fetch_reference` would refuse, without what calling again can do. The stub reads an expiry from the references in its result where they name its uri, and the fetch from the reference as the host last recorded it, so where one result names a uri both expired and fresh, the stub says it's expired while a fetch can succeed. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, one whose address doesn't parse, a uri off its server's origin, one with a user name or password, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. - A label in a stub (a name, a type, a server's id, a provenance or a saved path) loses any line or paragraph separator (U+2028, U+2029), as it loses control characters. - A script's demand fetch of a reference not yet fetched reads that reference's own digest and expiry, where it read whatever an earlier reference to the same uri had claimed. diff --git a/src/framework.ts b/src/framework.ts index 6a5a4c7ae..ecace646e 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,7 +1,7 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { constants as bufferConstants } from 'node:buffer'; import { dirname, join } from 'node:path'; -import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ResultClaims, type ServerAddress } from './mcpl/references.js'; +import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, isReferenceExpired, lineClaimsBeforeTreatment, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ResultClaims, type ServerAddress } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, @@ -117,6 +117,7 @@ import type { ToolResultGuard, WithheldSpill } from './tool-result-guard.js'; import { goesNative, nonErrorResultText, + nonErrorResultTextOrThrow, resultClaimsOrThrow, scriptViewClaims, shownToScript, @@ -22599,6 +22600,7 @@ export class AgentFramework { try { lineClaims = resultClaimsOrThrow(result.data, result.dataIsContent); claims = showsBlocks ? scriptViewClaims(result.data, false) : lineClaims; + if (showsBlocks) lineClaims = lineClaimsBeforeTreatment(lineClaims, claims); } catch (err) { return failed(err); } @@ -22654,7 +22656,7 @@ export class AgentFramework { text = isError ? result.error ?? 'tool call failed' : result.data === undefined || result.data === null - ? nonErrorResultText(result) + ? nonErrorResultTextOrThrow(result) : toolResultDataToHistoryString(result.data, undefined, { dataIsContent: Boolean(result.dataIsContent), claims: claims ?? resultClaimsOrThrow(result.data, result.dataIsContent), @@ -29099,7 +29101,7 @@ export class AgentFramework { const read = SCRIPT_VIEW_CLAIMS.get(direct.data); if (read && 'error' in read) throw read.error; claims = read ? read.claims : scriptViewClaims(direct.data, true); - lineClaims = resultClaimsOrThrow(direct.data, true); + lineClaims = lineClaimsBeforeTreatment(resultClaimsOrThrow(direct.data, true), claims); } catch (err) { return unconvertible(err); } @@ -29206,7 +29208,10 @@ export class AgentFramework { // references only registered (Ada-1017, room-203 #201013). let mode: 'register' | 'model' | 'script' = scriptReads ? 'script' : 'model'; let claims: ResultClaims | undefined; - if (scriptReads && Array.isArray(result.content)) { + // Only where the script reads a view of the blocks (reading()'s own + // condition): without structured content it reads the model lane's + // text, whose fetch reads each block's own claims, as the text does. + if (scriptReads && Object.prototype.hasOwnProperty.call(result, 'structuredContent') && Array.isArray(result.content)) { try { claims = scriptViewClaims(result.content, true); SCRIPT_VIEW_CLAIMS.set(result.content, { claims }); diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 524078caa..c852b5e4f 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -727,6 +727,31 @@ export interface StubOptions { */ export type ResultClaims = ReadonlyMap; +/** + * A uri as a link shows it, cut of what sanitizeLabel cuts. A `never` is + * about what a link shows, so it's found by this text (neverShown): a claim + * of `never` on a uri withholds every link that shows the same text, two + * uris that cut alike sharing it. Every other claim is about the record, + * which the fetch reads by its raw uri, so it's kept by the raw uri (Ada-1017, + * room-203 #202239). + */ +export function shownUri(uri: string): string { + return sanitizeLabel(uri, REFERENCE_LIMITS.uri); +} + +const NEVER_SHOWN = new WeakMap>(); + +/** The texts a link shows (shownUri) for the uris a result's claims mark + * `never`, read once per claims. A ResultClaims isn't changed once made. */ +function neverShown(claims: ResultClaims): ReadonlySet { + let shown = NEVER_SHOWN.get(claims); + if (!shown) { + shown = new Set([...claims].filter(([, c]) => c.never).map(([uri]) => shownUri(uri))); + NEVER_SHOWN.set(claims, shown); + } + return shown; +} + /** What a result's references say of one uri, read as one. */ export interface UriClaims { /** Whether any of them marks it `disposition: never`. */ @@ -747,15 +772,15 @@ export function resultClaims(testimonies: Iterable, nevers: // classifier's contradiction (inlineNeverUri): withheld unconditionally // (RFC-005 §5), though the block is no reference. Only its never counts, // since its other claims could loosen (Ada-1017, room-203 #201013). - for (const uri of nevers) { - const c = claims.get(uri); - if (c) c.never = true; - else claims.set(uri, { never: true, rejected: false, unparseable: false, earliestMs: null }); - } + const entry = (uri: string): UriClaims => { + let c = claims.get(uri); + if (!c) claims.set(uri, c = { never: false, rejected: false, unparseable: false, earliestMs: null }); + return c; + }; + for (const uri of nevers) entry(uri).never = true; for (const t of testimonies) { if (t.fromLink) continue; - let c = claims.get(t.uri); - if (!c) claims.set(t.uri, c = { never: false, rejected: false, unparseable: false, earliestMs: null }); + const c = entry(t.uri); if (t.disposition === 'never') c.never = true; if (t.rejectedFields.includes('expiresAt')) { c.rejected = true; @@ -793,6 +818,28 @@ export function inlineNeverUri(block: unknown, c: BlockClassification = classify return b.disposition === 'never' && typeof b.uri === 'string' && b.uri !== '' ? b.uri : null; } +/** + * The claims the model's line for a result reads (`line`), for a script's + * link objects to judge their uri by (referenceBlockForScript's lineClaims), + * with, for each uri the script's view claims (`view`) and the line doesn't, + * that uri's record's own claims as they stand now, before the treatment: + * the line falls back to the record there, and the treatment registers what + * it shows as it goes, so a reference nested before a link would otherwise + * change what the link reads (an expiry, or a plain reference that clears an + * earlier result's never), and the link's uri would turn on the order of + * the blocks. + */ +export function lineClaimsBeforeTreatment(line: ResultClaims, view: ResultClaims): ResultClaims { + const frozen = new Map(line); + for (const uri of view.keys()) { + if (frozen.has(uri)) continue; + const record = referenceRegistry.findByUri(uri); + frozen.set(uri, (record ? resultClaims([record.testimony]).get(uri) : undefined) + ?? { never: false, rejected: false, unparseable: false, earliestMs: null }); + } + return frozen; +} + /** The claims a decision about a record reads: its result's for its uri, * where the result's references name it, and otherwise the record's own * testimony. */ @@ -801,9 +848,10 @@ function uriClaims(record: ReferenceRecord, claims?: ResultClaims): UriClaims | } /** Whether a record's uri is withheld: marked `disposition: never` by its - * result's references, or by its own testimony where none of them names - * it. */ + * result's references, for any uri a link shows as the same text + * (shownUri), or by its own testimony where none of them names it. */ export function claimedNever(record: ReferenceRecord, claims?: ResultClaims): boolean { + if (claims && neverShown(claims).has(shownUri(record.testimony.uri))) return true; return uriClaims(record, claims)?.never === true; } @@ -834,13 +882,14 @@ export function claimedExpiry(record: ReferenceRecord, claims: ResultClaims | un * sanitizeLabel strips gone, so a uri split by a control character is read * as it would read: `ht\u0000tps://user:pass@…` parses as a path, and shows * as userinfo. A `disposition: "never"` for the uri keeps it out whatever - * the link says: any reference's in the same result (claims), or, where no - * reference in the result names the uri, the record's. + * the link says: any reference's in the same result (claims), for this uri + * or one shown as the same text, or, where no reference in the result names + * the uri, the record's. */ export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, nowMs = Date.now(), claims?: ResultClaims): string | null { if (!t.fromLink || claimedNever(record, claims)) return null; if (unfetchableReason(record, nowMs, claims) === null) return null; - const uri = sanitizeLabel(t.uri, REFERENCE_LIMITS.uri); + const uri = shownUri(t.uri); return uriMayCarryCredential(uri) ? null : uri; } @@ -994,7 +1043,8 @@ export function referenceBlockForScript(block: unknown, serverId?: string, claim // and never where the view's claims withhold it: reading more claims // can loosen as well as tighten, since an expiry makes a link one the // host can't fetch, whose uri the line shows (Ada-1017, room-203 - // #201013). A script never sees a uri the model's line withholds. + // #201013). A script never sees a uri the model's line withholds, + // through its blocks; structured content goes as sent, as on main. const lineShows = shownLinkUri(t, record, nowMs, lineClaims ?? claims); const uri = lineShows !== null && lineClaims !== undefined && claimedNever(record, claims) ? null : lineShows; // Why a fetch_reference with this object's refId would be refused, in diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 9708b9d7b..6bd98899f 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -299,19 +299,10 @@ export function nonErrorResultText( let blocks: [HistoryBlockSpan[], ToolResultBlock[]] | undefined; const onBlocks = options?.onBlocks; try { - // Read inside the try too: the direct Agent API's result is its - // caller's own object, whose `data` can be a getter. - const data = result.data; - if (data === undefined || data === null) { - const error = result.error; - if (typeof error === 'string' && error) { - const text = error.toWellFormed(); - return maxChars ? truncateForHistory(text, maxChars) : text; - } - } - const text = toolResultDataToHistoryString(data, maxChars, { + // The result's own fields are read inside the try too: the direct Agent + // API's result is its caller's own object, whose `data` can be a getter. + const text = nonErrorResultTextOrThrow(result, maxChars, { ...options, - dataIsContent: Boolean(result.dataIsContent), ...(onBlocks ? { onBlocks: (spans: HistoryBlockSpan[], made: ToolResultBlock[]) => { blocks = [spans, made]; } } : {}), }); if (onBlocks && blocks) onBlocks(blocks[0], blocks[1]); @@ -328,6 +319,28 @@ export function nonErrorResultText( } } +/** + * nonErrorResultText's text, throwing whatever stops reading or showing the + * result, such as a stopped claims walk (historyJson), for a caller that + * answers a stop itself: a script's routes, which fail naming the tool + * (Ada-1017, room-203 #200374), never the model lane's notice. + */ +export function nonErrorResultTextOrThrow( + result: { data?: unknown; error?: string; dataIsContent?: boolean }, + maxChars?: number, + options?: HistorySerializeOptions, +): string { + const data = result.data; + if (data === undefined || data === null) { + const error = result.error; + if (typeof error === 'string' && error) { + const text = error.toWellFormed(); + return maxChars ? truncateForHistory(text, maxChars) : text; + } + } + return toolResultDataToHistoryString(data, maxChars, { ...options, dataIsContent: Boolean(result.dataIsContent) }); +} + /** The first line of what was thrown, for a log line. */ function thrownLine(error: unknown): string { try { @@ -398,7 +411,10 @@ function governedByRfc005(b: Record): boolean { // on the model's lanes and a script's, as main has every link at depth: // its fields hold no uri RFC-005 governs, though a value in them, such as // an untyped object's `uri`, is data as anywhere else. As a block of a - // content array, the classifier reads it, and there it's the marker. + // content array on the model's line, the classifier reads it, and there + // it's the marker; a script's view of a module's array, which reads every + // block here, shows its fields, as module data reached a script on main + // (Ada-1017, room-203 #202138). // That's the object the script lane makes of a link whose uri it leaves // out (referenceBlockForScript), which a script's view, given it again, // leaves as it is, as it leaves a reference's stub. @@ -473,8 +489,8 @@ function claimsInHistory(data: unknown): ResultClaims | null { try { return claimsInHistoryOrThrow(data); } catch { - // What the rest of the data claims is unknown: historyJson shows it as - // UNCHECKED_RESULT_TEXT, and a withheld result's kept fields keep nothing. + // What the rest of the data claims is unknown, so a withheld result's + // kept fields keep nothing (keptWhenWithheld). return null; } } @@ -588,35 +604,38 @@ export function rfc005TextOf(value: unknown): string | null { } /** - * What a tool result's data shows as where the walk of its claims stops - * (claimsInHistory), so RFC-005's rules can't be read through it: the tool - * ran, and its result isn't shown, rather than shown with no claims, where a - * link could show a uri a reference in the result withholds. The round - * completes, as a withheld result's kept fields keep nothing then - * (keptWhenWithheld). The claims walk meets data no JSON can hold, such as - * a cycle or a BigInt, or data too deep to write, before the render does, - * so such data shows this too, where the render threw (Desmond-2579's run). - * nonErrorResultText, the model lane's entry, shows it as well for anything - * else that stops reading or showing a result (Ada-1017, room-203 #201013). - * A script never gets it: both of its routes read the claims first and call - * the serializer directly, and fail naming the tool (#200344, #200374). + * What the model's lane shows for a tool result it couldn't read through, + * check for references and show (nonErrorResultText, the lane's one catch, + * which logs the tool and the error): the tool ran, and its result isn't + * shown, rather than shown with fewer claims, where a link could show a uri a + * reference in the result withholds, and rather than a throw, which would + * leave the round unstored and unsent. Where the walk of its claims stops + * (historyJson throws then), the claims walk meets data no JSON can hold, + * such as a cycle or a BigInt, or data too deep to write, before the render + * does, so such data shows this, where the render threw (Desmond-2579's + * run); so does a render that stops after the walk, or a getter that throws + * (Ada-1017, room-203 #200344, #201013). The round completes, as a withheld + * result's kept fields keep nothing then (keptWhenWithheld). A script never + * gets it: its routes read the claims first and call the serializer, or + * nonErrorResultTextOrThrow, directly, and fail naming the tool (#200374). */ export const UNCHECKED_RESULT_TEXT = - "[Result not shown: the tool ran, but its result couldn't be read through to check it for references, so this host withheld it.]"; + "[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]"; /** * JSON of a tool result's data with each value in it shown as history shows * it (shownInHistory), at any depth, by the claims `given` or else its own - * (claimsInHistory). Data that is itself shown as text, such as a bare - * reference, is that text, and data whose claims can't be read is - * UNCHECKED_RESULT_TEXT. + * (claimsInHistoryOrThrow), throwing where that walk stops. Data that is + * itself shown as text, such as a bare reference, is that text. */ function historyJson(data: unknown, given?: ResultClaims): string | undefined { // A walk that stopped can't say what the data claims, and showing it with // no claims would let a link show a uri a reference withholds, should the // render's own walk, with other replacers, get past the same value. - const claims = given ?? claimsInHistory(data); - if (claims === null) return UNCHECKED_RESULT_TEXT; + // Where the walk stops this throws: the model's lane answers with + // UNCHECKED_RESULT_TEXT and a log line (nonErrorResultText), a script's + // routes with a failure naming the tool. + const claims = given ?? claimsInHistoryOrThrow(data); const root = shownInHistory(data, claims); if (root !== data && typeof root === 'string') return root; return JSON.stringify(data, (_key, value: unknown) => shownInHistory(value, claims)) as string | undefined; diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index 60dd3311b..e1ce59ad7 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -45,11 +45,11 @@ const blocks = (test: string) => [ // A reference whose payload the fetch saves. Its labels carry a bidi // override, a bidi mark and a control character. { type: 'resource', uri: uris(test).report, name: 'report\u202E.pdf\u0007', mimeType: 'application/pdf\u200F', sizeBytes: 1234, - digest: DIGEST, expiresAt: '2026-12-31T00:00:00Z' }, + digest: DIGEST, expiresAt: '2099-12-31T00:00:00Z' }, // The same reference again, saying something else about itself. { type: 'resource', uri: uris(test).report, name: 'report-copy.pdf' }, // A reference the fetch doesn't save. - { type: 'image', uri: uris(test).big, mimeType: 'image/png', disposition: 'ref', expiresAt: '2027-01-01T00:00:00Z' }, + { type: 'image', uri: uris(test).big, mimeType: 'image/png', disposition: 'ref', expiresAt: '2099-12-31T00:00:00Z' }, // Inline data that claims a bulk disposition, and inline data with a uri // beside it: withheld. { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png', disposition: 'ref' }, @@ -67,7 +67,7 @@ const blocks = (test: string) => [ // A reference that counts as expired: with a bidi override after it, its // expiresAt doesn't parse, so it fails closed. It isn't fetched, and the // script is told why. - { type: 'image', uri: uris(test).old, mimeType: 'image/png', expiresAt: '2027-01-01T00:00:00Z\u202E' }, + { type: 'image', uri: uris(test).old, mimeType: 'image/png', expiresAt: '2099-12-31T00:00:00Z\u202E' }, ]; /** A `never` reference, which RFC-005 withholds, with its uri, from every lane the model reads. */ @@ -225,9 +225,9 @@ async function withPeers(fn: (h: Harness) => Promise, opts: { eager?: bool /** What a script reads for those blocks from an MCPL peer, given each reference's id. */ const treated = (report: string, big: string, old: string, link: string) => [ { type: 'resource', refId: report, name: 'report.pdf', mimeType: 'application/pdf', sizeBytes: 1234, digest: DIGEST, - expiresAt: '2026-12-31T00:00:00Z', savedTo: SAVED }, + expiresAt: '2099-12-31T00:00:00Z', savedTo: SAVED }, { type: 'resource', refId: report, name: 'report-copy.pdf', savedTo: SAVED }, - { type: 'image', refId: big, mimeType: 'image/png', expiresAt: '2027-01-01T00:00:00Z', savedTo: null }, + { type: 'image', refId: big, mimeType: 'image/png', expiresAt: '2099-12-31T00:00:00Z', savedTo: null }, { type: 'text', text: INLINE_WITHHELD_TEXT }, { type: 'text', text: INLINE_WITHHELD_TEXT }, { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png' }, @@ -235,7 +235,7 @@ const treated = (report: string, big: string, old: string, link: string) => [ { type: 'text', text: '[Reference not shown: it arrived without a link.]' }, { type: 'resource_link', refId: link, name: 'link', savedTo: null }, { type: 'text', text: 'two attachments' }, - { type: 'image', refId: old, mimeType: 'image/png', expiresAt: '2027-01-01T00:00:00Z', savedTo: null, expired: EXPIRY_REASON.unparseable, + { type: 'image', refId: old, mimeType: 'image/png', expiresAt: '2099-12-31T00:00:00Z', savedTo: null, expired: EXPIRY_REASON.unparseable, unfetchable: `the reference is ${EXPIRY_REASON.unparseable}` }, ]; @@ -726,6 +726,111 @@ test("an MCPL peer's script-demand fetch reads the view's claims: a nested expir }); }); +test("a nested link's uri doesn't turn on the order of the blocks: a reference nested before it doesn't change what its line reads", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script }) => { + // Fetchable, so the line, reading no claim of this result for the uri, + // shows no uri. A nested expired reference registered before the link, + // as the treatment's own pass registers what it shows, would make the + // record unfetchable for the link and show its uri, in one order only. + for (const [i, order] of ([['expired', 'link'], ['link', 'expired']] as const).entries()) { + const uri = `https://files.example.invalid/block-order-${i}/report.csv`; + const fields: Record = {}; + for (const name of order) { + fields[name] = name === 'link' + ? { type: 'resource_link', uri, name: 'report.csv' } + : { type: 'resource', uri, name: 'report.csv', expiresAt: '2000-01-01T00:00:00Z' }; + } + const content = [{ type: 'text', text: 'two attachments', ...fields }]; + answer({ content, structuredContent: {} }); + const fromPeer = await script('mcpl--files--attach'); + assert.ok(!fromPeer.includes(`block-order-${i}`), `${order.join(' then ')}: ${fromPeer}`); + } + // An earlier result's never, which a later result's plain reference, + // nested before the link, would clear on the record. + const sealedUri = 'https://elsewhere.example.invalid/order-never/report.csv'; + answer({ content: [{ type: 'resource', uri: sealedUri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + const later = [{ type: 'text', text: 'again', plain: { type: 'resource', uri: sealedUri, name: 'report.csv' }, link: { type: 'resource_link', uri: sealedUri, name: 'report.csv' } }]; + answer({ content: later, structuredContent: {} }); + const out = await script('mcpl--files--attach'); + // The view reads the result's plain reference, the latest word, so its + // never no longer holds there; the line, which shows neither, reads the + // record as it stood, and withholds the uri; so does the script. + assert.ok(!out.includes('order-never'), out); + }); +}); + +test("an MCPL peer's result a script reads as text has its demand fetch read each block, as the text does, not a view it never gets", async () => { + await withPeers(async ({ answer, script, fetched }) => { + // No structured content: the script reads the model lane's text, whose + // stubs read the top-level blocks' claims, so a reference nested in a + // block's field, which the text doesn't show, stops no fetch. + const top = blocks('text-route')[0] as { uri: string }; + answer({ content: [top, { type: 'text', text: 'an old copy', attachment: { type: 'resource', uri: top.uri, expiresAt: '2000-01-01T00:00:00Z' } }] }); + const out = await script('mcpl--files--attach'); + assert.deepEqual(fetched, [top.uri]); + assert.ok(out.includes(SAVED), out); + }); +}); + +test("an MCPL peer's view reads the claims its dispatch read for the demand fetch, not a second reading of its blocks", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script, fetched }) => { + // A nested reference that says never until the demand fetch has run, as + // only a value read twice could: a view that read its claims again would + // see none, and show the link's uri. + const uri = 'https://elsewhere.example.invalid/read-once/report.csv'; + const sealed = { type: 'resource', uri, name: 'report.csv', get disposition(): string | undefined { return fetched.length === 0 ? 'never' : undefined; } }; + answer({ + content: [blocks('read-once')[0], { type: 'text', text: 'sealed', sealed }, { type: 'resource_link', uri, name: 'report.csv' }], + structuredContent: {}, + }); + const out = await script('mcpl--files--attach'); + assert.equal(fetched.length, 1, 'the demand fetch ran between the two readings'); + assert.ok(!out.includes('read-once/report.csv'), out); + }); +}); + +test("inline data's expiry claims nothing for a link to its uri, which the host can fetch, so the link shows no uri", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script, modelCall }) => { + // Fetchable, so only an expiry claimed for the uri would make it a link + // the host can't fetch, whose uri shows. Inline data's claims beside its + // data count for nothing but never (Ada-1017, room-203 #201013). + const uri = 'https://files.example.invalid/inline-expiry/report.csv'; + const content = () => [ + { type: 'image', data: 'iVBORw0KGgo=', mimeType: 'image/png', uri, disposition: 'ref', expiresAt: '2000-01-01T00:00:00Z' }, + { type: 'resource_link', uri, name: 'report.csv' }, + ]; + answer({ content: content(), structuredContent: {} }); + const line = nonErrorResultText(await modelCall('attach')); + assert.ok(!line.includes('inline-expiry'), line); + answer({ content: content(), structuredContent: {} }); + const out = await script('mcpl--files--attach'); + assert.ok(!out.includes('inline-expiry'), out); + }); +}); + +test("an expiry on one uri says nothing of a link to another that shows as the same text, which the host can fetch", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script, modelCall }) => { + // Only a `never` is found by the text a link shows; an expiry is about the + // record, which the fetch reads by its raw uri (Ada-1017, room-203 + // #202239). Read by the text, this expiry would make the link one the + // host can't fetch, showing its uri and words saying so. + const uri = 'https://files.example.invalid/expiry-cut/report.csv'; + const content = () => [ + { type: 'resource', uri, name: 'old.csv', expiresAt: '2000-01-01T00:00:00Z' }, + { type: 'resource_link', uri: 'https://files.example.invalid/expiry-cut/\u200Freport.csv', name: 'report.csv' }, + ]; + answer({ content: content(), structuredContent: {} }); + const line = nonErrorResultText(await modelCall('attach')); + assert.ok(!line.includes('expiry-cut'), line); + answer({ content: content(), structuredContent: {} }); + const out = await script('mcpl--files--attach'); + assert.ok(!out.includes('expiry-cut'), out); + const link = (JSON.parse(out) as { content: Array> }).content[1]!; + assert.deepEqual(['uri' in link, 'unfetchable' in link, 'expired' in link, typeof link.refId], [false, false, false, 'string'], out); + }); +}); + test("an MCPL peer's top-level link, treated once, keeps the record of the uri its result named, whatever sanitizeLabel cut from the uri it shows", async () => { await withPeers(async ({ answer, script }) => { // Off the server's origin, so its uri is shown, cut of its bidi override. @@ -814,6 +919,12 @@ test("where the walk of a result's claims stops, a script's text route fails nam return { get type(): string { if (!read) { read = true; throw new RangeError('Maximum call stack size exceeded'); } return 'note'; } }; }; // No structured content, so the text route; data that isn't content. + // No data and an error that can't be read: still a failure, never the + // model lane's notice (nonErrorResultTextOrThrow). + const noError = { success: true, data: null, get error(): string { throw new Error('zz-unreadable error'); } }; + const failedRead = JSON.parse(await exit(noError as unknown as ToolResult, 'fwd--whole', false)) as { content: [{ text: string }]; isError: boolean }; + assert.equal(failedRead.isError, true, JSON.stringify(failedRead)); + assert.match(failedRead.content[0].text, /^tool 'fwd--whole' returned a result that can't be converted to JSON/); for (const data of [{ deep: once() }, [{ deep: once() }]]) { const out = JSON.parse(await exit({ success: true, data }, 'fwd--whole', false)) as { content: [{ text: string }]; isError: boolean }; assert.equal(out.isError, true, JSON.stringify(out)); diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 40f0a47f6..672f60a09 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -47,7 +47,7 @@ import { CoalesceError, validateCoalescedContent } from '../src/mcpl/push-coales import { ReferenceFetcher } from '../src/mcpl/reference-fetcher.js'; import { checkedToolResult } from '../src/module-registry.js'; import { parseImagePlaceholders } from '../src/tool-image-ledger.js'; -import { UNCHECKED_RESULT_TEXT, nonErrorResultText, shownToScript, toolResultDataToHistoryString, truncateForHistory } from '../src/tool-result-history.js'; +import { UNCHECKED_RESULT_TEXT, nonErrorResultText, scriptViewClaims, shownToScript, toolResultDataToHistoryString, truncateForHistory } from '../src/tool-result-history.js'; import { MockMembrane, createMockResponse } from './helpers/mock-membrane.js'; import { fixture } from './helpers/coalescing-fixture.js'; @@ -1100,7 +1100,16 @@ describe('one answer for every block of a result', () => { // The claims walk meets the cycle before the render does, so the result // is the notice where main's serializer threw (Ada-1017, room-203 // #200374), and nothing past the cycle is registered either way. - assert.equal(toolResultDataToHistoryString({ loop, later: { type: 'resource', uri, name: 'past.csv', disposition: 'never' } }), UNCHECKED_RESULT_TEXT); + const data = { loop, later: { type: 'resource', uri, name: 'past.csv', disposition: 'never' } }; + const error = console.error; + console.error = () => {}; + try { + assert.equal(nonErrorResultText({ data }), UNCHECKED_RESULT_TEXT); + } finally { + console.error = error; + } + // A script's routes take the serializer's own answer: it throws. + assert.throws(() => toolResultDataToHistoryString(data), TypeError); assert.equal(referenceRegistry.findByUri(uri), undefined); }); @@ -1415,7 +1424,7 @@ describe('one answer for every block of a result', () => { for (const shown of [ toolResultDataToHistoryString([sealed, link], undefined, { dataIsContent: true }), toolResultDataToHistoryString({ sealed, link }), - JSON.stringify(shownToScript(link, referencesModule.contentClaims([sealed, link]))), + JSON.stringify(shownToScript(link, scriptViewClaims([sealed, link], false))), ]) { assert.ok(!shown.includes('inline-never'), shown); } @@ -1424,6 +1433,23 @@ describe('one answer for every block of a result', () => { assert.ok(toolResultDataToHistoryString([dated, link], undefined, { dataIsContent: true }).includes('inline-never')); }); + it("withholds a never uri from a link whose uri shows as the same text, its bidi control cut", () => { + const uri = 'https://zz-elsewhere.example/cut-never/report.csv'; + const sealed = { type: 'resource', uri, name: 'report.csv', disposition: 'never' }; + // The same uri with a right-to-left mark, which sanitizeLabel cuts, so + // the link would show exactly the text the never withholds. + const link = { type: 'resource_link', uri: 'https://zz-elsewhere.example/cut-never/\u200Freport.csv', name: 'report.csv' }; + const control = toolResultDataToHistoryString([link], undefined, { dataIsContent: true }); + assert.ok(control.includes('cut-never/report.csv'), control); + for (const shown of [ + toolResultDataToHistoryString([sealed, link], undefined, { dataIsContent: true }), + toolResultDataToHistoryString({ sealed, link }), + JSON.stringify(shownToScript(link, scriptViewClaims([sealed, link], false))), + ]) { + assert.ok(!shown.includes('cut-never'), shown); + } + }); + it("shows the model the unchecked notice, with a line naming the tool, for whatever stops reading or showing a result, where the serializer throws for a script", () => { const logged: string[] = []; const error = console.error; @@ -1440,6 +1466,14 @@ describe('one answer for every block of a result', () => { // direct Agent API can be. const result = { get data(): unknown { throw new Error('zz-unreadable data'); } }; assert.equal(nonErrorResultText(result, undefined, { toolName: 'zz_data' }), UNCHECKED_RESULT_TEXT); + const noError = { data: null, get error(): string { throw new Error('zz-unreadable error'); } }; + assert.equal(nonErrorResultText(noError, undefined, { toolName: 'zz_error' }), UNCHECKED_RESULT_TEXT); + const noFlag = { data: [{ type: 'text', text: 'x' }], get dataIsContent(): boolean { throw new Error('zz-unreadable flag'); } }; + assert.equal(nonErrorResultText(noFlag, undefined, { toolName: 'zz_flag' }), UNCHECKED_RESULT_TEXT); + // What's thrown can be a value String() can't read itself; the line + // still goes out, saying so. + const unprintable = { get data(): unknown { throw Object.create(null); } }; + assert.equal(nonErrorResultText(unprintable, undefined, { toolName: 'zz_unprintable' }), UNCHECKED_RESULT_TEXT); // A render that stops where the claims walk got through: the render // registers what it shows, and the claims walk registers nothing. const uri = 'https://zz-elsewhere.example/render-stop/report.csv'; @@ -1458,10 +1492,13 @@ describe('one answer for every block of a result', () => { } finally { console.error = error; } - assert.equal(logged.length, 3, logged.join('\n')); + assert.equal(logged.length, 6, logged.join('\n')); assert.match(logged[0]!, /tool 'zz_tool'.*zz-unreadable text/); assert.match(logged[1]!, /tool 'zz_data'.*zz-unreadable data/); - assert.match(logged[2]!, /tool 'zz_deep'.*Maximum call stack size exceeded/); + assert.match(logged[2]!, /tool 'zz_error'.*zz-unreadable error/); + assert.match(logged[3]!, /tool 'zz_flag'.*zz-unreadable flag/); + assert.match(logged[4]!, /tool 'zz_unprintable'.*a value that can't be read/); + assert.match(logged[5]!, /tool 'zz_deep'.*Maximum call stack size exceeded/); }); it('shows data whose claims walk stops as the unchecked notice, never with no claims, and a round with it completes, both copies alike', async () => { @@ -1474,16 +1511,25 @@ describe('one answer for every block of a result', () => { }; // A link beside a never to its uri, which a render with no claims would show. const data = () => ({ link: { type: 'resource_link', uri: PLAIN_URI, name: 'report.csv' }, sealed: { type: 'resource', uri: PLAIN_URI, disposition: 'never' }, deep: once() }); - assert.equal(toolResultDataToHistoryString(data()), UNCHECKED_RESULT_TEXT); + assert.throws(() => toolResultDataToHistoryString(data()), RangeError, 'a script\'s routes take the serializer\'s own answer'); const fw = Object.create(AgentFramework.prototype) as { buildStoredToolResultContent(agent: string, calls: unknown[], maxChars: number | undefined): Promise<{ blocks: Array<{ content: unknown }>; spilled: Map }>; toMembraneToolResult(callId: string, result: ToolResult, maxChars: number | undefined, precomputed: unknown): Live; }; Object.assign(fw, { toolImageLedgers: new Map(), writeToolResultFile: async () => null, moduleRegistry: { getAllModules: () => [] } }); const call = { id: 'zz-unchecked', name: 'zz_tool', input: {}, result: { success: true, data: data() } }; - const { blocks, spilled } = await fw.buildStoredToolResultContent('zz-agent', [call], 24_000); - assert.deepEqual(blocks.map((block) => block.content), [UNCHECKED_RESULT_TEXT]); - assert.equal(fw.toMembraneToolResult(call.id, call.result, 24_000, spilled.get(call.id)).content, UNCHECKED_RESULT_TEXT); + const logged: string[] = []; + const error = console.error; + console.error = (...args: unknown[]) => { logged.push(args.map(String).join(' ')); }; + try { + const { blocks, spilled } = await fw.buildStoredToolResultContent('zz-agent', [call], 24_000); + assert.deepEqual(blocks.map((block) => block.content), [UNCHECKED_RESULT_TEXT]); + assert.equal(fw.toMembraneToolResult(call.id, call.result, 24_000, spilled.get(call.id)).content, UNCHECKED_RESULT_TEXT); + } finally { + console.error = error; + } + // The stored copy's builder names the tool in the model lane's log line. + assert.ok(logged.some((line) => /tool 'zz_tool'.*Maximum call stack size exceeded/.test(line)), logged.join('\n')); }); it("keeps a result's native live copy when its turn's results are stored by their shares of the readers' bound", async () => { From 3aad97bcbdfde00e19da311693ca0662cee671b4 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 06:46:23 -0400 Subject: [PATCH 26/40] refactor(framework): a script's text route takes the serializer's own reading of the claims The removal variant text-route-notice survived at 7bd4f19e (by run): since historyJson throws where its claims walk stops (claimsInHistoryOrThrow), the text route's own resultClaimsOrThrow read, there so a stop failed the script rather than showing the notice, reads exactly what the serializer reads (contentClaims for a content array, the depth walk otherwise) and throws where it would. So the route passes the claims the fetches read, where they read any, and otherwise leaves the reading to the serializer; the variant unchecked-renders now runs against the script tests too. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index ecace646e..5b92c1ed3 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -22649,9 +22649,9 @@ export class AgentFramework { } } else { // As in history, a reference is its stub at any depth and inline data - // claiming a uri or a disposition is withheld (RFC-005), by the - // result's claims, read here where no fetch read them, a stop in their - // walk failing as a stopped render does. + // claiming a uri or a disposition is withheld (RFC-005), by the claims + // the fetches read, or else the serializer's own reading of them, + // whose walk throws where it stops, failing as a stopped render does. try { text = isError ? result.error ?? 'tool call failed' @@ -22659,7 +22659,7 @@ export class AgentFramework { ? nonErrorResultTextOrThrow(result) : toolResultDataToHistoryString(result.data, undefined, { dataIsContent: Boolean(result.dataIsContent), - claims: claims ?? resultClaimsOrThrow(result.data, result.dataIsContent), + ...(claims ? { claims } : {}), }); } catch (err) { return failed(err); From 5147cb132dbd981f4e0cbfd0af31ff65cf20f970 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 07:19:16 -0400 Subject: [PATCH 27/40] fix(references): review fixes: a registry journal for the line's records, the shown text well formed The calls on the review's findings: - A script's link object decides its uri by the model's line, which reads each record as it stood before the script lane registered anything. lineClaimsBeforeTreatment copied only the testimony claims of the uris the view claims, merged into the line's result-wide claims, so a link-only uri's record evicted by the treatment's own registrations came back fresh without an earlier never, an eviction lost the record's server binding, and an earlier never on one uri reached a cut-alike link through neverShown. In its place, the registry's journal (openJournal): each open keeps its own copies, a record copied, servers included, the first time a registration changes, creates or evicts it, and close lets them go. The line is a ScriptLine, its claims (the result's own) and recordBefore (the journal's before); a uri with no record before reads as the record the link's own registration makes. The module route opens it before its demand fetch and closes it in a finally after the treatment; the peer route around its treatment. - shownUri takes .toWellFormed(), so a never matches the text every copy shows, a lone surrogate as U+FFFD. Texts: the dispatch's comment and a test name say the text route's fetch passes over what each top-level block's own expiry marks, as a model's call does, and that no fetch reads a never; an error result's own field and the guard's kept fields throw as on main, in the changelog, the body and the notice's doc; the notice's doc no longer says the text route reads the claims first; a script never sees a withheld uri through a result's blocks, structured content going as sent; isError; the host's record holds the latest reference until its bytes are saved; the label cut named. Tests: a registry journal through a change, a second server, an eviction, two overlapping opens and a close (mcpl-reference-fetcher); a link's uri through an eviction 2,001 nested references make, peer and module; a module's nested link keeping an earlier never in both orders; an earlier never on one uri not withholding a cut-alike link the line shows; a view-only never withholding a cut-alike link; the shown-text never both ways, for a bidi mark and a lone surrogate; the order, read-once and inline-expiry tests checking the view was written; the notice's words pinned once. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 6 +- src/framework.ts | 177 ++++++++++++---------- src/mcpl/references.ts | 102 +++++++++---- src/tool-result-history.ts | 22 +-- test/mcpl-reference-fetcher.test.ts | 40 ++++- test/script-structured-references.test.ts | 94 +++++++++++- test/tool-result-blocks.test.ts | 51 +++++-- 7 files changed, 353 insertions(+), 139 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index bee056895..98a8078e2 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,5 +1,5 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text once the controls a label drops are cut, or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host last recorded it, did, since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. -- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads, so a script never sees a uri the model's line withholds. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cut drops controls and bidi marks, trims surrounding space and stops at 4,096 characters, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as it stood before the script was given the result, so a script never sees, through a result's blocks, a uri the model's line withholds; a result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. - A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. -- A tool result's data that can't be read through to apply RFC-005's rules, or shown once it has been, shows `[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]` on the model's lane, in the stored and sent copies alike, with a log line naming the tool and the error. It was shown with none of its references' claims where the walk reading them stopped and the render's didn't, or, for data no JSON can hold, such as a cycle or a BigInt, a getter that throws, or data too deep to write, the copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception, with the tool-result guard off; with it on, its archive of the result still throws, as before. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. +- A tool result's data that can't be read through to apply RFC-005's rules, or shown once it has been, shows `[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]` on the model's lane, in the stored and sent copies alike, with a log line naming the tool and the error. It was shown with none of its references' claims where the walk reading them stopped and the render's didn't, or, for data no JSON can hold, such as a cycle or a BigInt, a getter that throws, or data too deep to write, the copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception, with the tool-result guard off; with it on, its archive of the result, and the fields it keeps of a result it withholds, still throw, as before, as does an error result whose own error field can't be read. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. diff --git a/src/framework.ts b/src/framework.ts index 5b92c1ed3..ec731b254 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -1,7 +1,7 @@ import { ToolPresentation, presentationTools, isPresentationTool, renderCatalogue, type PresentationSnapshot } from "./tool-presentation.js"; import { constants as bufferConstants } from 'node:buffer'; import { dirname, join } from 'node:path'; -import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, isReferenceExpired, lineClaimsBeforeTreatment, referenceBlockForScript, referenceRegistry, sanitizeLabel, type ResultClaims, type ServerAddress } from './mcpl/references.js'; +import { EXPIRY_REASON, REFERENCE_LIMITS, claimedExpiry, classifyBlock, isReferenceExpired, referenceBlockForScript, referenceRegistry, sanitizeLabel, type RegistryJournal, type ResultClaims, type ScriptLine, type ServerAddress } from './mcpl/references.js'; import { ReferenceFetcher, DEFAULT_FETCH_MAX_BYTES, EAGER_FETCH_TIMEOUT_MS } from './mcpl/reference-fetcher.js'; import { appendFileSync, closeSync, existsSync, fsyncSync, mkdirSync, openSync, readFileSync, renameSync, unlinkSync, @@ -22590,93 +22590,105 @@ export class AgentFramework { // Where the walk stops, the script's answer is a // failure naming the tool, before any fetch or treatment, so neither runs // without them (Ada-1017, room-203 #200250, #200374): on both of a - // script's routes, `is_error` says nothing could be shown, where the + // script's routes, `isError` says nothing could be shown, where the // model's lane shows UNCHECKED_RESULT_TEXT. let claims: ResultClaims | undefined; - // The claims the model's line for the result reads, which decide where a - // link's object shows its uri (referenceBlockForScript's lineClaims). - let lineClaims: ResultClaims | undefined; - if ((!isError || structured) && Array.isArray(result.data) && fromModule) { - try { - lineClaims = resultClaimsOrThrow(result.data, result.dataIsContent); - claims = showsBlocks ? scriptViewClaims(result.data, false) : lineClaims; - if (showsBlocks) lineClaims = lineClaimsBeforeTreatment(lineClaims, claims); - } catch (err) { - return failed(err); + // The model's line for the result, its claims and its records as it reads + // them, which decide where a link's object shows its uri + // (referenceBlockForScript's line). Its records are the registry's as + // they stood before this lane registered anything (openJournal): the + // demand fetch below and the treatment register as they go, which the + // line never sees (Ada-1017, room-203 #202683). + let line: ScriptLine | undefined; + let journal: RegistryJournal | undefined; + try { + if ((!isError || structured) && Array.isArray(result.data) && fromModule) { + try { + const lineClaims = resultClaimsOrThrow(result.data, result.dataIsContent); + claims = showsBlocks ? scriptViewClaims(result.data, false) : lineClaims; + if (showsBlocks) { + journal = referenceRegistry.openJournal(); + line = { claims: lineClaims, recordBefore: journal.before }; + } + } catch (err) { + return failed(err); + } + const fetcher = this.getReferenceFetcher(); + for (const block of result.data) { + const c = classifyBlock(block); + if (c.kind !== 'reference' || c.testimony.fromLink) continue; + // Registered, not only found, so a re-issued reference's claims + // replace a link's, or a stale reference's, before the fetch reads + // them, as on any other path, while the record isn't fetched yet. A + // record its uri already has keeps the server it's bound to. + const record = referenceRegistry.register(c.testimony); + // A uri any of the result's references gives an expiry that counts + // isn't fetched, as autoFetchReferences passes over one that any block + // of an MCPL result marks expired. + const expired = record.fetchedPath ? null : claimedExpiry(record, claims, Date.now()); + if (expired !== null) { + console.error(`[rfc005] script-demand fetch of ${record.refId} not made: by the result's claims, the reference is ${EXPIRY_REASON[expired]}`); + continue; + } + const outcome = await fetcher.fetch(record); + if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); + } } - const fetcher = this.getReferenceFetcher(); - for (const block of result.data) { - const c = classifyBlock(block); - if (c.kind !== 'reference' || c.testimony.fromLink) continue; - // Registered, not only found, so a re-issued reference's claims - // replace a link's, or a stale reference's, before the fetch reads - // them, as on any other path, while the record isn't fetched yet. A - // record its uri already has keeps the server it's bound to. - const record = referenceRegistry.register(c.testimony); - // A uri any of the result's references gives an expiry that counts - // isn't fetched, as autoFetchReferences passes over one that any block - // of an MCPL result marks expired. - const expired = record.fetchedPath ? null : claimedExpiry(record, claims, Date.now()); - if (expired !== null) { - console.error(`[rfc005] script-demand fetch of ${record.refId} not made: by the result's claims, the reference is ${EXPIRY_REASON[expired]}`); - continue; + // An MCPL peer's result whose blocks JSON couldn't hold, so that RFC-005's + // treatment couldn't be given them at dispatch. + if (SCRIPT_RESULT_UNCONVERTIBLE.has(result)) return failed(SCRIPT_RESULT_UNCONVERTIBLE.get(result)); + let content: unknown[]; + let text: string | undefined; + if (showsBlocks) { + content = Array.isArray(result.data) ? result.data : []; + // A module's blocks, such as an MCPL peer's it got from ctx.callTool + // and returns whole, are read as history reads a module's data: what + // RFC-005 governs, at any depth, as the script lane shows it, and + // anything else as sent. What a script prints comes back to the model, + // so no uri the model's lanes withhold reaches it through the blocks + // (structured content goes as sent, as on main). What this gives is + // a stub, a notice or the value itself, so a module that returns + // blocks it already gave has them left as they are. + if (fromModule) { + try { + content = JSON.parse(JSON.stringify(content, (_key, value: unknown) => shownToScript(value, claims, line))) as unknown[]; + } catch (err) { + return failed(err); + } } - const outcome = await fetcher.fetch(record); - if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); - } - } - // An MCPL peer's result whose blocks JSON couldn't hold, so that RFC-005's - // treatment couldn't be given them at dispatch. - if (SCRIPT_RESULT_UNCONVERTIBLE.has(result)) return failed(SCRIPT_RESULT_UNCONVERTIBLE.get(result)); - let content: unknown[]; - let text: string | undefined; - if (showsBlocks) { - content = Array.isArray(result.data) ? result.data : []; - // A module's blocks, such as an MCPL peer's it got from ctx.callTool - // and returns whole, are read as history reads a module's data: what - // RFC-005 governs, at any depth, as the script lane shows it, and - // anything else as sent. What a script prints comes back to the model, - // so no uri the model's lanes withhold reaches it through the blocks - // (structured content goes as sent, as on main). What this gives is - // a stub, a notice or the value itself, so a module that returns - // blocks it already gave has them left as they are. - if (fromModule) { + } else { + // As in history, a reference is its stub at any depth and inline data + // claiming a uri or a disposition is withheld (RFC-005), by the claims + // the fetches read, or else the serializer's own reading of them, + // whose walk throws where it stops, failing as a stopped render does. try { - content = JSON.parse(JSON.stringify(content, (_key, value: unknown) => shownToScript(value, claims, lineClaims))) as unknown[]; + text = isError + ? result.error ?? 'tool call failed' + : result.data === undefined || result.data === null + ? nonErrorResultTextOrThrow(result) + : toolResultDataToHistoryString(result.data, undefined, { + dataIsContent: Boolean(result.dataIsContent), + ...(claims ? { claims } : {}), + }); } catch (err) { return failed(err); } + content = [{ type: 'text', text }]; } - } else { - // As in history, a reference is its stub at any depth and inline data - // claiming a uri or a disposition is withheld (RFC-005), by the claims - // the fetches read, or else the serializer's own reading of them, - // whose walk throws where it stops, failing as a stopped render does. + let json: string; try { - text = isError - ? result.error ?? 'tool call failed' - : result.data === undefined || result.data === null - ? nonErrorResultTextOrThrow(result) - : toolResultDataToHistoryString(result.data, undefined, { - dataIsContent: Boolean(result.dataIsContent), - ...(claims ? { claims } : {}), - }); + json = JSON.stringify({ + content, + structuredContent: structured && result.structured !== undefined ? result.structured : null, + isError, + }); } catch (err) { return failed(err); } - content = [{ type: 'text', text }]; - } - let json: string; - try { - json = JSON.stringify({ - content, - structuredContent: structured && result.structured !== undefined ? result.structured : null, - isError, - }); - } catch (err) { - return failed(err); + return { ok: true, json, isError, brief: text ?? scriptResultBrief(content) }; + } finally { + journal?.close(); } - return { ok: true, json, isError, brief: text ?? scriptResultBrief(content) }; } /** @@ -29094,29 +29106,35 @@ export class AgentFramework { }; let claims: ResultClaims; // The model's line for a peer's content reads its blocks' claims - // alone, which decide where a link's object shows its uri. + // alone, and each record as it stood before the treatment registered + // anything (openJournal), which decide where a link's object shows + // its uri (Ada-1017, room-203 #202683). let lineClaims: ResultClaims; try { // As the dispatch read them for its demand fetch, where it did. const read = SCRIPT_VIEW_CLAIMS.get(direct.data); if (read && 'error' in read) throw read.error; claims = read ? read.claims : scriptViewClaims(direct.data, true); - lineClaims = lineClaimsBeforeTreatment(resultClaimsOrThrow(direct.data, true), claims); + lineClaims = resultClaimsOrThrow(direct.data, true); } catch (err) { return unconvertible(err); } - const blocks = direct.data.map((block: unknown) => referenceBlockForScript(block, serverId, claims, lineClaims)); + const journal = referenceRegistry.openJournal(); try { + const line: ScriptLine = { claims: lineClaims, recordBefore: journal.before }; + const blocks = direct.data.map((block: unknown) => referenceBlockForScript(block, serverId, claims, line)); // The top-level blocks were treated just above; the pass treats // what's nested in them. Reading a treated block again would take // a link object's uri, as sanitizeLabel cut it, for a new link, // registered apart from the one the result named. const nested = function (this: unknown, _key: string, value: unknown): unknown { - return this === blocks ? value : shownToScriptFromServer(value, serverId, claims, lineClaims); + return this === blocks ? value : shownToScriptFromServer(value, serverId, claims, line); }; return { ...direct, data: JSON.parse(JSON.stringify(blocks, nested)) as unknown[] }; } catch (err) { return unconvertible(err); + } finally { + journal.close(); } } // A standard peer's content that isn't a list of blocks is a malformed @@ -29210,7 +29228,8 @@ export class AgentFramework { let claims: ResultClaims | undefined; // Only where the script reads a view of the blocks (reading()'s own // condition): without structured content it reads the model lane's - // text, whose fetch reads each block's own claims, as the text does. + // text, and its fetch passes over what each top-level block's own + // expiry marks, as a model's call does. No fetch reads a `never`. if (scriptReads && Object.prototype.hasOwnProperty.call(result, 'structuredContent') && Array.isArray(result.content)) { try { claims = scriptViewClaims(result.content, true); diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index c852b5e4f..7e5140dad 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -582,8 +582,38 @@ export class ReferenceRegistry { * its own origin, for a uri it delivered. */ private byKey = new Map(); private counter = 0; + /** Each open journal's copies (openJournal), by uri. */ + private journals = new Set>(); + + /** + * Keep, until `close`, each record as it stood when this opened: a copy + * taken the first time a registration changes, creates or evicts it, or + * null where the uri had none. `before(uri)` gives that record, or the + * record as it is where nothing registered has touched it since, which is + * the same. A fetch that saves a copy in the meantime sets its record's + * saved path without registering, so that is read as it is; a fetch only + * saves what its checks passed. Each open has its own copies, so lanes + * that overlap keep theirs apart, and closing one lets its copies go. + */ + openJournal(): RegistryJournal { + const copies = new Map(); + this.journals.add(copies); + return { + before: (uri) => (copies.has(uri) ? copies.get(uri) ?? undefined : this.findByUri(uri)), + close: () => { this.journals.delete(copies); }, + }; + } + + /** Copy a uri's record into each open journal that has none for it yet. */ + private keepFirst(uri: string): void { + if (this.journals.size === 0) return; + const record = this.findByUri(uri); + const copy = record ? { ...record, serverIds: [...record.serverIds] } : null; + for (const copies of this.journals) if (!copies.has(uri)) copies.set(uri, copy); + } register(testimony: ReferenceTestimony, serverId?: string): ReferenceRecord { + this.keepFirst(testimony.uri); const existingId = this.byKey.get(testimony.uri); if (existingId !== undefined) { const existing = this.records.get(existingId); @@ -611,6 +641,7 @@ export class ReferenceRegistry { const oldest = this.records.keys().next().value; if (oldest !== undefined) { const old = this.records.get(oldest); + if (old) this.keepFirst(old.testimony.uri); this.records.delete(oldest); if (old) this.byKey.delete(old.testimony.uri); } @@ -663,6 +694,23 @@ export class ReferenceRegistry { export const referenceRegistry = new ReferenceRegistry(); +/** A registry's records as they stood when it opened (openJournal). */ +export interface RegistryJournal { + before(uri: string): ReferenceRecord | undefined; + close(): void; +} + +/** + * The model's line for a result as a script's link object reads it: the + * claims that line reads, and each record as it stood when the script lane + * began to treat the result (a journal's `before`), whatever the treatment + * has registered, rebound or evicted since, which the line never sees. + */ +export interface ScriptLine { + claims: ResultClaims; + recordBefore(uri: string): ReferenceRecord | undefined; +} + // ============================================================================ // Stub building (§5) // ============================================================================ @@ -728,7 +776,9 @@ export interface StubOptions { export type ResultClaims = ReadonlyMap; /** - * A uri as a link shows it, cut of what sanitizeLabel cuts. A `never` is + * A uri as a link shows it: cut of what sanitizeLabel cuts (controls and bidi + * marks, surrounding space, past REFERENCE_LIMITS.uri), and well formed, as + * every copy is before it's shown, a lone surrogate showing as U+FFFD. A `never` is * about what a link shows, so it's found by this text (neverShown): a claim * of `never` on a uri withholds every link that shows the same text, two * uris that cut alike sharing it. Every other claim is about the record, @@ -736,7 +786,7 @@ export type ResultClaims = ReadonlyMap; * room-203 #202239). */ export function shownUri(uri: string): string { - return sanitizeLabel(uri, REFERENCE_LIMITS.uri); + return sanitizeLabel(uri, REFERENCE_LIMITS.uri).toWellFormed(); } const NEVER_SHOWN = new WeakMap>(); @@ -818,28 +868,6 @@ export function inlineNeverUri(block: unknown, c: BlockClassification = classify return b.disposition === 'never' && typeof b.uri === 'string' && b.uri !== '' ? b.uri : null; } -/** - * The claims the model's line for a result reads (`line`), for a script's - * link objects to judge their uri by (referenceBlockForScript's lineClaims), - * with, for each uri the script's view claims (`view`) and the line doesn't, - * that uri's record's own claims as they stand now, before the treatment: - * the line falls back to the record there, and the treatment registers what - * it shows as it goes, so a reference nested before a link would otherwise - * change what the link reads (an expiry, or a plain reference that clears an - * earlier result's never), and the link's uri would turn on the order of - * the blocks. - */ -export function lineClaimsBeforeTreatment(line: ResultClaims, view: ResultClaims): ResultClaims { - const frozen = new Map(line); - for (const uri of view.keys()) { - if (frozen.has(uri)) continue; - const record = referenceRegistry.findByUri(uri); - frozen.set(uri, (record ? resultClaims([record.testimony]).get(uri) : undefined) - ?? { never: false, rejected: false, unparseable: false, earliestMs: null }); - } - return frozen; -} - /** The claims a decision about a record reads: its result's for its uri, * where the result's references name it, and otherwise the record's own * testimony. */ @@ -1014,7 +1042,7 @@ export function referenceStubOrNull(block: unknown, provenance?: string, serverI * empty, the withheld notice, each as a text block. * A text block, or inline data that claims nothing, is returned as sent. */ -export function referenceBlockForScript(block: unknown, serverId?: string, claims?: ResultClaims, lineClaims?: ResultClaims): unknown { +export function referenceBlockForScript(block: unknown, serverId?: string, claims?: ResultClaims, line?: ScriptLine): unknown { const c = classifyBlock(block); switch (c.kind) { case 'text': return block; @@ -1039,14 +1067,22 @@ export function referenceBlockForScript(block: unknown, serverId?: string, claim // with each other across an expiry. const nowMs = Date.now(); // A link's uri where the model's line for the result shows it, by the - // claims the line reads (lineClaims, where they differ from the view's), - // and never where the view's claims withhold it: reading more claims - // can loosen as well as tighten, since an expiry makes a link one the - // host can't fetch, whose uri the line shows (Ada-1017, room-203 - // #201013). A script never sees a uri the model's line withholds, - // through its blocks; structured content goes as sent, as on main. - const lineShows = shownLinkUri(t, record, nowMs, lineClaims ?? claims); - const uri = lineShows !== null && lineClaims !== undefined && claimedNever(record, claims) ? null : lineShows; + // claims the line reads and its record as the line reads it (`line`, + // where they differ from the view's), and never where the view's + // claims withhold it: reading more claims can loosen as well as + // tighten, since an expiry makes a link one the host can't fetch, whose + // uri the line shows (Ada-1017, room-203 #201013). The line's record is + // the one before the script lane registered anything (#202683), since + // the treatment registers, rebinds and evicts as it goes, which the + // line never sees; where there was none, it's the one the link's own + // registration makes. A script never sees a uri the model's line + // withholds, through its blocks; structured content goes as sent, as + // on main. + const lineRecord = line + ? line.recordBefore(t.uri) ?? { refId: record.refId, serverIds: serverId === undefined ? [] : [serverId], testimony: t, receivedAt: record.receivedAt } + : record; + const lineShows = shownLinkUri(t, lineRecord, nowMs, line ? line.claims : claims); + const uri = lineShows !== null && line !== undefined && claimedNever(record, claims) ? null : lineShows; // Why a fetch_reference with this object's refId would be refused, in // the words a stub gives, whether or not the model's line for it is a // stub: the line has no id to fetch with, and the object does. As the diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index 6bd98899f..fd8642363 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -38,6 +38,7 @@ import { resultClaims, type ReferenceTestimony, type ResultClaims, + type ScriptLine, } from './mcpl/references.js'; import { formatToolImagePlaceholder, toolResultCutAt } from './tool-image-ledger.js'; @@ -538,19 +539,20 @@ function claimsInHistoryOrThrow(data: unknown): ResultClaims { * withheld notice, each as a text block. A null, absent or empty uri claims * none (hasUri), so inline data beside one is itself. Any other value is * itself. `claims` are those of everything the script's view shows - * (scriptViewClaims), and `lineClaims` those the model's line for the - * result reads, which decide where a link's uri is shown. What it gives is either a stub or a notice, which + * (scriptViewClaims), and `line` the model's line for the result, its claims + * and its records as it reads them (ScriptLine), which decide where a link's + * uri is shown. What it gives is either a stub or a notice, which * RFC-005 doesn't govern, or the value itself, which it gives again, so * applying it twice gives what applying it once gave. */ -export function shownToScript(value: unknown, claims?: ResultClaims, lineClaims?: ResultClaims): unknown { +export function shownToScript(value: unknown, claims?: ResultClaims, line?: ScriptLine): unknown { if (!value || typeof value !== 'object' || Array.isArray(value)) return value; const b = value as Record; if (!governedByRfc005(b)) return value; if ((b.type === 'image' || b.type === 'audio') && typeof b.data === 'string') { return hasUri(b.uri) || b.disposition !== undefined ? { type: 'text', text: INLINE_WITHHELD_TEXT } : value; } - return referenceBlockForScript(b, undefined, claims, lineClaims); + return referenceBlockForScript(b, undefined, claims, line); } /** @@ -562,7 +564,7 @@ export function shownToScript(value: unknown, claims?: ResultClaims, lineClaims? * shown only where the model's line shows it, by the claims the line reads * (contentClaims, which reads a block and not its fields): reading more * claims can loosen as well as tighten, since an expiry makes a link one the - * host can't fetch (referenceBlockForScript's `lineClaims`). An MCPL peer's + * host can't fetch (referenceBlockForScript's `line`). An MCPL peer's * top-level block the classifier rejects is * its notice, its fields unshown, so they don't count (`fromServer`). It * throws what stops the walk, such as a value too deep to convert or a @@ -586,10 +588,10 @@ export function scriptViewClaims(content: readonly unknown[], fromServer: boolea * governed, so a block the top-level pass already treated is left as it is. * `claims` are those of the script's whole view (scriptViewClaims). */ -export function shownToScriptFromServer(value: unknown, serverId: string, claims?: ResultClaims, lineClaims?: ResultClaims): unknown { +export function shownToScriptFromServer(value: unknown, serverId: string, claims?: ResultClaims, line?: ScriptLine): unknown { if (!value || typeof value !== 'object' || Array.isArray(value)) return value; const b = value as Record; - return governedByRfc005(b) ? referenceBlockForScript(b, serverId, claims, lineClaims) : value; + return governedByRfc005(b) ? referenceBlockForScript(b, serverId, claims, line) : value; } /** @@ -616,8 +618,10 @@ export function rfc005TextOf(value: unknown): string | null { * run); so does a render that stops after the walk, or a getter that throws * (Ada-1017, room-203 #200344, #201013). The round completes, as a withheld * result's kept fields keep nothing then (keptWhenWithheld). A script never - * gets it: its routes read the claims first and call the serializer, or - * nonErrorResultTextOrThrow, directly, and fail naming the tool (#200374). + * gets it: its routes call the serializer, or nonErrorResultTextOrThrow, + * directly, whose walk throws where it stops, and fail naming the tool + * (#200374). An error result's own error field is read as before, and so + * are the fields the tool-result guard keeps of a result it withholds. */ export const UNCHECKED_RESULT_TEXT = "[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]"; diff --git a/test/mcpl-reference-fetcher.test.ts b/test/mcpl-reference-fetcher.test.ts index a8d3f6091..42b1e22f4 100644 --- a/test/mcpl-reference-fetcher.test.ts +++ b/test/mcpl-reference-fetcher.test.ts @@ -9,7 +9,7 @@ import assert from 'node:assert/strict'; import { createServer } from 'node:http'; import { createHash } from 'node:crypto'; import { DEFAULT_FETCH_MAX_BYTES, ReferenceFetcher, normalizedOrigin, type ServerResolution } from '../src/mcpl/reference-fetcher.js'; -import { EXPIRY_REASON, noAddressReason, ReferenceRegistry, UNBOUND_REASON, classifyBlock, referenceExpiry, referenceRegistry, shownServerId } from '../src/mcpl/references.js'; +import { EXPIRY_REASON, noAddressReason, ReferenceRegistry, UNBOUND_REASON, classifyBlock, referenceExpiry, referenceRegistry, shownServerId, type ReferenceTestimony } from '../src/mcpl/references.js'; import { AgentFramework } from '../src/framework.js'; const PAYLOAD = Buffer.from('RIFF-not-really-a-wav-'.repeat(100)); @@ -103,6 +103,44 @@ test('happy path: origin-bound fetch, bearer auth applied host-side, digest veri assert.equal(again.path, out.path); }); +test("a registry journal keeps each record as it stood when it opened, through a change, a rebinding and an eviction, each open its own copies", () => { + const reg = new ReferenceRegistry(); + const t = (uri: string, extra: Record = {}): ReferenceTestimony => { + const c = classifyBlock({ type: 'resource', uri, name: 'x.csv', ...extra }); + assert.equal(c.kind, 'reference'); + return (c as { testimony: ReferenceTestimony }).testimony; + }; + const u = 'https://zz-journal.example/u.csv'; + const w = 'https://zz-journal.example/w.csv'; + reg.register(t(u, { disposition: 'never' }), 'first'); + reg.register(t(w)); + const one = reg.openJournal(); + // A later testimony and a second server, then a second journal, which + // takes the record as it stands then, and a third change neither sees. + reg.register(t(u), 'second'); + const two = reg.openJournal(); + reg.register(t(u, { expiresAt: '2000-01-01T00:00:00Z' }), 'third'); + assert.deepEqual([one.before(u)?.testimony.disposition, one.before(u)?.serverIds], ['never', ['first']]); + assert.deepEqual([two.before(u)?.testimony.disposition, two.before(u)?.testimony.expiresAt, two.before(u)?.serverIds], [undefined, undefined, ['first', 'second']]); + assert.deepEqual(reg.findByUri(u)?.serverIds, ['first', 'second', 'third']); + // A uri with no record when a journal opened has none before; one nothing + // has touched since reads as it is. + const v = 'https://zz-journal.example/v.csv'; + reg.register(t(v)); + assert.equal(one.before(v), undefined); + assert.equal(one.before(w), reg.findByUri(w)); + // A closed journal copies nothing more. + one.close(); + reg.register(t(w, { expiresAt: '2000-01-01T00:00:00Z' })); + assert.equal(one.before(w)?.testimony.expiresAt, '2000-01-01T00:00:00Z'); + // An eviction: the registry holds 2,000 records, so 2,001 more take u's, + // which the open journal still has as it stood. + for (let i = 0; i < 2001; i++) reg.register(t(`https://zz-journal.example/many/${i}.csv`)); + assert.equal(reg.findByUri(u), undefined); + assert.deepEqual([two.before(u)?.testimony.disposition, two.before(u)?.serverIds], [undefined, ['first', 'second']]); + two.close(); +}); + test('vector 5: digest mismatch is never presented', async () => { const reg = new ReferenceRegistry(); const f = makeFetcher({ srv: { url: ORIGIN } }); diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index e1ce59ad7..d9a1ba900 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -727,7 +727,7 @@ test("an MCPL peer's script-demand fetch reads the view's claims: a nested expir }); test("a nested link's uri doesn't turn on the order of the blocks: a reference nested before it doesn't change what its line reads", { timeout: 30_000 }, async () => { - await withPeers(async ({ answer, script }) => { + await withPeers(async ({ answer, script, exit }) => { // Fetchable, so the line, reading no claim of this result for the uri, // shows no uri. A nested expired reference registered before the link, // as the treatment's own pass registers what it shows, would make the @@ -744,6 +744,8 @@ test("a nested link's uri doesn't turn on the order of the blocks: a reference n answer({ content, structuredContent: {} }); const fromPeer = await script('mcpl--files--attach'); assert.ok(!fromPeer.includes(`block-order-${i}`), `${order.join(' then ')}: ${fromPeer}`); + const view = JSON.parse(fromPeer) as { content: Array>>; isError: boolean }; + assert.deepEqual([view.isError, typeof view.content[0]!.link!.refId, 'uri' in view.content[0]!.link!], [false, 'string', false], fromPeer); } // An earlier result's never, which a later result's plain reference, // nested before the link, would clear on the record. @@ -757,10 +759,73 @@ test("a nested link's uri doesn't turn on the order of the blocks: a reference n // never no longer holds there; the line, which shows neither, reads the // record as it stood, and withholds the uri; so does the script. assert.ok(!out.includes('order-never'), out); + const view = JSON.parse(out) as { content: Array>>; isError: boolean }; + assert.deepEqual([view.isError, typeof view.content[0]!.link!.refId], [false, 'string'], out); + // A module's result the same way, for a uri of its own, in both orders. + for (const [i, nestedFirst] of [true, false].entries()) { + const moduleUri = `https://elsewhere.example.invalid/order-never-module-${i}/report.csv`; + answer({ content: [{ type: 'resource', uri: moduleUri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + const plain = { type: 'resource', uri: moduleUri, name: 'report.csv' }; + const link = { type: 'resource_link', uri: moduleUri, name: 'report.csv' }; + const block = nestedFirst ? { type: 'text', text: 'again', plain, link } : { type: 'text', text: 'again', link, plain }; + const fromModule = await exit({ success: true, data: [block], structured: {} }, 'fwd--whole', false); + assert.ok(!fromModule.includes(`order-never-module-${i}`), fromModule); + const moduleView = JSON.parse(fromModule) as { content: Array>>; isError: boolean }; + assert.deepEqual([moduleView.isError, typeof moduleView.content[0]!.link!.refId], [false, 'string'], fromModule); + } + }); +}); + +test("a link's uri reads its record as it stood before the script lane registered anything, through an eviction the treatment's own registrations make", { timeout: 60_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + for (const route of ['peer', 'module'] as const) { + // An earlier result marks the uri never; off the server's origin, so a + // link to it with no such claim shows its uri. + const uri = `https://elsewhere.example.invalid/evicted-${route}/report.csv`; + answer({ content: [{ type: 'resource', uri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + // More references nested before the link than the registry holds, so + // the treatment evicts the uri's record before it reaches the link. + const many = Array.from({ length: 2001 }, (_, i) => ({ type: 'resource', uri: `https://files.example.invalid/many-${route}/${i}.csv`, name: `${i}.csv` })); + const content = [{ type: 'text', text: 'many attachments', many, link: { type: 'resource_link', uri, name: 'report.csv' } }]; + let out: string; + if (route === 'peer') { + answer({ content, structuredContent: {} }); + out = await script('mcpl--files--attach'); + } else { + out = await exit({ success: true, data: content, structured: {} }, 'fwd--whole', false); + } + assert.ok(!out.includes(`evicted-${route}/report.csv`), `${route}: ${out.slice(-600)}`); + const view = JSON.parse(out) as { content: Array>>; isError: boolean }; + assert.deepEqual([view.isError, typeof view.content[0]!.link!.refId], [false, 'string'], route); + } + }); +}); + +test("an earlier result's never on one uri doesn't withhold, from a script, a link to another that shows as the same text, where the model's line shows it", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script, modelCall }) => { + // Each record speaks for its own uri: the earlier never is the marked + // uri's, and the link names the clean one, whose record has none. + const marked = 'https://elsewhere.example.invalid/cut-history/\u200Freport.csv'; + const clean = 'https://elsewhere.example.invalid/cut-history/report.csv'; + answer({ content: [{ type: 'resource', uri: marked, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + const content = () => [ + { type: 'text', text: 'again', attachment: { type: 'resource', uri: marked, name: 'report.csv' } }, + { type: 'resource_link', uri: clean, name: 'report.csv' }, + ]; + answer({ content: content(), structuredContent: {} }); + const line = nonErrorResultText(await modelCall('attach')); + assert.ok(line.includes('cut-history/report.csv'), line); + answer({ content: content(), structuredContent: {} }); + const out = await script('mcpl--files--attach'); + const link = (JSON.parse(out) as { content: Array> }).content[1]!; + assert.equal(link.uri, clean, out); }); }); -test("an MCPL peer's result a script reads as text has its demand fetch read each block, as the text does, not a view it never gets", async () => { +test("an MCPL peer's result a script reads as text has its demand fetch read each top-level block's own expiry, as a model's call does, not a view it never gets", async () => { await withPeers(async ({ answer, script, fetched }) => { // No structured content: the script reads the model lane's text, whose // stubs read the top-level blocks' claims, so a reference nested in a @@ -787,6 +852,8 @@ test("an MCPL peer's view reads the claims its dispatch read for the demand fetc const out = await script('mcpl--files--attach'); assert.equal(fetched.length, 1, 'the demand fetch ran between the two readings'); assert.ok(!out.includes('read-once/report.csv'), out); + const view = JSON.parse(out) as { content: Array>; isError: boolean }; + assert.deepEqual([view.isError, view.content.length, typeof view.content[2]!.refId], [false, 3, 'string'], out); }); }); @@ -806,6 +873,29 @@ test("inline data's expiry claims nothing for a link to its uri, which the host answer({ content: content(), structuredContent: {} }); const out = await script('mcpl--files--attach'); assert.ok(!out.includes('inline-expiry'), out); + const view = JSON.parse(out) as { content: Array>; isError: boolean }; + assert.deepEqual([view.isError, typeof view.content[1]!.refId], [false, 'string'], out); + }); +}); + +test("a never nested where only the script's view reads it withholds a link whose uri shows as the same text", { timeout: 30_000 }, async () => { + await withPeers(async ({ answer, script, modelCall }) => { + // Off the server's origin, so the line, which reads the top-level blocks + // alone, shows the link's uri; the view reads the nested never, found by + // the text the link shows. + const uri = 'https://elsewhere.example.invalid/view-cut/report.csv'; + const content = () => [ + { type: 'text', text: 'sealed', attachment: { type: 'resource', uri, name: 'report.csv', disposition: 'never' } }, + { type: 'resource_link', uri: 'https://elsewhere.example.invalid/view-cut/\u200Freport.csv', name: 'report.csv' }, + ]; + answer({ content: content(), structuredContent: {} }); + const line = nonErrorResultText(await modelCall('attach')); + assert.ok(line.includes('view-cut/report.csv'), line); + answer({ content: content(), structuredContent: {} }); + const out = await script('mcpl--files--attach'); + assert.ok(!out.includes('view-cut'), out); + const link = (JSON.parse(out) as { content: Array> }).content[1]!; + assert.deepEqual(['uri' in link, typeof link.refId], [false, 'string'], out); }); }); diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 672f60a09..c3d0e9fb4 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1433,20 +1433,45 @@ describe('one answer for every block of a result', () => { assert.ok(toolResultDataToHistoryString([dated, link], undefined, { dataIsContent: true }).includes('inline-never')); }); + it("withholds a never uri from a link whose uri shows as the same text once well formed, a lone surrogate showing as U+FFFD", () => { + // Each side as the other shows it: a never with the replacement character + // and a link with a lone surrogate, and the other way round. + // Each pair at a path of its own, since the registry keeps each never. + for (const [i, [sealedEnd, linkEnd]] of [['\uFFFD', '\uD800'], ['\uD800', '\uFFFD']].entries()) { + const at = `https://zz-elsewhere.example/sur-never-${i}/`; + const sealed = { type: 'resource', uri: `${at}${sealedEnd}.csv`, name: 'report.csv', disposition: 'never' }; + const link = { type: 'resource_link', uri: `${at}${linkEnd}.csv`, name: 'report.csv' }; + const control = toolResultDataToHistoryString([link], undefined, { dataIsContent: true }); + assert.ok(control.includes(`sur-never-${i}`), control); + for (const shown of [ + toolResultDataToHistoryString([sealed, link], undefined, { dataIsContent: true }), + toolResultDataToHistoryString({ sealed, link }), + JSON.stringify(shownToScript(link, scriptViewClaims([sealed, link], false))), + ]) { + assert.ok(!shown.includes(`sur-never-${i}`), shown); + } + } + }); + it("withholds a never uri from a link whose uri shows as the same text, its bidi control cut", () => { - const uri = 'https://zz-elsewhere.example/cut-never/report.csv'; - const sealed = { type: 'resource', uri, name: 'report.csv', disposition: 'never' }; // The same uri with a right-to-left mark, which sanitizeLabel cuts, so - // the link would show exactly the text the never withholds. - const link = { type: 'resource_link', uri: 'https://zz-elsewhere.example/cut-never/\u200Freport.csv', name: 'report.csv' }; - const control = toolResultDataToHistoryString([link], undefined, { dataIsContent: true }); - assert.ok(control.includes('cut-never/report.csv'), control); - for (const shown of [ - toolResultDataToHistoryString([sealed, link], undefined, { dataIsContent: true }), - toolResultDataToHistoryString({ sealed, link }), - JSON.stringify(shownToScript(link, scriptViewClaims([sealed, link], false))), - ]) { - assert.ok(!shown.includes('cut-never'), shown); + // the link would show exactly the text the never withholds: the mark on + // the link's side, and on the never's, each pair at a path of its own, + // since the registry keeps each never. + for (const [i, markOnLink] of [true, false].entries()) { + const clean = `https://zz-elsewhere.example/cut-never-${i}/report.csv`; + const marked = `https://zz-elsewhere.example/cut-never-${i}/\u200Freport.csv`; + const sealed = { type: 'resource', uri: markOnLink ? clean : marked, name: 'report.csv', disposition: 'never' }; + const link = { type: 'resource_link', uri: markOnLink ? marked : clean, name: 'report.csv' }; + const control = toolResultDataToHistoryString([link], undefined, { dataIsContent: true }); + assert.ok(control.includes(`cut-never-${i}/report.csv`), control); + for (const shown of [ + toolResultDataToHistoryString([sealed, link], undefined, { dataIsContent: true }), + toolResultDataToHistoryString({ sealed, link }), + JSON.stringify(shownToScript(link, scriptViewClaims([sealed, link], false))), + ]) { + assert.ok(!shown.includes(`cut-never-${i}`), shown); + } } }); @@ -1473,6 +1498,8 @@ describe('one answer for every block of a result', () => { // What's thrown can be a value String() can't read itself; the line // still goes out, saying so. const unprintable = { get data(): unknown { throw Object.create(null); } }; + // The words themselves, which the agent reads, pinned here once. + assert.equal(UNCHECKED_RESULT_TEXT, "[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]"); assert.equal(nonErrorResultText(unprintable, undefined, { toolName: 'zz_unprintable' }), UNCHECKED_RESULT_TEXT); // A render that stops where the claims walk got through: the render // registers what it shows, and the claims walk registers nothing. From b9ca38c31babd3459f50b94c49586c91be348b53 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 07:21:36 -0400 Subject: [PATCH 28/40] docs(references): openJournal's doc says which saved paths a journal reads A nit from a read of 5147cb13: a fetch's saved path is read as it is only where no registration in the lane copied the record first; a module's demand fetch registers before it fetches, so its record reads as it stood, unsaved, as the model's line reads it. Comment only. Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 7e5140dad..33ca85f09 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -591,8 +591,11 @@ export class ReferenceRegistry { * null where the uri had none. `before(uri)` gives that record, or the * record as it is where nothing registered has touched it since, which is * the same. A fetch that saves a copy in the meantime sets its record's - * saved path without registering, so that is read as it is; a fetch only - * saves what its checks passed. Each open has its own copies, so lanes + * saved path without registering, so a record no registration has copied + * yet is read saved, and one a registration copied first, as a module's + * demand fetch registers before it fetches, is read as it stood, unsaved, + * as the model's line, which makes no demand fetch, reads it. A fetch saves + * only what its checks passed. Each open has its own copies, so lanes * that overlap keep theirs apart, and closing one lets its copies go. */ openJournal(): RegistryJournal { From 7b682f19f0141e7c86179ce2ce5f39d29f77ced7 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 07:53:29 -0400 Subject: [PATCH 29/40] fix(references, framework): review fixes: the module demand loop registers links in order, its journal after the loop, a copy per journal, the whole uri's credential The calls on the review's findings: - A module's demand loop registers every reference block in the result's order, links included, and fetches none of the links. Skipping links, the loop's registration of a later reference could evict, in a full registry, the record of a link before it that the model's own render keeps, an earlier never with it, and a script reading the result as text, which renders against the live registry, then saw the uri. - The loop's registrations are now the ones the model's line makes, so the module route's journal opens after the loop; it still keeps the line's records from everything the treatment registers, rebinds or evicts. Where the result's own reference evicts an earlier never before its link, script and line both show the uri, which is the registry's bound, named in the description. - keepFirst makes a copy for each open journal. - shownLinkUri's credential check reads the whole uri as well as the cut it shows, so userinfo whose @ falls past the 4,096-character cut still withholds the link's uri (older than these review fixes); the fragment says so. Also ReferenceRegistry.journalCount, which tests read to see each lane closed its journal; openJournal's doc says a copy is taken the first time a registration names the uri, whatever it changes, or evicts it, and which saved paths it reads; the peer route's comment names shownUri. Tests: a module's text route keeps an earlier never on a link before a later reference in a full registry; a module's view and the model's line both show a uri the result's own reference evicted the never of; a link's record read through an eviction keeps its server binding; the eviction test checks the eviction happened and the journals closed; two journals opened with no registration between keep distinct copies; a link whose userinfo falls past the cut shows no uri on any lane. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 4 +- src/framework.ts | 34 +++++++--- src/mcpl/references.ts | 46 ++++++++----- test/mcpl-reference-fetcher.test.ts | 13 ++++ test/script-structured-references.test.ts | 81 ++++++++++++++++++++++- test/tool-result-blocks.test.ts | 16 +++++ 6 files changed, 162 insertions(+), 32 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 98a8078e2..8021a96b5 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,5 +1,5 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cut drops controls and bidi marks, trims surrounding space and stops at 4,096 characters, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. -- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as it stood before the script was given the result, so a script never sees, through a result's blocks, a uri the model's line withholds; a result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. The check reads the whole uri as well as the text a link would show, so userinfo past where that text is cut, at 4,096 characters, still withholds it. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cut drops controls and bidi marks, trims surrounding space and stops at 4,096 characters, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as that line leaves the registry, before the script lane's own registrations, so a script never sees, through a result's blocks, a uri the model's line withholds; a result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. - A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. - A tool result's data that can't be read through to apply RFC-005's rules, or shown once it has been, shows `[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]` on the model's lane, in the stored and sent copies alike, with a log line naming the tool and the error. It was shown with none of its references' claims where the walk reading them stopped and the render's didn't, or, for data no JSON can hold, such as a cycle or a BigInt, a getter that throws, or data too deep to write, the copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception, with the tool-result guard off; with it on, its archive of the result, and the fields it keeps of a result it withholds, still throw, as before, as does an error result whose own error field can't be read. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. diff --git a/src/framework.ts b/src/framework.ts index ec731b254..789eefc7f 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -22596,32 +22596,39 @@ export class AgentFramework { // The model's line for the result, its claims and its records as it reads // them, which decide where a link's object shows its uri // (referenceBlockForScript's line). Its records are the registry's as - // they stood before this lane registered anything (openJournal): the - // demand fetch below and the treatment register as they go, which the - // line never sees (Ada-1017, room-203 #202683). + // the model's line leaves it, before the lane's own registrations + // (openJournal): the demand loop below registers the result's top-level + // blocks in its order, as the line's render does, and the treatment then + // registers, rebinds and evicts as it goes, which the line never sees + // (Ada-1017, room-203 #202683, #203011). let line: ScriptLine | undefined; let journal: RegistryJournal | undefined; try { if ((!isError || structured) && Array.isArray(result.data) && fromModule) { + let lineClaims: ResultClaims; try { - const lineClaims = resultClaimsOrThrow(result.data, result.dataIsContent); + lineClaims = resultClaimsOrThrow(result.data, result.dataIsContent); claims = showsBlocks ? scriptViewClaims(result.data, false) : lineClaims; - if (showsBlocks) { - journal = referenceRegistry.openJournal(); - line = { claims: lineClaims, recordBefore: journal.before }; - } } catch (err) { return failed(err); } const fetcher = this.getReferenceFetcher(); for (const block of result.data) { const c = classifyBlock(block); - if (c.kind !== 'reference' || c.testimony.fromLink) continue; + if (c.kind !== 'reference') continue; // Registered, not only found, so a re-issued reference's claims // replace a link's, or a stale reference's, before the fetch reads // them, as on any other path, while the record isn't fetched yet. A // record its uri already has keeps the server it's bound to. const record = referenceRegistry.register(c.testimony); + // A link is registered too, in the result's order, and not fetched: + // the registry then holds its records in the order the model's own + // render makes, so a registration here evicts what that render's + // would, short of more references in one result than the registry + // holds, and the text route, which reads the live registry, decides + // each link's uri as the model's line does (Ada-1017, room-203 + // #203000). + if (c.testimony.fromLink) continue; // A uri any of the result's references gives an expiry that counts // isn't fetched, as autoFetchReferences passes over one that any block // of an MCPL result marks expired. @@ -22633,6 +22640,13 @@ export class AgentFramework { const outcome = await fetcher.fetch(record); if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); } + // The loop's registrations are the ones the model's line makes; the + // treatment's, below, are the script lane's own, which the journal + // keeps the line's records from (Ada-1017, room-203 #203011). + if (showsBlocks) { + journal = referenceRegistry.openJournal(); + line = { claims: lineClaims, recordBefore: journal.before }; + } } // An MCPL peer's result whose blocks JSON couldn't hold, so that RFC-005's // treatment couldn't be given them at dispatch. @@ -29125,7 +29139,7 @@ export class AgentFramework { const blocks = direct.data.map((block: unknown) => referenceBlockForScript(block, serverId, claims, line)); // The top-level blocks were treated just above; the pass treats // what's nested in them. Reading a treated block again would take - // a link object's uri, as sanitizeLabel cut it, for a new link, + // a link object's uri, as shownUri shows it, for a new link, // registered apart from the one the result named. const nested = function (this: unknown, _key: string, value: unknown): unknown { return this === blocks ? value : shownToScriptFromServer(value, serverId, claims, line); diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 33ca85f09..846eb028e 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -587,16 +587,16 @@ export class ReferenceRegistry { /** * Keep, until `close`, each record as it stood when this opened: a copy - * taken the first time a registration changes, creates or evicts it, or - * null where the uri had none. `before(uri)` gives that record, or the - * record as it is where nothing registered has touched it since, which is - * the same. A fetch that saves a copy in the meantime sets its record's - * saved path without registering, so a record no registration has copied - * yet is read saved, and one a registration copied first, as a module's - * demand fetch registers before it fetches, is read as it stood, unsaved, - * as the model's line, which makes no demand fetch, reads it. A fetch saves - * only what its checks passed. Each open has its own copies, so lanes - * that overlap keep theirs apart, and closing one lets its copies go. + * taken the first time a registration names its uri, whatever it changes, + * or evicts its record, or null where the uri had none then. `before(uri)` + * gives that copy, or, where no registration has named the uri since, the + * record as it is, which is the same. A fetch that saves a copy while it's + * open sets its record's saved path without registering, so a record no + * registration has named since is read saved, and one a registration + * named first is read as it stood, unsaved. A fetch saves only what passed + * the checks a link's line reads too, so either reads as a record the host + * can fetch. Each open has its own copies, so lanes that overlap keep + * theirs apart, and closing one lets its copies go. */ openJournal(): RegistryJournal { const copies = new Map(); @@ -607,12 +607,19 @@ export class ReferenceRegistry { }; } - /** Copy a uri's record into each open journal that has none for it yet. */ + /** Copy a uri's record into each open journal that has none for it yet, + * a copy of its own for each. */ private keepFirst(uri: string): void { if (this.journals.size === 0) return; const record = this.findByUri(uri); - const copy = record ? { ...record, serverIds: [...record.serverIds] } : null; - for (const copies of this.journals) if (!copies.has(uri)) copies.set(uri, copy); + for (const copies of this.journals) { + if (!copies.has(uri)) copies.set(uri, record ? { ...record, serverIds: [...record.serverIds] } : null); + } + } + + /** How many journals are open: none once every lane has closed its own. */ + get journalCount(): number { + return this.journals.size; } register(testimony: ReferenceTestimony, serverId?: string): ReferenceRecord { @@ -921,7 +928,9 @@ export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, now if (!t.fromLink || claimedNever(record, claims)) return null; if (unfetchableReason(record, nowMs, claims) === null) return null; const uri = shownUri(t.uri); - return uriMayCarryCredential(uri) ? null : uri; + // The whole uri as well as the cut it shows: userinfo whose `@` falls past + // the cut still carries a credential, and the cut would show part of it. + return uriMayCarryCredential(uri) || uriMayCarryCredential(sanitizeLabel(t.uri, Infinity).toWellFormed()) ? null : uri; } /** @@ -1075,10 +1084,11 @@ export function referenceBlockForScript(block: unknown, serverId?: string, claim // claims withhold it: reading more claims can loosen as well as // tighten, since an expiry makes a link one the host can't fetch, whose // uri the line shows (Ada-1017, room-203 #201013). The line's record is - // the one before the script lane registered anything (#202683), since - // the treatment registers, rebinds and evicts as it goes, which the - // line never sees; where there was none, it's the one the link's own - // registration makes. A script never sees a uri the model's line + // the registry's as the model's line leaves it, before the script + // lane's own registrations (#202683, #203011), since the treatment + // registers, rebinds and evicts as it goes, which the line never sees; + // where there was none, it's the one the link's own registration + // makes. A script never sees a uri the model's line // withholds, through its blocks; structured content goes as sent, as // on main. const lineRecord = line diff --git a/test/mcpl-reference-fetcher.test.ts b/test/mcpl-reference-fetcher.test.ts index 42b1e22f4..c98c001b3 100644 --- a/test/mcpl-reference-fetcher.test.ts +++ b/test/mcpl-reference-fetcher.test.ts @@ -139,6 +139,19 @@ test("a registry journal keeps each record as it stood when it opened, through a assert.equal(reg.findByUri(u), undefined); assert.deepEqual([two.before(u)?.testimony.disposition, two.before(u)?.serverIds], [undefined, ['first', 'second']]); two.close(); + // Two journals open at once, with no registration between their opens, + // each get a copy of their own. + const x = 'https://zz-journal.example/x.csv'; + reg.register(t(x), 'first'); + const three = reg.openJournal(); + const four = reg.openJournal(); + reg.register(t(x), 'second'); + assert.notEqual(three.before(x), four.before(x)); + assert.deepEqual(three.before(x)?.serverIds, ['first']); + assert.notEqual(three.before(x)?.serverIds, four.before(x)?.serverIds); + three.close(); + four.close(); + assert.equal(reg.journalCount, 0); }); test('vector 5: digest mismatch is never presented', async () => { diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index d9a1ba900..a18af55f8 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -24,8 +24,8 @@ import { join } from 'node:path'; import { AgentFramework, WorkspaceModule } from '../src/index.js'; import type { EventResponse, Module, ModuleContext, ToolCall, ToolDefinition, ToolResult } from '../src/index.js'; -import { EXPIRY_REASON, INLINE_WITHHELD_TEXT, UNBOUND_REASON, referenceRegistry } from '../src/mcpl/references.js'; -import type { ReferenceRecord } from '../src/mcpl/references.js'; +import { EXPIRY_REASON, INLINE_WITHHELD_TEXT, UNBOUND_REASON, classifyBlock, referenceRegistry } from '../src/mcpl/references.js'; +import type { ReferenceRecord, ReferenceTestimony } from '../src/mcpl/references.js'; import { nonErrorResultText } from '../src/tool-result-history.js'; import { MockMembrane } from './helpers/mock-membrane.js'; @@ -799,10 +799,87 @@ test("a link's uri reads its record as it stood before the script lane registere assert.ok(!out.includes(`evicted-${route}/report.csv`), `${route}: ${out.slice(-600)}`); const view = JSON.parse(out) as { content: Array>>; isError: boolean }; assert.deepEqual([view.isError, typeof view.content[0]!.link!.refId], [false, 'string'], route); + // The never record was evicted: the uri's record now is the link's own. + assert.equal(referenceRegistry.findByUri(uri)?.testimony.fromLink, true, route); + assert.equal(referenceRegistry.journalCount, 0, `${route}: the lane closed its journal`); } }); }); +/** A testimony, as the classifier reads a block. */ +const testimonyOf = (block: Record): ReferenceTestimony => (classifyBlock(block) as { testimony: ReferenceTestimony }).testimony; + +/** Make `uri`'s record the registry's oldest, with the registry full: it + * holds 2,000, so 1,999 new records after it take the place of every older + * one. */ +const fillAfter = (tag: string): void => { + for (let i = 0; i < 1999; i++) referenceRegistry.register(testimonyOf({ type: 'resource', uri: `https://files.example.invalid/fill-${tag}/${i}.csv`, name: `${i}.csv` })); +}; + +test("a module's result a script reads as text registers its links in the result's order, so a later reference keeps the model's line's never on a link before it", { timeout: 60_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + const uri = 'https://elsewhere.example.invalid/text-order/report.csv'; + answer({ content: [{ type: 'resource', uri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + fillAfter('text-order'); + // The model's render reaches the link first, which keeps its record, and + // then registers the new reference, which evicts another. + const data = [ + { type: 'resource_link', uri, name: 'report.csv' }, + { type: 'resource', uri: 'https://files.example.invalid/text-order/new.csv', name: 'new.csv' }, + ]; + const out = await exit({ success: true, data }, 'fwd--whole', false); + assert.ok(!out.includes('text-order/report.csv'), out); + assert.equal(referenceRegistry.findByUri(uri)?.testimony.disposition, 'never'); + }); +}); + +test("a module's view reads the registry as the model's line leaves it: where the result's own reference evicts an earlier never before its link, script and line both show the uri", { timeout: 60_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + // The demand loop registers the result's blocks in the order the model's + // render does, so the new reference evicts the never record before the + // link, for both: an earlier result's never lasts as long as the registry + // keeps it (Ada-1017, room-203 #203011). + const uri = 'https://elsewhere.example.invalid/view-evicted/report.csv'; + const data = () => [ + { type: 'resource', uri: 'https://files.example.invalid/view-evicted/new.csv', name: 'new.csv' }, + { type: 'resource_link', uri, name: 'report.csv' }, + ]; + const seed = async (): Promise => { + answer({ content: [{ type: 'resource', uri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + }; + await seed(); + fillAfter('view-evicted-line'); + const line = nonErrorResultText({ data: data() }); + assert.ok(line.includes('view-evicted/report.csv'), line); + await seed(); + fillAfter('view-evicted-script'); + const out = await exit({ success: true, data: data(), structured: {} }, 'fwd--whole', false); + const link = (JSON.parse(out) as { content: Array> }).content[1]!; + assert.equal(link.uri, uri, out); + assert.equal(referenceRegistry.journalCount, 0); + }); +}); + +test("a link's record read through an eviction keeps its server binding, so a link the host can fetch shows a script no uri", { timeout: 60_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + // Bound to the files server, on its origin, by the peer's dispatch. + const uri = 'https://files.example.invalid/bound-evicted/report.csv'; + answer({ content: [{ type: 'resource', uri, name: 'report.csv' }], structuredContent: {} }); + await script('mcpl--files--attach'); + assert.deepEqual(referenceRegistry.findByUri(uri)?.serverIds, ['files']); + // A module's nested references, registered with no server, evict it + // before the link, whose own registration then makes it unbound. + const many = Array.from({ length: 2001 }, (_, i) => ({ type: 'resource', uri: `https://files.example.invalid/bound-many/${i}.csv`, name: `${i}.csv` })); + const out = await exit({ success: true, data: [{ type: 'text', text: 'many', many, link: { type: 'resource_link', uri, name: 'report.csv' } }], structured: {} }, 'fwd--whole', false); + assert.ok(!out.includes('bound-evicted/report.csv'), out.slice(-600)); + assert.deepEqual(referenceRegistry.findByUri(uri)?.serverIds, [], 'evicted, and made again unbound'); + const view = JSON.parse(out) as { content: Array>> }; + assert.equal(typeof view.content[0]!.link!.refId, 'string'); + }); +}); + test("an earlier result's never on one uri doesn't withhold, from a script, a link to another that shows as the same text, where the model's line shows it", { timeout: 30_000 }, async () => { await withPeers(async ({ answer, script, modelCall }) => { // Each record speaks for its own uri: the earlier never is the marked diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index c3d0e9fb4..6776c80c2 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1433,6 +1433,22 @@ describe('one answer for every block of a result', () => { assert.ok(toolResultDataToHistoryString([dated, link], undefined, { dataIsContent: true }).includes('inline-never')); }); + it("withholds a link's uri whose userinfo falls past where its shown text is cut", () => { + // Off any server's origin, so the host can't fetch it, and a uri that + // can't carry a credential would show; this one's `@` falls past the + // 4,096 characters a link's uri is cut to. + const link = { type: 'resource_link', uri: `https://zz-user:${'p'.repeat(4200)}@zz-elsewhere.example/long-userinfo.csv`, name: 'a.csv' }; + const control = { type: 'resource_link', uri: 'https://zz-elsewhere.example/short-userinfo-control.csv', name: 'a.csv' }; + assert.ok(toolResultDataToHistoryString([control], undefined, { dataIsContent: true }).includes('short-userinfo-control')); + for (const shown of [ + toolResultDataToHistoryString([link], undefined, { dataIsContent: true }), + toolResultDataToHistoryString({ link }), + JSON.stringify(shownToScript(link, scriptViewClaims([link], false))), + ]) { + assert.ok(!shown.includes('zz-user:'), shown.slice(0, 300)); + } + }); + it("withholds a never uri from a link whose uri shows as the same text once well formed, a lone surrogate showing as U+FFFD", () => { // Each side as the other shows it: a never with the replacement character // and a link with a lone surrogate, and the other way round. From ef6c7d044011b3ad6d6860241ed369f78e2ceda4 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 08:27:41 -0400 Subject: [PATCH 30/40] test(tool-result-blocks): the cut-userinfo test uses a uri whose cut text alone would show it The removal variant credential-cut-only survived at 7b682f19 (by run): the test's https uri, its @ past the 4,096-character cut inside a password, was withheld by the cut text alone, which doesn't parse there. A postgres: uri whose long name alone runs past the cut parses, cut, as a host with no userinfo, so only the whole-uri check withholds it. Co-Authored-By: Claude Opus 5.5 --- test/tool-result-blocks.test.ts | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 6776c80c2..d9376525f 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1434,18 +1434,21 @@ describe('one answer for every block of a result', () => { }); it("withholds a link's uri whose userinfo falls past where its shown text is cut", () => { - // Off any server's origin, so the host can't fetch it, and a uri that - // can't carry a credential would show; this one's `@` falls past the - // 4,096 characters a link's uri is cut to. - const link = { type: 'resource_link', uri: `https://zz-user:${'p'.repeat(4200)}@zz-elsewhere.example/long-userinfo.csv`, name: 'a.csv' }; - const control = { type: 'resource_link', uri: 'https://zz-elsewhere.example/short-userinfo-control.csv', name: 'a.csv' }; + // A scheme the fetcher refuses, so the host can't fetch it, and a uri + // that can't carry a credential would show. Its userinfo, a long name + // alone, runs past the 4,096 characters a link's uri is cut to: cut, + // the text parses as a host with no userinfo, so only the whole uri + // shows the credential. (Where the cut falls in a password, the text + // doesn't parse, which withholds it anyway.) + const link = { type: 'resource_link', uri: `postgres://zz-user-${'t'.repeat(4200)}@zz-db.example/long-userinfo`, name: 'a.csv' }; + const control = { type: 'resource_link', uri: 'postgres://zz-db.example/short-userinfo-control', name: 'a.csv' }; assert.ok(toolResultDataToHistoryString([control], undefined, { dataIsContent: true }).includes('short-userinfo-control')); for (const shown of [ toolResultDataToHistoryString([link], undefined, { dataIsContent: true }), toolResultDataToHistoryString({ link }), JSON.stringify(shownToScript(link, scriptViewClaims([link], false))), ]) { - assert.ok(!shown.includes('zz-user:'), shown.slice(0, 300)); + assert.ok(!shown.includes('zz-user-'), shown.slice(0, 300)); } }); From 2ba363bbab3b65df3af744bf33bd90ade8223a76 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 08:30:54 -0400 Subject: [PATCH 31/40] revert(references): the whole-uri credential check, which no link reaches The fourth review found that the credential check, reading the shown text, missed userinfo past the 4,096-character cut, and 7b682f19 added a check of the whole uri. A probe at ef6c7d04 shows no link's uri is ever cut: parseReferenceFields refuses a uri longer than REFERENCE_LIMITS.uri (REFERENCE_LINK_TOO_LONG_TEXT), and sanitizeLabel cuts only text longer than that, which cleaning can't make. So the check never changed an answer, which is why its removal variant survived twice, and it goes, with its test and the fragment's sentence; shownUri's doc says no uri is shortened, and the changelog no longer names the cut among what makes two uris show alike. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- src/mcpl/references.ts | 9 ++++----- test/tool-result-blocks.test.ts | 19 ------------------- 3 files changed, 5 insertions(+), 25 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 8021a96b5..6dc1074f7 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,4 +1,4 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. The check reads the whole uri as well as the text a link would show, so userinfo past where that text is cut, at 4,096 characters, still withholds it. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cut drops controls and bidi marks, trims surrounding space and stops at 4,096 characters, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cut drops controls and bidi marks and trims surrounding space, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. - In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as that line leaves the registry, before the script lane's own registrations, so a script never sees, through a result's blocks, a uri the model's line withholds; a result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. - A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 846eb028e..4ca85a6d3 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -787,8 +787,9 @@ export type ResultClaims = ReadonlyMap; /** * A uri as a link shows it: cut of what sanitizeLabel cuts (controls and bidi - * marks, surrounding space, past REFERENCE_LIMITS.uri), and well formed, as - * every copy is before it's shown, a lone surrogate showing as U+FFFD. A `never` is + * marks, surrounding space; a uri past REFERENCE_LIMITS.uri is no reference, + * so none is shortened), and well formed, as every copy is before it's + * shown, a lone surrogate showing as U+FFFD. A `never` is * about what a link shows, so it's found by this text (neverShown): a claim * of `never` on a uri withholds every link that shows the same text, two * uris that cut alike sharing it. Every other claim is about the record, @@ -928,9 +929,7 @@ export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, now if (!t.fromLink || claimedNever(record, claims)) return null; if (unfetchableReason(record, nowMs, claims) === null) return null; const uri = shownUri(t.uri); - // The whole uri as well as the cut it shows: userinfo whose `@` falls past - // the cut still carries a credential, and the cut would show part of it. - return uriMayCarryCredential(uri) || uriMayCarryCredential(sanitizeLabel(t.uri, Infinity).toWellFormed()) ? null : uri; + return uriMayCarryCredential(uri) ? null : uri; } /** diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index d9376525f..c3d0e9fb4 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -1433,25 +1433,6 @@ describe('one answer for every block of a result', () => { assert.ok(toolResultDataToHistoryString([dated, link], undefined, { dataIsContent: true }).includes('inline-never')); }); - it("withholds a link's uri whose userinfo falls past where its shown text is cut", () => { - // A scheme the fetcher refuses, so the host can't fetch it, and a uri - // that can't carry a credential would show. Its userinfo, a long name - // alone, runs past the 4,096 characters a link's uri is cut to: cut, - // the text parses as a host with no userinfo, so only the whole uri - // shows the credential. (Where the cut falls in a password, the text - // doesn't parse, which withholds it anyway.) - const link = { type: 'resource_link', uri: `postgres://zz-user-${'t'.repeat(4200)}@zz-db.example/long-userinfo`, name: 'a.csv' }; - const control = { type: 'resource_link', uri: 'postgres://zz-db.example/short-userinfo-control', name: 'a.csv' }; - assert.ok(toolResultDataToHistoryString([control], undefined, { dataIsContent: true }).includes('short-userinfo-control')); - for (const shown of [ - toolResultDataToHistoryString([link], undefined, { dataIsContent: true }), - toolResultDataToHistoryString({ link }), - JSON.stringify(shownToScript(link, scriptViewClaims([link], false))), - ]) { - assert.ok(!shown.includes('zz-user-'), shown.slice(0, 300)); - } - }); - it("withholds a never uri from a link whose uri shows as the same text once well formed, a lone surrogate showing as U+FFFD", () => { // Each side as the other shows it: a never with the replacement character // and a link with a lone surrogate, and the other way round. From 1754ec8798d7d5c05707be5db1cebb9d01bbec8d Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 08:47:12 -0400 Subject: [PATCH 32/40] fix(framework): review fixes: the module demand loop is the line's only where the line renders the blocks The call on the review's findings: 7b682f19's loop registered links, and its journal opened after the loop, on the premise that the loop's registrations are the model's line's. That holds only where the line renders those blocks, a content array that isn't a failure. For an array the line walks at depth, the loop registered a later top-level link or reference before the walk reached a nested link, evicting an earlier never before the journal opened; for a failure, whose line is its error and registers nothing, the loop registered blocks the line never does. Both showed a script a uri b9ca38c3 withheld, by run. Now blocksAreLines (!isError && isContentArray(data, dataIsContent)) decides: there the loop registers links in order and the journal opens after it; elsewhere the journal opens before the loop, which leaves links to the treatment, as at b9ca38c3. isContentArray is exported for it. Tests: a module's array the line walks at depth keeps the never on a nested link before a later top-level one, on the view and the text route, as the model's line does; a failed result with structured content keeps a never its blocks' registrations would evict; the text-order test checks its answer rendered the link's stub; journalCount is read with journals open and after each route's failure exit; a test title says the line reads the binding as it stood. openJournal's doc says a record named first is read as it stood then, saved or not. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 45 +++++++++++------- src/mcpl/references.ts | 13 +++--- src/tool-result-history.ts | 2 +- test/mcpl-reference-fetcher.test.ts | 2 + test/script-structured-references.test.ts | 56 ++++++++++++++++++++++- 5 files changed, 94 insertions(+), 24 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 789eefc7f..06363fdd8 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -118,6 +118,7 @@ import { goesNative, nonErrorResultText, nonErrorResultTextOrThrow, + isContentArray, resultClaimsOrThrow, scriptViewClaims, shownToScript, @@ -22597,10 +22598,10 @@ export class AgentFramework { // them, which decide where a link's object shows its uri // (referenceBlockForScript's line). Its records are the registry's as // the model's line leaves it, before the lane's own registrations - // (openJournal): the demand loop below registers the result's top-level - // blocks in its order, as the line's render does, and the treatment then - // registers, rebinds and evicts as it goes, which the line never sees - // (Ada-1017, room-203 #202683, #203011). + // (openJournal): where the line renders the result's blocks, the demand + // loop below registers them in their order, as the line's render does; + // the treatment then registers, rebinds and evicts as it goes, which the + // line never sees (Ada-1017, room-203 #202683, #203011, #203172). let line: ScriptLine | undefined; let journal: RegistryJournal | undefined; try { @@ -22612,22 +22613,34 @@ export class AgentFramework { } catch (err) { return failed(err); } + // The loop's registrations are the line's own only where the line + // renders these blocks: a content array's, block by block at the top + // level, and not a failure's, whose line is its error. There the loop + // registers links too, in the result's order, and the journal opens + // after it. Elsewhere the line walks at depth, or registers nothing, + // so the journal opens before the loop, which leaves links to the + // treatment (Ada-1017, room-203 #203172). + const blocksAreLines = !isError && isContentArray(result.data, result.dataIsContent); + if (showsBlocks && !blocksAreLines) { + journal = referenceRegistry.openJournal(); + line = { claims: lineClaims, recordBefore: journal.before }; + } const fetcher = this.getReferenceFetcher(); for (const block of result.data) { const c = classifyBlock(block); - if (c.kind !== 'reference') continue; + if (c.kind !== 'reference' || (c.testimony.fromLink && !blocksAreLines)) continue; // Registered, not only found, so a re-issued reference's claims // replace a link's, or a stale reference's, before the fetch reads // them, as on any other path, while the record isn't fetched yet. A // record its uri already has keeps the server it's bound to. const record = referenceRegistry.register(c.testimony); - // A link is registered too, in the result's order, and not fetched: - // the registry then holds its records in the order the model's own - // render makes, so a registration here evicts what that render's - // would, short of more references in one result than the registry - // holds, and the text route, which reads the live registry, decides - // each link's uri as the model's line does (Ada-1017, room-203 - // #203000). + // A link of blocks the line renders is registered too, in the + // result's order, and not fetched: the registry then holds its + // records in the order the model's own render makes, so a + // registration here evicts what that render's would, short of more + // references in one result than the registry holds, and the text + // route, which reads the live registry, decides each link's uri as + // the model's line does (Ada-1017, room-203 #203000). if (c.testimony.fromLink) continue; // A uri any of the result's references gives an expiry that counts // isn't fetched, as autoFetchReferences passes over one that any block @@ -22640,10 +22653,10 @@ export class AgentFramework { const outcome = await fetcher.fetch(record); if (!outcome.ok) console.error(`[rfc005] script-demand fetch of ${record.refId} failed: ${outcome.error}`); } - // The loop's registrations are the ones the model's line makes; the - // treatment's, below, are the script lane's own, which the journal - // keeps the line's records from (Ada-1017, room-203 #203011). - if (showsBlocks) { + // Where the loop's registrations are the ones the model's line makes, + // the treatment's, below, are the script lane's own, which the + // journal keeps the line's records from (Ada-1017, room-203 #203011). + if (showsBlocks && blocksAreLines) { journal = referenceRegistry.openJournal(); line = { claims: lineClaims, recordBefore: journal.before }; } diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 4ca85a6d3..f800127c5 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -591,12 +591,13 @@ export class ReferenceRegistry { * or evicts its record, or null where the uri had none then. `before(uri)` * gives that copy, or, where no registration has named the uri since, the * record as it is, which is the same. A fetch that saves a copy while it's - * open sets its record's saved path without registering, so a record no - * registration has named since is read saved, and one a registration - * named first is read as it stood, unsaved. A fetch saves only what passed - * the checks a link's line reads too, so either reads as a record the host - * can fetch. Each open has its own copies, so lanes that overlap keep - * theirs apart, and closing one lets its copies go. + * open sets its record's saved path without registering: a record no + * registration has named since is read with it, and one a registration + * named first is read as it stood then, saved or not. For a record a fetch + * saved meanwhile, both readings are of one the host can fetch, since a + * fetch saves only what passed the checks a link's line reads too. Each + * open has its own copies, so lanes that overlap keep theirs apart, and + * closing one lets its copies go. */ openJournal(): RegistryJournal { const copies = new Map(); diff --git a/src/tool-result-history.ts b/src/tool-result-history.ts index fd8642363..c231cf426 100644 --- a/src/tool-result-history.ts +++ b/src/tool-result-history.ts @@ -121,7 +121,7 @@ export function toolResultBlocks(data: unknown, dataIsContent?: boolean, claims? /** Whether a tool result's data is a content array, which toolResultBlocks * shows block by block. */ -function isContentArray(data: unknown, dataIsContent?: boolean): data is unknown[] { +export function isContentArray(data: unknown, dataIsContent?: boolean): data is unknown[] { return Array.isArray(data) && (dataIsContent === true || data.every((element) => isPlainRecord(element) && MCP_BLOCK_TYPES.has(element.type as string))); } diff --git a/test/mcpl-reference-fetcher.test.ts b/test/mcpl-reference-fetcher.test.ts index c98c001b3..b0ac74dc3 100644 --- a/test/mcpl-reference-fetcher.test.ts +++ b/test/mcpl-reference-fetcher.test.ts @@ -145,11 +145,13 @@ test("a registry journal keeps each record as it stood when it opened, through a reg.register(t(x), 'first'); const three = reg.openJournal(); const four = reg.openJournal(); + assert.equal(reg.journalCount, 2); reg.register(t(x), 'second'); assert.notEqual(three.before(x), four.before(x)); assert.deepEqual(three.before(x)?.serverIds, ['first']); assert.notEqual(three.before(x)?.serverIds, four.before(x)?.serverIds); three.close(); + assert.equal(reg.journalCount, 1); four.close(); assert.equal(reg.journalCount, 0); }); diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index a18af55f8..5d3196659 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -830,6 +830,10 @@ test("a module's result a script reads as text registers its links in the result ]; const out = await exit({ success: true, data }, 'fwd--whole', false); assert.ok(!out.includes('text-order/report.csv'), out); + const reply = JSON.parse(out) as { content: [{ text: string }]; isError: boolean }; + assert.equal(reply.isError, false, out); + // The link's line rendered, as the withheld stub that names it. + assert.match(reply.content[0].text, /^\[ref_\w+\] report\.csv/m, out); assert.equal(referenceRegistry.findByUri(uri)?.testimony.disposition, 'never'); }); }); @@ -862,7 +866,54 @@ test("a module's view reads the registry as the model's line leaves it: where th }); }); -test("a link's record read through an eviction keeps its server binding, so a link the host can fetch shows a script no uri", { timeout: 60_000 }, async () => { +test("a module's array the model's line walks at depth keeps its never on a nested link the walk reaches before a later top-level one, as a script reads it", { timeout: 90_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + // Not a content array, so the model's line walks it depth-first: it + // reaches the nested link first, which keeps its record, and then the + // later top-level link, whose registration evicts another. + const uri = 'https://elsewhere.example.invalid/walk-order/report.csv'; + const data = () => [ + { wrap: { type: 'resource_link', uri, name: 'report.csv' } }, + { type: 'resource_link', uri: 'https://files.example.invalid/walk-order/new.csv', name: 'new.csv' }, + ]; + const seed = async (tag: string): Promise => { + answer({ content: [{ type: 'resource', uri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + fillAfter(`walk-order-${tag}`); + }; + await seed('line'); + const line = nonErrorResultText({ data: data() }); + assert.ok(!line.includes('walk-order/report.csv'), line); + for (const structured of [true, false]) { + await seed(structured ? 'view' : 'text'); + const out = await exit({ success: true, data: data(), ...(structured ? { structured: {} } : {}) }, 'fwd--whole', false); + assert.ok(!out.includes('walk-order/report.csv'), `${structured ? 'view' : 'text'}: ${out}`); + assert.equal((JSON.parse(out) as { isError: boolean }).isError, false, out); + } + assert.equal(referenceRegistry.journalCount, 0); + }); +}); + +test("a module's failed result, whose model's line is its error and registers nothing, keeps a never its blocks' registrations would evict, as a script reads it", { timeout: 60_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + const uri = 'https://elsewhere.example.invalid/error-order/report.csv'; + answer({ content: [{ type: 'resource', uri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + fillAfter('error-order'); + const data = [ + { type: 'resource', uri: 'https://files.example.invalid/error-order/new.csv', name: 'new.csv' }, + { type: 'resource_link', uri, name: 'report.csv' }, + ]; + const out = await exit({ success: false, isError: true, error: 'the attach failed', data, structured: {} } as unknown as ToolResult, 'fwd--whole', false); + const view = JSON.parse(out) as { content: Array>; isError: boolean }; + assert.equal(view.isError, true, out); + assert.ok(!out.includes('error-order/report.csv'), out); + assert.equal(typeof view.content[1]!.refId, 'string', out); + assert.equal(referenceRegistry.journalCount, 0); + }); +}); + +test("a link's line reads its record's server binding as it stood before an eviction the treatment makes, so a link the host could fetch shows a script no uri", { timeout: 60_000 }, async () => { await withPeers(async ({ answer, script, exit }) => { // Bound to the files server, on its origin, by the peer's dispatch. const uri = 'https://files.example.invalid/bound-evicted/report.csv'; @@ -1030,9 +1081,12 @@ test("where the treatment fails after the claims walk read the view, a peer's re const fromPeer = JSON.parse(await script('mcpl--files--attach')) as { content: [{ text: string }]; isError: boolean }; assert.equal(fromPeer.isError, true); assert.match(fromPeer.content[0].text, /^tool 'mcpl--files--attach' returned a result that can't be converted to JSON/); + // Each route closed its journal on the way out of the failure too. + assert.equal(referenceRegistry.journalCount, 0, 'peer'); const fromModule = JSON.parse(await exit({ success: true, data: content(), structured: {} }, 'fwd--whole', false)) as { content: [{ text: string }]; isError: boolean }; assert.equal(fromModule.isError, true); assert.match(fromModule.content[0].text, /^tool 'fwd--whole' returned a result that can't be converted to JSON/); + assert.equal(referenceRegistry.journalCount, 0, 'module'); } finally { referenceRegistry.register = real; } From 17633af716a04fd79f6fab72c31c91256916ce55 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 09:14:49 -0400 Subject: [PATCH 33/40] fix(framework, references): review fixes: the flag as the render reads it, the registry's bound named The calls on the review's findings, which end its series of rounds: - Past 2,000 references in one result, a module's content array can lose an earlier result's never on a script's view and text route where the model's line, rendering in order, hasn't yet: the demand loop registers the whole array before anything is shown (by run: [link(U), 2,000 links] withholds U on the line, shows it to a script). Named, not built round: the changelog's "a script never sees" now holds within that bound, and says what happens past it; the comment beside blocksAreLines says so too. - blocksAreLines and the line's claims read Boolean(dataIsContent), as the render does, so a flag that isn't a boolean decides the same way. - The comment says a failure's line registers nothing and data at depth registers in its own walk's order, so the journal before the loop gives records from before that walk, which can only be stricter. - shownUri's doc and the changelog say cleaning, not cut. - openJournal's doc names the two-frameworks exception. Tests: a module's array walked at depth with a later top-level resource keeps the never the early journal holds; an array content by its own flag, an unknown block among it, has its link registered first; the walk-order test checks the nested link was shown. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 4 +- src/framework.ts | 17 +++++--- src/mcpl/references.ts | 18 +++++---- test/script-structured-references.test.ts | 49 ++++++++++++++++++++++- 4 files changed, 72 insertions(+), 16 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 6dc1074f7..a3950e818 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,5 +1,5 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cut drops controls and bidi marks and trims surrounding space, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. -- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as that line leaves the registry, before the script lane's own registrations, so a script never sees, through a result's blocks, a uri the model's line withholds; a result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cleaning drops controls and bidi marks and trims surrounding space, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as that line leaves the registry, before the script lane's own registrations, so a script never sees, through a result's blocks, a uri the model's line withholds, within 2,000 references in one result: past that, either can lose an earlier result's `never`, and a script reading a module's content array can lose it where the line, rendering in order, hasn't yet. A result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. - A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. - A tool result's data that can't be read through to apply RFC-005's rules, or shown once it has been, shows `[Result not shown: the tool ran, but its result couldn't be read through, checked for references and shown, so this host withheld it.]` on the model's lane, in the stored and sent copies alike, with a log line naming the tool and the error. It was shown with none of its references' claims where the walk reading them stopped and the render's didn't, or, for data no JSON can hold, such as a cycle or a BigInt, a getter that throws, or data too deep to write, the copies' serializer threw. So a host passing such a result to the direct Agent API gets the notice where it got an exception, with the tool-result guard off; with it on, its archive of the result, and the fields it keeps of a result it withholds, still throw, as before, as does an error result whose own error field can't be read. A script meets it as a failure naming the tool, on its text route as on its structured view, and from an MCPL peer's structured view as a result it can't convert. diff --git a/src/framework.ts b/src/framework.ts index 06363fdd8..203b72478 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -22608,7 +22608,9 @@ export class AgentFramework { if ((!isError || structured) && Array.isArray(result.data) && fromModule) { let lineClaims: ResultClaims; try { - lineClaims = resultClaimsOrThrow(result.data, result.dataIsContent); + // The flag as the render reads it, so a value other than a + // boolean decides the same way here (isContentArray reads `true`). + lineClaims = resultClaimsOrThrow(result.data, Boolean(result.dataIsContent)); claims = showsBlocks ? scriptViewClaims(result.data, false) : lineClaims; } catch (err) { return failed(err); @@ -22617,10 +22619,15 @@ export class AgentFramework { // renders these blocks: a content array's, block by block at the top // level, and not a failure's, whose line is its error. There the loop // registers links too, in the result's order, and the journal opens - // after it. Elsewhere the line walks at depth, or registers nothing, - // so the journal opens before the loop, which leaves links to the - // treatment (Ada-1017, room-203 #203172). - const blocksAreLines = !isError && isContentArray(result.data, result.dataIsContent); + // after it. Elsewhere the line registers nothing (a failure) or + // registers in its own walk's order (data at depth), which the loop + // can't follow, so the journal opens before the loop, which leaves + // links to the treatment: the line's records are then the registry + // before the line's own walk, which can only be stricter, by an + // eviction (Ada-1017, room-203 #203172). Past 2,000 references in one + // result, a content array's loop can evict an earlier never before + // the line, rendering in order, would have (#203184). + const blocksAreLines = !isError && isContentArray(result.data, Boolean(result.dataIsContent)); if (showsBlocks && !blocksAreLines) { journal = referenceRegistry.openJournal(); line = { claims: lineClaims, recordBefore: journal.before }; diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index f800127c5..9ef36c78b 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -595,7 +595,9 @@ export class ReferenceRegistry { * registration has named since is read with it, and one a registration * named first is read as it stood then, saved or not. For a record a fetch * saved meanwhile, both readings are of one the host can fetch, since a - * fetch saves only what passed the checks a link's line reads too. Each + * fetch saves only what passed the checks a link's line reads too, except + * where two frameworks configure one server id differently, which a stub's + * words can't tell apart either (setServerAddress). Each * open has its own copies, so lanes that overlap keep theirs apart, and * closing one lets its copies go. */ @@ -787,13 +789,13 @@ export interface StubOptions { export type ResultClaims = ReadonlyMap; /** - * A uri as a link shows it: cut of what sanitizeLabel cuts (controls and bidi - * marks, surrounding space; a uri past REFERENCE_LIMITS.uri is no reference, - * so none is shortened), and well formed, as every copy is before it's - * shown, a lone surrogate showing as U+FFFD. A `never` is - * about what a link shows, so it's found by this text (neverShown): a claim - * of `never` on a uri withholds every link that shows the same text, two - * uris that cut alike sharing it. Every other claim is about the record, + * A uri as a link shows it: cleaned as sanitizeLabel cleans a label (controls + * and bidi marks dropped, surrounding space trimmed; a uri past + * REFERENCE_LIMITS.uri is no reference, so none is shortened), and well + * formed, as every copy is before it's shown, a lone surrogate showing as + * U+FFFD. A `never` is about what a link shows, so it's found by this text + * (neverShown): a claim of `never` on a uri withholds every link that shows + * the same text, two uris that clean alike sharing it. Every other claim is about the record, * which the fetch reads by its raw uri, so it's kept by the raw uri (Ada-1017, * room-203 #202239). */ diff --git a/test/script-structured-references.test.ts b/test/script-structured-references.test.ts index 5d3196659..d9b6d3c2a 100644 --- a/test/script-structured-references.test.ts +++ b/test/script-structured-references.test.ts @@ -888,12 +888,59 @@ test("a module's array the model's line walks at depth keeps its never on a nest await seed(structured ? 'view' : 'text'); const out = await exit({ success: true, data: data(), ...(structured ? { structured: {} } : {}) }, 'fwd--whole', false); assert.ok(!out.includes('walk-order/report.csv'), `${structured ? 'view' : 'text'}: ${out}`); - assert.equal((JSON.parse(out) as { isError: boolean }).isError, false, out); + const reply = JSON.parse(out) as { content: Array> & { text?: string }>; isError: boolean }; + assert.equal(reply.isError, false, out); + // The nested link was shown, as its object or its stub, not dropped. + if (structured) assert.equal(typeof reply.content[0]!.wrap!.refId, 'string', out); + else assert.match(String(reply.content[0]!.text), /\[ref_\w+\] report\.csv/, out); } assert.equal(referenceRegistry.journalCount, 0); }); }); +test("a module's array the model's line walks at depth reads its records from before the demand loop, so the loop's own reference can't evict a never the view's nested link keeps", { timeout: 60_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + // The later top-level element is a reference, which the loop registers, + // in a full registry evicting the never record; the journal, open before + // the loop for data the line walks at depth, keeps it. + const uri = 'https://elsewhere.example.invalid/walk-resource/report.csv'; + answer({ content: [{ type: 'resource', uri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + fillAfter('walk-resource'); + const data = [ + { wrap: { type: 'resource_link', uri, name: 'report.csv' } }, + { type: 'resource', uri: 'https://files.example.invalid/walk-resource/new.csv', name: 'new.csv' }, + ]; + const out = await exit({ success: true, data, structured: {} }, 'fwd--whole', false); + assert.ok(!out.includes('walk-resource/report.csv'), out); + const view = JSON.parse(out) as { content: Array>>; isError: boolean }; + assert.deepEqual([view.isError, typeof view.content[0]!.wrap!.refId], [false, 'string'], out); + assert.equal(referenceRegistry.findByUri(uri)?.testimony.fromLink, true, 'the loop evicted the never record'); + assert.equal(referenceRegistry.journalCount, 0); + }); +}); + +test("a module's array its result says is content has its links registered in order, an unknown block among them", { timeout: 60_000 }, async () => { + await withPeers(async ({ answer, script, exit }) => { + // Content by the result's own flag, though one element is of no MCP + // type: the model's render shows it block by block, so the loop + // registers the link first, as the render does, keeping its record. + const uri = 'https://elsewhere.example.invalid/flag-order/report.csv'; + answer({ content: [{ type: 'resource', uri, name: 'report.csv', disposition: 'never' }], structuredContent: {} }); + await script('mcpl--files--attach'); + fillAfter('flag-order'); + const data = [ + { type: 'resource_link', uri, name: 'report.csv' }, + { type: 'resource', uri: 'https://files.example.invalid/flag-order/new.csv', name: 'new.csv' }, + { type: 'zz-unknown', value: 1 }, + ]; + const out = await exit({ success: true, data, dataIsContent: true } as ToolResult, 'fwd--whole', false); + assert.ok(!out.includes('flag-order/report.csv'), out); + assert.equal((JSON.parse(out) as { isError: boolean }).isError, false, out); + assert.equal(referenceRegistry.findByUri(uri)?.testimony.disposition, 'never'); + }); +}); + test("a module's failed result, whose model's line is its error and registers nothing, keeps a never its blocks' registrations would evict, as a script reads it", { timeout: 60_000 }, async () => { await withPeers(async ({ answer, script, exit }) => { const uri = 'https://elsewhere.example.invalid/error-order/report.csv'; From 056b5b5d3f6160fa891a695c618776688f6bc71a Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 13:05:23 -0400 Subject: [PATCH 34/40] fix(references): a uri in a parameter's value is read for credentials; a data: link never shows its uri From the maintainer review at 17633af7, both by run there: - A listed credential percent-encoded inside another parameter's value, as in an image proxy's `?url=&w=640`, passed the check, and the link's line showed `X-Amz-Signature` one decode level down (and two, through a second proxy). uriMayCarryCredential now reads each parameter's value that parses as an absolute uri (new URL alone, so an ordinary value never meets the relative reading's fail-closed answer) the same way, in both readings, nested up to four deep (CREDENTIAL_NESTING), and a uri nested deeper may carry one: a bound that let the next level through would be the same gap one level down. It's the one function, so a tool result's line and a script's object both take it. - A data: link the host can't fetch showed its uri, so a base64 payload up to the uri limit reached the stored line as text. shownLinkUri now never shows a data: uri, any letter case: it holds its content rather than naming where it lies. Such a link gets the stub that says why it can't be fetched, with no new words. OAuth's `?code=` stays off the list: ordinary links use it, and an authorization code is single-use and needs the client's secret or PKCE verifier. Tests: the proxy shape at one and two levels, userinfo and a fragment's token in a nested uri, an ordinary nested uri shown at one level and at four, a fifth level withheld, and a value that reads as a path shown; a data: link, in either letter case, gets the stub on the line and no uri in a script's object, and from a server, the scheme's refusal. Both new tests fail against 17633af7's src, and the other 74 in the file pass. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- changelog.d/tool-result-blocks.fixed.md | 2 +- src/mcpl/references.ts | 44 ++++++++++++++++---- test/tool-result-blocks.test.ts | 49 +++++++++++++++++++++++ 4 files changed, 87 insertions(+), 10 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index a3950e818..64cd8ac05 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,4 +1,4 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cleaning drops controls and bidi marks and trims surrounding space, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. A parameter's value that is itself a uri, such as an image proxy's `?url=` holding a percent-encoded signed URL, is read the same way, four levels deep, and a uri nested deeper counts too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cleaning drops controls and bidi marks and trims surrounding space, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. - In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as that line leaves the registry, before the script lane's own registrations, so a script never sees, through a result's blocks, a uri the model's line withholds, within 2,000 references in one result: past that, either can lose an earlier result's `never`, and a script reading a module's content array can lose it where the line, rendering in order, hasn't yet. A result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. - A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. diff --git a/changelog.d/tool-result-blocks.fixed.md b/changelog.d/tool-result-blocks.fixed.md index 12c0c86b3..2427e6063 100644 --- a/changelog.d/tool-result-blocks.fixed.md +++ b/changelog.d/tool-result-blocks.fixed.md @@ -1,5 +1,5 @@ - A tool result's content array with a block agent-framework couldn't show, such as inline audio, an image with no media type or no data, or a block of a type it doesn't know, was stored, and sent to the model, as the whole array's JSON. Inline data's base64 went into the agent's context and history, and so did a block's uri, and an image beside it went as base64 text instead of an image. Each block now shows on its own, in the stored copy and the one the model reads alike: as itself, as a reference's stub, or as a marker saying what it was and why it isn't shown. Beside an image, the model reads them as blocks while the result's links and markers fit its inline cap together, and past that it reads the stored text, cut at the cap. Inline audio reads `[Audio not shown: this host's tool results carry only text and images (, ).]`, and the other blocks read the markers the push lane shows for them. An MCPL tool's result is content whatever its blocks' types. Any other array, such as a module's that doesn't set `dataIsContent`, is content only when every element is a block of one of MCP's types (text, image, audio, resource, resource_link). Any other array is data, written as JSON as before. Results stored before the upgrade keep the text they were stored with. -- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link whose uri is empty, null or not a string is the marker that says so. A link that names no uri at all is the marker as a block of a content array the model reads, and at depth, or in a script's view of a module's result, stays data, as before, since its fields hold no uri this reads. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed). On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. +- MCP's `resource_link` is a reference (RFC-005 §8.1): its `uri`, `name`, `mimeType` and `size` (as `sizeBytes`) make its record, with no disposition. On every lane, a link `fetch_reference` can fetch shows as a reference's stub, with its name, type and size. In a tool result, whose JSON showed every link's uri, one it can't fetch shows as `[resource link "": , ; not fetched]`, keeping the uri, the agent's only handle on it, as a standard MCP server's link shows. That holds at any depth in a tool result's data too, where a link's fields were written as data, and a link whose uri is empty, null or not a string is the marker that says so. A link that names no uri at all is the marker as a block of a content array the model reads, and at depth, or in a script's view of a module's result, stays data, as before, since its fields hold no uri this reads. That covers a uri that isn't http(s), one off its server's origin, one from a server with no address, and a link no server delivered. Where the uri may carry a credential, the link shows a stub instead (under Changed), and so does a link whose uri is a `data:` uri, which holds its content rather than naming where it lies. On every other lane, a link that can't be fetched shows the stub that says why it can't be fetched, with no uri, where the push lane showed an unknown-block marker. A link is registered with the server that sent it, but it's never fetched eagerly or on a script's demand: a uri a link names is fetched only when a block of the result that isn't a link names it too. A link's claims never replace a reference's in the registry, so a later link to a uri can't lift its expiry or its digest check. - A reference's stub now offers `fetch_reference` only where a fetch can reach the reference through a server that delivered it and the reference hasn't expired, as the servers' configs have it. Where two frameworks in one process configure one server id at different urls, a stub reads the address recorded most recently by a framework that hasn't since removed the server or stopped. A framework records its servers' addresses as it configures them and again as it starts. Otherwise it ends with why `fetch_reference` would refuse, without what calling again can do. The stub reads an expiry from the references in its result where they name its uri, and the fetch from the reference as the host last recorded it, so where one result names a uri both expired and fresh, the stub says it's expired while a fetch can succeed. A fetch's own limits, a spent budget or a claim over its ceiling, still apply when it's made. That covers a server with no address, such as a stdio one, a server the host doesn't know, one whose address doesn't parse, a uri off its server's origin, one with a user name or password, or a scheme the fetcher doesn't fetch. A third-party origin and a refused scheme are described there without naming the origins or the scheme. Before, the stub offered the fetch for any reference a server delivered. A server's id in a stub, or in `fetch_reference`'s refusal, is bounded and stripped of controls, as a label is, and so is a scheme in that refusal. - A label in a stub (a name, a type, a server's id, a provenance or a saved path) loses any line or paragraph separator (U+2028, U+2029), as it loses control characters. - A script's demand fetch of a reference not yet fetched reads that reference's own digest and expiry, where it read whatever an earlier reference to the same uri had claimed. diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 9ef36c78b..e1dae55a1 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -258,7 +258,8 @@ export function shownServerId(serverId: string): string { * `accessToken`, `access-token` and `access_token[]` are one name. RFC-005 §5: "Signed URLs and query capabilities are * bearer credentials that look like locations", so a uri is shown only where * the host's policy classifies it as non-capability-bearing, which here is - * one with no userinfo and none of these parameters (uriMayCarryCredential). + * one with no userinfo and none of these parameters, nor any uri in a + * parameter's value that has them (uriMayCarryCredential). * For example: * - AWS's signed URLs: `https://b.s3.amazonaws.com/k?X-Amz-Algorithm=…&X-Amz-Credential=…&X-Amz-Security-Token=…&X-Amz-Signature=…` * - Google Cloud Storage's: `https://storage.googleapis.com/b/k?X-Goog-Algorithm=…&X-Goog-Credential=…&X-Goog-Signature=…` @@ -301,9 +302,13 @@ function credentialName(name: string): string { * anchor (`#token`) or `?token=` doesn't. A `?` or `;` inside either may * separate parameters too, as in a route kept in a fragment * (`#/reset?token=…`), so the parameters are read both with and without - * them as separators, and one with a value in either reading counts. A uri - * that doesn't parse, even as a relative reference, may, since its parts - * can't be read. + * them as separators, and one with a value in either reading counts. A + * parameter's value that is itself an absolute uri, as an image proxy's + * `?url=` is once decoded, is read the same + * way, nested up to CREDENTIAL_NESTING deep, and a uri nested deeper may + * carry one, since a bound that let the next level through would be the + * same gap one level down. A uri that doesn't parse, even as a relative + * reference, may, since its parts can't be read. */ export function uriMayCarryCredential(uri: string): boolean { let u: URL; @@ -318,16 +323,37 @@ export function uriMayCarryCredential(uri: string): boolean { return true; } } + return urlMayCarryCredential(u, 0); +} + +/** How deep uriMayCarryCredential reads uris nested in parameters' values, + * each one a parameter's value in the uri before it. */ +const CREDENTIAL_NESTING = 4; + +function urlMayCarryCredential(u: URL, depth: number): boolean { if (u.username !== '' || u.password !== '') return true; // Read both ways: with '&' alone separating parameters, as a server // reading the query as WHATWG does sees them, and with '?' and ';' too, so // a value that opens with one (`token=;…`) still counts in the first. const named = (params: string) => [params, params.replace(/[?;]/g, '&')].some((read) => - [...new URLSearchParams(read).entries()] - .some(([name, value]) => value !== '' && CREDENTIAL_PARAMETERS.has(credentialName(name)))); + [...new URLSearchParams(read).entries()].some(([name, value]) => value !== '' + && (CREDENTIAL_PARAMETERS.has(credentialName(name)) || nestedMayCarry(value, depth)))); return named(u.search) || named(u.hash.slice(1)); } +/** Whether a parameter's value, decoded, is an absolute uri that may carry + * a credential. Only an absolute one is read, so an ordinary value never + * meets the relative reading's fail-closed answer. */ +function nestedMayCarry(value: string, depth: number): boolean { + let inner: URL; + try { + inner = new URL(value); + } catch { + return false; + } + return depth >= CREDENTIAL_NESTING || urlMayCarryCredential(inner, depth + 1); +} + /** How the host reaches a server, as far as fetching its references goes: * the WebSocket url it dials an MCPL server at, or none: for a server it * spawns, such as a stdio one, or a modern MCP server over HTTP, whose @@ -915,7 +941,9 @@ export function claimedExpiry(record: ReferenceRecord, claims: ResultClaims | un * The uri a tool result's link shows, or null where it shows none: a link * (fromLink) the host can't fetch, whose uri can't carry a credential, and * that no reference withholds with `disposition: never`, keeps its uri, the - * agent's only handle on it. A tool result's link line (buildReferenceStub, + * agent's only handle on it. A `data:` uri holds its content rather than + * naming where it lies, so it's never shown: a link's line isn't a way for + * inline data to reach context. A tool result's link line (buildReferenceStub, * unfetchedLinkUri) and a script's object for the link * (referenceBlockForScript) both ask this, so the model and a script, whose * output comes back to the model, see a link's uri alike. @@ -932,7 +960,7 @@ export function shownLinkUri(t: ReferenceTestimony, record: ReferenceRecord, now if (!t.fromLink || claimedNever(record, claims)) return null; if (unfetchableReason(record, nowMs, claims) === null) return null; const uri = shownUri(t.uri); - return uriMayCarryCredential(uri) ? null : uri; + return uriMayCarryCredential(uri) || /^data:/i.test(uri) ? null : uri; } /** diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index c3d0e9fb4..3fbddcd62 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -354,6 +354,55 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { } }); + it("reads a uri in a parameter's value as it reads the link's own, four levels deep, and withholds one nested deeper", () => { + const presigned = `https://zz-bucket.s3.amazonaws.com/k.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=zz&X-Amz-Signature=${SECRET}`; + const line = (uri: string) => toolResultDataToHistoryString([TEXT, { type: 'resource_link', uri, name: 'report.csv' }]).split('\n')[1]!; + const stub = /^\[ref_[^\]]+\] report\.csv — from tool result — it's not tied to any server, so there's nowhere to fetch it from$/; + // An image proxy's shape: the signed URL percent-encoded as one + // parameter's value, one decode level down, and two. + const once = `https://zz-site.example/_next/image?url=${encodeURIComponent(presigned)}&w=640`; + const twice = `https://zz-camo.example/?u=${encodeURIComponent(once)}`; + // Userinfo in a nested uri, and a credential in a nested uri's fragment. + const userinfo = `https://zz-site.example/go?to=${encodeURIComponent('https://zz-user:zz-pass@zz-elsewhere.example/r.csv')}`; + const fragment = `https://zz-site.example/go#next=${encodeURIComponent(`https://zz-elsewhere.example/cb#access_token=${SECRET}`)}`; + for (const uri of [once, twice, userinfo, fragment]) assert.match(line(uri), stub, uri); + // An ordinary nested uri shows, four levels down too; a fifth level is + // past the bound, and withheld whatever it holds. + const nest = (levels: number) => { + let uri = 'https://zz-files.example/page?v=1'; + for (let i = 0; i < levels; i += 1) uri = `https://zz-site${i}.example/r?next=${encodeURIComponent(uri)}`; + return uri; + }; + for (const levels of [1, 4]) { + const uri = nest(levels); + assert.equal(line(uri), `[resource link "report.csv": ${uri}; not fetched]`, String(levels)); + } + assert.match(line(nest(5)), stub); + // A value that isn't an absolute uri is a value, though it reads as a + // path: the relative reading's fail-closed answer never meets it. + const plain = 'https://zz-files.example/search?q=zz%20elsewhere%20report&page=zz-reports/2'; + assert.equal(line(plain), `[resource link "report.csv": ${plain}; not fetched]`); + }); + + it("never shows a data: link's uri, which holds its content rather than naming where it lies", () => { + const payload = 'iVBORw0KGgo'.repeat(40); + for (const uri of [`data:image/png;base64,${payload}`, `DATA:text/plain,${payload}`]) { + const link = { type: 'resource_link', uri, name: 'inline.png' }; + const shown = toolResultDataToHistoryString([TEXT, link]); + assert.match(shown, /^zz-text\n\[ref_[^\]]+\] inline\.png — from tool result — it's not tied to any server, so there's nowhere to fetch it from$/, uri.slice(0, 20)); + assert.ok(!shown.includes(payload), shown.slice(0, 200)); + // A script's object for it carries no uri either, as the line shows none. + const object = referenceBlockForScript(link, undefined) as { uri?: string }; + assert.equal(object.uri, undefined); + } + // From a server, the stub says why the fetch is refused, without the scheme. + servingFiles(); + const link = { type: 'resource_link', uri: `data:image/png;base64,${payload}`, name: 'inline.png' }; + referenceRegistry.register(testimonyOf(link), 'zz-files'); + const shown = toolResultDataToHistoryString([TEXT, link]); + assert.ok(shown.endsWith('inline.png — from tool result — scheme not allowed: this host fetches only http and https uris') && !shown.includes(payload), shown.slice(0, 200)); + }); + it("in a tool result, shows the stub that says why where the host can't fetch it and the uri may carry a credential", () => { const carriers = [ // userinfo, whatever the scheme From 24ec92b1f2372da2d3dfadd5364c3a7425ecc5e2 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 13:11:59 -0400 Subject: [PATCH 35/40] fix(references): a relative uri in a parameter's value, and one encoded twice inside it, are read for credentials MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two shapes past 056b5b5d's nested reading, each by run there: - A relative uri in a parameter's value: a login's `?next=%2Freset%3Ftoken%3D…`, or a protocol-relative `?url=%2F%2Fuser%3Apass%40host%2F…`. Unencoded, the first was already caught through the `?`-as-separator reading, so encoding the value was what let it past. - A uri encoded twice inside one parameter, as a redirector that encodes an already encoded url leaves it: URLSearchParams decodes a value once, and what's left begins `https%3A…`, which doesn't parse. nestedMayCarry now reads each value in this order: as an absolute uri; else, where its form is a relative reference's (it starts with `/`, or holds `?` or `#`), resolved against the uri it sits in, less that uri's query and fragment; else, where decoding it once more changes it, decoded and read again from the start. Each read and each decode is a level within CREDENTIAL_NESTING, and a uri reached past it may carry one. A value that is none of these, such as a word or a malformed `%`, answers false, and a decode alone never withholds, so an ordinary value never fails closed. The base drops the query and fragment because a value of a fragment alone, such as a colour's `#ff0000`, resolved against the whole uri, brings back the query it sits in, and is read again at each level until the bound withholds an ordinary link (by run: `?color=%23ff0000`). Tests: the login and protocol-relative shapes; an absolute and a protocol-relative uri each encoded twice in one parameter; a decode counted as a level (three nestings under one value encoded once show, encoded twice they're past the bound); and controls that still show: `next=/home`, `lang=en:US&tag=type:image`, a value with a space, and a colour. Co-Authored-By: Claude Opus 5.5 --- changelog.d/tool-result-blocks.changed.md | 2 +- src/mcpl/references.ts | 59 +++++++++++++++++------ test/tool-result-blocks.test.ts | 35 ++++++++++++++ 3 files changed, 80 insertions(+), 16 deletions(-) diff --git a/changelog.d/tool-result-blocks.changed.md b/changelog.d/tool-result-blocks.changed.md index 64cd8ac05..4a41a4cf6 100644 --- a/changelog.d/tool-result-blocks.changed.md +++ b/changelog.d/tool-result-blocks.changed.md @@ -1,4 +1,4 @@ -- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. A parameter's value that is itself a uri, such as an image proxy's `?url=` holding a percent-encoded signed URL, is read the same way, four levels deep, and a uri nested deeper counts too. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cleaning drops controls and bidi marks and trims surrounding space, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. +- In an MCPL tool's result, or a module's, a link (MCP's `resource_link`) the host can't fetch whose uri may carry a credential now shows a stub that says why it can't be fetched, with no uri, where the result's JSON showed the uri. So an agent no longer sees that uri, and has no other handle on the link here. Such a uri has userinfo, or a query or fragment parameter that has a value and is named like a signed URL's or a token, such as `X-Amz-Signature`, `sig`, `hm`, `token`, `access_token` or `key`, compared by letters and digits alone, so `accessToken` and `token[]` count too. A parameter's value that is itself a uri is read the same way, whether absolute (an image proxy's `?url=` holding a percent-encoded signed URL), protocol-relative, or relative with a query or fragment (a login's `?next=%2Freset%3Ftoken%3D…`), and also once decoded again where a redirector encoded it twice, each read and decode a level, up to four; a uri reached deeper counts too, and one in another encoding, such as base64, isn't read. RFC-005 §5 gives the reason: signed URLs and query capabilities are bearer credentials that look like locations. So does a link to a uri that any reference in the same result marks `disposition: never`, or to a uri that shows as the same text (a label's cleaning drops controls and bidi marks and trims surrounding space, and a lone surrogate shows as U+FFFD), or, where no reference in the result names the uri (a link names it without claiming anything), whose reference from an earlier result, as the host's record of the uri holds it, did (the latest reference until its bytes are saved, then the one they were verified against), since RFC-005 §5 removes such a uri unconditionally. An ordinary link with a query, such as `watch?v=…`, and an anchor such as `#token`, which has no value, still show their uri. A standard MCP server's result, a modern server's or an MCP-only one's, isn't read this way: its links show their uri as the server sent it. - In a script's structured result, from an MCPL peer or a module, a reference arrives as its object without its uri, as before, except a link (MCP's `resource_link`), which carries `uri` where the model's line for it shows the uri. One the fetcher would refuse carries `unfetchable`, the words a stub gives in place of advising `fetch_reference`, as one that counts as expired carries `expired`. Both give why a `fetch_reference` with the object's `refId` would be refused, as the result's references claim it, so a link whose line shows its uri carries them too. An MCPL peer's link reached a script as the unknown-block notice, and a module's as its fields, its uri among them. A script's view reads the claims of every reference it shows, those nested in a block's fields included, so a `disposition: never` nested in one block withholds that uri from a link elsewhere in the same result, where the model's line, which shows a block and not its fields, can still show it. A link's object carries its uri only where the model's line for the result shows it, by the claims that line reads and the uri's record as that line leaves the registry, before the script lane's own registrations, so a script never sees, through a result's blocks, a uri the model's line withholds, within 2,000 references in one result: past that, either can lose an earlier result's `never`, and a script reading a module's content array can lose it where the line, rendering in order, hasn't yet. A result's structured content reaches a script as sent, as before. A script's demand fetch of an MCPL peer's references reads the same claims as its view, as a module's does, so a uri a reference nested in a block calls expired isn't fetched. - A uri inline data marks `disposition: never` beside its data, which is withheld, is withheld from every other block of the result too, such as a link naming it, as a reference's `never` is (RFC-005 §5). - Inline data beside an empty `uri` reaches a script as sent, from an MCPL peer as from a module, where 0.26.1 and 0.26.2 gave a script an MCPL peer's as the withheld notice. An empty uri carries nothing a credential could ride on. Inline data beside one that also claims a disposition is still withheld. diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index e1dae55a1..7b5cbcc8d 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -301,14 +301,20 @@ function credentialName(name: string): string { * `name=value&…` as a query is. A name with no value carries nothing, as an * anchor (`#token`) or `?token=` doesn't. A `?` or `;` inside either may * separate parameters too, as in a route kept in a fragment - * (`#/reset?token=…`), so the parameters are read both with and without - * them as separators, and one with a value in either reading counts. A - * parameter's value that is itself an absolute uri, as an image proxy's - * `?url=` is once decoded, is read the same - * way, nested up to CREDENTIAL_NESTING deep, and a uri nested deeper may - * carry one, since a bound that let the next level through would be the - * same gap one level down. A uri that doesn't parse, even as a relative - * reference, may, since its parts can't be read. + * (`#/reset?token=…`), so the parameters are read both with and without them + * as separators, and one with a value in either reading counts. A + * parameter's value that is itself a uri is read the same way + * (nestedMayCarry): an absolute one, as an image proxy's `?url=` is once decoded; a protocol-relative one, or + * a relative one with a query or fragment, as a login's + * `?next=%2Freset%3Ftoken%3D…` is; and one encoded again inside the value, + * as a redirector that encodes an already encoded url leaves it, each read + * and each decode a level of its own. That goes up to CREDENTIAL_NESTING + * levels deep, and a uri reached deeper may carry one, since a bound that + * let the next level through would be the same gap one level down. A uri + * carried in another encoding, such as base64, isn't read. A uri that + * doesn't parse, even as a relative reference, may, since its parts can't be + * read. */ export function uriMayCarryCredential(uri: string): boolean { let u: URL; @@ -337,21 +343,44 @@ function urlMayCarryCredential(u: URL, depth: number): boolean { // a value that opens with one (`token=;…`) still counts in the first. const named = (params: string) => [params, params.replace(/[?;]/g, '&')].some((read) => [...new URLSearchParams(read).entries()].some(([name, value]) => value !== '' - && (CREDENTIAL_PARAMETERS.has(credentialName(name)) || nestedMayCarry(value, depth)))); + && (CREDENTIAL_PARAMETERS.has(credentialName(name)) || nestedMayCarry(value, u, depth)))); return named(u.search) || named(u.hash.slice(1)); } -/** Whether a parameter's value, decoded, is an absolute uri that may carry - * a credential. Only an absolute one is read, so an ordinary value never - * meets the relative reading's fail-closed answer. */ -function nestedMayCarry(value: string, depth: number): boolean { - let inner: URL; +/** Whether a parameter's value is a uri that may carry a credential, read in + * this order: as an absolute uri; else, where its form is a relative + * reference's (it starts with `/`, or holds `?` or `#`), resolved against + * the uri it sits in, less that uri's query and fragment, since a value of + * a fragment alone, such as a colour's `#ff0000`, would otherwise bring + * back the query it sits in and be read again at each level; else, where + * decoding it once more changes it, decoded and read again from the start. + * Each read or decode is a level. A value that is none of these, such as a + * word, or a malformed `%`, answers false, so an ordinary value never meets + * the relative reading's fail-closed answer. */ +function nestedMayCarry(value: string, outer: URL, depth: number): boolean { + let inner: URL | null = null; try { inner = new URL(value); + } catch { + if (/^\/|[?#]/.test(value)) { + try { + const base = new URL(outer.href); + base.search = ''; + base.hash = ''; + inner = new URL(value, base); + } catch { + inner = null; + } + } + } + if (inner !== null) return depth >= CREDENTIAL_NESTING || urlMayCarryCredential(inner, depth + 1); + let decoded: string; + try { + decoded = decodeURIComponent(value); } catch { return false; } - return depth >= CREDENTIAL_NESTING || urlMayCarryCredential(inner, depth + 1); + return decoded !== value && nestedMayCarry(decoded, outer, depth + 1); } /** How the host reaches a server, as far as fetching its references goes: diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 3fbddcd62..c2c7a2c63 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -384,6 +384,41 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { assert.equal(line(plain), `[resource link "report.csv": ${plain}; not fetched]`); }); + it("reads a relative uri in a parameter's value, and one encoded again inside it, each read and decode a level", () => { + const presigned = `https://zz-bucket.s3.amazonaws.com/k.png?X-Amz-Signature=${SECRET}`; + const line = (uri: string) => toolResultDataToHistoryString([TEXT, { type: 'resource_link', uri, name: 'report.csv' }]).split('\n')[1]!; + const stub = /^\[ref_[^\]]+\] report\.csv — from tool result — it's not tied to any server, so there's nowhere to fetch it from$/; + const twice = (value: string) => encodeURIComponent(encodeURIComponent(value)); + for (const uri of [ + // A relative one with a query, and a protocol-relative one with userinfo. + `https://zz-app.example/login?next=${encodeURIComponent(`/reset?token=${SECRET}`)}`, + `https://zz-proxy.example/img?url=${encodeURIComponent('//zz-user:zz-pass@zz-bucket.example/k')}`, + // Encoded twice inside one parameter, as a redirector leaves it: an + // absolute one, and a protocol-relative one read once decoded. + `https://zz-r.example/go?u=${twice(presigned)}`, + `https://zz-r.example/go?u=${twice('//zz-user:zz-pass@zz-bucket.example/k')}`, + ]) assert.match(line(uri), stub, uri); + // A decode is a level: three nestings below one value encoded twice put + // the fourth at level five, past the bound, where encoded once it's read. + const nest = (levels: number) => { + let uri = 'https://zz-files.example/page?v=1'; + for (let i = 0; i < levels; i += 1) uri = `https://zz-site${i}.example/r?next=${encodeURIComponent(uri)}`; + return uri; + }; + const once = `https://zz-r.example/go?u=${encodeURIComponent(nest(3))}`; + assert.equal(line(once), `[resource link "report.csv": ${once}; not fetched]`); + assert.match(line(`https://zz-r.example/go?u=${twice(nest(3))}`), stub); + // Ordinary values still show: a path, values with a colon, a value with + // a space, and a colour, whose fragment alone resolves against the uri + // less its query, so it isn't read again at each level. + for (const uri of [ + 'https://zz-files.example/x?next=/home', + 'https://zz-files.example/x?lang=en:US&tag=type:image', + 'https://zz-files.example/x?q=zz%20elsewhere%20report', + 'https://zz-files.example/badge?color=%23ff0000&label=%23zz', + ]) assert.equal(line(uri), `[resource link "report.csv": ${uri}; not fetched]`, uri); + }); + it("never shows a data: link's uri, which holds its content rather than naming where it lies", () => { const payload = 'iVBORw0KGgo'.repeat(40); for (const uri of [`data:image/png;base64,${payload}`, `DATA:text/plain,${payload}`]) { From 7ca4e7a71ef6873aaae7404a8e88136a8d499e37 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 13:15:46 -0400 Subject: [PATCH 36/40] fix(references): a value's decodes stop at the bound, and one that would still decode past it is withheld From the reads of 24ec92b1: the decode step had no bound of its own, only progress, so a value built to change under every decode (`%` and `25` repeated) was decoded about 2,000 times in 4,096 characters, tens of milliseconds per crafted link, read two ways and on both lanes, so a result of a few hundred such links would hold the event loop for seconds. Now a decode past CREDENTIAL_NESTING answers true where the value would still change: the check's work stops where its reading does, and a uri found deeper would be withheld anyway. Within the bound a decode alone still never withholds, so `%20` stays false. Tests: a plain value percent-encoded five times over in its uri, four decodes past the parameter's own, shows, and six times over is withheld; the controls still show. Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 8 ++++++-- test/tool-result-blocks.test.ts | 7 +++++++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 7b5cbcc8d..01bbea95d 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -356,7 +356,8 @@ function urlMayCarryCredential(u: URL, depth: number): boolean { * decoding it once more changes it, decoded and read again from the start. * Each read or decode is a level. A value that is none of these, such as a * word, or a malformed `%`, answers false, so an ordinary value never meets - * the relative reading's fail-closed answer. */ + * the relative reading's fail-closed answer; within the bound a decode + * alone never withholds, and past it a value that would still decode does. */ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { let inner: URL | null = null; try { @@ -380,7 +381,10 @@ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { } catch { return false; } - return decoded !== value && nestedMayCarry(decoded, outer, depth + 1); + if (decoded === value) return false; + // Past the bound, a value that would still decode is withheld: the check's + // work stops where its reading does, and a uri found deeper would be. + return depth >= CREDENTIAL_NESTING || nestedMayCarry(decoded, outer, depth + 1); } /** How the host reaches a server, as far as fetching its references goes: diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index c2c7a2c63..8fa752de7 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -408,6 +408,13 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { const once = `https://zz-r.example/go?u=${encodeURIComponent(nest(3))}`; assert.equal(line(once), `[resource link "report.csv": ${once}; not fetched]`); assert.match(line(`https://zz-r.example/go?u=${twice(nest(3))}`), stub); + // A value percent-encoded past the bound is withheld, uri or not, so the + // check's work stops where its reading does; one level less shows. + let encoded = 'zz plain words'; + for (let i = 0; i < 5; i += 1) encoded = encodeURIComponent(encoded); + const four = `https://zz-files.example/x?q=${encoded}`; + assert.equal(line(four), `[resource link "report.csv": ${four}; not fetched]`); + assert.match(line(`https://zz-files.example/x?q=${encodeURIComponent(encoded)}`), stub); // Ordinary values still show: a path, values with a colon, a value with // a space, and a colour, whose fragment alone resolves against the uri // less its query, so it isn't read again at each level. From 7c5a7c0570988ecfdfca1d0e7c1c2f04e9df3e6e Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 13:19:41 -0400 Subject: [PATCH 37/40] fix(references): a value is decoded again leniently, as the parameter itself was From the reads of 7ca4e7a7: the top level decodes a parameter leniently (URLSearchParams), leaving a stray `%` as it is, but the step that decodes a value again used decodeURIComponent, which throws on one. So a value encoded twice with one stray `%` appended answered false at its first decode, and the uri inside it, which a lenient server would still reach, was never read: the code not matching what the description says it reads. The step now decodes as the top level does, through URLSearchParams, with `+` and `&` escaped first so they come back as themselves: unescaped, an `&` that the first decode made out of `%26` would split the value and leave only the part before it. Tests: a protocol-relative uri with userinfo encoded twice with a stray `%` appended, and a relative value whose `&` precedes its `?token=`, each withheld; a value with a stray `%` and no uri still shows. Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 15 +++++++-------- test/tool-result-blocks.test.ts | 5 +++++ 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 01bbea95d..beb3f27a1 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -354,8 +354,9 @@ function urlMayCarryCredential(u: URL, depth: number): boolean { * a fragment alone, such as a colour's `#ff0000`, would otherwise bring * back the query it sits in and be read again at each level; else, where * decoding it once more changes it, decoded and read again from the start. - * Each read or decode is a level. A value that is none of these, such as a - * word, or a malformed `%`, answers false, so an ordinary value never meets + * Each read or decode is a level, and a decode is lenient, as the + * parameter's own was, so a stray `%` doesn't stop it. A value that is none + * of these, such as a word, answers false, so an ordinary value never meets * the relative reading's fail-closed answer; within the bound a decode * alone never withholds, and past it a value that would still decode does. */ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { @@ -375,12 +376,10 @@ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { } } if (inner !== null) return depth >= CREDENTIAL_NESTING || urlMayCarryCredential(inner, depth + 1); - let decoded: string; - try { - decoded = decodeURIComponent(value); - } catch { - return false; - } + // Decoded as the parameter itself was, leniently: a stray `%` stays as it + // is rather than stopping the reading, as it doesn't stop a server's. + // `+` and `&` are escaped first, so they come back as themselves. + const decoded = new URLSearchParams(`v=${value.replace(/[+&]/g, encodeURIComponent)}`).get('v') ?? value; if (decoded === value) return false; // Past the bound, a value that would still decode is withheld: the check's // work stops where its reading does, and a uri found deeper would be. diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 8fa752de7..4191a2b93 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -397,6 +397,10 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { // absolute one, and a protocol-relative one read once decoded. `https://zz-r.example/go?u=${twice(presigned)}`, `https://zz-r.example/go?u=${twice('//zz-user:zz-pass@zz-bucket.example/k')}`, + // Decoded leniently, as the parameter itself was: a stray `%` doesn't + // stop the reading, and an `&` the first decode made stays in the value. + `https://zz-r.example/go?u=${twice('//zz-user:zz-pass@zz-bucket.example/k')}%`, + `https://zz-r.example/go?u=${encodeURIComponent(`%2Fpath&x%3Ftoken%3D${SECRET}`)}`, ]) assert.match(line(uri), stub, uri); // A decode is a level: three nestings below one value encoded twice put // the fourth at level five, past the bound, where encoded once it's read. @@ -422,6 +426,7 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { 'https://zz-files.example/x?next=/home', 'https://zz-files.example/x?lang=en:US&tag=type:image', 'https://zz-files.example/x?q=zz%20elsewhere%20report', + 'https://zz-files.example/x?q=100%25%20off%', 'https://zz-files.example/badge?color=%23ff0000&label=%23zz', ]) assert.equal(line(uri), `[resource link "report.csv": ${uri}; not fetched]`, uri); }); From 15c7eb6598e6743a9a6bce2459d80fd016792777 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 13:25:53 -0400 Subject: [PATCH 38/40] fix(references): a relative value that doesn't resolve may carry a credential, as a uri that doesn't parse may From the reads of 7c5a7c05: uriMayCarryCredential withholds a uri that doesn't parse, since its parts can't be read, but a parameter's value that is a relative reference by its form and doesn't resolve fell through to the decode, which left it unchanged, and answered false. Only a protocol-relative value whose authority doesn't parse comes there (`//user:pass@host%` once decoded, `// note`, `//host:99999/x`; a path, query or fragment value always resolves), and the first of those shows userinfo in the outer uri's text however a server would read it. Such a value now answers true, the documented rule applied to the branch that skipped it. The cost, named in the comment and the description: an ordinary value starting with `//` whose authority doesn't parse, such as `q=// note`, withholds its link. Tests: `//zz-user:zz-pass@zz-host%` encoded twice in a parameter is withheld; `/a b?c d#e f` as a value still shows. Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 20 ++++++++++++-------- test/tool-result-blocks.test.ts | 4 ++++ 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index beb3f27a1..f9d80a5f0 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -352,13 +352,13 @@ function urlMayCarryCredential(u: URL, depth: number): boolean { * reference's (it starts with `/`, or holds `?` or `#`), resolved against * the uri it sits in, less that uri's query and fragment, since a value of * a fragment alone, such as a colour's `#ff0000`, would otherwise bring - * back the query it sits in and be read again at each level; else, where - * decoding it once more changes it, decoded and read again from the start. - * Each read or decode is a level, and a decode is lenient, as the - * parameter's own was, so a stray `%` doesn't stop it. A value that is none - * of these, such as a word, answers false, so an ordinary value never meets - * the relative reading's fail-closed answer; within the bound a decode - * alone never withholds, and past it a value that would still decode does. */ + * back the query it sits in and be read again at each level, and one that + * doesn't resolve may carry one; else, where decoding it once more changes + * it, decoded and read again from the start. Each read or decode is a + * level, and a decode is lenient, as the parameter's own was, so a stray + * `%` doesn't stop it. A value that is none of these, such as a word, + * answers false; within the bound a decode alone never withholds, and past + * it a value that would still decode does. */ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { let inner: URL | null = null; try { @@ -371,7 +371,11 @@ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { base.hash = ''; inner = new URL(value, base); } catch { - inner = null; + // One that doesn't resolve may carry one, as a uri that doesn't parse + // may: its parts can't be read. Only a protocol-relative value whose + // authority doesn't parse comes here, so an ordinary value such as + // `q=// note` is withheld too. + return true; } } } diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 4191a2b93..b003e0d94 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -401,6 +401,9 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { // stop the reading, and an `&` the first decode made stays in the value. `https://zz-r.example/go?u=${twice('//zz-user:zz-pass@zz-bucket.example/k')}%`, `https://zz-r.example/go?u=${encodeURIComponent(`%2Fpath&x%3Ftoken%3D${SECRET}`)}`, + // A relative value that doesn't resolve can't be read, so it may carry + // one, as a uri that doesn't parse may: here its authority holds a `%`. + `https://zz-r.example/go?u=${twice('//zz-user:zz-pass@zz-host%')}`, ]) assert.match(line(uri), stub, uri); // A decode is a level: three nestings below one value encoded twice put // the fourth at level five, past the bound, where encoded once it's read. @@ -427,6 +430,7 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { 'https://zz-files.example/x?lang=en:US&tag=type:image', 'https://zz-files.example/x?q=zz%20elsewhere%20report', 'https://zz-files.example/x?q=100%25%20off%', + `https://zz-files.example/x?q=${encodeURIComponent('/a b?c d#e f')}`, 'https://zz-files.example/badge?color=%23ff0000&label=%23zz', ]) assert.equal(line(uri), `[resource link "report.csv": ${uri}; not fetched]`, uri); }); From d28980eb7fc01e33bb775239e548531d9482a351 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 13:29:23 -0400 Subject: [PATCH 39/40] fix(references): only a protocol-relative value that doesn't resolve may carry a credential; a backslash one is read as relative From the reads of 15c7eb65: a uri with an opaque path, such as a `mailto:`, `urn:` or `tag:` one, can't be a base, so every relative value inside it that isn't a fragment alone fails to resolve, and since 15c7eb65 answered true: an ordinary `mailto:help@example.com?subject=Question%3F` withheld its link, where nothing resolves against such a uri, so nothing hides behind it (by run). Now a resolve that throws answers true only where the value is protocol-relative, `//` or the backslash form a special scheme reads as `//`, whose authority can't be read; any other answers false. A value starting with a backslash is read as a relative reference too, so the backslash form is resolved and its userinfo read rather than passed. Tests: a backslash protocol-relative value with userinfo is withheld, `//zz-user:zz-pass@zz-host%` encoded twice stays withheld, and a `mailto:` with `subject=Question%3F` shows. Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 34 +++++++++++++++++---------------- test/tool-result-blocks.test.ts | 5 +++++ 2 files changed, 23 insertions(+), 16 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index f9d80a5f0..468c24956 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -349,33 +349,35 @@ function urlMayCarryCredential(u: URL, depth: number): boolean { /** Whether a parameter's value is a uri that may carry a credential, read in * this order: as an absolute uri; else, where its form is a relative - * reference's (it starts with `/`, or holds `?` or `#`), resolved against - * the uri it sits in, less that uri's query and fragment, since a value of - * a fragment alone, such as a colour's `#ff0000`, would otherwise bring - * back the query it sits in and be read again at each level, and one that - * doesn't resolve may carry one; else, where decoding it once more changes - * it, decoded and read again from the start. Each read or decode is a - * level, and a decode is lenient, as the parameter's own was, so a stray - * `%` doesn't stop it. A value that is none of these, such as a word, - * answers false; within the bound a decode alone never withholds, and past - * it a value that would still decode does. */ + * reference's (it starts with `/` or `\`, or holds `?` or `#`), resolved + * against the uri it sits in, less that uri's query and fragment, since a + * value of a fragment alone, such as a colour's `#ff0000`, would otherwise + * bring back the query it sits in and be read again at each level, and a + * protocol-relative one that doesn't resolve may carry one; else, where + * decoding it once more changes it, decoded and read again from the start. + * Each read or decode is a level, and a decode is lenient, as the + * parameter's own was, so a stray `%` doesn't stop it. A value that is none + * of these, such as a word, answers false; within the bound a decode alone + * never withholds, and past it a value that would still decode does. */ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { let inner: URL | null = null; try { inner = new URL(value); } catch { - if (/^\/|[?#]/.test(value)) { + if (/^[/\\]|[?#]/.test(value)) { try { const base = new URL(outer.href); base.search = ''; base.hash = ''; inner = new URL(value, base); } catch { - // One that doesn't resolve may carry one, as a uri that doesn't parse - // may: its parts can't be read. Only a protocol-relative value whose - // authority doesn't parse comes here, so an ordinary value such as - // `q=// note` is withheld too. - return true; + // A protocol-relative one (`//`, or `\\` as a special scheme reads it) + // that doesn't resolve may carry one, as a uri that doesn't parse may: + // its authority can't be read. So an ordinary value such as + // `q=// note` is withheld too. Any other relative value that doesn't + // resolve sits in a uri that can't be a base, such as a `mailto:`'s + // subject, against which nothing resolves it, so it carries nothing. + return /^[/\\]{2}/.test(value); } } } diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index b003e0d94..57230a363 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -404,6 +404,8 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { // A relative value that doesn't resolve can't be read, so it may carry // one, as a uri that doesn't parse may: here its authority holds a `%`. `https://zz-r.example/go?u=${twice('//zz-user:zz-pass@zz-host%')}`, + // The backslash form a special scheme reads as `//`. + `https://zz-r.example/go?u=${encodeURIComponent('\\\\zz-user:zz-pass@zz-bucket.example\\k')}`, ]) assert.match(line(uri), stub, uri); // A decode is a level: three nestings below one value encoded twice put // the fourth at level five, past the bound, where encoded once it's read. @@ -431,6 +433,9 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { 'https://zz-files.example/x?q=zz%20elsewhere%20report', 'https://zz-files.example/x?q=100%25%20off%', `https://zz-files.example/x?q=${encodeURIComponent('/a b?c d#e f')}`, + // Nothing resolves against a uri with an opaque path, so a relative + // value there that doesn't resolve carries nothing. + 'mailto:zz-help@zz-files.example?subject=Question%3F', 'https://zz-files.example/badge?color=%23ff0000&label=%23zz', ]) assert.equal(line(uri), `[resource link "report.csv": ${uri}; not fetched]`, uri); }); From 3bdc670970b59c67db064a2e2b61d2f78e4c02fe Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Sat, 24 Oct 2026 13:33:40 -0400 Subject: [PATCH 40/40] fix(references): a relative value is read against one fixed base, by its form as the parser reads it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From the reads of d28980eb: a uri with an opaque path, such as a `mailto:` one, can't be a base, so every relative value in its parameters but a fragment alone failed to resolve and went unread: `mailto:x@y?body=%3Ftoken%3D…` showed, where the same value in an https uri's parameter is withheld (by run). The outer uri was never needed as the base: stripped of its query and fragment it supplied only a scheme, host and path, and a credential lives in the value's own authority, query or fragment. So nestedMayCarry resolves every relative value against RELATIVE_BASE, `http://relative.invalid/`, the base uriMayCarryCredential already reads a relative uri against, and drops its `outer` parameter and the lines that copied and stripped the outer uri. A backslash value inside a non-special outer uri now reads as protocol-relative, which is stricter. A value's form is now tested as the URL parser reads it, which first trims C0 controls and spaces: `\t//user:pass@host/k`, ` //…` and `\u0001//…` resolve as protocol-relative uris with that userinfo, and were never resolved (by run). The tabs and newlines the parser also drops can't change the form once the lead is trimmed, since none is a `/`, `\`, `?` or `#`, so the test is one regex. And a value that fails to resolve answers true: a fuzz of about 137,000 relative-form values against the fixed base found every one that threw protocol-relative as the parser reads it, on its authority, so the narrower test there was redundant where it was right and leaked where it read the raw value. Tests: a `mailto:` body holding `?token=` is withheld, and so are the tab-, space- and C0-led protocol-relative values with userinfo, and a tab-led one whose authority doesn't parse, through the catch; the `mailto:` subject `Question%3F`, the colours and the other controls still show. Every earlier probe of this round answers as before (19 withheld, 17 shown). Co-Authored-By: Claude Opus 5.5 --- src/mcpl/references.ts | 53 +++++++++++++++++---------------- test/tool-result-blocks.test.ts | 15 +++++++--- 2 files changed, 39 insertions(+), 29 deletions(-) diff --git a/src/mcpl/references.ts b/src/mcpl/references.ts index 468c24956..c03f439f5 100644 --- a/src/mcpl/references.ts +++ b/src/mcpl/references.ts @@ -324,7 +324,7 @@ export function uriMayCarryCredential(uri: string): boolean { u = new URL(uri); } catch { try { - u = new URL(uri, 'http://relative.invalid/'); + u = new URL(uri, RELATIVE_BASE); } catch { return true; } @@ -332,6 +332,11 @@ export function uriMayCarryCredential(uri: string): boolean { return urlMayCarryCredential(u, 0); } +/** The base a relative uri is read against: it supplies only a scheme and a + * path, and a credential lives in the uri's own authority, query or + * fragment. */ +const RELATIVE_BASE = 'http://relative.invalid/'; + /** How deep uriMayCarryCredential reads uris nested in parameters' values, * each one a parameter's value in the uri before it. */ const CREDENTIAL_NESTING = 4; @@ -343,41 +348,39 @@ function urlMayCarryCredential(u: URL, depth: number): boolean { // a value that opens with one (`token=;…`) still counts in the first. const named = (params: string) => [params, params.replace(/[?;]/g, '&')].some((read) => [...new URLSearchParams(read).entries()].some(([name, value]) => value !== '' - && (CREDENTIAL_PARAMETERS.has(credentialName(name)) || nestedMayCarry(value, u, depth)))); + && (CREDENTIAL_PARAMETERS.has(credentialName(name)) || nestedMayCarry(value, depth)))); return named(u.search) || named(u.hash.slice(1)); } /** Whether a parameter's value is a uri that may carry a credential, read in * this order: as an absolute uri; else, where its form is a relative * reference's (it starts with `/` or `\`, or holds `?` or `#`), resolved - * against the uri it sits in, less that uri's query and fragment, since a - * value of a fragment alone, such as a colour's `#ff0000`, would otherwise - * bring back the query it sits in and be read again at each level, and a - * protocol-relative one that doesn't resolve may carry one; else, where - * decoding it once more changes it, decoded and read again from the start. - * Each read or decode is a level, and a decode is lenient, as the - * parameter's own was, so a stray `%` doesn't stop it. A value that is none - * of these, such as a word, answers false; within the bound a decode alone - * never withholds, and past it a value that would still decode does. */ -function nestedMayCarry(value: string, outer: URL, depth: number): boolean { + * against RELATIVE_BASE, never against the uri it sits in, which may have + * no path to resolve against, as a `mailto:` hasn't; else, where decoding + * it once more changes it, decoded and read again from the start. Each read + * or decode is a level, and a decode is lenient, as the parameter's own + * was, so a stray `%` doesn't stop it. A value that is none of these, such + * as a word, answers false; within the bound a decode alone never + * withholds, and past it a value that would still decode does. */ +function nestedMayCarry(value: string, depth: number): boolean { let inner: URL | null = null; try { inner = new URL(value); } catch { - if (/^[/\\]|[?#]/.test(value)) { + // Its form as the URL parser reads it, which first trims C0 controls and + // spaces, so `\t//user:pass@…` is the protocol-relative uri it resolves + // as. The tabs and newlines it also drops can't change this form once the + // lead is trimmed: none is a `/`, `\\`, `?` or `#`. + if (/^[\u0000-\u0020]*[/\\]|[?#]/.test(value)) { try { - const base = new URL(outer.href); - base.search = ''; - base.hash = ''; - inner = new URL(value, base); + inner = new URL(value, RELATIVE_BASE); } catch { - // A protocol-relative one (`//`, or `\\` as a special scheme reads it) - // that doesn't resolve may carry one, as a uri that doesn't parse may: - // its authority can't be read. So an ordinary value such as - // `q=// note` is withheld too. Any other relative value that doesn't - // resolve sits in a uri that can't be a base, such as a `mailto:`'s - // subject, against which nothing resolves it, so it carries nothing. - return /^[/\\]{2}/.test(value); + // Against a hierarchical base only a value that is protocol-relative + // as the parser reads it (`//`, or `\\` as a special scheme reads it) + // fails to resolve, on its authority, and it may carry one, as a uri + // that doesn't parse may: its authority can't be read. So an + // ordinary value such as `q=// note` is withheld too. + return true; } } } @@ -389,7 +392,7 @@ function nestedMayCarry(value: string, outer: URL, depth: number): boolean { if (decoded === value) return false; // Past the bound, a value that would still decode is withheld: the check's // work stops where its reading does, and a uri found deeper would be. - return depth >= CREDENTIAL_NESTING || nestedMayCarry(decoded, outer, depth + 1); + return depth >= CREDENTIAL_NESTING || nestedMayCarry(decoded, depth + 1); } /** How the host reaches a server, as far as fetching its references goes: diff --git a/test/tool-result-blocks.test.ts b/test/tool-result-blocks.test.ts index 57230a363..b6d2b7c62 100644 --- a/test/tool-result-blocks.test.ts +++ b/test/tool-result-blocks.test.ts @@ -406,6 +406,14 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { `https://zz-r.example/go?u=${twice('//zz-user:zz-pass@zz-host%')}`, // The backslash form a special scheme reads as `//`. `https://zz-r.example/go?u=${encodeURIComponent('\\\\zz-user:zz-pass@zz-bucket.example\\k')}`, + // Its form read as the URL parser reads it: led by a tab, a space or a + // C0 control, a protocol-relative value is still one, and one whose + // authority doesn't parse is withheld through its failed resolve. + ...['\t', ' ', '\u0001'].map((lead) => `https://zz-r.example/go?u=${encodeURIComponent(`${lead}//zz-user:zz-pass@zz-bucket.example/k`)}`), + `https://zz-r.example/go?u=${encodeURIComponent('\t//zz-user:zz-pass@zz-host%?x')}`, + // Read against a fixed base, so a uri with no path to resolve against, + // such as a mailto:, still has its parameters' relative values read. + `mailto:zz-help@zz-files.example?body=${encodeURIComponent(`?token=${SECRET}`)}`, ]) assert.match(line(uri), stub, uri); // A decode is a level: three nestings below one value encoded twice put // the fourth at level five, past the bound, where encoded once it's read. @@ -425,16 +433,15 @@ describe("MCP's resource_link, a reference (RFC-005 §8.1)", () => { assert.equal(line(four), `[resource link "report.csv": ${four}; not fetched]`); assert.match(line(`https://zz-files.example/x?q=${encodeURIComponent(encoded)}`), stub); // Ordinary values still show: a path, values with a colon, a value with - // a space, and a colour, whose fragment alone resolves against the uri - // less its query, so it isn't read again at each level. + // a space, and a colour, a fragment alone with no parameter in it. for (const uri of [ 'https://zz-files.example/x?next=/home', 'https://zz-files.example/x?lang=en:US&tag=type:image', 'https://zz-files.example/x?q=zz%20elsewhere%20report', 'https://zz-files.example/x?q=100%25%20off%', `https://zz-files.example/x?q=${encodeURIComponent('/a b?c d#e f')}`, - // Nothing resolves against a uri with an opaque path, so a relative - // value there that doesn't resolve carries nothing. + // A relative value with an empty query carries nothing, in a mailto: + // as anywhere. 'mailto:zz-help@zz-files.example?subject=Question%3F', 'https://zz-files.example/badge?color=%23ff0000&label=%23zz', ]) assert.equal(line(uri), `[resource link "report.csv": ${uri}; not fetched]`, uri);