feat(issue-545): cache mcp tool lists - #702
Merged
Merged
Conversation
…eResult (#545) Client (McpManager.listTools): TTL cache keyed like the connection pool — public scope under the bare server id, private/unknown/absent under the pool key (server id + token hash). Server TTLs clamped to 15 min; absent ttlMs gets a 60s default (deliberate spec deviation, ADR-0009; OMADIA_MCP_TOOLLIST_TTL_MS=0 restores spec-strict). Purged on close()/ closeAll() and immediately on notifications/tools/list_changed; lazy expiry on read. Discovery and the security rescan bypass via fresh: true. The bare-id probe only accepts entries filed as public (sharedPublic): a token-less caller's private list shares that key and must never cross auth contexts. Cached descriptors are deep-copied on write and read so caller mutation cannot poison the cache. Own servers now emit CacheableResult: loopback ttlMs 300000/public (frozen per-turn list), public server ttlMs 60000/private (per-API-key filtered list; tools/call keeps checking bindings live).
…light fetch (#545) The SDK dispatches notifications via Promise.resolve().then(), so a list_changed arriving in the same stdio chunk as the tools/list response could purge the cache BEFORE listTools primed it — the invalidation was lost and the stale-marked list stayed cached for its full TTL. Flaked in CI where pipe buffering coalesces the frames. Guard the cache write with a purge-generation counter: snapshot before the fetch, skip the write when any purge advanced it (purgeToolLists, closeAll). The stdio fixture now writes response + notification in one stdout.write, so the existing test exercises the coalesced delivery deterministically (red without the guard, green with it).
#700 split the public endpoint's `tools/list` handler in two, one per protocol era. The `ttlMs`/`cacheScope` this branch added landed on the 2025 leg alone — which is the one era where the fields are not vocabulary and no client can read them, while the 2026-07-28 leg (the only one that can) emitted none. Both legs now carry the same hints, with the same reasoning: `private` is mandatory because the list is filtered per API key, and the TTL is short because a revoked binding lingers in a cached list for that long. `McpListToolsResult` now declares `ttlMs`/`cacheScope` instead of the read site casting to reach them. The cast made "the server sent nothing" and "this type forgot the fields" indistinguishable, so a change that really did stop them arriving would have compiled in silence and the cache would have fallen back to its default TTL forever. The comment justifying that read is corrected too. Loose passthrough is why the fields survive on the v1 legs; on the v2 leg they are modelled 2026-07-28 fields that arrive verbatim. Measured, because there was reason to doubt it — the v2 `listTools` runs with `cacheMode: 'bypass'`, and bypass turns out to skip only the SDK's own response cache, not the hints. Not collapsed into "the SDK is lenient": that is exactly the reasoning which proved false for `resultType` on a legacy-era v2 decode (#562 phase 2). ADR-0009 gains the structural half of its own premise: for `stdio` and `sse` peers the ecosystem never catches up, because `server/discover` is answered at the HTTP edge only. The default TTL is the permanent path there, not a transitional one. Mutation check: dropping the hints from the modern leg turns the new guard red, and only that one. Refs #545
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Client-side TTL cache for
tools/listinMcpManagerplusttlMs/cacheScopeemission from the loopback and public MCP servers (MCP 2026-07-28
CacheableResult). Closes #545.Why
Fewer discovery round-trips and stable tool blocks for prompt caching. No SDK
bump needed: SDK 1.30.0 parses results leniently, the fields survive (same
pattern as
resultType, #544). Servers withoutttlMsget a 60s default —deliberate spec deviation, recorded in ADR-0009 (spec-strict "absent ⇒ 0" would
make the cache a no-op against today's ecosystem;
OMADIA_MCP_TOOLLIST_TTL_MS=0opts back in). Discovery and the security rescan bypass the cache
(
fresh: true);close()andnotifications/tools/list_changedpurge it.Test plan
test/mcpToolListCache.test.ts— 22 tests: TTL expiry, missing/negative/zerottlMs, 15-min clamp, unknown scope ⇒ private,freshbypass, purge onclose()/list_changed, token rotation ⇒ cache miss, no cross-auth-contextsharing, caller mutation cannot poison the cache
test/cliBridge/loopbackMcpServer.test.ts(
300000/public) andtest/publicMcp/publicMcpEndpoint.e2e.test.ts(
60000/private) — 76/76 across the three suitestsc --noEmitclean (middleware + harness-orchestrator)eslintclean on changed filesRisk / blast radius
OMADIA_MCP_TOOLLIST_TTL_MS— documented in.env.exampleandmiddleware-agent-handoff.md; no compose change needed (rides the existingenv_file). Package-local pattern (no zod schema): malformed values fall backto the default silently.
listTools()signature gains an optionaloptsparam; new pure-functionexports from
@omadia/orchestrator. Both additive, no breaking change.the list,
tools/callkeeps checking bindings live.@modelcontextprotocol/*@2— HTTP first (spike done) #562 phase 2): stdio/sse peers never sendttlMsunder v2era negotiation — they permanently ride the ADR-0009 default, which is the
designed behavior.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.