Progressive tool discovery for the CPZAI MCP server

[chris-cpz]

Three changes, one theme: the tool catalogue should be the one this caller can actually use.

1. Progressive tool discovery

31 tools and ~27KB of JSON Schema landed in every client's context on every request. Clients that defer tool loading themselves (Claude Code, and the MCP connector's mcp_toolset.default_config.defer_loading) were already fine. Everything else paid in full, and tool-selection accuracy degrades past roughly thirty tools.

POST /mcp is unchanged. New POST /mcp/compact:

• search_tools: natural-language or keyword query, optional category, returns each match with its full input and output schema. Searches names, titles, descriptions, parameter names and parameter descriptions, so unsettled or entity_id finds the tool that takes it.
• call_tool: invokes a discovered tool and returns its result unchanged.
• Advertised up front: every state-changing tool, plus list_accounts, list_positions, list_orders, get_market_data.

call_tool takes read-only tools only. A client gates approval on the tool name it can see, so routing an order through a generic dispatcher would hide it from exactly the check meant to catch it. It also refuses an already-advertised read tool, answers an unknown name with near matches, and validates arguments against the real schema before dispatch.

2. Scope-aware tools/list

tools/list described the whole server to every credential. The spec permits the advertised set to vary by the authorization presented, since credentials are per-request input rather than connection state. Scopes come from GET /me (needs CPZ-Lab/cpzai#1635), cached per credential digest for 60s with a periodic sweep and a size bound, looked up only for tools/list, never on the call path, under a 3s budget of its own so a slow API costs an unfiltered list rather than a stalled client.

search_tools will not return an out-of-scope tool and call_tool will not dispatch one.

Unknown is not none. If the lookup fails, or the REST API has no scopes field yet, the full catalogue is advertised and the reason is logged. Verified against an unreachable API, so the two deploys are order independent. Enforcement stays in the REST API.

3. Declared output schemas

24 tools declare the envelope they actually return ({data, count}, {data}, {message}), so upstream drift fails loudly instead of returning a plausible-looking object. Only shapes this server controls are declared: the action tools that proxy elsewhere, plus place_order and create_connection, promise nothing. The SDK skips output validation on isError results, so error envelopes are unaffected.

Measured

Endpoint	Full scopes	data scope only
/mcp	31 tools, 31,290 B	10 tools, ~9,400 B
/mcp/compact	15 tools, 13,220 B	6 tools, ~5,500 B

Not two catalogues

captureTools runs the existing registerTools / registerExpandedTools against a recorder, then the permitted subset is replayed onto the server. One definition per tool; advertised, searchable and dispatchable surfaces cannot drift.

Also

SDK 1.27.1 to 1.30.0. Still protocol 2025-11-25: the 2026-07-28 implementation ships as separate @modelcontextprotocol/server@beta packages that are explicitly not production ready with APIs that may still change, so it stays off an order-placing surface until it stabilizes. Its headline change, a stateless core, is already how this server runs.

api-client gains an optional timeoutMs so a caller can set its own request budget.

Verification

• 156 tests pass, 21 new across tests/tool-search.test.ts and tests/scopes.test.ts. Includes: dispatch matches a direct call and applies the tool's own argument transforms; place_order through call_tool is refused and never reaches the API; a data-scoped key gets exactly the 10 expected tools; an identity-only credential gets get_profile alone; scope lookup caches per credential and returns null loudly on failure.
• Smoke-tested over HTTP on both paths: tools/list returns 31 and 15 tools, 24 and 7 with output schemas; unauthenticated POST still returns 401 with the WWW-Authenticate challenge; with an unreachable REST API both paths still serve the full catalogue and log the reason.
• tool-catalog.json is generated from full mode and is unchanged.

New cpzai://guides/discovery resource and README sections document both endpoints, the defer_loading config for the MCP connector, and the scope behaviour.

Deploy

Not git-triggered. S3 zip, CodeBuild aquila-mcp-build, then aws ecs update-service --cluster aquila-mcp --service aquila-mcp --force-new-deployment.