From a0510a8676041e71719115ba77f9e27be2b45323 Mon Sep 17 00:00:00 2001 From: plombeer31 Date: Sat, 5 Sep 2026 00:47:46 +0300 Subject: [PATCH] feat(composio): teach the model to reach for Composio Mounting Composio's meta-tools is not enough on its own. In the catalog they read as four opaque mcp.composio.COMPOSIO_* entries with nothing to say that "email this to Ivan" is now possible, so the model answers "I can't send email" while holding a tool that sends email. Adds a four-line `### integrations` section to the stable prefix that names the flow: search first, never guess an app tool's name, execute what you found, and if the account is not connected yet put the sign-in link in a `reply` so the user can click it. That last point is why no new rendering code is needed. Tool results are not linkified in chat, but replies are -- they already get OSC 8 links and clickable chips -- so routing the connect URL through `reply` reuses the surface that exists instead of teaching the tool renderer about links. Derived, not flagged: the section keys off `mcp.composio.COMPOSIO_SEARCH_TOOLS` being present in the descriptor list, so it cannot drift out of sync with what actually mounted, and it needs no new plumbing through BuildPromptInput. The text is ours rather than Composio's `experimental.assistive_prompt` (which their session response offers, ready to drop in). Piping a remote-controlled string into the system prompt would let a third party silently re-steer the agent, and any edit on their side would invalidate the KV-cached prefix for every user at once. The list of already-connected apps is deliberately left out. It would be useful, but it changes the moment the operator authorises something -- mid-session -- and the stable prefix is the one part of the prompt that must not move. Placement is between `### capabilities` and `### instructions`, so persona, rules, skills and the whole tools catalog stay byte-identical whether or not Composio is configured; with no key the section is absent entirely and the prefix matches pre-integration builds. No approval work was needed after all: Composio annotates SEARCH_TOOLS and GET_TOOL_SCHEMAS `readOnlyHint: true` and the two mutating tools destructive, which is exactly what mcp-tool-adapter already gates on. Discovery flows unprompted and every write to a real account still hits the approval gate, with no per-tool ACL override and no change to the documented MCP invariants. Verified on a real runtime: with a key the section renders after a 4-tool Composio catalog; with no key it is absent. --- AGENTS.md | 3 +- src/prompt/composio-guidance.test.ts | 97 ++++++++++++++++++++++++++++ src/prompt/composio-guidance.ts | 53 +++++++++++++++ src/prompt/stable-prefix.ts | 9 +++ 4 files changed, 161 insertions(+), 1 deletion(-) create mode 100644 src/prompt/composio-guidance.test.ts create mode 100644 src/prompt/composio-guidance.ts diff --git a/AGENTS.md b/AGENTS.md index eae9ce14..8ebc893c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1797,7 +1797,8 @@ Locked invariants (pinned by [src/composio/resolve-composio-server.test.ts](src/ 3. **Failure is soft.** Composio unreachable, rate-limiting, or rejecting a stale key logs a warning and boots without it. A third-party SaaS broker must never stand between the operator and their own shell, files and browser. 4. **`userId` is a minted UUID, persisted, and never an email.** Composio scopes connected accounts to it, so regenerating it silently orphans every app the operator has already authorised. An email would also hand PII to a third party for no benefit. 5. **The workbench stays disabled.** `createComposioSession` always posts `workbench: { enable: false }`, dropping `COMPOSIO_REMOTE_WORKBENCH` / `COMPOSIO_REMOTE_BASH_TOOL`. They duplicate `os.shell.run` and would quietly route the operator's shell work through a third-party sandbox. -6. **Trust stays `approval_gated`.** Discovery is still unprompted, because `mcp-tool-adapter.ts` exempts tools annotated `readOnlyHint === true` and Composio tags `COMPOSIO_SEARCH_TOOLS` / `COMPOSIO_GET_TOOL_SCHEMAS` exactly that way, while tagging `COMPOSIO_MULTI_EXECUTE_TOOL` / `COMPOSIO_MANAGE_CONNECTIONS` destructive. Every write to a real SaaS account therefore hits the approval gate, and the seamlessness costs nothing in consent. Loosening the server's trust to `pure_read` would un-gate the writes too — do not. +6. **The `### integrations` prefix section is derived, not flagged.** [src/prompt/composio-guidance.ts](src/prompt/composio-guidance.ts) keys off `mcp.composio.COMPOSIO_SEARCH_TOOLS` being in the descriptor list, so the guidance cannot drift out of sync with what actually mounted. It renders between `### capabilities` and `### instructions`, leaving persona / rules / skills / the tools catalog byte-identical whether or not Composio is configured, and is absent entirely with no key. The text is **ours**, not Composio's `experimental.assistive_prompt`: piping a remote-controlled string into the system prompt would let a third party re-steer the agent, and any edit on their side would invalidate the KV-cached prefix for every user at once. The list of connected apps is deliberately left out — it changes mid-session, and the stable prefix must not move. +7. **Trust stays `approval_gated`.** Discovery is still unprompted, because `mcp-tool-adapter.ts` exempts tools annotated `readOnlyHint === true` and Composio tags `COMPOSIO_SEARCH_TOOLS` / `COMPOSIO_GET_TOOL_SCHEMAS` exactly that way, while tagging `COMPOSIO_MULTI_EXECUTE_TOOL` / `COMPOSIO_MANAGE_CONNECTIONS` destructive. Every write to a real SaaS account therefore hits the approval gate, and the seamlessness costs nothing in consent. Loosening the server's trust to `pure_read` would un-gate the writes too — do not. ## Project path resolution (`os.fs.locate_project`) diff --git a/src/prompt/composio-guidance.test.ts b/src/prompt/composio-guidance.test.ts new file mode 100644 index 00000000..aecd133f --- /dev/null +++ b/src/prompt/composio-guidance.test.ts @@ -0,0 +1,97 @@ +import { describe, expect, it } from "vitest"; + +import { + COMPOSIO_GUIDANCE, + COMPOSIO_SEARCH_TOOL, + isComposioActive, +} from "./composio-guidance.js"; +import { buildStablePrefix, type ToolDescriptor } from "./stable-prefix.js"; +import type { CapabilitiesSummary } from "./capabilities.js"; + +function descriptor(name: string): ToolDescriptor { + return { name, summary: `${name} summary`, argsSchema: "{}" }; +} + +const CAPS: CapabilitiesSummary = { + platform: "linux", +} as unknown as CapabilitiesSummary; + +function prefixWith(descriptors: readonly ToolDescriptor[]): string { + return buildStablePrefix({ + toolDescriptors: descriptors, + capabilities: CAPS, + skillCatalog: [], + }); +} + +describe("isComposioActive", () => { + it("keys off the search tool actually being mounted", () => { + expect(isComposioActive([descriptor(COMPOSIO_SEARCH_TOOL)])).toBe(true); + expect(isComposioActive([descriptor("os.fs.read")])).toBe(false); + expect(isComposioActive([])).toBe(false); + }); + + it("is not fooled by an unrelated mcp server", () => { + expect(isComposioActive([descriptor("mcp.github.search")])).toBe(false); + }); +}); + +describe("the ### integrations prefix section", () => { + it("is absent when Composio is not mounted", () => { + // An install with no key must pay nothing for the integration -- + // not a token, not a byte of KV-cached prefix. + const prefix = prefixWith([descriptor("os.fs.read")]); + expect(prefix).not.toContain("### integrations"); + expect(prefix).not.toContain("Composio"); + }); + + it("appears once Composio is mounted", () => { + const prefix = prefixWith([ + descriptor("os.fs.read"), + descriptor(COMPOSIO_SEARCH_TOOL), + ]); + expect(prefix).toContain("### integrations"); + expect(prefix).toContain(COMPOSIO_GUIDANCE); + }); + + it("sits between capabilities and instructions", () => { + // Placement is load-bearing for the KV cache: everything above it + // -- persona, rules, skills, the whole tools catalog -- stays + // byte-identical whether or not Composio is configured. + const prefix = prefixWith([descriptor(COMPOSIO_SEARCH_TOOL)]); + expect(prefix.indexOf("### capabilities")).toBeLessThan( + prefix.indexOf("### integrations"), + ); + expect(prefix.indexOf("### integrations")).toBeLessThan( + prefix.indexOf("### instructions"), + ); + }); + + it("leaves everything above it byte-identical", () => { + const without = prefixWith([descriptor("os.fs.read")]); + const with_ = prefixWith([descriptor("os.fs.read")]); + expect(with_.slice(0, with_.indexOf("### capabilities"))).toBe( + without.slice(0, without.indexOf("### capabilities")), + ); + }); + + it("names the search tool first and the execute tool after", () => { + // The failure mode this guards is the model guessing an app tool + // name instead of discovering it, which Composio rejects. + const search = COMPOSIO_GUIDANCE.indexOf("COMPOSIO_SEARCH_TOOLS"); + const exec = COMPOSIO_GUIDANCE.indexOf("COMPOSIO_MULTI_EXECUTE_TOOL"); + expect(search).toBeGreaterThanOrEqual(0); + expect(exec).toBeGreaterThan(search); + }); + + it("tells the model to surface the connect link through reply", () => { + // Tool results are not linkified in chat; a `reply` is. Routing the + // URL through reply is what makes it clickable for the user. + expect(COMPOSIO_GUIDANCE).toContain("`reply`"); + expect(COMPOSIO_GUIDANCE).toContain("COMPOSIO_MANAGE_CONNECTIONS"); + }); + + it("stays short enough to live in every turn's prefix", () => { + expect(COMPOSIO_GUIDANCE.length).toBeLessThan(1200); + }); +}); diff --git a/src/prompt/composio-guidance.ts b/src/prompt/composio-guidance.ts new file mode 100644 index 00000000..2a3b2c15 --- /dev/null +++ b/src/prompt/composio-guidance.ts @@ -0,0 +1,53 @@ +/** + * The `### integrations` section of the stable prefix. + * + * Composio's meta-tools are useless if the model never reaches for + * them. Without this section the catalogue reads as four opaque + * `mcp.composio.COMPOSIO_*` entries with no hint that "email this to + * Ivan" is a thing they can do — so the model answers "I can't send + * email" while holding a tool that sends email. This block is the + * difference between the tools existing and the tools being used. + * + * The text is ours, not Composio's. Their session response ships an + * `experimental.assistive_prompt` that would drop in here verbatim, + * but wiring a remote-controlled string straight into the system + * prompt means a third party can silently re-steer the agent, and any + * edit on their side invalidates the KV-cached prefix for every user + * at once. A short local paragraph costs a few dozen tokens and keeps + * both properties. + * + * Deliberately omitted: the list of already-connected apps. It would + * be genuinely useful, but it changes the moment the operator + * authorises something — i.e. mid-session — and the stable prefix is + * the one part of the prompt that must not move. The model can ask + * Composio directly; the cache stays intact. + */ + +import type { ToolDescriptor } from "./stable-prefix.js"; + +/** Discovery tool whose presence means a Composio session is mounted. */ +export const COMPOSIO_SEARCH_TOOL = "mcp.composio.COMPOSIO_SEARCH_TOOLS"; + +/** + * Live iff the Composio search tool is in the catalog. + * + * Derived from the descriptors rather than passed in as a flag: the + * descriptors already reflect exactly what got mounted this boot, so + * the guidance cannot drift out of sync with the tools it describes. + */ +export function isComposioActive( + descriptors: readonly ToolDescriptor[], +): boolean { + return descriptors.some((d) => d.name === COMPOSIO_SEARCH_TOOL); +} + +/** + * The section body. Kept to four sentences: it sits in the KV-cached + * prefix of every single turn, so each line has to earn its tokens. + */ +export const COMPOSIO_GUIDANCE = [ + "External accounts — Gmail, Slack, Notion, Linear, Jira, GitHub, Discord and ~1500 more apps — are reachable through Composio, which also handles their sign-in.", + "When the user asks for something that lives in one of those apps, call `mcp.composio.COMPOSIO_SEARCH_TOOLS` with the use case (e.g. `{ queries: [{ use_case: \"send an email\" }] }`) before anything else — never guess an app tool's name or arguments.", + "Then run what it found via `mcp.composio.COMPOSIO_MULTI_EXECUTE_TOOL` (use `mcp.composio.COMPOSIO_GET_TOOL_SCHEMAS` first if you need the exact arguments).", + "If the account is not connected yet, `mcp.composio.COMPOSIO_MANAGE_CONNECTIONS` returns a sign-in link: put that URL in a `reply` so the user can click it, wait for them to confirm, then retry. Connections persist, so this happens once per app.", +].join("\n"); diff --git a/src/prompt/stable-prefix.ts b/src/prompt/stable-prefix.ts index 3f3bd02f..5a38292a 100644 --- a/src/prompt/stable-prefix.ts +++ b/src/prompt/stable-prefix.ts @@ -1,4 +1,8 @@ import type { ToolCallTransport } from "../llm/provider/completion-types.js"; +import { + COMPOSIO_GUIDANCE, + isComposioActive, +} from "./composio-guidance.js"; import { formatSkillCatalogLine } from "../skills/skill-catalog.js"; /** @@ -183,6 +187,10 @@ export const WINDOWS_PLATFORM_HINT = [ ].join("\n"); export function buildStablePrefix(input: StablePrefixInput): string { + // Present only while a Composio session is mounted, so an install + // with no key pays nothing for it and its prefix is byte-identical + // to before the integration existed. + const composioActive = isComposioActive(input.toolDescriptors); const nativeTools = input.toolTransport === "native_tools"; const persona = input.systemPersona ?? @@ -242,6 +250,7 @@ export function buildStablePrefix(input: StablePrefixInput): string { `### capabilities`, caps, ``, + ...(composioActive ? [`### integrations`, COMPOSIO_GUIDANCE, ``] : []), `### instructions`, // The emission instructions are the one transport-dependent block. // Grammar links parse text-JSON (GBNF-constrained locally), so they