[mega-issue] MCP Analytics

[lucasheriques]

MCP analytics helps teams see how agents and users actually use their MCP servers. It captures sessions, tool calls, errors, and intent, so you can debug, measure real usage, and understand what people are trying to do.

It answers the questions a team building MCP tools actually has:

• which tools get called, and which are slow or erroring?
• what are people trying to do, at the level of "find a churn cohort", not "ran query_run 14 times"?
• where do conversations stall or fall through to feedback?

Not a goal: replacing LLM analytics/AI o11y. Generation traces, prompt/response inspection, and cost belong there.

Want this for your MCP server?

If you're building an MCP server and want analytics on it, drop a comment here and we'll reach out. We're looking for teams to pilot with and build alongside.

A few things stay internal (a Slack canvas, PostHog only): mainly the list of folks we're talking to for pilots.

Current status

SDKs

• TypeScript: @posthog/mcp on npm at 0.4.x (alpha). instrument(server, posthog) wraps a Server/McpServer; PostHogMCP covers custom dispatchers (hono, edge). Source: https://github.com/PostHog/posthog-js/tree/main/packages/mcp
• Python: merged into posthog-python (feat(mcp): PostHog MCP analytics SDK for Python (posthog.mcp) posthog-python#691). Plain pip install posthog, then from posthog.mcp import instrument — the mcp SDK is a peer dependency, so there's nothing extra to install. Wraps FastMCP and the low-level server with event parity with the TS SDK.

Install & onboarding

• npx -y @posthog/wizard@latest mcp-analytics instruments an existing TS/JS server automatically.
• In-app onboarding is event-gated: the product detects $mcp_initialize / $mcp_tool_call and flips to the dashboard when data arrives.

Product

• Dashboard (KPIs, activity, per-tool usage/error charts, client harness breakdown), sessions with LLM intent summaries, tool quality (per-tool error rate, latency percentiles, failures, description drift), and intent clustering with per-cluster journeys.
• The whole product is queryable over MCP too (session list, tool stats, intent clusters, missing-capability reports).
• Known gap, explicit goal: robust session tracking for stateless, serverless, and multi-replica deployments — where in-process session state doesn't survive between tool calls, sessions today can degrade to one call each. Server-side stitching plus $mcp_conversation_id is the direction.

Docs & landing page

• docs: https://posthog.com/docs/mcp-analytics
• landing page: live (https://posthog.com/mcp-analytics)

Roadmap note: near-term priorities move through sprint planning rather than living in this issue; see the comments for research and findings that feed it.

Links

• npm: https://www.npmjs.com/package/@posthog/mcp
• TS SDK source: https://github.com/PostHog/posthog-js/tree/main/packages/mcp
• Python SDK: feat(mcp): PostHog MCP analytics SDK for Python (posthog.mcp) posthog-python#691
• product code: https://github.com/PostHog/posthog/tree/master/products/mcp_analytics
• docs: https://posthog.com/docs/mcp-analytics
• landing page: https://posthog.com/mcp-analytics

[pauldambra]

• MCP analytics via MCP feat(mcp-analytics): MCP analytics skills + flag-gate feedback endpoints #63886

questions feedback from internal thread https://posthog.slack.com/archives/C09SK2PAGKF/p1781511992093959

• maybe some stab at how many "humans"
• any notion of a session in this world?
• might want some session level golden metrics of some sort if possible over time ?
• some way to break out exec ?
  • lots of people going to do this trick so i wonder if some way to see params as tools <-- suuper messy of course
• individual tool call chart overtime so can easily see changes as we work on different tools etc
• or maybe this is a click into a specific tool to give me all about that tool
• maybe some read vs write slices or dims too perhaps?
  • eg if i see a spike in sensitive sort of write actions etc i might be worried
• how does it link to self-driving / scouts / etc
• user paths or journeys but for tool calls
• detect cycles - e.g. struggle / frustration
• Avg input/output tokens avg and p90, p95 – helps to track outliers needed to be optimized.
• Three dimensions that we track for tools: input/output tokens, latency, and error rate.
• Deeper analysis of clients – clients tend to change their MCP implementation, so ideally, I’d like to have visibility into the MCP behavior in specific clients.
• Tool exclusion lists (exactly what Andy mentioned above) – to filter out the exec tool (might be related to the instrumentation).
• filter_test_accounts
• and filters (more specifically, feature flag filters). For instance, I’d like to see aggregated sessions for the render-ui feature flag.
• “Calls (broken down by error)” here is a bit hard to understand. Maybe a different chart with green/red indicators, so it’s easier to perceive.
• I think validation exceptions are not captured (MCPs usually validate on the client, then on the server) – could help to find broken schemas.
• Adding flexible instrumentation options to the MCP analytics package. It only supports the MCP SDK, which we don’t use for the MCP.

[sampennington]

individual tool call chart overtime so can easily see changes as we work on different tools etc
or maybe this is a click into a specific tool to give me all about that tool

I think these are done?

This page

More we could add though! Token spend sounds super useful at least

[lucasheriques]

Python SDK: first cut up for review 🐍

The #1 "what to prioritize next" item (Python SDK) now has a draft PR:

• SDK: feat(mcp): PostHog MCP analytics SDK for Python (posthog.mcp) posthog-python#691 — posthog.mcp, install via pip install posthog[mcp]. instrument(server, posthog) wraps a FastMCP or low-level mcp.server.Server; PostHogMCP covers custom dispatchers. Captures the identical $mcp_* events (intent, identify, exceptions, get_more_tools, conversation_id), so the existing dashboard ingests Python-server data with no backend changes.
• Docs: docs(mcp-analytics): add Python SDK install section posthog.com#17827 — adds a Python section to the installation page.

Dogfooded against project 2; the full $mcp_* event set ingests with correct intent/error/source. Alpha; resources/prompts events intentionally omitted to match the TS instrument() surface.

Note to future Lucas: update here once this is merged