feat(ai-observability): rebuild the skill around the SDK wrapper and the session tree

context/commandments.yaml

@@ -316,3 +316,11 @@ commandments:
     - posthog-kmp is a thin wrapper that delegates to the native posthog-android, posthog-ios, and posthog-js SDKs, so session replay and autocapture availability follow the underlying native platform
     - Use PostHogConfig.HOST_US or PostHogConfig.HOST_EU for the host, and read the project token from generated build config or environment values rather than hardcoding it
     - The SDK is 0.x pre-release; check the latest posthog-kmp version on Maven Central (https://central.sonatype.com/artifact/com.posthog/posthog-kmp) before pinning instead of hardcoding a stale version
+
+  # Carve-outs for the ai-observability skill; every variant carries this tag.
+  # Worded to override the generic language rules regardless of list order.
+  ai_observability:
+    - "AI Observability carve-out: this skill instruments LLM calls and is not product-analytics coverage. Do NOT add posthog.capture() events for user actions, captureException() error handlers, or a reverse proxy unless the user explicitly asks for them"
+    - "AI Observability carve-out: only the wrapper-client and manual-capture install paths construct a PostHog client. The OTel and framework-hook paths have no client at all, so any rule in this file about the Posthog()/PostHog() constructor, exception autocapture, atexit/shutdown registration or flushing simply does not apply — never invent a client just to satisfy one"
+    - "AI Observability carve-out: the $ai_* payload properties ($ai_input, $ai_output_choices, and the rest) intentionally carry user-generated prompt and completion text, so this file's PII rules do NOT apply to them. Those rules still govern every other property you set"
+    - "AI Observability carve-out: read the PostHog key and host exactly as the variant's install doc reads them. A direct os.environ[\"POSTHOG_API_KEY\"] / process.env lookup already fails loudly and idiomatically when unset, which satisfies this file's missing-configuration rule — do NOT add a separate presence check, guard branch, or custom raise around a bootstrap that is only a few lines long"

context/skills/ai-observability/config.yaml

@@ -12,6 +12,8 @@ shared_docs:
   - https://posthog.com/docs/ai-observability/basics.md
   - https://posthog.com/docs/ai-observability/generations.md
   - https://posthog.com/docs/ai-observability/traces.md
+  - https://posthog.com/docs/ai-observability/sessions.md
+  - https://posthog.com/docs/ai-observability/spans.md
 variants:
   - id: openai-python
     framework: openai

context/skills/ai-observability/description.md

@@ -1,23 +1,23 @@
 # PostHog AI Observability for {display_name}
 
-Wire up PostHog's AI Observability so calls made through {display_name} emit `$ai_generation` events into LLM Analytics.
+Wire up PostHog's AI Observability so calls made through {display_name} land in LLM Analytics as a full **session → trace → span → generation** tree — not just isolated `$ai_generation` events.
 
 ## Prerequisite — vendor LLM SDK
 
 This skill instruments the LLM calls the project *already makes*. It does **not** install the vendor SDK for you.
 
-Check the project's manifest for the provider's package (e.g. `openai`, `@anthropic-ai/sdk`, `langchain`, `ai`, `@google/genai`). If a vendor SDK is present, pick the matching variant. If none is present, switch to the `manual-capture` variant — it posts `$ai_generation` events directly and works standalone.
+Check the project's manifest for an LLM package. The catalog is far wider than the obvious providers — 68 variants covering agent frameworks (`openai-agents`, `claude-agent-sdk`, LangGraph, CrewAI, Mastra, …) and OpenAI-compatible gateways (Groq, OpenRouter, Together, Ollama, …), which an app reaches through the `openai` package plus a `baseURL` override. `1-begin.md` carries the ordered decision rules; follow them rather than matching on the first familiar package name. If no LLM SDK is present, switch to the `manual-capture` variant — it posts `$ai_generation` events directly and works standalone.
 
-Everything else this skill needs — PostHog credentials, the OTel packages, env vars — the skill installs and configures itself. It does **not** require a pre-existing `posthog.init(...)`. If one is already there, reuse its env-var names in `3-otel-setup.md`; if not, that step sets fresh values via `set_env_values`.
+Everything else this skill needs — PostHog credentials, instrumentation packages, env vars — the skill installs and configures itself. It does **not** require a pre-existing `posthog.init(...)`. If one is already there, reuse its env-var names in `3-instrument.md`; if not, that step sets fresh values via `set_env_values`.
 
 ## Steps
 
 Read every referenced file **before editing**. Then work through them in order:
 
-1. **Begin** — see `references/1-begin.md`. Pick the right variant from the vendor SDK the project declares, then locate the LLM call sites so you know where the instrumentation will apply.
-2. **Install** — see `references/2-install.md`. Add the OpenTelemetry SDK, PostHog's span-processor package, and the provider-specific instrumentation package to the manifest.
-3. **Set up OpenTelemetry** — see `references/3-otel-setup.md`. Initialize the OTel TracerProvider once with `PostHogSpanProcessor`, attach the provider-specific instrumentor, and route the project token / host through environment variables.
-4. **Verify** — see `references/4-verify.md`. Describe a single call the user can trigger and how to confirm `$ai_generation` events land in PostHog.
+1. **Begin** — see `references/1-begin.md`. Pick the variant with the ordered rules (framework before provider, gateway base URL before the SDK it borrows), then read four facts from the code: the conversation, the user, the turn, and whether the app registers tools.
+2. **Install** — see `references/2-install.md`. Declare the variant's packages in the manifest — and only those. For providers and gateways that's the PostHog SDK alongside the vendor SDK, with no OpenTelemetry packages.
+3. **Instrument** — see `references/3-instrument.md`. Swap the vendor client for PostHog's wrapper, attach `$ai_session_id`, a per-turn `posthog_trace_id`, and the distinct id to every call, and capture tool runs as `$ai_span` events. This step is what turns isolated generations into a session tree.
+4. **Verify** — see `references/4-verify.md`. Describe a request the user can trigger, and grade what lands in PostHog — one session, grouped traces, right attribution — rather than what the diff contains.
 
 ## Reference files
 
@@ -28,9 +28,12 @@ The linked install page carries the exact code blocks for this variant's languag
 ## Key principles
 
 - **Environment variables.** Read `<ph_project_token>` and `<ph_client_api_host>` from env, using the framework's env-var convention. Never hardcode either value.
-- **Minimal changes.** OTel initialization is a single call that runs once at process start. Place it alongside any existing PostHog init rather than restructuring the entry point.
-- **Match the docs.** Package names, instrumentor imports, and processor names change between AIO releases. The install page for this variant is the source of truth.
-- **Don't touch what isn't yours.** This skill instruments generations only. Identify calls, event tracking, error tracking, and dashboards belong to the base `integration` skill — do not add or edit them here.
+- **The SDK wrapper is the default, not OpenTelemetry.** OTel makes the session tree awkward to build and maintain, so provider and gateway variants use PostHog's drop-in wrapper client. Reserve OTel for the `opentelemetry-*` variants and LlamaIndex, and never swap a framework's own tracing hook for an instrumentor.
+- **Minimal changes.** The wrapper swaps a client constructor and adds parameters to existing calls. Don't restructure the app, and don't wrap the setup in an init function or module globals.
+- **Match the docs.** Package names and wrapper imports change between AIO releases. The install page for this variant is the source of truth.
+- **Cardinality is what gets graded.** One `$ai_session_id` per conversation, one `posthog_trace_id` per turn, shared by every call in it. An id minted per call is worse than none — it looks instrumented and groups nothing.
+- **Tools become spans.** When the app registers tools, capture each execution as an `$ai_span` event sharing the turn's trace id — the wrapper never sees your dispatch loop. Framework variants emit these themselves; an app with no tools correctly has none.
+- **Don't touch what isn't yours.** This skill instruments LLM observability only — generations, traces, sessions, spans. Identify calls, event tracking, error tracking, and dashboards belong to the base `integration` skill — do not add or edit them here.
 
 ## Emit a run record
 
@@ -40,6 +43,8 @@ When you finish, write `.posthog-wizard-cache/.posthog-ai.json` at the project r
 { "provider": "openai", "package": "@posthog/ai", "otel_init_file": "src/instrumentation.ts" }
 ```
 
+`otel_init_file` keeps its name for the report's sake, but on the wrapper path there is no OTel init — set it to the file where the wrapper client was constructed (or, on the manual path, where the capture helper lives).
+
 The `report/` step reads this file to render an AI Observability section in the setup report. If the cache directory does not exist, create it.
 
 ## Framework guidelines

context/skills/ai-observability/references/1-begin.md

@@ -1,52 +1,70 @@
 ---
 next_step: 2-install.md
 title: AI Observability Setup - Begin
-description: Pick the right variant, confirm prerequisites, and locate the LLM call sites before editing
+description: Pick the variant that matches this project, then read the four facts the instrumentation needs
 ---
 
-Before touching any code, decide which variant of this skill to install, confirm the two prerequisites, and get a read on where in the project LLM calls actually happen. AI Observability instruments an existing setup — if the setup isn't there, this skill can't do its job.
+Pick the variant, then read the code. Do not edit anything in this step.
 
 ## Pick the variant
 
-The `ai-observability` skill has one variant per LLM provider × language (e.g. `ai-observability-openai-python`, `ai-observability-anthropic-node`, …). You are running the group-level entry; before installing, pick the specific variant that matches this project.
+This skill ships 68 variants. Call `load_skill_menu` with `category: "ai-observability"`. That list is the source of truth.
 
-Scan the manifest (`package.json`, `pyproject.toml`, `requirements.txt`, `Gemfile`) for a vendor LLM package. Typical package names:
+Apply these rules in order. The first match wins. Frameworks wrap providers, and gateways look like OpenAI, so the order matters.
 
-- OpenAI — `openai`
-- Anthropic — `@anthropic-ai/sdk` (Node) or `anthropic` (Python)
-- LangChain — `langchain` / `@langchain/core` (plus a provider adapter like `langchain-openai` / `@langchain/openai`)
-- Vercel AI SDK — `ai` (plus a provider like `@ai-sdk/openai`)
-- Google Gemini — `google-genai` (Python) or `@google/genai` (Node)
+### 1. A framework wins over the provider under it
 
-Decision rules — apply in order:
+| Package in the manifest | Variant |
+|---|---|
+| `openai-agents` | `openai-agents` |
+| `claude-agent-sdk` | `claude-agent-sdk` |
+| `langchain`, `@langchain/core` | `langchain-{python,node}` |
+| `langgraph`, `@langchain/langgraph` | `langgraph-{python,node}` |
+| `ai` (Vercel AI SDK) | `vercel-ai` |
+| `llama-index`, `llamaindex` | `llamaindex` |
+| `crewai` | `crewai` |
+| `pyautogen`, `autogen-agentchat` | `autogen` |
+| `dspy`, `dspy-ai` | `dspy` |
+| `pydantic-ai` | `pydantic-ai` |
+| `semantic-kernel` | `semantic-kernel` |
+| `smolagents` | `smolagents` |
+| `mirascope` | `mirascope` |
+| `instructor` | `instructor-{python,node}` |
+| `litellm` | `litellm` |
+| `mastra`, `@mastra/core` | `mastra` |
+| `convex` | `convex` |
 
-1. **Exactly one vendor SDK found** → pick the corresponding variant. Language follows the manifest (`package.json` → Node, `pyproject.toml`/`requirements.txt` → Python). Tell the user which variant you picked and why in a `[STATUS]` line, then call `install_skill` with the full variant id (e.g. `ai-observability-openai-python`).
-2. **Multiple vendor SDKs found** (e.g. LangChain wraps OpenAI, so both may be declared) → prefer the higher-level abstraction: LangChain > direct provider SDK. If still ambiguous, use `wizard_ask` to have the user pick, listing the candidates as options.
-3. **No vendor SDK found** → install `ai-observability-manual-capture`. This variant posts `$ai_generation` events explicitly and works without any auto-instrumentation.
-4. **You're not sure** → `wizard_ask` the user with a multi-choice picker listing every provider you have a variant for. Do not guess when there's real ambiguity.
+Instrument the framework, not the provider below it. A provider instrumentor keeps the model calls and loses the agent, tool, and handoff structure.
 
-## Check for an existing PostHog setup (informational — not a blocker)
+### 2. An `openai` client with a base URL override means a gateway
 
-Grep the project for one of:
+Most OpenAI-compatible providers ship no SDK. Apps call them with the `openai` package aimed at another host. Check the client constructor and `OPENAI_BASE_URL`. Common hosts are `api.groq.com`, `openrouter.ai`, `api.together.xyz`, and `localhost:11434`.
 
-- `posthog.init(` — most JS/TS SDKs
-- `PostHog(` — Python, Ruby, Go SDK constructors
-- `AddPostHog(` — .NET DI registration
+Pick the variant that names the provider. The install shape matches plain OpenAI, but the provider name does not. Step 3 explains why that matters.
 
-**This is not a prerequisite.** The OTel-based variants use `PostHogSpanProcessor`, a self-contained exporter that just takes an API key + host — it does not depend on a `posthog.init(...)` call anywhere. The `manual-capture` variant uses `posthog.capture(...)`, which needs the traditional SDK, but the install step will add it if it isn't there.
+### 3. A plain provider SDK maps to that provider
 
-If a `posthog.init(...)` (or equivalent) **is** already present, note the env-var names it reads (`POSTHOG_API_KEY`, `NEXT_PUBLIC_POSTHOG_KEY`, etc.) and reuse them in `3-otel-setup.md` — don't invent parallel names. If nothing is there, `3-otel-setup.md` will set fresh values via `set_env_values`.
+`openai`, `anthropic`, `@anthropic-ai/sdk`, `google-genai`, `@google/genai`, `mistralai`, and `cohere` each have a variant. `boto3` with `bedrock-runtime` maps to `aws-bedrock`.
 
-## Locate the LLM call sites
+### 4. Anything else
 
-Grep for where the vendor SDK is imported and called. This is not a full analysis — one or two representative sites is enough for you to reason about where OTel initialization has to run before those calls execute:
+- Several candidates and no framework: prefer the higher-level one. If it stays unclear, use `wizard_ask` with the candidates as options.
+- The app already emits its own OTel spans: `opentelemetry-{python,node}`.
+- No LLM SDK at all: `manual-capture`.
 
-- OpenAI: `import OpenAI`, `openai.OpenAI(`, `new OpenAI(`
-- Anthropic: `Anthropic(`, `new Anthropic(`
-- LangChain: `ChatOpenAI(`, `from langchain`, `import { ChatOpenAI } from '@langchain/openai'`
-- Vercel AI: `generateText(`, `streamText(`, `import ... from 'ai'`
-- Google Gemini: `genai.Client(`, `new GoogleGenerativeAI(`
+Language follows the manifest. A `package.json` means Node. A `pyproject.toml` or `requirements.txt` means Python. Framework variants have no language suffix.
 
-Note the app's entry point (server startup file, `main.py`, `index.ts`, `instrumentation.ts` in Next.js, etc.) — OTel must be initialized *before* the vendor SDK is imported, and the entry point is where that happens.
+Report the variant and the reason in a `[STATUS]` line, then call `install_skill` with the full id.
 
-Do not edit yet. Once you have a note of the entry point and the call sites, move on to `2-install.md`.
+## Read four facts from the code
+
+The install doc holds the code. It cannot know this app. Step 3 uses these answers and nothing else.
+
+1. **Conversation.** The field that groups turns, such as `thread_id` or `conversation_id`. If the app has none, the process run is the conversation.
+2. **User.** The user id in scope at the call sites. If the app has none, the events stay anonymous. Do not invent one.
+3. **Turn.** The function that takes one question and returns one answer. It may call the model several times.
+4. **Tools.** Does the app register tools with its calls? Look for a `tools=` argument or a tool decorator. Find the loop that runs them.
+
+Note the module that builds the vendor client. Step 3 replaces that constructor. If the project already calls `posthog.init(...)` or `PostHog(...)`, reuse its env-var names and its client.
+
+Go to `2-install.md`.