PostHog · sarahxsanders · Sep 9, 2026
@@ -0,0 +1,47 @@
+---
+type: ai-observability
+flow: integration-v2
+label: Add AI Observability
+model_pi: openai/gpt-5.6-terra
+effort_pi: medium
+model_sdk: claude-sonnet-5
+effort_sdk: high
+skills: []
+allowedTools: [Read, Write, Edit, Glob, Grep, load_skill_menu, install_skill]
+disallowedTools: [enqueue_task]
+dependsOn: [capture, error-tracking]
+---
+
+## Goal
+
+Add AI Observability to the LLM calls this project already makes. Read the
+upstream handoffs and inspect the call sites, not just the dependency names.
+If there are no LLM calls, complete as `not needed` with the reason and make no
+changes. Do not install a vendor SDK, add a demo call, or use manual capture to
+invent an AI feature in an app that has none.
+
+For existing LLM calls, load the `ai-observability` skill menu and install the
+variant matching the calling code's language and provider or agent framework.
+Follow that skill's selection rules and instrumentation references. Manual
+capture is appropriate only for existing calls without a supported wrapper.
+Reuse the PostHog client and identity established earlier in this run. If a
+variant cannot be chosen from the code, report the ambiguity as `not needed`;
+this default run does not ask the user to choose a provider.
+
+This is the instrumentation part of a larger integration. Declare dependencies
+in the manifest; the following review task installs them and checks imports,
+builds, and lint. Defer the skill's package-manager and import-check commands
+to that task. For Go, leave the required modules in the handoff for review to
+add with `go get` and `go mod tidy`; do not hand-edit `go.mod` or `go.sum`.
+Do not run the app or make paid LLM calls. Use `check_env_keys` and
+`set_env_values` for environment files, using the project credentials supplied
+by the wizard.
+
+## How you know you succeeded
+
+Existing LLM calls are wired using the selected skill, or the handoff clearly
+explains why this task was not needed. List the variant, changed files, declared
+dependencies, and a concrete call path the user can trigger to verify the
+session/trace/generation tree. Pass any deferred dependency work to review.
+Describe delivery as unverified unless this run actually observed it; do not
+publish a separate setup report.
@@ -23,11 +23,19 @@ Plan a PostHog integration and seed the task queue with this graph:
   the SDK installed and initialized, not the events.
 - `capture`, after `identify` — it decides the events and instruments them, and it
   reads how identity is already established before it instruments anything.
-- `review`, after `install`, `init`, `identify`, `capture`, and `error-tracking` —
+- `ai-observability`, after `capture` and `error-tracking` — it instruments existing
+  LLM calls, or reports that none apply. Queue it on every default run; the task
+  checks applicability rather than the planner guessing from a package name.
+- `logs`, after `ai-observability` — it configures log capture on supported
+  platforms, or reports why it does not apply. Queue it even when AIO will be
+  skipped. These tasks run sequentially because they can edit the same dependency
+  manifests and initialization files as the earlier instrumentation tasks.
+- `review`, after `install`, `init`, `identify`, `capture`, `error-tracking`,
+  `ai-observability`, and `logs` —
   it installs the dependencies, verifies the project builds/typechecks/lints, and
   reviews every change the run made, fixing what fails. There is no separate build
   step: verifying and reviewing are one pass over the same changeset.
-- `dashboard`, after `capture`, parallel to `review` — it builds insights from the
+- `dashboard`, after `capture`, independent of AIO, Logs, and `review` — it builds insights from the
   instrumented events, which `capture` has already defined; it needs no code review.
 - `report`, after `dashboard` **and** `review` — it writes the setup report last, so
   it describes the integration as reviewed rather than as first written.

@@ -0,0 +1,47 @@
+---
+type: logs
+flow: integration-v2
+label: Add log capture
+model_pi: openai/gpt-5.6-terra
+effort_pi: medium
+model_sdk: claude-sonnet-5
+effort_sdk: high
+skills: []
+allowedTools: [Read, Write, Edit, Glob, Grep, load_skill_menu, install_skill]
+disallowedTools: [enqueue_task]
+dependsOn: [ai-observability]
+---
+
+## Goal
+
+Configure PostHog log capture for this project's supported runtimes. Load the
+`logs` skill menu, install its matching skill, and read the platform reference
+before editing. Inspect the runtime and existing logging setup; use the
+documented OTLP exporter or SDK-native log capture for that platform. A
+browser-only app must not receive a server exporter. If no documented setup
+applies, complete as `not needed` and explain why.
+
+Reuse any existing PostHog log export and leave its handlers and outputs
+intact. If capture is already configured, report that without adding a second
+exporter. Preserve any tracing provider AIO configured in the previous step.
+Keep changes to log setup and existing logging paths; do not scatter new logs
+through unrelated code or export secrets, request bodies, or user data.
+
+This is the instrumentation part of a larger integration. Instead of the
+standalone skill's install command, declare required packages in the manifest
+and leave installation, lockfile updates, builds, and lint to review. For
+manifests managed by tooling (such as Go), pass the required packages to review
+instead of editing them by hand. Use `detect_package_manager` if needed. Use
+the wizard's supplied project and region and the env names from upstream
+handoffs; inspect and write environment files only through `check_env_keys`
+and `set_env_values`. Do not request credentials or select a different project.
+If required configuration is unavailable, report what is missing instead of
+guessing a region.
+
+## How you know you succeeded
+
+Log capture is configured once for the supported runtime, was already present,
+or has a clear skip reason. The handoff names the changed files, packages, env
+variable names (never values), and any dependency work for review. Give a
+specific existing log path the user can trigger and find in PostHog Logs;
+delivery remains unverified until observed. Do not publish a separate report.
@@ -45,7 +45,8 @@ before those events mean anything.
 
 One `publish_handoff` call went through with the full report: what was
 installed and initialized, the events captured, whether identify was wired or
-skipped, error tracking added, the dashboard link, any build conflict in full,
+skipped, error tracking added, AI Observability and Logs configured or skipped
+with reasons, the dashboard link, any build conflict in full,
 and the next steps for the user. Every claim in it traces to a handoff, and
 what the run could not confirm reads as unconfirmed. The same report is
 mirrored into a PostHog notebook whose URL is emitted with the

@@ -9,7 +9,7 @@ effort_sdk: high
 skills: [posthog-best-practices, integration-v2-build]
 allowedTools: [Read, Edit, Glob, Grep, Bash]
 disallowedTools: [enqueue_task]
-dependsOn: [install, init, identify, error-tracking, capture]
+dependsOn: [install, init, identify, error-tracking, capture, ai-observability, logs]
 ---
 
 ## Goal

@@ -26,6 +26,9 @@ Draw on two sources only:
   (from `.posthog-wizard-cache/.posthog-events.json`).
 - Whether user identification was wired or skipped, and why.
 - The error tracking added.
+- AI Observability and Logs: what was configured or already present, or why each
+  was skipped. Use the task handoffs, including changed files and outstanding
+  verification; do not treat a skipped task as a successful setup.
 - The dashboard link.
 - Any build conflict, in full.
 - Clear next steps for the user.
@@ -46,6 +49,8 @@ code changed this run and drop the ones that don't fit:
   for CSP violations — a blocked SDK queues events silently and never sends.
 - If LLM analytics was set up: trigger the instrumented call path and confirm
   `$ai_generation` events appear in PostHog.
+- If Logs was set up: trigger the log path named in its handoff and confirm the
+  entry appears in PostHog Logs with the expected service and severity.
 - If auth exists and identify was wired: the returning-visitor path also calls
   identify, so returning sessions don't fragment onto anonymous distinct IDs.
 

@@ -20,6 +20,37 @@ It's essential to do this in both client code and server code, so that user beha
 
 You should also add PostHog exception capture error tracking to these files where relevant.
 
+## AI Observability and Logs
+
+Include both in a default integration where applicable, after the SDK and event
+instrumentation above and before the revise and conclusion steps. Run AIO first,
+then Logs, so edits to manifests and initialization files do not conflict.
+
+1. **AI Observability:** inspect existing LLM call sites. If there are none, skip
+   AIO and record that reason; do not add a vendor SDK or artificial model calls.
+   Otherwise call `load_skill_menu` with `category: "ai-observability"`, install
+   the matching variant with `install_skill`, and follow its references. Choose
+   the provider/framework and language from the actual calling code. Use manual
+   capture only for existing calls without a supported wrapper. Reuse the client
+   and identity already established. If the provider remains ambiguous, skip with
+   that reason instead of asking a question during the default run.
+2. **Logs:** call `load_skill_menu` with `category: "logs"`, install the matching
+   skill, and follow the documented setup for this runtime. Use SDK-native log
+   capture where documented, otherwise the platform's OTLP exporter. Do not add a
+   server exporter to browser-only code. If no documented setup applies, skip with
+   the reason. Preserve existing handlers, outputs, and any AIO tracing provider;
+   an existing PostHog exporter needs no duplicate. Keep edits to log setup and
+   existing logging paths, without adding logs to unrelated code.
+
+For both skills, use the project credentials and region supplied by the wizard.
+Inspect and change environment files only through `check_env_keys` and
+`set_env_values`, reusing existing variable names. Do not ask for credentials or
+guess a region. Follow this runtime's tool restrictions; defer any dependency
+installation or verification still needed to the revise step. Do not make paid
+LLM calls or claim delivery based on code changes. Keep each result (configured,
+already present, or skipped with a reason) for the final report, including a
+concrete path the user can trigger to check delivery.
+
 Remember: Do not alter the fundamental architecture of existing files. Make your additions minimal and targeted.
 
 Remember the documentation and example project resources you were provided at the beginning. Read them now.

@@ -8,6 +8,11 @@ Check the project for errors. Read the package.json file for any type checking o
 
 Ensure that any components created were actually used.
 
+Install any dependencies declared by the AI Observability and Logs steps before
+verification, using the project's package manager. Include their changed files
+in this review and check that Logs preserved any AIO tracing setup. Verification
+of delivery belongs to the user's checklist; do not make paid LLM calls.
+
 Once all other tasks are complete, run any linter or prettier-like scripts found in the package.json, but ONLY on the files you have edited or created during this session. Do not run formatting or linting across the entire project's codebase.
 
 ## Status

@@ -78,6 +78,11 @@ Do not spawn subagents.
 
 Compose the setup report as markdown — do NOT write it to a file in the project. It should include a summary of the integration edits, a table with the event names, event descriptions, and files where events were added, a list of links for the dashboard and insights created, and a "Verify before merging" checklist (see below). Follow this format:
 
+Include separate AI Observability and Logs results: what was configured, what was
+already present, or why each was skipped. Name the changed files and distinguish
+code verification from observed delivery. A skipped product is not a successful
+setup.
+
 <wizard-report>
 # PostHog post-wizard report
 
@@ -108,6 +113,8 @@ For the "Verify before merging" checklist, write GitHub-style checkboxes (`- [ ]
 - If you added environment variables: "Add the exact PostHog env var names you added to `.env.example` and any monorepo/bootstrap scripts so collaborators know what to set."
 - If this integration ships a minified production browser bundle (most SPA/SSR web frameworks — e.g. Next.js, Nuxt, SvelteKit, Astro, Vite-based apps): "Wire source-map upload (`posthog-cli sourcemap` or your bundler's upload step) into CI so production stack traces de-minify."
 - If LLM analytics was set up in this run: "Trigger the LLM call path(s) you instrumented and confirm `$ai_generation` events appear in PostHog AI Observability."
+- If Logs was set up in this run: trigger the existing log path you configured and
+  confirm the entry appears in PostHog Logs with the expected service and severity.
 - If the app has user auth and an `identify` call was added: "Confirm the returning-visitor path also calls `identify` — a handler that only identifies on fresh login can leave returning sessions on anonymous distinct IDs."
 
 Do not invent items beyond what applies. If only the two "Always" items apply, the checklist is just those two.

@@ -0,0 +1,65 @@
+import { afterEach, describe, expect, it } from 'vitest';
+import { mkdtempSync, readFileSync, readdirSync, rmSync } from 'fs';
+import { tmpdir } from 'os';
+import { join } from 'path';
+import matter from 'gray-matter';
+
+import { buildAgents } from '../agent-generator.js';
+import { expandSkillGroups, loadSkillsConfig } from '../skill-generator.js';
+
+const configDir = join(process.cwd(), 'context');
+const flowDir = join(configDir, 'agents', 'integration-v2');
+const prompts = readdirSync(flowDir)
+    .filter((file) => file.endsWith('.md'))
+    .map((file) => matter(readFileSync(join(flowDir, file), 'utf8')).data);
+const byType = new Map(prompts.map((prompt) => [prompt.type, prompt]));
+
+function ancestors(type, visiting = new Set()) {
+    if (visiting.has(type)) throw new Error(`Dependency cycle at ${type}`);
+    const prompt = byType.get(type);
+    if (!prompt) throw new Error(`Unknown dependency ${type}`);
+    const next = new Set([...visiting, type]);
+    return new Set((prompt.dependsOn ?? []).flatMap((dep) => [dep, ...ancestors(dep, next)]));
+}
+
+describe('default integration observability flow', () => {
+    let distDir;
+    afterEach(() => {
+        if (distDir) rmSync(distDir, { recursive: true, force: true });
+    });
+
+    it('publishes both tasks with tools that can discover their existing product skills', () => {
+        distDir = mkdtempSync(join(tmpdir(), 'integration-observability-'));
+        buildAgents({ configDir, distDir, baseUrl: 'https://example.test' });
+        const menu = JSON.parse(readFileSync(join(distDir, 'agents', 'agent-menu.json'), 'utf8'));
+        const skills = expandSkillGroups(loadSkillsConfig(configDir), configDir);
+
+        for (const type of ['ai-observability', 'logs']) {
+            const entry = menu.agents.find((agent) => agent.flow === 'integration-v2' && agent.id === type);
+            expect(entry).toBeDefined();
+            const asset = entry.downloadUrl.split('/').pop();
+            const { data } = matter(readFileSync(join(distDir, 'agents', asset), 'utf8'));
+            expect(data.allowedTools).toEqual(expect.arrayContaining(['load_skill_menu', 'install_skill']));
+            expect(data.allowedTools).not.toContain('wizard_ask');
+            expect(data.allowedTools).not.toContain('Bash');
+            expect(data.seed).not.toBe(true);
+            expect(data.runnerSeeded).not.toBe(true);
+            expect(skills.some((skill) => skill._category === type)).toBe(true);
+        }
+    });
+
+    it('serializes the new writers after existing instrumentation and before review', () => {
+        for (const type of ['install', 'init', 'identify', 'capture', 'error-tracking']) {
+            expect(ancestors('ai-observability').has(type)).toBe(true);
+        }
+        expect(ancestors('logs').has('ai-observability')).toBe(true);
+        const reviewed = ancestors('review');
+        for (const prompt of prompts.filter((p) => p.type !== 'review' && p.allowedTools?.some((t) => ['Write', 'Edit'].includes(t)))) {
+            expect(reviewed.has(prompt.type), `${prompt.type} must finish before review`).toBe(true);
+        }
+        expect(ancestors('report').has('logs')).toBe(true);
+        expect(ancestors('report').has('ai-observability')).toBe(true);
+        expect(byType.get('report').sink).toBe(true);
+        expect(ancestors('dashboard').has('logs')).toBe(false);
+    });
+});