From 8e2d4a0b673a1fab986f3876c88a41947781e539 Mon Sep 17 00:00:00 2001 From: "Vincent (Wen Yu) Ge" Date: Fri, 18 Sep 2026 14:18:00 -0400 Subject: [PATCH] fix(audit): seed the ledger in each narrow audit skill Five audit leaves stated that the wizard seeds their ledger. It does not: the wizard pre-seeds only the comprehensive audit, so a leaf ran with no ledger at all. Every audit_resolve_checks call failed, the Audit plan tab stayed empty, and the report step rendered [label] placeholders. Each skill now seeds its own checklist in step 1, the way audit-attribution already does, with one row per id the skill resolves. audit-identify also declares the server-SDK area its step 5 uses. Generated-By: PostHog Desktop Task-Id: 6490a5fd-5ee9-4203-b9c4-0b4ff180c45b --- .../audit-attribution/references/4-report.md | 2 +- .../skills/audit-autocapture/description.md | 3 +- .../references/1-presence.md | 22 +++++++++++ .../audit-autocapture/references/4-report.md | 2 +- context/skills/audit-events/description.md | 3 +- .../audit-events/references/1-presence.md | 19 +++++++++ .../audit-events/references/4-report.md | 2 +- .../skills/audit-feature-flags/description.md | 3 +- .../references/1-presence.md | 23 ++++++++++- .../references/4-report.md | 2 +- context/skills/audit-identify/description.md | 5 ++- .../audit-identify/references/1-presence.md | 39 ++++++++++++++++--- .../audit-identify/references/6-report.md | 2 +- .../audit-session-replay/description.md | 3 +- .../references/1-presence.md | 22 ++++++++++- .../references/4-report.md | 2 +- 16 files changed, 135 insertions(+), 19 deletions(-) diff --git a/context/skills/audit-attribution/references/4-report.md b/context/skills/audit-attribution/references/4-report.md index bb9e1aa7..c7415091 100644 --- a/context/skills/audit-attribution/references/4-report.md +++ b/context/skills/audit-attribution/references/4-report.md @@ -4,7 +4,7 @@ next_step: null # Step 4 — Generate the audit report -The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check the wizard seeded for this skill ends up in the report, even passes; nothing is invented. +The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check Step 1 seeded ends up in the report, even passes; nothing is invented. ## Status diff --git a/context/skills/audit-autocapture/description.md b/context/skills/audit-autocapture/description.md index 7dd94310..8e90aadd 100644 --- a/context/skills/audit-autocapture/description.md +++ b/context/skills/audit-autocapture/description.md @@ -11,7 +11,7 @@ The audit covers two lenses: The audit runs as a step chain. **The exact step list lives in the reference files themselves, not in this overview.** Step 1 lives at `references/1-presence.md`; each step file ends with a `next_step:` frontmatter pointer to the next, and the final step has `next_step: null`. Follow them in the order they point. You must resolve each step in order before any source-tree exploration. -The audit ledger is seeded by the wizard with one pending check per autocapture check. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. +Step 1 seeds the audit ledger itself with one pending check per autocapture check through `mcp__wizard-tools__audit_seed_checks`, because the runtime does not pre-seed this skill. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. **Start by reading the path relative to this file at `references/1-presence.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md. @@ -33,6 +33,7 @@ The wizard intercepts these and updates the spinner. Use them freely — they ar The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**: +- `mcp__wizard-tools__audit_seed_checks({ checks })` — writes the full pending checklist once at Step 1 and replaces the file atomically, so one call per run is safe. - `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call. All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so. diff --git a/context/skills/audit-autocapture/references/1-presence.md b/context/skills/audit-autocapture/references/1-presence.md index 4f2dd6d5..2a9b1dbf 100644 --- a/context/skills/audit-autocapture/references/1-presence.md +++ b/context/skills/audit-autocapture/references/1-presence.md @@ -12,6 +12,7 @@ Emit: ``` [STATUS] Detecting PostHog autocapture configuration +[STATUS] Seeding audit checklist ``` ## Action @@ -30,6 +31,27 @@ If init sites are found, `Read` each file once and inspect the init options for ## Decision - **Grep returns zero hits anywhere in the project:** emit `[ABORT] PostHog SDK initialization not found` and stop. The wizard catches `[ABORT]` and terminates the run. +- **Init found:** seed the audit ledger, then continue based on the autocapture configuration. + +## Seed the audit ledger + +The runtime does not pre-seed this skill's ledger, so call `mcp__wizard-tools__audit_seed_checks` with the exact payload below. The tool replaces the file atomically, so one call at the start of every run is safe. Do not seed the ledger when this step aborts. Seeding is not resolving checks. + +```json +{ + "checks": [ + { "id": "autocapture-intentional", "area": "Autocapture", "label": "Autocapture is explicitly configured", "status": "pending" }, + { "id": "autocapture-mask-config", "area": "Autocapture", "label": "Autocapture PII masking stays enabled", "status": "pending" }, + { "id": "autocapture-allowlists", "area": "Autocapture", "label": "Autocapture is scoped on high-traffic projects", "status": "pending" }, + { "id": "autocapture-ratio-to-custom", "area": "Autocapture — Optimize", "label": "Autocapture and custom events have a healthy mix", "status": "pending" }, + { "id": "autocapture-copied-text", "area": "Autocapture — Optimize", "label": "Copied text capture stays off on high-traffic sites", "status": "pending" }, + { "id": "autocapture-dead-clicks-vs-heatmap", "area": "Autocapture — Optimize", "label": "Dead-click autocapture is off when heatmaps suffice", "status": "pending" } + ] +} +``` + +## Continue + - **Init found, `autocapture: false` is explicitly set on every init site:** autocapture is fully off. Resolve all three Step 2 fix checks (`autocapture-intentional`, `autocapture-mask-config`, `autocapture-allowlists`) in a single `audit_resolve_checks` call with `status: "pass"` and `details: "skip: autocapture explicitly disabled in init config"`. Then continue to Step 3 — optimize-side checks still have work to do for the dead-clicks and ratio checks. - **Init found, autocapture not fully disabled:** continue normally. diff --git a/context/skills/audit-autocapture/references/4-report.md b/context/skills/audit-autocapture/references/4-report.md index 68a1dec7..ab61639e 100644 --- a/context/skills/audit-autocapture/references/4-report.md +++ b/context/skills/audit-autocapture/references/4-report.md @@ -4,7 +4,7 @@ next_step: null # Step 4 — Generate the audit report -The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check the wizard seeded for this skill ends up in the report, even passes; nothing is invented. +The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check Step 1 seeded ends up in the report, even passes; nothing is invented. ## Status diff --git a/context/skills/audit-events/description.md b/context/skills/audit-events/description.md index 25731cd7..928ebf0f 100644 --- a/context/skills/audit-events/description.md +++ b/context/skills/audit-events/description.md @@ -11,7 +11,7 @@ The audit covers two lenses: The audit runs as a step chain. **The exact step list lives in the reference files themselves, not in this overview.** Step 1 lives at `references/1-presence.md`; each step file ends with a `next_step:` frontmatter pointer to the next, and the final step has `next_step: null`. Follow them in the order they point. You must resolve each step in order before any source-tree exploration. -The audit ledger is seeded by the wizard with one pending check per event check. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. +Step 1 seeds the audit ledger itself with one pending check per event check through `mcp__wizard-tools__audit_seed_checks`, because the runtime does not pre-seed this skill. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. **Start by reading the path relative to this file at `references/1-presence.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md. @@ -33,6 +33,7 @@ The wizard intercepts these and updates the spinner. Use them freely — they ar The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**: +- `mcp__wizard-tools__audit_seed_checks({ checks })` — writes the full pending checklist once, at Step 1. The tool replaces the file atomically, so one call per run is safe. - `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call. All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so. diff --git a/context/skills/audit-events/references/1-presence.md b/context/skills/audit-events/references/1-presence.md index 0447db17..29df3d60 100644 --- a/context/skills/audit-events/references/1-presence.md +++ b/context/skills/audit-events/references/1-presence.md @@ -12,6 +12,7 @@ Emit: ``` [STATUS] Detecting PostHog event capture usage +[STATUS] Seeding audit checklist ``` ## Action @@ -27,6 +28,24 @@ Run **two `Grep` calls in parallel**, both with `output_mode: "files_with_matche - **Init found, capture not found:** continue. Step 2 (fix) will detect this and resolve its four ledger checks with skip details. Step 3 (optimize) still has work to do because pageview defaults and downstream usage may still matter. - **Both found:** continue normally. +## Seed audit ledger + +The runtime does not pre-seed this skill's ledger, so call `mcp__wizard-tools__audit_seed_checks` here with the exact payload below. The tool replaces the file atomically, so one call at the start of every run is safe. Do not seed when this step aborts. + +```json +{ + "checks": [ + { "id": "capture-event-names-static", "area": "Event Capture", "label": "Event names use static strings", "status": "pending" }, + { "id": "event-naming-standardization", "area": "Event Capture", "label": "Event names follow a consistent convention", "status": "pending" }, + { "id": "event-duplicates-and-bloat", "area": "Event Capture", "label": "Event captures have no duplicates or bloat", "status": "pending" }, + { "id": "event-quality-context-review", "area": "Event Capture", "label": "Event captures contain no material quality issues", "status": "pending" }, + { "id": "event-usage-coverage", "area": "Event Capture — Optimize", "label": "Captured events are used in PostHog artifacts", "status": "pending" }, + { "id": "events-pageview-defaults", "area": "Event Capture — Optimize", "label": "Automatic pageviews do not dominate event volume", "status": "pending" }, + { "id": "events-env-pollution", "area": "Event Capture — Optimize", "label": "Production project excludes non-production events", "status": "pending" } + ] +} +``` + Do not read any files in this step. Do not call `audit_resolve_checks`. Do not preload future steps. Continue to **`2-events-fix.md`**. diff --git a/context/skills/audit-events/references/4-report.md b/context/skills/audit-events/references/4-report.md index 5afb981e..d4d3fdb1 100644 --- a/context/skills/audit-events/references/4-report.md +++ b/context/skills/audit-events/references/4-report.md @@ -4,7 +4,7 @@ next_step: null # Step 4 — Generate the audit report -The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check the wizard seeded for this skill ends up in the report, even passes; nothing is invented. +The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check Step 1 seeded ends up in the report, even passes; nothing is invented. ## Status diff --git a/context/skills/audit-feature-flags/description.md b/context/skills/audit-feature-flags/description.md index 39232516..8c786625 100644 --- a/context/skills/audit-feature-flags/description.md +++ b/context/skills/audit-feature-flags/description.md @@ -13,7 +13,7 @@ The billed feature flag endpoint is `/flags` (the renamed `/decide`). All refere The audit runs as a step chain. **The exact step list lives in the reference files themselves, not in this overview.** Step 1 lives at `references/1-presence.md`; each step file ends with a `next_step:` frontmatter pointer to the next, and the final step has `next_step: null`. Follow them in the order they point. You must resolve each step in order before any source-tree exploration. -The audit ledger is seeded by the wizard with one pending check per feature flag check. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. +Step 1 seeds the audit ledger itself with one pending check per feature flag check through `mcp__wizard-tools__audit_seed_checks`, because the runtime does not pre-seed this skill. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. **Start by reading the path relative to this file at `references/1-presence.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md. @@ -35,6 +35,7 @@ The wizard intercepts these and updates the spinner. Use them freely — they ar The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**: +- `mcp__wizard-tools__audit_seed_checks({ checks })` — writes the full pending checklist once, at Step 1. It replaces the file atomically, so one call per run is safe. - `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call. All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so. diff --git a/context/skills/audit-feature-flags/references/1-presence.md b/context/skills/audit-feature-flags/references/1-presence.md index 921d8a92..00a4e948 100644 --- a/context/skills/audit-feature-flags/references/1-presence.md +++ b/context/skills/audit-feature-flags/references/1-presence.md @@ -12,6 +12,7 @@ Emit: ``` [STATUS] Detecting PostHog feature flag usage +[STATUS] Seeding audit checklist ``` ## Action @@ -25,9 +26,29 @@ Run **two `Grep` calls in parallel**, both with `output_mode: "files_with_matche ## Decision -- **Surface grep returns zero hits anywhere in the project:** emit `[ABORT] No PostHog feature flag usage found` and stop. The wizard catches `[ABORT]` and terminates the run. +- **Surface grep returns zero hits anywhere in the project:** emit `[ABORT] No PostHog feature flag usage found` and stop. The wizard catches `[ABORT]` and terminates the run. Do not seed the ledger. - **Surface grep finds hits:** continue. +## Seed the audit ledger + +The ledger lives at `.posthog-audit-checks.json` and renders live in the wizard sidebar / "Audit plan" tab. **The runtime does not pre-seed this skill's ledger**, so call `mcp__wizard-tools__audit_seed_checks` here with the exact payload below. Do not seed when this step aborts. The tool replaces the file atomically, so one call at the start of every run is safe. + +```json +{ + "checks": [ + { "id": "ff-bootstrap-when-known-set", "area": "Feature Flags", "label": "Known initial flag sets use bootstrap", "status": "pending" }, + { "id": "ff-await-readiness", "area": "Feature Flags", "label": "Flag evaluation waits for readiness", "status": "pending" }, + { "id": "ff-default-values", "area": "Feature Flags", "label": "Flag evaluations have safe default values", "status": "pending" }, + { "id": "ff-bootstrap-distinct-id-mismatch", "area": "Feature Flags", "label": "Bootstrap distinct ID matches stable identity", "status": "pending" }, + { "id": "ff-identified-only-pre-auth-targeting", "area": "Feature Flags", "label": "Pre-auth flag targeting works for anonymous users", "status": "pending" }, + { "id": "ff-active-but-unreferenced", "area": "Feature Flags — Optimize", "label": "Active flags have codebase references", "status": "pending" }, + { "id": "ff-local-eval-polling-interval", "area": "Feature Flags — Optimize", "label": "Local evaluation uses an intentional polling interval", "status": "pending" }, + { "id": "ff-local-eval-in-edge-handlers", "area": "Feature Flags — Optimize", "label": "Edge handlers avoid local flag evaluation", "status": "pending" }, + { "id": "ff-test-ci-gating", "area": "Feature Flags — Optimize", "label": "Test and CI runs gate flag evaluation", "status": "pending" } + ] +} +``` + ## Record local-evaluation detection Local evaluation is detected when the second grep returns **at least one hit** (the project either initializes a server SDK with `personal_api_key` / a feature-flags secure API key, or calls a local-evaluation-only API like `getAllFlagsAndPayloads` / `getAllFlags` on the server). Keep this signal in working memory — Step 3 uses it to decide whether to run two of the optimize subagents or skip them as `pass` with `details: "skip: local evaluation not detected"`. diff --git a/context/skills/audit-feature-flags/references/4-report.md b/context/skills/audit-feature-flags/references/4-report.md index b1ad9122..1c274511 100644 --- a/context/skills/audit-feature-flags/references/4-report.md +++ b/context/skills/audit-feature-flags/references/4-report.md @@ -4,7 +4,7 @@ next_step: null # Step 4 — Generate the audit report -The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check the wizard seeded for this skill ends up in the report, even passes; nothing is invented. +The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check Step 1 seeded ends up in the report, even passes; nothing is invented. ## Status diff --git a/context/skills/audit-identify/description.md b/context/skills/audit-identify/description.md index 23811759..c3e17d76 100644 --- a/context/skills/audit-identify/description.md +++ b/context/skills/audit-identify/description.md @@ -12,7 +12,7 @@ The audit covers three lenses: The audit runs as a step chain. **The exact step list lives in the reference files themselves, not in this overview.** Step 1 lives at `references/1-presence.md`; each step file ends with a `next_step:` frontmatter pointer to the next, and the final step has `next_step: null`. Follow them in the order they point. You must resolve each step in order before any source-tree exploration. -The audit ledger is seeded by the wizard with one pending check per identify check. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. +Step 1 seeds the audit ledger itself with one pending check per identify check through `mcp__wizard-tools__audit_seed_checks`, because the runtime does not pre-seed this skill. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. **Start by reading the path relative to this file at `references/1-presence.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md. @@ -34,6 +34,7 @@ The wizard intercepts these and updates the spinner. Use them freely — they ar The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**: +- `mcp__wizard-tools__audit_seed_checks({ checks })` — writes the full pending checklist once at Step 1. It replaces the file atomically, so one call per run is safe. - `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call. All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so. @@ -41,7 +42,7 @@ All audit ledger calls are atomic and serialize internally — **concurrent call ### Check entry shape - `id` — stable kebab-case slug. Reuse the existing seeded ids exactly when calling `audit_resolve_checks`. -- `area` — short group name. This skill seeds three areas: `Identification` (fix), `Identification — Lifecycle` (quality), and `Identification — Optimize` (cost). +- `area` — short group name. This skill seeds four areas: `Identification` (fix), `Identification — Lifecycle` (quality), `Identification — Optimize` (cost), and `Identification — Server SDK` (server-side hygiene). - `label` — short human name. - `status` — `pending` | `pass` | `error` | `warning` | `suggestion`. - `file` — optional `path:line` for findings tied to a location. diff --git a/context/skills/audit-identify/references/1-presence.md b/context/skills/audit-identify/references/1-presence.md index 68ee3878..9becb5cb 100644 --- a/context/skills/audit-identify/references/1-presence.md +++ b/context/skills/audit-identify/references/1-presence.md @@ -2,9 +2,9 @@ next_step: 2-identify-fix.md --- -# Step 1 — Presence detector +# Step 1 — Presence detector + seed the ledger -This step decides whether the rest of the audit has anything to look at. Run it **before** any other work. Resolve zero ledger checks here — this step is gating only. +This step decides whether the rest of the audit has anything to look at and seeds the audit ledger. Run it **before** any other work. Resolve zero ledger checks here — this step is gating only. ## Status @@ -12,6 +12,7 @@ Emit: ``` [STATUS] Detecting PostHog identify usage +[STATUS] Seeding audit checklist ``` ## Action @@ -23,10 +24,38 @@ Run **two `Grep` calls in parallel**, both with `output_mode: "files_with_matche ## Decision -- **Both greps return zero hits anywhere in the project:** emit `[ABORT] PostHog SDK initialization not found` and stop. The wizard catches `[ABORT]` and terminates the run. -- **Init found, identify not found:** continue. Step 2 (fix) will detect this and resolve its four ledger checks with skip details. Step 3 (optimize) still has work to do because `person_profiles` config still matters even without identify calls. +- **Both greps return zero hits anywhere in the project:** emit `[ABORT] PostHog SDK initialization not found` and stop. The wizard catches `[ABORT]` and terminates the run. Do not seed the ledger. +- **Init found, identify not found:** continue. Step 2 (fix) will detect this and resolve its five ledger checks with skip details. Step 3 (optimize) still has work to do because `person_profiles` config still matters even without identify calls. - **Both found:** continue normally. -Do not read any files in this step. Do not call `audit_resolve_checks`. Do not preload future steps. +## Seed the audit ledger + +The ledger lives at `.posthog-audit-checks.json` and renders live in the wizard sidebar / "Audit plan" tab. **The runtime does not pre-seed this skill's ledger** — call `mcp__wizard-tools__audit_seed_checks` directly here with the exact payload below. The tool replaces the file atomically, so calling it once at the start of every run is safe. Do not seed when this step aborts. + +```json +{ + "checks": [ + { "id": "identify-stable-distinct-id", "area": "Identification", "label": "Identify uses stable authenticated distinct IDs", "status": "pending" }, + { "id": "identify-not-late", "area": "Identification", "label": "Identify runs before captures and flag evaluations", "status": "pending" }, + { "id": "cross-runtime-distinct-id", "area": "Identification", "label": "Client and server use the same distinct ID", "status": "pending" }, + { "id": "identify-reset-on-logout", "area": "Identification", "label": "Logout and account switches call posthog.reset()", "status": "pending" }, + { "id": "identify-sequential-calls", "area": "Identification", "label": "Each flow has one consistent identify call", "status": "pending" }, + { "id": "identify-set-discipline", "area": "Identification — Lifecycle", "label": "Person properties use $set and $set_once correctly", "status": "pending" }, + { "id": "identify-alias-usage", "area": "Identification — Lifecycle", "label": "Alias usage does not block identity merges", "status": "pending" }, + { "id": "identify-groupidentify-correctness", "area": "Identification — Lifecycle", "label": "Group calls use valid types and stable keys", "status": "pending" }, + { "id": "identify-person-profiles-mode", "area": "Identification — Optimize", "label": "person_profiles matches traffic shape", "status": "pending" }, + { "id": "identify-isidentified-guard", "area": "Identification — Optimize", "label": "Identify calls are guarded against repeat firing", "status": "pending" }, + { "id": "identify-duplicate-identify-per-session", "area": "Identification — Optimize", "label": "$identify events do not repeat within sessions", "status": "pending" }, + { "id": "identify-duplicate-groupidentify-per-session", "area": "Identification — Optimize", "label": "$groupidentify events do not repeat within sessions", "status": "pending" }, + { "id": "server-process-person-profile", "area": "Identification — Server SDK", "label": "Server captures disable unwanted person processing", "status": "pending" }, + { "id": "server-sdk-flush-on-exit", "area": "Identification — Server SDK", "label": "Short-lived server SDKs flush before exit", "status": "pending" }, + { "id": "server-set-without-identify", "area": "Identification — Server SDK", "label": "Server $set captures have a canonical identify call", "status": "pending" } + ] +} +``` + +Seeding is not resolving. Do not call `audit_resolve_checks` in this step. + +Do not read any files in this step. Do not preload future steps. Continue to **`2-identify-fix.md`**. diff --git a/context/skills/audit-identify/references/6-report.md b/context/skills/audit-identify/references/6-report.md index 0449fb24..ba3bceb2 100644 --- a/context/skills/audit-identify/references/6-report.md +++ b/context/skills/audit-identify/references/6-report.md @@ -4,7 +4,7 @@ next_step: null # Step 6 — Generate the audit report -The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check the wizard seeded for this skill ends up in the report, even passes; nothing is invented. +The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check Step 1 seeded ends up in the report, even passes; nothing is invented. ## Status diff --git a/context/skills/audit-session-replay/description.md b/context/skills/audit-session-replay/description.md index 86994133..eea0fa43 100644 --- a/context/skills/audit-session-replay/description.md +++ b/context/skills/audit-session-replay/description.md @@ -11,7 +11,7 @@ The audit covers two lenses: The audit runs as a step chain. **The exact step list lives in the reference files themselves, not in this overview.** Step 1 lives at `references/1-presence.md`; each step file ends with a `next_step:` frontmatter pointer to the next, and the final step has `next_step: null`. Follow them in the order they point. You must resolve each step in order before any source-tree exploration. -The audit ledger is seeded by the wizard with one pending check per session replay check. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. +Step 1 seeds the audit ledger itself with one pending check per session replay check through `mcp__wizard-tools__audit_seed_checks`, because the runtime does not pre-seed this skill. **Each step gracefully handles a missing check id**: if a step's expected id is not in the ledger, it skips its `audit_resolve_checks` call for that id and continues. Use `mcp__wizard-tools__audit_resolve_checks` to patch each check as you finish it. **Start by reading the path relative to this file at `references/1-presence.md`.** Do not Glob, ls, or find the skill directory. Do not preload future steps. Do not re-read a step file once you've moved past it. Do not re-read SKILL.md. @@ -33,6 +33,7 @@ The wizard intercepts these and updates the spinner. Use them freely — they ar The ledger lives at `.posthog-audit-checks.json` and is rendered live in the "Audit plan" tab. It is owned by MCP tools — **never `Write` this file directly**: +- `mcp__wizard-tools__audit_seed_checks({ checks })` — writes the full pending checklist once, at Step 1. It replaces the file atomically, so one call per run is safe. - `mcp__wizard-tools__audit_resolve_checks({ updates })` — patch one or more checks by `id`. Each `update` is `{ id, status, file?, details? }`. Batch updates from the same step into a single call. All audit ledger calls are atomic and serialize internally — **concurrent calls from parallel subagents cannot lose updates**, so feel free to fan out runtime checks across `Agent` subagents when a step says so. diff --git a/context/skills/audit-session-replay/references/1-presence.md b/context/skills/audit-session-replay/references/1-presence.md index a06df2bf..ead4bc65 100644 --- a/context/skills/audit-session-replay/references/1-presence.md +++ b/context/skills/audit-session-replay/references/1-presence.md @@ -4,7 +4,7 @@ next_step: 2-session-replay-fix.md # Step 1 — Presence detector -This step decides whether the rest of the audit has anything to look at. Run it **before** any other work. Resolve zero ledger checks here — this step is gating only. +This step decides whether the rest of the audit has anything to look at and seeds the audit ledger. Run it **before** any other work. Resolve zero ledger checks here — this step gates execution and seeds the ledger. ## Status @@ -12,6 +12,7 @@ Emit: ``` [STATUS] Detecting PostHog session replay configuration +[STATUS] Seeding audit checklist ``` ## Action @@ -27,6 +28,25 @@ Run **two `Grep` calls in parallel**, both with `output_mode: "files_with_matche - **Init found, replay APIs not found:** continue. Some Step 3 (optimize) checks still apply via MCP project settings (sampling rate, triggers) even when the codebase has no replay-specific config. - **Both found:** continue normally. +## Seed the audit ledger + +Only after continuing from the decision above, seed the ledger. **The runtime does not pre-seed this skill's ledger** — call `mcp__wizard-tools__audit_seed_checks` directly here with the exact payload below. The tool replaces the file atomically, so calling it once at the start of every run is safe. Do not seed the ledger if this step aborts. + +```json +{ + "checks": [ + { "id": "replay-minimum-duration-set", "area": "Session Replay", "label": "Replay has a positive minimum duration", "status": "pending" }, + { "id": "replay-mask-config", "area": "Session Replay", "label": "Replay keeps inputs masked on PII surfaces", "status": "pending" }, + { "id": "replay-disabled-in-test-envs", "area": "Session Replay", "label": "Replay is disabled in test and CI environments", "status": "pending" }, + { "id": "replay-strict-minimum-duration", "area": "Session Replay", "label": "Replay uses strict minimum duration", "status": "pending" }, + { "id": "replay-sampling-rate", "area": "Session Replay — Optimize", "label": "Replay sampling rate matches recording volume", "status": "pending" }, + { "id": "replay-triggers-configured", "area": "Session Replay — Optimize", "label": "Replay triggers focus on important sessions", "status": "pending" }, + { "id": "replay-network-recording-filtered", "area": "Session Replay — Optimize", "label": "Replay network recording filters payloads", "status": "pending" }, + { "id": "replay-mobile-sampling", "area": "Session Replay — Optimize", "label": "Mobile replay uses sampling below 100%", "status": "pending" } + ] +} +``` + Do not read any files in this step. Do not call `audit_resolve_checks`. Do not preload future steps. Continue to **`2-session-replay-fix.md`**. diff --git a/context/skills/audit-session-replay/references/4-report.md b/context/skills/audit-session-replay/references/4-report.md index c63802c0..b6dadcca 100644 --- a/context/skills/audit-session-replay/references/4-report.md +++ b/context/skills/audit-session-replay/references/4-report.md @@ -4,7 +4,7 @@ next_step: null # Step 4 — Generate the audit report -The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check the wizard seeded for this skill ends up in the report, even passes; nothing is invented. +The audit report is rendered **directly from `.posthog-audit-checks.json`** — that file is the source of truth. Every check Step 1 seeded ends up in the report, even passes; nothing is invented. ## Status