Skip to content

fix(audit): seed the ledger in each narrow audit skill - #400

Merged
gewenyu99 merged 1 commit into
mainfrom
posthog/audit-leaf-seeds
Sep 18, 2026
Merged

gewenyu99 merged 1 commit into
mainfrom
posthog/audit-leaf-seeds

Conversation

@gewenyu99

Copy link
Copy Markdown
Collaborator

Problem

Five audit leaves tell the agent that the runtime hands them a ledger:

The audit ledger is seeded by the wizard with one pending check per … check.

It does not. The wizard pre-seeds only the comprehensive audit skill (and the retired events-audit program). A skill promoted to an audit-family leaf runs through the wizard's generic agent-skill program, which seeds nothing, so today wizard audit events and its four siblings run with no ledger on disk:

  • every audit_resolve_checks call fails with unknown check id(s)
  • the wizard's "Audit plan" tab stays empty for the whole run
  • the report step, which renders directly from the ledger, emits [label] placeholders
  • the same emptiness reaches the task stream

The same files contradict themselves: each one's check-entry section already says "This skill seeds N areas".

Change

Each skill seeds its own checklist in step 1, the pattern audit-attribution already uses — it states outright that the runtime does not pre-seed, and calls audit_seed_checks with an explicit payload.

Skill Rows seeded Areas
audit-events 7 Event Capture, Event Capture — Optimize
audit-autocapture 6 Autocapture, Autocapture — Optimize
audit-feature-flags 9 Feature Flags, Feature Flags — Optimize
audit-session-replay 8 Session Replay, Session Replay — Optimize
audit-identify 15 Identification, — Lifecycle, — Optimize, — Server SDK

Every id is one the skill's own steps resolve, in the order the steps reach them, with a label written from that check's rule text. audit-identify also declares the server-SDK area its step 5 uses; its check-entry section previously named three areas and omitted it.

The report steps said "Every check the wizard seeded for this skill ends up in the report" — now "Every check Step 1 seeded". That line is corrected in audit-attribution too, which already seeded itself.

Verification

  • node scripts/build.js succeeds, and the built audit-events.zip and audit-identify.zip carry the seed call.
  • vitest run scripts/plugins/tests scripts/lib/tests — 24 files, 207 tests pass.
  • Checked every seeded id against the ids each skill's steps resolve: no missing rows, no duplicates, no label over 55 characters, every area inside the wizard's 18-character area column.

Labels and areas are what the wizard renders in its live check list, and after PostHog/wizard#1263 the areas are also what the task stream publishes as progress rows.


Created with PostHog Desktop

@gewenyu99
gewenyu99 requested a review from a team as a code owner September 18, 2026 18:18

@edwinyjlim edwinyjlim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

man these are kinda messy

Five audit leaves stated that the wizard seeds their ledger. It does not:
the wizard pre-seeds only the comprehensive audit, so a leaf ran with no
ledger at all. Every audit_resolve_checks call failed, the Audit plan tab
stayed empty, and the report step rendered [label] placeholders.

Each skill now seeds its own checklist in step 1, the way audit-attribution
already does, with one row per id the skill resolves. audit-identify also
declares the server-SDK area its step 5 uses.

Generated-By: PostHog Desktop
Task-Id: 6490a5fd-5ee9-4203-b9c4-0b4ff180c45b
@gewenyu99
gewenyu99 force-pushed the posthog/audit-leaf-seeds branch from 5478e2b to 8e2d4a0 Compare September 18, 2026 18:35
@gewenyu99
gewenyu99 merged commit 5da51c3 into main Sep 18, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants