-
Notifications
You must be signed in to change notification settings - Fork 17
feat(skills): add cull-feature-flags skill #387
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
83c4fb7
f461b28
f8899df
90ef930
b13f78c
4675bdb
fcea7eb
f7c2b6d
4c561f8
cb88b3a
adda307
2c7b442
16c36cd
7cb43b3
338fda1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # Cull stale feature flags. The wizard seeds the ledger deterministically | ||
| # (scan + fetch + classify host-side); this skill verifies, asks once, applies, | ||
| # and reports. One variant per framework the wizard's scanner supports; | ||
| # `framework` is the detection id the wizard resolves the variant from. | ||
| # Reachable via `wizard cull-feature-flags` (wizard-registered flat command) | ||
| # and `wizard skill cull-feature-flags-<variant>`. | ||
| type: docs-only | ||
| template: description.md | ||
| description: Remove stale PostHog feature flags from a {display_name} codebase and disable them in PostHog, from a ledger the wizard seeds | ||
| tags: [feature-flags] | ||
| cli: | ||
| role: skill | ||
| references: | ||
| preamble: "**Read ONLY this file.** Do not read any other reference file until this one tells you to." | ||
| shared_docs: | ||
| - https://posthog.com/docs/feature-flags/best-practices.md | ||
| variants: | ||
| - id: nextjs | ||
| framework: nextjs | ||
| default: true | ||
| display_name: Next.js | ||
| tags: [nextjs, nextjs-feature-flags, react, javascript, javascript_web, javascript_node] | ||
| docs_urls: | ||
| - https://posthog.com/docs/feature-flags/installation/react.md | ||
| - https://posthog.com/docs/libraries/next-js.md |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # Cull stale PostHog feature flags in a {display_name} project | ||
|
|
||
| This skill removes feature flags that have outlived their purpose: rolled out to everyone, off for everyone, archived or deleted in PostHog but still checked in code, or defined in PostHog and never evaluated anywhere. The wizard already did the detection before you started: it scanned the source tree for flag calls, fetched the project's flags, classified every flag with plain rules, and wrote one row per flag into `.posthog-audit-checks.json` at the project root. That ledger is the ground truth. You never grep for flags, never re-classify a row, and never promote a healthy flag into a removal. | ||
|
|
||
| Your job is three steps: verify each proposed row at its call site, ask the user once which rows to apply, cull the approved ones (code first, PostHog second), then write the report. | ||
|
|
||
| **Start by reading `references/1-verify-call-sites.md`.** Do not Glob, ls, or find the skill directory. Do not read `2-apply.md` or `3-report.md` until the current step tells you to. | ||
|
|
||
| ## State | ||
|
|
||
| | Lane | Ledger areas | What culling does | | ||
| |---|---|---| | ||
| | Decided in PostHog | `Rolled out` | Keeps the code that runs today, drops the check, and disables the flag. | | ||
| | Decided in PostHog | `Off for everyone` | Keeps the off branch, drops the check, and disables the flag. | | ||
| | Off in PostHog, still in code | `Archived in PostHog`, `Disabled in PostHog`, `Deleted in PostHog` | Keeps the off branch, drops the check, and makes no PostHog change. | | ||
| | In PostHog, not in code | `Unreferenced`, `Comment only`, `Dead code` | Disables the flag and removes the mention or deletes the dead module when present. | | ||
| | Nothing to cull | `Many call sites`, `Healthy` | Report only. | | ||
|
|
||
| - `.posthog-audit-checks.json` (project root): the ledger. `area` is one of: `Rolled out` (100% to everyone, no conditions), `Off for everyone` (0% everywhere, possibly rolled back), `Archived in PostHog`, `Disabled in PostHog`, `Unreferenced` (in PostHog, never evaluated in code), `Comment only` (key appears only in a comment or config string), `Dead code` (the only file evaluating it is unreachable), `Deleted in PostHog` (code evaluates a key PostHog does not have), `Many call sites` (three or more files evaluate it directly, suggestion only), `Healthy`. `Read` it once at the start of each step. Each row is `{ id, area, label, status, file?, details? }` where `id` is the flag key, `area` is the bucket the wizard assigned, `label` names the proposed action, `file` is the first call site as `path:line`, and `details` carries the rollout summary and every call site. | ||
| - Rows change only through `mcp__wizard-tools__audit_resolve_checks`. Never `Edit` or `Write` the ledger file. Never delete it. | ||
| - Row statuses this skill uses: `pending` (seeded, not yet verified), `warning` (verified, proposed to the user), `pass` (kept, left for you, or culled), `error` (cull failed, reason in `details`). | ||
|
|
||
| The wizard prompt tells you whether the repo uses bulk evaluation (`getAllFlags`) or dynamic flag keys. When it does, every `Unreferenced` or `Comment only` row needs a real check at the bulk or dynamic call site before it can be proposed. | ||
|
|
||
| ## Status | ||
|
|
||
| Report progress with `[STATUS] <message>` lines. Each step lists the exact strings to emit. They are cheap; use them at every sub-step. | ||
|
|
||
| ## Abort | ||
|
|
||
| Report unrecoverable preconditions with exactly one `[ABORT] <reason>` line and stop. The wizard terminates the run; do not halt yourself. | ||
|
|
||
| - `[ABORT] No flag cull ledger found` when `.posthog-audit-checks.json` does not exist (this skill needs `wizard cull-feature-flags` to seed it). | ||
| - `[ABORT] No PostHog feature flag usage found` when the ledger has zero rows. | ||
|
|
||
| ## Rules | ||
|
|
||
| 1. **Disable, never delete.** The only PostHog mutation this skill makes is disabling a flag. Archiving and deleting are for the user to do in the app. | ||
| 2. **Code before PostHog.** For a row that needs both a code edit and a disable, the edit lands first and the disable only after the edit succeeded, so a failed edit never leaves a disabled flag behind live code. | ||
| 3. **One consent call.** Exactly one `wizard_ask` for the whole run: one pick list per lane, then the report-only or cull choice last. Nothing is culled without it. | ||
| 4. **Winning branch only.** Removing a flag check means keeping the branch the flag would resolve to and deleting the other one, plus the now-unused import or hook. Do not restructure beyond that. | ||
| 5. **Downgrade freely, never upgrade.** During verification a row may be resolved to `pass` (keep) with a reason. A `pass` row seeded as healthy is never touched. | ||
|
|
||
| ## Available tools | ||
|
|
||
| {{> mcp-tool-calling}} | ||
|
|
||
| **Verify:** `Read` on each call-site file. | ||
|
|
||
| **Confirm:** `mcp__wizard-tools__wizard_ask`, called once (see `2-apply.md`). | ||
|
|
||
| **Apply:** `Edit` for code; through `exec`, the flag disable tool (discover it with `search feature-flag`, run `info` on it, then `call`). | ||
|
|
||
| **Ledger:** `mcp__wizard-tools__audit_resolve_checks` for every status change. | ||
|
|
||
| ## Reference files | ||
|
|
||
| {references} | ||
|
|
||
| ## Framework guidelines | ||
|
|
||
| {commandments} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| --- | ||
| next_step: 2-apply.md | ||
| --- | ||
|
|
||
| # Step 1: Verify each proposed row at its call site | ||
|
|
||
| This step turns every `pending` ledger row into `warning` (verified, will be proposed) or `pass` (keep, with a reason). It does NOT ask the user anything and does NOT edit code or PostHog; that belongs to step 2. | ||
|
|
||
| ## Status | ||
|
|
||
| Emit at the start: | ||
|
|
||
| ``` | ||
| [STATUS] Reading the flag cull ledger | ||
| ``` | ||
|
|
||
| ## Read the ledger | ||
|
|
||
| `Read` `.posthog-audit-checks.json` at the project root. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Mmmm creative use of this ledger
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On purpose, the ledger already had the tui and the resolve tool. Open to a rename. |
||
|
|
||
| - File missing: emit `[ABORT] No flag cull ledger found` and stop. | ||
| - Zero rows: emit `[ABORT] No PostHog feature flag usage found` and stop. | ||
|
|
||
| Rows with `status: pass` were seeded healthy. Leave them alone for the whole run. | ||
|
|
||
| ## Verify the pending rows | ||
|
|
||
| Emit: | ||
|
|
||
| ``` | ||
| [STATUS] Verifying <n> flag call sites | ||
| ``` | ||
|
|
||
| For each `pending` row, in ledger order, emit `[STATUS] Verifying <key>`. For `Rolled out` and `Off for everyone`, `Read` the call site named in `file` and every extra site listed in `details` as `also <path:line>`, around the given line. For `Archived in PostHog`, `Disabled in PostHog`, and `Deleted in PostHog`, `Read` only the first call site named in `file`. Emit `[STATUS] Reading <file>` before each call-site file `Read`. `Dead code` keeps its one `Grep` for the module's basename; do not read call-site files for that row. Decide one of: | ||
|
|
||
| | bucket (`area`) | confirm as `warning` when | downgrade to `pass` when | | ||
| |---|---|---| | ||
| | `Rolled out` | the call site is a boolean check whose true branch is the current behaviour | the flag gates something that must stay switchable (kill switch, ops toggle named as such) | | ||
| | `Off for everyone` | the call site is a boolean check whose false branch is the current behaviour | the feature is clearly mid-build (recent scaffolding, TODOs pointing at it), or the flag or its call site reads like a kill switch or rollback (name, comment, a recent change back to 0%) | | ||
| | `Archived in PostHog`, `Disabled in PostHog` | the call site is a boolean check; keep the false branch | never, these are always safe to propose | | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| | `Deleted in PostHog` | no live flag in PostHog resembles the key | the key looks like a typo of a healthy flag key in the ledger (note the match in `details`) | | ||
| | `Unreferenced`, `Comment only` | the prompt says no bulk evaluation and no dynamic keys | the prompt reports `getAllFlags` or dynamic keys and the bulk or dynamic call site could reach this key | | ||
| | `Dead code` | nothing imports the file (the wizard checked; confirm with one `Grep` for the module's basename) | something imports it after all | | ||
| | `Many call sites` | always `warning`, it is a suggestion, not a removal | never | | ||
|
|
||
| **Truncated scan.** When the Scan facts say the scan was truncated, `Unreferenced` and `Comment only` are unproven: the wizard stopped before reading every file. Downgrade those rows to `pass` with `; kept: scan truncated, not proven unreferenced` so they land in the report's Kept table with that reason. Do not Grep for the key; the rule against grepping for flags holds. | ||
|
|
||
| Resolve each row through `mcp__wizard-tools__audit_resolve_checks` as soon as it is decided (one call per row, so the run screen moves while you work): | ||
|
|
||
| - confirmed: `status: "warning"`, `details` = the seeded details plus `; winning branch: true|false|n/a` | ||
| - downgraded: `status: "pass"`, `details` = the seeded details plus `; kept: <one-line reason>` | ||
|
|
||
| ## Output | ||
|
|
||
| Every row that was `pending` is now `warning` or `pass`. No other file changed. Emit: | ||
|
|
||
| ``` | ||
| [STATUS] <w> flags proposed, <k> kept | ||
| ``` | ||
|
|
||
| Then emit: | ||
|
|
||
| ``` | ||
| [STATUS] Ready to ask | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| --- | ||
| next_step: 3-report.md | ||
| --- | ||
|
|
||
| # Step 2: Ask once, then cull the approved rows | ||
|
|
||
| This step asks the user which `warning` rows to apply, then applies them: code edit first, PostHog disable second. It does NOT write the report; that belongs to step 3. Step 1 already decided the winning branch for every row; do not re-verify. | ||
|
|
||
| ## Ask | ||
|
|
||
| Emit: | ||
|
|
||
| ``` | ||
| [STATUS] Waiting for confirmation | ||
| ``` | ||
|
|
||
| `Read` `.posthog-audit-checks.json`. Collect every row with `status: "warning"` whose `area` is not `Many call sites` (that bucket is report-only). | ||
|
|
||
| Zero such rows: do not call `wizard_ask`; skip to the Output section, nothing to apply. | ||
|
|
||
| Call `mcp__wizard-tools__wizard_ask` exactly once. Build its questions in this order. Include a `multi` question only when its listed areas contain at least one collected row. | ||
|
|
||
| 1. For `Rolled out`, use `id: "rolled-out"`, `kind: "multi"`, and `required: false`. Prompt: | ||
|
|
||
| > Pick rolled-out flags to cull. These flags are at 100% for everyone with no conditions, so the code only ever runs the "on" branch. Culling keeps that branch, drops the check, and disables the flag in PostHog. Re-enable it with one toggle; rollout conditions are kept. | ||
|
|
||
| Example description: `100% to everyone with no conditions; keeps the code that runs today; src/checkout.ts:42, src/cart.ts:18; disables the flag`. | ||
|
|
||
| 2. For `Off for everyone`, use `id: "off-for-everyone"`, `kind: "multi"`, and `required: false`. Prompt: | ||
|
|
||
| > Pick flags that are off for everyone to cull. These flags are at 0% for everyone. A flag rolled back after an incident looks exactly like one that never shipped, so a 0% flag may be a rollback lever. Keeping it costs nothing. Culling keeps the "off" branch, drops the check, and disables the flag. | ||
|
|
||
| Example description: `0% to everyone after the checkout rollback; keeps the off branch; src/checkout.ts:42; disables the flag`. | ||
|
|
||
| 3. For `Archived in PostHog`, `Disabled in PostHog`, and `Deleted in PostHog`, use `id: "off-in-posthog"`, `kind: "multi"`, and `required: false`. Prompt: | ||
|
|
||
| > Pick flags that are already off or absent in PostHog to cull. PostHog already turned these off or no longer has them, but the code still checks them. Culling keeps the "off" branch and drops the check. Nothing changes in PostHog for these flags. | ||
|
|
||
| Example description: `disabled in PostHog; keeps the off branch; src/search.ts:27; no PostHog change`. | ||
|
|
||
| 4. For `Unreferenced`, `Comment only`, and `Dead code`, use `id: "not-in-code"`, `kind: "multi"`, and `required: false`. Prompt: | ||
|
|
||
| > Pick flags not evaluated in this repository to cull. PostHog has these flags, but nothing in this repository evaluates them. Only this repository was scanned, so a flag read by another service, a mobile app, or a bulk fetch elsewhere will look unreferenced here. Culling disables the flag and removes the comment or deletes the dead module when present. | ||
|
|
||
| Example description: `the only evaluation is in an unreachable module; deletes the module; src/legacy-checkout.ts:1; disables the flag`. | ||
|
|
||
| 5. Always include `id: "mode"`, `kind: "single"`, last. Prompt verbatim: | ||
|
|
||
| > Cull the flags you picked, or report only and change nothing? | ||
|
|
||
| Options, in this order: | ||
| - label `Report only, change nothing`, value `report-only` | ||
| - label `Cull the flags I picked`, value `cull` | ||
|
|
||
| For every `multi` question, keep rows in ledger order and do not preselect an option. Build every option as follows: | ||
|
|
||
| - `label`: the flag key. | ||
| - `value`: the flag key. | ||
| - `description`: one line containing, in order, the seeded rollout summary clause from `details`; the code outcome; every call site as `path:line`, starting with `file` and then each `also <path:line>` entry from `details`; and the PostHog outcome. | ||
|
|
||
| Translate `winning branch: true` to `keeps the code that runs today` and `winning branch: false` to `keeps the off branch`. Use `removes the mention` for `Comment only` and `deletes the module` for `Dead code`. Omit the code outcome for `Unreferenced`. Use `disables the flag` for `Rolled out`, `Off for everyone`, `Unreferenced`, `Comment only`, and `Dead code`. Use `no PostHog change` for `Archived in PostHog`, `Disabled in PostHog`, and `Deleted in PostHog`. Omit any empty call-site fragment. | ||
|
|
||
| Approved rows are the union of the four `multi` answers only when `mode` is `cull`. Nothing is approved when `mode` is `report-only`, the union is empty, or the call errors or is cancelled with Esc. Do not retry the call. | ||
|
|
||
| Resolve every collected row not approved to `status: "pass"` with `details` = seeded details plus `; declined by user`. | ||
|
|
||
| When nothing is approved, skip the Cull section and follow the Output instructions. | ||
|
|
||
| ## Cull | ||
|
|
||
| ### Edit the code | ||
|
|
||
| For each approved row, in ledger order: | ||
|
|
||
| 1. Emit: | ||
|
|
||
| ``` | ||
| [STATUS] Culling <key> | ||
| ``` | ||
|
|
||
| 2. `Unreferenced` has no call site: skip the rest of this list, the row goes straight to the verify and disable passes. `Comment only` removes the mention only. | ||
| 3. `Read` the call site named in `file` and every extra site listed in `details`. | ||
| 4. Emit `[STATUS] Editing <file>` before each file edit. | ||
| 5. Keep the winning branch recorded in `details` (`winning branch: true` keeps the code that ran when the flag was on; `false` keeps the code that ran when it was off). Delete the other branch, the flag call, and any import or hook that is now unused. | ||
| 6. `Dead code`: delete the unreachable file instead of editing it. | ||
| 7. `Deleted in PostHog`: code edit only, there is no flag to disable. | ||
|
|
||
| A failed code edit resolves the row to `status: "error"` with `details` = the seeded details plus `; failed: <one-line reason>`. That row is finished: it never reaches the disable pass, and PostHog is never touched for it. | ||
|
|
||
| ### Verify the edited files | ||
|
|
||
| Once per run, after every approved row has been edited: | ||
|
|
||
| 1. Emit `[STATUS] Type checking <n> files`. | ||
| 2. `Read` `package.json` for the project's lint and typecheck scripts. Run them only on the files edited in this session, never across the whole project. Capture stdout and stderr; truncate long output to the failure region. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this case, this isn't quite true actually. Broken imports can happen cross package, for verification like type checks, definitely run the whole project |
||
| 3. Re-run after each fix until clean. Fix only cull-induced failures, prioritizing files you edited. | ||
|
|
||
| A row whose file still fails resolves to `error` in the format above and never reaches the disable pass. | ||
|
|
||
| ### Disable and resolve | ||
|
|
||
| Before the loop below, run `exec({ "command": "search feature-flag" })` once, pick the tool whose description says it disables a flag, then run `exec({ "command": "info <tool_name>" })` once. | ||
|
|
||
| For each approved row whose code edit and file verification passed, in ledger order: | ||
|
|
||
| 1. `Deleted in PostHog` has no flag, and `Archived in PostHog` and `Disabled in PostHog` are already off: these rows need no PostHog call, go straight to step 4. | ||
| 2. Emit `[STATUS] Disabling <key> in PostHog`, then run only `exec({ "command": "call <tool_name> <json> })` with the flag key or id from `details`. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. wait, the local changes haven't been deployed yet? This would break production, no? |
||
| 3. Never call a delete or archive tool. | ||
| 4. Resolve the row: `status: "pass"`, `details` = seeded details plus `; culled`. Anything that failed in this pass resolves to `error` in the format above instead. | ||
|
|
||
| ## Output | ||
|
|
||
| Every `warning` row outside `Many call sites` is now `pass` or `error`. `Read` the ledger once more and count from it, not from memory: culled = rows whose `details` end with `; culled`, failed = rows with `status: "error"`, left for you = rows whose `details` end with `; declined by user`. Emit: | ||
|
|
||
| ``` | ||
| [STATUS] Culled <a> flags, <f> failed, <d> left for you | ||
| ``` | ||
|
|
||
| When nothing was approved, emit this once instead and continue to the report step: | ||
|
|
||
| ``` | ||
| [STATUS] Report only, nothing changed | ||
| ``` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand the intent, but disabling greps outright probably:
I think we need to restrict a slightly different way