Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions context/skills/cull-feature-flags/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Cull stale feature flags. The wizard seeds the ledger deterministically
# (scan + fetch + classify host-side); this skill verifies, asks once, applies,
# and reports. One variant per framework the wizard's scanner supports;
# `framework` is the detection id the wizard resolves the variant from.
# Reachable via `wizard cull-feature-flags` (wizard-registered flat command)
# and `wizard skill cull-feature-flags-<variant>`.
type: docs-only
template: description.md
description: Remove stale PostHog feature flags from a {display_name} codebase and disable them in PostHog, from a ledger the wizard seeds
tags: [feature-flags]
cli:
role: skill
references:
preamble: "**Read ONLY this file.** Do not read any other reference file until this one tells you to."
shared_docs:
- https://posthog.com/docs/feature-flags/best-practices.md
variants:
- id: nextjs
framework: nextjs
default: true
display_name: Next.js
tags: [nextjs, nextjs-feature-flags, react, javascript, javascript_web, javascript_node]
docs_urls:
- https://posthog.com/docs/feature-flags/installation/react.md
- https://posthog.com/docs/libraries/next-js.md
62 changes: 62 additions & 0 deletions context/skills/cull-feature-flags/description.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Cull stale PostHog feature flags in a {display_name} project

This skill removes feature flags that have outlived their purpose: rolled out to everyone, off for everyone, archived or deleted in PostHog but still checked in code, or defined in PostHog and never evaluated anywhere. The wizard already did the detection before you started: it scanned the source tree for flag calls, fetched the project's flags, classified every flag with plain rules, and wrote one row per flag into `.posthog-audit-checks.json` at the project root. That ledger is the ground truth. You never grep for flags, never re-classify a row, and never promote a healthy flag into a removal.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the intent, but disabling greps outright probably:

  • really messes with the ability for the agent to properly carry out the task to independently verify. In the wild, flags are weird and projects are weird
  • The agent will just use another tool call xD

I think we need to restrict a slightly different way


Your job is three steps: verify each proposed row at its call site, ask the user once which rows to apply, cull the approved ones (code first, PostHog second), then write the report.

**Start by reading `references/1-verify-call-sites.md`.** Do not Glob, ls, or find the skill directory. Do not read `2-apply.md` or `3-report.md` until the current step tells you to.

## State

| Lane | Ledger areas | What culling does |
|---|---|---|
| Decided in PostHog | `Rolled out` | Keeps the code that runs today, drops the check, and disables the flag. |
| Decided in PostHog | `Off for everyone` | Keeps the off branch, drops the check, and disables the flag. |
| Off in PostHog, still in code | `Archived in PostHog`, `Disabled in PostHog`, `Deleted in PostHog` | Keeps the off branch, drops the check, and makes no PostHog change. |
| In PostHog, not in code | `Unreferenced`, `Comment only`, `Dead code` | Disables the flag and removes the mention or deletes the dead module when present. |
| Nothing to cull | `Many call sites`, `Healthy` | Report only. |

- `.posthog-audit-checks.json` (project root): the ledger. `area` is one of: `Rolled out` (100% to everyone, no conditions), `Off for everyone` (0% everywhere, possibly rolled back), `Archived in PostHog`, `Disabled in PostHog`, `Unreferenced` (in PostHog, never evaluated in code), `Comment only` (key appears only in a comment or config string), `Dead code` (the only file evaluating it is unreachable), `Deleted in PostHog` (code evaluates a key PostHog does not have), `Many call sites` (three or more files evaluate it directly, suggestion only), `Healthy`. `Read` it once at the start of each step. Each row is `{ id, area, label, status, file?, details? }` where `id` is the flag key, `area` is the bucket the wizard assigned, `label` names the proposed action, `file` is the first call site as `path:line`, and `details` carries the rollout summary and every call site.
- Rows change only through `mcp__wizard-tools__audit_resolve_checks`. Never `Edit` or `Write` the ledger file. Never delete it.
- Row statuses this skill uses: `pending` (seeded, not yet verified), `warning` (verified, proposed to the user), `pass` (kept, left for you, or culled), `error` (cull failed, reason in `details`).

The wizard prompt tells you whether the repo uses bulk evaluation (`getAllFlags`) or dynamic flag keys. When it does, every `Unreferenced` or `Comment only` row needs a real check at the bulk or dynamic call site before it can be proposed.

## Status

Report progress with `[STATUS] <message>` lines. Each step lists the exact strings to emit. They are cheap; use them at every sub-step.

## Abort

Report unrecoverable preconditions with exactly one `[ABORT] <reason>` line and stop. The wizard terminates the run; do not halt yourself.

- `[ABORT] No flag cull ledger found` when `.posthog-audit-checks.json` does not exist (this skill needs `wizard cull-feature-flags` to seed it).
- `[ABORT] No PostHog feature flag usage found` when the ledger has zero rows.

## Rules

1. **Disable, never delete.** The only PostHog mutation this skill makes is disabling a flag. Archiving and deleting are for the user to do in the app.
2. **Code before PostHog.** For a row that needs both a code edit and a disable, the edit lands first and the disable only after the edit succeeded, so a failed edit never leaves a disabled flag behind live code.
3. **One consent call.** Exactly one `wizard_ask` for the whole run: one pick list per lane, then the report-only or cull choice last. Nothing is culled without it.
4. **Winning branch only.** Removing a flag check means keeping the branch the flag would resolve to and deleting the other one, plus the now-unused import or hook. Do not restructure beyond that.
5. **Downgrade freely, never upgrade.** During verification a row may be resolved to `pass` (keep) with a reason. A `pass` row seeded as healthy is never touched.

## Available tools

{{> mcp-tool-calling}}

**Verify:** `Read` on each call-site file.

**Confirm:** `mcp__wizard-tools__wizard_ask`, called once (see `2-apply.md`).

**Apply:** `Edit` for code; through `exec`, the flag disable tool (discover it with `search feature-flag`, run `info` on it, then `call`).

**Ledger:** `mcp__wizard-tools__audit_resolve_checks` for every status change.

## Reference files

{references}

## Framework guidelines

{commandments}
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
next_step: 2-apply.md
---

# Step 1: Verify each proposed row at its call site

This step turns every `pending` ledger row into `warning` (verified, will be proposed) or `pass` (keep, with a reason). It does NOT ask the user anything and does NOT edit code or PostHog; that belongs to step 2.

## Status

Emit at the start:

```
[STATUS] Reading the flag cull ledger
```

## Read the ledger

`Read` `.posthog-audit-checks.json` at the project root.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmmm creative use of this ledger

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On purpose, the ledger already had the tui and the resolve tool. Open to a rename.


- File missing: emit `[ABORT] No flag cull ledger found` and stop.
- Zero rows: emit `[ABORT] No PostHog feature flag usage found` and stop.

Rows with `status: pass` were seeded healthy. Leave them alone for the whole run.

## Verify the pending rows

Emit:

```
[STATUS] Verifying <n> flag call sites
```

For each `pending` row, in ledger order, emit `[STATUS] Verifying <key>`. For `Rolled out` and `Off for everyone`, `Read` the call site named in `file` and every extra site listed in `details` as `also <path:line>`, around the given line. For `Archived in PostHog`, `Disabled in PostHog`, and `Deleted in PostHog`, `Read` only the first call site named in `file`. Emit `[STATUS] Reading <file>` before each call-site file `Read`. `Dead code` keeps its one `Grep` for the module's basename; do not read call-site files for that row. Decide one of:

| bucket (`area`) | confirm as `warning` when | downgrade to `pass` when |
|---|---|---|
| `Rolled out` | the call site is a boolean check whose true branch is the current behaviour | the flag gates something that must stay switchable (kill switch, ops toggle named as such) |
| `Off for everyone` | the call site is a boolean check whose false branch is the current behaviour | the feature is clearly mid-build (recent scaffolding, TODOs pointing at it), or the flag or its call site reads like a kill switch or rollback (name, comment, a recent change back to 0%) |
| `Archived in PostHog`, `Disabled in PostHog` | the call site is a boolean check; keep the false branch | never, these are always safe to propose |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never, these are always safe to propose
Not always true. Sometimes we temporarily disable a flag. I think premise is correct but wording can be tweaked

| `Deleted in PostHog` | no live flag in PostHog resembles the key | the key looks like a typo of a healthy flag key in the ledger (note the match in `details`) |
| `Unreferenced`, `Comment only` | the prompt says no bulk evaluation and no dynamic keys | the prompt reports `getAllFlags` or dynamic keys and the bulk or dynamic call site could reach this key |
| `Dead code` | nothing imports the file (the wizard checked; confirm with one `Grep` for the module's basename) | something imports it after all |
| `Many call sites` | always `warning`, it is a suggestion, not a removal | never |

**Truncated scan.** When the Scan facts say the scan was truncated, `Unreferenced` and `Comment only` are unproven: the wizard stopped before reading every file. Downgrade those rows to `pass` with `; kept: scan truncated, not proven unreferenced` so they land in the report's Kept table with that reason. Do not Grep for the key; the rule against grepping for flags holds.

Resolve each row through `mcp__wizard-tools__audit_resolve_checks` as soon as it is decided (one call per row, so the run screen moves while you work):

- confirmed: `status: "warning"`, `details` = the seeded details plus `; winning branch: true|false|n/a`
- downgraded: `status: "pass"`, `details` = the seeded details plus `; kept: <one-line reason>`

## Output

Every row that was `pending` is now `warning` or `pass`. No other file changed. Emit:

```
[STATUS] <w> flags proposed, <k> kept
```

Then emit:

```
[STATUS] Ready to ask
```
123 changes: 123 additions & 0 deletions context/skills/cull-feature-flags/references/2-apply.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
next_step: 3-report.md
---

# Step 2: Ask once, then cull the approved rows

This step asks the user which `warning` rows to apply, then applies them: code edit first, PostHog disable second. It does NOT write the report; that belongs to step 3. Step 1 already decided the winning branch for every row; do not re-verify.

## Ask

Emit:

```
[STATUS] Waiting for confirmation
```

`Read` `.posthog-audit-checks.json`. Collect every row with `status: "warning"` whose `area` is not `Many call sites` (that bucket is report-only).

Zero such rows: do not call `wizard_ask`; skip to the Output section, nothing to apply.

Call `mcp__wizard-tools__wizard_ask` exactly once. Build its questions in this order. Include a `multi` question only when its listed areas contain at least one collected row.

1. For `Rolled out`, use `id: "rolled-out"`, `kind: "multi"`, and `required: false`. Prompt:

> Pick rolled-out flags to cull. These flags are at 100% for everyone with no conditions, so the code only ever runs the "on" branch. Culling keeps that branch, drops the check, and disables the flag in PostHog. Re-enable it with one toggle; rollout conditions are kept.

Example description: `100% to everyone with no conditions; keeps the code that runs today; src/checkout.ts:42, src/cart.ts:18; disables the flag`.

2. For `Off for everyone`, use `id: "off-for-everyone"`, `kind: "multi"`, and `required: false`. Prompt:

> Pick flags that are off for everyone to cull. These flags are at 0% for everyone. A flag rolled back after an incident looks exactly like one that never shipped, so a 0% flag may be a rollback lever. Keeping it costs nothing. Culling keeps the "off" branch, drops the check, and disables the flag.

Example description: `0% to everyone after the checkout rollback; keeps the off branch; src/checkout.ts:42; disables the flag`.

3. For `Archived in PostHog`, `Disabled in PostHog`, and `Deleted in PostHog`, use `id: "off-in-posthog"`, `kind: "multi"`, and `required: false`. Prompt:

> Pick flags that are already off or absent in PostHog to cull. PostHog already turned these off or no longer has them, but the code still checks them. Culling keeps the "off" branch and drops the check. Nothing changes in PostHog for these flags.

Example description: `disabled in PostHog; keeps the off branch; src/search.ts:27; no PostHog change`.

4. For `Unreferenced`, `Comment only`, and `Dead code`, use `id: "not-in-code"`, `kind: "multi"`, and `required: false`. Prompt:

> Pick flags not evaluated in this repository to cull. PostHog has these flags, but nothing in this repository evaluates them. Only this repository was scanned, so a flag read by another service, a mobile app, or a bulk fetch elsewhere will look unreferenced here. Culling disables the flag and removes the comment or deletes the dead module when present.

Example description: `the only evaluation is in an unreachable module; deletes the module; src/legacy-checkout.ts:1; disables the flag`.

5. Always include `id: "mode"`, `kind: "single"`, last. Prompt verbatim:

> Cull the flags you picked, or report only and change nothing?

Options, in this order:
- label `Report only, change nothing`, value `report-only`
- label `Cull the flags I picked`, value `cull`

For every `multi` question, keep rows in ledger order and do not preselect an option. Build every option as follows:

- `label`: the flag key.
- `value`: the flag key.
- `description`: one line containing, in order, the seeded rollout summary clause from `details`; the code outcome; every call site as `path:line`, starting with `file` and then each `also <path:line>` entry from `details`; and the PostHog outcome.

Translate `winning branch: true` to `keeps the code that runs today` and `winning branch: false` to `keeps the off branch`. Use `removes the mention` for `Comment only` and `deletes the module` for `Dead code`. Omit the code outcome for `Unreferenced`. Use `disables the flag` for `Rolled out`, `Off for everyone`, `Unreferenced`, `Comment only`, and `Dead code`. Use `no PostHog change` for `Archived in PostHog`, `Disabled in PostHog`, and `Deleted in PostHog`. Omit any empty call-site fragment.

Approved rows are the union of the four `multi` answers only when `mode` is `cull`. Nothing is approved when `mode` is `report-only`, the union is empty, or the call errors or is cancelled with Esc. Do not retry the call.

Resolve every collected row not approved to `status: "pass"` with `details` = seeded details plus `; declined by user`.

When nothing is approved, skip the Cull section and follow the Output instructions.

## Cull

### Edit the code

For each approved row, in ledger order:

1. Emit:

```
[STATUS] Culling <key>
```

2. `Unreferenced` has no call site: skip the rest of this list, the row goes straight to the verify and disable passes. `Comment only` removes the mention only.
3. `Read` the call site named in `file` and every extra site listed in `details`.
4. Emit `[STATUS] Editing <file>` before each file edit.
5. Keep the winning branch recorded in `details` (`winning branch: true` keeps the code that ran when the flag was on; `false` keeps the code that ran when it was off). Delete the other branch, the flag call, and any import or hook that is now unused.
6. `Dead code`: delete the unreachable file instead of editing it.
7. `Deleted in PostHog`: code edit only, there is no flag to disable.

A failed code edit resolves the row to `status: "error"` with `details` = the seeded details plus `; failed: <one-line reason>`. That row is finished: it never reaches the disable pass, and PostHog is never touched for it.

### Verify the edited files

Once per run, after every approved row has been edited:

1. Emit `[STATUS] Type checking <n> files`.
2. `Read` `package.json` for the project's lint and typecheck scripts. Run them only on the files edited in this session, never across the whole project. Capture stdout and stderr; truncate long output to the failure region.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, this isn't quite true actually. Broken imports can happen cross package, for verification like type checks, definitely run the whole project

3. Re-run after each fix until clean. Fix only cull-induced failures, prioritizing files you edited.

A row whose file still fails resolves to `error` in the format above and never reaches the disable pass.

### Disable and resolve

Before the loop below, run `exec({ "command": "search feature-flag" })` once, pick the tool whose description says it disables a flag, then run `exec({ "command": "info <tool_name>" })` once.

For each approved row whose code edit and file verification passed, in ledger order:

1. `Deleted in PostHog` has no flag, and `Archived in PostHog` and `Disabled in PostHog` are already off: these rows need no PostHog call, go straight to step 4.
2. Emit `[STATUS] Disabling <key> in PostHog`, then run only `exec({ "command": "call <tool_name> <json> })` with the flag key or id from `details`.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait, the local changes haven't been deployed yet? This would break production, no?

3. Never call a delete or archive tool.
4. Resolve the row: `status: "pass"`, `details` = seeded details plus `; culled`. Anything that failed in this pass resolves to `error` in the format above instead.

## Output

Every `warning` row outside `Many call sites` is now `pass` or `error`. `Read` the ledger once more and count from it, not from memory: culled = rows whose `details` end with `; culled`, failed = rows with `status: "error"`, left for you = rows whose `details` end with `; declined by user`. Emit:

```
[STATUS] Culled <a> flags, <f> failed, <d> left for you
```

When nothing was approved, emit this once instead and continue to the report step:

```
[STATUS] Report only, nothing changed
```
Loading