feat(feature-flags): add the wizard feature-flags command - #1322
Open
johncwaters wants to merge 7 commits into
Open
johncwaters wants to merge 7 commits into
johncwaters wants to merge 7 commits into
Conversation
This was referenced Sep 23, 2026
Merged
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a Context Mill branch:
Add Results will be posted here when complete. |
johncwaters
force-pushed
the
johnw/feature-flags-command
branch
from
September 23, 2026 22:24
adab746 to
f3eb515
Compare
johncwaters
force-pushed
the
johnw/feature-flags-command
branch
2 times, most recently
from
September 24, 2026 18:55
34213e2 to
b70460a
Compare
johncwaters
marked this pull request as ready for review
September 24, 2026 20:38
Adds `wizard feature-flags`: an orchestrator flow with bundled prompts that creates one example flag per side of the app (backend, frontend), inactive at 0%, evaluates each once on its own side, and writes a report with an optional check that the evaluations reach PostHog.
The feature-flags program gets its own intro screen, a run-screen learn deck explaining what flags are and what the run sets up, and sidebar tips, in place of the generic agent-skill screens.
Give feature-flags its own OAuth scope list and share the step skill id between the evaluate prompt and the intro screen.
A Django run whose templates load no app JavaScript added a PostHog snippet to host the frontend flag. The seed now defines the frontend by JavaScript the app owns and loads, including through import maps or Vite, and names script-less server templates as backend-only.
johncwaters
force-pushed
the
johnw/feature-flags-command
branch
from
September 25, 2026 15:07
b70460a to
b977652
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Feedback wanted: polish. Feature complete, with unit tests and workbench e2e grading. The companion context-mill and wizard-workbench PRs are merged.
agentPromptsreaches the orchestrator through the run config, the same wayagentFlowdoes. Does that seam still fit the program host in feat(programs): B2 surface shell — runProgram types and stub signatures #1307, or should it move there when that lands?Setting up a first feature flag means creating it in PostHog, installing the right SDKs, and evaluating it correctly: once per request, with the identified distinct id, the key in one place, and the evaluation reported back. The wizard has no command for this.
Companion PRs (merged): context-mill (PostHog/context-mill#405) and wizard-workbench (PostHog/wizard-workbench#4136).
Changes
wizard feature-flags: a native command running an orchestrator flow with bundled prompts (seed, install, init, create-flag, evaluate, report).sidesinput. A side counts only when the app already has code there: the frontend is JavaScript the app owns and loads (inline, as files, or through import maps or Vite), so server-rendered templates that load none are backend-only.wizard-example-backend-flagandwizard-example-frontend-flag, inactive at 0% rollout. An existing flag with the same key is reused unchanged.integration-v2-feature-flags-stepskill from context-mill for the framework's posthog.com flag docs; the instructions stay in the prompt.posthog-feature-flags-report.md, then offers oneverify-flagcheck for recent$feature_flag_calledevents.feature-flags-introscreen, a run-screen learn deck explaining what flags are (gating code, gradual rollout, targeting, the off switch, experiments) and what the run sets up, and sidebar tips on payloads, local evaluation, and experiments. The intro offers only Cancel while detection runs, when no framework is detected, or when the framework version is unsupported, so a run never reaches login without a supported framework.agentPrompts), and the command gets its own OAuth scope list (feature_flag:read,feature_flag:write), registry entry, and e2e profile.Before merge:
feature-flagsmust be added toWIZARD_GATEWAY_PROGRAM_IDS(PostHog/charts#16288, held in draft so the program cannot mint before it ships), and a gateway operator must add awizard:feature-flagsbudget row. Until then the gateway refuses the mint withprogram_unknown. Two e2e runs cost $1.05 and $0.93, under every posture cap, so the program needs no cap entry of its own.Test plan
pnpm typecheckandpnpm testpass.cli.test.tsandhealth-checks.test.tstime out when the machine is loaded and pass when run alone; this PR does not touch them.--e2eagainst a test project, starting with no example flags. Next.js passed with both flags created inactive at 0% and each evaluated once on its own side. Django passed with only the backend flag, because its templates contain no scripts. Each run took about 5 minutes, using a gateway token minted locally. A Next.js rerun with the context-mill skill trimmed to docs only also passed. After tightening the side rule: three Django runs were backend-only, Next.js got both sides, and a Rails import-map app (workbenchrails/fizzy) got both sides, with the SDK gems in the Gemfile, the snippet in the layout, and no npm. After rebasing onto the refactor's agent layer, both apps passed again with the flags deleted first: Next.js created both flags, Django only the backend flag.pnpm wizard-ci feature-flags/<app> --e2epassed every check:next-js/15-app-router-saascreated both flags inactive at 0% with the keys inlib/posthog-flags.ts, anddjango/django3-saascreated only the backend flag with the key inconfig/posthog_feature_flags.py.LLM context
Written with Claude Code.