Repository navigation
feat(wizard-ci): read the e2e gateway key from POSTHOG_GATEWAY_TOKEN - #4192
Merged
Merged
Conversation
Local e2e runs needed a minted phe_ gateway token in a file, which meant a browser login, a weekly mint quota and a 24h expiry. A project secret key (phs_ with the llm_gateway:read scope) works against the gateway directly, so it can live in .env or be injected with `op run`. The runner prefers POSTHOG_GATEWAY_TOKEN, writes it to a 0600 file in a temporary directory for the harness (which only reads WIZARD_CI_GATEWAY_TOKEN_FILE), keeps the variable out of the child env, and removes the directory after the run or on SIGINT/SIGTERM. CI still sets only the file variable, so its path is unchanged. Constraint: the wizard harness reads the gateway credential only from a file Rejected: calling `op read` from the runner | CI has no op CLI, and `op run` needs no code Directive: keep POSTHOG_GATEWAY_TOKEN ahead of the file variable so a stale .env file entry never wins Confidence: high Scope-risk: narrow
johncwaters
marked this pull request as ready for review
September 24, 2026 20:55
gewenyu99
reviewed
Sep 24, 2026
|
|
||
| # AI gateway key for model calls: a phs_ project secret key with the llm_gateway:read scope. | ||
| # With `op run --env-file .env -- pnpm wizard-ci ...` this can be an op:// reference instead of the key. | ||
| POSTHOG_GATEWAY_TOKEN=phs_... |
Contributor
There was a problem hiding this comment.
Suggested change
| POSTHOG_GATEWAY_TOKEN=phs_... | |
| POSTHOG_GATEWAY_TOKEN=op://... |
Contributor
There was a problem hiding this comment.
Let's just defualt to the good path kek
gewenyu99
approved these changes
Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Feedback wanted: ready to merge. Local-only change; the CI path (
WIZARD_CI_GATEWAY_TOKEN_FILE) is unchanged.Running
wizard-ci --e2elocally needed a mintedphe_gateway token in a file: a browser login, a weekly mint quota, a 24h expiry, and the mint refuses programs it doesn't know yet (program_unknownforfeature-flags). A project secret key (phs_with the admin-grantedllm_gateway:readscope) works against the gateway directly and bills its own team, while the run and its MCP actions still target the project you pick.No token-issuing or mint code: the runner only forwards a key you already hold, and a key spends only its own team's gateway wallet, so this opens no new inference path.
Changes
services/wizard-ci/e2e.ts: readsPOSTHOG_GATEWAY_TOKENfirst (the name pr-evaluator already uses), then falls back toWIZARD_CI_GATEWAY_TOKEN_FILE, so a stale file entry in.envnever wins. The harness only reads a file, so the key goes into a 0600 file in amkdtempdirectory. The variable is kept out of the child env, and the directory is removed after the run or on SIGINT/SIGTERM. Fails early with a clear message when neither is set..env.example: documentsPOSTHOG_GATEWAY_TOKEN, plusop run --env-file .env -- pnpm wizard-ci ...for keeping anop://reference in.envinstead of the key.How did you test this code?
All on the team test project,
feature-flags/next-js/15-app-router-saas --e2e, wizardb70460a8with no local gateway allowlist:WIZARD_CI_GATEWAY_TOKEN_FILEpointing at the service key: pass (all 12 feature-flag checks)POSTHOG_GATEWAY_TOKENviaop runwith anop://reference: pass, temp directory removedPOSTHOG_GATEWAY_TOKENviaop runwithWIZARD_CI_GATEWAY_TOKEN_FILE=/nonexistent/...: pass, which shows the env var takes precedencetsc --noEmitclean