feat(error-tracking): add the error-tracking program on the orchestrator flow - #1185
Draft
ablaszkiewicz wants to merge 5 commits into
Draft
feat(error-tracking): add the error-tracking program on the orchestrator flow#1185ablaszkiewicz wants to merge 5 commits into
ablaszkiewicz wants to merge 5 commits into
Conversation
…tor flow New `wizard error-tracking` command. Modeled on replay-vision: a detect step carries the framework id into orchestrator preflight, the context-mill `error-tracking` flow seeds install/init (integration-v2 step-skills) when the repo has no PostHog, then exception capture, then the source-map subgraph where the platform needs it. The detect step also pre-installs posthog-cli for symbol-upload platforms, which the agent cannot (warlock blocks npm install -g). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a Context Mill branch:
Add Results will be posted here when complete. |
This was referenced Sep 1, 2026
Binding harness anthropic→pi, like metrics: the binding routes only, and every stage's model and effort are pinned context-mill side in the flow's frontmatter (terra seed, sol tasks, luna report). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
e2e.json flow definition + profile registration: the driver confirms the intro, answers the flow's api-key ask from SOURCE_MAPS_CLI_KEY (secret rule, vaulted by the ask tool) and declines the test-affordance offer. Two variations: the pinned orchestrator-on-pi binding and the linear fallback. Unlocks snapshots mode and the workbench e2e runner for the command. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The screen was registered in the action registry (satisfying the exhaustiveness test) but decideE2eAction keeps its own intro switch, and an unlisted screen falls through to wait — the driver sat on the intro until the host died. Adds the missing case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The bin resolves --local-* flags in its yargs middleware into the local-dev singleton that getSkillsBaseUrl() reads. The tui-host bypasses yargs, so its env-backed local flags reached the session but never the singleton — agent prompts always resolved to production, and an unpublished flow died in preflight with "No seed agent prompt". Initialize the singleton from the same env spellings before the store starts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Setting up PostHog error tracking today takes multiple commands: the default integration flow, then
wizard upload-source-maps— and the source-maps command aborts on repos without PostHog instead of installing it. Replay-vision already solved this shape (install PostHog as step 0 via the orchestrator seed); error tracking deserves the same one-command experience.Changes
New
wizard error-trackingcommand, modeled on replay-vision:src/lib/programs/error-tracking/): detect step puts the framework id onsession.skillIdfor orchestrator preflight,agentFlow: 'error-tracking'pinned,ciPreRunfor headless runs. No platform allow-list — every detectable framework has anintegration-v2-error-tracking-stepvariant.posthog-clifor symbol-upload platforms (swift, android, react-native, flutter, go, rust) — the agent can't, warlock blocksnpm install -g. Warn-don't-fail, mirroring the source-maps program.error_tracking:read,product_enablement:write), custom intro screen (no single skillId exists for the generic intro), command wrapper +bin.ts, e2e action registry entry.context/agents/error-tracking/): seed checks whether PostHog is integrated and which uploader variant applies, then queues install/init → capture-exceptions → the source-map subgraph (credentials/configure/wire-ci/test-setup) → report. Companion PRs in context-mill and wizard-workbench.wizard upload-source-mapsstays as-is — it remains a valid standalone entry point.Test plan
pnpm build && pnpm test && pnpm lint— 2586/2586 tests pass, including newerror-tracking.test.tsand the updated lockstep guards (switchboard bindings, flags isolation, e2e action registry exhaustiveness).--local-context-millvia the workbench stack); not yet run against a live app.Tests
Per framework: run the wizard against a PostHog-less app, add a button that
captures an exception, build and serve, click it in a real browser, then read the
issue back from PostHog. A trace that resolves to original sources means
source-map upload worked. Project: Test (Oli).
✅ worked end to end ·⚠️ worked, but something needed fixing first · ⛔ could not be verified
✅ no-posthog-angular/angular-saas
no-posthog-angular/angular-saas— Angular 20 standalone on the stock@angular/buildbuilder. The run populatedsrc/environments/.env.tswith the public token and host, so theenvironment.tslookup resolves; upload via@posthog/clichained intobuild, credentials in.envasPOSTHOG_CLI_*matching theposthog-cli --dotenv-file .envstep, releaseangular-boilerplate@6d56bfe.TypeError: Cannot read properties of undefined (reading 'retries') — minified
i/br/vrresolved back toKaboomButtonComponent.handleClick/kaboom/readRetryBudget, with the Angular runtime frames resolved to their realplatform-browserandrender3paths.Build output
Stack trace
✅ no-posthog-javascript-node/express-todo
no-posthog-javascript-node/express-todo— Express 4 on Node,posthog-nodewithsetupExpressErrorHandler,dotenvloaded above the init, esbuild bundle, upload viaposthog-clichained intobuild, releaseexpress-todo.npm startruns the built bundle. Server-side, so the error is triggered byGET /kaboom.TypeError: Cannot read properties of undefined (reading 'retries') —
library: posthog-node, resolved through the esbuild bundle back toreadRetryBudget->kaboominindex.js.Build output
Stack trace
✅ no-posthog-javascript-node/fastify-blog
no-posthog-javascript-node/fastify-blog— Fastify on Node (ESM),posthog-nodeviafastify.setErrorHandler,dotenv/configimported above the init, esbuild bundle, upload viaposthog-clichained intobuild, releasefastify-blog.npm startruns the built bundle. Triggered byGET /kaboom(port 3001).TypeError: Cannot read properties of undefined (reading 'retries') —
library: posthog-node, resolved through the esbuild bundle toreadRetryBudget->kaboominindex.js.Build output
Stack trace
✅ no-posthog-javascript-node/hono-links
no-posthog-javascript-node/hono-links— Hono on Node,posthog-nodeviaapp.onErrorwith an explicitawait posthog.flush(),dotenv/configimported above the init, esbuild bundle, upload viaposthog-cli, releasehono-links.npm startruns the built bundle. Triggered byGET /kaboom(port 3002).TypeError: Cannot read properties of undefined (reading 'retries') — resolved through the esbuild bundle to
readRetryBudget->kaboominindex.js.Build output
Stack trace
✅ no-posthog-javascript-node/koa-notes
no-posthog-javascript-node/koa-notes— Koa + koa-router,posthog-nodein a dedicatedposthog.jsmodule wired throughapp.on('error'),dotenv/configimported above the init. The run appended itsposthog-clistep to the project's existing esbuild build and leftstart:productionpointing atdist/index.js; releasekoa-notes. Triggered byGET /kaboom(port 3003).TypeError: Cannot read properties of undefined (reading 'retries') — resolved
readRetryBudget->kaboominindex.js.Build output
Stack trace
✅ no-posthog-javascript-node/native-http-contacts
no-posthog-javascript-node/native-http-contacts— rawnode:httpserver,posthog-nodewithcaptureException+await flush()in the handler's catch,dotenv/configimported above the init. The run appended itsposthog-clistep to the project's existing esbuild build and leftstart:productionpointing atdist/index.js; releasenative-http-contacts. Triggered byGET /kaboom(port 3004).TypeError: Cannot read properties of undefined (reading 'retries') — resolved
readRetryBudget->kaboom->serverinindex.js.Build output
Stack trace
✅ no-posthog-javascript-web/saas-dashboard
no-posthog-javascript-web/saas-dashboard— plain browser JS on Vite,posthog-js+@posthog/rollup-plugin, releasewizard-e2e-saas-dashboard@2a836b3.TypeError: Cannot read properties of undefined (reading 'retries') — resolved
readRetryBudget->kaboom-> click handler insrc/kaboom-button.js.Build output
Stack trace
✅ no-posthog-next-js/15-app-router-saas
no-posthog-next-js/15-app-router-saas— Next 15 App Router SaaS starter (Postgres/Stripe),@posthog/nextjs-config, 57 chunks uploaded across both build passes.Notably this run wrote the cleanest
next.config.tsof the Next apps: it hoists the env vars into consts and guardswithPostHogConfigbehindposthogApiKey && posthogProjectId, which narrows the types naturally — no!assertion needed, and PostHog config is simply absent when unconfigured.TypeError: Cannot read properties of undefined (reading 'retries') — resolved to
components/debug/kaboom-button.tsx:14.Unrelated to the wizard: the build needs
POSTGRES_URL(the fixture is a Postgres/Stripe starter), so a placeholder value was set to get pastCollecting page data.Build output
Stack trace
✅ no-posthog-next-js/15-app-router-todo
no-posthog-next-js/15-app-router-todo— Next 15.5.7,@posthog/nextjs-config.TypeError: Cannot read properties of undefined (reading 'retries') — resolved to
components/debug/kaboom-button.tsx:14.Build output
Stack trace
✅ no-posthog-next-js/15-pages-router-saas
no-posthog-next-js/15-pages-router-saas— Next 15 Pages Router SaaS starter (Postgres/Stripe),@posthog/nextjs-config, 52 chunks uploaded across both build passes. Same guardednext.config.tsshape as the App Router SaaS app — env vars hoisted into consts andwithPostHogConfigapplied only when both are set, so no!assertion.TypeError: Cannot read properties of undefined (reading 'retries') — resolved to
components/debug/kaboom-button.tsx:14.Unrelated to the wizard: needs
POSTGRES_URLforCollecting page data, so a placeholder was set.Build output
Stack trace
✅ no-posthog-next-js/15-pages-router-todo
no-posthog-next-js/15-pages-router-todo— Next 15 Pages Router,@posthog/nextjs-config. Built clean, 21 chunks uploaded across the two build passes.TypeError: Cannot read properties of undefined (reading 'retries') — resolved to
components/debug/kaboom-button.tsx:14. As with the App Router app, React collapses the inner frames so only the click handler survives.Build output
Stack trace
✅ no-posthog-nuxt/movies-nuxt-3-6
no-posthog-nuxt/movies-nuxt-3-6— Nuxt 3,plugins/posthog.client.tswith avue:errorhook, upload via anitro:build:public-assetshook runningposthog-cli, releasewizard-e2e-movies-nuxt-3-6@a1def4b, 22 chunks uploaded.TypeError: Cannot read properties of undefined (reading 'retries') — resolved the full chain
readRetryBudget->kaboom->handleClickincomponents/KaboomButton.vue. (Groups with the Vue issue: same error and file name.)Build output
Stack trace
✅ no-posthog-nuxt/movies-nuxt-4
no-posthog-nuxt/movies-nuxt-4— Nuxt 4 via the@posthog/nuxtmodule (posthogConfigwithpublicKey,hostand asourcemapsblock) plus aplugins/posthog.client.ts;posthog-jsandposthog-nodeadded. The only app in the set that gets the Nuxt module rather than aposthog-clibuild step. 38 chunks uploaded.TypeError: Cannot read properties of undefined (reading 'retries') — minified
s/r/nresolved back tohandleClick/kaboom/readRetryBudget.Build output
Stack trace
✅ no-posthog-react-router/react-router-v7-project
no-posthog-react-router/react-router-v7-project— React Router v7 SSR,@posthog/reactprovider,@posthog/rollup-plugin. Unlikeshopper, this run set noreleaseName, so the plugin's neither-field path applied and the build was clean either way.TypeError: Cannot read properties of undefined (reading 'retries') — minified names
o/E/Aresolved back tohandleClick/kaboom/readRetryBudget. (Groups with theshopperissue: same error and file name.)Build output
Stack trace
✅ no-posthog-react-router/rrv7-starter
no-posthog-react-router/rrv7-starter— React Router v7 SSR starter (GSAP route transitions, customstartscript),@posthog/react,@posthog/rollup-plugin, noreleaseNameset.TypeError: Cannot read properties of undefined (reading 'retries') — minified
t/Vv/Xvresolved back tohandleClick/kaboom/readRetryBudget. (Groups with the other React Router runs: same error and file name.)Build output
Stack trace
✅ no-posthog-react-router/saas-template
no-posthog-react-router/saas-template— React Router v7 SSR SaaS template (Prisma/Supabase/Stripe, zod-validated env),@posthog/react,@posthog/rollup-plugin.TypeError: Cannot read properties of undefined (reading 'retries') — minified
e/we/xeresolved back tohandleClick/kaboom/readRetryBudget.Unrelated to the wizard, this fixture needs real setup before it builds or boots:
prisma generate(which itself needsDATABASE_URL) to produce~/generated/browser, then twelve zod-validated variables (APP_URL,COOKIE_SECRET, Supabase, Stripe, storage). Placeholders were supplied for all of them.Build output
Stack trace
✅ no-posthog-react-router/shopper
no-posthog-react-router/shopper— React Router v7 SSR,@posthog/reactprovider +PostHogErrorBoundary,@posthog/rollup-plugin.TypeError: Cannot read properties of undefined (reading 'retries') — resolved the full chain
readRetryBudget->kaboom->handleClickinapp/components/KaboomButton.tsx.Build output
Stack trace
✅ no-posthog-sveltekit/CMSaasStarter
no-posthog-sveltekit/CMSaasStarter— SvelteKit (Supabase/Stripe),posthog-jsviasrc/hooks.client.ts(init+handleError),posthog-nodeinsrc/lib/server/posthog.ts,@posthog/rollup-plugin; 151 chunks uploaded across the client and server passes.TypeError: Cannot read properties of undefined (reading 'retries') — minified
d/e/presolved back tohandleClick/kaboom/readRetryBudget.The build needs placeholder Supabase/Stripe values, since
$env/static/publicresolves at build time.Build output
Stack trace
✅ no-posthog-vue/movies
no-posthog-vue/movies— Vue 3 + Vite,@posthog/rollup-plugin, capture viaapp.config.errorHandler.TypeError: Cannot read properties of undefined (reading 'retries') — resolved the full call chain
readRetryBudget->kaboom->handleClickinsrc/components/KaboomButton.vue.Build output
Stack trace
✅ node-webpack
node-webpack— webpack 5 + ts-loader,posthog-nodealready in the manifest but its key defined nowhere. The run wired@posthog/webpack-pluginwithdevtool: 'hidden-source-map'anddeleteAfterUpload, and wrote the runtime token asPOSTHOG_PROJECT_TOKENbeside the uploader'sPOSTHOG_API_KEY/POSTHOG_PROJECT_ID; releasewizard-e2e-node-webpack. Triggered bynode dist/index.js.TypeError: Cannot read properties of undefined (reading 'retries') — minified
lt/ctresolved toreadRetryBudget/kaboomin./src/index.ts.Build output
Stack trace
✅ node-rollup
node-rollup— Rollup 4, no TS plugin,posthog-nodealready in the manifest but its key defined nowhere. The run wired@posthog/rollup-pluginwithdeleteAfterUploadbeside the existingsourcemap: true, and wrote the runtime token asPOSTHOG_PROJECT_TOKENbeside the uploader'sPOSTHOG_API_KEY/POSTHOG_PROJECT_ID; releasewizard-e2e-node-rollup. Triggered bynode dist/index.js.TypeError: Cannot read properties of undefined (reading 'retries') — resolved
readRetryBudget->kaboomin../src/index.ts, with source context lines.Build output
Stack trace
✅ node-rollup-typescript-plugin
node-rollup-typescript-plugin— Rollup 4 +@rollup/plugin-typescript,posthog-nodealready in the manifest but its key defined nowhere. The run wired@posthog/rollup-pluginafter the TypeScript plugin withdeleteAfterUpload, added astartscript for the bundle, and wrote the runtime token asPOSTHOG_PROJECT_TOKENbeside the uploader'sPOSTHOG_API_KEY/POSTHOG_PROJECT_ID; releasewizard-e2e-node-rollup-typescript-plugin. Triggered bynpm run start.TypeError: Cannot read properties of undefined (reading 'tsRetryBudget') — compiled
dist/index.jsframes resolved back to the.tssource, with context lines.Build output
Stack trace
✅ node-raw
node-raw— plaintscbuild,posthog-nodealready in the manifest but its key defined nowhere, whilestartalready rannode --env-file=.env. The run installed@posthog/cliand appended aposthog-cli --dotenv-file .env sourcemap processstep tobuild, wrote the uploader'sPOSTHOG_CLI_API_KEY/POSTHOG_CLI_PROJECT_ID/POSTHOG_CLI_HOSTto match that flag, and the runtime token asPOSTHOG_PROJECT_TOKEN; releasenode-raw. Triggered bynpm run start.TypeError: Cannot read properties of undefined (reading 'rawRetryBudget') — compiled
dist/index.jsframes resolved back to the.tssource, with context lines.Build output
Stack trace
✅ next
next— Next.js 16 on Turbopack,posthog-jsalready in the manifest but its key defined nowhere. The run wired@posthog/nextjs-configviawithPostHogConfigwithdeleteAfterUpload, and wroteNEXT_PUBLIC_POSTHOG_KEY/_HOSTbeside the uploader'sPOSTHOG_API_KEY/POSTHOG_PROJECT_IDin.env.local; releasewizard-e2e-next. Clicked in Chromium againstnext start.TypeError: Cannot read properties of undefined (reading 'nextRetryBudget') — minified
o/iresolved toreadNextRetryBudget/kaboomNextinapp/KaboomButton.tsx.Build output
Stack trace
✅ nuxt-3-6
nuxt-3-6— Nuxt 3.6,posthog-jsalready in the manifest andruntimeConfigfalling back to a literalphc_placeholder, so nothing threw while events went nowhere. The run replaced that fallback with a realNUXT_PUBLIC_POSTHOG_KEY, addedsourcemap.clientand anitro:build:public-assetshook runningposthog-cli sourcemap inject/upload --delete-after, with the uploader'sPOSTHOG_CLI_*in.env; releasewizard-e2e-nuxt-3-6. Clicked in Chromium againstnuxt preview.TypeError: Cannot read properties of undefined (reading 'nuxt36RetryBudget') — minified
s/rresolved toreadNuxt36RetryBudget/kaboomNuxt36incomponents/KaboomButton.vue.Build output
Stack trace
✅ react-vite
react-vite— Vite 7 + React,posthog-jsalready in the manifest but its key defined nowhere. The run addedbuild.sourcemap: 'hidden', wired@posthog/rollup-pluginthroughloadEnvwithdeleteAfterUpload, and wroteVITE_PUBLIC_POSTHOG_KEY/_HOSTbeside the uploader'sPOSTHOG_API_KEY/POSTHOG_PROJECT_ID; releasewizard-e2e-react-vite. Clicked in Chromium againstvite preview.TypeError: Cannot read properties of undefined (reading 'viteRetryBudget') — one minified
assets/index-DM2dbKgY.jsline resolved to the exact lines ofsrc/KaboomButton.tsx. Rollup's minified map carries nonamesentry for the two helpers, so those frames keep their mangled_c/gclabels while file and line are exact.Build output
Stack trace
✅ nuxt-4-3
nuxt-4-3— Nuxt 4.3,posthog-jsalready in the manifest, a client plugin readingruntimeConfig.public.posthogKey, and noruntimeConfigblock defining it. The run took the@posthog/nuxtmodule path:sourcemap.client: 'hidden',sourcemapExcludeSources: false, aposthogConfig.sourcemapsblock, andNUXT_PUBLIC_POSTHOG_KEY/_HOSTbeside the uploader'sPOSTHOG_API_KEY/POSTHOG_PROJECT_ID; releasewizard-e2e-nuxt-4-3. Clicked in Chromium againstnuxt preview.TypeError: Cannot read properties of undefined (reading 'nuxt43RetryBudget') — minified
s/rresolved toreadNuxt43RetryBudget/kaboomNuxt43inapp/components/KaboomButton.vue.The module and the pre-existing client plugin both call
posthog.init, so the page logsYou have already initialized PostHog! Re-initializing is a no-op. Capture works; the plugin's owncapture_exceptionsoptions are the part that gets dropped.Build output
Stack trace
✅ python-flask
python-flask— Flask 3, no PostHog at all. The seed classified it as a readable-stack platform and queued no uploader subgraph: four tasks, nocredentials/configure/wire-ci. The run installedposthogandpython-dotenv, wiredload_dotenv()with aPosthog(...)client underenable_exception_autocapture, added an@app.errorhandler(Exception)that re-raisesHTTPExceptionand callscapture_exceptionfor the rest, and wrotePOSTHOG_PROJECT_TOKEN/POSTHOG_HOST. Triggered byGET /kaboom(port 8000).TypeError: 'NoneType' object is not subscriptable — frames arrived readable with no symbol set, which is the point of the skip.
Task graph (no uploader subgraph)
Stack trace
✅ no-posthog-django/django3-saas
no-posthog-django/django3-saas— Django 4.2 SaaS app, no PostHog at all. The seed classified it as a readable-stack platform and queued no uploader subgraph: four tasks, nocredentials/configure/wire-ci. The run addedposthogtorequirements.txt, registeredposthog.integrations.django.PosthogContextMiddleware, built the process-wide client inaccounts/apps.pyunderAppConfig.ready()withenable_exception_autocaptureand handed it to the middleware throughsettings.POSTHOG_MW_CLIENT, and reused the project's existingPOSTHOG_PROJECT_TOKEN/POSTHOG_HOST. Triggered byGET /kaboom/withDEBUG=False(port 8001).TypeError: 'NoneType' object is not subscriptable — frames arrived readable with no symbol set, which is the point of the skip.
requirements.txtends up with twoposthoglines —posthog>=6.0.0andposthog==6.9.3. pip resolves both to 6.9.3, so the install works.Task graph (no uploader subgraph)
Stack trace
LLM context
Built with Claude Code; flow design mirrors
replay-vision(wizard) anderror-tracking-upload-source-maps(context-mill agents).🤖 Generated with Claude Code
https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY