feat(agents): add the error-tracking orchestrator flow - #377
Draft
ablaszkiewicz wants to merge 11 commits into
Draft
feat(agents): add the error-tracking orchestrator flow#377ablaszkiewicz wants to merge 11 commits into
ablaszkiewicz wants to merge 11 commits into
Conversation
Nine prompts under context/agents/error-tracking/ backing the wizard's new `error-tracking` command. The seed establishes whether PostHog is integrated and which uploader variant applies, then queues install/init (integration-v2 step-skills, like replay-vision) on uninstrumented repos, exception capture via integration-v2-error-tracking-step, the source-map subgraph (adapted from the error-tracking-upload-source-maps flow) where the platform ships minified bundles or stripped binaries, and a report that merges it all. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a wizard branch:
Add Results will be posted here when complete. |
…ness install/init move terra→sol low, capture-exceptions terra low→sol medium. Pairs with the wizard binding change routing the program to the orchestrator on pi; seed stays terra and report stays luna, matching the metrics flow's pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…nager A configure run in a pnpm workspace copied the docs' npm install and npm hard-fails on workspace:* deps (EUNSUPPORTEDPROTOCOL); the agent retried npm three ways and never switched. New skill bullet: detect the manager from the lockfile, translate the docs' npm commands, and read EUNSUPPORTEDPROTOCOL as wrong-manager, never a flag to retry. The error-tracking flow's configure task now also mandates detect_package_manager before the first install. Verified: the pnpm monorepo fixture run that failed now completes 4/4, recovering to `pnpm add -D` on first contact with the error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…anning A seed run saw "PostHog already integrated" and enqueued only capture+report, never applying the uploader precedence to a tsc-built Node service — while an identical sibling fixture planned the full graph. The two facts are now declared independent, a compiled/bundled JS project is explicitly never "none", and the success criteria demand the plan state the uploader decision (or the readable-stack reason) explicitly. Verified: the fixture that under-planned now queues the full node-uploader graph and completes 5/5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The integration init skill documents its public keys in .env.example; the credentials task wrote its upload variables only to the real env file, so the names never reached the committed template. New step 4: append the names with placeholders (never values) — for the next developer, and so a later run's check_env_keys reports the expectation. Scoped to the error-tracking flow's task prompt; the shared uploader skill is untouched, so the standalone upload-source-maps program keeps its current behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
Astro inlines scripts below its asset limit into the HTML, so a build can emit a .map with no .js beside it. posthog-cli then finds zero pairs, exits non-zero, and takes the whole build down with it. Treat Astro as a no-variant platform: install and exception capture still run, the build script is left untouched, and the report says why upload was skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…Node The shared integration-v2-init skill assumes the framework auto-loads .env, which is true for Next/Nuxt/Astro/SvelteKit/Vite but not for plain Node backends. On Express, Fastify, Hono, Koa and raw node:http the run wrote process.env reads and a .env file with nothing to bridge them, so every app died at module load on its own missing-variable guard. Teach the init task to wire the loading (dotenv, or --env-file) and to start the app once to confirm it boots. Also have configure check that some script actually runs a newly emitted bundle, since two apps built to dist/ while start still ran the source. Scoped to this flow's task prompts; the shared skill is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…n Angular On the stock @angular/build builder nothing defines process.env or import.meta.env in the browser bundle, but the run wrote import.meta.env['NG_APP_POSTHOG_PROJECT_TOKEN'] into src/environments/*. Reading a property off undefined throws while the module evaluates, so <app-root> stayed empty and the app rendered a blank page. Name the mechanism's real source (@ngx-env/builder supplies import.meta.env and the NG_APP_ prefix) and require it to be an existing dependency before use. Otherwise fall back to the skill's existing "no valid environment to read from" case: embed the public project token in the committed environment files, which is where it ends up in the bundle anyway. Scoped to this flow's task prompt; the shared skill is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The earlier rule stopped the run reaching for import.meta.env, but it then read env['NG_APP_POSTHOG_PROJECT_TOKEN'] from the project's generated src/environments/.env.ts, which only defines npm_package_version. The key came back undefined, and because a production build takes the guard's quiet return path the app rendered normally while PostHog never initialised — no requests at all. Name that second empty container, require the lookup's source to be opened and the key seen before wiring it, and make the task prove initialisation by serving the build and confirming a request reaches the PostHog host. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
Pointing `start` at dist/index.js is only safe if that bundle can execute. On koa-notes the esbuild command had no --format=esm while package.json sets "type": "module", so the emitted CommonJS died at boot with `ReferenceError: module is not defined in ES module scope` — a breakage that only appeared once `start` stopped running the source. Require the task to run the build and start the script once, and name the module-format mismatch as the usual cause. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…uild rule The new "run the build and start that script once" instruction sat directly above "Do not run the build", leaving the task with contradictory orders. Carve the single verification out of the prohibition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The wizard is gaining a one-command error-tracking setup (
wizard error-tracking, PostHog/wizard#1185) that installs PostHog first when the repo doesn't have it — the replay-vision pattern. The orchestrator flow backing it needs to live here.Changes
New flow
context/agents/error-tracking/— nine prompts:setup-error-tracking.md(seed) establishes two facts: is PostHog integrated, and which source-map uploader variant applies (or none, on readable-stack-trace platforms). It then queues one graph:install+init(only when PostHog is missing) →capture-exceptions→ the upload subgraph (credentialsas a root so the API-key ask reaches the user early,configureafter the code edits,wire-ci,test-setup) →report.install/initreuse theintegration-v2step-skills, near-verbatim from replay-vision's.capture-exceptionsuses the existingintegration-v2-error-tracking-stepbundle, adapted from integration-v2's task (verification is the user-driven test-setup step here, not a build task).configure/credentials/wire-ci/test-setupare adapted from theerror-tracking-upload-source-mapsflow, secret-vault handling included; the seed passes{ skillId, displayName }inputs so no task re-detects.reportmerges the handoffs, enables the Error Tracking product viaposthog_exec(non-fatal on failure), and writesposthog-error-tracking-report.md.Companion PRs: PostHog/wizard#1185 (program + wiring), wizard-workbench (picker registration).
Test plan
node scripts/build.js— the flow validates and all nine prompts land indist/agents/+agent-menu.json.pnpm test— 173/173.🤖 Generated with Claude Code
https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY