Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
dbbb8ca
feat(agents): add the error-tracking orchestrator flow
ablaszkiewicz Sep 1, 2026
644a7a7
feat(agents): run the error-tracking code tasks on sol for the pi har…
ablaszkiewicz Sep 1, 2026
870f697
fix(skills): make source-map installs follow the project's package ma…
ablaszkiewicz Sep 2, 2026
c59bda3
fix(agents): make the error-tracking seed settle both facts before pl…
ablaszkiewicz Sep 2, 2026
8be48c8
feat(agents): document upload variable names in .env.example
ablaszkiewicz Sep 3, 2026
2b067cc
feat(agents): skip the source-map uploader on Astro
ablaszkiewicz Sep 3, 2026
64955b1
fix(agents): make the error-tracking init wire .env loading on plain …
ablaszkiewicz Sep 8, 2026
31cc726
fix(agents): stop the error-tracking init inventing import.meta.env o…
ablaszkiewicz Sep 8, 2026
5d77c07
fix(agents): require the Angular init key to resolve, not just exist
ablaszkiewicz Sep 8, 2026
e79c007
fix(agents): make configure prove the built bundle actually runs
ablaszkiewicz Sep 8, 2026
9b14ab1
fix(agents): reconcile the configure build-verification with the no-b…
ablaszkiewicz Sep 8, 2026
6223e9a
fix(agents): drop the build/run verification steps from error-tracking
ablaszkiewicz Sep 9, 2026
413c0ae
fix(agents): make the env variable names an explicit contract
ablaszkiewicz Sep 9, 2026
12bc01b
feat(error-tracking): make init check the variable an existing init r…
ablaszkiewicz Sep 9, 2026
4de201c
feat(error-tracking): treat a keyless init as not integrated
ablaszkiewicz Sep 9, 2026
7a4f082
feat(error-tracking): keep the type assertion in a typed build config
ablaszkiewicz Sep 9, 2026
779b999
feat(error-tracking): forward the release identity across the CI boun…
ablaszkiewicz Sep 9, 2026
9983432
feat(error-tracking): match the app's own env names at the deploy bou…
ablaszkiewicz Sep 9, 2026
c9f89e5
fix(error-tracking): forward CI git vars only where a provider sets them
ablaszkiewicz Sep 9, 2026
010ff33
feat(error-tracking): make the run script load the env file it depend…
ablaszkiewicz Sep 10, 2026
ee6f8f2
feat(error-tracking): give the capture step its own docs matrix
ablaszkiewicz Sep 10, 2026
a98931c
fix(error-tracking): stop the capture task splicing two SDK setups
ablaszkiewicz Sep 10, 2026
567bd04
fix(error-tracking): default Nuxt to the 3.7+ capture docs
ablaszkiewicz Sep 11, 2026
a146290
fix(error-tracking): keep the capture task off the SDK's own listeners
ablaszkiewicz Sep 13, 2026
3927aba
test(skills): check error-tracking-step covers every integration variant
ablaszkiewicz Sep 14, 2026
1700aae
fix(agents): use allowed wizard models in the error-tracking flow
ablaszkiewicz Sep 14, 2026
488e759
fix(agents): pick the react-native uploader before native markers
ablaszkiewicz Sep 14, 2026
136a9dd
fix(agents): make the error-tracking init type-check
ablaszkiewicz Sep 14, 2026
811a5e9
fix(skills): do not set a devtool beside the PostHog webpack plugin
ablaszkiewicz Sep 14, 2026
843bd1e
fix(agents): stop turning on Error Tracking from the report task
ablaszkiewicz Sep 14, 2026
476c40f
fix(agents): pick the Android uploader only for Android Gradle projects
ablaszkiewicz Sep 14, 2026
e98bb0e
chore(agents): run error-tracking install and init at medium effort
ablaszkiewicz Sep 15, 2026
fa69873
fix(agents): write the error-tracking report before the optional test
ablaszkiewicz Sep 15, 2026
0886e3b
fix(agents): carry a missing posthog-cli into the error-tracking report
ablaszkiewicz Sep 15, 2026
ba65456
fix(warlock): triage through the gateway's ci product
ablaszkiewicz Sep 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions context/agents/error-tracking/capture-exceptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
type: capture-exceptions
flow: error-tracking
label: Wire up exception capture
model_pi: openai/gpt-5.6-sol
effort_pi: medium
model_sdk: claude-sonnet-5
effort_sdk: high
skills: [integration-v2-error-tracking-step, posthog-best-practices]
allowedTools: [Read, Write, Edit, Glob, Grep]
disallowedTools: [enqueue_task]
dependsOn: [install, init]
---

## Goal

Make the errors the app does not catch reach PostHog, by whatever means the
SDK offers for that. Which means depends on the SDK: some autocapture
exceptions once you enable it at init, some wire into the framework's own
error handler, some give you a boundary to mount at the app entry. Follow the
docs and the reference example for this one, and set it up in one place —
never manual capture calls sprinkled across files.

A page often shows two setups side by side: a global initializer, and an
explicit client you hold. Follow one of them the whole way through. Taking the
wiring from one and the calls from the other reads fine and compiles nowhere —
an initializer that hands back nothing leaves you nothing to call methods on.

The SDK is installed and initialized — either it already was, or the install
and init tasks before you did it (see their handoffs); build on that, do not
re-check it.

This is an instrument-only task. Do not install dependencies, run the build,
run tests, or start the app — the user-driven test-setup step at the end of
the flow verifies, when the user wants it. Do not touch the build config
either way; when the flow includes a configure task, it owns those files.
Stay inside this project's directory and set up that one place; that is the
whole job.

## Do not copy the SDK's own listeners

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be very fragile. I think this info should be maintained in docs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So our docs already explain how our auto capture works. But then agent during implementation really tried to to its best job and sometimes started hooking into things our auto capture already hooks into.

I don't think we should write in every technology doc "do not hook into x.y, it's already covered by auto capture". List here is not strict and doesn't really need to be updated. It's more of a general guidance to an agent so it can distinguish auto capture vs userful framework hooks


Exception autocapture already registers the global handlers: `window.onerror`
and `unhandledrejection` in the browser, `uncaughtException` and
`unhandledRejection` in Node, `sys.excepthook` in Python, the panic hook in
Rust, and the uncaught-exception and signal handlers on iOS and Android. Turn
that on with the SDK's own option — `capture_exceptions`,
`enableExceptionAutocapture`, `enable_exception_autocapture`, `capture_panics`,
`errorTrackingConfig.autoCapture` — and register none of those handlers
yourself. A second listener on the same event sends every error twice, and it
stops matching the SDK's handling the moment the SDK changes.

What the SDK cannot see is yours to add: errors a framework catches before any
global handler fires. Express error middleware, Fastify `setErrorHandler`, Hono
`onError`, Vue `app.config.errorHandler`, Angular `ErrorHandler`, SvelteKit
`handleError`, a React error boundary, Next.js `global-error` — hook those,
because the framework swallows the error and the global listener never hears
of it. A platform with no autocapture at all, such as Go, is the other case:
there the capture boundary at the entry point is the mechanism, not a copy of
one.

## How you know you succeeded

An error the app does not catch reaches PostHog, through the mechanism this
SDK gives you rather than one you invented, and no global error listener of
your own sits beside the SDK's autocapture. You did not install anything, run
a build, lint, or tests, search outside the project, or read through the whole
app or hand-wrap individual components or routes. Your handoff names the files
you changed and the capture mechanism, so the report can explain it to the
user.
113 changes: 113 additions & 0 deletions context/agents/error-tracking/configure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
type: configure
flow: error-tracking
label: Apply build-config changes
model_pi: openai/gpt-5.6-sol
effort_pi: medium
model_sdk: claude-sonnet-5
effort_sdk: medium
skills: []
Comment thread
ablaszkiewicz marked this conversation as resolved.
allowedTools: [Read, Write, Edit, Glob, Grep, Bash, load_skill_menu, install_skill, check_env_keys]
disallowedTools: [enqueue_task]
dependsOn: [capture-exceptions]
---

## Goal

Make this project's production build emit and upload source maps (or, for Go
and Rust, native debug symbols). Install the skill your task input names
(`install_skill` with the `skillId`) and read it — it is the source of truth
for the per-framework build-config and the uploader wiring.

Two of the skill's steps are yours:

- **"Apply build-config changes"** — make the bundler / build-config edits the
skill instructs for this platform, so the build produces and injects the
chunk IDs PostHog needs and runs the uploader.
- **"Make credentials available at build time"** — do the skill's step so the
build can read the upload credentials from the environment. If it calls for a
loader (e.g. `dotenv`), install it SILENTLY with the project's package
manager. Skip this step entirely when the platform already auto-loads `.env`.

Install every dependency with the project's own package manager: call
`detect_package_manager` before the first install and use its answer,
translating any `npm install` the skill or docs show (`pnpm add -D …`,
`yarn add -D …`). In a pnpm or yarn workspace, npm fails outright on
`workspace:*` dependencies (`EUNSUPPORTEDPROTOCOL`) — that error means the
wrong manager, never a flag to retry with.

When the upload step you wire runs a machine-global `posthog-cli` rather than
one from the project's own dependencies, check that it is on the `PATH`
(`command -v posthog-cli`). Do not try to install it: a global install is
blocked for you, and the wizard already tried before your task. When it is
missing, name it in your handoff as a manual follow-up — run
`npm install -g @posthog/cli@latest` before the next release build — because
without it the upload step cannot run.

When your changes make the build emit a bundle to a new directory (`dist/`,
`build/`), check that some script actually runs that output. A project whose
`build` writes `dist/index.js` while `start` still runs the original source
never executes the bundle the maps were uploaded for, so every uploaded map
goes unused. Add or fix the script that serves the built output.

That run script has its own environment to satisfy, and it is not the build's.
The uploader's credentials are handled — your build step passes them itself. The
separate question is the variables **the application reads when it starts**: the
project token and host its init looks up. Those live in the same gitignored env
file, and a compiled binary or a bare interpreter loads nothing on its own.

So open the init point, note the variables it reads, and make the run script
provide exactly those — export the env file ahead of the command, pass the
runtime's own flag for it, whatever that platform offers. Skip it and the
artifact starts, prints its own "variable missing" guard, and reports nothing:
the build is green, the symbols are uploaded, and the single command you hand
the user to verify with is the one command that cannot capture.

Put a multi-step build somewhere the tool actually runs it — a script file, a
make target, the manifest's own scripts — never an alias mechanism you are
assuming exists, borrowed from a neighbouring tool.

Match the bundle's module format to the package's type while you write the
command — you cannot run it, so it has to be right by construction. `esbuild
--platform=node` emits CommonJS unless you pass `--format=esm`, so in a package
whose `package.json` sets `"type": "module"` the bundle dies at boot with
`ReferenceError: module is not defined in ES module scope`. Read the `type`
field before writing the command.

Types are the same kind of trap. In a config the compiler checks — a `.ts`
config, or JS under `checkJs` — `process.env.ANYTHING` is `string | undefined`,
while a plugin's options usually require `string`. Where your skill's example
asserts or defaults that lookup, keep that part exactly: dropping a `!` or a
`?? ''` turns a working example into a build that fails type checking. The
upload step often runs before the type check, so the maps land and the build
still exits non-zero — a broken build that looks half-successful in the log.

## The names are a contract

An env variable only works if the name the code reads is the name in the file.
You write the code that reads them; the `credentials` task writes the file, and
you two run in parallel. So never invent a name that already exists somewhere
else — look for the other half of the contract first, and adopt it:

- **Call `check_env_keys` before you write the config.** It returns names, never
values. If the PostHog upload variables are already there, make your config
read exactly those names, whatever they are, even when they are not the ones
your skill's example shows.
- **Only when they are absent yet** are you the one deciding. Use the names your
skill documents for the mechanism you are wiring, and name them in your
handoff in full so `credentials` and `wire-ci` can match them.

Names that merely look plausible are the failure here. A build reading
`POSTHOG_API_KEY` beside an env file holding `POSTHOG_CLI_API_KEY` throws no
error anywhere — the upload is skipped silently, the build looks clean, and
every stack trace stays minified.

Do not write any credential values and do not create env files — the
`credentials` task owns that, in parallel with you. Do not run the build.

## How you know you succeeded

The build config carries the skill's source-map / debug-symbol changes and can
read its credentials from the environment at build time. Your handoff names
every file you changed and the exact build-config keys you added, so the CI
task can wire the same variables through the pipeline.
80 changes: 80 additions & 0 deletions context/agents/error-tracking/credentials.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
type: credentials
flow: error-tracking
label: Get and write the upload credentials
model_pi: openai/gpt-5.6-sol
effort_pi: medium
model_sdk: claude-sonnet-5
effort_sdk: medium
skills: []
allowedTools: [Read, Write, Edit, Glob, Grep, Bash, load_skill_menu, install_skill, wizard_ask]
disallowedTools: [enqueue_task]
dependsOn: []
---

## Goal

Put the PostHog source-map upload credentials into this project's environment.
Install the skill your task input names (`install_skill` with the `skillId`)
and follow its **"Write credentials to the env file"** step for the variable
names and the env file to pick.

The upload needs a PostHog **personal API key** at build time. Only the user can
mint one — never call the PostHog API or any tool to create it. Get it and
write it in this one task (the key never survives across tasks):

1. Ask with `wizard_ask`, exactly:
`{ id: "api-key", prompt: "Paste your PostHog personal API key below.\n\nDon't have one yet? Create one here:\n<SETTINGS_URL>\n\nWhen creating the key, choose the 'Source map upload' preset, then come back and paste it here.", kind: "text", sensitive: true }`
You receive `{ secretRef: "secret:..." }` — a vaulted reference, never the raw
value. If `wizard_ask` is unavailable (non-interactive run), report this task
with status `not needed` and say in your handoff that the user must create
the key and set the variables themselves; do not block.
2. Pick the env file per the skill (reuse the one PostHog's SDK already writes
its `POSTHOG_*` vars to, when there is one). Call `check_env_keys` on it
first (it returns present/absent, never values — never read the file
directly).
3. Settle the variable names before you write them — see "The names are a
contract" below. Then call `set_env_values`, passing the secretRef as a
value object, not a literal string — e.g.
`values: { "POSTHOG_CLI_API_KEY": { secretRef: "<the ref>" }, "POSTHOG_CLI_PROJECT_ID": "<PROJECT_ID>", "POSTHOG_CLI_HOST": "<UI_HOST>" }`
with whichever names you settled on. The wizard resolves the ref locally, so
you never see the key value.
4. Document the same variable names for other developers: append them to
`.env.example` (create it if the project has none) with empty or
placeholder values — never a real value, and never the key itself. The
example file is committed and is the only `.env*` you may write directly;
it is how the next developer, and the next wizard run's `check_env_keys`,
learns the project expects these variables.

Replace `<SETTINGS_URL>`, `<PROJECT_ID>`, and `<UI_HOST>` from your project
context. Do not touch the build config — the `configure` task owns that.

## The names are a contract

An env variable only works if the name in the file is the name the code reads.
You write the file; the `configure` task writes the code that reads it, and you
two run in parallel. So never invent a name that already exists somewhere else
— look for the other half of the contract first, and adopt it:

- **Read the build config before you choose.** If it already references PostHog
env variables, use exactly those names, whatever they are, even when they are
not the ones your skill would suggest. Reading config is safe — it holds
names, not secrets.
- **Only when nothing references them yet** are you the one deciding. Use the
names your skill documents for the mechanism that was actually wired.
- **Either way, say which set you chose in your handoff, in full.** That is how
`configure` and `wire-ci` learn what to match. A handoff that says "the usual
variables" hands the next task the same guess you just made.

Names that merely look plausible are the failure here. A build reading
`POSTHOG_API_KEY` beside an env file holding `POSTHOG_CLI_API_KEY` throws no
error anywhere — the upload simply never runs, and every stack trace stays
minified.

## How you know you succeeded

The env file holds the upload variables (the key as a resolved secret, the
non-secret project id and host as literals), written through the wizard tools,
never hardcoded in source, and `.env.example` documents the same names with
placeholders. Your handoff names the env file and every variable name — never
a value — so the CI task carries the same names into the pipeline.
117 changes: 117 additions & 0 deletions context/agents/error-tracking/init.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
type: init
flow: error-tracking
label: Set up PostHog initialization
model_pi: openai/gpt-5.6-terra
Comment thread
ablaszkiewicz marked this conversation as resolved.
effort_pi: medium
model_sdk: claude-sonnet-5
effort_sdk: medium
skills: [integration-v2-init, posthog-best-practices]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's fucking go @edwinyjlim @sarahxsanders @daniloc Our dream of orchestrated, reusable skills is coming alive!

allowedTools: [Read, Write, Edit, Glob, Grep, Bash, check_env_keys, set_env_values]
disallowedTools: [enqueue_task]
dependsOn: []
---

## Goal

Make sure PostHog is initialized. If the project already has a working
`posthog.init(...)` (or the framework's equivalent) with its env keys wired,
leave it alone and say so in your handoff. If it doesn't, create it following
your skill — it owns the how: the framework's init point, the env-var wiring
through the wizard tools, and `.env.example`.

You exist in this flow because the user asked for error tracking on a repo
whose PostHog init is missing or unproven. Initialize the SDK so exceptions can
flow and stop — no instrumentation, no extras. Don't set up exception capture
either way; the capture-exceptions task after you owns that.

## An existing init still needs its variable defined

"Already initialised" is a property of the pair, not of the call. An init point
that reads `process.env.SOMETHING` only works when `SOMETHING` is defined in the
env file the project loads. A repo can carry an init that has never once run:
the call is there, the variable it names is nowhere, and the client is built
from an empty string.

So before you leave an existing init alone, call `check_env_keys` on that env
file and look for the exact name the init reads. It returns names, never values.

- **Present** — the pair is complete. Leave it alone and say so.
- **Absent** — the init is not wired yet, whoever wrote it. Write that variable
with `set_env_values` under the name the code already reads, and document it
in `.env.example`. Do not rename the code to match a name you would rather
have written; the code is the half that already exists.

An empty key is the quiet failure here. A client constructed from `''` throws
nothing and logs nothing. The build is clean, the app starts, every capture call
returns — and no event ever arrives.

## Make the environment actually reachable

Writing the keys to `.env` is only half the job — something has to load that
file at runtime, or the app throws on boot and captures nothing.

Most frameworks do it for you: Next, Nuxt, Astro and SvelteKit auto-load `.env`,
and Vite auto-loads it for client code. Nothing to do there.

A plain Node backend does not — Express, Fastify, Hono, Koa, a raw `node:http`
server — and neither does a bare Rollup or webpack config. When your init point
reads `process.env` on one of those, wire the loading too, either way:

- install `dotenv` with the project's own package manager (detect it from the
lockfile) and import it above the PostHog init — `require('dotenv').config()`,
or `import 'dotenv/config'` for ESM; or
- add `--env-file=.env` to the `start` and `dev` scripts, when the project is on
Node 20.6+ and would rather not take a new dependency.

You cannot run the app, so this has to be right by construction. Writing the
guard without the loader is what produces
`POSTHOG_… variable required by PostHog is missing or un-configured` at module
load: the guard you wrote firing against an env file nothing reads.

Some platforms have no environment to read at all, and there the answer is not
a loader. Angular on the stock `@angular/build` builder is the common one:
nothing populates `process.env`, `import.meta.env`, or the project's own
`src/environments/.env.ts` with your keys.

The trap is that each of those *looks* like a mechanism. `import.meta.env` and

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, but I'd move all the .env handling and init out to the shared skill. Also that if a .env is not reachable, don't fail the run report it clearly in the handoff

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest a separate PR as it affects other flows? You mean to do it such that replay vision also uses that?

the `NG_APP_` prefix only exist with `@ngx-env/builder` installed, and a
generated `.env.ts` usually carries one unrelated key such as
`npm_package_version` and nothing else. Read from either and your key is
`undefined`: in development the guard throws while the module evaluates and the
app renders a blank page, and in a production build the guard returns quietly,
so the app looks fine while PostHog never initialises at all.

So do not wire a lookup unless you have opened the thing it reads from and seen
your key defined there. When nothing populates it, write the real public project
token as a literal in the committed `src/environments/*` files. This is the
skill's "no valid environment to read from" case, and the public token is
publishable — it ships inside the browser bundle either way.

## The init has to type-check

You cannot run the build either. So in a project the compiler checks —
TypeScript, or JavaScript under `checkJs` — the init has to compile by
construction. `process.env.ANYTHING` is `string | undefined`, and the SDK
constructor wants a `string`.

Pass the variable that you checked. `if (!key) { … } else { new PostHog(key, …) }`
narrows `key` to `string`. A check on a different variable does not: store the
missing name in `missing`, test `missing`, then pass `key`, and the compiler
still sees `string | undefined`. The build then stops with "Argument of type
'string | undefined' is not assignable to parameter of type 'string'", emits
no bundle, and uploads no source maps. When the existing code already passes
`process.env.KEY ?? ''` or `process.env.KEY!`, keep that part as it is.

## How you know you succeeded

An init point exists with the PostHog env keys present — whether it already
did or you just created it — keys in the env file and confirmed there with
`check_env_keys`, never hardcoded. On a platform that does not auto-load
`.env`, the loading is wired; on a platform with no environment at all, the
token is a literal rather than a lookup into something that never defines it.
In a type-checked project, the value you pass to the SDK is narrowed to a
`string`.
Your handoff names the files involved, how the client is constructed, and how
`.env` reaches it, so the capture-exceptions task can find the init options
without re-discovering them.
Loading
Loading