Skip to content

feat(control): WIP serve a control API over a unix socket - #1277

Closed
gewenyu99 wants to merge 8 commits into
posthog/split-p3-movesfrom
posthog/split-p4-control
Closed

gewenyu99 wants to merge 8 commits into
posthog/split-p3-movesfrom
posthog/split-p4-control

Conversation

@gewenyu99

Copy link
Copy Markdown
Collaborator

Serve a control API over a unix socket behind --control-socket.

HTTP/1.1, JSON in and out: state with long polling, actions through store setters, credentials, run arming on the TUI surface, detection and independent runs with a ledger on the headless surface, shutdown. Published builds keep it for headless runs and refuse the flag on the TUI, whose bundle never contains the server; scripts/smoke-test.sh audits the chunks and CI diffs the chunk manifests against the committed fixtures. Stacked on #1276.


Created with PostHog Desktop

`--control-socket <path>` attaches an HTTP/1.1 server that reads one store
and commits through its setters: health, state with long polling, actions,
credentials, run arming on the TUI surface, detection and independent runs on
the headless surface, a run ledger, and shutdown. The store owns the server,
the projection, the generic actions, and the ledger; programs add their own
actions through FlowStep.controlActions; the cli supplies the hooks that
resolve credentials, run detection, and start one agent run with explicit
context. Published builds keep the server for headless runs and refuse the
flag on the TUI, which never loads it; the smoke test audits the built chunks
and CI diffs the chunk manifests against the committed fixtures. `--e2e-ask`
replaces the E2E_ASK environment read in dev builds.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

Each POST /runs clears the previous run's outcome, publishes to a task stream
session of its own, and records its task list in the ledger, so nothing from
one run leaks into the next while credentials and framework context persist.
The headless runner attaches no process-level stream when a socket is served.
scripts/controlled-headless-smoke.no-jest.ts drives the surface end to end.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
@gewenyu99

gewenyu99 commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator Author

Control API: why each piece exists

Top to bottom is the direction of a request. Every box says what it is for in running the TUI, the store, and the agent apart; every arrow names the contract the data crosses.

flowchart TB
  classDef wave_entry fill:#fff8e6,stroke:#d9a400,stroke-width:2px,color:#d9a400;
  classDef card_entry fill:#fff1c7,stroke:#d9a400,stroke-width:1px,color:#1d1d1f;
  classDef wave_cli fill:#fff1e8,stroke:#e8632c,stroke-width:2px,color:#e8632c;
  classDef card_cli fill:#ffe1cc,stroke:#e8632c,stroke-width:1px,color:#1d1d1f;
  classDef wave_tui fill:#e9f0ff,stroke:#2f6fed,stroke-width:2px,color:#2f6fed;
  classDef card_tui fill:#d6e4ff,stroke:#2f6fed,stroke-width:1px,color:#1d1d1f;
  classDef wave_agent fill:#f1e9ff,stroke:#7a3fe0,stroke-width:2px,color:#7a3fe0;
  classDef card_agent fill:#e6d8ff,stroke:#7a3fe0,stroke-width:1px,color:#1d1d1f;
  classDef wave_store fill:#e8f8ee,stroke:#1f9d55,stroke-width:2px,color:#1f9d55;
  classDef card_store fill:#d4f2df,stroke:#1f9d55,stroke-width:1px,color:#1d1d1f;
  classDef wave_control fill:#e4f5f3,stroke:#148f7a,stroke-width:2px,color:#148f7a;
  classDef card_control fill:#cfeeea,stroke:#148f7a,stroke-width:1px,color:#1d1d1f;
  classDef wave_harness fill:#f2f2f2,stroke:#6e6e73,stroke-width:2px,color:#6e6e73;
  classDef card_harness fill:#e6e6e6,stroke:#6e6e73,stroke-width:1px,color:#1d1d1f;
  classDef wave_guard fill:#fbe9e7,stroke:#c0392b,stroke-width:2px,color:#c0392b;
  classDef card_guard fill:#f8d7d2,stroke:#c0392b,stroke-width:1px,color:#1d1d1f;
  subgraph W1["PARENT: runs the wizard with no keyboard and no Ink of its own"]
    direction LR
    W1C1("<b>ControlClient / curl</b><br/>one typed call per route, so a parent<br/>never learns the store or the screens")
    W1C2("<b>snapshot route + MCP proxy</b><br/>CI and coding agents drive the real TUI<br/>through the socket, never its stdin")
    W1C1 ~~~ W1C2
  end
  class W1 wave_harness
  class W1C1,W1C2 card_harness
  subgraph W2["src/store/control: the store put on a socket, nothing more"]
    direction LR
    W2C1("<b>server.ts</b><br/>one route per store question or setter;<br/>hooks only where the store has no answer")
    W2C2("<b>state.ts</b><br/>projects the committed session whitelist<br/>so a parent sees what the TUI sees, no secrets")
    W2C3("<b>actions.ts</b><br/>each action is the setter the screen key<br/>handler calls: parity with the keyboard")
    W2C4("<b>runs.ts + driver.ts</b><br/>remembers each independent run and its end<br/>state; waits on the version instead of polling")
    W2C1 ~~~ W2C2 ~~~ W2C3 ~~~ W2C4
  end
  class W2 wave_control
  class W2C1,W2C2,W2C3,W2C4 card_control
  subgraph W3["src/store: one run's truth, the flow that chains runs, and what publishes them"]
    direction LR
    W3C1("<b>state/  RunStore</b><br/>the committed state of one agent run: its session<br/>copy, tasks, phase, outro, the questions it asks")
    W3C2("<b>state/  FlowStore</b><br/>chains runs on top: the flow, gates, interrupts, the<br/>session every run inherits; mirrors the active run")
    W3C3("<b>task-stream/</b><br/>store-level publisher: mirrors the run state<br/>outward to PostHog and files, renderer-blind")
    W3C4("<b>ui/  WizardUI + StoreUI</b><br/>the only way an agent writes: StoreUI turns<br/>its calls into store commits")
    W3C1 ~~~ W3C2 ~~~ W3C3 ~~~ W3C4
  end
  class W3 wave_store
  class W3C1,W3C2,W3C3,W3C4 card_store
  subgraph W4["src/cli: owns the store and composes runs above it"]
    direction LR
    W4C1("<b>control-hooks.ts</b><br/>does the parent's bidding: auth, detection,<br/>one scoped run at a time on a fresh RunStore, exit")
    W4C2("<b>runners/</b><br/>create the store, attach the server where a<br/>surface runs (TUI in dev builds, headless everywhere)")
    W4C3("<b>the TUI on top</b><br/>chains agent runs through FlowStore.startRun;<br/>UiStore holds presentation above, never inside a run")
    W4C1 ~~~ W4C2 ~~~ W4C3
  end
  class W4 wave_cli
  class W4C1,W4C2,W4C3 card_cli
  subgraph W5["src/agent: one run, no screen, no socket"]
    direction LR
    W5C1("<b>runAgent</b><br/>runs one program from its run config; writes<br/>back only through WizardUI, never to a store or screen directly")
  end
  class W5 wave_agent
  class W5C1 card_agent
  W1 -->|"HTTP/1.1 over a 0600 unix socket, JSON<br/>the whole parent contract: read state, commit one action, ask for what the store cannot do"| W2
  W2 -->|"reads ControlState · commits through exactly one setter · requestRun<br/>the mirror is the store itself; nothing is computed for the wire"| W3
  W3 -->|"ControlHooks: credentials, detection, one run, exit<br/>the parts above the store, answered by the layer that owns the store"| W4
  W4 -->|"RunAgent(runConfig, runSession, composed)<br/>one independent run: the agent sees its run config and session copy, nothing else"| W5
Loading

Interfaces, one sheet each

┌─ ControlState ───────────────────────────────────────────────────────────────────────────────┐
│ src/store/control/types.ts · GET /state, every POST reply, RunRecord.result                  │
│                                                                                              │
│ version: number                        currentScreen: string  (flow key or Interrupt)        │
│ session: ControlSession                tasks: TaskItem[]        statusMessages: string[]     │
│ eventPlan: PlannedEvent[]              handoffText: string | null                            │
│ setupQuestions: Omit<SetupQuestion, "detect">[]     actions: ActionView[]                    │
├──────────────────────────────────────────────────────────────────────────────────────────────┤
│ · a projection of one FlowStore: every field is store data, nothing is computed for the API  │
│ · ControlSession = Pick<WizardSession, CONTROL_SESSION_KEYS> & { hasCredentials, projectId } │
│ · frameworkContext redacted: secret:<id> values → [secret-ref]; key/token/secret words →     │
│   [redacted]                                                                                 │
│ · outroData.errorDetail passes through sanitizeErrorDetail; never accessToken, apiKey,       │
│   apiUser, answers                                                                           │
│ · version bumps on every commit; ?wait=&since= resolves on version > since, on wait ms, or   │
│   on close                                                                                   │
└──────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ DriverAction · ActionView ──────────────────────────────────────────────────────────────────┐
│ src/store/control/types.ts · POST /actions/:id  body { params }                              │
│                                                                                              │
│ id: string    description: string    params?: Record<name, hint>                             │
│ apply(store, params): void     ActionView = the same without apply                           │
├──────────────────────────────────────────────────────────────────────────────────────────────┤
│ · legal set = actionsFor(flow, currentScreen): the step's controlActions, then               │
│   GENERIC_ACTIONS                                                                            │
│ · apply calls exactly one store setter or resolver: the one the screen's key handler calls   │
│ · params are validated, never coerced: a wrong type or value is a 400 (BadParamError)        │
│ · the default intro's confirm_setup follows the scan-sharing toggle like Enter (share param) │
└──────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ ControlHooks ───────────────────────────────────────────────────────────────────────────────┐
│ src/store/control/types.ts · implemented once, in src/cli/control-hooks.ts                   │
│                                                                                              │
│ setCredentials(): Promise<void>               POST /credentials   (400 without an API key)   │
│ detect(req: DetectRequest): Promise<void>     POST /detect        (headless surface)         │
│ startRun(req: RunRequest): Promise<void>      POST /runs          (headless surface)         │
│ shutdown(): Promise<void>                     POST /shutdown                                 │
├──────────────────────────────────────────────────────────────────────────────────────────────┤
│ · only what the store cannot do itself: network auth, program detection, an agent run, exit  │
│ · POST /run is not a hook: it is store.requestRun(); run-wizard.ts waits on                  │
│   session.runRequested                                                                       │
│ · credentials, detect, and shutdown answer 409 while a run is in flight (ledger or runPhase) │
│ · startRun resolves when the run ends; the server then stores RunRecord.result = readState() │
└──────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ DetectRequest · RunRequest ─────────────────────────────────────────────────────────────────┐
│ request bodies, validated in server.ts                                                       │
│                                                                                              │
│ DetectRequest = { programId?: ProgramId } & Partial<Pick<WizardSession, "installDir">>       │
│ RunRequest    = { programId: ProgramId }                                                     │
│               & Partial<Pick<WizardSession, "installDir" | "frameworkContext" | "skillId">>  │
├──────────────────────────────────────────────────────────────────────────────────────────────┤
│ · programId checked against PROGRAM_REGISTRY, else 400; a non-object frameworkContext is a   │
│   400                                                                                        │
│ · installDir resolved once in the server (relative joins the live dir); ledger and hook see  │
│   the same path                                                                              │
│ · frameworkContext merges over the live one for that run only; the live session is never     │
│   rewritten                                                                                  │
│ · skillId precedence: request, program config, live session                                  │
└──────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ RunRecord ──────────────────────────────────────────────────────────────────────────────────┐
│ src/store/control/types.ts · GET /runs, POST /runs reply                                     │
│                                                                                              │
│ runId: uuid    programId: ProgramId    installDir: string    status: running | done | failed │
│ error: string | null    startedAt, finishedAt: ISO 8601    result: ControlState | null       │
├──────────────────────────────────────────────────────────────────────────────────────────────┤
│ · one record per POST /runs, in start order; a second POST while one is running → 409        │
│ · result is the state exactly as GET /state read it when the run ended (same shape, no new   │
│   type)                                                                                      │
│ · a hook that rejects records status failed, error = message, and the state at that moment   │
└──────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ RunStream ──────────────────────────────────────────────────────────────────────────────────┐
│ src/cli/control-hooks.ts · the task stream of one run, over its RunStore                     │
│                                                                                              │
│ attach(): void    shutdown(timeoutMs): Promise<void>    built by runStream(config, run:      │
│   RunStore)                                                                                  │
├──────────────────────────────────────────────────────────────────────────────────────────────┤
│ · startRun: store.startRun(runSession) → run.setRunPhase(Running) → attach → runAgent →      │
│   settle → runCleanups → shutdown                                                            │
│ · a new RunStore per run: tasks, status, plan, handoff, outro start empty; credentials,      │
│   context, and artefact urls are inherited                                                   │
│ · the stream is keyed by the run's skillId, not the launch program's                         │
└──────────────────────────────────────────────────────────────────────────────────────────────┘

How to run this step

Headless surface, every build (the key travels in the environment, an issued gateway bearer replaces the mint):

POSTHOG_WIZARD_API_KEY=phx_... WIZARD_CI_GATEWAY_TOKEN_FILE=/path/to/token.txt \
  npx tsx bin.ts --headless-DONOTUSE-EXPERIMENTAL --control-socket /tmp/w/w.sock \
  --project-id <id> --region us --install-dir /tmp/app
mkdir -p /tmp/w
curl -s --unix-socket /tmp/w/w.sock http://localhost/health
curl -s --unix-socket /tmp/w/w.sock -X POST -H 'content-type: application/json' -d '{}' http://localhost/detect
curl -s --unix-socket /tmp/w/w.sock -X POST -H 'content-type: application/json' \
  -d '{"programId":"posthog-integration"}' http://localhost/runs
curl -s --unix-socket /tmp/w/w.sock 'http://localhost/state?wait=60000&since=0' | jq '.state.session.runPhase, .state.tasks'
curl -s --unix-socket /tmp/w/w.sock http://localhost/runs
curl -s --unix-socket /tmp/w/w.sock -X POST http://localhost/shutdown

The same sequence, scripted, printing every request and a redacted view of every response (the first transcript below is its output):

APP_DIR=/tmp/app PROJECT_ID=<id> POSTHOG_REGION=us POSTHOG_KEY_FILE=/path/to/phx-key.txt \
  WIZARD_CI_GATEWAY_TOKEN_FILE=/path/to/token.txt \
  npx tsx scripts/controlled-headless-smoke.no-jest.ts posthog-integration metrics

Controlled TUI, dev builds only (the real screens in a terminal, a parent on the socket):

POSTHOG_WIZARD_API_KEY=phx_... npx tsx bin.ts --ci --control-socket /tmp/w/w.sock --project-id <id> --region us --install-dir /tmp/app
curl -s --unix-socket /tmp/w/w.sock http://localhost/state | jq '.state.currentScreen, [.state.actions[].id]'
curl -s --unix-socket /tmp/w/w.sock -X POST -H 'content-type: application/json' -d '{}' http://localhost/actions/confirm_setup
curl -s --unix-socket /tmp/w/w.sock -X POST http://localhost/credentials
curl -s --unix-socket /tmp/w/w.sock -X POST http://localhost/run

Published builds refuse --control-socket on the TUI: node dist/bin.js --control-socket /tmp/x --install-dir /tmp/app exits non-zero with "only available with the experimental headless flag". Tests: pnpm test:store (server, state, actions), pnpm test:cli (hooks, dispatch, refusal), pnpm test:arch, and pnpm build && pnpm build:ci for the chunk audit.

Evidence: controlled headless, basic integration then metrics, two independent runs over one socket

Run on 2026-09-19 against a throwaway copy of express-todo, project 228144, issued gateway bearer, no mint. Run again after the review fixes (409 fences, close semantics, param validation): both ledger records done with result.session.runPhase = completed; the integration run created the dashboard and notebook linked in the ledger; POST /shutdown exited 0 and removed the socket.

Transcript (long polls collapsed to the last line per run)
$ npx tsx bin.ts --headless-DONOTUSE-EXPERIMENTAL --control-socket /var/folders/kz/r8kl49v5683fdf4prxcylqqh0000gn/T/wizard-ctl-HXPLtq/w.sock --install-dir /tmp/wz-ctl-evidence-app --project-id 228144 --region us   # POSTHOG_WIZARD_API_KEY in env
GET /health -> {"ok":true,"version":"2.76.0","surface":"headless","pid":36904,"program":"posthog-integration"}
GET /state -> {"version":1,"screen":"intro","runPhase":"idle","integration":null,"detectionComplete":false,"hasCredentials":false,"tasks":[],"dashboardUrl":null,"notebookUrl":null,"outro":null}
POST /detect {} -> {"version":16,"screen":"intro","runPhase":"idle","integration":"javascript_node","detectionComplete":true,"hasCredentials":true,"tasks":[],"dashboardUrl":null,"notebookUrl":null,"outro":null}
POST /runs {"programId":"posthog-integration"} -> {"runId":"fc8dcba3-7324-41d2-bdbd-481a7de35fa8","programId":"posthog-integration","installDir":"/tmp/wz-ctl-evidence-app","status":"running","error":null,"startedAt":"2026-09-19T18:13:15.335Z","finishedAt":null,"result":null}
  GET /state?wait=60000&since=86 -> screen=intro runPhase=completed tasks=9/9 last=completed:Write setup report
  final -> {"version":86,"screen":"intro","runPhase":"completed","integration":"javascript_node","detectionComplete":true,"hasCredentials":true,"tasks":["completed:Analyze project","completed:Plan events","completed:Install SDK","completed:Set env vars","completed:Initialize PostHog","completed:Instrument events","completed:Verify","completed:Create dashboard","completed:Write setup report"],"dashboardUrl":"https://us.posthog.com/project/228144/dashboard/2114467","notebookUrl":"https://us.posthog.com/project/228144/notebooks/9zomaGFI","outro":null}
POST /runs {"programId":"metrics"} -> {"runId":"669fc70c-8109-4571-872b-582dcb62f872","programId":"metrics","installDir":"/tmp/wz-ctl-evidence-app","status":"running","error":null,"startedAt":"2026-09-19T18:16:38.279Z","finishedAt":null,"result":null}
  GET /state?wait=60000&since=115 -> screen=intro runPhase=completed tasks=6/6 last=completed:Write setup report
  final -> {"version":115,"screen":"intro","runPhase":"completed","integration":"javascript_node","detectionComplete":true,"hasCredentials":true,"tasks":["completed:Analyze project structure","completed:Configure metrics on PostHog client","completed:Instrument request middleware metrics","completed:Instrument business throughput metrics","completed:Verify build and imports","completed:Write setup report"],"dashboardUrl":null,"notebookUrl":null,"outro":null}
GET /runs -> [
  {
    "runId": "fc8dcba3-7324-41d2-bdbd-481a7de35fa8",
    "programId": "posthog-integration",
    "installDir": "/tmp/wz-ctl-evidence-app",
    "status": "done",
    "error": null,
    "startedAt": "2026-09-19T18:13:15.335Z",
    "finishedAt": "2026-09-19T18:16:38.279Z",
    "result": {
      "version": 86,
      "currentScreen": "intro",
      "session": {
        "installDir": "/tmp/wz-ctl-evidence-app",
        "integration": "javascript_node",
        "detectedFrameworkLabel": null,
        "detectionComplete": true,
        "frameworkContext": {},
        "setupConfirmed": false,
        "integrate": null,
        "llmOptIn": false,
        "discoveredFeatures": [],
        "runRequested": false,
        "runPhase": "completed",
        "completedRuns": [],
        "pendingQuestion": null,
        "taskNotice": null,
        "outroData": null,
        "outroDismissed": false,
        "dashboardUrl": "https://us.posthog.com/project/228144/dashboard/2114467",
        "notebookUrl": "https://us.posthog.com/project/228144/notebooks/9zomaGFI",
        "mcpComplete": false,
        "slackStepDismissed": false,
        "skillsComplete": false,
        "hasCredentials": true,
        "projectId": 228144
      },
      "tasks": [
        {
          "label": "Analyze project",
          "activeForm": "Analyzing project structure and analytics opportunities",
          "status": "completed",
          "done": true
        },
        {
          "label": "Plan events",
          "activeForm": "Planning meaningful server-side events",
          "status": "completed",
          "done": true
        },
        {
          "label": "Install SDK",
          "activeForm": "Installing the PostHog Node.js SDK",
          "status": "completed",
          "done": true
        },
        {
          "label": "Set env vars",
          "activeForm": "Configuring PostHog environment variables",
          "status": "completed",
          "done": true
        },
        {
          "label": "Initialize PostHog",
          "activeForm": "Initializing PostHog for the server runtime",
          "status": "completed",
          "done": true
        },
        {
          "label": "Instrument events",
          "activeForm": "Instrumenting actions, identities, and errors",
          "status": "completed",
          "done": true
        },
        {
          "label": "Verify",
          "activeForm": "Verifying the analytics integration",
          "status": "completed",
          "done": true
        },
        {
          "label": "Create dashboard",
          "activeForm": "Creating the PostHog dashboard and insights",
          "status": "completed",
          "done": true
        },
        {
          "label": "Write setup report",
          "activeForm": "Writing the final setup report",
          "status": "completed",
          "done": true
        }
      ],
      "statusMessages": [
        "Completing source-level verification because the project has no permitted build, typecheck, or lint script, then loading conclusion steps.",
        "Loading the current query and artifact guidance before creating the live analytics dashboard.",
        "Discovering dashboard, insight, notebook, and current documentation tools required for the final PostHog artifacts.",
        "Inspecting each artifact tool schema before constructing any dashboard, insight, or notebook payload.",
        "Creating the parent dashboard while checking current documentation for saved event insights and Node.js capture practices.",
        "Adding lifecycle conversion, overall activity, completion-state, and deletion insights to the new dashboard.",
        "Configured dashboard: https://us.posthog.com/project/228144/dashboard/2114467 — now publishing the setup report to the wizard session.",
        "Published setup report to the wizard session; creating the shareable PostHog notebook copy.",
        "Created notebook: https://us.posthog.com/project/228144/notebooks/9zomaGFI — finalizing the tracked setup work.",
        "PostHog integration complete"
      ],
      "eventPlan": [
        {
          "name": "todo_created",
          "description": "A todo was successfully created through the API."
        },
        {
          "name": "todo_updated",
          "description": "A todo was successfully updated through the API."
        },
        {
          "name": "todo_deleted",
          "description": "A todo was successfully deleted through the API."
        }
      ],
      "handoffText": "# PostHog post-wizard report\n\nThe wizard completed a server-side PostHog integration for this Express application. It installed `posthog-node` and `dotenv`, configured `POSTHOG_PROJECT_TOKEN` and `POSTHOG_HOST` in the local environment, initialized a guarded singleton with exception autocapture, and added graceful shutdown flushing. The API now captures create, update, and delete actions without sending todo titles or other user-entered content as event properties. Incoming PostHog distinct and session headers are reused when available, with a generated non-PII distinct ID otherwise. An Express error handler captures exceptions, and a live dashboard contains lifecycle, activity, completion-state, and deletion insights.\n\n| Event name | Description | File |\n| --- | --- | --- |\n| `todo_created` | A todo was successfully created through the API. | `index.js` |\n| `todo_updated` | A todo was successfully updated through the API. | `index.js` |\n| `todo_deleted` | A todo was successfully deleted through the API. | `index.js` |\n\n## Next steps\n\nWe've built insights and a dashboard to monitor behavior based on the events instrumented in this run:\n\n- [Analytics basics (wizard) dashboard](https://us.posthog.com/project/228144/dashboard/2114467)\n- [Todo lifecycle funnel (wizard)](https://us.posthog.com/project/228144/insights/kt7GLPKT)\n- [Todo activity over time (wizard)](https://us.posthog.com/project/228144/insights/0q53xOAH)\n- [Todo updates by completion state (wizard)](https://us.posthog.com/project/228144/insights/nzzZWI6l)\n- [Todo deletion trend (wizard)](https://us.posthog.com/project/228144/insights/cTPSJXJR)\n\nImplementation reference: [PostHog Node.js Product Analytics installation](https://posthog.com/docs/product-analytics/installation/nodejs).\n\n## Verify before merging\n\n- [ ] Run a full production build (the wizard only verified the files it touched) and fix any lint or type errors introduced by the generated code.\n- [ ] Run the test suite — call sites that were rewritten or instrumented may need updated mocks or fixtures.\n- [ ] Add `POSTHOG_PROJECT_TOKEN` and `POSTHOG_HOST` to `.env.example` and any bootstrap scripts so collaborators know what to set.\n\n### Agent skill\n\nWe've left an agent skill folder in the project. You can use this context for further agent development when using Claude Code. This helps ensure the model uses current approaches for integrating PostHog.",
      "setupQuestions": [],
      "actions": [
        {
          "id": "confirm_setup",
          "description": "Confirm the intro and continue. share: true grants and false declines sharing scan results; absent keeps the toggle (granted when undecided).",
          "params": {
            "share": "boolean (optional)"
          }
        }
      ]
    }
  },
  {
    "runId": "669fc70c-8109-4571-872b-582dcb62f872",
    "programId": "metrics",
    "installDir": "/tmp/wz-ctl-evidence-app",
    "status": "done",
    "error": null,
    "startedAt": "2026-09-19T18:16:38.279Z",
    "finishedAt": "2026-09-19T18:19:23.597Z",
    "result": {
      "version": 115,
      "currentScreen": "intro",
      "session": {
        "installDir": "/tmp/wz-ctl-evidence-app",
        "integration": "javascript_node",
        "detectedFrameworkLabel": null,
        "detectionComplete": true,
        "frameworkContext": {},
        "setupConfirmed": false,
        "integrate": null,
        "llmOptIn": false,
        "discoveredFeatures": [],
        "runRequested": false,
        "runPhase": "completed",
        "completedRuns": [],
        "pendingQuestion": null,
        "taskNotice": null,
        "outroData": null,
        "outroDismissed": false,
        "dashboardUrl": null,
        "notebookUrl": null,
        "mcpComplete": false,
        "slackStepDismissed": false,
        "skillsComplete": false,
        "hasCredentials": true,
        "projectId": 228144
      },
      "tasks": [
        {
          "label": "Analyze project structure",
          "activeForm": "Analyzing project structure",
          "status": "completed",
          "done": true
        },
        {
          "label": "Configure metrics on PostHog client",
          "activeForm": "Configuring metrics on PostHog client",
          "status": "completed",
          "done": true
        },
        {
          "label": "Instrument request middleware metrics",
          "activeForm": "Instrumenting request middleware metrics",
          "status": "completed",
          "done": true
        },
        {
          "label": "Instrument business throughput metrics",
          "activeForm": "Instrumenting business throughput metrics",
          "status": "completed",
          "done": true
        },
        {
          "label": "Verify build and imports",
          "activeForm": "Verifying build and imports",
          "status": "completed",
          "done": true
        },
        {
          "label": "Write setup report",
          "activeForm": "Writing setup report",
          "status": "completed",
          "done": true
        }
      ],
      "statusMessages": [
        "Initializing Wizard agent...",
        "Verbose logs: /tmp/posthog-wizard.log",
        "Agent initialized. Let's get cooking!",
        "Setting up application metrics...",
        "Configuring metrics on the existing PostHog client",
        "Application metrics configured! View the report at ./posthog-metrics-report.md"
      ],
      "eventPlan": [],
      "handoffText": "# PostHog Application Metrics Setup\n\n## Summary\n\nAdded `posthog.metrics` (counters and histograms) to the existing Express\n`express-todo` service, reusing the single `posthog-node` client already\ncreated in `posthog.js`. No new client, no changes to existing event\ncapture, exception tracking, or shutdown behavior.\n\n- **SDK**: `posthog-node` v5.52.4 (already in `package.json`, verified installed\n  in `node_modules` — satisfies the `>= 5.43.0` metrics requirement)\n- **Client**: `posthog.js` — added a `metrics` config block to the existing\n  `new PostHog(...)` call\n- **Service name**: `express-todo` (set via `metrics.serviceName`, plus\n  `metrics.environment` from `NODE_ENV`)\n\n## Changes\n\n### `posthog.js`\n\nAdded a `metrics` config to the existing client instead of creating a second\nclient:\n\n```js\nconst posthog = token && host\n  ? new PostHog(token, {\n      host,\n      enableExceptionAutocapture: true,\n      flushAt: 1,\n      flushInterval: 0,\n      metrics: {\n        serviceName: 'express-todo',\n        environment: process.env.NODE_ENV,\n      },\n    })\n  : null;\n```\n\n### `index.js`\n\n1. **Request/response middleware** (choke point, added once — not per route):\n   - `http.requests` — `count`, attributes: `method`, `route` (route pattern,\n     e.g. `/api/todos/:id`, not the raw path), `status`\n   - `http.request.duration` — `histogram` (`unit: \"ms\"`), same attributes\n\n   ```js\n   app.use((req, res, next) => {\n     const start = process.hrtime.bigint();\n     res.on('finish', () => {\n       if (posthog) {\n         const durationMs = Number(process.hrtime.bigint() - start) / 1e6;\n         const attributes = {\n           method: req.method,\n           route: req.route ? req.route.path : req.path,\n           status: String(res.statusCode),\n         };\n         posthog.metrics.count('http.requests', 1, { attributes });\n         posthog.metrics.histogram('http.request.duration', durationMs, {\n           unit: 'ms',\n           attributes,\n         });\n       }\n     });\n     next();\n   });\n   ```\n\n2. **Business throughput counters**, added next to the existing `capture()`\n   call for the same action (no new distinct-id or user-scoped data — pure\n   totals):\n   - `todos.created` — in `POST /api/todos`, next to the `todo_created` event\n   - `todos.updated` — in `PATCH /api/todos/:id`, next to the `todo_updated` event\n   - `todos.deleted` — in `DELETE /api/todos/:id`, next to the `todo_deleted` event\n\n3. **Error counter** in the existing error-handling middleware, next to the\n   existing `captureException` call:\n   - `http.errors` — `count`, attribute: `route`\n\n## Attribute cardinality\n\nAll attributes are bounded, low-cardinality dimensions: HTTP `method` (a\nhandful of verbs), `route` (the Express route *pattern*, e.g.\n`/api/todos/:id` — not the raw URL, so todo IDs never leak into series), and\n`status` (HTTP status codes). No user IDs, session IDs, or other unbounded\nvalues are attached to any metric.\n\n## Not instrumented\n\n- **Background jobs / queues**: none exist in this service.\n- **Outbound external dependencies**: none exist in this service (in-memory\n  `todos` array, no DB or external API calls).\n\nIf either is added later, add `count`/`histogram`/`gauge` calls at those\ncall sites following the same pattern.\n\n## Verification\n\n- Confirmed `posthog-node@5.52.4` is present in `node_modules` (>= the\n  `5.43.0` floor required for the `posthog.metrics` API) by running\n  `npm install` and inspecting `node_modules/posthog-node/dist/client.d.ts`\n  and `types.d.ts`, which expose `metrics` config and the `count`/`gauge`/\n  `histogram` methods.\n- Reviewed the final `index.js` and `posthog.js` for syntax correctness (no\n  build/typecheck script is defined in `package.json` to run automatically).\n- `POSTHOG_PROJECT_TOKEN` and `POSTHOG_HOST` are already present in `.env`,\n  so no new environment variables were needed.\n\n## Next steps\n\n1. Start the app (`npm start`) and exercise `GET/POST/PATCH/DELETE\n   /api/todos` a few times.\n2. Open **Metrics** in the PostHog sidebar\n   ([Metrics](https://us.posthog.com/project/228144/metrics)) and confirm\n   `http.requests`, `http.request.duration`, `todos.created`,\n   `todos.updated`, `todos.deleted`, and (on a forced error) `http.errors`\n   appear under service `express-todo`.\n3. If metrics aren't enabled yet for this project, open the Metrics page and\n   select **Enable metrics** in the onboarding view (metrics is in open alpha).\n",
      "setupQuestions": [],
      "actions": [
        {
          "id": "confirm_setup",
          "description": "Confirm the intro and continue. share: true grants and false declines sharing scan results; absent keeps the toggle (granted when undecided).",
          "params": {
            "share": "boolean (optional)"
          }
        }
      ]
    }
  }
]
POST /shutdown -> {"ok":true}
wizard exit code: 0
socket removed: true
wizard console:
┌  Welcome to the PostHog setup wizard
│  Running posthog-integration in Headless mode

Evidence: controlled TUI in a pseudo-terminal (no agent run)

The real TUI under node-pty, driven only over the socket: intro, readiness probe, auth resolved by POST /credentials, the run screen parked because nothing armed the run, POST /detect refused with 501 on this surface, exit on POST /shutdown.

$ /Users/vincent/work-code/workbench/wizard/node_modules/.bin/tsx bin.ts --ci --control-socket /var/folders/kz/r8kl49v5683fdf4prxcylqqh0000gn/T/wz-tui-probe-kO6FKV/t.sock --install-dir /tmp/wz-tui-probe-app --project-id 228144 --region us   # POSTHOG_WIZARD_API_KEY in env
socket mode: 600
GET /health -> {"ok":true,"version":"2.76.0","surface":"tui","pid":10942,"program":"posthog-integration"}
GET /state -> {"version":7,"screen":"intro","actions":["confirm_setup"],"hasCredentials":false,"runRequested":false,"runPhase":"idle"}
POST /actions/confirm_setup -> {"version":8,"screen":"health-check","actions":["dismiss_outage"],"hasCredentials":false,"runRequested":false,"runPhase":"idle"}
GET /state?wait=5000&since=9 -> {"version":9,"screen":"auth","actions":[],"hasCredentials":false,"runRequested":false,"runPhase":"idle"}
POST /credentials -> {"version":11,"screen":"run","actions":[],"hasCredentials":true,"runRequested":false,"runPhase":"idle"}
POST /detect -> 501 POST /detect is not available on the tui surface
GET /runs -> []
POST /shutdown -> {"ok":true}
wizard exited; socket removed: true

Bundle audit

scripts/smoke-test.sh on both builds: exactly one chunk carries wizard-control-server; the chunk carrying wizard-tui-entry does not import it; bin.js does not import it; --control-socket alone is refused; with the headless flag it reaches the headless path. scripts/__fixtures__/chunk-manifest.{prod,ci}.json record the module groups; the src/store/control group is imported only by the cli runners group, and both manifests are unchanged by this branch's last commits.

A headless run with WIZARD_CI_GATEWAY_TOKEN_FILE in its environment configures
the gateway from that file, as --ci does, instead of minting a token.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
The control hooks mark a run that ended while running as completed and mark
detection complete after a program's ciPreRun, so the ledger, the stream, and
the state agree with the headless runner. The chunk manifest is keyed by
source-module group instead of chunk file name, because rolldown's chunk
names differ by platform; CI's fixture diff now compares content. The
development skill documents the surfaces and how to drive the control API.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
GET /state is the committed session whitelist (CONTROL_SESSION_KEYS,
credentials as a flag), the run atoms, and the actions legal on the
current screen. The run status, the overlay flag, the context digest, and
the reduced views go; a ledger record's result is the state at the end of
the run. POST /run is requestRun on the store, so ControlHooks shrinks to
what the store cannot do itself: credentials, detection, one run, shutdown.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Routes that rewrite the session or end the process (POST /credentials,
/detect, /shutdown) answer 409 while a run is in flight; a stale-socket
probe only unlinks a socket; close() answers pending long polls before it
drops connections and unlinks synchronously; the 413 reaches the client;
a bad action id is a 400; the ledger records the resolved install dir and
survives a hook that throws synchronously. Detection publishes what
ciPreRun wrote to its copy as well as what it committed; every run calls
runCleanups and keys its task stream by its own skill; the headless
destination reads credentials from the live store; the CI gateway bearer
is dev-build only and skipped for programs without an agent run. The
default intro's confirm_setup follows the scan-sharing toggle like Enter,
and action params are validated instead of coerced.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
The bundle audit compares these with the manifests CI builds; the new modules
and the regrouped chunks belong in them. A failed comparison now uploads its
manifests and reports both files.

Generated-By: PostHog Desktop
Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
@gewenyu99 gewenyu99 closed this Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant