feat(telemetry): PR 4 — wire 7 core events to call sites

[brennanb2025]

Summary

Wires all 7 v1 telemetry events to their call sites, completing the event set per telemetry-implementation.md §PR 4. Stacks on #1372 (foundations), #1374 (transport), and #1385 (first-launch UX). No new schema work.

agent_started attribution flows from each renderer launch site through the queued startup → pty:spawn IPC, and main fires the event only after provider.spawn() resolves — bare-shell tabs and failed spawns produce no event. agent_error rides the same catch path for claude spawns and the work-item readiness timeout (5s → error_class: 'unknown', since process-startup has no v1 enum slot). repo_added is suppressed when the user re-picks an existing repo; settings_changed filters through SETTINGS_CHANGED_WHITELIST with a pre/post diff so no-op blur saves don't inflate adoption signal. workspace_created.source is validated against the closed enum at the IPC boundary — unknown surfaces map to 'unknown' rather than dropping, so dashboard gaps surface as a slice.

The renderer threading hops launch site → queueTabStartupCommand → pendingStartupByTabId → pty-connection → pty-transport → pty:spawn. The EventProps<'agent_started'> type carries the payload through every hop so a missing field is a typecheck error, and main re-validates each enum field on receive as defense-in-depth against a malformed or spoofed IPC.

Test plan

• Unit: pnpm test — all suites green, including the new classify-error.test.ts, pty.test.ts → 'agent_started telemetry', and worktree-activation.test.ts telemetry-forwarding case.
• Activation funnel order: fresh profile → app_opened → folder-picker repo_added → composer workspace_created → agent click agent_started, in order, in PostHog Live Events.
• Spawn failure: trigger ENOENT on a missing claude binary → agent_error with error_class: 'binary_not_found'; agent_started does NOT fire.
• agent_error no-leak: confirm in the PostHog event inspector that message and stack are absent — only error_class (+ optional whitelisted error_name) ride.
• workspace_created.source per surface: sidebar +, Landing, Cmd+J, keyboard shortcut, Tasks page → expected source per surface; hand-crafted IPC with bogus source → 'unknown'.
• settings_changed no-op suppression: re-save the same experimental toggle → no event. Flip it → event with value_kind: 'bool'. Flip a non-whitelisted setting (e.g. theme) → no event.
• app_opened reload dedup: Cmd+R the renderer → no second app_opened in the same session.
• Re-add suppression: pick the same folder twice → exactly one repo_added.
• Packaged RC build / dashboards smoke: build with ORCA_BUILD_IDENTITY=rc ORCA_POSTHOG_WRITE_KEY=phc_... → run the funnel → all 7 events show in PostHog 406068 with orca_channel: rc, $process_person_profile: false, no $geoip_* / $ip.

Manual smoke test results — packaged build vs PostHog 406068

Ran a packaged official build (ORCA_BUILD_IDENTITY=stable ORCA_POSTHOG_WRITE_KEY=phc_…) against an isolated user-data-dir. All 7 V1 events round-tripped to PostHog. Build artifact: arm64 Orca.app, phc key embedded in app.asar.

Core funnel — new-user install (default opted-in)

Event	Result
app_opened	✓ fires once on first did-finish-load
repo_added	✓ method: folder_picker
workspace_created	✓ source: sidebar, from_existing_branch: false
agent_started	✓ all three enum fields ride correctly (claude-code/command_palette, codex/sidebar)
agent_error	✓ error_class: unknown, agent_kind: codex (renderer-side path)
settings_changed	✓ setting_key: editorAutoSave, value_kind: bool
telemetry_opted_in / telemetry_opted_out	✓ via correct

Consent gating

• Pre-banner cohort (existedBeforeTelemetryRelease:true, optedIn:null): zero events transmitted in 10 min — gate works.
• Banner "Turn off" → telemetry_opted_out { via: 'first_launch_banner' } ✓
• Banner "Sure" (i.e. setOptIn(true) from pending_banner) → telemetry_opted_in { via: 'settings' }. This is by design — deriveOptInVia (src/main/ipc/telemetry.ts:92) deliberately refuses to tag setOptIn(true) as first_launch_banner to keep that signal unforgeable from a compromised renderer; the silent-acknowledge ✕ path is the only legitimate optedIn:true transition for that cohort.

Edge cases verified

• repo_added re-add suppression: re-add same path twice → exactly 1 event ✓
• settings_changed no-op suppression: re-save identical value → 0 events; flip → 1 event ✓
• settings_changed whitelist: non-whitelisted setting (terminalFontSize) → 0 events; second whitelist member (experimentalAgentDashboard) → 1 event ✓
• workspace_created.source unknown fallback: bogus surface AND omitted telemetrySource both land as source: 'unknown' ✓
• workspace_created.from_existing_branch: true only when baseBranch is non-empty ✓
• agent_started bare-shell: spawn without telemetry field → 0 events ✓
• agent_started invalid enum: spawn with agent_kind: 'NOT_A_REAL_AGENT' → dropped at the safeParse boundary ✓
• app_opened reload dedup: location.reload() in renderer → no second app_opened (gate held) ✓
• Strict-schema rejection: extra keys on app_opened, unknown event names, non-whitelisted error_name, raw error_message on agent_error — all dropped, none transmitted ✓
• Consent-mutation rate-limit: 6 rapid telemetrySetOptIn flips → only first 5 transmitted, 6th dropped AND state did not mutate (rate-limit fires before the write) ✓

Review-fix flows (post-rebuild verification)

The review fix replaced the mark-only gate with trackAppOpenedOnce() and added app_opened emission to two more code paths. Re-verified end-to-end:

Flow	Expected	PostHog result
New user, default opted-in (first window load via trackAppOpenedOnce())	app_opened	✓ fired
Banner "Sure" (setOptIn(true) from pending_banner)	app_opened BEFORE telemetry_opted_in	✓ both present, app_opened first
Banner ✕ silent acknowledge	app_opened only, no telemetry_opted_in	✓ exactly that

Notes

• I could not exercise the main-side agent_error catch path on pty.ts:805-836 from a renderer-driven smoke test — pty:spawn swallows non-existent cwd and bad shellOverride rather than throwing. Coverage there comes from the unit tests (classify-error.test.ts, pty.test.ts → 'agent_error'). The renderer-side agent_error (5s readiness timeout in launch-work-item-direct.ts) was exercised end-to-end and lands correctly.

Made with Orca 🐋