From 9eb20ffe9637a9952b7116de17d90916c5f60047 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 15:23:07 +0200 Subject: [PATCH 01/20] docs(admin): design daily setup CLI outcomes --- ...6-08-15-daily-setup-cli-outcomes-design.md | 310 ++++++++++++++++++ 1 file changed, 310 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-15-daily-setup-cli-outcomes-design.md diff --git a/docs/superpowers/specs/2026-08-15-daily-setup-cli-outcomes-design.md b/docs/superpowers/specs/2026-08-15-daily-setup-cli-outcomes-design.md new file mode 100644 index 0000000000..e1ff7cd4af --- /dev/null +++ b/docs/superpowers/specs/2026-08-15-daily-setup-cli-outcomes-design.md @@ -0,0 +1,310 @@ +# Daily Setup → CLI Outcomes (v2) Design + +**Status:** Approved for implementation planning + +**Date:** 2026-08-15 + +## Summary + +Add a new daily stacked-column chart to the onboarding analytics dashboard that explains what people do after reaching the v2 Setup page. Each UTC day contains two adjacent stacks: First-time and Returning. Each person can contribute to at most one mutually exclusive outcome per UTC day, and downstream actions are observed for up to 24 hours after that day's Setup anchor. + +The existing aggregate **Setup → CLI outcomes (v2)** chart remains unchanged. The new chart reuses `AdminStackedBarChart` through a backward-compatible grouped-stack extension; it does not introduce another chart component. + +## Problem + +The existing aggregate Setup-to-CLI chart answers a range-level question and collapses each person across the full selected period. It also uses a broad onboarding-start signal. It cannot answer: + +- how Setup cohorts behave day by day; +- whether a person is appearing for the first time or returning within the selected range; +- which Setup copy action preceded CLI activity; +- whether the person ran `init`, ran another CLI command, or ran no CLI command; +- whether a returning Setup view should own a later action instead of the earlier view. + +Repeated Setup views are important product behavior, so the new chart needs person-day semantics rather than range-level unique-person semantics. + +## Goals + +- Show daily v2 Setup outcomes as adjacent First-time and Returning stacks. +- Count a person at most once per UTC day. +- Classify every counted person-day into exactly one of 12 outcomes. +- Observe copy and CLI events for the next 24 hours. +- Attribute an action only to the most recent eligible Setup day. +- Reuse and generically extend `AdminStackedBarChart`. +- Show only outcome categories that actually occur in the displayed range. +- Keep the existing aggregate Setup-to-CLI graph and its semantics unchanged. + +## Non-goals + +- Replacing or modifying the existing **Setup → CLI outcomes (v2)** graph. +- Changing the existing range-level `v2_setup_cli_outcomes` API field. +- Defining First-time against a person's complete product history. It is relative to the selected date range. +- Counting onboarding attempts rather than people. +- Introducing a database migration or persistent analytics table. +- Introducing a new chart component. +- Treating broad `onboarding-v2` channel or Builder onboarding-step events as proof that `init` was run in this new chart. The new chart distinguishes actual CLI command invocations. + +## Definitions + +### Setup anchor + +A Setup anchor is the first qualifying v2 Setup view for one PostHog person on one UTC calendar day: + +- event: `onboarding_step_viewed`; +- `properties.flow = 'pre_org'`; +- `properties.onboarding_version = 2`; +- `properties.step = 'setup'`. + +Later Setup views by that person on the same UTC day do not create another cohort entry and do not reset the 24-hour window. + +### First-time and Returning + +First-time and Returning are calculated only from Setup anchors inside the selected range: + +- the person's earliest Setup day in the selected range is First-time; +- every later Setup day in the selected range is Returning; +- Setup views before the selected range are intentionally ignored. + +This means changing the selected date range can change whether a day is labelled First-time or Returning. + +### Copy signals + +- CLI command copied: `onboarding_cli_command_copied`. +- AI instructions copied: `onboarding_ai_instructions_copied`. + +Both copy signals must also have `properties.flow = 'pre_org'`, `properties.onboarding_version = 2`, and `properties.step = 'setup'`. Copy order does not matter. Both flags can be true for one person-day. + +### CLI signals + +- Init run: `CLI Command Invoked` with `properties.command_path = 'init'`. +- Other CLI run: `CLI Command Invoked` with a non-`init` command path. + +If both Init and Other CLI are present, Init takes precedence when selecting the outcome. + +## Attribution Window + +Each Setup anchor owns a half-open action window beginning at the anchor timestamp: + +```text +[setup anchor, min(setup anchor + 24 hours, next Setup anchor)) +``` + +The next Setup anchor is the person's next qualifying UTC-day anchor, including an ownership-only anchor in the 24-hour tail after the selected range. Truncating the earlier window at the next anchor ensures one action cannot upgrade two daily columns. + +Ownership-only anchors after the selected end are not emitted as chart cohorts and do not affect First-time/Returning labels. They exist only to prevent the final displayed day from claiming actions that actually followed a newer Setup view. For example, a September 1 Setup view can truncate an August 31 window when August is selected, but September 1 is not displayed. + +Example: + +- August 3: person reaches Setup for the first time. +- August 4: the same person reaches Setup again and then runs `init`. +- The `init` action belongs only to August 4 Returning. +- August 3 remains classified from the actions assigned before the August 4 anchor, potentially No action. + +An action exactly at a new Setup anchor belongs to the new anchor. An action exactly 24 hours after an anchor is outside that anchor's window. + +## Outcome Taxonomy + +Every Setup person-day maps to exactly one outcome. + +| Key | Init run | Other CLI run | Copy combination | Display label | +| --- | --- | --- | --- | --- | +| `cli_copy_init` | Yes | Any | CLI only | CLI copy + init | +| `ai_copy_init` | Yes | Any | AI only | AI copy + init | +| `both_copy_init` | Yes | Any | Both | Both copied + init | +| `no_copy_init` | Yes | Any | Neither | No copy + init | +| `cli_copy_other_cli` | No | Yes | CLI only | CLI copy + other CLI | +| `ai_copy_other_cli` | No | Yes | AI only | AI copy + other CLI | +| `both_copy_other_cli` | No | Yes | Both | Both copied + other CLI | +| `no_copy_other_cli` | No | Yes | Neither | No copy + other CLI | +| `cli_copy_no_cli` | No | No | CLI only | CLI copied · no CLI run | +| `ai_copy_no_cli` | No | No | AI only | AI copied · no CLI run | +| `both_copy_no_cli` | No | No | Both | Both copied · no CLI run | +| `no_action` | No | No | Neither | No action | + +Classification order is: + +1. If Init was run, choose one of the four Init outcomes from the copy flags. +2. Otherwise, if another CLI command was run, choose one of the four Other CLI outcomes. +3. Otherwise, choose one of the three copied-with-no-CLI outcomes or No action. + +## Backend Data Flow + +### Dedicated PostHog query + +The current onboarding analytics query groups by onboarding attempt and retains only the earliest Setup timestamp for that attempt. Extending that result would lose repeated daily Setup views. Add a dedicated HogQL query for the new graph and leave `buildFrontendOnboardingHogql` and `v2_setup_cli_outcomes` unchanged. + +The new query will: + +1. Identify people with a qualifying Setup event inside the selected range. +2. For only those cohort people, read qualifying Setup events from the selected start through the 24-hour follow-up tail. +3. Read the two qualifying v2/pre-org/Setup copy events and all `CLI Command Invoked` events for the same cohort people through `selected end + 24 hours`. +4. Return normalized event rows containing the person ID, timestamp, event kind, and CLI command path when applicable. + +Restricting action rows to people who reached Setup avoids scanning unrelated CLI identities in the result set. The query must expose total-row metadata and use an explicit row limit. The server must fail loudly and log when the total exceeds the limit rather than returning a truncated chart. + +### Pure analytics model + +A pure TypeScript model will: + +- validate the normalized PostHog rows; +- group Setup events into the first anchor per person and UTC day; +- mark anchors inside the selected range as display cohorts and tail anchors as ownership-only; +- derive First-time/Returning from display cohorts only; +- assign each copy and CLI event to the most recent eligible Setup anchor using the half-open attribution window; +- classify each person-day into one outcome key; +- aggregate counts by UTC date and lifecycle; +- zero-fill every UTC date in the selected range; +- return all outcome keys with numeric counts, including zeros. + +Keeping anchoring, attribution, classification, and date filling in a pure model makes the complete business rule directly unit-testable and avoids duplicating it in HogQL or embedding display labels there. + +### API response + +Extend `FrontendOnboardingAnalytics` with a new field; do not change the existing aggregate field: + +```ts +type DailySetupCliOutcomeKey = + | 'cli_copy_init' + | 'ai_copy_init' + | 'both_copy_init' + | 'no_copy_init' + | 'cli_copy_other_cli' + | 'ai_copy_other_cli' + | 'both_copy_other_cli' + | 'no_copy_other_cli' + | 'cli_copy_no_cli' + | 'ai_copy_no_cli' + | 'both_copy_no_cli' + | 'no_action' + +interface DailySetupCliOutcomePoint { + date: string + first_time: Record + returning: Record +} +``` + +The response field is `daily_setup_cli_outcomes: DailySetupCliOutcomePoint[]`. + +The endpoint should execute the existing analytics query and the new dedicated query within the same request. If either fails or is truncated, the existing dashboard error state is used; partial results must not be presented as complete analytics. + +## Chart Component Extension + +Extend `AdminStackedBarChart` without changing existing callers: + +- add optional `stack` metadata to a series/dataset; +- add optional human-readable `stackLabel` metadata for tooltips; +- pass `stack` through to Chart.js so equal stack IDs combine and different stack IDs render side by side; +- preserve current behavior when `stack` is absent; +- compute tooltip totals from datasets in the hovered stack only when stack metadata is present; +- include the stack label in grouped-stack tooltips; +- deduplicate legend entries that share an outcome label; +- make a deduplicated legend item toggle every dataset with that outcome label; +- omit legend entries whose matching datasets total zero across the displayed range. + +For the new graph, each active outcome produces two datasets with the same label and color: + +- `stack = 'first_time'`, displayed on the left; +- `stack = 'returning'`, displayed on the right. + +The frontend includes both lifecycle datasets for an active outcome so the paired layout stays stable. An outcome is active when either lifecycle has a non-zero count anywhere in the selected range. + +## Dashboard Presentation + +Add a new `ChartCard` immediately after the existing aggregate **Setup → CLI outcomes (v2)** card. + +- Title: **Daily Setup → CLI outcomes (v2)**. +- Description: explain that each person is counted once per UTC day, the left stack is First-time, the right stack is Returning, and actions use the next 24 hours. +- Do not show a range-level people total in the card header because the measure is person-days and a returning person can legitimately appear on multiple dates. +- Use one consistent color per outcome across both lifecycle stacks. +- Use related color families for Init, Other CLI, and No CLI outcomes; reserve a neutral gray for No action. +- Render First-time on the left and Returning on the right for every date. +- Keep zero-count dates on the x-axis to preserve the complete selected daily range. +- If the whole range has no Setup person-days, use the existing `ChartCard` empty state. + +### Legend behavior + +The legend is derived from the displayed data: + +- show an outcome once if its combined First-time and Returning count is greater than zero anywhere in the selected range; +- do not show categories that are zero throughout the range; +- never show separate duplicate legend items for First-time and Returning; +- retain First-time/Returning distinction through column position and tooltip text. + +### Tooltip behavior + +For the hovered stack, show: + +- UTC date; +- First-time or Returning; +- outcome label; +- person count; +- percentage of that lifecycle stack's total for the day. + +The denominator must not combine First-time and Returning. + +## Error Handling and Empty States + +- Invalid dates and ranges continue to use the endpoint's existing validation. +- The action query boundary must be derived safely as `end + 24 hours` and remain within the supported PostHog timestamp range. +- Invalid row shapes are rejected instead of coerced into misleading counts. +- Query failures and row-limit overflow fail the analytics request and are logged with request ID and total/limit metadata. +- A valid query with no Setup anchors returns the complete date range with zero counts; the frontend displays the chart card's normal no-data state. + +## Testing Strategy + +### Backend model tests + +- all 12 mutually exclusive categories; +- copy order does not affect the category; +- Init takes precedence over Other CLI; +- one person is counted once on a UTC day despite repeated Setup views; +- the first selected-range Setup day is First-time and later days are Returning; +- Setup history before the selected range does not change lifecycle classification; +- an action in overlapping nominal 24-hour windows belongs only to the most recent Setup anchor; +- an ownership-only Setup anchor after the selected end truncates the final displayed anchor without appearing in the result; +- actions at the next anchor and at the 24-hour boundary follow the half-open interval rules; +- UTC day boundaries and zero-filled missing dates; +- totals across all categories equal the number of Setup person-days for each date/lifecycle. + +### Query and endpoint tests + +- the query filters v2 pre-org Setup events correctly; +- both copy events require v2/pre-org/Setup properties, and all `CLI Command Invoked` events are selected; +- the action boundary extends exactly 24 hours beyond the selected end; +- event-row total metadata and row-limit overflow are validated; +- the existing analytics response fields and aggregate outcome graph remain unchanged; +- a new-query failure produces the established analytics error behavior rather than partial data. + +### Chart and frontend tests + +- ungrouped existing `AdminStackedBarChart` callers retain current data and tooltip behavior; +- grouped datasets render with two stack IDs; +- grouped tooltip percentages use only the hovered stack; +- duplicate outcome legend labels collapse into one item; +- toggling a deduplicated legend item toggles both lifecycle datasets; +- zero-only categories are absent from the legend; +- a category present only in one lifecycle still appears once; +- the series mapper preserves every date and filters only range-wide zero outcomes; +- the existing aggregate chart remains rendered and the new daily chart is rendered separately below it; +- all new user-facing strings are translation keys in `messages/en.json` with no inline fallback text. + +## Expected Implementation Areas + +- `supabase/functions/_backend/utils/frontend_onboarding_analytics.ts` +- `supabase/functions/_backend/utils/frontend_onboarding_analytics_model.ts` +- `src/services/adminFrontendOnboarding.ts` +- `src/components/admin/AdminStackedBarChart.vue` +- `src/components/admin/adminStackedBarChart.ts` +- `src/pages/admin/dashboard/frontend-onboarding.vue` +- `messages/en.json` +- existing onboarding analytics, dashboard, and stacked-chart unit tests + +## Acceptance Criteria + +- The existing aggregate Setup-to-CLI chart is visually and semantically unchanged. +- A separate daily chart displays paired First-time and Returning stacks for every selected UTC date. +- Each Setup person-day belongs to exactly one of the 12 approved outcomes. +- Actions are attributed only to the latest eligible Setup anchor within 24 hours. +- The legend contains only outcomes present somewhere in the displayed range and contains no lifecycle duplicates. +- Existing `AdminStackedBarChart` usages remain backward compatible. +- Backend, frontend model, chart-helper, dashboard, lint, and type-check validation pass. From 471fa4d89a945c243eaf48cb4423ef7160bbe993 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 15:43:23 +0200 Subject: [PATCH 02/20] docs(admin): plan daily Setup CLI outcomes --- .../2026-08-15-daily-setup-cli-outcomes.md | 1668 +++++++++++++++++ 1 file changed, 1668 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-15-daily-setup-cli-outcomes.md diff --git a/docs/superpowers/plans/2026-08-15-daily-setup-cli-outcomes.md b/docs/superpowers/plans/2026-08-15-daily-setup-cli-outcomes.md new file mode 100644 index 0000000000..5c69992525 --- /dev/null +++ b/docs/superpowers/plans/2026-08-15-daily-setup-cli-outcomes.md @@ -0,0 +1,1668 @@ +# Daily Setup → CLI Outcomes (v2) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a daily paired-stack chart that classifies each v2 Setup person-day into one of 12 CLI/copy outcomes while preserving the existing aggregate chart. + +**Architecture:** A dedicated PostHog adapter returns bounded, normalized Setup/copy/CLI events for people who reached Setup in the selected range. A pure TypeScript model owns UTC-day anchoring, First-time/Returning classification, latest-anchor attribution, 12-way outcome classification, and zero filling. The existing admin endpoint adds the resulting field, while `AdminStackedBarChart` gains optional grouped-stack behavior used by a new dashboard card. + +**Tech Stack:** TypeScript, Hono/Deno backend utilities, PostHog HogQL, Vue 3, Chart.js, vue-chartjs, Vitest, Bun + +**Design spec:** `docs/superpowers/specs/2026-08-15-daily-setup-cli-outcomes-design.md` + +--- + +## File map + +### New files + +- `supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts` — typed event model, UTC person-day anchoring, action attribution, outcome classification, and daily aggregation. +- `supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts` — dedicated HogQL builder, total validation, row parsing, PostHog failure handling, and limit logging. +- `tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts` — pure business-rule coverage for all outcomes, repeat views, lifecycle, attribution, boundaries, and zero filling. +- `tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts` — HogQL, parsing, query failure, and event-limit coverage. + +### Modified files + +- `supabase/functions/_backend/utils/frontend_onboarding_analytics.ts` — run the existing and new PostHog reads together and append `daily_setup_cli_outcomes` without changing `v2_setup_cli_outcomes`. +- `tests/frontend-onboarding-analytics.unit.test.ts` — cover two-query orchestration, the +24-hour daily boundary, response integration, and failure behavior. +- `src/components/admin/adminStackedBarChart.ts` — optional stack metadata, grouped tooltip totals, dynamic deduplicated legend generation, and grouped legend toggling. +- `src/components/admin/AdminStackedBarChart.vue` — pass optional stack metadata and select grouped options only when grouped data is present. +- `tests/admin-stacked-bar-chart.unit.test.ts` — grouped data, stack-scoped tooltips, zero-only legend filtering, deduplication, toggling, and legacy behavior. +- `src/services/adminFrontendOnboarding.ts` — API types, outcome definitions/colors, and the range-aware paired-series mapper. +- `tests/admin-frontend-onboarding-dashboard.unit.test.ts` — mapper behavior, dynamic categories, separate-card placement, old-chart preservation, and translations. +- `src/pages/admin/dashboard/frontend-onboarding.vue` — computed labels/series and the new chart card below the existing aggregate card. +- `messages/en.json` — title, description, lifecycle labels, and 12 outcome labels. + +--- + +### Task 1: Define and test the 12-way outcome taxonomy + +**Files:** +- Create: `supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts` +- Create: `tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts` + +- [ ] **Step 1: Write the failing table-driven taxonomy test** + +Create `tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts` with the taxonomy cases first: + +```ts +import { describe, expect, it } from 'vitest' +import { + classifyFrontendOnboardingDailySetupCliOutcome, + FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS, +} from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts' + +describe('frontend onboarding daily Setup CLI outcomes model', () => { + it.each([ + [{ cliCopied: true, aiCopied: false, initRun: true, otherCliRun: false }, 'cli_copy_init'], + [{ cliCopied: false, aiCopied: true, initRun: true, otherCliRun: false }, 'ai_copy_init'], + [{ cliCopied: true, aiCopied: true, initRun: true, otherCliRun: false }, 'both_copy_init'], + [{ cliCopied: false, aiCopied: false, initRun: true, otherCliRun: false }, 'no_copy_init'], + [{ cliCopied: true, aiCopied: false, initRun: false, otherCliRun: true }, 'cli_copy_other_cli'], + [{ cliCopied: false, aiCopied: true, initRun: false, otherCliRun: true }, 'ai_copy_other_cli'], + [{ cliCopied: true, aiCopied: true, initRun: false, otherCliRun: true }, 'both_copy_other_cli'], + [{ cliCopied: false, aiCopied: false, initRun: false, otherCliRun: true }, 'no_copy_other_cli'], + [{ cliCopied: true, aiCopied: false, initRun: false, otherCliRun: false }, 'cli_copy_no_cli'], + [{ cliCopied: false, aiCopied: true, initRun: false, otherCliRun: false }, 'ai_copy_no_cli'], + [{ cliCopied: true, aiCopied: true, initRun: false, otherCliRun: false }, 'both_copy_no_cli'], + [{ cliCopied: false, aiCopied: false, initRun: false, otherCliRun: false }, 'no_action'], + ] as const)('classifies %o as %s', (signals, expected) => { + expect(classifyFrontendOnboardingDailySetupCliOutcome(signals)).toBe(expected) + }) + + it('gives init precedence over another CLI command', () => { + expect(classifyFrontendOnboardingDailySetupCliOutcome({ + cliCopied: true, + aiCopied: true, + initRun: true, + otherCliRun: true, + })).toBe('both_copy_init') + }) + + it('publishes every category exactly once in display order', () => { + expect(FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS).toEqual([ + 'cli_copy_init', + 'ai_copy_init', + 'both_copy_init', + 'no_copy_init', + 'cli_copy_other_cli', + 'ai_copy_other_cli', + 'both_copy_other_cli', + 'no_copy_other_cli', + 'cli_copy_no_cli', + 'ai_copy_no_cli', + 'both_copy_no_cli', + 'no_action', + ]) + }) +}) +``` + +- [ ] **Step 2: Run the new model test and verify the missing module failure** + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +``` + +Expected: FAIL because `frontend_onboarding_daily_setup_cli_outcomes_model.ts` does not exist. + +- [ ] **Step 3: Add the model types, empty-count factory, and classifier** + +Create `supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts`: + +```ts +export const FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS = [ + 'cli_copy_init', + 'ai_copy_init', + 'both_copy_init', + 'no_copy_init', + 'cli_copy_other_cli', + 'ai_copy_other_cli', + 'both_copy_other_cli', + 'no_copy_other_cli', + 'cli_copy_no_cli', + 'ai_copy_no_cli', + 'both_copy_no_cli', + 'no_action', +] as const + +export type FrontendOnboardingDailySetupCliOutcomeKey = typeof FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS[number] +export type FrontendOnboardingDailySetupCliLifecycle = 'first_time' | 'returning' +export type FrontendOnboardingDailySetupCliEventKind = 'setup' | 'cli_copy' | 'ai_copy' | 'cli_command' + +export interface FrontendOnboardingDailySetupCliEvent { + personId: string + timestampMs: number + kind: FrontendOnboardingDailySetupCliEventKind + commandPath?: string +} + +export interface FrontendOnboardingDailySetupCliSignals { + cliCopied: boolean + aiCopied: boolean + initRun: boolean + otherCliRun: boolean +} + +export type FrontendOnboardingDailySetupCliOutcomeCounts = Record + +export interface FrontendOnboardingDailySetupCliOutcomePoint { + date: string + first_time: FrontendOnboardingDailySetupCliOutcomeCounts + returning: FrontendOnboardingDailySetupCliOutcomeCounts +} + +export function createFrontendOnboardingDailySetupCliOutcomeCounts(): FrontendOnboardingDailySetupCliOutcomeCounts { + return Object.fromEntries( + FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS.map(key => [key, 0]), + ) as FrontendOnboardingDailySetupCliOutcomeCounts +} + +export function classifyFrontendOnboardingDailySetupCliOutcome( + signals: FrontendOnboardingDailySetupCliSignals, +): FrontendOnboardingDailySetupCliOutcomeKey { + const copyPrefix = signals.cliCopied && signals.aiCopied + ? 'both_copy' + : signals.cliCopied + ? 'cli_copy' + : signals.aiCopied + ? 'ai_copy' + : 'no_copy' + + if (signals.initRun) + return `${copyPrefix}_init` as FrontendOnboardingDailySetupCliOutcomeKey + if (signals.otherCliRun) + return `${copyPrefix}_other_cli` as FrontendOnboardingDailySetupCliOutcomeKey + if (copyPrefix !== 'no_copy') + return `${copyPrefix}_no_cli` as FrontendOnboardingDailySetupCliOutcomeKey + return 'no_action' +} +``` + +- [ ] **Step 4: Run the taxonomy test and verify it passes** + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +``` + +Expected: PASS with 14 assertions/cases and no other test failures. + +- [ ] **Step 5: Commit the taxonomy model** + +```bash +git add supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +git commit -m "feat(admin): define daily Setup CLI outcomes" +``` + +--- + +### Task 2: Implement UTC person-day anchoring and latest-anchor attribution + +**Files:** +- Modify: `supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts` +- Modify: `tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts` + +- [ ] **Step 1: Add failing tests for daily uniqueness, returning users, attribution, tail anchors, and zero-filled dates** + +Append inside the existing `describe` block: + +```ts + it('counts one anchor per UTC day and assigns overlapping actions only to the latest anchor', () => { + const startMs = Date.UTC(2026, 7, 3) + const endMs = Date.UTC(2026, 7, 6) + const hour = 60 * 60 * 1000 + const events = [ + { personId: 'person-a', timestampMs: startMs + 9 * hour, kind: 'setup' as const }, + { personId: 'person-a', timestampMs: startMs + 12 * hour, kind: 'setup' as const }, + { personId: 'person-a', timestampMs: startMs + 13 * hour, kind: 'ai_copy' as const }, + { personId: 'person-a', timestampMs: startMs + 24 * hour + 8 * hour, kind: 'setup' as const }, + { personId: 'person-a', timestampMs: startMs + 24 * hour + 9 * hour, kind: 'cli_command' as const, commandPath: 'init' }, + { personId: 'person-b', timestampMs: startMs + 24 * hour + 10 * hour, kind: 'setup' as const }, + { personId: 'person-b', timestampMs: startMs + 24 * hour + 11 * hour, kind: 'cli_copy' as const }, + { personId: 'person-b', timestampMs: startMs + 24 * hour + 12 * hour, kind: 'cli_command' as const, commandPath: 'app-debug' }, + ] + + const points = buildFrontendOnboardingDailySetupCliOutcomes(events, startMs, endMs) + + expect(points).toHaveLength(3) + expect(points[0].date).toBe('2026-08-03') + expect(points[0].first_time.ai_copy_no_cli).toBe(1) + expect(points[0].returning.no_action).toBe(0) + expect(points[1].date).toBe('2026-08-04') + expect(points[1].returning.no_copy_init).toBe(1) + expect(points[1].first_time.cli_copy_other_cli).toBe(1) + expect(Object.values(points[2].first_time).reduce((sum, value) => sum + value, 0)).toBe(0) + expect(Object.values(points[2].returning).reduce((sum, value) => sum + value, 0)).toBe(0) + }) + + it('uses an ownership-only tail anchor to prevent the last displayed day stealing an action', () => { + const startMs = Date.UTC(2026, 7, 31) + const endMs = Date.UTC(2026, 8, 1) + const minute = 60 * 1000 + const events = [ + { personId: 'person-a', timestampMs: endMs - 30 * minute, kind: 'setup' as const }, + { personId: 'person-a', timestampMs: endMs + 15 * minute, kind: 'setup' as const }, + { personId: 'person-a', timestampMs: endMs + 20 * minute, kind: 'cli_command' as const, commandPath: 'init' }, + ] + + const [point] = buildFrontendOnboardingDailySetupCliOutcomes(events, startMs, endMs) + + expect(point.first_time.no_action).toBe(1) + expect(point.first_time.no_copy_init).toBe(0) + expect(point.returning.no_copy_init).toBe(0) + }) + + it('uses half-open action windows at the next anchor and 24-hour boundary', () => { + const startMs = Date.UTC(2026, 7, 1) + const endMs = Date.UTC(2026, 7, 4) + const day = 24 * 60 * 60 * 1000 + const events = [ + { personId: 'next-anchor', timestampMs: startMs, kind: 'setup' as const }, + { personId: 'next-anchor', timestampMs: startMs + day, kind: 'setup' as const }, + { personId: 'next-anchor', timestampMs: startMs + day, kind: 'cli_command' as const, commandPath: 'init' }, + { personId: 'boundary', timestampMs: startMs, kind: 'setup' as const }, + { personId: 'boundary', timestampMs: startMs + day, kind: 'cli_command' as const, commandPath: 'init' }, + ] + + const points = buildFrontendOnboardingDailySetupCliOutcomes(events, startMs, endMs) + + expect(points[0].first_time.no_action).toBe(2) + expect(points[0].first_time.no_copy_init).toBe(0) + expect(points[1].returning.no_copy_init).toBe(1) + }) + + it.each([ + [['cli_copy', 'ai_copy'] as const], + [['ai_copy', 'cli_copy'] as const], + ])('treats copy order as irrelevant for %o', ([firstCopy, secondCopy]) => { + const startMs = Date.UTC(2026, 7, 1) + const endMs = Date.UTC(2026, 7, 2) + const events = [ + { personId: 'person-a', timestampMs: startMs + 1_000, kind: 'setup' as const }, + { personId: 'person-a', timestampMs: startMs + 2_000, kind: firstCopy }, + { personId: 'person-a', timestampMs: startMs + 3_000, kind: secondCopy }, + { personId: 'person-a', timestampMs: startMs + 4_000, kind: 'cli_command' as const, commandPath: 'init' }, + ] + + const [point] = buildFrontendOnboardingDailySetupCliOutcomes(events, startMs, endMs) + + expect(point.first_time.both_copy_init).toBe(1) + }) + + it('ignores Setup history before the selected range when assigning lifecycle', () => { + const startMs = Date.UTC(2026, 7, 1) + const endMs = Date.UTC(2026, 7, 2) + const events = [ + { personId: 'person-a', timestampMs: startMs - 1_000, kind: 'setup' as const }, + { personId: 'person-a', timestampMs: startMs + 1_000, kind: 'setup' as const }, + ] + + const [point] = buildFrontendOnboardingDailySetupCliOutcomes(events, startMs, endMs) + + expect(point.first_time.no_action).toBe(1) + expect(Object.values(point.returning).reduce((sum, value) => sum + value, 0)).toBe(0) + }) + + it('rejects invalid bounds and invalid normalized events', () => { + const startMs = Date.UTC(2026, 7, 1) + const endMs = Date.UTC(2026, 7, 2) + + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([], endMs, startMs)).toThrow(RangeError) + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([ + { personId: '', timestampMs: startMs, kind: 'setup' }, + ], startMs, endMs)).toThrow('daily Setup CLI event is invalid') + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([ + { personId: 'person-a', timestampMs: startMs, kind: 'cli_command' }, + ], startMs, endMs)).toThrow('daily Setup CLI command path is invalid') + }) +``` + +Update the import to include `buildFrontendOnboardingDailySetupCliOutcomes`. + +- [ ] **Step 2: Run the model test and verify the missing builder failure** + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +``` + +Expected: FAIL because `buildFrontendOnboardingDailySetupCliOutcomes` is not exported. + +- [ ] **Step 3: Implement the person-day builder in the pure model** + +Append to `frontend_onboarding_daily_setup_cli_outcomes_model.ts`: + +```ts +const DAY_MS = 24 * 60 * 60 * 1000 + +interface SetupAnchor { + personId: string + date: string + timestampMs: number + lifecycle: FrontendOnboardingDailySetupCliLifecycle | null + signals: FrontendOnboardingDailySetupCliSignals +} + +function utcDate(timestampMs: number): string { + return new Date(timestampMs).toISOString().slice(0, 10) +} + +function eachUtcDate(startMs: number, endMs: number): string[] { + const start = new Date(startMs) + let cursor = Date.UTC(start.getUTCFullYear(), start.getUTCMonth(), start.getUTCDate()) + const dates: string[] = [] + while (cursor < endMs) { + dates.push(utcDate(cursor)) + cursor += DAY_MS + } + return dates +} + +function validateEvent(event: FrontendOnboardingDailySetupCliEvent): void { + if (!event.personId.trim() || !Number.isFinite(event.timestampMs)) + throw new TypeError('daily Setup CLI event is invalid') + if (event.kind === 'cli_command' && !event.commandPath?.trim()) + throw new TypeError('daily Setup CLI command path is invalid') +} + +function createSignals(): FrontendOnboardingDailySetupCliSignals { + return { cliCopied: false, aiCopied: false, initRun: false, otherCliRun: false } +} + +function applyAction(anchor: SetupAnchor, event: FrontendOnboardingDailySetupCliEvent): void { + if (event.kind === 'cli_copy') + anchor.signals.cliCopied = true + else if (event.kind === 'ai_copy') + anchor.signals.aiCopied = true + else if (event.kind === 'cli_command' && event.commandPath === 'init') + anchor.signals.initRun = true + else if (event.kind === 'cli_command') + anchor.signals.otherCliRun = true +} + +export function buildFrontendOnboardingDailySetupCliOutcomes( + events: readonly FrontendOnboardingDailySetupCliEvent[], + startMs: number, + endMs: number, +): FrontendOnboardingDailySetupCliOutcomePoint[] { + if (!Number.isFinite(startMs) || !Number.isFinite(endMs) || endMs <= startMs) + throw new RangeError('daily Setup CLI bounds are invalid') + + const followupEndMs = endMs + DAY_MS + const firstSetupByPersonDay = new Map() + + for (const event of events) { + validateEvent(event) + if (event.kind !== 'setup' || event.timestampMs < startMs || event.timestampMs >= followupEndMs) + continue + const key = `${event.personId}\u0000${utcDate(event.timestampMs)}` + const current = firstSetupByPersonDay.get(key) + if (!current || event.timestampMs < current.timestampMs) + firstSetupByPersonDay.set(key, event) + } + + const anchorsByPerson = new Map() + for (const setup of firstSetupByPersonDay.values()) { + const anchors = anchorsByPerson.get(setup.personId) ?? [] + anchors.push({ + personId: setup.personId, + date: utcDate(setup.timestampMs), + timestampMs: setup.timestampMs, + lifecycle: null, + signals: createSignals(), + }) + anchorsByPerson.set(setup.personId, anchors) + } + + for (const anchors of anchorsByPerson.values()) { + anchors.sort((left, right) => left.timestampMs - right.timestampMs) + let displayedAnchors = 0 + for (const anchor of anchors) { + if (anchor.timestampMs >= endMs) + continue + anchor.lifecycle = displayedAnchors === 0 ? 'first_time' : 'returning' + displayedAnchors++ + } + } + + for (const event of events) { + if (event.kind === 'setup' || event.timestampMs < startMs || event.timestampMs >= followupEndMs) + continue + const anchors = anchorsByPerson.get(event.personId) ?? [] + for (let index = anchors.length - 1; index >= 0; index--) { + const anchor = anchors[index] + if (event.timestampMs < anchor.timestampMs) + continue + const nextAnchorMs = anchors[index + 1]?.timestampMs ?? Number.POSITIVE_INFINITY + const windowEndMs = Math.min(anchor.timestampMs + DAY_MS, nextAnchorMs) + if (event.timestampMs < windowEndMs && anchor.lifecycle) + applyAction(anchor, event) + break + } + } + + const points = eachUtcDate(startMs, endMs).map(date => ({ + date, + first_time: createFrontendOnboardingDailySetupCliOutcomeCounts(), + returning: createFrontendOnboardingDailySetupCliOutcomeCounts(), + })) + const pointsByDate = new Map(points.map(point => [point.date, point])) + + for (const anchors of anchorsByPerson.values()) { + for (const anchor of anchors) { + if (!anchor.lifecycle) + continue + const point = pointsByDate.get(anchor.date) + if (!point) + continue + const outcome = classifyFrontendOnboardingDailySetupCliOutcome(anchor.signals) + point[anchor.lifecycle][outcome]++ + } + } + + return points +} +``` + +- [ ] **Step 4: Run the model tests and verify every attribution case passes** + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +``` + +Expected: PASS, including August 31/September 1 ownership and exact-boundary cases. + +- [ ] **Step 5: Commit the attribution model** + +```bash +git add supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +git commit -m "feat(admin): attribute daily Setup CLI actions" +``` + +--- + +### Task 3: Add the bounded PostHog event adapter + +**Files:** +- Create: `supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts` +- Create: `tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts` + +- [ ] **Step 1: Write failing query, parsing, and limit tests** + +Create `tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts`: + +```ts +import type { Context } from 'hono' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { + assertFrontendOnboardingDailySetupCliEventTotal, + buildFrontendOnboardingDailySetupCliHogql, + FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT, + getFrontendOnboardingDailySetupCliEvents, +} from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts' + +const { cloudlogErrMock, queryPosthogHogqlMock } = vi.hoisted(() => ({ + cloudlogErrMock: vi.fn(), + queryPosthogHogqlMock: vi.fn(), +})) + +vi.mock('../supabase/functions/_backend/utils/posthog_read.ts', () => ({ + queryPosthogHogql: queryPosthogHogqlMock, +})) +vi.mock('../supabase/functions/_backend/utils/logging.ts', () => ({ + cloudlogErr: cloudlogErrMock, +})) + +function createContext(): Context { + return { get: () => 'request-id' } as unknown as Context +} + +beforeEach(() => { + cloudlogErrMock.mockReset() + queryPosthogHogqlMock.mockReset() +}) + +describe('daily Setup CLI PostHog adapter', () => { + it('selects only cohort people and relevant v2 Setup, copy, and CLI events', () => { + const query = buildFrontendOnboardingDailySetupCliHogql( + '2026-08-01T00:00:00.000Z', + '2026-09-01T00:00:00.000Z', + '2026-09-02T00:00:00.000Z', + ) + + expect(query).toContain("event = 'onboarding_step_viewed'") + expect(query).toContain("event IN ('onboarding_cli_command_copied', 'onboarding_ai_instructions_copied')") + expect(query).toContain("event = 'CLI Command Invoked'") + expect(query).toContain("JSONExtractString(toString(properties), 'flow') = 'pre_org'") + expect(query).toContain('toIntOrZero(toString(properties.onboarding_version)) = 2') + expect(query).toContain("JSONExtractString(toString(properties), 'step') = 'setup'") + expect(query).toContain('INNER JOIN setup_people AS cohort') + expect(query).toContain("timestamp < parseDateTimeBestEffort('2026-09-02T00:00:00.000Z')") + expect(query).toContain('count() OVER () AS total_events') + expect(query).toContain('ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC') + expect(query).toContain(`LIMIT ${FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT}`) + }) + + it('maps normalized rows and trims CLI command paths', async () => { + queryPosthogHogqlMock.mockResolvedValue({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: 'person-a', timestamp_ms: 1_000, event_kind: 'setup', command_path: '', total_events: 2 }, + { person_id: 'person-a', timestamp_ms: 2_000, event_kind: 'cli_command', command_path: ' init ', total_events: 2 }, + ], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-09-01T00:00:00.000Z', + '2026-09-02T00:00:00.000Z', + )).resolves.toEqual([ + { personId: 'person-a', timestampMs: 1_000, kind: 'setup' }, + { personId: 'person-a', timestampMs: 2_000, kind: 'cli_command', commandPath: 'init' }, + ]) + }) + + it('fails closed and logs when total event metadata exceeds the limit', async () => { + queryPosthogHogqlMock.mockResolvedValue({ + configured: true, + connected: true, + failureReason: null, + rows: [{ total_events: FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT + 1 }], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-09-01T00:00:00.000Z', + '2026-09-02T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query exceeded event limit') + expect(cloudlogErrMock).toHaveBeenCalledWith(expect.objectContaining({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_event_limit_exceeded', + })) + }) + + it('rejects malformed rows and unsuccessful PostHog responses', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [{ person_id: 'person-a', timestamp_ms: 1_000, event_kind: 'cli_command', command_path: '', total_events: 1 }], + }) + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-09-01T00:00:00.000Z', + '2026-09-02T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics row is invalid') + + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: false, + failureReason: 'unavailable', + rows: [], + }) + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-09-01T00:00:00.000Z', + '2026-09-02T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics PostHog query failed') + }) + + it('validates total metadata directly', () => { + expect(() => assertFrontendOnboardingDailySetupCliEventTotal(1, 1)).not.toThrow() + expect(() => assertFrontendOnboardingDailySetupCliEventTotal('1', 1)).toThrow('daily Setup CLI analytics query returned invalid total metadata') + }) +}) +``` + +- [ ] **Step 2: Run the adapter test and verify the missing module failure** + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts +``` + +Expected: FAIL because the adapter module does not exist. + +- [ ] **Step 3: Implement the HogQL builder, parser, limit guard, and query wrapper** + +Create `supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts`: + +```ts +import type { Context } from 'hono' +import type { + FrontendOnboardingDailySetupCliEvent, + FrontendOnboardingDailySetupCliEventKind, +} from './frontend_onboarding_daily_setup_cli_outcomes_model.ts' +import { cloudlogErr } from './logging.ts' +import { queryPosthogHogql } from './posthog_read.ts' + +const INVALID_TOTAL_ERROR = 'daily Setup CLI analytics query returned invalid total metadata' +const LIMIT_EXCEEDED_ERROR = 'daily Setup CLI analytics query exceeded event limit' +const INVALID_ROW_ERROR = 'daily Setup CLI analytics row is invalid' + +export const FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT = 50_000 + +function sqlStr(value: string): string { + return `'${value.replace(/'/g, '\'\'')}'` +} + +export function assertFrontendOnboardingDailySetupCliEventTotal( + totalEvents: unknown, + limit = FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT, +): number { + if (typeof totalEvents !== 'number' + || !Number.isFinite(totalEvents) + || !Number.isInteger(totalEvents) + || totalEvents < 0) { + throw new Error(INVALID_TOTAL_ERROR) + } + if (totalEvents > limit) + throw new Error(LIMIT_EXCEEDED_ERROR) + return totalEvents +} + +export function buildFrontendOnboardingDailySetupCliHogql( + startDate: string, + endDate: string, + followupEndDate: string, +): string { + return ` + WITH setup_people AS ( + SELECT person_id + FROM events + WHERE event = 'onboarding_step_viewed' + AND JSONExtractString(toString(properties), 'flow') = 'pre_org' + AND toIntOrZero(toString(properties.onboarding_version)) = 2 + AND JSONExtractString(toString(properties), 'step') = 'setup' + AND timestamp >= parseDateTimeBestEffort(${sqlStr(startDate)}) + AND timestamp < parseDateTimeBestEffort(${sqlStr(endDate)}) + GROUP BY person_id + ), relevant_events AS ( + SELECT + toString(source.person_id) AS person_id, + toUnixTimestamp64Milli(source.timestamp) AS timestamp_ms, + multiIf( + source.event = 'onboarding_step_viewed', 'setup', + source.event = 'onboarding_cli_command_copied', 'cli_copy', + source.event = 'onboarding_ai_instructions_copied', 'ai_copy', + 'cli_command' + ) AS event_kind, + if( + source.event = 'CLI Command Invoked', + JSONExtractString(toString(source.properties), 'command_path'), + '' + ) AS command_path + FROM events AS source + INNER JOIN setup_people AS cohort ON source.person_id = cohort.person_id + WHERE source.timestamp >= parseDateTimeBestEffort(${sqlStr(startDate)}) + AND source.timestamp < parseDateTimeBestEffort(${sqlStr(followupEndDate)}) + AND ( + (source.event = 'onboarding_step_viewed' + AND JSONExtractString(toString(source.properties), 'flow') = 'pre_org' + AND toIntOrZero(toString(source.properties.onboarding_version)) = 2 + AND JSONExtractString(toString(source.properties), 'step') = 'setup') + OR (source.event IN ('onboarding_cli_command_copied', 'onboarding_ai_instructions_copied') + AND JSONExtractString(toString(source.properties), 'flow') = 'pre_org' + AND toIntOrZero(toString(source.properties.onboarding_version)) = 2 + AND JSONExtractString(toString(source.properties), 'step') = 'setup') + OR source.event = 'CLI Command Invoked' + ) + ) + SELECT + person_id, + timestamp_ms, + event_kind, + command_path, + count() OVER () AS total_events + FROM relevant_events + ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC + LIMIT ${FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT}` +} + +function mapRows(rows: Record[]): FrontendOnboardingDailySetupCliEvent[] { + const validKinds = new Set(['setup', 'cli_copy', 'ai_copy', 'cli_command']) + return rows.map((row) => { + const personId = typeof row.person_id === 'string' ? row.person_id.trim() : '' + const timestampMs = typeof row.timestamp_ms === 'number' ? row.timestamp_ms : Number(row.timestamp_ms) + const kind = typeof row.event_kind === 'string' ? row.event_kind as FrontendOnboardingDailySetupCliEventKind : null + const commandPath = typeof row.command_path === 'string' ? row.command_path.trim() : '' + + if (!personId || !Number.isFinite(timestampMs) || timestampMs <= 0 || !kind || !validKinds.has(kind)) + throw new Error(INVALID_ROW_ERROR) + if (kind === 'cli_command' && !commandPath) + throw new Error(INVALID_ROW_ERROR) + + return { + personId, + timestampMs, + kind, + ...(kind === 'cli_command' ? { commandPath } : {}), + } + }) +} + +export async function getFrontendOnboardingDailySetupCliEvents( + c: Context, + startDate: string, + endDate: string, + followupEndDate: string, +): Promise { + const posthog = await queryPosthogHogql( + c, + buildFrontendOnboardingDailySetupCliHogql(startDate, endDate, followupEndDate), + ) + if (!posthog.configured || !posthog.connected || posthog.failureReason !== null) + throw new Error('daily Setup CLI analytics PostHog query failed') + + if (posthog.rows.length > 0) { + try { + assertFrontendOnboardingDailySetupCliEventTotal(posthog.rows[0].total_events) + } + catch (error) { + cloudlogErr({ + requestId: c.get('requestId'), + message: error instanceof Error && error.message === LIMIT_EXCEEDED_ERROR + ? 'frontend_onboarding_daily_setup_cli_event_limit_exceeded' + : 'frontend_onboarding_daily_setup_cli_invalid_total_events', + event_limit: FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT, + total_events: posthog.rows[0].total_events, + returned_rows: posthog.rows.length, + }) + throw error + } + } + + return mapRows(posthog.rows) +} +``` + +- [ ] **Step 4: Run the adapter tests and verify they pass** + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts +``` + +Expected: PASS with query boundaries, identity join, row mapping, and fail-closed limits covered. + +- [ ] **Step 5: Commit the PostHog adapter** + +```bash +git add supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts +git commit -m "feat(admin): query daily Setup CLI events" +``` + +--- + +### Task 4: Integrate daily outcomes into the existing analytics endpoint + +**Files:** +- Modify: `supabase/functions/_backend/utils/frontend_onboarding_analytics.ts:1-10,227-283` +- Modify: `tests/frontend-onboarding-analytics.unit.test.ts:1-332` + +- [ ] **Step 1: Add a failing endpoint integration test** + +In `tests/frontend-onboarding-analytics.unit.test.ts`, add this test inside `describe('getAdminFrontendOnboardingAnalytics')`: + +```ts + it('adds daily Setup CLI outcomes without changing the aggregate outcome field', async () => { + const start = '2026-08-01T00:00:00.000Z' + const end = '2026-08-03T00:00:00.000Z' + const startMs = Date.parse(start) + + queryPosthogHogqlMock + .mockResolvedValueOnce({ configured: true, connected: true, failureReason: null, rows: [] }) + .mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: 'person-a', timestamp_ms: startMs + 1_000, event_kind: 'setup', command_path: '', total_events: 3 }, + { person_id: 'person-a', timestamp_ms: startMs + 2_000, event_kind: 'cli_copy', command_path: '', total_events: 3 }, + { person_id: 'person-a', timestamp_ms: startMs + 3_000, event_kind: 'cli_command', command_path: 'init', total_events: 3 }, + ], + }) + + const result = await getAdminFrontendOnboardingAnalytics(createContext(), start, end) + + expect(result.daily_setup_cli_outcomes[0].first_time.cli_copy_init).toBe(1) + expect(result.daily_setup_cli_outcomes[1].first_time.cli_copy_init).toBe(0) + expect(result.v2_setup_cli_outcomes).toEqual({ + total_users: 0, + cli_only: 0, + cli_and_ai_instructions: 0, + no_cli: 0, + }) + }) + + it('fails the analytics request when the dedicated daily query fails', async () => { + queryPosthogHogqlMock + .mockResolvedValueOnce({ configured: true, connected: true, failureReason: null, rows: [] }) + .mockResolvedValueOnce({ configured: true, connected: false, failureReason: 'unavailable', rows: [] }) + + await expect(getAdminFrontendOnboardingAnalytics( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics PostHog query failed') + }) +``` + +Update the boundary tests so successful requests expect two PostHog calls. Assert that the existing query still ends at `end + 48 hours`, while the second query starts at the selected start and ends at `end + 24 hours`: + +```ts + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) + expect(queryPosthogHogqlMock.mock.calls[0][1]).toContain( + `timestamp < parseDateTimeBestEffort('${new Date(Date.parse(end) + 2 * DAY_MS).toISOString()}')`, + ) + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain( + `source.timestamp >= parseDateTimeBestEffort('${start}')`, + ) + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain( + `source.timestamp < parseDateTimeBestEffort('${new Date(Date.parse(end) + DAY_MS).toISOString()}')`, + ) +``` + +- [ ] **Step 2: Run the endpoint tests and verify the missing response field failure** + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-analytics.unit.test.ts +``` + +Expected: FAIL because `daily_setup_cli_outcomes` is absent and successful calls still execute one query. + +- [ ] **Step 3: Wire the dedicated query and pure builder into the endpoint** + +Add imports in `frontend_onboarding_analytics.ts`: + +```ts +import { getFrontendOnboardingDailySetupCliEvents } from './frontend_onboarding_daily_setup_cli_outcomes.ts' +import { buildFrontendOnboardingDailySetupCliOutcomes } from './frontend_onboarding_daily_setup_cli_outcomes_model.ts' +``` + +Replace the single existing PostHog call in `getAdminFrontendOnboardingAnalytics` with: + +```ts + const normalizedStartDate = new Date(startMs).toISOString() + const normalizedEndDate = new Date(endMs).toISOString() + const dailyFollowupEndDate = new Date(endMs + FRONTEND_ONBOARDING_FOLLOWUP_MS).toISOString() + + const [posthog, dailySetupCliEvents] = await Promise.all([ + queryPosthogHogql( + c, + buildFrontendOnboardingHogql( + new Date(queryStartMs).toISOString(), + normalizedEndDate, + new Date(followupEndMs).toISOString(), + ), + ), + getFrontendOnboardingDailySetupCliEvents( + c, + normalizedStartDate, + normalizedEndDate, + dailyFollowupEndDate, + ), + ]) +``` + +After `buildFrontendOnboardingAnalytics`, create the daily result: + +```ts + const dailySetupCliOutcomes = buildFrontendOnboardingDailySetupCliOutcomes( + dailySetupCliEvents, + startMs, + endMs, + ) +``` + +Add it to the returned object without editing `buildV2SetupCliOutcomes` or `v2_setup_cli_outcomes`: + +```ts + return { + ...analytics, + daily_setup_cli_outcomes: dailySetupCliOutcomes, + posthog_configured: posthog.configured, + posthog_connected: posthog.connected, + } +``` + +- [ ] **Step 4: Update existing one-query expectations and run both backend suites** + +Change successful `toHaveBeenCalledTimes(1)` expectations in `tests/frontend-onboarding-analytics.unit.test.ts` to `2`. Keep invalid-bound tests at zero calls. Where a test inspects the current aggregate query, continue using `mock.calls[0][1]`; use `mock.calls[1][1]` only for the dedicated daily query. + +Run: + +```bash +bunx vitest run tests/frontend-onboarding-analytics.unit.test.ts tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts tests/frontend-onboarding-analytics-model.unit.test.ts +``` + +Expected: PASS. The existing aggregate-model suite must remain unchanged and green. + +- [ ] **Step 5: Commit the endpoint integration** + +```bash +git add supabase/functions/_backend/utils/frontend_onboarding_analytics.ts tests/frontend-onboarding-analytics.unit.test.ts +git commit -m "feat(admin): expose daily Setup CLI outcomes" +``` + +--- + +### Task 5: Extend `AdminStackedBarChart` for optional paired stacks + +**Files:** +- Modify: `src/components/admin/adminStackedBarChart.ts:1-81` +- Modify: `src/components/admin/AdminStackedBarChart.vue:1-81` +- Modify: `tests/admin-stacked-bar-chart.unit.test.ts:1-63` + +- [ ] **Step 1: Add failing grouped-stack helper tests** + +Update imports in `tests/admin-stacked-bar-chart.unit.test.ts` to include: + +```ts + buildAdminStackedBarLegendItems, + getAdminStackedBarTooltipTotal, + toggleAdminStackedBarLegendGroup, +``` + +Append these tests: + +```ts + it.concurrent('preserves optional stack metadata and uses stack-scoped tooltip totals', () => { + const grouped = buildAdminStackedBarChartData(['Aug 1'], [ + { label: 'No action', data: [3], color: '#94a3b8', stack: 'first_time', stackLabel: 'First-time' }, + { label: 'No action', data: [2], color: '#94a3b8', stack: 'returning', stackLabel: 'Returning' }, + { label: 'CLI + init', data: [1], color: '#10b981', stack: 'first_time', stackLabel: 'First-time' }, + ]) + + expect(grouped.datasets.map(dataset => ({ + stack: dataset.stack, + stackLabel: dataset.stackLabel, + }))).toEqual([ + { stack: 'first_time', stackLabel: 'First-time' }, + { stack: 'returning', stackLabel: 'Returning' }, + { stack: 'first_time', stackLabel: 'First-time' }, + ]) + expect(getAdminStackedBarTooltipTotal(grouped.datasets, 0, 0)).toBe(4) + expect(getAdminStackedBarTooltipTotal(grouped.datasets, 1, 0)).toBe(2) + }) + + it.concurrent('deduplicates active grouped legend labels and removes zero-only categories', () => { + const grouped = buildAdminStackedBarChartData(['Aug 1', 'Aug 2'], [ + { label: 'No action', data: [3, 0], color: '#94a3b8', stack: 'first_time' }, + { label: 'No action', data: [0, 1], color: '#94a3b8', stack: 'returning' }, + { label: 'Never happened', data: [0, 0], color: '#ef4444', stack: 'first_time' }, + { label: 'Never happened', data: [0, 0], color: '#ef4444', stack: 'returning' }, + ]) + const chart = { + data: grouped, + isDatasetVisible: () => true, + } as any + + expect(buildAdminStackedBarLegendItems(chart).map(item => item.text)).toEqual(['No action']) + }) + + it.concurrent('toggles every lifecycle dataset represented by one legend item', () => { + const visibility = [true, true, true] + const chart = { + data: { datasets: [{ label: 'No action' }, { label: 'No action' }, { label: 'CLI + init' }] }, + isDatasetVisible: (index: number) => visibility[index], + setDatasetVisibility: (index: number, visible: boolean) => { visibility[index] = visible }, + update: () => undefined, + } as any + + toggleAdminStackedBarLegendGroup(chart, 'No action') + expect(visibility).toEqual([false, false, true]) + toggleAdminStackedBarLegendGroup(chart, 'No action') + expect(visibility).toEqual([true, true, true]) + }) + + it.concurrent('uses grouped interaction only when requested', () => { + expect((buildAdminStackedBarChartOptions(false, false) as any).interaction.mode).toBe('index') + expect((buildAdminStackedBarChartOptions(false, true) as any).interaction.mode).toBe('nearest') + }) +``` + +- [ ] **Step 2: Run the chart tests and verify the missing helper failures** + +Run: + +```bash +bunx vitest run tests/admin-stacked-bar-chart.unit.test.ts +``` + +Expected: FAIL because grouped metadata and helper exports are absent. + +- [ ] **Step 3: Implement grouped datasets, tooltips, and legends in the chart helper** + +Extend `AdminStackedBarDataset`: + +```ts +export interface AdminStackedBarDataset { + label: string + data: number[] + color: string + stack?: string + stackLabel?: string +} +``` + +In `buildAdminStackedBarChartData`, pass through both optional values: + +```ts + stack: item.stack, + stackLabel: item.stackLabel, +``` + +Change the Chart.js type import to: + +```ts +import type { Chart as ChartJs, ChartData, ChartOptions, LegendItem } from 'chart.js' +``` + +Add these reusable chart-data types below `AdminStackedBarDataset`, and change `buildAdminStackedBarChartData` plus `applyAdminStackedBarAccessibleBorders` to return `AdminStackedBarChartData`: + +```ts +export type AdminStackedBarChartDataset = ChartData<'bar'>['datasets'][number] & { stackLabel?: string } +export type AdminStackedBarChartData = Omit, 'datasets'> & { datasets: AdminStackedBarChartDataset[] } +``` + +Add these helpers before `buildAdminStackedBarChartOptions`: + +```ts +type AdminLegendDataChart = Pick, 'data' | 'isDatasetVisible'> +type AdminLegendToggleChart = Pick, 'data' | 'isDatasetVisible' | 'setDatasetVisibility' | 'update'> + +function datasetTotal(dataset: AdminStackedBarChartDataset): number { + return dataset.data.reduce((sum, value) => sum + (typeof value === 'number' ? value : 0), 0) +} + +export function getAdminStackedBarTooltipTotal( + datasets: readonly AdminStackedBarChartDataset[], + datasetIndex: number, + dataIndex: number, +): number { + const activeStack = datasets[datasetIndex]?.stack + return datasets.reduce((sum, dataset) => { + if (activeStack !== undefined && dataset.stack !== activeStack) + return sum + return sum + Number(dataset.data[dataIndex] ?? 0) + }, 0) +} + +export function buildAdminStackedBarLegendItems(chart: AdminLegendDataChart): LegendItem[] { + const seen = new Set() + return chart.data.datasets.flatMap((rawDataset, datasetIndex) => { + const dataset = rawDataset as AdminStackedBarChartDataset + const label = dataset.label ?? '' + if (!label || seen.has(label) || datasetTotal(dataset) === 0) + return [] + const matching = chart.data.datasets + .map(candidate => candidate as AdminStackedBarChartDataset) + .filter(candidate => candidate.label === label) + if (matching.every(candidate => datasetTotal(candidate) === 0)) + return [] + seen.add(label) + return [{ + text: label, + fillStyle: typeof dataset.backgroundColor === 'string' ? dataset.backgroundColor : '#94a3b8', + strokeStyle: typeof dataset.borderColor === 'string' ? dataset.borderColor : '#94a3b8', + lineWidth: typeof dataset.borderWidth === 'number' ? dataset.borderWidth : 0, + hidden: chart.data.datasets + .map((candidate, index) => ({ candidate, index })) + .filter(({ candidate }) => candidate.label === label) + .every(({ index }) => !chart.isDatasetVisible(index)), + datasetIndex, + pointStyle: 'circle' as const, + }] + }) +} + +export function toggleAdminStackedBarLegendGroup(chart: AdminLegendToggleChart, label: string): void { + const indexes = chart.data.datasets + .map((dataset, index) => ({ dataset, index })) + .filter(({ dataset }) => dataset.label === label) + .map(({ index }) => index) + const show = indexes.every(index => !chart.isDatasetVisible(index)) + for (const index of indexes) + chart.setDatasetVisibility(index, show) + chart.update() +} +``` + +Change the options signature and interaction mode: + +```ts +export function buildAdminStackedBarChartOptions( + isDark: boolean, + groupedStacks = false, +): ChartOptions<'bar'> { +``` + +```ts + interaction: { + mode: groupedStacks ? 'nearest' : 'index', + intersect: false, + }, +``` + +Inside `plugins.legend`, retain the existing label styling and add grouped-only generation/toggling: + +```ts + labels: { + color: textColor, + boxHeight: 10, + boxWidth: 10, + font: { size: 12, weight: 500 }, + padding: 18, + usePointStyle: true, + ...(groupedStacks ? { generateLabels: buildAdminStackedBarLegendItems } : {}), + }, + ...(groupedStacks + ? { onClick: (_event, item, legend) => toggleAdminStackedBarLegendGroup(legend.chart, item.text) } + : {}), +``` + +Replace the tooltip total calculation with stack-aware logic and prefix grouped labels: + +```ts + const total = getAdminStackedBarTooltipTotal( + context.chart.data.datasets, + context.datasetIndex, + context.dataIndex, + ) + const dataset = context.dataset as AdminStackedBarChartDataset + const label = dataset.stackLabel + ? `${dataset.stackLabel} · ${dataset.label ?? ''}` + : dataset.label ?? '' + return formatAdminStackedBarTooltip(label, value, total) +``` + +- [ ] **Step 4: Pass grouped metadata through the Vue wrapper without changing legacy callers** + +Extend the local `DataSeries` interface in `AdminStackedBarChart.vue`: + +```ts +interface DataSeries { + label: string + data: Array<{ date: string, value: number }> + color: string + stack?: string + stackLabel?: string +} +``` + +Pass the fields into datasets: + +```ts + stack: item.stack, + stackLabel: item.stackLabel, +``` + +Detect grouped mode and pass it to the options builder: + +```ts +const hasGroupedStacks = computed(() => props.series.some(item => item.stack !== undefined)) +const chartOptions = computed(() => buildAdminStackedBarChartOptions(isDark.value, hasGroupedStacks.value)) +``` + +- [ ] **Step 5: Run the chart tests and verify legacy and grouped behavior pass together** + +Run: + +```bash +bunx vitest run tests/admin-stacked-bar-chart.unit.test.ts +``` + +Expected: PASS. Existing non-grouped options remain `index` mode, while grouped data uses paired stacks and a dynamic legend. + +- [ ] **Step 6: Commit the reusable chart extension** + +```bash +git add src/components/admin/adminStackedBarChart.ts src/components/admin/AdminStackedBarChart.vue tests/admin-stacked-bar-chart.unit.test.ts +git commit -m "feat(admin): support grouped stacked bar charts" +``` + +--- + +### Task 6: Add frontend API types and the dynamic paired-series mapper + +**Files:** +- Modify: `src/services/adminFrontendOnboarding.ts:1-165` +- Modify: `tests/admin-frontend-onboarding-dashboard.unit.test.ts:1-165` + +- [ ] **Step 1: Extend the fixture and write a failing dynamic-series test** + +Add `buildFrontendOnboardingDailySetupCliSeries` to the service imports in `tests/admin-frontend-onboarding-dashboard.unit.test.ts`. + +Add this field to the shared `analytics` fixture after `v2_setup_cli_outcomes`: + +```ts + daily_setup_cli_outcomes: [{ + date: '2026-08-10', + first_time: { + cli_copy_init: 2, + ai_copy_init: 0, + both_copy_init: 0, + no_copy_init: 0, + cli_copy_other_cli: 0, + ai_copy_other_cli: 0, + both_copy_other_cli: 0, + no_copy_other_cli: 0, + cli_copy_no_cli: 0, + ai_copy_no_cli: 0, + both_copy_no_cli: 0, + no_action: 0, + }, + returning: { + cli_copy_init: 0, + ai_copy_init: 0, + both_copy_init: 0, + no_copy_init: 0, + cli_copy_other_cli: 0, + ai_copy_other_cli: 0, + both_copy_other_cli: 0, + no_copy_other_cli: 0, + cli_copy_no_cli: 0, + ai_copy_no_cli: 1, + both_copy_no_cli: 0, + no_action: 0, + }, + }], +``` + +Add the mapper test: + +```ts + it.concurrent('builds paired stacks only for outcome categories present in the range', () => { + const labels = { + cli_copy_init: 'CLI copy + init', + ai_copy_init: 'AI copy + init', + both_copy_init: 'Both copied + init', + no_copy_init: 'No copy + init', + cli_copy_other_cli: 'CLI copy + other CLI', + ai_copy_other_cli: 'AI copy + other CLI', + both_copy_other_cli: 'Both copied + other CLI', + no_copy_other_cli: 'No copy + other CLI', + cli_copy_no_cli: 'CLI copied · no CLI run', + ai_copy_no_cli: 'AI copied · no CLI run', + both_copy_no_cli: 'Both copied · no CLI run', + no_action: 'No action', + } + + const series = buildFrontendOnboardingDailySetupCliSeries( + analytics.daily_setup_cli_outcomes, + labels, + 'First-time', + 'Returning', + ) + + expect(series).toHaveLength(4) + expect(series.map(item => ({ label: item.label, stack: item.stack, stackLabel: item.stackLabel, value: item.data[0].value }))).toEqual([ + { label: 'CLI copy + init', stack: 'first_time', stackLabel: 'First-time', value: 2 }, + { label: 'CLI copy + init', stack: 'returning', stackLabel: 'Returning', value: 0 }, + { label: 'AI copied · no CLI run', stack: 'first_time', stackLabel: 'First-time', value: 0 }, + { label: 'AI copied · no CLI run', stack: 'returning', stackLabel: 'Returning', value: 1 }, + ]) + expect(series.some(item => item.label === 'No action')).toBe(false) + }) +``` + +- [ ] **Step 2: Run the dashboard service tests and verify the missing type/mapper failures** + +Run: + +```bash +bunx vitest run tests/admin-frontend-onboarding-dashboard.unit.test.ts +``` + +Expected: FAIL because the response type lacks `daily_setup_cli_outcomes` and the mapper is not exported. + +- [ ] **Step 3: Add API types, outcome order/colors, and the mapper** + +Add near the top of `src/services/adminFrontendOnboarding.ts`: + +```ts +export const FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS = [ + 'cli_copy_init', + 'ai_copy_init', + 'both_copy_init', + 'no_copy_init', + 'cli_copy_other_cli', + 'ai_copy_other_cli', + 'both_copy_other_cli', + 'no_copy_other_cli', + 'cli_copy_no_cli', + 'ai_copy_no_cli', + 'both_copy_no_cli', + 'no_action', +] as const + +export type FrontendOnboardingDailySetupCliOutcomeKey = typeof FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS[number] +export type FrontendOnboardingDailySetupCliOutcomeCounts = Record + +export interface FrontendOnboardingDailySetupCliOutcomePoint { + date: string + first_time: FrontendOnboardingDailySetupCliOutcomeCounts + returning: FrontendOnboardingDailySetupCliOutcomeCounts +} +``` + +Add to `FrontendOnboardingAnalytics`: + +```ts + daily_setup_cli_outcomes: FrontendOnboardingDailySetupCliOutcomePoint[] +``` + +Extend `FrontendOnboardingDailySeries`: + +```ts + stack?: 'first_time' | 'returning' + stackLabel?: string +``` + +Add the stable color map and mapper below `buildFrontendOnboardingDailySeries`: + +```ts +const DAILY_SETUP_CLI_OUTCOME_COLORS: Record = { + cli_copy_init: '#047857', + ai_copy_init: '#10b981', + both_copy_init: '#34d399', + no_copy_init: '#86efac', + cli_copy_other_cli: '#1d4ed8', + ai_copy_other_cli: '#3b82f6', + both_copy_other_cli: '#7c3aed', + no_copy_other_cli: '#a78bfa', + cli_copy_no_cli: '#c2410c', + ai_copy_no_cli: '#f97316', + both_copy_no_cli: '#fbbf24', + no_action: '#94a3b8', +} + +export function buildFrontendOnboardingDailySetupCliSeries( + points: readonly FrontendOnboardingDailySetupCliOutcomePoint[], + labels: Record, + firstTimeLabel: string, + returningLabel: string, +): FrontendOnboardingDailySeries[] { + const activeKeys = FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS.filter(key => points.some( + point => point.first_time[key] + point.returning[key] > 0, + )) + + return activeKeys.flatMap(key => ([ + { + label: labels[key], + color: DAILY_SETUP_CLI_OUTCOME_COLORS[key], + stack: 'first_time' as const, + stackLabel: firstTimeLabel, + data: points.map(point => ({ date: point.date, value: point.first_time[key] })), + }, + { + label: labels[key], + color: DAILY_SETUP_CLI_OUTCOME_COLORS[key], + stack: 'returning' as const, + stackLabel: returningLabel, + data: points.map(point => ({ date: point.date, value: point.returning[key] })), + }, + ])) +} +``` + +- [ ] **Step 4: Run the dashboard service tests and verify the dynamic mapper passes** + +Run: + +```bash +bunx vitest run tests/admin-frontend-onboarding-dashboard.unit.test.ts +``` + +Expected: PASS, including the new mapper test and the still-unchanged page structure assertions. + +- [ ] **Step 5: Commit the typed mapper** + +```bash +git add src/services/adminFrontendOnboarding.ts tests/admin-frontend-onboarding-dashboard.unit.test.ts +git commit -m "feat(admin): map daily Setup CLI chart series" +``` + +--- + +### Task 7: Add the separate daily chart card and translations + +**Files:** +- Modify: `src/pages/admin/dashboard/frontend-onboarding.vue:7-136,456-487` +- Modify: `messages/en.json` +- Modify: `tests/admin-frontend-onboarding-dashboard.unit.test.ts:360-521` + +- [ ] **Step 1: Add failing structural and translation assertions** + +Update the existing component-count expectations: + +```ts + expect(source.match(/)/g)).toHaveLength(6) + expect(source.match(/)/g)).toHaveLength(1) + expect(source.match(/)/g)).toHaveLength(2) +``` + +Add the new card ordering assertion while preserving the aggregate chart assertion: + +```ts + const dailyCliOutcomeIndex = source.indexOf(`t('frontend-onboarding-daily-setup-cli-outcomes-v2')`) + expect(source).toContain(':values="setupCliOutcomeValues"') + expect(source).toContain('visibleAnalytics.value?.v2_setup_cli_outcomes') + expect(cliOutcomeIndex).toBeLessThan(dailyCliOutcomeIndex) + expect(dailyCliOutcomeIndex).toBeLessThan(legacyIndex) +``` + +Add translation assertions to the English-label test: + +```ts + expect(messages['frontend-onboarding-daily-setup-cli-outcomes-v2']).toBe('Daily Setup → CLI outcomes (v2)') + expect(messages['frontend-onboarding-daily-setup-cli-first-time']).toBe('First-time') + expect(messages['frontend-onboarding-daily-setup-cli-returning']).toBe('Returning') + expect(messages['frontend-onboarding-daily-setup-cli-no-action']).toBe('No action') +``` + +- [ ] **Step 2: Run the dashboard tests and verify the card/translation failures** + +Run: + +```bash +bunx vitest run tests/admin-frontend-onboarding-dashboard.unit.test.ts +``` + +Expected: FAIL because the page still has five cards/one stacked chart and the new translation keys are missing. + +- [ ] **Step 3: Add translated labels and paired-series computed state to the page** + +Add `FrontendOnboardingDailySetupCliOutcomeKey` to the existing type-only service import in `frontend-onboarding.vue`, and add the mapper to the existing value import: + +```ts +import type { + FrontendOnboardingAnalytics, + FrontendOnboardingDailySetupCliOutcomeKey, +} from '~/services/adminFrontendOnboarding' +``` + +```ts + buildFrontendOnboardingDailySetupCliSeries, +``` + +After `hasSetupCliOutcomeData`, add: + +```ts +const dailySetupCliOutcomeLabels = computed>(() => ({ + cli_copy_init: t('frontend-onboarding-daily-setup-cli-cli-copy-init'), + ai_copy_init: t('frontend-onboarding-daily-setup-cli-ai-copy-init'), + both_copy_init: t('frontend-onboarding-daily-setup-cli-both-copy-init'), + no_copy_init: t('frontend-onboarding-daily-setup-cli-no-copy-init'), + cli_copy_other_cli: t('frontend-onboarding-daily-setup-cli-cli-copy-other-cli'), + ai_copy_other_cli: t('frontend-onboarding-daily-setup-cli-ai-copy-other-cli'), + both_copy_other_cli: t('frontend-onboarding-daily-setup-cli-both-copy-other-cli'), + no_copy_other_cli: t('frontend-onboarding-daily-setup-cli-no-copy-other-cli'), + cli_copy_no_cli: t('frontend-onboarding-daily-setup-cli-cli-copy-no-cli'), + ai_copy_no_cli: t('frontend-onboarding-daily-setup-cli-ai-copy-no-cli'), + both_copy_no_cli: t('frontend-onboarding-daily-setup-cli-both-copy-no-cli'), + no_action: t('frontend-onboarding-daily-setup-cli-no-action'), +})) +const dailySetupCliSeries = computed(() => buildFrontendOnboardingDailySetupCliSeries( + visibleAnalytics.value?.daily_setup_cli_outcomes ?? [], + dailySetupCliOutcomeLabels.value, + t('frontend-onboarding-daily-setup-cli-first-time'), + t('frontend-onboarding-daily-setup-cli-returning'), +)) +const hasDailySetupCliOutcomeData = computed(() => dailySetupCliSeries.value.length > 0) +``` + +- [ ] **Step 4: Add the separate chart card below the existing aggregate card** + +Insert this block immediately after the existing `AdminBarChart` outcome `ChartCard` and before the legacy v1 section: + +```vue + + + + +``` + +Do not add `:total` or `:unit`; the measure is person-days and returning people can appear on multiple dates. + +- [ ] **Step 5: Add all English translations** + +Add these keys to `messages/en.json` next to the existing Setup CLI outcome keys: + +```json +"frontend-onboarding-daily-setup-cli-outcomes-v2": "Daily Setup → CLI outcomes (v2)", +"frontend-onboarding-daily-setup-cli-outcomes-description": "Each person is counted once per UTC day. Left: first-time Setup views; right: returning views. Actions are attributed for up to 24 hours.", +"frontend-onboarding-daily-setup-cli-first-time": "First-time", +"frontend-onboarding-daily-setup-cli-returning": "Returning", +"frontend-onboarding-daily-setup-cli-cli-copy-init": "CLI copy + init", +"frontend-onboarding-daily-setup-cli-ai-copy-init": "AI copy + init", +"frontend-onboarding-daily-setup-cli-both-copy-init": "Both copied + init", +"frontend-onboarding-daily-setup-cli-no-copy-init": "No copy + init", +"frontend-onboarding-daily-setup-cli-cli-copy-other-cli": "CLI copy + other CLI", +"frontend-onboarding-daily-setup-cli-ai-copy-other-cli": "AI copy + other CLI", +"frontend-onboarding-daily-setup-cli-both-copy-other-cli": "Both copied + other CLI", +"frontend-onboarding-daily-setup-cli-no-copy-other-cli": "No copy + other CLI", +"frontend-onboarding-daily-setup-cli-cli-copy-no-cli": "CLI copied · no CLI run", +"frontend-onboarding-daily-setup-cli-ai-copy-no-cli": "AI copied · no CLI run", +"frontend-onboarding-daily-setup-cli-both-copy-no-cli": "Both copied · no CLI run", +"frontend-onboarding-daily-setup-cli-no-action": "No action" +``` + +- [ ] **Step 6: Run the complete frontend feature tests** + +Run: + +```bash +bunx vitest run tests/admin-stacked-bar-chart.unit.test.ts tests/admin-frontend-onboarding-dashboard.unit.test.ts +``` + +Expected: PASS with six `ChartCard` instances, two `AdminStackedBarChart` instances, the existing aggregate `AdminBarChart`, the new card below it, and all mapper/legend behavior green. + +- [ ] **Step 7: Commit the dashboard card** + +```bash +git add src/pages/admin/dashboard/frontend-onboarding.vue src/services/adminFrontendOnboarding.ts messages/en.json tests/admin-frontend-onboarding-dashboard.unit.test.ts +git commit -m "feat(admin): chart daily Setup CLI outcomes" +``` + +--- + +### Task 8: Run full validation and inspect the final diff + +**Files:** +- Verify all files listed in Tasks 1-7 + +- [ ] **Step 1: Run formatting/lint validation for frontend and backend** + +Run: + +```bash +bun lint +bun lint:backend +``` + +Expected: both commands exit 0 with no lint errors. + +- [ ] **Step 2: Run TypeScript validation** + +Run: + +```bash +bun typecheck +``` + +Expected: exit 0 with no frontend, backend, or CLI type errors. + +- [ ] **Step 3: Run the complete unit suite** + +Run: + +```bash +bun test:unit +``` + +Expected: all unit tests pass, including the four onboarding analytics suites and the stacked-chart/dashboard suites. + +- [ ] **Step 4: Verify formatting and scope** + +Run: + +```bash +git diff --check 07ba500a2..HEAD +git status --short +git diff 07ba500a2..HEAD -- supabase/functions/_backend/utils/frontend_onboarding_analytics_model.ts +``` + +Expected: + +- `git diff --check` prints nothing. +- `git status --short` shows only the pre-existing unrelated `codedb.snapshot` modification, if it is still present. +- The final diff command prints nothing, proving the existing aggregate analytics model and `buildV2SetupCliOutcomes` were not modified. + +- [ ] **Step 5: Review the implementation against the design acceptance criteria** + +Confirm all of these from tests and the final diff: + +- the old aggregate card and `v2_setup_cli_outcomes` remain intact; +- the new API field contains every selected UTC date; +- every displayed person-day is in exactly one category; +- a later Setup anchor owns overlapping actions; +- a follow-up-tail anchor truncates the final displayed day; +- grouped tooltips use a single lifecycle denominator; +- zero-only outcome categories are absent from the legend; +- the daily card has no misleading range-level people total. From 78959f0736037b91a96d42ad9e31c3c40b245218 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 15:47:36 +0200 Subject: [PATCH 03/20] feat(admin): define daily Setup CLI outcomes --- ...boarding_daily_setup_cli_outcomes_model.ts | 80 +++++++++++++++++++ ...aily-setup-cli-outcomes-model.unit.test.ts | 51 ++++++++++++ 2 files changed, 131 insertions(+) create mode 100644 supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts create mode 100644 tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts new file mode 100644 index 0000000000..8ab468387c --- /dev/null +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts @@ -0,0 +1,80 @@ +export const FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS = [ + 'cli_copy_init', + 'ai_copy_init', + 'both_copy_init', + 'no_copy_init', + 'cli_copy_other_cli', + 'ai_copy_other_cli', + 'both_copy_other_cli', + 'no_copy_other_cli', + 'cli_copy_no_cli', + 'ai_copy_no_cli', + 'both_copy_no_cli', + 'no_action', +] as const + +export type FrontendOnboardingDailySetupCliOutcomeKey = typeof FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS[number] +export type FrontendOnboardingDailySetupCliOutcomeLifecycle = 'first_time' | 'returning' +export type FrontendOnboardingDailySetupCliOutcomeEventKind = 'setup' | 'cli_copy' | 'ai_copy' | 'cli_command' + +export interface FrontendOnboardingDailySetupCliOutcomeEvent { + personId: string + timestampMs: number + kind: FrontendOnboardingDailySetupCliOutcomeEventKind + commandPath?: string +} + +export interface FrontendOnboardingDailySetupCliOutcomeSignals { + cliCopied: boolean + aiCopied: boolean + initRun: boolean + otherCliRun: boolean +} + +export type FrontendOnboardingDailySetupCliOutcomeCounts = Record + +export interface FrontendOnboardingDailySetupCliOutcomeDailyPoint { + date: string + first_time: FrontendOnboardingDailySetupCliOutcomeCounts + returning: FrontendOnboardingDailySetupCliOutcomeCounts +} + +export function createFrontendOnboardingDailySetupCliOutcomeCounts(): FrontendOnboardingDailySetupCliOutcomeCounts { + return { + cli_copy_init: 0, + ai_copy_init: 0, + both_copy_init: 0, + no_copy_init: 0, + cli_copy_other_cli: 0, + ai_copy_other_cli: 0, + both_copy_other_cli: 0, + no_copy_other_cli: 0, + cli_copy_no_cli: 0, + ai_copy_no_cli: 0, + both_copy_no_cli: 0, + no_action: 0, + } +} + +export function classifyFrontendOnboardingDailySetupCliOutcome( + signals: FrontendOnboardingDailySetupCliOutcomeSignals, +): FrontendOnboardingDailySetupCliOutcomeKey { + const copyPrefix = signals.cliCopied && signals.aiCopied + ? 'both_copy' + : signals.cliCopied + ? 'cli_copy' + : signals.aiCopied + ? 'ai_copy' + : 'no_copy' + + if (signals.initRun) + return `${copyPrefix}_init` as FrontendOnboardingDailySetupCliOutcomeKey + + if (signals.otherCliRun) + return `${copyPrefix}_other_cli` as FrontendOnboardingDailySetupCliOutcomeKey + + if (copyPrefix !== 'no_copy') + return `${copyPrefix}_no_cli` as FrontendOnboardingDailySetupCliOutcomeKey + + return 'no_action' +} diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts new file mode 100644 index 0000000000..a7d2ef2ec4 --- /dev/null +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts @@ -0,0 +1,51 @@ +import { describe, expect, it } from 'vitest' +import { + classifyFrontendOnboardingDailySetupCliOutcome, + FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS, +} from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts' + +describe('classifyFrontendOnboardingDailySetupCliOutcome', () => { + it.each([ + [true, false, true, false, 'cli_copy_init'], + [false, true, true, false, 'ai_copy_init'], + [true, true, true, false, 'both_copy_init'], + [false, false, true, false, 'no_copy_init'], + [true, false, false, true, 'cli_copy_other_cli'], + [false, true, false, true, 'ai_copy_other_cli'], + [true, true, false, true, 'both_copy_other_cli'], + [false, false, false, true, 'no_copy_other_cli'], + [true, false, false, false, 'cli_copy_no_cli'], + [false, true, false, false, 'ai_copy_no_cli'], + [true, true, false, false, 'both_copy_no_cli'], + [false, false, false, false, 'no_action'], + ] as const)('classifies cliCopied=%s, aiCopied=%s, initRun=%s, otherCliRun=%s as %s', (cliCopied, aiCopied, initRun, otherCliRun, expected) => { + expect(classifyFrontendOnboardingDailySetupCliOutcome({ cliCopied, aiCopied, initRun, otherCliRun })).toBe(expected) + }) + + it('prioritizes init when init and another CLI command both run', () => { + expect(classifyFrontendOnboardingDailySetupCliOutcome({ + cliCopied: true, + aiCopied: true, + initRun: true, + otherCliRun: true, + })).toBe('both_copy_init') + }) + + it('exports every outcome key once in taxonomy order', () => { + expect(FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS).toEqual([ + 'cli_copy_init', + 'ai_copy_init', + 'both_copy_init', + 'no_copy_init', + 'cli_copy_other_cli', + 'ai_copy_other_cli', + 'both_copy_other_cli', + 'no_copy_other_cli', + 'cli_copy_no_cli', + 'ai_copy_no_cli', + 'both_copy_no_cli', + 'no_action', + ]) + expect(new Set(FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS)).toHaveLength(12) + }) +}) From 9fafdfadaa75c3a76897e275b29ad9df3931f336 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 15:53:35 +0200 Subject: [PATCH 04/20] fix(admin): align daily Setup CLI outcomes model --- ...boarding_daily_setup_cli_outcomes_model.ts | 31 ++++++------------- ...aily-setup-cli-outcomes-model.unit.test.ts | 20 +++++++++--- 2 files changed, 26 insertions(+), 25 deletions(-) diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts index 8ab468387c..09c90db471 100644 --- a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts @@ -14,17 +14,17 @@ export const FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS = [ ] as const export type FrontendOnboardingDailySetupCliOutcomeKey = typeof FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS[number] -export type FrontendOnboardingDailySetupCliOutcomeLifecycle = 'first_time' | 'returning' -export type FrontendOnboardingDailySetupCliOutcomeEventKind = 'setup' | 'cli_copy' | 'ai_copy' | 'cli_command' +export type FrontendOnboardingDailySetupCliLifecycle = 'first_time' | 'returning' +export type FrontendOnboardingDailySetupCliEventKind = 'setup' | 'cli_copy' | 'ai_copy' | 'cli_command' -export interface FrontendOnboardingDailySetupCliOutcomeEvent { +export interface FrontendOnboardingDailySetupCliEvent { personId: string timestampMs: number - kind: FrontendOnboardingDailySetupCliOutcomeEventKind + kind: FrontendOnboardingDailySetupCliEventKind commandPath?: string } -export interface FrontendOnboardingDailySetupCliOutcomeSignals { +export interface FrontendOnboardingDailySetupCliSignals { cliCopied: boolean aiCopied: boolean initRun: boolean @@ -33,31 +33,20 @@ export interface FrontendOnboardingDailySetupCliOutcomeSignals { export type FrontendOnboardingDailySetupCliOutcomeCounts = Record -export interface FrontendOnboardingDailySetupCliOutcomeDailyPoint { +export interface FrontendOnboardingDailySetupCliOutcomePoint { date: string first_time: FrontendOnboardingDailySetupCliOutcomeCounts returning: FrontendOnboardingDailySetupCliOutcomeCounts } export function createFrontendOnboardingDailySetupCliOutcomeCounts(): FrontendOnboardingDailySetupCliOutcomeCounts { - return { - cli_copy_init: 0, - ai_copy_init: 0, - both_copy_init: 0, - no_copy_init: 0, - cli_copy_other_cli: 0, - ai_copy_other_cli: 0, - both_copy_other_cli: 0, - no_copy_other_cli: 0, - cli_copy_no_cli: 0, - ai_copy_no_cli: 0, - both_copy_no_cli: 0, - no_action: 0, - } + return Object.fromEntries( + FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS.map(key => [key, 0]), + ) as FrontendOnboardingDailySetupCliOutcomeCounts } export function classifyFrontendOnboardingDailySetupCliOutcome( - signals: FrontendOnboardingDailySetupCliOutcomeSignals, + signals: FrontendOnboardingDailySetupCliSignals, ): FrontendOnboardingDailySetupCliOutcomeKey { const copyPrefix = signals.cliCopied && signals.aiCopied ? 'both_copy' diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts index a7d2ef2ec4..3823f03dcd 100644 --- a/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts @@ -1,6 +1,7 @@ import { describe, expect, it } from 'vitest' import { classifyFrontendOnboardingDailySetupCliOutcome, + createFrontendOnboardingDailySetupCliOutcomeCounts, FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS, } from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts' @@ -22,13 +23,18 @@ describe('classifyFrontendOnboardingDailySetupCliOutcome', () => { expect(classifyFrontendOnboardingDailySetupCliOutcome({ cliCopied, aiCopied, initRun, otherCliRun })).toBe(expected) }) - it('prioritizes init when init and another CLI command both run', () => { + it.each([ + [false, false, 'no_copy_init'], + [true, false, 'cli_copy_init'], + [false, true, 'ai_copy_init'], + [true, true, 'both_copy_init'], + ] as const)('prioritizes init over another CLI command for cliCopied=%s and aiCopied=%s', (cliCopied, aiCopied, expected) => { expect(classifyFrontendOnboardingDailySetupCliOutcome({ - cliCopied: true, - aiCopied: true, + cliCopied, + aiCopied, initRun: true, otherCliRun: true, - })).toBe('both_copy_init') + })).toBe(expected) }) it('exports every outcome key once in taxonomy order', () => { @@ -48,4 +54,10 @@ describe('classifyFrontendOnboardingDailySetupCliOutcome', () => { ]) expect(new Set(FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS)).toHaveLength(12) }) + + it('creates zero counts in canonical outcome order', () => { + expect(Object.entries(createFrontendOnboardingDailySetupCliOutcomeCounts())).toEqual( + FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS.map(key => [key, 0]), + ) + }) }) From 8a54c84e6ae80d5a4891089832845019488a9e48 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 15:59:39 +0200 Subject: [PATCH 05/20] feat(admin): attribute daily Setup CLI actions --- ...boarding_daily_setup_cli_outcomes_model.ts | 173 ++++++++++++++++++ ...aily-setup-cli-outcomes-model.unit.test.ts | 150 +++++++++++++++ 2 files changed, 323 insertions(+) diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts index 09c90db471..5654045899 100644 --- a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts @@ -39,6 +39,16 @@ export interface FrontendOnboardingDailySetupCliOutcomePoint { returning: FrontendOnboardingDailySetupCliOutcomeCounts } +interface FrontendOnboardingDailySetupCliAnchor { + personId: string + timestampMs: number + date: string + lifecycle: FrontendOnboardingDailySetupCliLifecycle | undefined + signals: FrontendOnboardingDailySetupCliSignals +} + +const UTC_DAY_MS = 24 * 60 * 60 * 1000 + export function createFrontendOnboardingDailySetupCliOutcomeCounts(): FrontendOnboardingDailySetupCliOutcomeCounts { return Object.fromEntries( FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS.map(key => [key, 0]), @@ -67,3 +77,166 @@ export function classifyFrontendOnboardingDailySetupCliOutcome( return 'no_action' } + +export function buildFrontendOnboardingDailySetupCliOutcomes( + events: readonly FrontendOnboardingDailySetupCliEvent[], + startMs: number, + endMs: number, +): FrontendOnboardingDailySetupCliOutcomePoint[] { + if (!Number.isFinite(startMs) || !Number.isFinite(endMs) || endMs <= startMs) + throw new RangeError('startMs and endMs must be finite bounds with endMs greater than startMs') + + const normalizedEvents = events.map((event) => { + const personId = event.personId?.trim() + if (!personId) + throw new Error('Frontend onboarding event personId must be a non-empty string') + + if (!Number.isFinite(event.timestampMs)) + throw new Error('Frontend onboarding event timestampMs must be finite') + + if (!['setup', 'cli_copy', 'ai_copy', 'cli_command'].includes(event.kind)) + throw new Error(`Unsupported frontend onboarding event kind: ${event.kind}`) + + const commandPath = event.commandPath?.trim() + if (event.kind === 'cli_command' && !commandPath) + throw new Error('CLI command event commandPath must be a non-empty string') + + return { + ...event, + personId, + commandPath, + } + }) + + const anchorsByPersonDate = new Map() + for (const event of normalizedEvents) { + if (event.kind !== 'setup') + continue + + const date = getUtcDate(event.timestampMs) + const key = `${event.personId}\u0000${date}` + const existingAnchor = anchorsByPersonDate.get(key) + if (existingAnchor && existingAnchor.timestampMs <= event.timestampMs) + continue + + anchorsByPersonDate.set(key, { + personId: event.personId, + timestampMs: event.timestampMs, + date, + lifecycle: undefined, + signals: createFrontendOnboardingDailySetupCliSignals(), + }) + } + + const tailEndMs = endMs + UTC_DAY_MS + const anchorsByPerson = new Map() + for (const anchor of anchorsByPersonDate.values()) { + if (anchor.timestampMs < startMs || anchor.timestampMs >= tailEndMs) + continue + + const anchors = anchorsByPerson.get(anchor.personId) ?? [] + anchors.push(anchor) + anchorsByPerson.set(anchor.personId, anchors) + } + + for (const anchors of anchorsByPerson.values()) { + anchors.sort((left, right) => left.timestampMs - right.timestampMs) + let hasDisplayedAnchor = false + for (const anchor of anchors) { + if (anchor.timestampMs >= endMs) + continue + + anchor.lifecycle = hasDisplayedAnchor ? 'returning' : 'first_time' + hasDisplayedAnchor = true + } + } + + for (const event of normalizedEvents) { + if (event.kind === 'setup') + continue + + const anchors = anchorsByPerson.get(event.personId) + if (!anchors) + continue + + const anchorIndex = findLatestAnchorIndex(anchors, event.timestampMs) + if (anchorIndex === -1) + continue + + const anchor = anchors[anchorIndex] + const nextAnchor = anchors[anchorIndex + 1] + const windowEndMs = Math.min(anchor.timestampMs + UTC_DAY_MS, nextAnchor?.timestampMs ?? Infinity) + if (event.timestampMs >= windowEndMs || !anchor.lifecycle) + continue + + if (event.kind === 'cli_copy') + anchor.signals.cliCopied = true + else if (event.kind === 'ai_copy') + anchor.signals.aiCopied = true + else if (event.commandPath === 'init') + anchor.signals.initRun = true + else + anchor.signals.otherCliRun = true + } + + const pointsByDate = new Map() + for (let timestampMs = getUtcDayStart(startMs); timestampMs < endMs; timestampMs += UTC_DAY_MS) { + const date = getUtcDate(timestampMs) + pointsByDate.set(date, { + date, + first_time: createFrontendOnboardingDailySetupCliOutcomeCounts(), + returning: createFrontendOnboardingDailySetupCliOutcomeCounts(), + }) + } + + for (const anchors of anchorsByPerson.values()) { + for (const anchor of anchors) { + if (!anchor.lifecycle) + continue + + const point = pointsByDate.get(anchor.date) + if (!point) + continue + + const outcome = classifyFrontendOnboardingDailySetupCliOutcome(anchor.signals) + point[anchor.lifecycle][outcome]++ + } + } + + return [...pointsByDate.values()] +} + +function createFrontendOnboardingDailySetupCliSignals(): FrontendOnboardingDailySetupCliSignals { + return { + cliCopied: false, + aiCopied: false, + initRun: false, + otherCliRun: false, + } +} + +function findLatestAnchorIndex(anchors: readonly FrontendOnboardingDailySetupCliAnchor[], timestampMs: number): number { + for (let index = anchors.length - 1; index >= 0; index--) { + if (anchors[index].timestampMs <= timestampMs) + return index + } + + return -1 +} + +function getUtcDate(timestampMs: number): string { + const date = new Date(timestampMs) + if (Number.isNaN(date.getTime())) + throw new RangeError('Frontend onboarding event timestampMs must be a valid UTC date') + + return date.toISOString().slice(0, 10) +} + +function getUtcDayStart(timestampMs: number): number { + const date = new Date(timestampMs) + if (Number.isNaN(date.getTime())) + throw new RangeError('Bounds must be valid UTC dates') + + date.setUTCHours(0, 0, 0, 0) + return date.getTime() +} diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts index 3823f03dcd..465aade736 100644 --- a/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts @@ -1,10 +1,24 @@ +import type { FrontendOnboardingDailySetupCliEvent } from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts' import { describe, expect, it } from 'vitest' import { + buildFrontendOnboardingDailySetupCliOutcomes, classifyFrontendOnboardingDailySetupCliOutcome, createFrontendOnboardingDailySetupCliOutcomeCounts, FRONTEND_ONBOARDING_DAILY_SETUP_CLI_OUTCOME_KEYS, } from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts' +function utcMs(date: string): number { + return Date.parse(`${date}T00:00:00.000Z`) +} + +function setup(personId: string, timestampMs: number): FrontendOnboardingDailySetupCliEvent { + return { personId, timestampMs, kind: 'setup' } +} + +function categoryTotal(point: ReturnType[number], lifecycle: 'first_time' | 'returning'): number { + return Object.values(point[lifecycle]).reduce((total, count) => total + count, 0) +} + describe('classifyFrontendOnboardingDailySetupCliOutcome', () => { it.each([ [true, false, true, false, 'cli_copy_init'], @@ -61,3 +75,139 @@ describe('classifyFrontendOnboardingDailySetupCliOutcome', () => { ) }) }) + +describe('buildFrontendOnboardingDailySetupCliOutcomes', () => { + it('keeps only the earliest Setup view for a person UTC day', () => { + const august3 = utcMs('2026-08-03') + + expect(buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person', august3 + 4_000), + setup('person', august3 + 1_000), + { personId: 'person', timestampMs: august3 + 2_000, kind: 'cli_copy' }, + ], august3, august3 + 86_400_000)).toMatchObject([ + { + date: '2026-08-03', + first_time: { cli_copy_no_cli: 1 }, + returning: { no_action: 0 }, + }, + ]) + }) + + it('classifies first-time and returning Setup person-days and zero-fills dates', () => { + const august3 = utcMs('2026-08-03') + const august4 = utcMs('2026-08-04') + const august6 = utcMs('2026-08-06') + const points = buildFrontendOnboardingDailySetupCliOutcomes([ + setup('returning-person', august3 + 1_000), + { personId: 'returning-person', timestampMs: august3 + 2_000, kind: 'ai_copy' }, + setup('returning-person', august4 + 1_000), + { personId: 'returning-person', timestampMs: august4 + 2_000, kind: 'cli_command', commandPath: 'init' }, + setup('independent-person', august4 + 3_000), + { personId: 'independent-person', timestampMs: august4 + 4_000, kind: 'cli_copy' }, + { personId: 'independent-person', timestampMs: august4 + 5_000, kind: 'cli_command', commandPath: 'deploy' }, + ], august3, august6) + + expect(points).toMatchObject([ + { date: '2026-08-03', first_time: { ai_copy_no_cli: 1 } }, + { + date: '2026-08-04', + first_time: { cli_copy_other_cli: 1 }, + returning: { no_copy_init: 1 }, + }, + { date: '2026-08-05' }, + ]) + expect(categoryTotal(points[2], 'first_time')).toBe(0) + expect(categoryTotal(points[2], 'returning')).toBe(0) + }) + + it('uses an end-boundary Setup only to truncate the final displayed window', () => { + const august31 = utcMs('2026-08-31') + const september1 = utcMs('2026-09-01') + + expect(buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person', august31 + 1_000), + setup('person', september1 + 1_000), + { personId: 'person', timestampMs: september1 + 2_000, kind: 'cli_command', commandPath: 'init' }, + ], august31, september1)).toMatchObject([ + { date: '2026-08-31', first_time: { no_action: 1 } }, + ]) + }) + + it('assigns a next-anchor boundary action to that new anchor and excludes the 24-hour boundary', () => { + const august3 = utcMs('2026-08-03') + const august4 = utcMs('2026-08-04') + const august5 = utcMs('2026-08-05') + const points = buildFrontendOnboardingDailySetupCliOutcomes([ + setup('next-anchor', august3 + 1_000), + setup('next-anchor', august4 + 1_000), + { personId: 'next-anchor', timestampMs: august4 + 1_000, kind: 'cli_copy' }, + setup('twenty-four-hours', august3 + 2_000), + { personId: 'twenty-four-hours', timestampMs: august4 + 2_000, kind: 'cli_copy' }, + ], august3, august5) + + expect(points).toMatchObject([ + { date: '2026-08-03', first_time: { no_action: 2 } }, + { date: '2026-08-04', returning: { cli_copy_no_cli: 1 } }, + ]) + }) + + it.each([ + ['cli then ai', ['cli_copy', 'ai_copy']], + ['ai then cli', ['ai_copy', 'cli_copy']], + ] as const)('classifies %s copy order as both_copy_init', (_description, copyKinds) => { + const august3 = utcMs('2026-08-03') + + const points = buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person', august3 + 1_000), + { personId: 'person', timestampMs: august3 + 2_000, kind: copyKinds[0] }, + { personId: 'person', timestampMs: august3 + 3_000, kind: copyKinds[1] }, + { personId: 'person', timestampMs: august3 + 4_000, kind: 'cli_command', commandPath: 'init' }, + ], august3, august3 + 86_400_000) + + expect(points[0].first_time.both_copy_init).toBe(1) + }) + + it('treats an in-range Setup as first-time despite pre-range Setup history', () => { + const august2 = utcMs('2026-08-02') + const august3 = utcMs('2026-08-03') + + expect(buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person', august2 + 1_000), + setup('person', august3 + 1_000), + ], august3, august3 + 86_400_000)[0].first_time.no_action).toBe(1) + }) + + it('rejects invalid bounds and normalized events', () => { + const august3 = utcMs('2026-08-03') + + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([], august3, august3)).toThrow(RangeError) + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([], Number.NaN, august3)).toThrow(RangeError) + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([ + setup(' ', august3), + ], august3, august3 + 86_400_000)).toThrow(/personId/i) + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([ + { personId: 'person', timestampMs: august3, kind: 'cli_command' }, + ], august3, august3 + 86_400_000)).toThrow(/commandPath/i) + }) + + it('keeps category totals equal to displayed Setup person-days for each date and lifecycle', () => { + const august3 = utcMs('2026-08-03') + const august5 = utcMs('2026-08-05') + const points = buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person-a', august3 + 1_000), + setup('person-a', august3 + 2_000), + setup('person-a', august3 + 86_400_000 + 1_000), + setup('person-b', august3 + 3_000), + { personId: 'person-b', timestampMs: august3 + 4_000, kind: 'cli_copy' }, + ], august3, august5) + + expect(points.map(point => ({ + date: point.date, + first_time: categoryTotal(point, 'first_time'), + returning: categoryTotal(point, 'returning'), + }))).toEqual([ + { date: '2026-08-03', first_time: 2, returning: 0 }, + { date: '2026-08-04', first_time: 0, returning: 1 }, + ]) + }) +}) From d990e47e6399104c096de68ba6b73d2e630446cf Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 16:05:28 +0200 Subject: [PATCH 06/20] fix(admin): correct daily Setup action attribution --- ...boarding_daily_setup_cli_outcomes_model.ts | 11 +++---- ...aily-setup-cli-outcomes-model.unit.test.ts | 31 +++++++++++++++++-- 2 files changed, 32 insertions(+), 10 deletions(-) diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts index 5654045899..4e4da30d06 100644 --- a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts @@ -86,6 +86,7 @@ export function buildFrontendOnboardingDailySetupCliOutcomes( if (!Number.isFinite(startMs) || !Number.isFinite(endMs) || endMs <= startMs) throw new RangeError('startMs and endMs must be finite bounds with endMs greater than startMs') + const tailEndMs = endMs + UTC_DAY_MS const normalizedEvents = events.map((event) => { const personId = event.personId?.trim() if (!personId) @@ -97,8 +98,8 @@ export function buildFrontendOnboardingDailySetupCliOutcomes( if (!['setup', 'cli_copy', 'ai_copy', 'cli_command'].includes(event.kind)) throw new Error(`Unsupported frontend onboarding event kind: ${event.kind}`) - const commandPath = event.commandPath?.trim() - if (event.kind === 'cli_command' && !commandPath) + const commandPath = event.commandPath + if (event.kind === 'cli_command' && !commandPath?.trim()) throw new Error('CLI command event commandPath must be a non-empty string') return { @@ -110,7 +111,7 @@ export function buildFrontendOnboardingDailySetupCliOutcomes( const anchorsByPersonDate = new Map() for (const event of normalizedEvents) { - if (event.kind !== 'setup') + if (event.kind !== 'setup' || event.timestampMs < startMs || event.timestampMs >= tailEndMs) continue const date = getUtcDate(event.timestampMs) @@ -128,12 +129,8 @@ export function buildFrontendOnboardingDailySetupCliOutcomes( }) } - const tailEndMs = endMs + UTC_DAY_MS const anchorsByPerson = new Map() for (const anchor of anchorsByPersonDate.values()) { - if (anchor.timestampMs < startMs || anchor.timestampMs >= tailEndMs) - continue - const anchors = anchorsByPerson.get(anchor.personId) ?? [] anchors.push(anchor) anchorsByPerson.set(anchor.personId, anchors) diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts index 465aade736..01dd51c220 100644 --- a/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes-model.unit.test.ts @@ -93,6 +93,18 @@ describe('buildFrontendOnboardingDailySetupCliOutcomes', () => { ]) }) + it('does not let a pre-range Setup suppress an in-range Setup on the same UTC day', () => { + const august3 = utcMs('2026-08-03') + const startMs = august3 + 12 * 60 * 60 * 1_000 + + expect(buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person', august3 + 9 * 60 * 60 * 1_000), + setup('person', august3 + 13 * 60 * 60 * 1_000), + ], startMs, august3 + 86_400_000)).toMatchObject([ + { date: '2026-08-03', first_time: { no_action: 1 } }, + ]) + }) + it('classifies first-time and returning Setup person-days and zero-fills dates', () => { const august3 = utcMs('2026-08-03') const august4 = utcMs('2026-08-04') @@ -125,9 +137,9 @@ describe('buildFrontendOnboardingDailySetupCliOutcomes', () => { const september1 = utcMs('2026-09-01') expect(buildFrontendOnboardingDailySetupCliOutcomes([ - setup('person', august31 + 1_000), - setup('person', september1 + 1_000), - { personId: 'person', timestampMs: september1 + 2_000, kind: 'cli_command', commandPath: 'init' }, + setup('person', august31 + 23 * 60 * 60 * 1_000 + 30 * 60 * 1_000), + setup('person', september1 + 15 * 60 * 1_000), + { personId: 'person', timestampMs: september1 + 20 * 60 * 1_000, kind: 'cli_command', commandPath: 'init' }, ], august31, september1)).toMatchObject([ { date: '2026-08-31', first_time: { no_action: 1 } }, ]) @@ -167,6 +179,15 @@ describe('buildFrontendOnboardingDailySetupCliOutcomes', () => { expect(points[0].first_time.both_copy_init).toBe(1) }) + it('treats whitespace-padded init as another CLI command', () => { + const august3 = utcMs('2026-08-03') + + expect(buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person', august3 + 1_000), + { personId: 'person', timestampMs: august3 + 2_000, kind: 'cli_command', commandPath: ' init ' }, + ], august3, august3 + 86_400_000)[0].first_time.no_copy_other_cli).toBe(1) + }) + it('treats an in-range Setup as first-time despite pre-range Setup history', () => { const august2 = utcMs('2026-08-02') const august3 = utcMs('2026-08-03') @@ -182,9 +203,13 @@ describe('buildFrontendOnboardingDailySetupCliOutcomes', () => { expect(() => buildFrontendOnboardingDailySetupCliOutcomes([], august3, august3)).toThrow(RangeError) expect(() => buildFrontendOnboardingDailySetupCliOutcomes([], Number.NaN, august3)).toThrow(RangeError) + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([], august3, august3 - 1)).toThrow(RangeError) expect(() => buildFrontendOnboardingDailySetupCliOutcomes([ setup(' ', august3), ], august3, august3 + 86_400_000)).toThrow(/personId/i) + expect(() => buildFrontendOnboardingDailySetupCliOutcomes([ + setup('person', Number.NaN), + ], august3, august3 + 86_400_000)).toThrow(/timestampMs/i) expect(() => buildFrontendOnboardingDailySetupCliOutcomes([ { personId: 'person', timestampMs: august3, kind: 'cli_command' }, ], august3, august3 + 86_400_000)).toThrow(/commandPath/i) From ad1b88229b0927bdd5be3d2d2607d72775865302 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 16:18:13 +0200 Subject: [PATCH 07/20] feat(admin): query daily Setup CLI events --- ...end_onboarding_daily_setup_cli_outcomes.ts | 147 +++++++++++ ...ding-daily-setup-cli-outcomes.unit.test.ts | 241 ++++++++++++++++++ 2 files changed, 388 insertions(+) create mode 100644 supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts create mode 100644 tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts new file mode 100644 index 0000000000..eb2b305378 --- /dev/null +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts @@ -0,0 +1,147 @@ +import type { Context } from 'hono' +import type { + FrontendOnboardingDailySetupCliEvent, + FrontendOnboardingDailySetupCliEventKind, +} from './frontend_onboarding_daily_setup_cli_outcomes_model.ts' +import { cloudlogErr } from './logging.ts' +import { queryPosthogHogql } from './posthog_read.ts' + +const INVALID_TOTAL_EVENTS_ERROR = 'daily Setup CLI analytics query returned invalid total metadata' +const EVENT_LIMIT_EXCEEDED_ERROR = 'daily Setup CLI analytics query exceeded event limit' +const INVALID_ROW_ERROR = 'daily Setup CLI analytics row is invalid' +const EVENT_KINDS: readonly FrontendOnboardingDailySetupCliEventKind[] = ['setup', 'cli_copy', 'ai_copy', 'cli_command'] + +export const FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT = 50_000 + +function sqlStr(value: string): string { + return `'${value.replace(/'/g, '\'\'')}'` +} + +function timestampMs(value: unknown): number | null { + const timestamp = typeof value === 'number' + ? value + : typeof value === 'string' && value.trim() !== '' + ? Number(value) + : Number.NaN + + return Number.isFinite(timestamp) && timestamp > 0 ? timestamp : null +} + +function mapEvent(row: Record): FrontendOnboardingDailySetupCliEvent { + const personId = typeof row.person_id === 'string' ? row.person_id.trim() : '' + const timestamp = timestampMs(row.timestamp_ms) + const kind = EVENT_KINDS.includes(row.event_kind as FrontendOnboardingDailySetupCliEventKind) + ? row.event_kind as FrontendOnboardingDailySetupCliEventKind + : null + + if (!personId || timestamp === null || kind === null) + throw new Error(INVALID_ROW_ERROR) + + if (kind === 'cli_command') { + const commandPath = row.command_path + if (typeof commandPath !== 'string' || commandPath.trim() === '') + throw new Error(INVALID_ROW_ERROR) + + return { personId, timestampMs: timestamp, kind, commandPath } + } + + return { personId, timestampMs: timestamp, kind } +} + +export function assertFrontendOnboardingDailySetupCliEventTotal( + totalEvents: unknown, + limit = FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT, +): number { + if (typeof totalEvents !== 'number' + || !Number.isFinite(totalEvents) + || !Number.isInteger(totalEvents) + || totalEvents < 0) { + throw new Error(INVALID_TOTAL_EVENTS_ERROR) + } + if (totalEvents > limit) + throw new Error(EVENT_LIMIT_EXCEEDED_ERROR) + + return totalEvents +} + +export function buildFrontendOnboardingDailySetupCliHogql( + startDate: string, + endDate: string, + followupEndDate: string, +): string { + return ` + WITH setup_people AS ( + SELECT DISTINCT + toString(person_id) AS person_id + FROM events + WHERE event = 'onboarding_step_viewed' + AND JSONExtractString(toString(properties), 'flow') = 'pre_org' + AND toIntOrZero(toString(properties.onboarding_version)) = 2 + AND JSONExtractString(toString(properties), 'step') = 'setup' + AND timestamp >= parseDateTimeBestEffort(${sqlStr(startDate)}) + AND timestamp < parseDateTimeBestEffort(${sqlStr(endDate)}) + ) + SELECT + toString(selected_events.person_id) AS person_id, + toUnixTimestamp64Milli(selected_events.timestamp) AS timestamp_ms, + multiIf( + selected_events.event = 'onboarding_step_viewed', 'setup', + selected_events.event = 'onboarding_cli_command_copied', 'cli_copy', + selected_events.event = 'onboarding_ai_instructions_copied', 'ai_copy', + 'cli_command' + ) AS event_kind, + if(selected_events.event = 'CLI Command Invoked', JSONExtractString(toString(selected_events.properties), 'command_path'), '') AS command_path, + count() OVER () AS total_events + FROM events AS selected_events + INNER JOIN setup_people AS cohort + ON toString(selected_events.person_id) = cohort.person_id + WHERE selected_events.timestamp >= parseDateTimeBestEffort(${sqlStr(startDate)}) + AND selected_events.timestamp < parseDateTimeBestEffort(${sqlStr(followupEndDate)}) + AND ( + selected_events.event = 'CLI Command Invoked' + OR ( + selected_events.event IN ('onboarding_step_viewed', 'onboarding_cli_command_copied', 'onboarding_ai_instructions_copied') + AND JSONExtractString(toString(selected_events.properties), 'flow') = 'pre_org' + AND toIntOrZero(toString(selected_events.properties.onboarding_version)) = 2 + AND JSONExtractString(toString(selected_events.properties), 'step') = 'setup' + ) + ) + ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC + LIMIT ${FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT}` +} + +export async function getFrontendOnboardingDailySetupCliEvents( + c: Context, + startDate: string, + endDate: string, + followupEndDate: string, +): Promise { + const posthog = await queryPosthogHogql( + c, + buildFrontendOnboardingDailySetupCliHogql(startDate, endDate, followupEndDate), + ) + if (!posthog.configured || !posthog.connected || posthog.failureReason !== null) + throw new Error('daily Setup CLI analytics PostHog query failed') + + if (posthog.rows.length > 0) { + const totalEvents = posthog.rows[0].total_events + try { + assertFrontendOnboardingDailySetupCliEventTotal(totalEvents) + } + catch (error) { + const message = error instanceof Error && error.message === EVENT_LIMIT_EXCEEDED_ERROR + ? 'frontend_onboarding_daily_setup_cli_event_limit_exceeded' + : 'frontend_onboarding_daily_setup_cli_invalid_total_events' + cloudlogErr({ + requestId: c.get('requestId'), + message, + event_limit: FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT, + total_events: totalEvents, + returned_rows: posthog.rows.length, + }) + throw error + } + } + + return posthog.rows.map(mapEvent) +} diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts new file mode 100644 index 0000000000..fca25ab891 --- /dev/null +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts @@ -0,0 +1,241 @@ +import type { Context } from 'hono' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { + assertFrontendOnboardingDailySetupCliEventTotal, + buildFrontendOnboardingDailySetupCliHogql, + FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT, + getFrontendOnboardingDailySetupCliEvents, +} from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts' + +const { cloudlogErrMock, queryPosthogHogqlMock } = vi.hoisted(() => ({ + cloudlogErrMock: vi.fn(), + queryPosthogHogqlMock: vi.fn(), +})) + +vi.mock('../supabase/functions/_backend/utils/posthog_read.ts', () => ({ + queryPosthogHogql: queryPosthogHogqlMock, +})) + +vi.mock('../supabase/functions/_backend/utils/logging.ts', () => ({ + cloudlogErr: cloudlogErrMock, +})) + +function createContext(): Context { + return { get: () => 'request-id' } as unknown as Context +} + +beforeEach(() => { + cloudlogErrMock.mockReset() + queryPosthogHogqlMock.mockReset() + queryPosthogHogqlMock.mockResolvedValue({ + configured: true, + connected: true, + failureReason: null, + rows: [], + }) +}) + +describe('buildFrontendOnboardingDailySetupCliHogql', () => { + it('selects the bounded v2 Setup cohort and its Setup, copy, and unfiltered CLI events', () => { + const query = buildFrontendOnboardingDailySetupCliHogql( + '2026-08-01T00:00:00.123Z', + '2026-08-03T00:00:00.456Z', + '2026-08-04T00:00:00.789Z', + ) + + expect(query).toContain('WITH setup_people AS') + expect(query).toContain('event = \'onboarding_step_viewed\'') + expect(query).toContain('JSONExtractString(toString(properties), \'flow\') = \'pre_org\'') + expect(query).toContain('toIntOrZero(toString(properties.onboarding_version)) = 2') + expect(query).toContain('JSONExtractString(toString(properties), \'step\') = \'setup\'') + expect(query).toContain('event IN (\'onboarding_step_viewed\', \'onboarding_cli_command_copied\', \'onboarding_ai_instructions_copied\')') + expect(query).toContain('event = \'CLI Command Invoked\'') + expect(query).not.toContain('JSONExtractString(toString(properties), \'channel\')') + expect(query).not.toContain('JSONExtractString(toString(properties), \'command_path\') = \'init\'') + expect(query).toContain('timestamp >= parseDateTimeBestEffort(\'2026-08-01T00:00:00.123Z\')') + expect(query).toContain('timestamp < parseDateTimeBestEffort(\'2026-08-03T00:00:00.456Z\')') + expect(query).toContain('selected_events.timestamp >= parseDateTimeBestEffort(\'2026-08-01T00:00:00.123Z\')') + expect(query).toContain('selected_events.timestamp < parseDateTimeBestEffort(\'2026-08-04T00:00:00.789Z\')') + expect(query).toContain('JSONExtractString(toString(selected_events.properties), \'flow\') = \'pre_org\'') + expect(query).toContain('toIntOrZero(toString(selected_events.properties.onboarding_version)) = 2') + expect(query).toContain('JSONExtractString(toString(selected_events.properties), \'step\') = \'setup\'') + expect(query).toContain('INNER JOIN setup_people AS cohort') + expect(query).toContain('ON toString(selected_events.person_id) = cohort.person_id') + expect(query).toContain('toString(selected_events.person_id) AS person_id') + expect(query).toContain('toUnixTimestamp64Milli(selected_events.timestamp) AS timestamp_ms') + expect(query).toContain('if(selected_events.event = \'CLI Command Invoked\', JSONExtractString(toString(selected_events.properties), \'command_path\'), \'\') AS command_path') + expect(query).toContain('count() OVER () AS total_events') + expect(query).toContain('ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC') + expect(query).toContain('LIMIT 50000') + expect(query).not.toContain('LIMIT 50001') + }) + + it('escapes interpolated date strings', () => { + const query = buildFrontendOnboardingDailySetupCliHogql('start\'value', 'end\'value', 'followup\'value') + + expect(query).toContain('parseDateTimeBestEffort(\'start\'\'value\')') + expect(query).toContain('parseDateTimeBestEffort(\'end\'\'value\')') + expect(query).toContain('parseDateTimeBestEffort(\'followup\'\'value\')') + }) +}) + +describe('assertFrontendOnboardingDailySetupCliEventTotal', () => { + it('accepts finite non-negative integer totals through the limit', () => { + expect(assertFrontendOnboardingDailySetupCliEventTotal(0)).toBe(0) + expect(assertFrontendOnboardingDailySetupCliEventTotal(1, 1)).toBe(1) + expect(assertFrontendOnboardingDailySetupCliEventTotal(FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT)) + .toBe(FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT) + expect(FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT).toBe(50_000) + }) + + it.each([undefined, null, '1', Number.NaN, Number.POSITIVE_INFINITY, -1, 1.5])( + 'rejects malformed total metadata %s', + (totalEvents) => { + expect(() => assertFrontendOnboardingDailySetupCliEventTotal(totalEvents)) + .toThrow('daily Setup CLI analytics query returned invalid total metadata') + }, + ) + + it('uses a distinct error when the total exceeds the limit', () => { + expect(() => assertFrontendOnboardingDailySetupCliEventTotal(2, 1)) + .toThrow('daily Setup CLI analytics query exceeded event limit') + }) +}) + +describe('getFrontendOnboardingDailySetupCliEvents', () => { + it('strictly maps Setup, copy, and CLI rows while preserving the raw CLI command path', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: ' person-1 ', timestamp_ms: '1000', event_kind: 'setup', command_path: 'ignored', total_events: 5 }, + { person_id: 'person-1', timestamp_ms: 1100, event_kind: 'cli_copy', command_path: '', total_events: 5 }, + { person_id: 'person-1', timestamp_ms: 1200, event_kind: 'ai_copy', command_path: '', total_events: 5 }, + { person_id: 'person-1', timestamp_ms: 1300, event_kind: 'cli_command', command_path: 'init', total_events: 5 }, + { person_id: 'person-1', timestamp_ms: 1400, event_kind: 'cli_command', command_path: ' init ', total_events: 5 }, + ], + }) + + const result = await getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + ) + + expect(queryPosthogHogqlMock).toHaveBeenCalledWith( + expect.anything(), + buildFrontendOnboardingDailySetupCliHogql( + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + ), + ) + expect(result).toEqual([ + { personId: 'person-1', timestampMs: 1000, kind: 'setup' }, + { personId: 'person-1', timestampMs: 1100, kind: 'cli_copy' }, + { personId: 'person-1', timestampMs: 1200, kind: 'ai_copy' }, + { personId: 'person-1', timestampMs: 1300, kind: 'cli_command', commandPath: 'init' }, + { personId: 'person-1', timestampMs: 1400, kind: 'cli_command', commandPath: ' init ' }, + ]) + }) + + it('returns an empty list for a successful empty query', async () => { + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).resolves.toEqual([]) + }) + + it.each([ + { configured: false, connected: true, failureReason: null }, + { configured: true, connected: false, failureReason: null }, + { configured: false, connected: false, failureReason: 'unconfigured' }, + { configured: true, connected: false, failureReason: 'unavailable' }, + { configured: true, connected: false, failureReason: 'timeout' }, + { configured: true, connected: true, failureReason: 'too_large' }, + ])('fails closed for configured=$configured, connected=$connected, failureReason=$failureReason', async (posthog) => { + queryPosthogHogqlMock.mockResolvedValueOnce({ ...posthog, rows: [] }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics PostHog query failed') + }) + + it('logs total metadata and rejects when the event limit is exceeded', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [{ person_id: 'person-1', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 50_001 }], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query exceeded event limit') + expect(cloudlogErrMock).toHaveBeenCalledWith({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_event_limit_exceeded', + event_limit: 50_000, + total_events: 50_001, + returned_rows: 1, + }) + }) + + it('logs a distinct message and rejects invalid total metadata', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [{ person_id: 'person-1', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: '1' }], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query returned invalid total metadata') + expect(cloudlogErrMock).toHaveBeenCalledWith({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_invalid_total_events', + event_limit: 50_000, + total_events: '1', + returned_rows: 1, + }) + }) + + it.each([ + ['empty person', { person_id: ' ', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], + ['non-string person', { person_id: 42, timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], + ['zero timestamp', { person_id: 'person-1', timestamp_ms: 0, event_kind: 'setup', command_path: '', total_events: 1 }], + ['non-numeric timestamp', { person_id: 'person-1', timestamp_ms: 'later', event_kind: 'setup', command_path: '', total_events: 1 }], + ['unknown kind', { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'unknown', command_path: '', total_events: 1 }], + ['missing CLI path', { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'cli_command', total_events: 1 }], + ['whitespace CLI path', { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'cli_command', command_path: ' ', total_events: 1 }], + ])('rejects the whole query result for a malformed %s row', async (_name, row) => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [row], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics row is invalid') + }) +}) From f987748dad5c232eb44ae18e2fd8e96045c8f6cd Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 16:27:24 +0200 Subject: [PATCH 08/20] fix(admin): validate daily Setup CLI totals --- ...end_onboarding_daily_setup_cli_outcomes.ts | 17 +- ...ding-daily-setup-cli-outcomes.unit.test.ts | 161 +++++++++++++++--- 2 files changed, 153 insertions(+), 25 deletions(-) diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts index eb2b305378..28a460ed2a 100644 --- a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts @@ -106,7 +106,7 @@ export function buildFrontendOnboardingDailySetupCliHogql( AND JSONExtractString(toString(selected_events.properties), 'step') = 'setup' ) ) - ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC + ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC, command_path ASC LIMIT ${FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT}` } @@ -124,9 +124,20 @@ export async function getFrontendOnboardingDailySetupCliEvents( throw new Error('daily Setup CLI analytics PostHog query failed') if (posthog.rows.length > 0) { - const totalEvents = posthog.rows[0].total_events + let totalEvents = posthog.rows[0].total_events try { - assertFrontendOnboardingDailySetupCliEventTotal(totalEvents) + const expectedTotalEvents = assertFrontendOnboardingDailySetupCliEventTotal(totalEvents) + for (let index = 1; index < posthog.rows.length; index++) { + const row = posthog.rows[index] + totalEvents = row.total_events + const rowTotalEvents = assertFrontendOnboardingDailySetupCliEventTotal(totalEvents) + if (rowTotalEvents !== expectedTotalEvents) + throw new Error(INVALID_TOTAL_EVENTS_ERROR) + } + + totalEvents = expectedTotalEvents + if (posthog.rows.length > expectedTotalEvents) + throw new Error(INVALID_TOTAL_EVENTS_ERROR) } catch (error) { const message = error instanceof Error && error.message === EVENT_LIMIT_EXCEEDED_ERROR diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts index fca25ab891..127126e692 100644 --- a/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts @@ -43,30 +43,43 @@ describe('buildFrontendOnboardingDailySetupCliHogql', () => { '2026-08-03T00:00:00.456Z', '2026-08-04T00:00:00.789Z', ) + const setupPeople = query.match(/WITH setup_people AS \(([\s\S]*?)\n {4}\)\n {4}SELECT/)?.[1] ?? '' + const selectedProjectionStart = query.indexOf('\n SELECT\n toString(selected_events.person_id)') + const selectedFromStart = query.indexOf('\n FROM events AS selected_events') + const selectedOrderStart = query.indexOf('\n ORDER BY person_id ASC') + const selectedProjection = query.slice(selectedProjectionStart, selectedFromStart) + const selectedEventsFromWhere = query.slice(selectedFromStart, selectedOrderStart) + const selectedSetupCopyBranch = selectedEventsFromWhere.match(/OR \(\n([\s\S]*?)\n {8}\)\n {6}\)/)?.[1] ?? '' + + expect(setupPeople).toContain('WHERE event = \'onboarding_step_viewed\'') + expect(setupPeople).toContain('JSONExtractString(toString(properties), \'flow\') = \'pre_org\'') + expect(setupPeople).toContain('toIntOrZero(toString(properties.onboarding_version)) = 2') + expect(setupPeople).toContain('JSONExtractString(toString(properties), \'step\') = \'setup\'') + expect(setupPeople).toContain('timestamp >= parseDateTimeBestEffort(\'2026-08-01T00:00:00.123Z\')') + expect(setupPeople).toContain('timestamp < parseDateTimeBestEffort(\'2026-08-03T00:00:00.456Z\')') + expect(setupPeople).not.toContain('2026-08-04T00:00:00.789Z') + + expect(selectedProjection).toContain('toString(selected_events.person_id) AS person_id') + expect(selectedProjection).toContain('toUnixTimestamp64Milli(selected_events.timestamp) AS timestamp_ms') + expect(selectedProjection).toContain('if(selected_events.event = \'CLI Command Invoked\', JSONExtractString(toString(selected_events.properties), \'command_path\'), \'\') AS command_path') + expect(selectedProjection).toContain('count() OVER () AS total_events') + expect(selectedProjection).toContain('CLI Command Invoked') + + expect(selectedEventsFromWhere).toContain('INNER JOIN setup_people AS cohort') + expect(selectedEventsFromWhere).toContain('ON toString(selected_events.person_id) = cohort.person_id') + expect(selectedEventsFromWhere).toContain('selected_events.timestamp >= parseDateTimeBestEffort(\'2026-08-01T00:00:00.123Z\')') + expect(selectedEventsFromWhere).toContain('selected_events.timestamp < parseDateTimeBestEffort(\'2026-08-04T00:00:00.789Z\')') + expect(selectedEventsFromWhere).not.toContain('2026-08-03T00:00:00.456Z') + expect(selectedEventsFromWhere).toContain('selected_events.event = \'CLI Command Invoked\'\n OR (') + expect(selectedEventsFromWhere).not.toContain('JSONExtractString(toString(selected_events.properties), \'channel\')') + expect(selectedEventsFromWhere).not.toContain('JSONExtractString(toString(selected_events.properties), \'command_path\') = \'init\'') + expect(selectedSetupCopyBranch).toContain('selected_events.event IN (\'onboarding_step_viewed\', \'onboarding_cli_command_copied\', \'onboarding_ai_instructions_copied\')') + expect(selectedSetupCopyBranch).toContain('JSONExtractString(toString(selected_events.properties), \'flow\') = \'pre_org\'') + expect(selectedSetupCopyBranch).toContain('toIntOrZero(toString(selected_events.properties.onboarding_version)) = 2') + expect(selectedSetupCopyBranch).toContain('JSONExtractString(toString(selected_events.properties), \'step\') = \'setup\'') - expect(query).toContain('WITH setup_people AS') - expect(query).toContain('event = \'onboarding_step_viewed\'') - expect(query).toContain('JSONExtractString(toString(properties), \'flow\') = \'pre_org\'') - expect(query).toContain('toIntOrZero(toString(properties.onboarding_version)) = 2') - expect(query).toContain('JSONExtractString(toString(properties), \'step\') = \'setup\'') - expect(query).toContain('event IN (\'onboarding_step_viewed\', \'onboarding_cli_command_copied\', \'onboarding_ai_instructions_copied\')') - expect(query).toContain('event = \'CLI Command Invoked\'') - expect(query).not.toContain('JSONExtractString(toString(properties), \'channel\')') - expect(query).not.toContain('JSONExtractString(toString(properties), \'command_path\') = \'init\'') - expect(query).toContain('timestamp >= parseDateTimeBestEffort(\'2026-08-01T00:00:00.123Z\')') - expect(query).toContain('timestamp < parseDateTimeBestEffort(\'2026-08-03T00:00:00.456Z\')') - expect(query).toContain('selected_events.timestamp >= parseDateTimeBestEffort(\'2026-08-01T00:00:00.123Z\')') - expect(query).toContain('selected_events.timestamp < parseDateTimeBestEffort(\'2026-08-04T00:00:00.789Z\')') - expect(query).toContain('JSONExtractString(toString(selected_events.properties), \'flow\') = \'pre_org\'') - expect(query).toContain('toIntOrZero(toString(selected_events.properties.onboarding_version)) = 2') - expect(query).toContain('JSONExtractString(toString(selected_events.properties), \'step\') = \'setup\'') - expect(query).toContain('INNER JOIN setup_people AS cohort') - expect(query).toContain('ON toString(selected_events.person_id) = cohort.person_id') - expect(query).toContain('toString(selected_events.person_id) AS person_id') - expect(query).toContain('toUnixTimestamp64Milli(selected_events.timestamp) AS timestamp_ms') - expect(query).toContain('if(selected_events.event = \'CLI Command Invoked\', JSONExtractString(toString(selected_events.properties), \'command_path\'), \'\') AS command_path') expect(query).toContain('count() OVER () AS total_events') - expect(query).toContain('ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC') + expect(query).toContain('ORDER BY person_id ASC, timestamp_ms ASC, event_kind ASC, command_path ASC') expect(query).toContain('LIMIT 50000') expect(query).not.toContain('LIMIT 50001') }) @@ -215,6 +228,110 @@ describe('getFrontendOnboardingDailySetupCliEvents', () => { }) }) + it('logs and rejects malformed total metadata on a later row', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 2 }, + { person_id: 'person-1', timestamp_ms: 1100, event_kind: 'cli_copy', command_path: '', total_events: '2' }, + ], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query returned invalid total metadata') + expect(cloudlogErrMock).toHaveBeenCalledWith({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_invalid_total_events', + event_limit: 50_000, + total_events: '2', + returned_rows: 2, + }) + }) + + it('logs and rejects over-limit total metadata on a later row', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 2 }, + { person_id: 'person-1', timestamp_ms: 1100, event_kind: 'cli_copy', command_path: '', total_events: 50_001 }, + ], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query exceeded event limit') + expect(cloudlogErrMock).toHaveBeenCalledWith({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_event_limit_exceeded', + event_limit: 50_000, + total_events: 50_001, + returned_rows: 2, + }) + }) + + it('logs and rejects inconsistent totals across rows', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 2 }, + { person_id: 'person-1', timestamp_ms: 1100, event_kind: 'cli_copy', command_path: '', total_events: 3 }, + ], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query returned invalid total metadata') + expect(cloudlogErrMock).toHaveBeenCalledWith({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_invalid_total_events', + event_limit: 50_000, + total_events: 3, + returned_rows: 2, + }) + }) + + it('logs and rejects when returned rows exceed the asserted total', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }, + { person_id: 'person-1', timestamp_ms: 1100, event_kind: 'cli_copy', command_path: '', total_events: 1 }, + ], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query returned invalid total metadata') + expect(cloudlogErrMock).toHaveBeenCalledWith({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_invalid_total_events', + event_limit: 50_000, + total_events: 1, + returned_rows: 2, + }) + }) + it.each([ ['empty person', { person_id: ' ', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], ['non-string person', { person_id: 42, timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], From fa8d368784f92e2dcc3380685d178ea60f6b4350 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 16:31:05 +0200 Subject: [PATCH 09/20] fix(admin): reject incomplete Setup CLI events --- ...end_onboarding_daily_setup_cli_outcomes.ts | 2 +- ...ding-daily-setup-cli-outcomes.unit.test.ts | 29 +++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts index 28a460ed2a..cd59bfd0f7 100644 --- a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts @@ -136,7 +136,7 @@ export async function getFrontendOnboardingDailySetupCliEvents( } totalEvents = expectedTotalEvents - if (posthog.rows.length > expectedTotalEvents) + if (posthog.rows.length !== expectedTotalEvents) throw new Error(INVALID_TOTAL_EVENTS_ERROR) } catch (error) { diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts index 127126e692..70d3f3a969 100644 --- a/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts @@ -61,6 +61,10 @@ describe('buildFrontendOnboardingDailySetupCliHogql', () => { expect(selectedProjection).toContain('toString(selected_events.person_id) AS person_id') expect(selectedProjection).toContain('toUnixTimestamp64Milli(selected_events.timestamp) AS timestamp_ms') + expect(selectedProjection).toContain('selected_events.event = \'onboarding_step_viewed\', \'setup\'') + expect(selectedProjection).toContain('selected_events.event = \'onboarding_cli_command_copied\', \'cli_copy\'') + expect(selectedProjection).toContain('selected_events.event = \'onboarding_ai_instructions_copied\', \'ai_copy\'') + expect(selectedProjection).toContain('\'cli_command\'\n ) AS event_kind') expect(selectedProjection).toContain('if(selected_events.event = \'CLI Command Invoked\', JSONExtractString(toString(selected_events.properties), \'command_path\'), \'\') AS command_path') expect(selectedProjection).toContain('count() OVER () AS total_events') expect(selectedProjection).toContain('CLI Command Invoked') @@ -332,6 +336,31 @@ describe('getFrontendOnboardingDailySetupCliEvents', () => { }) }) + it('logs and rejects when returned rows are fewer than the asserted total', async () => { + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [ + { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 2 }, + ], + }) + + await expect(getFrontendOnboardingDailySetupCliEvents( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + '2026-08-04T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics query returned invalid total metadata') + expect(cloudlogErrMock).toHaveBeenCalledWith({ + requestId: 'request-id', + message: 'frontend_onboarding_daily_setup_cli_invalid_total_events', + event_limit: 50_000, + total_events: 2, + returned_rows: 1, + }) + }) + it.each([ ['empty person', { person_id: ' ', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], ['non-string person', { person_id: 42, timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], From 67e59512cc61e4f901667ebd8b518c0f58bc7fea Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 16:38:50 +0200 Subject: [PATCH 10/20] fix(admin): harden Setup CLI event parsing --- .../frontend_onboarding_daily_setup_cli_outcomes.ts | 4 ++-- ...tend-onboarding-daily-setup-cli-outcomes.unit.test.ts | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts index cd59bfd0f7..5ae18b530a 100644 --- a/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts +++ b/supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes.ts @@ -14,7 +14,7 @@ const EVENT_KINDS: readonly FrontendOnboardingDailySetupCliEventKind[] = ['setup export const FRONTEND_ONBOARDING_DAILY_SETUP_CLI_EVENT_LIMIT = 50_000 function sqlStr(value: string): string { - return `'${value.replace(/'/g, '\'\'')}'` + return `'${value.replace(/\\/g, '\\\\').replace(/'/g, '\'\'')}'` } function timestampMs(value: unknown): number | null { @@ -24,7 +24,7 @@ function timestampMs(value: unknown): number | null { ? Number(value) : Number.NaN - return Number.isFinite(timestamp) && timestamp > 0 ? timestamp : null + return Number.isSafeInteger(timestamp) && timestamp > 0 ? timestamp : null } function mapEvent(row: Record): FrontendOnboardingDailySetupCliEvent { diff --git a/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts index 70d3f3a969..cbdb7d22c5 100644 --- a/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts +++ b/tests/frontend-onboarding-daily-setup-cli-outcomes.unit.test.ts @@ -95,6 +95,12 @@ describe('buildFrontendOnboardingDailySetupCliHogql', () => { expect(query).toContain('parseDateTimeBestEffort(\'end\'\'value\')') expect(query).toContain('parseDateTimeBestEffort(\'followup\'\'value\')') }) + + it('escapes a backslash immediately before a quote in interpolated date strings', () => { + const query = buildFrontendOnboardingDailySetupCliHogql(String.raw`start\'value`, 'end', 'followup') + + expect(query).toContain(String.raw`parseDateTimeBestEffort('start\\''value')`) + }) }) describe('assertFrontendOnboardingDailySetupCliEventTotal', () => { @@ -365,6 +371,9 @@ describe('getFrontendOnboardingDailySetupCliEvents', () => { ['empty person', { person_id: ' ', timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], ['non-string person', { person_id: 42, timestamp_ms: 1000, event_kind: 'setup', command_path: '', total_events: 1 }], ['zero timestamp', { person_id: 'person-1', timestamp_ms: 0, event_kind: 'setup', command_path: '', total_events: 1 }], + ['fractional timestamp', { person_id: 'person-1', timestamp_ms: 1000.5, event_kind: 'setup', command_path: '', total_events: 1 }], + ['unsafe numeric timestamp', { person_id: 'person-1', timestamp_ms: Number.MAX_SAFE_INTEGER + 1, event_kind: 'setup', command_path: '', total_events: 1 }], + ['unsafe numeric string timestamp', { person_id: 'person-1', timestamp_ms: String(Number.MAX_SAFE_INTEGER + 1), event_kind: 'setup', command_path: '', total_events: 1 }], ['non-numeric timestamp', { person_id: 'person-1', timestamp_ms: 'later', event_kind: 'setup', command_path: '', total_events: 1 }], ['unknown kind', { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'unknown', command_path: '', total_events: 1 }], ['missing CLI path', { person_id: 'person-1', timestamp_ms: 1000, event_kind: 'cli_command', total_events: 1 }], From 91e50200d271f1467f39ecc1a678282f9111b43b Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 16:45:12 +0200 Subject: [PATCH 11/20] feat(admin): expose daily Setup CLI outcomes --- .../utils/frontend_onboarding_analytics.ts | 33 +++-- ...frontend-onboarding-analytics.unit.test.ts | 118 +++++++++++++++--- 2 files changed, 128 insertions(+), 23 deletions(-) diff --git a/supabase/functions/_backend/utils/frontend_onboarding_analytics.ts b/supabase/functions/_backend/utils/frontend_onboarding_analytics.ts index b5192292e9..001064bca1 100644 --- a/supabase/functions/_backend/utils/frontend_onboarding_analytics.ts +++ b/supabase/functions/_backend/utils/frontend_onboarding_analytics.ts @@ -5,6 +5,8 @@ import { FRONTEND_ONBOARDING_FOLLOWUP_MS, FRONTEND_ONBOARDING_VERSIONS, } from './frontend_onboarding_analytics_model.ts' +import { getFrontendOnboardingDailySetupCliEvents } from './frontend_onboarding_daily_setup_cli_outcomes.ts' +import { buildFrontendOnboardingDailySetupCliOutcomes } from './frontend_onboarding_daily_setup_cli_outcomes_model.ts' import { cloudlogErr } from './logging.ts' import { queryPosthogHogql } from './posthog_read.ts' @@ -244,22 +246,33 @@ export async function getAdminFrontendOnboardingAnalytics(c: Context, startDate: if (durationMs > FRONTEND_ONBOARDING_MAX_RANGE_MS) throw new RangeError('frontend onboarding analytics date range cannot exceed 365 days') + const normalizedStartDate = new Date(startMs).toISOString() + const normalizedEndDate = new Date(endMs).toISOString() const previousStartMs = startMs - durationMs const queryStartMs = Math.min(previousStartMs, startMs - FRONTEND_ONBOARDING_FOLLOWUP_MS) - const followupEndMs = endMs + 2 * FRONTEND_ONBOARDING_FOLLOWUP_MS + const dailyFollowupEndMs = endMs + FRONTEND_ONBOARDING_FOLLOWUP_MS + const aggregateFollowupEndMs = endMs + 2 * FRONTEND_ONBOARDING_FOLLOWUP_MS if (queryStartMs < POSTHOG_MIN_DATE_MS || queryStartMs >= POSTHOG_MAX_DATE_MS - || followupEndMs < POSTHOG_MIN_DATE_MS || followupEndMs >= POSTHOG_MAX_DATE_MS) { + || aggregateFollowupEndMs < POSTHOG_MIN_DATE_MS || aggregateFollowupEndMs >= POSTHOG_MAX_DATE_MS) { throw new RangeError('derived analytics date boundaries must be within the supported PostHog range') } - const posthog = await queryPosthogHogql( - c, - buildFrontendOnboardingHogql( - new Date(queryStartMs).toISOString(), - new Date(endMs).toISOString(), - new Date(followupEndMs).toISOString(), + const [posthog, dailySetupCliEvents] = await Promise.all([ + queryPosthogHogql( + c, + buildFrontendOnboardingHogql( + new Date(queryStartMs).toISOString(), + normalizedEndDate, + new Date(aggregateFollowupEndMs).toISOString(), + ), ), - ) + getFrontendOnboardingDailySetupCliEvents( + c, + normalizedStartDate, + normalizedEndDate, + new Date(dailyFollowupEndMs).toISOString(), + ), + ]) if (!posthog.configured || !posthog.connected || posthog.failureReason !== null) throw new Error('frontend onboarding analytics PostHog query failed') @@ -283,9 +296,11 @@ export async function getAdminFrontendOnboardingAnalytics(c: Context, startDate: } } const analytics = buildFrontendOnboardingAnalytics(mapAttempts(posthog.rows), startMs, endMs) + const dailySetupCliOutcomes = buildFrontendOnboardingDailySetupCliOutcomes(dailySetupCliEvents, startMs, endMs) return { ...analytics, + daily_setup_cli_outcomes: dailySetupCliOutcomes, posthog_configured: posthog.configured, posthog_connected: posthog.connected, } diff --git a/tests/frontend-onboarding-analytics.unit.test.ts b/tests/frontend-onboarding-analytics.unit.test.ts index b209fd0316..de77cef2c3 100644 --- a/tests/frontend-onboarding-analytics.unit.test.ts +++ b/tests/frontend-onboarding-analytics.unit.test.ts @@ -8,6 +8,7 @@ import { FRONTEND_ONBOARDING_MAX_RANGE_MS, getAdminFrontendOnboardingAnalytics, } from '../supabase/functions/_backend/utils/frontend_onboarding_analytics.ts' +import { createFrontendOnboardingDailySetupCliOutcomeCounts } from '../supabase/functions/_backend/utils/frontend_onboarding_daily_setup_cli_outcomes_model.ts' const { cloudlogErrMock, queryPosthogHogqlMock } = vi.hoisted(() => ({ cloudlogErrMock: vi.fn(), @@ -60,17 +61,17 @@ describe('buildFrontendOnboardingHogql', () => { expect(query).not.toMatch(/WITH\s+JSONExtractString/) expect(query).toContain('toString(person_id) AS person_id') expect(query).toContain('onboarding_attempts.person_id AS person_id') - expect(query).toContain("JSONExtractString(toString(properties), 'channel') = 'onboarding-v2'") - expect(query).toContain("event = 'CLI Command Invoked'") - expect(query).toContain("JSONExtractString(toString(properties), 'command_path') = 'init'") - expect(query).toContain("event = 'Builder Onboarding Step'") - expect(query).toContain("JSONExtractString(toString(properties), 'step') IN ('welcome', 'resume-prompt')") - expect(query).toContain("toUnixTimestamp64Milli(minIf(timestamp, event = 'onboarding_step_viewed' AND step = 'intent'))") - expect(query).toContain("toUnixTimestamp64Milli(minIf(timestamp, event = 'onboarding_step_viewed' AND step = 'details'))") - expect(query).toContain("toUnixTimestamp64Milli(minIf(timestamp, event = 'onboarding_step_viewed' AND step = 'organization'))") - expect(query).toContain("toUnixTimestamp64Milli(minIf(timestamp, event = 'onboarding_step_viewed' AND step = 'setup'))") - expect(query).toContain("groupUniqArrayIf(tuple(event, toUnixTimestamp64Milli(timestamp)), event IN (") - expect(query).toContain("groupUniqArrayIf(toUnixTimestamp64Milli(timestamp), event = 'onboarding_ai_instructions_copied') AS ai_instructions_copied_ms") + expect(query).toContain('JSONExtractString(toString(properties), \'channel\') = \'onboarding-v2\'') + expect(query).toContain('event = \'CLI Command Invoked\'') + expect(query).toContain('JSONExtractString(toString(properties), \'command_path\') = \'init\'') + expect(query).toContain('event = \'Builder Onboarding Step\'') + expect(query).toContain('JSONExtractString(toString(properties), \'step\') IN (\'welcome\', \'resume-prompt\')') + expect(query).toContain('toUnixTimestamp64Milli(minIf(timestamp, event = \'onboarding_step_viewed\' AND step = \'intent\'))') + expect(query).toContain('toUnixTimestamp64Milli(minIf(timestamp, event = \'onboarding_step_viewed\' AND step = \'details\'))') + expect(query).toContain('toUnixTimestamp64Milli(minIf(timestamp, event = \'onboarding_step_viewed\' AND step = \'organization\'))') + expect(query).toContain('toUnixTimestamp64Milli(minIf(timestamp, event = \'onboarding_step_viewed\' AND step = \'setup\'))') + expect(query).toContain('groupUniqArrayIf(tuple(event, toUnixTimestamp64Milli(timestamp)), event IN (') + expect(query).toContain('groupUniqArrayIf(toUnixTimestamp64Milli(timestamp), event = \'onboarding_ai_instructions_copied\') AS ai_instructions_copied_ms') expect(query).toContain('groupUniqArray(toUnixTimestamp64Milli(timestamp)) AS cli_started_ms') expect(query).toContain('GROUP BY onboarding_version, attempt_id') expect(query).toContain('LEFT JOIN cli_starts USING person_id') @@ -99,6 +100,7 @@ describe('getAdminFrontendOnboardingAnalytics', () => { it('maps grouped v1 and v3 rows, including repeated v3 interactions', async () => { const start = '2026-08-01T00:00:00.000Z' const intentMs = Date.parse(start) + 60 * 60 * 1000 + 123 + const dailySetupMs = Date.parse(start) + 2 * 60 * 60 * 1000 queryPosthogHogqlMock.mockResolvedValueOnce({ configured: true, connected: true, @@ -138,6 +140,30 @@ describe('getAdminFrontendOnboardingAnalytics', () => { total_attempts: 3, }], }) + queryPosthogHogqlMock.mockResolvedValueOnce({ + configured: true, + connected: true, + failureReason: null, + rows: [{ + person_id: 'person-1', + timestamp_ms: dailySetupMs, + event_kind: 'setup', + command_path: '', + total_events: 3, + }, { + person_id: 'person-1', + timestamp_ms: dailySetupMs + 1_000, + event_kind: 'cli_copy', + command_path: '', + total_events: 3, + }, { + person_id: 'person-1', + timestamp_ms: dailySetupMs + 2_000, + event_kind: 'cli_command', + command_path: 'init', + total_events: 3, + }], + }) const result = await getAdminFrontendOnboardingAnalytics(createContext(), start, '2026-08-03T00:00:00.000Z') @@ -164,7 +190,29 @@ describe('getAdminFrontendOnboardingAnalytics', () => { posthog_configured: true, posthog_connected: true, }) + expect(result.daily_setup_cli_outcomes).toEqual([ + { + date: '2026-08-01', + first_time: { + ...createFrontendOnboardingDailySetupCliOutcomeCounts(), + cli_copy_init: 1, + }, + returning: createFrontendOnboardingDailySetupCliOutcomeCounts(), + }, + { + date: '2026-08-02', + first_time: createFrontendOnboardingDailySetupCliOutcomeCounts(), + returning: createFrontendOnboardingDailySetupCliOutcomeCounts(), + }, + ]) + expect(result.v2_setup_cli_outcomes).toEqual({ + total_users: 1, + cli_only: 0, + cli_and_ai_instructions: 1, + no_cli: 0, + }) expect(result).not.toHaveProperty('onboarding_version') + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) }) it('returns zero analytics for a successful PostHog query with no matching attempts', async () => { @@ -179,6 +227,32 @@ describe('getAdminFrontendOnboardingAnalytics', () => { posthog_configured: true, posthog_connected: true, }) + expect(result.daily_setup_cli_outcomes).toEqual([ + { + date: '2026-08-01', + first_time: createFrontendOnboardingDailySetupCliOutcomeCounts(), + returning: createFrontendOnboardingDailySetupCliOutcomeCounts(), + }, + { + date: '2026-08-02', + first_time: createFrontendOnboardingDailySetupCliOutcomeCounts(), + returning: createFrontendOnboardingDailySetupCliOutcomeCounts(), + }, + ]) + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) + }) + + it('fails closed when the daily Setup CLI PostHog query fails', async () => { + queryPosthogHogqlMock + .mockResolvedValueOnce({ configured: true, connected: true, failureReason: null, rows: [] }) + .mockResolvedValueOnce({ configured: true, connected: false, failureReason: 'timeout', rows: [] }) + + await expect(getAdminFrontendOnboardingAnalytics( + createContext(), + '2026-08-01T00:00:00.000Z', + '2026-08-03T00:00:00.000Z', + )).rejects.toThrow('daily Setup CLI analytics PostHog query failed') + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) }) it.each([ @@ -234,6 +308,7 @@ describe('getAdminFrontendOnboardingAnalytics', () => { ] }, v3_graph: { nodes: [{ key: 'valid', count: 1 }] }, }) + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) }) it('queries the equal-length previous window through the current end plus 48 hours for post-setup outcomes', async () => { @@ -242,7 +317,7 @@ describe('getAdminFrontendOnboardingAnalytics', () => { await getAdminFrontendOnboardingAnalytics(createContext(), start, end) - expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(1) + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) expect(queryPosthogHogqlMock.mock.calls[0][1]).toContain( `timestamp >= parseDateTimeBestEffort('${new Date(Date.parse(start) - 2 * DAY_MS).toISOString()}')`, ) @@ -252,6 +327,15 @@ describe('getAdminFrontendOnboardingAnalytics', () => { expect(queryPosthogHogqlMock.mock.calls[0][1]).toContain( `AND intent_ms < toUnixTimestamp64Milli(parseDateTimeBestEffort('${end}'))`, ) + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain( + `timestamp >= parseDateTimeBestEffort('${start}')`, + ) + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain( + `timestamp < parseDateTimeBestEffort('${end}')`, + ) + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain( + `timestamp < parseDateTimeBestEffort('${new Date(Date.parse(end) + DAY_MS).toISOString()}')`, + ) }) it('queries a full 24-hour intent lookback for ranges shorter than the follow-up window', async () => { @@ -260,13 +344,16 @@ describe('getAdminFrontendOnboardingAnalytics', () => { await getAdminFrontendOnboardingAnalytics(createContext(), start, end) - expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(1) + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) expect(queryPosthogHogqlMock.mock.calls[0][1]).toContain( `timestamp >= parseDateTimeBestEffort('${new Date(Date.parse(start) - DAY_MS).toISOString()}')`, ) expect(queryPosthogHogqlMock.mock.calls[0][1]).toContain( `HAVING intent_ms >= toUnixTimestamp64Milli(parseDateTimeBestEffort('${new Date(Date.parse(start) - DAY_MS).toISOString()}'))`, ) + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain( + `timestamp >= parseDateTimeBestEffort('${start}')`, + ) }) it('accepts schema-valid sub-millisecond ISO fractions and normalizes them for PostHog', async () => { @@ -276,8 +363,11 @@ describe('getAdminFrontendOnboardingAnalytics', () => { '2026-08-03T00:00:00.5678Z', ) - expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(1) + expect(queryPosthogHogqlMock).toHaveBeenCalledTimes(2) expect(queryPosthogHogqlMock.mock.calls[0][1]).toContain('parseDateTimeBestEffort(\'2026-08-03T00:00:00.567Z\')') + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain('parseDateTimeBestEffort(\'2026-08-01T00:00:00.123Z\')') + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain('parseDateTimeBestEffort(\'2026-08-03T00:00:00.567Z\')') + expect(queryPosthogHogqlMock.mock.calls[1][1]).toContain('parseDateTimeBestEffort(\'2026-08-04T00:00:00.567Z\')') }) it('rejects date ranges wider than the dashboard maximum before querying PostHog', async () => { From 7d8d163e2511f667dfd36a7e042722f79d615183 Mon Sep 17 00:00:00 2001 From: WcaleNieWolny Date: Sat, 15 Aug 2026 16:59:13 +0200 Subject: [PATCH 12/20] feat(admin): support grouped stacked bar charts --- src/components/admin/AdminStackedBarChart.vue | 7 +- src/components/admin/adminStackedBarChart.ts | 112 ++++++++++++++++-- tests/admin-stacked-bar-chart.unit.test.ts | 94 +++++++++++++++ 3 files changed, 202 insertions(+), 11 deletions(-) diff --git a/src/components/admin/AdminStackedBarChart.vue b/src/components/admin/AdminStackedBarChart.vue index ad69bc54f2..ca4f37487d 100644 --- a/src/components/admin/AdminStackedBarChart.vue +++ b/src/components/admin/AdminStackedBarChart.vue @@ -19,6 +19,8 @@ interface DataSeries { label: string data: Array<{ date: string, value: number }> color: string + stack?: string + stackLabel?: string } const props = defineProps({ @@ -56,6 +58,8 @@ const chartData = computed(() => { label: item.label, data: item.data.map(point => point.value), color: item.color, + stack: item.stack, + stackLabel: item.stackLabel, })) return applyAdminStackedBarAccessibleBorders( @@ -65,7 +69,8 @@ const chartData = computed(() => { ) }) -const chartOptions = computed(() => buildAdminStackedBarChartOptions(isDark.value)) +const hasGroupedStacks = computed(() => props.series.some(item => item.stack !== undefined)) +const chartOptions = computed(() => buildAdminStackedBarChartOptions(isDark.value, hasGroupedStacks.value))