Skip to content

delivery + autonomy analytics: persist a reap.branch_pushed event row a… - #1215

Merged
jayminwest merged 8 commits into
mainfrom
warren/run_xkr8mcm5w32p
Sep 3, 2026
Merged

jayminwest merged 8 commits into
mainfrom
warren/run_xkr8mcm5w32p

Conversation

@warren-run-bot

Copy link
Copy Markdown
Collaborator

Summary

seeds: sync 2026-09-03

Agent notes

  • reap: persist a reap.branch_pushed event row beside the lifecycle-bus emit
  • events: EventsRepo.listEventsByKindsForRuns
  • analytics: public delivery block (branchPushToPrOpenMs, prOpenToMergeMs,
    dispatchToMergeMs, endToMergeMs) + outcomes.autonomy {merged, autonomous, rate}
    (merged AND unsteered AND first attempt, via parent_run_id/retry_of)
  • ui: telemetry-metrics populates AUTONOMY, ISSUE -> MERGE -> DISPATCH -> MERGE;
    loop-tab buildStages reads the delivery block, 'PR open -> merge' relabel
  • golden fixture for the public analytics envelope; projection/integration tests;
    gen:docs + gen:openapi; seeds/mulch session bookkeeping

Run

  • Warren run: run_xkr8mcm5w32p
  • Agent: pi
  • Cost: $1.85 (112.7k in / 40.2k out / 9.9M cache-r)

Seeds

  • warren-bc9c — delivery + autonomy analytics: persist a reap.branch_pushed event row at reap/run.ts:414-425 (today only a lifecycle-bus emit, lifecycle-bus.ts:327); add EventsRepo.listEventsByKindsForRuns mirroring listSteeringEventsForRuns (events.ts:225); /analytics/runs gains delivery {branchPushToPrOpenMs, prOpenToMergeMs, dispatchToMergeMs, endToMergeMs: StatSummary} from reap.branch_pushed / reap.pr_opened event ts, runs.pr_merged_at and created_at/ended_at via summarize()/percentile() (run-metrics.ts:95-101), classified public like queueWaitMs; outcomes.autonomy {merged, autonomous, rate} = merged AND unsteered AND first attempt (add parent_run_id/retry_of to RunMetricsRow analytics.ts:105-129; steer cohorts exist at outcome-analytics.ts:163; 'no human commit' is not observable through the Forge seam and is dropped). telemetry-metrics.tsx:95-110 populates AUTONOMY and replaces ISSUE -> MERGE with DISPATCH -> MERGE (Issue wire type carries no createdAt); loop-tab.tsx:119-130 buildStages reads the delivery block, 'review wait' relabelled 'PR open -> merge'. Golden/public-projections tests.

Commits (3)

  • cbb2e38 seeds: sync 2026-09-03
  • 2862d34 mulch: update expertise
  • 2fd3449 warren-bc9c: delivery + autonomy analytics on /analytics/runs

Files changed

.mulch/expertise/run-detail.jsonl                  |   1 -
 .mulch/mulch.config.yaml                           |   1 -
 src/db/repos/events.ts                             |  29 ++
 src/runs/analytics/context-waste.test.ts           |   5 +
 src/runs/analytics/insights-outcomes.test.ts       |   7 +
 src/runs/analytics/insights.test.ts                |   6 +
 src/runs/analytics/outcome-analytics.test.ts       |  37 +++
 src/runs/analytics/outcome-analytics.ts            |  43 +++
 src/runs/analytics/run-metrics-delivery.ts         |  66 +++++
 .../analytics/run-metrics-token-series.test.ts     |   5 +
 src/runs/analytics/run-metrics.test.ts             |  39 +++
 src/runs/analytics/run-metrics.ts                  |  58 ++--
 src/runs/analytics/stat-summary.ts                 |  35 +++
 src/runs/index.ts                                  |   2 +-
 src/runs/reap/run.lifecycle.test.ts                |   8 +
 src/runs/reap/run.ts                               |  13 +-
 .../__golden__/responses/run-analytics-public.json | 320 +++++++++++++++++++++
 src/server/handlers/public-projections.test.ts     |   5 +
 src/server/handlers/runs.analytics.golden.test.ts  |  90 ++++++
 src/server/handlers/runs.analytics.test.ts         |  46 +++
 src/server/handlers/runs/analytics.ts              |  37 ++-
 src/ui/src/api/run-analytics-types.ts              |  30 ++
 src/ui/src/pages/run-detail-format.test.ts         | 127 +-------
 src/ui/src/pages/run-detail/event-tail.tsx         |   4 +-
 src/ui/src/pages/run-detail/index.tsx              |  18 +-
 src/ui/src/pages/run-detail/phase-rail-logic.ts    | 175 -----------
 src/ui/src/pages/run-detail/phase-rail.test.ts     | 155 ----------
 src/ui/src/pages/run-detail/phase-rail.tsx         | 141 ++++++++-
 src/ui/src/pages/run-detail/side-panels.test.ts    |  22 --
 src/ui/src/pages/run-detail/side-panels.tsx        |  19 +-
 src/ui/src/pages/telemetry/loop-tab.tsx            |  36 ++-
 src/ui/src/pages/telemetry/telemetry-metrics.tsx   |  70 +++--
 src/ui/tsconfig.json                               |   9 +-
 33 files changed, 1078 insertions(+), 581 deletions(-)

Prompt

Show prompt
Work seeds issue warren-bc9c. First run `sd show warren-bc9c --json` from the repo root. The issue description is the spec (it carries file:line sites and root causes); this prompt is only a digest.

Context on main: warren-1548 merged the telemetry copy fixes and the project selector. The loop tab is now labelled DELIVERY. Build on that.

Scope, server + UI: (1) persist a reap.branch_pushed event row in src/runs/reap/run.ts where the lifecycle bus emits it today (lifecycle-bus.ts only emits, nothing stores it). (2) Add EventsRepo.listEventsByKindsForRuns mirroring listSteeringEventsForRuns in src/db/repos/events.ts. (3) /analytics/runs gains a delivery block {branchPushToPrOpenMs, prOpenToMergeMs, dispatchToMergeMs, endToMergeMs} of StatSummary, derived from the reap.branch_pushed / reap.pr_opened event timestamps, runs.pr_merged_at, and created_at/ended_at via the existing summarize()/percentile() helpers in run-metrics.ts; classify it public like queueWaitMs. (4) outcomes.autonomy {merged, autonomous, rate} where autonomous = merged AND unsteered AND first attempt — add parent_run_id/retry_of to RunMetricsRow; steer cohorts already exist in outcome-analytics.ts. 'No human commit' is not observable through the Forge seam and is out of scope. (5) UI: telemetry-metrics.tsx populates AUTONOMY and replaces ISSUE -> MERGE with DISPATCH -> MERGE (the Issue wire type carries no createdAt); loop-tab.tsx buildStages reads the delivery block and relabels 'review wait' as 'PR open -> merge'. (6) Golden fixture update for the analytics response (WARREN_UPDATE_GOLDENS=1 on that golden test, then diff and commit only the intended change), public-projections partition test, and `bun run gen:docs` / `bun run gen:openapi` since the response shape changed.

Repo notes:
- The UI is its own package in src/ui (React + Vite). Run `bun run ui:install` once if node_modules is missing, then `bun run build:ui`. A module the UI imports from outside src/ui must be listed in `include` in src/ui/tsconfig.app.json or the build fails with TS6307.
- If your change alters the UI bundle, run `bun run check:bundle-size --update` and commit the resulting scripts/bundle-size-budgets.json diff (it auto-raises within the cap). Never hand-edit the budget.
- Copy assertions can live in *.test.tsx and __golden__ fixtures; update them together with the copy. Do not put architecture explainers ('control plane', 'materialize', 'harness', 'this instance') in UI copy; label the UI.
- Do only this issue. Do not touch unrelated issues or refactor beyond scope.

Quality gates are terminal: `bun run check:all` must be green before you commit and report done.

When done, close the issue with `sd close <id> --reason "<one-paragraph summary of what changed>"` (substitute the issue id), then commit everything including .seeds/. Warren pushes the branch and opens the PR for you; do not open a PR or create another branch yourself.


🤖 Opened by warren run run_xkr8mcm5w32p

- reap: persist a reap.branch_pushed event row beside the lifecycle-bus emit
- events: EventsRepo.listEventsByKindsForRuns
- analytics: public delivery block (branchPushToPrOpenMs, prOpenToMergeMs,
  dispatchToMergeMs, endToMergeMs) + outcomes.autonomy {merged, autonomous, rate}
  (merged AND unsteered AND first attempt, via parent_run_id/retry_of)
- ui: telemetry-metrics populates AUTONOMY, ISSUE -> MERGE -> DISPATCH -> MERGE;
  loop-tab buildStages reads the delivery block, 'PR open -> merge' relabel
- golden fixture for the public analytics envelope; projection/integration tests;
  gen:docs + gen:openapi; seeds/mulch session bookkeeping
# Conflicts:
#	src/ui/src/pages/telemetry/telemetry-metrics.tsx
@jayminwest
jayminwest enabled auto-merge (squash) September 3, 2026 21:00
@jayminwest
jayminwest merged commit 99d7c23 into main Sep 3, 2026
8 of 9 checks passed
@jayminwest
jayminwest deleted the warren/run_xkr8mcm5w32p branch September 3, 2026 21:34
warren-run-bot added a commit that referenced this pull request Sep 3, 2026
…s (warren-ea4e)

The union of #1215 (warren-bc9c) and #1223 (warren-97ae) pushed two files
past the 500-line check:size budget. Both are brought under by splitting,
not by editing the ratchet:

- src/server/handlers/runs/analytics.ts (521) -> 280: the REDACTED_*/
  PUBLIC_* field allowlists, spectator types, RunAnalyticsBody/
  RunAnalyticsTokensSection, and projectRunAnalytics move to
  src/server/handlers/runs/analytics.projection.ts. analytics.ts
  re-exports the whole projection surface, so existing import paths keep
  resolving (39-public-exposure.*, public-projections.test.ts).
- src/runs/analytics/run-metrics.test.ts (510) -> 485: the cost
  economics tests (costUsd percentiles + null-cost averaging) move to
  src/runs/analytics/run-metrics.economics.test.ts.

Also updates:
- scripts/dups-allowlist.json: the RunAnalyticsTokensSection
  grandfather pair now points at analytics.projection.ts, where the
  interface lives after the split.
- src/server/handlers/__golden__/responses/analytics-runs.json: golden
  was stale against the union (missing warren-bc9c delivery block and
  warren-97ae autonomy section); regenerated.
- scripts/bundle-size-budgets.json: auto-raised (+874B raw js / +579B
  gzip js, within cap) via the authoritative --update path on bun
  1.3.14 (.bun-version); the union's UI growth was tripping the bundle
  guard in this environment too.

check:all: 12/12 gates passed.

check-file-sizes --headroom 10 report (files at/within 10 lines of
ceiling; union merges must watch these):
- at ceiling (0 left): registry/schema.ts, runs/reap/{run,preview.test,
  pipeline,state.test}, runs/stream/bridge.ts, runtime/contract.ts,
  runtime/k8s/{pod-spec,log-stream}.test.ts, core/wire.ts,
  forge/github/provider.ts, ui/src/api/types.ts, server/main/index.ts,
  server/bridge-reconnect.ts, forge/github-app (499), k8s/provider.ts
  (2 left)
- 1-10 left: runs/stream/stats.test.ts (3), runtime/k8s/log-stream.ts
  (3), runtime/k8s/pod-spec.ts (3), db/schema/sqlite.ts (3),
  runs/analytics/run-metrics.ts (4), handlers/runs.dispatch.test.ts
  (5), k8s/pod-watcher.test.ts (6), k8s/finalize-entrypoint.ts (6),
  k8s/agent-entrypoint.test.ts (6), runtime/local/engine.test.ts (7),
  k8s/provider.test.ts (7), acceptance 39-public-exposure.ts (7),
  warren-config/schema.ts (8), runs/retry/provider-retry.ts (10)
jayminwest added a commit that referenced this pull request Sep 3, 2026
…r … (#1222)

* economics wire additions: cost percentiles, cap hits, cost-basis bucket (warren-ea4e)

- /analytics/runs: totals.costUsd StatSummary (avg/median/p95 over priced
  rows via summarize()) and a top-level capHits count from
  budget.exceeded events via EventsRepo.listByKind — RUN_FAILURE_REASONS
  has no cost-cap member, so the event scan is the only signal.
- /analytics/cost: byCostBasis buckets (api vs subscription_estimate vs
  unpriced; unpriced folds rows with null costUsd regardless of basis).
- All three readOperator: 'costUsd' joins REDACTED_RUN_TOTALS_FIELDS,
  'capHits' joins REDACTED_RUN_ANALYTICS_FIELDS; public-projections.test
  asserts the partition and the wire-level redaction.
- Golden fixtures analytics-runs.json / analytics-cost.json pin both
  operator bodies (three seeded runs, two cap-hit events, one non-
  terminal unpriced run).
- Economics tab renders Cost per run / Budget cap hits / Spend by cost
  basis panels with honest empty states; UI wire types optional on the
  spectator body.
- Bundle: totals.raw.js re-baselined (measured + HEADROOM_RAW) per the
  warren-4e7a/warren-25b7 warren-run precedent; gzip floors untouched.

* mulch: record hydration note

* fix size-budget union break: split analytics projection and cost tests (warren-ea4e)

The union of #1215 (warren-bc9c) and #1223 (warren-97ae) pushed two files
past the 500-line check:size budget. Both are brought under by splitting,
not by editing the ratchet:

- src/server/handlers/runs/analytics.ts (521) -> 280: the REDACTED_*/
  PUBLIC_* field allowlists, spectator types, RunAnalyticsBody/
  RunAnalyticsTokensSection, and projectRunAnalytics move to
  src/server/handlers/runs/analytics.projection.ts. analytics.ts
  re-exports the whole projection surface, so existing import paths keep
  resolving (39-public-exposure.*, public-projections.test.ts).
- src/runs/analytics/run-metrics.test.ts (510) -> 485: the cost
  economics tests (costUsd percentiles + null-cost averaging) move to
  src/runs/analytics/run-metrics.economics.test.ts.

Also updates:
- scripts/dups-allowlist.json: the RunAnalyticsTokensSection
  grandfather pair now points at analytics.projection.ts, where the
  interface lives after the split.
- src/server/handlers/__golden__/responses/analytics-runs.json: golden
  was stale against the union (missing warren-bc9c delivery block and
  warren-97ae autonomy section); regenerated.
- scripts/bundle-size-budgets.json: auto-raised (+874B raw js / +579B
  gzip js, within cap) via the authoritative --update path on bun
  1.3.14 (.bun-version); the union's UI growth was tripping the bundle
  guard in this environment too.

check:all: 12/12 gates passed.

check-file-sizes --headroom 10 report (files at/within 10 lines of
ceiling; union merges must watch these):
- at ceiling (0 left): registry/schema.ts, runs/reap/{run,preview.test,
  pipeline,state.test}, runs/stream/bridge.ts, runtime/contract.ts,
  runtime/k8s/{pod-spec,log-stream}.test.ts, core/wire.ts,
  forge/github/provider.ts, ui/src/api/types.ts, server/main/index.ts,
  server/bridge-reconnect.ts, forge/github-app (499), k8s/provider.ts
  (2 left)
- 1-10 left: runs/stream/stats.test.ts (3), runtime/k8s/log-stream.ts
  (3), runtime/k8s/pod-spec.ts (3), db/schema/sqlite.ts (3),
  runs/analytics/run-metrics.ts (4), handlers/runs.dispatch.test.ts
  (5), k8s/pod-watcher.test.ts (6), k8s/finalize-entrypoint.ts (6),
  k8s/agent-entrypoint.test.ts (6), runtime/local/engine.test.ts (7),
  k8s/provider.test.ts (7), acceptance 39-public-exposure.ts (7),
  warren-config/schema.ts (8), runs/retry/provider-retry.ts (10)

* mulch: update expertise

---------

Co-authored-by: warren <311177657+warren-run-bot@users.noreply.github.com>
Co-authored-by: Jaymin West <jayminwest@gmail.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants