From a96268793b76d597b827ab3cb6fdf687d2d06676 Mon Sep 17 00:00:00 2001 From: Christoph Date: Fri, 24 Jul 2026 00:45:27 +0200 Subject: [PATCH 1/3] spec(project-analytics): dashboard development analytics and balance map OpenSpec change proposal (no implementation). Adds a project-analytics capability to the governance dashboard: a two-state summary (submitted = to-be, merged = current-state), a topic read against the project goal and invariants, a six-dimension radar balance map overlaying current-state and to-be, governance KPI ratios, and the escaped-defect (green-then-bug) KPI for building bug-free software. Signed-off-by: Christoph --- .../.openspec.yaml | 2 + .../project-development-analytics/proposal.md | 63 ++++++++ .../specs/project-analytics/spec.md | 136 ++++++++++++++++++ .../project-development-analytics/tasks.md | 35 +++++ 4 files changed, 236 insertions(+) create mode 100644 openspec/changes/project-development-analytics/.openspec.yaml create mode 100644 openspec/changes/project-development-analytics/proposal.md create mode 100644 openspec/changes/project-development-analytics/specs/project-analytics/spec.md create mode 100644 openspec/changes/project-development-analytics/tasks.md diff --git a/openspec/changes/project-development-analytics/.openspec.yaml b/openspec/changes/project-development-analytics/.openspec.yaml new file mode 100644 index 0000000..5e6d53a --- /dev/null +++ b/openspec/changes/project-development-analytics/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-24 diff --git a/openspec/changes/project-development-analytics/proposal.md b/openspec/changes/project-development-analytics/proposal.md new file mode 100644 index 0000000..712eb9d --- /dev/null +++ b/openspec/changes/project-development-analytics/proposal.md @@ -0,0 +1,63 @@ +## Why + +The governance dashboard shows the pipeline: pull requests bucketed by stage (awaiting review, in +progress, changes requested, merged), by lane. That answers "what is moving through the gate right now." +It does not answer the questions a project owner actually asks: + +- **What is the work about?** Across everything submitted and everything merged, what are the changes + mostly doing, and does that match the project's stated goal and its invariants? +- **Is the project healthy and balanced, or lopsided?** A project can merge steadily and still drift: + all features, no tests; all code, no docs; velocity up, correctness down. There is no single view of + where the project sits today and where it is heading if the open work merges. +- **Are we building bug-free software?** The point of the gate is fewer escaped defects. Nothing today + counts the failures that matter most: a change that passed review green, passed its tests, and still + turned out to carry a bug. + +GitHub already holds every raw fact (each PR, each check, each merge). The dashboard's job is not to +repeat GitHub; it is to **synthesise** those facts into a small number of quantifiable signals a human +can read at a glance, and to make the trajectory visible: current-state (what has merged) versus the +to-be state (what merges if the open PRs land). + +## What Changes + +Add a **project-analytics** section to the dashboard, above or beside the existing stage buckets. It is +derived entirely from PR facts the dashboard already fetches, plus a small set of declared link +conventions for the signals GitHub does not carry directly. Nothing here changes the gate; it is a +read-only view. + +1. **Two-state summary.** A count and short digest of every **submitted** PR (open = the *to-be* state) + and every **merged** PR (the *current-state*), so the two are always read side by side. + +2. **What the work is about.** The most common topics across submitted PRs and across merged PRs (by + lane, declared scope, and changed-area), and a plain-language read of what those topics mean against + the project's **invariants** and its **stated goal** (both declared in config). + +3. **A balance map (radar / spider chart).** A star-shaped chart with **6 fixed dimensions**, each scored + `0` at the centre to `5` at the point, drawn as **two overlaid polygons**: the current-state (from + merged PRs) and the to-be state (from merged plus open PRs, projecting the open work as if merged). The + shape shows at a glance whether the project is balanced across the principles of a healthy software + project or spiking in one direction, the way an attribute chart does for sports equipment or a game + character. The six dimensions and their scoring are defined in the spec. + +4. **Governance KPI ratios.** Totals and percentages that quantify how the process is going: how many + incoming PRs fit the spec/OpenSpec standard; how many pass review versus not (with the failing count + called out); tests created and tests passing and doc updates from merged PRs and their trend; which + review categories findings fall into; how many PRs conform to versus violate a MUST invariant; how many + attempted to change the core business logic of the code; and how many were flagged malicious. + +5. **Bug-free-software KPIs.** The headline performance numbers: submitted versus merged, and the + **escaped-defect count** - bugs reported against a PR that had passed review green and passed its tests + yet still shipped a defect. This is the number the whole gate exists to drive down. + +## Impact + +- Affected specs: new capability `project-analytics`. +- Affected code (at build time, not in this change): `cli/dashboard.py` (a new analytics model + render + section), its template/HTML, and `cli/dashboard.test.sh`. Reuses the existing `fetch`, `stage_of`, + `lane_of`, `parse_impact`, and `_normative_paths` helpers. +- New declared conventions (config + PR trailers) for the signals GitHub does not carry: the project goal + and invariants, the escaped-defect link, and the malicious/core-change markers. All are optional; every + KPI degrades to "not enough signal" rather than a wrong number when its input is absent. +- Privacy and provenance are unchanged from the dashboard's existing stance: every rendered fact is + already visible on GitHub, private deployments stay behind auth, and `--public` remains the explicit + opt-in for publication. diff --git a/openspec/changes/project-development-analytics/specs/project-analytics/spec.md b/openspec/changes/project-development-analytics/specs/project-analytics/spec.md new file mode 100644 index 0000000..1226370 --- /dev/null +++ b/openspec/changes/project-development-analytics/specs/project-analytics/spec.md @@ -0,0 +1,136 @@ +## ADDED Requirements + +### Requirement: Two-state PR summary +The dashboard MUST render a summary of every submitted (open) pull request and every merged pull request, +labelled so the two are read as distinct states: **submitted = the to-be state** (what the project becomes +if the open work merges) and **merged = the current-state** (what the project is now). Each state MUST show +a total count and a short per-PR digest (number, title, lane, declared scope). + +#### Scenario: Both states are shown side by side +- **WHEN** the dashboard renders for a repository with open and merged PRs +- **THEN** it shows a submitted (to-be) group and a merged (current-state) group, each with its total and a + per-PR digest +- **AND** a repository with no open PRs still shows the merged (current-state) group and an empty to-be + group, not an error + +### Requirement: What the work is about, against goal and invariants +The dashboard MUST report the most common topics across submitted PRs and, separately, across merged PRs, +derived from each PR's lane, declared scope, and changed areas. It MUST present those topics against the +project's declared **goal** and **invariants** (read from config), stating in plain language whether the +weight of the work supports the goal or diverges from it. When no goal or invariants are declared, it MUST +say so rather than invent a judgement. + +#### Scenario: Topics summarised for each state +- **WHEN** submitted and merged PRs carry lanes, declared scopes, and changed paths +- **THEN** the dashboard lists the leading topics for the to-be state and for the current-state separately + +#### Scenario: Topics read against the declared goal +- **WHEN** a project goal and invariants are declared in config +- **THEN** the dashboard states whether the leading topics support or diverge from that goal and names any + topic that touches an invariant +- **WHEN** no goal or invariants are declared +- **THEN** the dashboard reports the topics and states that no goal or invariants are declared to judge + them against + +### Requirement: Balance map across six fixed dimensions +The dashboard MUST render a radar (spider) chart with exactly **six fixed dimensions**, each on a scale +from `0` at the centre to `5` at the outer point, drawn as **two overlaid polygons**: the current-state +scored from merged PRs, and the to-be state scored from merged plus open PRs with each open PR projected as +if merged. The chart MUST make an unbalanced project visually obvious (a spike on one axis, a collapse on +another) the way an attribute chart does for equipment or a game character. + +#### Scenario: Two polygons on one chart +- **WHEN** the dashboard renders analytics for a repository +- **THEN** the radar shows a current-state polygon and a to-be polygon over the same six labelled axes, + each axis scaled 0 at centre to 5 at the point + +#### Scenario: To-be projects the open work +- **WHEN** open PRs exist +- **THEN** the to-be polygon is computed as if those open PRs were merged, so the gap between the two + polygons shows the direction the open work moves the project + +#### Scenario: A missing dimension does not break the chart +- **WHEN** a dimension has no signal to score (no relevant PRs yet) +- **THEN** that axis renders at 0 with a "not enough signal" note, and the rest of the chart still renders + +### Requirement: Definition of the six balance dimensions +The six dimensions MUST be the principles of a healthy software project below, each scored `0` to `5` from +PR-derived signals. Each dimension MUST be scored for the current-state (over merged PRs) and the to-be +state (over merged plus projected-open PRs) by the same rule, so the two are comparable. + +1. **Correctness** - the share of green-merged PRs (passed review, passed tests) that did NOT later produce + a reported escaped defect. `5` means no escaped defects. +2. **Verification** - the share of code-changing PRs that added or extended tests AND passed the test gate. + `5` means every code change is test-backed and green. +3. **Documentation** - the share of user-facing PRs that updated docs or the impact log. `5` means every + user-facing change is documented. +4. **Spec conformance** - the share of PRs that met the spec / OpenSpec readiness standard at intake. `5` + means every change was spec-driven. +5. **Governance integrity** - the share of merges that passed the agent and human review with no override + and no MUST-invariant violation, reduced by any malicious or against-invariant attempts. `5` means a + clean review record. +6. **Flow** - submitted-to-merged conversion and the absence of long-stuck PRs. `5` means work converts + healthily without a growing backlog. + +#### Scenario: Each dimension is scored for both states by one rule +- **WHEN** the analytics model runs +- **THEN** each of the six dimensions has a current-state score and a to-be score in `[0, 5]` produced by + the same scoring rule over the respective PR set + +### Requirement: Governance KPI ratios +The dashboard MUST render the following as totals and percentages, with the count that fails or violates +always shown alongside the percentage: + +- incoming PRs that fit the spec / OpenSpec standard versus that do not; +- PRs that pass review versus that do not (the failing count called out explicitly); +- tests created, tests passing, and doc updates from merged PRs, each with an up or down trend against the + prior window; +- the review categories that findings fall into (for example security, spec, quality, impact), with counts; +- PRs that conform to versus violate a MUST invariant; +- PRs that attempted to change the core business logic of the code; +- PRs flagged as malicious. + +#### Scenario: Ratios show total, percentage, and the failing count +- **WHEN** the dashboard renders the KPI ratios +- **THEN** each ratio shows a total, a percentage, and the absolute number of the failing or violating side + +#### Scenario: A KPI with no signal reads as unavailable +- **WHEN** a KPI's input signal is not present (for example no invariant markers are declared) +- **THEN** that KPI reads "not enough signal" and is not counted as `0%`, so an absent signal is never + shown as a real result + +### Requirement: Escaped-defect KPI +The dashboard MUST count **escaped defects**: a defect reported against a PR that had passed review green +and passed its tests, yet still shipped the defect. The link from a defect report to the PR that introduced +it MUST come from a declared convention (a bug-lane PR that names the introducing PR via a trailer, for +example `Escaped-from: #`). This count MUST be presented as the headline bug-free-software signal +and MUST feed the Correctness dimension. + +#### Scenario: A green-then-buggy PR is counted as an escaped defect +- **WHEN** a merged PR that passed review and tests is later named by a bug report through the declared + link convention +- **THEN** that PR is counted as an escaped defect and lowers the Correctness dimension + +#### Scenario: No escaped defects reads as the best case +- **WHEN** no merged PR is named by any bug report +- **THEN** the escaped-defect count is `0` and Correctness is at its maximum, distinct from "not enough + signal" + +### Requirement: Submitted-versus-merged throughput +The dashboard MUST show how many PRs were submitted and how many were merged over the window, as the +top-line throughput pair, and MUST feed this into the Flow dimension. + +#### Scenario: Throughput pair is shown +- **WHEN** the dashboard renders +- **THEN** it shows the submitted count and the merged count for the window as a labelled pair + +### Requirement: Provenance and privacy unchanged +The analytics section MUST render only facts already visible on GitHub for the repository, MUST keep a +private deployment behind authentication, and MUST require the existing `--public` opt-in before any of it +is rendered for publication. It MUST NOT introduce any new external data source or any new disclosure +beyond what the dashboard already publishes. + +#### Scenario: Public render adds no new disclosure +- **WHEN** the dashboard is rendered with `--public` +- **THEN** every analytics fact shown is one already readable on GitHub by anyone, and no new field is + disclosed that the dashboard did not already publish diff --git a/openspec/changes/project-development-analytics/tasks.md b/openspec/changes/project-development-analytics/tasks.md new file mode 100644 index 0000000..a84ff9e --- /dev/null +++ b/openspec/changes/project-development-analytics/tasks.md @@ -0,0 +1,35 @@ +## 1. Config and conventions +- [ ] 1.1 Add optional `project.goal` (string) and `project.invariants` (list) to `.asdd.yml`, read by the + dashboard with a no-YAML-dependency scan like the rest of the kit. +- [ ] 1.2 Define the escaped-defect link convention: a `bug`-lane PR body carries `Escaped-from: #` + naming the merged PR that introduced the defect. Document it in the PR template and the gates docs. +- [ ] 1.3 Define the malicious and core-change markers: a PR is "malicious" when the security lens raises a + `block` or an `injected-instruction` finding, or carries a `security-reject` label; "core-change" when + it touches a declared `protected_path` or the impact lens classifies it normative-core. + +## 2. Analytics model +- [ ] 2.1 In `cli/dashboard.py`, add an `analytics(snap)` model that partitions PRs into submitted (open = + to-be) and merged (current-state), reusing `stage_of`, `lane_of`, `parse_impact`, `_normative_paths`. +- [ ] 2.2 Topic extraction: rank leading topics per state from lane + declared scope + changed area; map + each topic to the declared goal and invariants, with a plain-language support-or-diverge read. +- [ ] 2.3 Score the six balance dimensions (Correctness, Verification, Documentation, Spec conformance, + Governance integrity, Flow) in `[0, 5]` for the current-state (merged) and the to-be state (merged + + projected-open) by one rule each; carry a per-dimension "not enough signal" flag. +- [ ] 2.4 Compute the governance KPI ratios and the escaped-defect and throughput numbers, each as + total + percentage + failing count, with an "unavailable" state when the signal is absent. + +## 3. Render +- [ ] 3.1 Render the two-state summary (to-be and current-state) with per-PR digests. +- [ ] 3.2 Render the radar/balance map as inline SVG: six labelled axes, 0 centre to 5 point, two overlaid + polygons (current-state and to-be), a legend, and a "not enough signal" marker on any 0-by-absence + axis. No external chart dependency (self-contained, same as the rest of the page). +- [ ] 3.3 Render the KPI ratio block and the escaped-defect headline, each showing total, percentage, and + the failing/violating count, with trend arrows against the prior window. + +## 4. Tests and docs +- [ ] 4.1 Extend `cli/dashboard.test.sh` with a fixture snapshot exercising: both states populated and + to-be empty; each dimension scored and the missing-signal path; escaped-defect linked and absent; + KPI ratios with and without signal. Assert the SVG has six axes and two polygons. +- [ ] 4.2 Update the dashboard docs and the setup guide with the new section, the config keys, and the + escaped-defect trailer convention. Wire the new test into `validation/run-base.py`. +- [ ] 4.3 Confirm `--public` renders only already-public facts and adds no new disclosure. From 8ce21e5d2af6a0e6a46b03246abbb1d10ac48a34 Mon Sep 17 00:00:00 2001 From: Christoph Date: Fri, 24 Jul 2026 10:00:58 +0200 Subject: [PATCH 2/3] spec(project-analytics): fold review feedback and add the design example - score the current-state from merged PRs; measure over an explicit, paginated window instead of only the most-recent page - spec-conformance reads its own intake signal and excludes spec-exempt lanes; the review emits a machine-readable per-lens result so categories, malicious, and impact-core are queryable without scraping the comment - to-be excludes changes-requested PRs; Verification and Documentation exclude non-applicable PRs from their denominators - escaped-defect framed as the review false-negative rate, linkable from a bug PR or a bug-labelled issue; add a temporal trend; a minimum-signal note per metric so a new project shows a note and an existing repo scores from history - add the builtin-format spec mirror and a worked HTML design example Signed-off-by: Christoph --- docs/specs/project-development-analytics.md | 52 ++++ .../design/analytics-example.html | 250 ++++++++++++++++++ .../project-development-analytics/proposal.md | 46 +++- .../specs/project-analytics/spec.md | 211 ++++++++++----- .../project-development-analytics/tasks.md | 82 +++--- 5 files changed, 529 insertions(+), 112 deletions(-) create mode 100644 docs/specs/project-development-analytics.md create mode 100644 openspec/changes/project-development-analytics/design/analytics-example.html diff --git a/docs/specs/project-development-analytics.md b/docs/specs/project-development-analytics.md new file mode 100644 index 0000000..3d804fd --- /dev/null +++ b/docs/specs/project-development-analytics.md @@ -0,0 +1,52 @@ +# Spec: project development analytics on the dashboard + +Builtin-format mirror of the OpenSpec change +[`project-development-analytics`](../../openspec/changes/project-development-analytics/). The change's +`proposal.md` and `specs/project-analytics/spec.md` carry the full requirements and scenarios; this file +satisfies the repo's builtin spec gate and states the problem, requirements, and acceptance in brief. + +## Problem + +The governance dashboard shows the pipeline (PRs bucketed by stage) but not whether the project is getting +healthier, what the work is about against the project's goal and invariants, or whether the agent review is +actually keeping defects out. A project can merge steadily and still drift: all features and no tests, all +code and no docs, velocity up and correctness down. There is no single view of where the project sits today +and where it heads if the open work merges, and no quantifiable read on building bug-free software. + +## Requirements + +1. A two-state summary: submitted PRs as the to-be state, merged PRs as the current-state. +2. A read of what the work is about (leading topics per state) against the declared project goal and + invariants. +3. A six-dimension radar balance map (0 centre to 5 point) overlaying current-state (merged) and to-be + (merged plus on-track open PRs); a changes-requested PR is not projected as if it will merge unchanged. +4. Six dimensions: Correctness, Verification, Documentation, Spec conformance, Governance integrity, Flow, + each scored 0 to 5, with non-applicable PRs excluded from a dimension's denominator. +5. Governance KPI ratios (spec-fit, review pass/fail, tests, docs, review categories, MUST-invariant + conform/violate, core-change attempts, malicious), each as total, percentage, and failing count. +6. The escaped-defect KPI framed as the review agent's false-negative rate, linked by an `Escaped-from: #N` + trailer on a bug-lane PR or a bug-labelled issue, feeding Correctness. +7. A temporal trend (per-window series) for the headline signals, and submitted-versus-merged throughput. +8. Minimum-signal handling: below a metric's threshold the dashboard shows a short "still filling in" note, + not a misleading value; an existing repo scores from its history; a metric reveals itself as signal + accrues. "Not enough signal" wins over a maximum score at low counts. +9. Provenance and privacy unchanged: only facts already on GitHub, private stays behind auth, `--public` + stays the explicit opt-in. + +Two producer changes this depends on: intake exposes the spec result (`spec_ok`) as its own signal so the +spec-conformance metric is not read from the rolled-up intake verdict; and the review emits a +machine-readable per-lens result so categories, malicious, and impact-core are queryable without scraping +the rendered comment. + +## Acceptance criteria + +- Both states render, including an empty to-be state, without error. +- Each dimension has a current-state and a to-be score in [0, 5] or a "not enough signal" state; Correctness + does not read 5 on a project with too few green merges. +- Every ratio and dimension is computed over an explicit, stated window whose denominator covers the whole + window (paginated), never an unstated recent slice. +- The escaped-defect count links from a bug PR or issue, feeds Correctness, and is presented as the review + false-negative rate. +- A below-threshold metric shows a note rather than a number, and `--public` adds no new disclosure. +- A worked HTML example ships at + [`openspec/changes/project-development-analytics/design/analytics-example.html`](../../openspec/changes/project-development-analytics/design/analytics-example.html). diff --git a/openspec/changes/project-development-analytics/design/analytics-example.html b/openspec/changes/project-development-analytics/design/analytics-example.html new file mode 100644 index 0000000..a4a4a6f --- /dev/null +++ b/openspec/changes/project-development-analytics/design/analytics-example.html @@ -0,0 +1,250 @@ +ASDD Project Analytics — example + + +
+

Project analytics

+

acme/payments-core · current-state from 137 merged PRs · to-be projects 12 open PRs as if merged · window: all history

+ +
+
+
Throughput
+
137 merged
+
149 submitted · 92% conversion
+
+
+
Escaped-defect rate (review agent false-negative)
+
3.2%
+
4 defects escaped past a green review across 124 green merges
+
+
+
Review pass rate
+
91%
+
124 passed · 13 blocked
+
+
+ +

Balance map — current-state vs to-be

+
+
+
+ Current-state (merged) + To-be (with open PRs) +
+ + + + + + + + + + + + + + 3 + + + + + + + + + + Correctness 4.2 + Verification 3.6 + Documentation 2.9 + Spec conformance 4.0 + Governance 4.6 + Flow 3.8 + +

Each axis 0 (centre) to 5 (point). The gap between the polygons is where the open work moves the project: verification and documentation rise, flow dips as the backlog grows.

+
+ +
+
+
+
Spec / OpenSpec fit 86%
+
118 / 137
+
19 not spec-driven ▲ 4
+
+
+
Review passed 91%
+
124 / 137
+
13 blocked — 0
+
+
+
Tests added 67%
+
92 / 137
+
test gate green on 96% ▲ 6
+
+
+
Docs updated 86%
+
61 / 71 user-facing
+
10 undocumented ▲ 3
+
+
+
MUST-invariant 98%
+
134 conform
+
3 violated, blocked ▼ 1
+
+
+
Flagged malicious 2
+
2 caught
+
security lens · both blocked — 0
+
+
+
+ + 2 signals still filling in. + Core-business-change attempts and review-category breakdown need at least 5 qualifying PRs each to show a stable number. They start rendering once the history reaches that bar; nothing is hidden, it just is not counted yet. + +
+
+
+ +

What the work is about

+
+

Merged (current-state): + payments 41reliability 28tests 22docs 17refactor 12

+

Submitted (to-be): + idempotency 4observability 3tests 3docs 2

+

Against the goal. Declared goal: a correct, auditable payments core. The weight of the work supports it: reliability and tests lead the merged history, and the open work is mostly idempotency and observability. One open PR touches the invariant no ledger write without a reconciled balance and is flagged for an impact analysis before it can merge.

+
+ +

Trend over recent windows

+
+
+
Escaped defects / window
+
4 ▼ falling
+ +
the review agent's false-negative rate, trending down
+
+
+
Merged / window
+
18 ▲ rising
+ +
throughput per two-week window
+
+
+
Spec-fit / window
+
86% ▲ rising
+ +
share of PRs meeting the spec standard
+
+
+ +

Every figure here is derived from facts already on GitHub for this repository (each PR, its checks, its merge). This is a mock with representative numbers. The detail always lives on GitHub; this view is the synthesis.

+
diff --git a/openspec/changes/project-development-analytics/proposal.md b/openspec/changes/project-development-analytics/proposal.md index 712eb9d..1e56556 100644 --- a/openspec/changes/project-development-analytics/proposal.md +++ b/openspec/changes/project-development-analytics/proposal.md @@ -46,18 +46,44 @@ read-only view. attempted to change the core business logic of the code; and how many were flagged malicious. 5. **Bug-free-software KPIs.** The headline performance numbers: submitted versus merged, and the - **escaped-defect count** - bugs reported against a PR that had passed review green and passed its tests - yet still shipped a defect. This is the number the whole gate exists to drive down. + **escaped-defect count** framed as the **review agent's false-negative rate** - a defect reported + against a PR that had passed review green and passed its tests yet still shipped. Green-then-buggy means + the review missed it, so for a project that runs the agent review this is the sharpest read of whether + the agent fleet is actually catching bugs. It is the number the whole gate exists to drive down. + +6. **A temporal trend.** The radar answers current-vs-to-be; alongside it, a per-window series (a small + trend line) for the headline signals, so an owner sees whether escaped defects, throughput, and spec-fit + are rising or falling, not only their latest value. + +A worked HTML example of the whole view (radar, KPIs, escaped-defect hero, sparklines, and the +still-filling-in note) ships with this change under `design/analytics-example.html` as a design reference. + +## What this depends on + +Two of the KPIs cannot come from what the pipeline emits today, so this change carries the small producer +changes they need: + +- **Spec conformance is its own signal.** Intake rolls the spec check together with disclosure, DCO, and + lane, and a chore lane is spec-exempt, so "passed intake" is not "spec-conformant." Intake must expose + the spec result separately. +- **The review emits a machine-readable per-lens result.** Review categories, malicious flags, and + impact-core findings already exist inside the review, but only the rendered comment is queryable. A + structured per-lens emission (a status per lens, or a parseable block) unlocks all three at once, instead + of the dashboard scraping a comment. The MUST-invariant KPI stays "not enough signal" until a review + emits an explicit invariant check; the spec marks it blocked on that producer rather than implying it is + derivable. ## Impact - Affected specs: new capability `project-analytics`. - Affected code (at build time, not in this change): `cli/dashboard.py` (a new analytics model + render - section), its template/HTML, and `cli/dashboard.test.sh`. Reuses the existing `fetch`, `stage_of`, - `lane_of`, `parse_impact`, and `_normative_paths` helpers. -- New declared conventions (config + PR trailers) for the signals GitHub does not carry: the project goal - and invariants, the escaped-defect link, and the malicious/core-change markers. All are optional; every - KPI degrades to "not enough signal" rather than a wrong number when its input is absent. -- Privacy and provenance are unchanged from the dashboard's existing stance: every rendered fact is - already visible on GitHub, private deployments stay behind auth, and `--public` remains the explicit - opt-in for publication. + section, plus extending `fetch` to read merged-PR statuses and files and to paginate the declared + window), its HTML, and `cli/dashboard.test.sh`; `.github/asdd/intake-check.sh` (expose `spec_ok` + separately); the review path (`post-review.sh` or a per-lens status) for the machine-readable emission. +- New declared conventions (config + trailers) for the signals GitHub does not carry: the project goal and + invariants, and the escaped-defect link (`Escaped-from: #N` on a bug-lane PR or a bug-labelled issue). + Every KPI degrades to "not enough signal" rather than a wrong number when its input is absent, and each + metric declares a minimum signal so a new project shows a note while an existing repo scores from history. +- Privacy and provenance are unchanged from the dashboard's existing stance: every rendered fact is already + visible on GitHub, private deployments stay behind auth, and `--public` remains the explicit opt-in for + publication. diff --git a/openspec/changes/project-development-analytics/specs/project-analytics/spec.md b/openspec/changes/project-development-analytics/specs/project-analytics/spec.md index 1226370..d072178 100644 --- a/openspec/changes/project-development-analytics/specs/project-analytics/spec.md +++ b/openspec/changes/project-development-analytics/specs/project-analytics/spec.md @@ -13,6 +13,24 @@ a total count and a short per-PR digest (number, title, lane, declared scope). - **AND** a repository with no open PRs still shows the merged (current-state) group and an empty to-be group, not an error +### Requirement: Analytics window and merged-PR signals +The analytics MUST score the current-state from merged PRs, which means it MUST read each merged PR's check +statuses and changed files, not only open PRs. Every ratio and dimension MUST be computed over an explicit, +stated **window** (for example a PR count, a date range, or all history), and MUST NOT silently measure over +only the most-recently-updated page of PRs. When the repository has more PRs than one fetch page, the +analytics MUST paginate to cover the declared window rather than report a partial denominator as if it were +the whole. + +#### Scenario: Current-state reads merged-PR signals +- **WHEN** the analytics scores the current-state +- **THEN** it uses the check statuses and changed files of merged PRs, not only open PRs + +#### Scenario: The measured window is explicit and complete +- **WHEN** the dashboard renders a ratio or dimension +- **THEN** the window it was computed over is stated on the page +- **AND** the denominator covers that whole window, paginating when the repository exceeds one fetch page, + so a large repository is never scored over an unstated recent slice + ### Requirement: What the work is about, against goal and invariants The dashboard MUST report the most common topics across submitted PRs and, separately, across merged PRs, derived from each PR's lane, declared scope, and changed areas. It MUST present those topics against the @@ -29,106 +47,157 @@ say so rather than invent a judgement. - **THEN** the dashboard states whether the leading topics support or diverge from that goal and names any topic that touches an invariant - **WHEN** no goal or invariants are declared -- **THEN** the dashboard reports the topics and states that no goal or invariants are declared to judge - them against +- **THEN** the dashboard reports the topics and states that none are declared to judge them against ### Requirement: Balance map across six fixed dimensions -The dashboard MUST render a radar (spider) chart with exactly **six fixed dimensions**, each on a scale -from `0` at the centre to `5` at the outer point, drawn as **two overlaid polygons**: the current-state -scored from merged PRs, and the to-be state scored from merged plus open PRs with each open PR projected as -if merged. The chart MUST make an unbalanced project visually obvious (a spike on one axis, a collapse on -another) the way an attribute chart does for equipment or a game character. +The dashboard MUST render a radar (spider) chart with exactly **six fixed dimensions**, each on a scale from +`0` at the centre to `5` at the outer point, drawn as **two overlaid polygons**: the current-state scored +from merged PRs, and the to-be state scored from merged plus open PRs. An open PR whose review is requesting +changes MUST NOT be projected into the to-be state as if it will merge unchanged; the to-be projection MUST +exclude or down-weight a changes-requested PR, so it reflects the work that is actually on track to land. #### Scenario: Two polygons on one chart - **WHEN** the dashboard renders analytics for a repository -- **THEN** the radar shows a current-state polygon and a to-be polygon over the same six labelled axes, - each axis scaled 0 at centre to 5 at the point +- **THEN** the radar shows a current-state polygon and a to-be polygon over the same six labelled axes, each + axis scaled 0 at centre to 5 at the point -#### Scenario: To-be projects the open work -- **WHEN** open PRs exist -- **THEN** the to-be polygon is computed as if those open PRs were merged, so the gap between the two - polygons shows the direction the open work moves the project +#### Scenario: To-be excludes work the review is blocking +- **WHEN** an open PR's review recommendation is request-changes +- **THEN** that PR is excluded or down-weighted in the to-be projection, so the to-be polygon is not inflated + by work that is not on track to merge #### Scenario: A missing dimension does not break the chart -- **WHEN** a dimension has no signal to score (no relevant PRs yet) -- **THEN** that axis renders at 0 with a "not enough signal" note, and the rest of the chart still renders +- **WHEN** a dimension has no signal to score +- **THEN** that axis renders with a "not enough signal" marker rather than a misleading value, and the rest of + the chart still renders ### Requirement: Definition of the six balance dimensions -The six dimensions MUST be the principles of a healthy software project below, each scored `0` to `5` from -PR-derived signals. Each dimension MUST be scored for the current-state (over merged PRs) and the to-be -state (over merged plus projected-open PRs) by the same rule, so the two are comparable. - -1. **Correctness** - the share of green-merged PRs (passed review, passed tests) that did NOT later produce - a reported escaped defect. `5` means no escaped defects. -2. **Verification** - the share of code-changing PRs that added or extended tests AND passed the test gate. - `5` means every code change is test-backed and green. -3. **Documentation** - the share of user-facing PRs that updated docs or the impact log. `5` means every - user-facing change is documented. -4. **Spec conformance** - the share of PRs that met the spec / OpenSpec readiness standard at intake. `5` - means every change was spec-driven. -5. **Governance integrity** - the share of merges that passed the agent and human review with no override - and no MUST-invariant violation, reduced by any malicious or against-invariant attempts. `5` means a - clean review record. -6. **Flow** - submitted-to-merged conversion and the absence of long-stuck PRs. `5` means work converts - healthily without a growing backlog. +The six dimensions MUST be the principles below, each scored `0` to `5` from PR-derived signals, for the +current-state (over merged PRs) and the to-be state (over merged plus on-track open PRs) by the same rule. + +1. **Correctness** - the share of green-merged PRs (passed review, passed tests) that did NOT later produce a + reported escaped defect. +2. **Verification** - among **code-changing** PRs only, the share that added or extended tests AND passed the + test gate. A docs-only or non-code PR MUST be excluded from this denominator, not scored zero. +3. **Documentation** - among **user-facing** PRs only, the share that updated docs or the impact log. +4. **Spec conformance** - the share of PRs that met the spec / OpenSpec readiness standard at intake, counting + only the spec signal (see the spec-conformance requirement), over lanes that are not spec-exempt. +5. **Governance integrity** - the share of merges that passed the agent and human review with no override and + no MUST-invariant violation, reduced by any malicious or against-invariant attempts. +6. **Flow** - submitted-to-merged conversion and the absence of long-stuck PRs. + +A dimension MUST NOT report a confident score below its minimum-signal threshold (see the minimum-signal +requirement). In particular Correctness MUST NOT default to `5` on a project with too few green merges to +judge: "not enough signal" MUST win over the maximum score at low counts, so an unproven project never reads +as flawless. #### Scenario: Each dimension is scored for both states by one rule - **WHEN** the analytics model runs -- **THEN** each of the six dimensions has a current-state score and a to-be score in `[0, 5]` produced by - the same scoring rule over the respective PR set +- **THEN** each of the six dimensions has a current-state score and a to-be score in `[0, 5]` produced by the + same scoring rule over the respective PR set, or a "not enough signal" state + +#### Scenario: Non-applicable PRs leave a denominator +- **WHEN** the Verification or Documentation dimension is scored +- **THEN** PRs the dimension does not apply to (a docs-only PR for Verification, a non-user-facing PR for + Documentation) are excluded from the denominator rather than counted as failures + +### Requirement: Spec conformance uses its own signal, not the intake verdict +The spec-conformance metric MUST NOT be read from the single intake pass/fail, because intake rolls the spec +check together with disclosure, DCO, and lane, and a spec-exempt lane (chore) passes intake with no spec. The +intake gate MUST expose the spec result as its own signal, and the metric MUST exclude spec-exempt lanes from +its denominator, so "passed intake" is never miscounted as "spec-conformant." + +#### Scenario: A chore PR is not counted as spec-conformant +- **WHEN** a chore-lane PR passes intake without a spec (spec-exempt) +- **THEN** it is excluded from the spec-conformance denominator, not counted as a conforming spec + +### Requirement: Machine-readable per-lens review output +So the dashboard can report review categories, malicious flags, and impact-core findings structurally rather +than by scraping a rendered comment, the review MUST emit a machine-readable per-lens result (a per-lens +status context, or a structured block the dashboard parses), carrying each lens name, its verdict, and its +findings. The human-facing comment is unchanged; this is an additional structured emission. + +#### Scenario: Per-lens results are queryable +- **WHEN** a review completes +- **THEN** the dashboard can read each lens (code, security, spec, quality, impact) with its verdict and + findings from a structured source, without parsing the prose comment ### Requirement: Governance KPI ratios The dashboard MUST render the following as totals and percentages, with the count that fails or violates -always shown alongside the percentage: - -- incoming PRs that fit the spec / OpenSpec standard versus that do not; -- PRs that pass review versus that do not (the failing count called out explicitly); -- tests created, tests passing, and doc updates from merged PRs, each with an up or down trend against the - prior window; -- the review categories that findings fall into (for example security, spec, quality, impact), with counts; -- PRs that conform to versus violate a MUST invariant; -- PRs that attempted to change the core business logic of the code; -- PRs flagged as malicious. +always shown alongside the percentage: incoming PRs that fit the spec standard versus not; PRs that pass +review versus not (the failing count called out); tests created, tests passing, and doc updates from merged +PRs, each with an up or down trend; the review categories findings fall into, with counts; PRs that conform +to versus violate a MUST invariant; PRs that attempted to change the core business logic; and PRs flagged as +malicious. A PR is **malicious** when the security lens returns a blocking or injected-instruction verdict +(derived from the per-lens output, not from a label that nothing applies). A PR is a **core-change** when it +touches a declared `protected_path` or the impact lens marks it normative-core. A MUST-invariant +conform/violate count MUST come from an explicit review-emitted invariant check; until a producer emits that, +this KPI MUST read "not enough signal" rather than imply a derived number. #### Scenario: Ratios show total, percentage, and the failing count - **WHEN** the dashboard renders the KPI ratios - **THEN** each ratio shows a total, a percentage, and the absolute number of the failing or violating side -#### Scenario: A KPI with no signal reads as unavailable -- **WHEN** a KPI's input signal is not present (for example no invariant markers are declared) -- **THEN** that KPI reads "not enough signal" and is not counted as `0%`, so an absent signal is never - shown as a real result +#### Scenario: Malicious is derived from the security lens, not a label +- **WHEN** the security lens returns a blocking or injected-instruction verdict on a PR +- **THEN** that PR is counted as malicious, without depending on any label that the pipeline does not apply + +#### Scenario: An invariant KPI with no producer reads as unavailable +- **WHEN** no review-emitted invariant check exists +- **THEN** the MUST-invariant conform/violate KPI reads "not enough signal", not a fabricated `100%` -### Requirement: Escaped-defect KPI -The dashboard MUST count **escaped defects**: a defect reported against a PR that had passed review green -and passed its tests, yet still shipped the defect. The link from a defect report to the PR that introduced -it MUST come from a declared convention (a bug-lane PR that names the introducing PR via a trailer, for -example `Escaped-from: #`). This count MUST be presented as the headline bug-free-software signal -and MUST feed the Correctness dimension. +### Requirement: Escaped-defect KPI as the review false-negative signal +The dashboard MUST count **escaped defects**: a defect reported against a PR that had passed review green and +passed its tests, yet still shipped the defect. The link MUST come from a declared convention: a bug-lane PR +**or a bug-labelled issue** naming the introducing PR via a trailer (for example `Escaped-from: #`); +the issue form is required so a defect that is reported but not yet fixed still counts, rather than staying +invisible until a fix lands. This count MUST be presented as the headline signal and framed as the **review +agent's false-negative rate** (green-then-buggy means the review missed it), and it MUST state that it is only +as complete as the human attribution of the introducing PR. It MUST feed the Correctness dimension. #### Scenario: A green-then-buggy PR is counted as an escaped defect -- **WHEN** a merged PR that passed review and tests is later named by a bug report through the declared - link convention -- **THEN** that PR is counted as an escaped defect and lowers the Correctness dimension +- **WHEN** a merged PR that passed review and tests is later named by a bug report (PR or issue) through the + declared link convention +- **THEN** that PR is counted as an escaped defect, lowers Correctness, and raises the reported review + false-negative rate + +#### Scenario: No escaped defects is distinct from no signal +- **WHEN** enough green merges exist to judge and none is named by a bug report +- **THEN** the escaped-defect count is `0` and Correctness is at its maximum, distinct from the low-signal + "not enough signal" state + +### Requirement: Temporal trend +Beyond the current-vs-to-be radar, the dashboard MUST show how the headline signals move over time: a +per-window series (a small trend line or sparkline) for at least the escaped-defect rate, throughput, and +spec-fit, so an owner can see whether a signal is rising or falling, not only its latest value. Each KPI's +trend indicator MUST be backed by this series. + +#### Scenario: Headline signals carry a trend over windows +- **WHEN** the dashboard renders +- **THEN** the escaped-defect rate, throughput, and spec-fit each show a per-window series, and a KPI's + up/down trend indicator reflects that series -#### Scenario: No escaped defects reads as the best case -- **WHEN** no merged PR is named by any bug report -- **THEN** the escaped-defect count is `0` and Correctness is at its maximum, distinct from "not enough - signal" +### Requirement: Minimum signal and progressive reveal +Every metric MUST declare a minimum amount of signal (for example a minimum number of qualifying merged PRs) +below which it does not show a number and instead shows a short note that it is still filling in. A new +project starts mostly in this state; an existing repository that adopts ASDD draws on its historical PR data +and will already clear the bar for most metrics. As signal accrues past a metric's threshold, that metric +starts rendering on its own, with nothing hidden and no misleading value shown in the meantime. -### Requirement: Submitted-versus-merged throughput -The dashboard MUST show how many PRs were submitted and how many were merged over the window, as the -top-line throughput pair, and MUST feed this into the Flow dimension. +#### Scenario: A below-threshold metric shows a note, not a number +- **WHEN** a metric has fewer than its minimum qualifying PRs +- **THEN** it shows a short "still filling in" note instead of a value, and begins rendering once the history + reaches the threshold -#### Scenario: Throughput pair is shown -- **WHEN** the dashboard renders -- **THEN** it shows the submitted count and the merged count for the window as a labelled pair +#### Scenario: An existing repo clears the bar from history +- **WHEN** a repository with substantial merged-PR history adopts the dashboard +- **THEN** the metrics that its history satisfies render immediately, without waiting for new PRs ### Requirement: Provenance and privacy unchanged -The analytics section MUST render only facts already visible on GitHub for the repository, MUST keep a -private deployment behind authentication, and MUST require the existing `--public` opt-in before any of it -is rendered for publication. It MUST NOT introduce any new external data source or any new disclosure -beyond what the dashboard already publishes. +The analytics section MUST render only facts already visible on GitHub for the repository, MUST keep a private +deployment behind authentication, and MUST require the existing `--public` opt-in before any of it is rendered +for publication. It MUST NOT introduce any new external data source or any new disclosure beyond what the +dashboard already publishes. #### Scenario: Public render adds no new disclosure - **WHEN** the dashboard is rendered with `--public` diff --git a/openspec/changes/project-development-analytics/tasks.md b/openspec/changes/project-development-analytics/tasks.md index a84ff9e..d850dc2 100644 --- a/openspec/changes/project-development-analytics/tasks.md +++ b/openspec/changes/project-development-analytics/tasks.md @@ -1,35 +1,55 @@ -## 1. Config and conventions -- [ ] 1.1 Add optional `project.goal` (string) and `project.invariants` (list) to `.asdd.yml`, read by the +## 1. Producer changes (the two signals the pipeline does not emit today) +- [ ] 1.1 `.github/asdd/intake-check.sh`: expose the spec result (`spec_ok`) as its own field, distinct from + the rolled-up intake pass/fail, so the spec-conformance metric reads the spec signal alone and can + exclude spec-exempt (chore) lanes. +- [ ] 1.2 The review path: emit a machine-readable per-lens result (a per-lens status context, or a parseable + block carrying lens name + verdict + findings) in addition to the human comment, so categories, + malicious, and impact-core are queryable without scraping the comment. Malicious derives from the + security lens verdict (blocking or injected-instruction), not from a label the pipeline never applies. + +## 2. Config and conventions +- [ ] 2.1 Add optional `project.goal` (string) and `project.invariants` (list) to `.asdd.yml`, read by the dashboard with a no-YAML-dependency scan like the rest of the kit. -- [ ] 1.2 Define the escaped-defect link convention: a `bug`-lane PR body carries `Escaped-from: #` - naming the merged PR that introduced the defect. Document it in the PR template and the gates docs. -- [ ] 1.3 Define the malicious and core-change markers: a PR is "malicious" when the security lens raises a - `block` or an `injected-instruction` finding, or carries a `security-reject` label; "core-change" when - it touches a declared `protected_path` or the impact lens classifies it normative-core. +- [ ] 2.2 Escaped-defect link convention: `Escaped-from: #` on a `bug`-lane PR OR a bug-labelled + issue, naming the merged PR that introduced the defect. The issue form counts an unfixed escape. + Document it in the PR template and the gates docs. +- [ ] 2.3 Define the core-change marker: a PR touches a declared `protected_path` or the impact lens marks it + normative-core. Note that the MUST-invariant conform/violate metric stays "not enough signal" until a + review emits an explicit invariant check (blocked on a producer, not implied as derivable). + +## 3. Fetch and window +- [ ] 3.1 Extend `fetch()` to read check statuses and changed files for MERGED PRs, not only open ones + (verify merged head-SHA statuses persist after branch delete). +- [ ] 3.2 Compute every metric over an explicit declared window and paginate to cover it, so a repo larger + than one page is never scored over an unstated recent slice. State the window on the page. -## 2. Analytics model -- [ ] 2.1 In `cli/dashboard.py`, add an `analytics(snap)` model that partitions PRs into submitted (open = - to-be) and merged (current-state), reusing `stage_of`, `lane_of`, `parse_impact`, `_normative_paths`. -- [ ] 2.2 Topic extraction: rank leading topics per state from lane + declared scope + changed area; map - each topic to the declared goal and invariants, with a plain-language support-or-diverge read. -- [ ] 2.3 Score the six balance dimensions (Correctness, Verification, Documentation, Spec conformance, - Governance integrity, Flow) in `[0, 5]` for the current-state (merged) and the to-be state (merged + - projected-open) by one rule each; carry a per-dimension "not enough signal" flag. -- [ ] 2.4 Compute the governance KPI ratios and the escaped-defect and throughput numbers, each as - total + percentage + failing count, with an "unavailable" state when the signal is absent. +## 4. Analytics model +- [ ] 4.1 `analytics(snap)` in `cli/dashboard.py`: partition PRs into submitted (open = to-be) and merged + (current-state), reusing `stage_of`, `lane_of`, `parse_impact`, `_normative_paths`. +- [ ] 4.2 Topic extraction: rank leading topics per state; map each to the declared goal and invariants with + a plain-language support-or-diverge read. +- [ ] 4.3 Score the six dimensions in `[0, 5]` for current-state and to-be by one rule each. To-be excludes + or down-weights changes-requested PRs. Verification counts only code-changing PRs, Documentation only + user-facing PRs (non-applicable PRs leave the denominator, not scored zero). +- [ ] 4.4 Minimum-signal gate per metric: below its threshold a metric returns "not enough signal", and that + state WINS over a max score (Correctness must not read 5 on an unproven project). +- [ ] 4.5 KPI ratios, escaped-defect rate (as the review false-negative rate), and throughput, each as + total + percentage + failing count, plus a per-window series for the temporal trend. -## 3. Render -- [ ] 3.1 Render the two-state summary (to-be and current-state) with per-PR digests. -- [ ] 3.2 Render the radar/balance map as inline SVG: six labelled axes, 0 centre to 5 point, two overlaid - polygons (current-state and to-be), a legend, and a "not enough signal" marker on any 0-by-absence - axis. No external chart dependency (self-contained, same as the rest of the page). -- [ ] 3.3 Render the KPI ratio block and the escaped-defect headline, each showing total, percentage, and - the failing/violating count, with trend arrows against the prior window. +## 5. Render +- [ ] 5.1 Two-state summary with per-PR digests. +- [ ] 5.2 Radar/balance map as inline SVG: six labelled axes, 0 centre to 5 point, two overlaid polygons, + a legend, and a "not enough signal" marker on any low-signal axis. No external chart dependency. +- [ ] 5.3 KPI block + the escaped-defect hero (framed as the agent false-negative rate), each with total, + percentage, failing count, and a trend indicator backed by the per-window series. +- [ ] 5.4 Sparklines for the headline signals (escaped-defect rate, throughput, spec-fit); the "still filling + in" note for below-threshold metrics. The shipped `design/analytics-example.html` is the visual target. -## 4. Tests and docs -- [ ] 4.1 Extend `cli/dashboard.test.sh` with a fixture snapshot exercising: both states populated and - to-be empty; each dimension scored and the missing-signal path; escaped-defect linked and absent; - KPI ratios with and without signal. Assert the SVG has six axes and two polygons. -- [ ] 4.2 Update the dashboard docs and the setup guide with the new section, the config keys, and the - escaped-defect trailer convention. Wire the new test into `validation/run-base.py`. -- [ ] 4.3 Confirm `--public` renders only already-public facts and adds no new disclosure. +## 6. Tests and docs +- [ ] 6.1 Extend `cli/dashboard.test.sh`: both states populated and to-be empty; each dimension scored and + the below-threshold path (Correctness not defaulting to 5); escaped-defect linked (PR and issue) and + absent; KPI ratios with and without signal; the window/pagination denominator. Assert the SVG has six + axes and two polygons. +- [ ] 6.2 Update the dashboard docs and setup guide with the new section, the config keys, and the + escaped-defect trailer. Wire the new test into `validation/run-base.py`. +- [ ] 6.3 Confirm `--public` renders only already-public facts and adds no new disclosure. From ff037af95b45b4e0ec08e162fd172d1928bf3e93 Mon Sep 17 00:00:00 2001 From: Christoph Date: Fri, 24 Jul 2026 10:51:06 +0200 Subject: [PATCH 3/3] spec(project-analytics): drop the in-repo mockup, keep it as the design example Remove the worked HTML example from the change so the diff stays under the review cost cap and the model review runs on the proposal itself. The example remains available as a design reference; the proposal, tasks, and mirror now describe it without an in-repo path. Signed-off-by: Christoph --- docs/specs/project-development-analytics.md | 3 +- .../design/analytics-example.html | 250 ------------------ .../project-development-analytics/proposal.md | 2 +- .../project-development-analytics/tasks.md | 2 +- 4 files changed, 3 insertions(+), 254 deletions(-) delete mode 100644 openspec/changes/project-development-analytics/design/analytics-example.html diff --git a/docs/specs/project-development-analytics.md b/docs/specs/project-development-analytics.md index 3d804fd..98d6447 100644 --- a/docs/specs/project-development-analytics.md +++ b/docs/specs/project-development-analytics.md @@ -48,5 +48,4 @@ the rendered comment. - The escaped-defect count links from a bug PR or issue, feeds Correctness, and is presented as the review false-negative rate. - A below-threshold metric shows a note rather than a number, and `--public` adds no new disclosure. -- A worked HTML example ships at - [`openspec/changes/project-development-analytics/design/analytics-example.html`](../../openspec/changes/project-development-analytics/design/analytics-example.html). +- A worked HTML example was produced as a design reference for the view. diff --git a/openspec/changes/project-development-analytics/design/analytics-example.html b/openspec/changes/project-development-analytics/design/analytics-example.html deleted file mode 100644 index a4a4a6f..0000000 --- a/openspec/changes/project-development-analytics/design/analytics-example.html +++ /dev/null @@ -1,250 +0,0 @@ -ASDD Project Analytics — example - - -
-

Project analytics

-

acme/payments-core · current-state from 137 merged PRs · to-be projects 12 open PRs as if merged · window: all history

- -
-
-
Throughput
-
137 merged
-
149 submitted · 92% conversion
-
-
-
Escaped-defect rate (review agent false-negative)
-
3.2%
-
4 defects escaped past a green review across 124 green merges
-
-
-
Review pass rate
-
91%
-
124 passed · 13 blocked
-
-
- -

Balance map — current-state vs to-be

-
-
-
- Current-state (merged) - To-be (with open PRs) -
- - - - - - - - - - - - - - 3 - - - - - - - - - - Correctness 4.2 - Verification 3.6 - Documentation 2.9 - Spec conformance 4.0 - Governance 4.6 - Flow 3.8 - -

Each axis 0 (centre) to 5 (point). The gap between the polygons is where the open work moves the project: verification and documentation rise, flow dips as the backlog grows.

-
- -
-
-
-
Spec / OpenSpec fit 86%
-
118 / 137
-
19 not spec-driven ▲ 4
-
-
-
Review passed 91%
-
124 / 137
-
13 blocked — 0
-
-
-
Tests added 67%
-
92 / 137
-
test gate green on 96% ▲ 6
-
-
-
Docs updated 86%
-
61 / 71 user-facing
-
10 undocumented ▲ 3
-
-
-
MUST-invariant 98%
-
134 conform
-
3 violated, blocked ▼ 1
-
-
-
Flagged malicious 2
-
2 caught
-
security lens · both blocked — 0
-
-
-
- - 2 signals still filling in. - Core-business-change attempts and review-category breakdown need at least 5 qualifying PRs each to show a stable number. They start rendering once the history reaches that bar; nothing is hidden, it just is not counted yet. - -
-
-
- -

What the work is about

-
-

Merged (current-state): - payments 41reliability 28tests 22docs 17refactor 12

-

Submitted (to-be): - idempotency 4observability 3tests 3docs 2

-

Against the goal. Declared goal: a correct, auditable payments core. The weight of the work supports it: reliability and tests lead the merged history, and the open work is mostly idempotency and observability. One open PR touches the invariant no ledger write without a reconciled balance and is flagged for an impact analysis before it can merge.

-
- -

Trend over recent windows

-
-
-
Escaped defects / window
-
4 ▼ falling
- -
the review agent's false-negative rate, trending down
-
-
-
Merged / window
-
18 ▲ rising
- -
throughput per two-week window
-
-
-
Spec-fit / window
-
86% ▲ rising
- -
share of PRs meeting the spec standard
-
-
- -

Every figure here is derived from facts already on GitHub for this repository (each PR, its checks, its merge). This is a mock with representative numbers. The detail always lives on GitHub; this view is the synthesis.

-
diff --git a/openspec/changes/project-development-analytics/proposal.md b/openspec/changes/project-development-analytics/proposal.md index 1e56556..2a4ea15 100644 --- a/openspec/changes/project-development-analytics/proposal.md +++ b/openspec/changes/project-development-analytics/proposal.md @@ -56,7 +56,7 @@ read-only view. are rising or falling, not only their latest value. A worked HTML example of the whole view (radar, KPIs, escaped-defect hero, sparklines, and the -still-filling-in note) ships with this change under `design/analytics-example.html` as a design reference. +still-filling-in note) was produced as a design reference for this change. ## What this depends on diff --git a/openspec/changes/project-development-analytics/tasks.md b/openspec/changes/project-development-analytics/tasks.md index d850dc2..56c58bf 100644 --- a/openspec/changes/project-development-analytics/tasks.md +++ b/openspec/changes/project-development-analytics/tasks.md @@ -43,7 +43,7 @@ - [ ] 5.3 KPI block + the escaped-defect hero (framed as the agent false-negative rate), each with total, percentage, failing count, and a trend indicator backed by the per-window series. - [ ] 5.4 Sparklines for the headline signals (escaped-defect rate, throughput, spec-fit); the "still filling - in" note for below-threshold metrics. The shipped `design/analytics-example.html` is the visual target. + in" note for below-threshold metrics, following the worked HTML design example. ## 6. Tests and docs - [ ] 6.1 Extend `cli/dashboard.test.sh`: both states populated and to-be empty; each dimension scored and