Repository navigation
π docs: rewrite README around plain-language evidence cards - #14603
Conversation
Generate dark/light SVG cards from the committed outcome snapshot (sent back before merge, two real pull requests, prompt injection, cost, an also strip and a how-it-works ladder); the README block lists Accuracy, Security, Cost and Also with no statistical vocabulary and a minimum base of 200 for production counts. Detailed charts and intervals stay in src/docs/benchmarks; the former overview moves to src/docs/what-is-hive.md. Adds prose-budget, vocabulary and card tests. Fixes #14602 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of No conflicting open PRs or same-file overlaps were found. |
The Compose quick start contract requires every install recipe to write src/.env and set HIVE_DASHBOARD_TOKEN. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
The Compose quick start contract requires every install recipe to write src/.env and set HIVE_DASHBOARD_TOKEN. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
43beb18
βΉοΈ Sentinel notice β informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @clubanderson, head
Hive did not add the |
Replace the sent-back-before-merge card, which counted reviewer objections rather than outcomes, with two follow-up measures computed per head from the same evidence bundles: the share of sent-back pull requests that came back fixed and were approved (369 of 563), and the typical time from change request to approval of the fixed version (24 minutes, 369 PRs). The report states why reverts, post-fix CI and reopened issues are not drawn yet, and the methodology defines the new measures. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
8a7a01d
βΉοΈ Sentinel notice β informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @clubanderson, head
Hive did not add the |
- One card builder: eyebrow, story headline, amber numerals with glow, footnote; 880px content width declared at 2x (1760) for crisp rendering - Warm near-black surface and off-white text; light theme via <picture> - Accuracy tiles merged into one card with two stats; two-bar time comparison in the Also strip; numbered how-it-works ladder - Drift check and tests updated; old came-back-fixed/time-to-approval assets removed Signed-off-by: Andrew Anderson <andy@clubanderson.com>
48a8777
βΉοΈ Sentinel notice β informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @clubanderson, head
Hive did not add the |
- Open with the maintainer who reads every pull request, then the CI analogy; numbers follow the picture - Two actor names only (coding agent, review agent); CI vocabulary in the how-it-works ladder (webhook trigger, ephemeral runner, required checks, PRs only -> merge on green, co-author trailer) - Before/after scenario card from hive#13523 with pluk#328 in the footnote - Dates as (oct '26); vocabulary ratchet bans insider nouns and ISO dates; prose ratchet lowered to 300 words Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
@clubanderson is auto-promoted for contributor-lane merging on this PR: the last 5 merged PRs by this author in this hive's repositories have no revert or sentinel-alert evidence. CI, reviewer verdict, diff fences, soak, holds and branch protection still apply. |
c3fb016
βΉοΈ Sentinel notice β informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @clubanderson, head
Hive did not add the |
|
merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at c3fb016. |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Fixes #14602
Builds on #14588 (merged). Opened as a draft on purpose: PRs under this login can auto-merge on green through the trusted-author lane, and the owner asked to review the README copy first. Do not mark ready without that review.
What changes
README β 1,727 β 336 prose words, 23 β 7 H2 sections. Order: tagline β scale line β Accuracy β Security β Cost β Also β How it works β Run it β Learn more. Every number is in a generated card; the prose is one or two flat sentences per section, with the limit of each number stated in the same section (not a controlled comparison; revert rates indistinguishable; cost is an estimate; prompt-injection numbers are from Hive's own test set).
Cards (
src/pkg/outcomebench/cards.go) β real-text SVGs in dark and light, swapped with<picture>/prefers-color-scheme, using the dashboard palette. Emitted byhivectl evidence benchmarknext to the existing charts and covered by the drift test, sosrc/scripts/outcome-benchmark.sh --readmeregenerates README and cards together.changes_requestedverdict pushed a later head that received anapproveverdict, Oct 2026 (Followup.FixedAndApproved)changes_requestedβ firstapproveon a later head, 369 PRs (Followup.TimeToApproval)stories-2026-10.json, timestamps from the evidence bundlesred-team-2026-10.json, from ioscan-red-team.md); labelled "not production traffic"cost-2026-10.json, saved/api/cost)Verdicts;TimeToMerge1,455 reviewed vs 2,224 without reviewAccuracy rework (second round) β the "1 in 3 sent back" card counted reviewer objections, not outcomes, and was dropped from the README (it stays in the report tables). Candidates computed over the full evidence history, all 10 repos (every reviewed merge is in Oct 2026, so widening the window did not change any base):
The README caveat sentence under the cards states the 140 / 133 / 19 split and that revert figures need older merges. Methodology bullets define both new measures; "What the README shows and leaves out" lists each cut with its base.
Also not shown in README, and why: revert rate (base 43 < 200 minimum, and indistinguishable); checks-passing at merge (check result recorded on only 55% of reviewed merges, under the 80% floor); fleet attention list and UI screenshots (over the card cap; no live-cost screenshot showing a priced run exists).
Generator rules β
--readme-min-base(default 200) for production-count cards, separate from the report's n β₯ 10;--storiesand--red-teaminputs; README block has no statistical vocabulary. Detailed charts, intervals and methodology remain insrc/docs/benchmarks/.Tests β
TestReadmeProseBudget(β€ 600 words, β€ 10 H2),TestReadmeVocabulary(statistical and marketing word list across README prose and card text),TestCardsRespectMinimumBase,TestLoadStoriesAndRedTeam; drift test extended to every card and the ladder picture.Relocated, not deleted β the former README overview (what Hive is, three rules, hub and spoke, contributor relay, project vs hosted, does-it-fit, run-it table, ACMM table, architecture, TUI, works-with, related, repositories/adopters/contributing/security pointers) is now
src/docs/what-is-hive.md, listed indocumentation-map.md. External anchors into the removed README sections (bin/README.md,security-insights.yml,CONTRIBUTING.md,examples/kubestellar/README.md) were redirected.Rendered
Latest push: written for readers who know CI/CD but not Hive. Opens with the maintainer who reads every pull request, then the CI analogy, then numbers. Only two actor names (coding agent, review agent); CI vocabulary in the ladder; dates as
(oct '26); auto-merge framed as the earned end state ("PRs only β merge on green"). Prose: 245 words, 7 sections (ratchet 300 / 10). Vocabulary ratchet now also bans insider nouns (reviewer, scanner, verdict, ACMM, spoke, kick, β¦) and ISO dates.Dark:
Light:
Before/after scenario card (hive#13523, pluk#328 in the footnote) and the how-it-works ladder:
For the owner's eye before merge
"Came back fixed and were approved" measures that the author acted on the reviewer and the fix passed re-review. It is not proof the shipped code was correct; the outcome measures (reverts, reopened issues) have too little matured reviewed history yet and are listed as not shown.
The 140 same-head merges are framed as "another review pass approved the same version" because 133 of those heads carry an
approvefrom a different perspective alongside thechanges_requested. Only 5 human actions are recorded on those heads, so this is gate consensus, not human override.The prompt-injection card is exempt from the 200 minimum base because it is a labelled self-run eval, not a production count. Say the word and it comes out.
The "Caught" sentences on the story card are my plain-English paraphrase of the posted reviews (linked from the card titles and the report).
Side-by-side layout of the small cards depends on GitHub's rendering width; each is 280 px.
The security sentence ("every merge is signed off and credits the issue author") describes the DCO sign-off and Co-authored-by attribution path; it is not measured by the benchmark.