Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 36 additions & 1 deletion NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -4819,6 +4819,41 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-----------------------------------------------------------------------------

Package: golang.org/x/image
Version: v0.47.0
License: BSD-3-Clause
Source: https://cs.opensource.google/go/x/image/+/v0.47.0:LICENSE

Copyright 2009 The Go Authors.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google LLC nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-----------------------------------------------------------------------------

Package: golang.org/x/net
Version: v0.62.0
License: BSD-3-Clause
Expand Down Expand Up @@ -4854,7 +4889,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

-----------------------------------------------------------------------------

Package: golang.org/x/sys/unix
Package: golang.org/x/sys
Version: v0.49.0
License: BSD-3-Clause
Source: https://cs.opensource.google/go/x/sys/+/v0.49.0:LICENSE
Expand Down
41 changes: 22 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,45 +7,48 @@
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/14261/badge)](https://www.bestpractices.dev/projects/14261)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

Every team has one: the maintainer who reads every pull request, never lets red merge, and sends work back in one line. Hive puts that person between your coding agents and your main branch.
You know the one. Reads every pull request. Never lets red merge. Sends it back with one line, and the line is right. Hive puts that person between your coding agents and main.

Hive is like a CI system, except the job is an AI coding agent and the gate is an AI review plus your existing CI.
If you run CI you know the shape. The job is an AI coding agent. The gate is an AI review plus your own CI.

<!-- outcomes:start -->
<!-- generated by src/scripts/outcome-benchmark.sh --readme; do not edit by hand. Report: src/docs/benchmarks/outcomes-2026-10.md -->
Numbers below: 11,571 pull requests, 10 repositories, 15,054 AI reviews (oct '26).
<!-- generated by src/scripts/outcome-benchmark.sh --readme. Do not edit by hand. Report: src/docs/benchmarks/outcomes-2026-10.md -->
Everything below comes from 11,571 live pull requests in 10 repositories and 15,054 reviews (oct '26).

## Accuracy

<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-accuracy-dark.svg"><img alt="2 in 3 pull requests the review agent sent back came back fixed and were approved; 24 minutes typical from sent back to approved" width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-accuracy-light.svg"></picture>
The review agent sent 563 pull requests back. Two out of three came back fixed and approved, usually within 24 minutes.

Humans still merge; what reaches them has passed review and CI. Start at "open PRs only"; once you trust it, turn the [setting](src/docs/acmm-policy-matrix.md) up to merge on green, like a merge queue.

<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-stories-dark.svg"><img alt="Before and after: hivecommons/hive #13523 was sent back once and merged 16 minutes after opening" width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-stories-light.svg"></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-accuracy-dark.png"><img alt="Two out of three pull requests the review agent sent back came back fixed and approved, usually within 24 minutes." width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-accuracy-light.png"></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-stories-dark.png"><img alt="hivecommons/hive #13523 was sent back once and merged 16 minutes after opening." width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-stories-light.png"></picture>

## Security

Hidden instructions in issues or files are screened out before an agent reads them; agents run as a GitHub App, not your PAT.
We planted 63 texts that try to redirect an agent in issues and files. Hive caught 61 before an agent read a word. One harmless text in 12 got flagged by mistake. Agents run as a GitHub App, not on your token.

<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-prompt-injection-dark.svg"><img alt="97% of planted text that tries to redirect an agent (prompt injection) was screened out before an agent read it, with 1 false alarm in 12 harmless texts" width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-prompt-injection-light.svg"></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-prompt-injection-dark.png"><img alt="Hive caught 61 of 63 planted texts that try to redirect an agent before an agent read a word. One harmless text in 12 got flagged by mistake." width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-prompt-injection-light.png"></picture>

## Cost

Tracked like CI minutes; each job's cost is on the dashboard.
Agents spent about $6,125 to merge 2,869 pull requests. Call it $2 a merge, our estimate (aug–oct '26).

<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-cost-dark.png"><img alt="Agents spent about $6,125 to merge 2,869 pull requests, call it $2 a merge." width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-cost-light.png"></picture>

## And the rest

<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-cost-dark.svg"><img alt="about $2 of agent spend per merged pull request" width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-cost-light.svg"></picture>
Every pull request gets the same one-line review, in your style guide's voice. It adds about 15 minutes to a merge.

## Also
<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-also-dark.png"><img alt="Also consistent, readable, efficient and fast." width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-also-light.png"></picture>

<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-also-dark.svg"><img alt="Also: consistent, readable, efficient, fast" width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-also-light.svg"></picture>
Humans still merge. Start with "open PRs only". When you trust it, [turn it up](src/docs/acmm-policy-matrix.md) and it merges on green, like a merge queue that reads the code.

Production data, not a lab run; counts regenerate from live records. Reverts are too new to report. [How we count →](src/docs/benchmarks/outcomes-2026-10.md)
Reverts are too new to report. [How we count →](src/docs/benchmarks/outcomes-2026-10.md)
<!-- outcomes:end -->

## How it works

<p align="center">
<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-how-it-works-dark.svg"><img alt="How Hive works: scan, claim, work, verify, merge, credit" width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-how-it-works-light.svg"></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="src/docs/benchmarks/assets/outcomes-2026-10-card-how-it-works-dark.png"><img alt="How Hive works, from scan to credit." width="880" src="src/docs/benchmarks/assets/outcomes-2026-10-card-how-it-works-light.png"></picture>
</p>

## Run it
Expand All @@ -57,14 +60,14 @@ printf 'HIVE_GITHUB_TOKEN=ghp_...\nHIVE_DASHBOARD_TOKEN=%s\n' "$(openssl rand -h
docker compose -f src/docker-compose.yaml up -d
```

Each line is explained in the [quick start](src/docs/quickstart-docker-compose.md); [Podman and Kubernetes](src/docs/what-is-hive.md#run-it).
The [quick start](src/docs/quickstart-docker-compose.md) explains each line. [Podman and Kubernetes](src/docs/what-is-hive.md#run-it) too.

## Learn more

[Overview](src/docs/what-is-hive.md) · [Architecture](src/docs/architecture.md) · [Security model](src/docs/security-model.md) · [Terminal UI](src/docs/hivectl.md#tui--live-terminal-dashboard) · [Integrations](src/docs/integrations.md) · [Contribute compute](src/docs/contributor-relay.md) · [Contributing](CONTRIBUTING.md) · [Code of Conduct](CODE_OF_CONDUCT.md) · [Adopters](ADOPTERS.md) · [Repositories](docs/repositories.md) · [Changelog](CHANGELOG.md) · [Vulnerabilities](SECURITY.md) · [Telemetry](src/docs/telemetry.md) · [Hosted service](https://hive.hivecommons.dev)

That maintainer now reads every repo. Only run images from `ghcr.io/hivecommons/hive`.
That maintainer now reads every repo. Only run images from `ghcr.io/hivecommons/hive`. Anything else is not ours.

---

Apache-2.0. Hive is a [Cloud Native Computing Foundation](https://www.cncf.io/) sandbox project ([onboarding status](src/docs/cncf-sandbox-onboarding.md)) and follows the [CNCF Code of Conduct](CODE_OF_CONDUCT.md). Copyright The Linux Foundation and Hive contributors; trademarks are used per the [Linux Foundation Trademark Usage page](https://www.linuxfoundation.org/legal/trademark-usage).
Apache-2.0. Hive is a [CNCF](https://www.cncf.io/) sandbox project ([onboarding status](src/docs/cncf-sandbox-onboarding.md)) and follows the [CNCF Code of Conduct](CODE_OF_CONDUCT.md). Copyright The Linux Foundation and Hive contributors. Trademarks follow [Linux Foundation policy](https://www.linuxfoundation.org/legal/trademark-usage).
2 changes: 2 additions & 0 deletions changelog.d/changed-readme-card-png.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
- `hivectl evidence benchmark` now also draws the README outcome cards as 2× PNGs with an embedded font (flat colours, no glow), gains `--from-json` to re-render committed assets without the evidence tree, and the README embeds the PNGs so the cards look the same in every browser.
- The README and its outcome cards are written as short full sentences in plain speech, with the numbers inside the sentences. A test now rejects semicolons and label lines in the README prose and card text.
7 changes: 4 additions & 3 deletions src/docs/benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Reproducible comparison of pull requests that went through Hive's review swarm (

| Month | Report | Charts |
|---|---|---|
| 2026-10 | [outcomes-2026-10.md](outcomes-2026-10.md) | [bars](assets/outcomes-2026-10-bars.svg) · [trend](assets/outcomes-2026-10-trend.svg) · README cards `assets/outcomes-2026-10-card-*.svg` |
| 2026-10 | [outcomes-2026-10.md](outcomes-2026-10.md) | [bars](assets/outcomes-2026-10-bars.svg) · [trend](assets/outcomes-2026-10-trend.svg) · README cards `assets/outcomes-2026-10-card-*.{svg,png}` |

Each report ships with `outcomes-YYYY-MM.json`, the computed `Report` the markdown, charts, README cards and README block are rendered from, plus the inputs that are not derivable from the evidence tree: `cost-YYYY-MM.json` (saved `GET /api/cost`), `stories-YYYY-MM.json` (curated caught → fixed → merged examples, timestamps taken from the evidence bundles) and `red-team-YYYY-MM.json` (prompt-injection evaluation counts from [ioscan-red-team.md](../ioscan-red-team.md)). `pkg/outcomebench` `TestCommittedReportMatchesReadme` re-renders the newest JSON and fails when the committed markdown, SVGs, cards or the repository README's `<!-- outcomes:start -->…<!-- outcomes:end -->` block differ, so none of them can drift.

Expand All @@ -17,7 +17,8 @@ The repository README shows this data as plain-language cards (one number, one s
- A production-count card is drawn only when its base is at least `--readme-min-base` (default 200), separate from the n ≥ 10 report threshold. The 43-merge revert bucket, for example, stays in the report.
- The checks-passing card is drawn only when a check result is recorded on at least 80 % of reviewed merges; below that the pass rate would describe a biased subset.
- No statistical vocabulary (intervals, medians, cohorts, n=) appears in the README or on a card; `TestReadmeVocabulary` enforces the list and `TestReadmeProseBudget` keeps README prose under 600 words and 10 sections.
- Every card is a real-text SVG in dark and light (`<picture>` + `prefers-color-scheme`) using the dashboard's own palette.
- Every card is drawn twice from the same instructions: a real-text SVG (the vector source) and a 2× PNG in dark and light. The README embeds the PNG (`<picture>` + `prefers-color-scheme`), because GitHub renders SVG text in whatever monospace the reader has and filters differ by browser; the PNG is rasterised in Go with an embedded IBM Plex Mono (SIL OFL 1.1, `pkg/outcomebench/fonts/`), no external renderer. Flat colours only: no gradient, glow or filter. Each PNG carries the SHA-256 of its SVG in a `tEXt` chunk; the drift test checks that hash and the dimensions rather than pixels, which differ slightly by platform.
- The three stat cards (accuracy, security, cost) are the same height, one figure each, so the README reads as a set.
- The prompt-injection card comes from Hive's own evaluation set, says so on the card, and is not subject to the minimum base because it is not a production count.

Each report's "What the README shows and leaves out" section records, per card, whether it was drawn and on what base.
Expand All @@ -35,7 +36,7 @@ src/scripts/outcome-benchmark.sh --root /data/evidence --month 2026-10 \
--readme # also rewrite README.md's outcomes block
```

The script wraps `hivectl evidence benchmark`; see [hivectl.md](../hivectl.md#evidence--benchmark) for every flag. To take a read-only copy of a spoke's evidence: `kubectl exec deploy/hive -c hive -- tar czf - -C /data evidence | tar xzf -`.
The script wraps `hivectl evidence benchmark`; see [hivectl.md](../hivectl.md#evidence--benchmark) for every flag. After changing the card layout or copy, redraw the committed assets and README block from the newest committed JSON without the evidence tree: `src/scripts/outcome-benchmark.sh --rerender --readme` (`hivectl evidence benchmark --from-json`). To take a read-only copy of a spoke's evidence: `kubectl exec deploy/hive -c hive -- tar czf - -C /data evidence | tar xzf -`.

## Metrics

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
35 changes: 16 additions & 19 deletions src/docs/benchmarks/assets/outcomes-2026-10-card-accuracy-dark.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading