rust-corpus builds 306 declared crates, 1,051 in the resolved graph, under Please. scripts/differential.sh puts the same request to cargo when a crate fails, which is what tells a bug in these rules apart from a crate that genuinely cannot be built as asked.
That is a two-way harness used as a debugging aid. Making it a four-way harness turns it into the evidence.
What it would be
The same crate list built under all four, with a published pass rate per system:
| system |
third-party resolution |
| Please |
rust_repo plus offline PubGrub in the build graph |
| Bazel |
rules_rust plus crate_universe, which runs cargo |
| Buck2 |
reindeer, which vendors and generates |
| Cargo |
itself |
Why it is worth the work
Every comparison claim currently rests on documented design for three of the four. docs/COMPARISON.md says so explicitly, and the rules_rust column carries a caveat rather than a number. A pass rate measured on the same 306 crates replaces every one of those caveats with a figure, and it is the thing that makes a headline claim in the README defensible rather than promotional.
It also cuts the other way, which is the point: if rules_rust builds crates we cannot, that is a list of bugs.
Shape
Lives in rust-corpus, not here. Each system gets its own setup directory and its own runner, and one script drives all four and writes a table. The crate list is already chosen for the mechanisms it breaks rather than by download count, which is what makes the result interesting.
Sequence worth considering
Bazel first, since rules_rust is the comparison's weakest column and the one people will challenge. Buck2 after.
How you would know
A table of four pass rates over the same crate list, reproducible, with each failure attributed to a system rather than to the crate.
rust-corpus builds 306 declared crates, 1,051 in the resolved graph, under Please.
scripts/differential.shputs the same request to cargo when a crate fails, which is what tells a bug in these rules apart from a crate that genuinely cannot be built as asked.That is a two-way harness used as a debugging aid. Making it a four-way harness turns it into the evidence.
What it would be
The same crate list built under all four, with a published pass rate per system:
rust_repoplus offline PubGrub in the build graphrules_rustpluscrate_universe, which runs cargoreindeer, which vendors and generatesWhy it is worth the work
Every comparison claim currently rests on documented design for three of the four.
docs/COMPARISON.mdsays so explicitly, and the rules_rust column carries a caveat rather than a number. A pass rate measured on the same 306 crates replaces every one of those caveats with a figure, and it is the thing that makes a headline claim in the README defensible rather than promotional.It also cuts the other way, which is the point: if rules_rust builds crates we cannot, that is a list of bugs.
Shape
Lives in rust-corpus, not here. Each system gets its own setup directory and its own runner, and one script drives all four and writes a table. The crate list is already chosen for the mechanisms it breaks rather than by download count, which is what makes the result interesting.
Sequence worth considering
Bazel first, since
rules_rustis the comparison's weakest column and the one people will challenge. Buck2 after.How you would know
A table of four pass rates over the same crate list, reproducible, with each failure attributed to a system rather than to the crate.