Skip to content

The rules_rust column is unverified, and one row was wrong #55

Description

@jackmarsh

docs/COMPARISON.md compares three systems across 60 features. Two columns are grounded: rust-rules is measured in this repository, and Cargo is recorded from documented behaviour. The rules_rust column says of itself that it is

recorded from its documented design and rule set rather than from runs on this machine, which is the one column here that has not been measured first hand.

That caveat covers unmeasured. It does not cover wrong, and one row was wrong.

The row that was wrong

Rust to C headers claimed rules_rust was "yes. Supported". It has no cbindgen at all:

  • extensions/ holds bindgen, mdbook, prost, pyo3 and wasm_bindgen
  • a code search over bazelbuild/rules_rust returns 0 hits for cbindgen, against 23 for rust_bindgen and 41 for wasm_bindgen, so the search works and the zero is real

Corrected. We were understating ourselves and overstating them in the same cell.

Why this is worth an issue rather than a fix

Twelve further rows say "yes. Supported" for rules_rust in exactly that wording, which is the phrasing that carried the error. Any of them could be the same mistake. The comparison is used to make claims about this project, so a column nobody has checked is a liability in both directions: it can flatter us, and it can flatter them.

What it would take

Check each rules_rust cell against the repository rather than against recollection: its rule set, its extensions, and its documented attributes. A code search per feature is enough to separate "has a rule for this" from "does not", which is what most of the rows assert.

Cheap and mechanical. The output is a column that can be cited.

How you would know

Every rules_rust cell names the rule, extension or attribute that backs it, or says plainly that it is absent.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationevidenceMeasurement, corpora and differential testing

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions