Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .config/nextest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,11 @@ slow-timeout = { period = "10s", terminate-after = 4 }
[[profile.ci.overrides]]
filter = "test(=fresh_init_then_gate_passes_for_every_class)"
slow-timeout = { period = "10s", terminate-after = 4 }

# These tests run real `cargo nextest` on a small crate, one build each, for
# the kit suite only; they match nothing in an adopter. Alone each ran in
# 1.4 s to 6.2 s. Under the full suite on a loaded runner they take longer,
# so they get 60 s; the profile timeout is not raised.
[[profile.ci.overrides]]
filter = "binary(gate_credit_nextest)"
slow-timeout = { period = "10s", terminate-after = 6 }
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,10 @@ fn encode_empty_emits_single_zero_block() {
}
```

#### 6.5.1 What the gate reads

The gate runs the `acceptance` layer the way it runs `property` (§9.5): the same nextest command, narrowed to the tests it selects by name, with its own filter `gates.acceptance.test_filter` (default `acceptance`). A test counts for the layer when one `::`-separated segment of its nextest name starts with the filter, so `acceptance_encode` and `journeys::acceptance_encode` count and `an_acceptance_helper` does not. When the layer takes the coverage run instead of running itself, it passes only if that run's JUnit report holds a passed test that matches.

### 6.6 Compile-fail / UI (proc-macros, tricky APIs)

`trybuild` or `ui_test`. These are the mutation tests of the type system: if a change makes an intended failure compile, the suite must go red.
Expand Down Expand Up @@ -596,6 +600,16 @@ Rules:

A good property kills operator mutants that example tests miss (`<` vs `<=`, off-by-one on max). If a survivor is an inequality on a bound, add a property at the bound, do not add a single magic example unless the bound is a spec constant.

### 9.5 What the gate reads

`gates.property.test_filter` (default `property`) names the tests that count as property tests: a test counts when one `::`-separated segment of its nextest name starts with the filter. The `property` layer passes only on a passed test that matches.

- **It runs itself** when no coverage run is credited. Its command is the shared nextest run with `-E 'test(/(^|::)<filter>/)'` and `--no-tests=fail`, so a filter that selects nothing fails the layer. A filter is letters, digits and underscore only (`^[A-Za-z0-9_]+$`), so it is a plain regex literal and needs no escaping. `check` refuses any other value, and the gate exits 2.
- **It takes the coverage run** when its command carries `--workspace`, `--all-features` and `--profile ci`. The coverage run must also have written a nextest JUnit report, holding at least one passed test that matches. The gate reads the file where nextest wrote it: the workspace root from `cargo metadata`, then nextest's `[store] dir` from `.config/nextest.toml` (default `target/nextest`), then `ci/<path>` with `path` from `[profile.ci.junit]`. `CARGO_TARGET_DIR` and `build.target-dir` do not move it. A report dated before the credited run began, or in the future, was not written by that run, so it is stale and not evidence; the gate never deletes it. No match, only failed or skipped matches, no report, or a stale report is `fail`. The report's `detail`, the failure line and `problems` all name the filter, the path read (relative to the workspace root), and the fix.
- **The same rule** applies to the whole workspace, not to one crate, so credit equals running.

The same keys exist for `acceptance` under `gates.acceptance` (§6.5.1).

---

## 10. Fuzzing (Tier 3)
Expand Down
3 changes: 3 additions & 0 deletions crates/cargo-tops/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,5 +39,8 @@ toml = "1"
# `openat`/`mkdirat` without `unsafe`. 1.1.6 clears RUSTSEC-2026-0486.
rustix = { version = "1.1.6", default-features = false, features = ["fs", "std"] }

[dev-dependencies]
proptest = "1"

[lints]
workspace = true
67 changes: 67 additions & 0 deletions crates/cargo-tops/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,32 @@ pub(crate) struct Gates {
/// Absent means the remote's default branch (`origin/HEAD`).
#[serde(skip_serializing_if = "Option::is_none")]
pub base_ref: Option<String>,
/// The `property` gate. Only `test_filter` is read here; the rest of the
/// object stays as written.
#[serde(default, skip_serializing)]
pub property: Option<serde_json::Value>,
/// The `acceptance` gate, as [`Gates::property`].
#[serde(default, skip_serializing)]
pub acceptance: Option<serde_json::Value>,
}

impl Gates {
/// The `test_filter` of `layer` (`property` or `acceptance`): the name
/// prefix a test must start a `::` segment with to count (PROTOCOL.md
/// §6, §9). It defaults to the layer's id.
pub(crate) fn test_filter(&self, layer: &str) -> String {
let gate = match layer {
"property" => &self.property,
"acceptance" => &self.acceptance,
_ => &None,
};
gate.as_ref()
.and_then(|gate| gate.get("test_filter"))
.and_then(serde_json::Value::as_str)
.filter(|filter| !filter.is_empty())
.unwrap_or(layer)
.to_string()
}
}

#[derive(Debug, Deserialize, Serialize, PartialEq)]
Expand All @@ -69,3 +95,44 @@ pub(crate) struct CrapGate {
pub(crate) struct MutationGate {
pub unexplained_survivors_max: i64,
}

#[cfg(test)]
mod tests {
use super::Gates;
use serde_json::json;

fn gates(extra: &serde_json::Value) -> Gates {
let mut doc =
json!({"coverage": {}, "crap": {}, "mutation": {"unexplained_survivors_max": 0}});
if let (Some(base), Some(more)) = (doc.as_object_mut(), extra.as_object()) {
base.extend(more.clone());
}
serde_json::from_value(doc).expect("gates")
}

#[test]
fn a_test_filter_defaults_to_the_layer_id() {
let none = gates(&json!({}));
assert_eq!(none.test_filter("property"), "property");
assert_eq!(none.test_filter("acceptance"), "acceptance");
let other = gates(&json!({"property": {"framework": "proptest"}}));
assert_eq!(other.test_filter("property"), "property");
}

#[test]
fn each_layer_reads_its_own_test_filter() {
let set = gates(&json!({
"property": {"test_filter": "prop_"},
"acceptance": {"test_filter": "journey_"},
}));
assert_eq!(set.test_filter("property"), "prop_");
assert_eq!(set.test_filter("acceptance"), "journey_");
assert_eq!(set.test_filter("unit"), "unit");
}

#[test]
fn an_empty_test_filter_is_the_default() {
let empty = gates(&json!({"property": {"test_filter": ""}}));
assert_eq!(empty.test_filter("property"), "property");
}
}
9 changes: 7 additions & 2 deletions crates/cargo-tops/src/credit.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
//! a layer and [`recorded_outcome`] after the source layer has finished.
//! A missing or failed coverage run is not a pass. A coverage command that
//! lacks `--workspace`, `--all-features`, or the nextest profile grants no
//! credit, and `unit`, `integration`, and `property` run themselves.
//! credit, and `unit`, `integration`, `property`, and `acceptance` run
//! themselves. `property` and `acceptance` also need evidence from the run
//! ([`crate::evidence`]).
#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used))]
#![allow(dead_code, reason = "main.rs calls the credit decision")]

Expand Down Expand Up @@ -66,7 +68,10 @@ pub(crate) fn recorded_outcome(query: &Query<'_>) -> Option<Outcome> {
fn takes_coverage(query: &Query<'_>) -> bool {
query.coverage_in_range
&& coverage_grants_credit(query.coverage_argv)
&& matches!(query.layer, "unit" | "integration" | "property")
&& matches!(
query.layer,
"unit" | "integration" | "property" | "acceptance"
)
}

fn takes_unit(query: &Query<'_>) -> bool {
Expand Down
Loading
Loading