Skip to content

Forbid check-pass/build-pass/run-pass directives in incremental tests #155589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rust-bors merged 4 commits into from
Apr 22, 2026
Merged

Forbid check-pass/build-pass/run-pass directives in incremental tests #155589

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
793c646
Migrate `bfail`/`build-pass` tests to `bpass` (1/2)
Zalathar Apr 21, 2026
0ea8958
Migrate `bfail`/`build-pass` tests to `bpass` (2/2)
Zalathar Apr 21, 2026
edbb862
Add support for `cpass` incremental revisions
Zalathar Apr 21, 2026
87ec57f
Forbid `*-pass` directives in incremental tests
Zalathar Apr 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions src/doc/rustc-dev-guide/src/tests/compiletest.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,12 @@ series of steps.
Compiletest starts with an empty directory with the `-C incremental` flag, and
then runs the compiler for each revision, reusing the incremental results from previous steps.

The revisions should start with:
Each revision name must start with one of:

* `bfail` — the test should fail to compile
* `bpass` — the test should compile successully
* `rpass` — the test should compile and run successfully
* `cpass` - the test must compile successfully (check build, no codegen)
* `bfail` — the test must fail to compile (full build, with codegen)
* `bpass` — the test must compile successully (full build, with codegen)
* `rpass` — the test must compile and run successfully

To make the revisions unique, you should add a suffix like `rpass1` and `rpass2`.

Expand Down
12 changes: 6 additions & 6 deletions src/doc/rustc-dev-guide/src/tests/directives.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,11 @@ See [Controlling pass/fail expectations](ui.md#controlling-passfail-expectations

| Directive | Explanation | Supported test suites | Possible values |
|-----------------------------|---------------------------------------------|-------------------------------------------|-----------------|
| `check-pass` | Building (no codegen) should pass | `ui`, `crashes`, `incremental` | N/A |
| `check-pass` | Building (no codegen) should pass | `ui`, `crashes` | N/A |
Comment thread
jieyouxu marked this conversation as resolved.
| `check-fail` | Building (no codegen) should fail | `ui`, `crashes` | N/A |
| `build-pass` | Building should pass | `ui`, `crashes`, `codegen`, `incremental` | N/A |
| `build-pass` | Building should pass | `ui`, `crashes`, `codegen` | N/A |
| `build-fail` | Building should fail | `ui`, `crashes` | N/A |
| `run-pass` | Program must exit with code `0` | `ui`, `crashes`, `incremental` | N/A |
| `run-pass` | Program must exit with code `0` | `ui`, `crashes` | N/A |
| `run-fail` | Program must exit with code `1..=127` | `ui`, `crashes` | N/A |
| `run-crash` | Program must crash | `ui` | N/A |
| `run-fail-or-crash` | Program must `run-fail` or `run-crash` | `ui` | N/A |
Expand All @@ -90,9 +90,9 @@ comparison](ui.md#output-comparison) and [Rustfix tests](ui.md#rustfix-tests) fo

| Directive | Explanation | Supported test suites | Possible values |
|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------|-----------------------------------------------------------------------------------------|
| `check-run-results` | Check run test binary `run-{pass,fail}` output snapshot | `ui`, `crashes`, `incremental` if `run-pass` | N/A |
| `error-pattern` | Check that output contains a specific string | `ui`, `crashes`, `incremental` if `run-pass` | String |
| `regex-error-pattern` | Check that output contains a regex pattern | `ui`, `crashes`, `incremental` if `run-pass` | Regex |
| `check-run-results` | Check run test binary `run-{pass,fail}` output snapshot | `ui`, `crashes`, `incremental` | N/A |
| `error-pattern` | Check that output contains a specific string | `ui`, `crashes`, `incremental` | String |
| `regex-error-pattern` | Check that output contains a regex pattern | `ui`, `crashes`, `incremental` | Regex |
| `check-stdout` | Check `stdout` against `error-pattern`s from running test binary[^check_stdout] | `ui`, `crashes`, `incremental` | N/A |
| `normalize-stderr-32bit` | Normalize actual stderr (for 32-bit platforms) with a rule `"<raw>" -> "<normalized>"` before comparing against snapshot | `ui`, `incremental` | `"<RAW>" -> "<NORMALIZED>"`, `<RAW>`/`<NORMALIZED>` is regex capture and replace syntax |
| `normalize-stderr-64bit` | Normalize actual stderr (for 64-bit platforms) with a rule `"<raw>" -> "<normalized>"` before comparing against snapshot | `ui`, `incremental` | `"<RAW>" -> "<NORMALIZED>"`, `<RAW>`/`<NORMALIZED>` is regex capture and replace syntax |
Expand Down
7 changes: 0 additions & 7 deletions src/tools/compiletest/src/directives.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -439,13 +439,6 @@ impl TestProps {
(TestMode::Ui, _) => (),
(TestMode::Crashes, _) => (),
(TestMode::Codegen, "build-pass") => (),
(TestMode::Incremental, _) => {
// FIXME(Zalathar): This only detects forbidden directives that are
// declared _after_ the incompatible `//@ revisions:` directive(s).
if self.revisions.iter().any(|r| !r.starts_with("bfail")) {
panic!("`{s}` directive is only supported in `bfail` incremental tests")
}
}
(mode, _) => panic!("`{s}` directive is not supported in `{mode}` tests"),
};
let pass_mode = if config.parse_name_directive(ln, "check-pass") {
Expand Down
9 changes: 6 additions & 3 deletions src/tools/compiletest/src/runtest.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,12 +327,15 @@ impl<'test> TestCx<'test> {
TestMode::Incremental => {
let revision =
self.revision.expect("incremental tests require a list of revisions");
if revision.starts_with("bpass") || revision.starts_with("rpass") {
if revision.starts_with("cpass")
|| revision.starts_with("bpass")
|| revision.starts_with("rpass")
{
true
} else if revision.starts_with("bfail") {
pm.is_some()
false
} else {
panic!("revision name must begin with `bfail`, `bpass`, or `rpass`");
panic!("revision name must begin with `cpass`, `bfail`, `bpass`, or `rpass`");
}
}
mode => panic!("unimplemented for mode {:?}", mode),
Expand Down
12 changes: 10 additions & 2 deletions src/tools/compiletest/src/runtest/incremental.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use super::{FailMode, ProcRes, TestCx, WillExecute};
use super::{Emit, FailMode, PassMode, ProcRes, TestCx, WillExecute};

impl TestCx<'_> {
pub(super) fn run_incremental_test(&self) {
Expand Down Expand Up @@ -31,7 +31,9 @@ impl TestCx<'_> {
write!(self.stdout, "revision={:?} props={:#?}", revision, self.props);
}

if revision.starts_with("bpass") {
if revision.starts_with("cpass") {
self.run_cpass_test();
} else if revision.starts_with("bpass") {
self.run_bpass_test();
} else if revision.starts_with("rpass") {
self.run_rpass_test();
Expand All @@ -42,6 +44,12 @@ impl TestCx<'_> {
}
}

fn run_cpass_test(&self) {
let proc_res = self.compile_test(WillExecute::No, Emit::Metadata);
self.check_if_test_should_compile(None, Some(PassMode::Check), &proc_res);
self.check_compiler_output_for_incr(&proc_res);
}

fn run_bpass_test(&self) {
let emit_metadata = self.should_emit_metadata(self.pass_mode());
let proc_res = self.compile_test(WillExecute::No, emit_metadata);
Expand Down
23 changes: 11 additions & 12 deletions tests/incremental/add_private_fn_at_krate_root_cc/struct_point.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,28 @@
// crate. This should not cause anything we use to be invalidated.
// Regression test for #36168.

//@ revisions: bfail1 bfail2
//@ revisions: bpass1 bpass2
//@ compile-flags: -Z query-dep-graph
//@ aux-build:point.rs
//@ build-pass
Comment on lines -5 to -8
Copy link
Copy Markdown
Member

@jieyouxu jieyouxu Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: I was staring at this for like a good minute trying to understand why this previously worked... This is because the previous revision names are just misleading, right?

Copy link
Copy Markdown
Member Author

@Zalathar Zalathar Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that was one of the big confusions that prompted me to go down the path of trying to clean this stuff up.

For whatever reason, it used to be the case that bfail (historically cfail) was the only revision kind that supported *-pass directives and diagnostic checks. So if someone wanted those checks, they would use a bfail revision and combine it with directives to explicitly expect success instead of failure. Very confusing.

//@ ignore-backends: gcc

#![feature(rustc_attrs)]
#![allow(dead_code)]
#![crate_type = "rlib"]

#![rustc_partition_reused(module="struct_point-fn_calls_methods_in_same_impl", cfg="bfail2")]
#![rustc_partition_reused(module="struct_point-fn_calls_free_fn", cfg="bfail2")]
#![rustc_partition_reused(module="struct_point-fn_read_field", cfg="bfail2")]
#![rustc_partition_reused(module="struct_point-fn_write_field", cfg="bfail2")]
#![rustc_partition_reused(module="struct_point-fn_make_struct", cfg="bfail2")]
#![rustc_partition_reused(module="struct_point-fn_calls_methods_in_same_impl", cfg="bpass2")]
#![rustc_partition_reused(module="struct_point-fn_calls_free_fn", cfg="bpass2")]
#![rustc_partition_reused(module="struct_point-fn_read_field", cfg="bpass2")]
#![rustc_partition_reused(module="struct_point-fn_write_field", cfg="bpass2")]
#![rustc_partition_reused(module="struct_point-fn_make_struct", cfg="bpass2")]

extern crate point;

/// A fn item that calls (public) methods on `Point` from the same impl
pub mod fn_calls_methods_in_same_impl {
use point::Point;

#[rustc_clean(cfg="bfail2")]
#[rustc_clean(cfg="bpass2")]
pub fn check() {
let x = Point { x: 2.0, y: 2.0 };
x.distance_from_origin();
Expand All @@ -35,7 +34,7 @@ pub mod fn_calls_methods_in_same_impl {
pub mod fn_calls_free_fn {
use point::{self, Point};

#[rustc_clean(cfg="bfail2")]
#[rustc_clean(cfg="bpass2")]
pub fn check() {
let x = Point { x: 2.0, y: 2.0 };
point::distance_squared(&x);
Expand All @@ -46,7 +45,7 @@ pub mod fn_calls_free_fn {
pub mod fn_make_struct {
use point::Point;

#[rustc_clean(cfg="bfail2")]
#[rustc_clean(cfg="bpass2")]
pub fn make_origin() -> Point {
Point { x: 2.0, y: 2.0 }
}
Expand All @@ -56,7 +55,7 @@ pub mod fn_make_struct {
pub mod fn_read_field {
use point::Point;

#[rustc_clean(cfg="bfail2")]
#[rustc_clean(cfg="bpass2")]
pub fn get_x(p: Point) -> f32 {
p.x
}
Expand All @@ -66,7 +65,7 @@ pub mod fn_read_field {
pub mod fn_write_field {
use point::Point;

#[rustc_clean(cfg="bfail2")]
#[rustc_clean(cfg="bpass2")]
pub fn inc_x(p: &mut Point) {
p.x += 1.0;
}
Expand Down
45 changes: 22 additions & 23 deletions tests/incremental/change_add_field/struct_point.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,34 +3,33 @@
// Fns with that type used only in their body are also recompiled, but
// their callers are not.

//@ revisions: bfail1 bfail2
//@ revisions: bpass1 bpass2
//@ compile-flags: -Z query-dep-graph
//@ build-pass
//@ ignore-backends: gcc

#![feature(rustc_attrs)]
#![allow(dead_code)]
#![crate_type = "rlib"]

// These are expected to require codegen.
#![rustc_partition_codegened(module="struct_point-point", cfg="bfail2")]
#![rustc_partition_codegened(module="struct_point-fn_with_type_in_sig", cfg="bfail2")]
#![rustc_partition_codegened(module="struct_point-call_fn_with_type_in_sig", cfg="bfail2")]
#![rustc_partition_codegened(module="struct_point-fn_with_type_in_body", cfg="bfail2")]
#![rustc_partition_codegened(module="struct_point-fn_make_struct", cfg="bfail2")]
#![rustc_partition_codegened(module="struct_point-fn_read_field", cfg="bfail2")]
#![rustc_partition_codegened(module="struct_point-fn_write_field", cfg="bfail2")]
#![rustc_partition_codegened(module="struct_point-point", cfg="bpass2")]
#![rustc_partition_codegened(module="struct_point-fn_with_type_in_sig", cfg="bpass2")]
#![rustc_partition_codegened(module="struct_point-call_fn_with_type_in_sig", cfg="bpass2")]
#![rustc_partition_codegened(module="struct_point-fn_with_type_in_body", cfg="bpass2")]
#![rustc_partition_codegened(module="struct_point-fn_make_struct", cfg="bpass2")]
#![rustc_partition_codegened(module="struct_point-fn_read_field", cfg="bpass2")]
#![rustc_partition_codegened(module="struct_point-fn_write_field", cfg="bpass2")]

#![rustc_partition_reused(module="struct_point-call_fn_with_type_in_body", cfg="bfail2")]
#![rustc_partition_reused(module="struct_point-call_fn_with_type_in_body", cfg="bpass2")]

pub mod point {
#[cfg(bfail1)]
#[cfg(bpass1)]
pub struct Point {
pub x: f32,
pub y: f32,
}

#[cfg(bfail2)]
#[cfg(bpass2)]
pub struct Point {
pub x: f32,
pub y: f32,
Expand All @@ -39,18 +38,18 @@ pub mod point {

impl Point {
pub fn origin() -> Point {
#[cfg(bfail1)]
#[cfg(bpass1)]
return Point { x: 0.0, y: 0.0 };

#[cfg(bfail2)]
#[cfg(bpass2)]
return Point { x: 0.0, y: 0.0, z: 0.0 };
}

pub fn total(&self) -> f32 {
#[cfg(bfail1)]
#[cfg(bpass1)]
return self.x + self.y;

#[cfg(bfail2)]
#[cfg(bpass2)]
return self.x + self.y + self.z;
}

Expand All @@ -70,7 +69,7 @@ pub mod point {
pub mod fn_with_type_in_sig {
use point::Point;

#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bfail2")]
#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bpass2")]
pub fn boop(p: Option<&Point>) -> f32 {
p.map(|p| p.total()).unwrap_or(0.0)
}
Expand All @@ -86,7 +85,7 @@ pub mod fn_with_type_in_sig {
pub mod call_fn_with_type_in_sig {
use fn_with_type_in_sig;

#[rustc_clean(except="typeck_root,optimized_mir", cfg="bfail2")]
#[rustc_clean(except="typeck_root,optimized_mir", cfg="bpass2")]
pub fn bip() -> f32 {
fn_with_type_in_sig::boop(None)
}
Expand All @@ -102,7 +101,7 @@ pub mod call_fn_with_type_in_sig {
pub mod fn_with_type_in_body {
use point::Point;

#[rustc_clean(except="typeck_root,optimized_mir", cfg="bfail2")]
#[rustc_clean(except="typeck_root,optimized_mir", cfg="bpass2")]
pub fn boop() -> f32 {
Point::origin().total()
}
Expand All @@ -115,7 +114,7 @@ pub mod fn_with_type_in_body {
pub mod call_fn_with_type_in_body {
use fn_with_type_in_body;

#[rustc_clean(cfg="bfail2")]
#[rustc_clean(cfg="bpass2")]
pub fn bip() -> f32 {
fn_with_type_in_body::boop()
}
Expand All @@ -125,7 +124,7 @@ pub mod call_fn_with_type_in_body {
pub mod fn_make_struct {
use point::Point;

#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bfail2")]
#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bpass2")]
pub fn make_origin(p: Point) -> Point {
Point { ..p }
}
Expand All @@ -135,7 +134,7 @@ pub mod fn_make_struct {
pub mod fn_read_field {
use point::Point;

#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bfail2")]
#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bpass2")]
pub fn get_x(p: Point) -> f32 {
p.x
}
Expand All @@ -145,7 +144,7 @@ pub mod fn_read_field {
pub mod fn_write_field {
use point::Point;

#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bfail2")]
#[rustc_clean(except="typeck_root,fn_sig,optimized_mir", cfg="bpass2")]
pub fn inc_x(p: &mut Point) {
p.x += 1.0;
}
Expand Down
8 changes: 4 additions & 4 deletions tests/incremental/change_crate_dep_kind.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
// detected then -Zincremental-verify-ich will trigger an assertion.

//@ needs-unwind
//@ revisions: bfail1 bfail2
//@ revisions: bpass1 bpass2
//@ compile-flags: -Z query-dep-graph -Cpanic=unwind
//@ needs-unwind
//@ build-pass (FIXME(62277): could be check-pass?)
//@ ignore-backends: gcc
// FIXME(#62277): could be check-pass?

#![cfg_attr(bfail1, feature(panic_unwind))]
#![cfg_attr(bpass1, feature(panic_unwind))]

// Turn the panic_unwind crate from an explicit into an implicit query:
#[cfg(bfail1)]
#[cfg(bpass1)]
extern crate panic_unwind;

fn main() {}
Loading
Loading