Testdir collision for doctests

[wiktor-k]

Hi,

I believe this may be out of scope for testdir but I think it still needs to be documented as it surprised me.

testdir::testdir!() generates the same directories when executed in a doctest.

Minimal reproducer (src/lib.rs):

/// Does one thing well.
///
/// ```
/// let dir = testdir::testdir!();
/// eprintln!("One: {dir:?}");
/// assert!(false);
/// ```
pub fn one() {}

/// Does another thing well.
///
/// ```
/// let dir = testdir::testdir!();
/// eprintln!("Two: {dir:?}");
/// assert!(false);
/// ```
pub fn two() {}

Running it with cargo test shows this output:

tderr:
Two: "/home/wiktor/tmp/cargo/testdir-198/rust_out/h9f5ab6561c35996e"
thread 'main' panicked at src/lib.rs:5:1:
assertion failed: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


---- src/lib.rs - one (line 3) stdout ----
Test executable failed (exit status: 101).

stderr:
One: "/home/wiktor/tmp/cargo/testdir-198/rust_out/h9f5ab6561c35996e"
thread 'main' panicked at src/lib.rs:5:1:
assertion failed: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Both of them generated the same path: "/home/wiktor/tmp/cargo/testdir-198/rust_out/h9f5ab6561c35996e" and some test may want to use isolated dirs.

This is similar to #6 but instead of rstest it's doctests.

[flub]

I think this is fair to consider in-scope, thanks for reporting.

I think the entire directory structure created here is not intuitive. Ideally this would indicate it is a doc test and still have the module structure somehow. Not sure if this is possible though.

[flub]

I've been looking into this a bit and doctest does not seem to give us much.

On edition 2021 the best I can find is to inspect the command line arguments, which is the absolute pathname of the test binary invoked. On linux it looks something like:

"/tmp/rustdoctest5Ud0Cn/rust_out

So here you could extract rustdoctest and also 5Ud0Cn which is unique per test. But does not identify the test at all.

On edition 2024 the same can be done. But also the existing mechanism already works better because the test "id" is encoded in the backtrace. The example you give ends up creating the following directory:

target/testdir-110/doctest_bundle_2024/__doctest_2/h3cd332b28e3061cb"

Here __doctest_2 is the "id" of the tests and each test already gets its own directory thanks to this.

Unfortunately I can't get any better name out of this. The test harness does not seem to allow access to that.

The best thing I can come up with is to use the macro with a string argument: testdir!("name_of_the_test"); and then you'll get a nice findable test.

But given the default behaviour already manages to generate a unique directory for each doctest I'm tempted to perhaps leave it at that.

[flub]

Forgot to add the helpful reference of how doctests work: https://blog.guillaume-gomez.fr/articles/2024-08-17+Doctests+-+How+were+they+improved%3F