gc: track and clean target directories

[tomsanbear]

What does this PR try to resolve?

Part of #13136.

This implements target-directory last-use tracking and cargo clean gc -Zgc cleanup for target/ directories.

The main fix is to make target-dir GC operate on physical target directories instead of individual database rows, which is necessary for shared CARGO_TARGET_DIR setups.

This follows the design in #13136 by storing target-dir associations as (workspace_manifest, target_dir) rows in a single target_directory table.
That supports both shared target dirs across workspaces and multiple target dirs for one workspace.

This PR also:

• records target-dir usage only for compile-family commands
• preserves shared target dirs if any valid recent association remains
• deletes leaked or orphaned target dirs when no valid association protects them
• counts shared target dirs only once for size-based cleanup
• normalizes equivalent target-dir path spellings for grouping
• skips GC deletion for unsafe target dirs with invalid CACHEDIR.TAG
• appends the target_directory migration at the end of the migration list

How to test and review this PR?

Review the changes in:

• src/cargo/ops/cargo_compile/mod.rs
• src/cargo/core/global_cache_tracker.rs
• tests/testsuite/global_cache_tracker.rs

Tests run:

• focused target-dir tracking and GC tests in tests/testsuite/global_cache_tracker.rs
• cargo test --test testsuite -- global_cache_tracker:: --quiet

The broader slice still hits the pre-existing global_cache_tracker::read_only_locking_auto_gc failure, which also reproduces on master.

[rustbot]

r? @weihanglo

rustbot has assigned @weihanglo.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @ehuss, @epage, @weihanglo
• @ehuss, @epage, @weihanglo expanded to ehuss, epage, weihanglo
• Random selection from ehuss, epage, weihanglo

[epage]

The issue for this is #13136. Since that issue, we now have the concept of build-dir in addition to target-dir which may overlap or be independent.

There was a previous attempt at #13846 that would be good to compare notes with.

Our contrib guide asks for a specific organization of commits, see https://doc.crates.io/contrib/process/working-on-cargo.html#submitting-a-pull-request. I've written more on each of those points at https://epage.github.io/dev/pr-style/. We haven't yet decided whether to integrate that into our docs. I'd particularly like to call out

• https://epage.github.io/dev/pr-style/#c-atomic
• https://epage.github.io/dev/pr-style/#c-test

[epage]

Looking up the target and build dirs can involve a bit of work. Might be good to check the Layout.

View changes since the review

[epage]

It would be good to discuss your database schema in the PR description.

[tomsanbear]

Thanks, I dug into the current build_dir behavior a bit more.

From the current workspace logic, build_dir falls back to target_dir unless
build.build-dir is explicitly configured, so the common case is still that
they are the same path. When they do diverge, build_dir is for intermediate
build artifacts and is documented as an implementation detail.

Since this is my first contribution here, I would rather err on the side of
taking reviewer guidance on scope. If you think build_dir should be handled in
this PR as well, I’m happy to extend it and follow up with additional commits.
If not, I can keep the PR scoped to target_dir and make that explicit in the
description.

[epage]

From the current workspace logic, build_dir falls back to target_dir unless
build.build-dir is explicitly configured, so the common case is still that
they are the same path. When they do diverge, build_dir is for intermediate
build artifacts and is documented as an implementation detail.

I'd recommend looking back over this because hat is not correct.

[tomsanbear]

From the current workspace logic, build_dir falls back to target_dir unless
build.build-dir is explicitly configured, so the common case is still that
they are the same path. When they do diverge, build_dir is for intermediate
build artifacts and is documented as an implementation detail.

I'd recommend looking back over this because hat is not correct.

Hmm, okay, I'll take another look. In the meantime, do you have any feeling regarding the inclusion of this within the scope to be covered within this pull request?