Migrate to xunit v3 and pin dependencies with lock files - #3
Merged
Merged
Conversation
xunit v2 is marked Legacy in the NuGet advisory data, which the SCA
workflow reported as a deprecation warning on every run. Move to
xunit.v3 4.0.0.
No test source changes were needed: Fact, Theory, InlineData,
MemberData, TheoryData, Assert and Record.Exception are all unchanged
in v3. All 966 tests pass.
The move does change how tests are hosted. xunit v3 builds on
Microsoft.Testing.Platform, so the test project is now an executable
that carries its own runner, and the .NET 10 SDK no longer runs such
projects through VSTest at all. That means:
- global.json selects the Microsoft.Testing.Platform runner, which is
the documented opt-in for the SDK's MTP mode of dotnet test
- Microsoft.NET.Test.Sdk and xunit.runner.visualstudio are gone; the
platform runner replaces both
- coverlet.collector is dropped with them, since it is a VSTest data
collector and coverage was never collected in CI. Fewer packages is
also less to audit.
- CI drops --nologo from dotnet test. MTP mode rejects that
VSTest-era option with exit code 5, which fails as "zero tests ran"
rather than as an obvious argument error.
TESTINGPLATFORM_TELEMETRY_OPTOUT is set alongside the existing .NET
telemetry opt-out, since the platform ships its own telemetry package.
Dependabot's test-dependencies group now matches Microsoft.Testing.*
instead of the packages that were removed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the OpenSSF Scorecard Pinned-Dependencies finding, which scored 7/10 on "nugetCommand not pinned by hash". Restore previously resolved package versions at build time, so a re-published or substituted package could enter a build without anything noticing. Directory.Build.props turns on RestorePackagesWithLockFile for every project, and the generated packages.lock.json files are committed. They record the exact resolved version and content hash of all 18 transitive dependencies of the test project. The library itself has no dependencies at all, so its lock file is empty, which is the point. CI now restores with --locked-mode in both the build and the SCA workflow. Verified that the gate actually fires: changing a package version without regenerating the lock fails restore with NU1004 rather than quietly resolving the new version. Content hashes are verified when packages are downloaded, which is the CI case; a warm local cache can serve a package without re-checking it. NuGetAuditMode is set to "all" so restore audits transitive packages rather than only direct references, at the lowest severity threshold. Most dependency risk sits in the transitive set, which is exactly what the default mode does not cover. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the two remaining actionable items. Two independent concerns, kept as two separate commits — but in one PR, because the lock file content depends on the exact package set, so splitting them would guarantee a conflict.
1. xunit v2 → v3 (
60dba84)xunit v2 is marked Legacy in NuGet's advisory data, which the SCA workflow reported as a deprecation warning on every run.
No test source changes were needed.
Fact,Theory,InlineData,MemberData,TheoryData,AssertandRecord.Exceptionare all unchanged in v3. All 966 tests pass.What did change is how tests are hosted. xunit v3 builds on Microsoft.Testing.Platform, so the test project is now an executable carrying its own runner, and the .NET 10 SDK no longer runs such projects through VSTest at all:
global.jsonselects the MTP runner — the documented opt-in for the SDK's MTP mode ofdotnet testMicrosoft.NET.Test.Sdkandxunit.runner.visualstudioare gone; the platform runner replaces bothcoverlet.collectordropped with them — a VSTest data collector, and coverage was never collected in CI. Fewer packages is also less to audit.--nologofromdotnet testThat last one is worth calling out: MTP mode rejects
--nologowith exit code 5, which surfaces as "Zero tests ran" rather than an obvious argument error. Easy to misread as a discovery failure.Also set
TESTINGPLATFORM_TELEMETRY_OPTOUT, since the platform ships its own telemetry package, and repointed Dependabot'stest-dependenciesgroup atMicrosoft.Testing.*now that the old packages are gone.Result:
dotnet list package --deprecated --include-transitiveis clean for both projects.2. NuGet lock files (
21fac37)Closes the Scorecard Pinned-Dependencies finding (7/10,
nugetCommand not pinned by hash). Restore previously resolved versions at build time, so a re-published or substituted package could enter a build unnoticed.Directory.Build.propsenablesRestorePackagesWithLockFilerepo-wide, and the generatedpackages.lock.jsonfiles are committed:PkLibSharp.TestsPkLibSharpCI restores with
--locked-modein both the build and the SCA workflow.I verified the gate actually fires rather than assuming it: changing a package version without regenerating the lock fails restore with
NU1004instead of quietly resolving the new version.One honest limitation found while testing — editing
resolvedinside the lock file alone was not rejected, because that package was already in the local NuGet cache. Content hashes are verified when packages are downloaded, which is the CI case (clean runner, cold cache); a warm local cache can serve a package without re-checking it. So this is a strong CI gate and a weaker local one.Finally,
NuGetAuditMode=allmakes restore audit transitive packages, not just direct references. The default covers only direct ones, which is not where most dependency risk lives.Verification
dotnet restore --locked-mode→ cleandotnet build -c Release -warnaserror→ 0 warnings, 0 errorsdotnet test -c Release --no-build→ 966 passed, 0 failed