Skip to content

Migrate to xunit v3 and pin dependencies with lock files - #3

Merged
AlyxSharkBite merged 2 commits into
mainfrom
chore/xunit-v3-and-lockfiles
Aug 28, 2026
Merged

AlyxSharkBite merged 2 commits into
mainfrom
chore/xunit-v3-and-lockfiles

Conversation

@AlyxSharkBite

Copy link
Copy Markdown
Owner

Fixes the two remaining actionable items. Two independent concerns, kept as two separate commits — but in one PR, because the lock file content depends on the exact package set, so splitting them would guarantee a conflict.


1. xunit v2 → v3 (60dba84)

xunit v2 is marked Legacy in NuGet's advisory data, which the SCA workflow reported as a deprecation warning on every run.

No test source changes were needed. Fact, Theory, InlineData, MemberData, TheoryData, Assert and Record.Exception are all unchanged in v3. All 966 tests pass.

What did change is how tests are hosted. xunit v3 builds on Microsoft.Testing.Platform, so the test project is now an executable carrying its own runner, and the .NET 10 SDK no longer runs such projects through VSTest at all:

  • global.json selects the MTP runner — the documented opt-in for the SDK's MTP mode of dotnet test
  • Microsoft.NET.Test.Sdk and xunit.runner.visualstudio are gone; the platform runner replaces both
  • coverlet.collector dropped with them — a VSTest data collector, and coverage was never collected in CI. Fewer packages is also less to audit.
  • CI drops --nologo from dotnet test

That last one is worth calling out: MTP mode rejects --nologo with exit code 5, which surfaces as "Zero tests ran" rather than an obvious argument error. Easy to misread as a discovery failure.

Also set TESTINGPLATFORM_TELEMETRY_OPTOUT, since the platform ships its own telemetry package, and repointed Dependabot's test-dependencies group at Microsoft.Testing.* now that the old packages are gone.

Result: dotnet list package --deprecated --include-transitive is clean for both projects.


2. NuGet lock files (21fac37)

Closes the Scorecard Pinned-Dependencies finding (7/10, nugetCommand not pinned by hash). Restore previously resolved versions at build time, so a re-published or substituted package could enter a build unnoticed.

Directory.Build.props enables RestorePackagesWithLockFile repo-wide, and the generated packages.lock.json files are committed:

Project Locked packages
PkLibSharp.Tests 18, all with content hashes
PkLibSharp 0 — the library genuinely has no dependencies

CI restores with --locked-mode in both the build and the SCA workflow.

I verified the gate actually fires rather than assuming it: changing a package version without regenerating the lock fails restore with NU1004 instead of quietly resolving the new version.

One honest limitation found while testing — editing resolved inside the lock file alone was not rejected, because that package was already in the local NuGet cache. Content hashes are verified when packages are downloaded, which is the CI case (clean runner, cold cache); a warm local cache can serve a package without re-checking it. So this is a strong CI gate and a weaker local one.

Finally, NuGetAuditMode=all makes restore audit transitive packages, not just direct references. The default covers only direct ones, which is not where most dependency risk lives.


Verification

  • dotnet restore --locked-mode → clean
  • dotnet build -c Release -warnaserror → 0 warnings, 0 errors
  • dotnet test -c Release --no-build → 966 passed, 0 failed
  • No vulnerable or deprecated packages in either project

AlyxSharkBite and others added 2 commits August 28, 2026 12:10
xunit v2 is marked Legacy in the NuGet advisory data, which the SCA
workflow reported as a deprecation warning on every run. Move to
xunit.v3 4.0.0.

No test source changes were needed: Fact, Theory, InlineData,
MemberData, TheoryData, Assert and Record.Exception are all unchanged
in v3. All 966 tests pass.

The move does change how tests are hosted. xunit v3 builds on
Microsoft.Testing.Platform, so the test project is now an executable
that carries its own runner, and the .NET 10 SDK no longer runs such
projects through VSTest at all. That means:

  - global.json selects the Microsoft.Testing.Platform runner, which is
    the documented opt-in for the SDK's MTP mode of dotnet test
  - Microsoft.NET.Test.Sdk and xunit.runner.visualstudio are gone; the
    platform runner replaces both
  - coverlet.collector is dropped with them, since it is a VSTest data
    collector and coverage was never collected in CI. Fewer packages is
    also less to audit.
  - CI drops --nologo from dotnet test. MTP mode rejects that
    VSTest-era option with exit code 5, which fails as "zero tests ran"
    rather than as an obvious argument error.

TESTINGPLATFORM_TELEMETRY_OPTOUT is set alongside the existing .NET
telemetry opt-out, since the platform ships its own telemetry package.

Dependabot's test-dependencies group now matches Microsoft.Testing.*
instead of the packages that were removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the OpenSSF Scorecard Pinned-Dependencies finding, which scored
7/10 on "nugetCommand not pinned by hash". Restore previously resolved
package versions at build time, so a re-published or substituted
package could enter a build without anything noticing.

Directory.Build.props turns on RestorePackagesWithLockFile for every
project, and the generated packages.lock.json files are committed. They
record the exact resolved version and content hash of all 18 transitive
dependencies of the test project. The library itself has no
dependencies at all, so its lock file is empty, which is the point.

CI now restores with --locked-mode in both the build and the SCA
workflow. Verified that the gate actually fires: changing a package
version without regenerating the lock fails restore with NU1004 rather
than quietly resolving the new version. Content hashes are verified
when packages are downloaded, which is the CI case; a warm local cache
can serve a package without re-checking it.

NuGetAuditMode is set to "all" so restore audits transitive packages
rather than only direct references, at the lowest severity threshold.
Most dependency risk sits in the transitive set, which is exactly what
the default mode does not cover.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@AlyxSharkBite
AlyxSharkBite merged commit d926826 into main Aug 28, 2026
8 checks passed
@AlyxSharkBite
AlyxSharkBite deleted the chore/xunit-v3-and-lockfiles branch August 28, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant