Skip to content

test: xunit v3 4.0.1 on Microsoft.Testing.Platform - #35

Merged
elvogel merged 2 commits into
mainfrom
test/xunit-4-mtp
Sep 29, 2026
Merged

elvogel merged 2 commits into
mainfrom
test/xunit-4-mtp

Conversation

@elvogel

@elvogel elvogel commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Closes the test-platform migration deferred on 2026-09-10, and supersedes #32 — which bumped the
versions without the migration and failed the build.

Why the property in the error message cannot fix it

xunit.v3 4.0 drops VSTest on the .NET 10 SDK:

error : Testing with VSTest target is no longer supported by Microsoft.Testing.Platform
on .NET 10 SDK and later. If you use dotnet test, you should opt-in to the new dotnet
test experience.

TestingPlatformDotnetTestSupport=true is the documented pre-.NET-10 bridge: it redirects the
VSTest target into InvokeTestingPlatform. MTP 2.4.0 removed that escape hatch, and the error is
raised from inside the bridge's own target:

<Target Name="_MTPBeforeVSTest" BeforeTargets="VSTest">
  <_SupportsGlobalJsonTestRunner Condition="... '$(_SdkMajorVersion)' >= '10'">true</...>
  <Error Text="Testing with VSTest target is no longer supported..."
         Condition="'$(IsTestingPlatformApplication)'=='true' AND ... '$(_SupportsGlobalJsonTestRunner)'=='true'" />

It is guarded on the SDK major version alone, so the property cannot suppress it — and the
refusal is conditioned on a better opt-in being available. The SDK agrees: dotnet test --help
on 10.0.401 says to opt in "via global.json". (docs/deferred-work.md guessed dotnet.config;
that is a different mechanism and not what this SDK reads. Corrected here.)

What changed

  • global.json gains "test": { "runner": "Microsoft.Testing.Platform" }.
  • Three packages removed rather than bumped, all VSTest-only and inert under MTP:
    Microsoft.NET.Test.Sdk, xunit.runner.visualstudio, coverlet.collector. The first two
    cannot work; the third would be worse than absent, since the collect flag would still be
    accepted and quietly produce nothing. No replacement added — nothing here has ever asked for a
    coverage number, and coverlet.mtp / Microsoft.Testing.Extensions.CodeCoverage are recorded
    in Directory.Packages.props for whoever does.
  • <OutputType>Exe</OutputType> for test projects, which Microsoft.NET.Test.Sdk used to
    supply implicitly. In Directory.Build.props, conditioned on the project name: IsTestProject
    is set by the test SDK targets long after .props is evaluated, so a condition on it would
    silently never match.
  • Documentation: the deferred-work entry closed, a ### The test platform section in
    CONTRIBUTING, a CHANGELOG entry.

What did not change, against expectation

The deferred-work entry predicted a filter rewrite in four places. It was unnecessary —
xunit.v3 4.x ships --filter accepting VSTest syntax, so both CI steps, CONTRIBUTING.md and the
corpus-audit instructions are unchanged and still correct. The one restriction is that a VSTest
filter cannot be combined with xunit's own --filter-class / --filter-query forms, which
nothing here does. No workflow file is touched.

One thing improves for free

A run that executes no tests now exits 8 instead of passing. Measured, not assumed:

$ dotnet test tests/Docmd.Word.Tests --filter "FullyQualifiedName~NoSuchTestNameAtAll"
  total: 0 ... Test run completed with non-success exit code: 8
REAL exit = 8

Under VSTest a filter that matched nothing was a silent green. That is the repository's own
"beware the vacuous pass" rule, now enforced by the platform.

Verification

Clean tree, every step's real exit code captured (not $? after a pipe):

Step Result
dotnet restore docmd.slnx --locked-mode ok
dotnet build docmd.slnx --no-restore -c Release ok, 0 warnings
CI Test step, verbatim 362 total, 361 pass, 1 skip
CI Performance gate step, verbatim 1 test, isolated
CI CLI smoke test, verbatim ok, embedded stylesheet parses
Test assembly run directly ok
Zero-test run exit 8, as intended

Pre-migration baseline was 362 total / 361 pass / 1 skip with the same filter. Full suite is
363 — the same tests plus the one performance test the filter excludes.

Incidental

src/Docmd.Cli/packages.lock.json still recorded Ooxml.Md.Core at 0.2.3 after the 0.2.4
release; regenerating the lock files corrected it. Worth knowing that --locked-mode does not
catch this — verified by restoring the stale value deliberately, which exits 0. It validates
package versions, not the project-reference versions in the same file, so that field goes stale
on every release and nothing complains.

🤖 Generated with Claude Code

https://claude.ai/code/session_018wZEgtuzaZPswykiGEBxbz

Lucas Vogel and others added 2 commits September 29, 2026 15:39
Closes the migration deferred on 2026-09-10, and supersedes Dependabot's PR #32,
which bumped the versions without it and failed the build.

xunit.v3 4.0 drops VSTest on the .NET 10 SDK. The error points at a property that
cannot fix it: TestingPlatformDotnetTestSupport is the pre-.NET-10 bridge that
redirects the VSTest target into InvokeTestingPlatform, and MTP 2.4.0 removed that
escape hatch. The error is raised from inside the bridge's own _MTPBeforeVSTest
target, guarded on the SDK major version and a variable named
_SupportsGlobalJsonTestRunner, so setting the property changes nothing. The SDK
says the same thing: `dotnet test --help` on 10.0.401 directs you to global.json.

So the opt-in is global.json, and three packages come out rather than up:

  Microsoft.NET.Test.Sdk        the VSTest targets and test host
  xunit.runner.visualstudio     the VSTest adapter
  coverlet.collector            a VSTest data collector

The first two cannot work under MTP. The third would be worse than absent: the
collect flag would still be accepted and quietly produce nothing. No replacement
is added, because nothing in this repository has ever asked for a coverage number.

Test projects gain <OutputType>Exe</OutputType>, which Microsoft.NET.Test.Sdk used
to supply implicitly. It is in Directory.Build.props conditioned on the project
name, because IsTestProject is set by the test SDK targets long after .props is
evaluated and a condition on it would silently never match.

The filter rewrite the deferred-work entry predicted turned out to be unnecessary.
xunit.v3 4.x ships --filter accepting VSTest syntax, so both CI steps, CONTRIBUTING
and the corpus-audit instructions are unchanged and still correct.

One thing improves for free: a run that executes no tests now exits 8 instead of
passing. Measured, not assumed -- a filter matching nothing exits 8 while the real
suite exits 0. A filter typo used to be a silent green.

Verified on a clean tree: locked-mode restore, Release build, both CI test steps
run verbatim, the CLI smoke test, and a test assembly executed directly.
363 tests: 362 pass, 1 skip (the corpus audit). That is the pre-migration count of
362 plus the one performance test the old filter excluded.

Also corrects src/Docmd.Cli/packages.lock.json, which still recorded
Ooxml.Md.Core at 0.2.3 after the 0.2.4 release. --locked-mode does not catch this:
it validates package versions, not the project-reference versions in the same file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018wZEgtuzaZPswykiGEBxbz
CONTRIBUTING draws a line between a fact the code owns and a dated
measurement. A test count is the first kind: a routine commit makes the
sentence false and nobody notices. Recording it as a measurement with its
date and its commit puts it on the right side of that line, which is the
same treatment 0.2.4's timing figures get.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018wZEgtuzaZPswykiGEBxbz
@elvogel
elvogel merged commit 6c6abc4 into main Sep 29, 2026
1 check passed
@elvogel
elvogel deleted the test/xunit-4-mtp branch September 29, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant