Skip to content

Fix - run CI tests on windows-latest as well as ubuntu - #265

Merged
demortes merged 6 commits into
mainfrom
ci/windows-test-leg
Nov 11, 2026
Merged

demortes merged 6 commits into
mainfrom
ci/windows-test-leg

Conversation

@demortes

@demortes demortes commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

What & why

Before: CI built and tested only on ubuntu-latest. The Windows release job builds the installer but runs no tests, so Windows-only code paths (PluginPathRules, PluginContentHash, SettingsStore, SapiVoice, the Windows branches in the plugin storage tests) never ran in CI, even though Windows is the primary platform.

After: the build-test job runs on a matrix of ubuntu-latest and windows-latest, so every PR and push to main builds and tests on both.

fail-fast: false keeps both legs running so a failure on one OS doesn't cancel the other's result.

Changes

  • .github/workflows/ci.yml: add strategy.matrix.os: [ubuntu-latest, windows-latest] and fail-fast: false; runs-on: ${{ matrix.os }}. Steps unchanged.
  • PluginContentBindingTests.FolderHash_IsDeterministic_AndIndependentOfCreationOrder: the first Windows run failed it (1 of ~2,830 tests). The test wrote a.txt and A.txt into one folder, which is a single file on NTFS, so the two trees got different content. Swapped A.txt for C.txt. PluginContentHash itself is unchanged; this was a test-only assumption of a case-sensitive file system.
  • PluginServiceLifecycleTests.ASecondDetach_Joins_AndReturnsOnceTheFirstHasFinished: passed on the first Windows run, failed on the second after exactly 5 s ("no exception was thrown"). Both detaches block pool threads; with the pool starved, the WaitAsync(200 ms) timeout callback couldn't run until the host's 5 s shutdown bound let the first detach finish, so the "not done yet" check saw a completed task. Now both detaches run on dedicated (LongRunning) threads and the probe is SpinWait.SpinUntil, which needs no pool thread. PluginService itself is unchanged.
  • PluginServiceReattachTests.Attach_WhileAnOlderTeardownIsStillRunning_..._TornDownOnExit: failed on run 3 at Assert.True(fresh.Returned.IsSet). ShutdownForExit(timeout) bounds the whole teardown, including the current runtime's, so with a 150 ms bound a slow runner returns before the fresh plugin's Shutdown has run. Exit staying strictly bounded is intended (decided on this PR), so the test now waits up to the 30 s ceiling for Shutdown instead of assuming it already ran.
  • PluginServiceExitTests.ShutdownForExit_WhenAHandlerNeverReturns_...: failed on run 4 at Assert.Null(service.Runtime); the bounded call returned after 150 ms before its worker had cleared the runtime. It and the two other tests that assert Runtime is null straight after a 150 ms bounded Detach (Detach_WithABound_WhenAHandlerNeverReturns_..., the reattach stuck-teardown helper) now wait for it with SpinWait.SpinUntil(..., Ceiling).
  • TwitchStreamCardServiceTests.A_clear_for_a_closed_game_that_fails_is_queued_and_dropped_once_acknowledged: failed on run 5 (RetryPendingAsync returned 1). The service's pump keeps retrying the failed clear and re-enqueues the entry before each send, so a concurrent pump retry could be mid-flight when the test counted. The test is about the queue's retry alone, so it now disposes the service first. EbsCleanupQueue and the service are unchanged.

How it was verified

  • YAML parses, and the job resolves to the expected matrix (checked locally with PyYAML).
  • Run 1: ubuntu green, Windows failed only the hash test. Run 2: hash test fixed, Windows failed only the detach test (an intermittent race; it passed in run 1). Run 3: both fixed, Windows failed only the reattach exit test above. Run 4: those fixed, Windows failed only the exit-handler test above. Run 5: those fixed, Windows failed only the Twitch closed-game clear test. Run 6: ubuntu and windows both green.
  • Couldn't run the suite locally (no .NET SDK reachable from the session), so CI is the verification.

Heads-up

  • Check names change from build-test to build-test (ubuntu-latest) and build-test (windows-latest). If branch protection requires build-test by name, update the required checks before merging.
  • Windows runners bill at 2x minutes on private repos; this repo is public, so no cost change.

Checklist

  • Builds clean, no new warnings (no code changes)
  • No secrets, tokens, or personal journal files committed

Generated by Claude Code

Windows is the primary platform, but CI only tested on ubuntu, so the
Windows branches of PluginPathRules, PluginContentHash, SettingsStore and
SapiVoice never ran in CI. Add a windows-latest matrix leg with
fail-fast off so each OS reports independently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor
Pre-merge checks: Passed 1 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Passed Passed checks: 1 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

On a case-insensitive file system (the new windows-latest CI leg) the two
names are one file, so the two trees ended up with different content
(a.txt=5 vs A.txt=1) and FolderHash_IsDeterministic_AndIndependentOfCreationOrder
failed. Use C.txt, which still sorts ahead of the lower-case names
ordinally but never collides. PluginContentHash itself is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ASecondDetach_Joins_AndReturnsOnceTheFirstHasFinished failed on the new
windows-latest leg after exactly 5 s with "no exception was thrown".
Both detaches block pool threads, and with the pool starved the
WaitAsync(200 ms) timeout callback could not run until the host's 5 s
shutdown bound let the first detach finish; by then the second had
completed, so the 'not done yet' assertion saw a finished task.

Run both detaches on dedicated threads (LongRunning) and probe with
SpinWait.SpinUntil, which needs no pool thread. Product code unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

demortes commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Run 3 on windows-latest: the two earlier fixes hold (hash test and ASecondDetach_Joins... both pass). One new failure, which needs a call before I push anything:

PluginServiceReattachTests.Attach_WhileAnOlderTeardownIsStillRunning_DoesNotWaitForIt_AndTheNewRuntimeIsStillTornDownOnExit fails at line 157 (Assert.True(fresh.Returned.IsSet)). It passed on runs 1 and 2.

Cause: ShutdownForExit(timeout) runs the whole of DetachCore on a worker and returns after work.Wait(timeout). That bounds the current runtime's teardown too, not only the wait on older teardowns. The test passes ShortBound (150 ms); on the slower Windows runner unloading the fresh plugin takes longer than that, so ShutdownForExit returns before the fresh plugin's Shutdown has run.

Not fixing it yet because the test name and DetachCore's doc say the current runtime is always torn down on exit, while ShutdownForExit's doc says it gives up after the timeout. Two ways out:

  1. Test-only: after ShutdownForExit, wait (bounded by Ceiling) for fresh.Returned instead of asserting it's already set. Product unchanged: exit stays strictly bounded.
  2. Product: make ShutdownForExit tear the current runtime down synchronously and bound only the join on older teardowns. Exit could then take up to the host's shutdown/unload bounds when a plugin hangs.

PluginServiceExitTests uses the same 150 ms ShutdownForExit pattern (lines 109, 147), so it may flake on Windows the same way.


Generated by Claude Code

…uming it

Attach_WhileAnOlderTeardownIsStillRunning_DoesNotWaitForIt_AndTheNewRuntimeIsStillTornDownOnExit
failed on windows-latest at Assert.True(fresh.Returned.IsSet).
ShutdownForExit(timeout) bounds the whole teardown, including the current
runtime's, so with the 150 ms bound a slow runner can return before the
fresh plugin's Shutdown has run. Exit staying strictly bounded is intended;
the test now waits (up to the 30 s ceiling) for Shutdown to run. App code
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…assuming it

ShutdownForExit_WhenAHandlerNeverReturns_GivesUpAtTheBound_AndReturnsFalse
failed on windows-latest at Assert.Null(service.Runtime): the bounded
call ran its teardown on a worker and returned after 150 ms before that
worker had even cleared the runtime. Same treatment as the reattach exit
test: wait (up to the ceiling) for the runtime to be cleared. Applied to
the two other tests that assert Runtime is null straight after a 150 ms
bounded Detach. App code unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

demortes commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Run 5 on windows-latest: all four earlier fixes hold. One new failure, outside the plugin tests:

TwitchStreamCardServiceTests.A_clear_for_a_closed_game_that_fails_is_queued_and_dropped_once_acknowledged fails at line 776: RetryPendingAsync() returns 1 pending instead of 0 after the fake EBS is switched to ClearedOk. It passed on runs 1–4.

Not pushing a fix yet. The likely cause is the queue's own background retry loop (EbsCleanupQueue, around line 310–326) or the service's direct clear running concurrently with the test's RetryPendingAsync, so the entry is re-recorded or not removed. That could be a test-timing issue or a real race in the queue, and I can't reproduce locally (no .NET SDK reachable from this session). Asking the owner how far to take this PR, since each Windows run has surfaced one more pre-existing timing-sensitive test (this file alone has 16 Task.Delay/Thread.Sleep waits).


Generated by Claude Code

@demortes
demortes marked this pull request as ready for review November 11, 2026 03:45
…s retry

A_clear_for_a_closed_game_that_fails_is_queued_and_dropped_once_acknowledged
failed on windows-latest: RetryPendingAsync returned 1 instead of 0.
After the first clear fails, the service's pump retries it every
2x the interval (up to MaxPublishRetries). Each retry enqueues the entry
again before sending, so when the test's manual RetryPendingAsync removes
it, a concurrent pump retry can re-add it and the count catches it while
that clear is still in flight. On a slower runner the pump is still
retrying when the test gets there.

The test is about the queue's own retry (as after an app restart), so
dispose the service first. Product code unchanged: the transient entry
during a direct clear is by design and settles on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@demortes
demortes merged commit dd2d5d5 into main Nov 11, 2026
3 checks passed
@demortes
demortes deleted the ci/windows-test-leg branch November 11, 2026 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants