Add headless remote engine test suite + CI - #4
Merged
Conversation
- Add remote_engine_test/: exercises RemoteGameEngine + ScoreboardState directly against a dockerized CRG scoreboard server over WebSocket, no Flutter widget tree involved. Mirrors the game-flow logic in integration_test/scoreboard_integration_test.dart but skips app UI/build, which is what makes the widget-driven tests slow. - Add scripts/test-remote-engine-scoreboards.sh to run it across all supported scoreboard versions in Docker, without an emulator. - Root-cause and fix a debounce bug: the CRG server's GameImpl.quickClockControl() silently drops or undoes Start/Stop/ Timeout actions sent within 1s of each other. Add a clockAction() cooldown helper and apply the same fix to the two call sites in scoreboard_integration_test.dart that bypass tapJamControl's existing cooldown (swipeToStartLineup, the overtime confirmation swipe). - Consolidate both test suites to create far fewer games/app instances: remote_engine_test drops from 11 tests to 6, scoreboard_integration_test from 11 to 9 (5 non-full-game tests down to 2). - Mark 'Full game start/stop - RDCL' in remote_engine_test as skipped: official score confirmation reliably hangs for RDCL specifically, with EnforceTimeToOr disabled and the tie broken, while the same flow works for WFTDA. Root cause not yet found. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The CRG server enforces a real ~30s delay (Rule.LINEUP_DURATION) before OfficialScore can be set, regardless of the EnforceTimeToOr rule we disable via WS. finishGame() now waits 32s for it to clear. Only the WFTDA "Full game start/stop" test pays that wait - official score confirmation isn't ruleset-specific, so "Full game start/stop - RDCL" now just confirms it reaches the post-game state instead of paying the 32s tax a second time. Un-skip the RDCL test and adjust timeouts accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
scripts/test-remote-engine-scoreboards.sh now writes each run's results as JSON via flutter test's built-in --file-reporter, alongside the existing console output. build-scoreboard-images.sh gained a matching --versions filter so CI can build just what's needed. Add a manual-only (workflow_dispatch) GitHub Actions workflow that builds the requested scoreboard image(s) (latest by default), runs the headless suite, and publishes the JSON via dorny/test-reporter as a proper GitHub Check - no custom parsing involved. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
workflow_dispatch can't target a workflow that only exists on a non-default branch, so add a push trigger scoped to this branch to exercise it before merging. Remove before merging to main. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Use latest github action versions
Since public-repo Actions minutes aren't metered, run the full version matrix (11 jobs) instead of defaulting to latest-only, each publishing its own GitHub Check. Drop the workflow_dispatch versions input since there's no longer a single run to parameterize. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace the temporary push-to-branch trigger with the real intended one now that the workflow exists to be targeted. Not yet marked as a required check in branch protection. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
remote_engine_test/: exercisesRemoteGameEngine+ScoreboardStatedirectly against a dockerized CRG scoreboard server over WebSocket, no Flutter widget tree involved. Mirrors the game-flow logic inintegration_test/scoreboard_integration_test.dartbut skips app UI/build, which is what makes the widget-driven tests slow.GameImpl.quickClockControl()silently drops or undoes Start/Stop/Timeout actions sent within 1s of each other) and a real ~30s server-enforced delay before official score can be confirmed.scripts/test-remote-engine-scoreboards.sh(and--versionssupport inscripts/build-scoreboard-images.sh) to run the new suite across scoreboard versions in Docker, without an emulator, producing machine-readable JSON reports viaflutter test --file-reporter..github/workflows/remote-engine-tests.yml: runs the new suite as a matrix across all supported scoreboard versions on every PR tomain, publishing per-version results as GitHub Checks viadorny/test-reporter.Test plan
remote_engine_test/passes locally against all matrix versions exceptv2023.7(currently excluded pending investigation into a possibly-legitimate failure)integration_test/scoreboard_integration_test.dartpasses against an Android emulator +v2025.9