From 77d0edaa2b70aa4756f079eaa87186f7b58c9829 Mon Sep 17 00:00:00 2001 From: Muhamed Isabegovic Date: Thu, 8 Oct 2026 09:43:37 +0200 Subject: [PATCH] [CI] Run the system tests on every pull request - bin/rails test skips test/system, so the 68 browser tests ran only by hand - New step: bin/rails test:system in headless Chrome, screenshots kept on failure - SimpleCov starts on COVERAGE alone: under CI=true the browser tests by themselves fell below the unit suite's threshold and failed the step - Brain: the CI section of the architecture overview --- .github/workflows/ci.yml | 11 +++++++++++ test/test_helper.rb | 6 ++++-- wiki/architecture/overview.md | 18 ++++++++++++++---- 3 files changed, 29 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 12d4033d..561a734a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -69,6 +69,17 @@ jobs: env: COVERAGE: true + - name: Run system tests + run: bin/rails test:system + + - name: Keep screenshots from failed system tests + uses: actions/upload-artifact@v4 + if: failure() + with: + name: screenshots + path: ${{ github.workspace }}/tmp/screenshots + if-no-files-found: ignore + - name: Run Undercover (coverage check for changed files) if: github.event_name == 'pull_request' continue-on-error: true # TODO: Remove after fixing pre-existing coverage gaps diff --git a/test/test_helper.rb b/test/test_helper.rb index 2ddcd088..4f5b5f5a 100644 --- a/test/test_helper.rb +++ b/test/test_helper.rb @@ -1,5 +1,7 @@ -# SimpleCov must be loaded BEFORE application code -if ENV["COVERAGE"] || ENV["CI"] +# SimpleCov must be loaded BEFORE application code. +# Asked for by COVERAGE alone: CI sets CI for every step, and the system tests +# on their own cover far less than the threshold the unit suite is held to. +if ENV["COVERAGE"] require "simplecov" require "simplecov-lcov" diff --git a/wiki/architecture/overview.md b/wiki/architecture/overview.md index d5288b3a..c35b04f3 100644 --- a/wiki/architecture/overview.md +++ b/wiki/architecture/overview.md @@ -2,7 +2,7 @@ title: Architecture overview kind: reference status: living -updated: 2026-09-30 +updated: 2026-10-08 repos: - usput.ba confidence: medium @@ -24,6 +24,8 @@ sources: - config/initializers/rack_attack.rb - config/ci.rb - .github/workflows/ci.yml +- test/test_helper.rb +- test/application_system_test_case.rb - README.md - CLAUDE.md - sources/planning/DEVELOPER_ONBOARDING.md @@ -152,14 +154,22 @@ schema, then runs in order: 1. `bundle exec rubocop --parallel` 2. `bundle exec erb_lint --lint-all` 3. `bin/rails test` with `COVERAGE=true` (SimpleCov with an LCOV formatter) -4. `bundle exec undercover --compare origin/` on pull requests only, +4. `bin/rails test:system`, the 68 browser tests in `test/system/`, in + headless Chrome through Selenium (`test/application_system_test_case.rb`, + using the Chrome the runner image ships). `bin/rails test` skips + `test/system/`, so before this step they ran only by hand. When the step + fails, the screenshots in `tmp/screenshots` are kept as an artifact. + SimpleCov starts only when `COVERAGE` is set (`test/test_helper.rb`), so + this step is not held to the coverage threshold of the unit suite, which + the browser tests alone fall far below. +5. `bundle exec undercover --compare origin/` on pull requests only, with `continue-on-error: true` and a note to remove that once older coverage gaps are fixed. `bin/ci` with `config/ci.rb` is a separate local pipeline: setup, rubocop, bundler-audit, importmap audit, Brakeman, unit tests, system tests and a -seed replant. Brakeman, bundler-audit and system tests run only there, not -in GitHub Actions. +seed replant. Brakeman and bundler-audit run only there, not in GitHub +Actions. ## The main parts and how they depend on each other