From 6b657bd36ef66184d01ee347811ff1f4efa16d35 Mon Sep 17 00:00:00 2001 From: Chris Laupama <42254688+chrislaupama@users.noreply.github.com> Date: Wed, 11 Nov 2026 19:02:03 +0000 Subject: [PATCH 1/4] CI: run tests and the app smoke tests side by side, behind the same required checks Each system now has a tests job (engine CMake build and ctest, cargo test) and an app job (release build, screenshot, self-test) running in parallel; build-test (macOS/Windows/Linux) becomes a gate that passes only when lint and every tests and app job passed, so the required check names stay. - Linux runs its two self-tests (--no-audio and --self-test-audio) side by side on separate X servers; they share no files. - Windows smoke tests are no longer continue-on-error. The WARP self-test timed out on four of the last five main runs and still showed green; it now fails the job, with a 60 minute limit, and also runs nightly. - Pull requests run only what their changes reach (tools/ci/changes.sh, tested by tools/ci/test-changes.sh); anything unknown runs everything, main and the nightly run always do. - tools/ci/prepare-target.sh touches every tracked source after the cache restore, so a cache saved between checkout and restore can never be reused for changed code, and documents why the engine build script output is still cleaned (restored CMake caches pin compiler paths). - New lint job: actionlint with shellcheck, the change classifier tests and the agent skill check. Fixes the two shellcheck warnings in release.yml. - Every action pinned to a commit; dtolnay/rust-toolchain replaced by rustup, which every hosted image already has. Job timeouts everywhere. - Debug and release targets are cached separately (cargo-v3 keys, falling back to the old combined cargo-v2 cache). --- .github/workflows/ci.yml | 552 ++++++++++++++++++++++------------ .github/workflows/perf.yml | 24 +- .github/workflows/release.yml | 54 ++-- CONTRIBUTING.md | 8 +- docs/building.md | 58 +++- engine/README.md | 8 +- tools/ci/changes.sh | 57 ++++ tools/ci/prepare-target.sh | 76 +++++ tools/ci/self-test.sh | 35 +++ tools/ci/test-changes.sh | 44 +++ 10 files changed, 670 insertions(+), 246 deletions(-) create mode 100755 tools/ci/changes.sh create mode 100755 tools/ci/prepare-target.sh create mode 100755 tools/ci/self-test.sh create mode 100755 tools/ci/test-changes.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 62b6861e..5888c4b4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -3,6 +3,11 @@ on: push: branches: [main] pull_request: + # Nightly on main: catches what changes without a commit (a new stable Rust, a runner image + # update) and runs the slow Windows self-test even on days without a merge. + schedule: + - cron: '23 4 * * *' + workflow_dispatch: concurrency: group: ci-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} @@ -10,22 +15,84 @@ concurrency: # Every job runs on a GitHub-hosted runner (free for public repositories): # macos-latest (Apple silicon), windows-latest and ubuntu-latest. There is no -# audio endpoint and no GPU on them, so the engine's device tests skip and the -# app smoke tests use a software renderer. Platform-specific steps are gated on -# runner.os. +# audio endpoint and no GPU on them, so the engine's device tests skip (except +# on Linux, which runs a headless PipeWire server) and the app smoke tests use +# a software renderer. # # The repository is public, so pull requests from forks run this workflow too. # It never uses a self-hosted runner (that would run a stranger's code on our -# machine), uses no secrets, and its token can only read the repository. +# machine), uses no secrets, its token can only read the repository, and pull +# requests only restore caches (only pushes to main and the nightly run save +# them), so a pull request cannot plant a cache that main would use. Actions +# are pinned to full commit hashes; Dependabot keeps them current. # -# Every action used here runs on Node 24 (checkout@v5, cache@v5) or is a -# composite/shell step (rust-toolchain, the MSVC environment step below). +# Layout. Each system has two jobs that run side by side: +# tests () the engine's standalone CMake build (warnings as errors) +# and ctest, then `cargo test` over the workspace (debug); +# app () the app's release build and its smoke tests (a +# screenshot and the scripted self-test). +# plus `lint` (workflow and script checks, seconds). The required checks are +# `build-test (macOS)`, `build-test (Windows)` and `build-test (Linux)`: each +# passes only when lint and every tests and app job passed, so one failure +# turns all three red (the failing job is named in the check list). +# +# Pull requests run only what their changes can reach (tools/ci/changes.sh): +# a documentation-only change builds nothing, an app-only change skips the +# standalone engine build, an engine-test-only change skips the app. Jobs that +# have nothing to do still run and pass, so the required checks always report. +# Pushes to main and the nightly run do everything. permissions: contents: read +env: + CARGO_TERM_COLOR: always + # CI builds start from a restored target dir, never edit-compile loops, so + # incremental data is dead weight (bigger cache, slower builds). Line tables + # keep file:line in test backtraces. + CARGO_INCREMENTAL: "0" + CARGO_PROFILE_DEV_DEBUG: line-tables-only + # Every C/C++ compile (the standalone engine build and the one the engine's + # build script runs) goes through ccache, so an unchanged engine file is + # compiled once and then served from the cache. ccache hashes the compiler + # (path, size, mtime), its flags and the preprocessed source, so a changed + # header, flag or compiler misses. + CMAKE_C_COMPILER_LAUNCHER: ccache + CMAKE_CXX_COMPILER_LAUNCHER: ccache + CCACHE_DIR: ${{ github.workspace }}/.ccache + CCACHE_BASEDIR: ${{ github.workspace }} + CCACHE_NOHASHDIR: "true" + CCACHE_MAXSIZE: 1G + HOMEBREW_NO_AUTO_UPDATE: "1" + HOMEBREW_NO_INSTALL_CLEANUP: "1" + jobs: - build-test: - name: build-test (${{ matrix.name }}) + lint: + name: lint + runs-on: ubuntu-24.04 + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + # actionlint checks every workflow (expressions, job graph, action inputs) + # and runs shellcheck over their `run:` scripts. Pinned release, checked + # against its published SHA-256. + - name: Workflows (actionlint + shellcheck) + run: | + v=1.7.12 + curl -sSfL -o "$RUNNER_TEMP/actionlint.tar.gz" "https://github.com/rhysd/actionlint/releases/download/v$v/actionlint_${v}_linux_amd64.tar.gz" + echo "8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8 $RUNNER_TEMP/actionlint.tar.gz" | sha256sum -c - + tar -xzf "$RUNNER_TEMP/actionlint.tar.gz" -C "$RUNNER_TEMP" actionlint + "$RUNNER_TEMP/actionlint" -color + - name: CI scripts (shellcheck) + run: shellcheck -S warning tools/ci/*.sh tools/release/*.sh tools/*.sh + - name: Change detection (tools/ci/changes.sh) + run: tools/ci/test-changes.sh + - name: Agent skills + run: python3 tools/agents/check_skills.py + + tests: + name: tests (${{ matrix.name }}) strategy: fail-fast: false matrix: @@ -37,50 +104,34 @@ jobs: - name: Linux runner: ubuntu-latest runs-on: ${{ matrix.runner }} - env: - CARGO_TERM_COLOR: always - # Speed: CI builds start from a restored target dir, never edit-compile - # loops, so incremental data is dead weight (bigger cache, slower - # builds). Line tables keep file:line in test backtraces. - CARGO_INCREMENTAL: "0" - CARGO_PROFILE_DEV_DEBUG: line-tables-only - # Every C/C++ compile (the standalone engine build and the two the - # engine build script runs, dev and release) goes through ccache, so an - # unchanged engine file is compiled once and then served from the cache. - CMAKE_C_COMPILER_LAUNCHER: ccache - CMAKE_CXX_COMPILER_LAUNCHER: ccache - CCACHE_DIR: ${{ github.workspace }}/.ccache - CCACHE_BASEDIR: ${{ github.workspace }} - CCACHE_NOHASHDIR: "true" - CCACHE_MAXSIZE: 1G + timeout-minutes: 45 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - fetch-depth: 2 # the merge commit and its base, for the docs-only check - # A pull request that changes only documentation skips the build and - # tests: every later step checks steps.changes.outputs.skip. The jobs - # still run (and pass), so the required build-test checks report. - - name: Changed files + fetch-depth: 2 # the merge commit and its base, for the change check + - name: What this change needs id: changes shell: bash run: | - skip=false - if [ "$GITHUB_EVENT_NAME" = "pull_request" ]; then - files=$(git diff --name-only HEAD^1 HEAD) - echo "$files" - # Anything outside these patterns needs the full run. - if [ -n "$files" ] && ! echo "$files" | grep -qvE '\.md$|^docs/|^specs/|^\.agents/|^LICENSE|^\.github/ISSUE_TEMPLATE/|^\.github/dependabot\.yml$'; then - skip=true - echo "Only documentation changed: skipping the build and tests." - fi + if [ "$GITHUB_EVENT_NAME" = pull_request ]; then + git diff --name-only HEAD^1 HEAD > "$RUNNER_TEMP/changed.txt" + cat "$RUNNER_TEMP/changed.txt" + tools/ci/changes.sh < "$RUNNER_TEMP/changed.txt" > "$RUNNER_TEMP/needs.txt" + else + printf 'engine=true\nrust=true\napp=true\n' > "$RUNNER_TEMP/needs.txt" fi - echo "skip=$skip" >> "$GITHUB_OUTPUT" - - uses: dtolnay/rust-toolchain@stable - if: steps.changes.outputs.skip != 'true' - + tee -a "$GITHUB_OUTPUT" < "$RUNNER_TEMP/needs.txt" + if grep -qE '^(engine|rust)=true' "$RUNNER_TEMP/needs.txt"; then run=true; else run=false; echo "Nothing in this change reaches the engine or Rust tests: skipping them."; fi + echo "run=$run" | tee -a "$GITHUB_OUTPUT" + - name: Rust toolchain (stable) + if: steps.changes.outputs.run == 'true' + shell: bash + run: | + rustup toolchain install stable --profile minimal --no-self-update + rustup default stable - name: Install build tools and system packages (Linux) - if: runner.os == 'Linux' && steps.changes.outputs.skip != 'true' + if: runner.os == 'Linux' && steps.changes.outputs.run == 'true' run: | sudo apt-get update sudo apt-get install -y --no-install-recommends ninja-build pkg-config ccache \ @@ -88,12 +139,12 @@ jobs: libpipewire-0.3-dev libasound2-dev \ libxkbcommon-dev libxkbcommon-x11-dev libwayland-dev libx11-dev libx11-xcb-dev \ libxcursor-dev libxrandr-dev libxi-dev libfontconfig1-dev libfreetype-dev libudev-dev \ - pipewire pipewire-bin wireplumber dbus xvfb + pipewire pipewire-bin wireplumber dbus - name: Install Ninja and ccache (macOS) - if: runner.os == 'macOS' && steps.changes.outputs.skip != 'true' + if: runner.os == 'macOS' && steps.changes.outputs.run == 'true' run: brew install ninja ccache - name: Install ccache (Windows) - if: runner.os == 'Windows' && steps.changes.outputs.skip != 'true' + if: runner.os == 'Windows' && steps.changes.outputs.run == 'true' shell: pwsh run: | $v = '4.12.1' @@ -104,7 +155,7 @@ jobs: Expand-Archive $zip -DestinationPath $env:RUNNER_TEMP Join-Path $env:RUNNER_TEMP "ccache-$v-windows-x86_64" | Out-File -FilePath $env:GITHUB_PATH -Append -Encoding utf8 - name: MSVC developer environment (Windows) - if: runner.os == 'Windows' && steps.changes.outputs.skip != 'true' + if: runner.os == 'Windows' && steps.changes.outputs.run == 'true' shell: pwsh run: | $vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" @@ -113,8 +164,8 @@ jobs: cmd /c "`"$vs\VC\Auxiliary\Build\vcvars64.bat`" >nul && set" | ForEach-Object { if ($_ -match '^([^=]+)=(.*)$') { "$($Matches[1])=$($Matches[2])" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 } } - - name: Tool versions - if: steps.changes.outputs.skip != 'true' + - name: Tool versions and build directories + if: steps.changes.outputs.run == 'true' shell: bash run: | cmake --version | head -1 @@ -122,154 +173,234 @@ jobs: ccache --version | head -1 rustc --version if [ "$RUNNER_OS" = "Windows" ]; then cl 2>&1 | head -1; else c++ --version | head -1; fi - - # Self-hosted runners keep a cargo target dir and an SDK download dir per - # runner outside the workspace (checkout wipes the workspace's ignored - # files), so each runner keeps its own incremental build and nothing is - # uploaded. Saving target/ to the GitHub cache took 26 minutes a run. - # GitHub-hosted runners start empty, so they use the workspace and - # actions/cache instead. - - name: Build directories - if: steps.changes.outputs.skip != 'true' - shell: bash - run: | - if [ "$RUNNER_ENVIRONMENT" = "github-hosted" ]; then - target="$GITHUB_WORKSPACE/target" - deps="$GITHUB_WORKSPACE/.deps" - else - base="$HOME/.cache/devicestudio-ci/$RUNNER_NAME" - target="$base/target" - deps="$base/deps" - fi - mkdir -p "$target" "$deps" - echo "CARGO_TARGET_DIR=$target" >> "$GITHUB_ENV" - echo "DSTUDIO_DEPS_DIR=$deps" >> "$GITHUB_ENV" - echo "target: $target" - echo "deps: $deps" + echo "CARGO_TARGET_DIR=$GITHUB_WORKSPACE/target" >> "$GITHUB_ENV" + echo "DSTUDIO_DEPS_DIR=$GITHUB_WORKSPACE/.deps" >> "$GITHUB_ENV" - name: Cache SDK downloads - if: runner.environment == 'github-hosted' && steps.changes.outputs.skip != 'true' - uses: actions/cache@v6 + if: steps.changes.outputs.engine == 'true' + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: .deps key: deps-${{ runner.os }}-${{ hashFiles('engine/CMakeLists.txt') }} - # The cargo target dir (about 2 GB a system) and the ccache store are - # restored on every run but saved only by pushes to main (the save steps - # after the release build), under a new key each time. Pull requests then - # always start from main's latest build, and their own saves no longer - # fill the repository's 10 GB cache and evict main's. + # The cargo target dir (debug builds only in this job) and the ccache + # store are restored on every run but saved only by pushes to main and + # the nightly run, under a new key each time. Pull requests then start + # from main's latest build, and their own saves cannot fill the + # repository's 10 GB cache and evict main's. The cargo-v2 fallback is the + # layout before the tests/app split (one cache with both profiles). - name: Restore cargo registry and target - if: runner.environment == 'github-hosted' && steps.changes.outputs.skip != 'true' + if: steps.changes.outputs.run == 'true' id: cargo-cache - uses: actions/cache/restore@v6 + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.cargo/registry/index ~/.cargo/registry/cache ~/.cargo/git/db target - key: cargo-v2-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}-${{ github.sha }} + key: cargo-v3-${{ runner.os }}-tests-${{ hashFiles('Cargo.lock') }}-${{ github.sha }} restore-keys: | + cargo-v3-${{ runner.os }}-tests-${{ hashFiles('Cargo.lock') }}- + cargo-v3-${{ runner.os }}-tests- cargo-v2-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}- cargo-v2-${{ runner.os }}- - name: Restore ccache - if: runner.environment == 'github-hosted' && steps.changes.outputs.skip != 'true' + if: steps.changes.outputs.run == 'true' id: ccache-cache - uses: actions/cache/restore@v6 + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: .ccache - key: ccache-${{ runner.os }}-${{ github.sha }} - restore-keys: ccache-${{ runner.os }}- - - name: Reset ccache statistics - if: steps.changes.outputs.skip != 'true' - run: ccache --zero-stats - # Only restored caches (hosted runners) can hold a CMakeCache.txt made - # in another checkout; the per-runner dirs always build in place. - - name: Drop CMake caches restored from another runner's checkout - if: runner.os != 'Windows' && runner.environment == 'github-hosted' && steps.changes.outputs.skip != 'true' + key: ccache-v2-${{ runner.os }}-tests-${{ github.sha }} + restore-keys: | + ccache-v2-${{ runner.os }}-tests- + ccache-${{ runner.os }}- + # See tools/ci/prepare-target.sh: keeps this job's profile only, makes + # every source newer than the restored build, drops CMake caches made in + # another checkout and the engine build script's output. + - name: Prepare the restored build + if: steps.changes.outputs.run == 'true' shell: bash - run: | - target=$(cd "$CARGO_TARGET_DIR" && pwd -P) - deps=$(cd "$DSTUDIO_DEPS_DIR" && pwd -P) - find "$target" "$deps" -name CMakeCache.txt > "$RUNNER_TEMP/cmake-caches.txt" - while IFS= read -r cache; do - [ -f "$cache" ] || continue - here=$(cd "$(dirname "$cache")" && pwd -P) - made=$(sed -n 's/^CMAKE_CACHEFILE_DIR:INTERNAL=//p' "$cache") - if [ -d "$made" ]; then made=$(cd "$made" && pwd -P); fi - [ "$made" = "$here" ] && continue - case "$here" in - "$target"/dstudio-third-party/*) stale="$target/dstudio-third-party" ;; - "$deps"/*) stale="$deps" ;; - *) stale="$here" ;; - esac - echo "Removing $stale (its CMakeCache.txt was made in $made)" - rm -rf "$stale" - done < "$RUNNER_TEMP/cmake-caches.txt" - # The target dir (persistent or restored) can hold dstudio-engine's build-script output - # (out/lib) and fingerprints saved from another commit; cargo would then - # skip CMake and link that old engine library. Cleaning the package - # forces build.rs (and so CMake) to run on every run. The shared - # FetchContent tree in $CARGO_TARGET_DIR/dstudio-third-party is kept. - - name: Force the engine build script to rerun - if: steps.changes.outputs.skip != 'true' - run: | - cargo clean -p dstudio-engine --locked - cargo clean -p dstudio-engine --release --locked - + run: tools/ci/prepare-target.sh debug - name: Engine — configure - if: steps.changes.outputs.skip != 'true' + if: steps.changes.outputs.engine == 'true' run: cmake -S engine -B engine/build-ci -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DDSTUDIO_ENGINE_WERROR=ON "-DFETCHCONTENT_BASE_DIR=${{ env.DSTUDIO_DEPS_DIR }}" - name: Engine — build - if: steps.changes.outputs.skip != 'true' + if: steps.changes.outputs.engine == 'true' run: cmake --build engine/build-ci -- -k 0 # Linux has no sound card on the hosted runner, but it can run a real # PipeWire server: a null sink (driven by the system clock like a # device) and a loopback from its monitor to a virtual source. The # device tests then open real PipeWire streams, and the duplex - # loopback test records what the engine plays and checks every click - # lands at its timeline position (tools/ci/headless-pipewire.sh). + # loopback tests (ctest and cargo) record what the engine plays and + # check every click lands at its timeline position + # (tools/ci/headless-pipewire.sh). - name: Headless PipeWire graph (Linux) - if: runner.os == 'Linux' && steps.changes.outputs.skip != 'true' + if: runner.os == 'Linux' && steps.changes.outputs.run == 'true' run: | tools/ci/headless-pipewire.sh | sed -e "s/^export //" -e "s/'//g" >> "$GITHUB_ENV" echo "DSTUDIO_TEST_LOOPBACK_INPUT=dstudio-loop-source" >> "$GITHUB_ENV" - name: Engine — test - if: steps.changes.outputs.skip != 'true' + if: steps.changes.outputs.engine == 'true' run: ctest --test-dir engine/build-ci --output-on-failure -j 4 - name: Engine — PipeWire device tests, verbose (Linux) - if: runner.os == 'Linux' && steps.changes.outputs.skip != 'true' && !cancelled() + if: runner.os == 'Linux' && steps.changes.outputs.engine == 'true' && !cancelled() run: ctest --test-dir engine/build-ci -V -R "^(device_|transport_commands_across_device)" # Plugin editor windows (X11): the main test run has no display, so the # editor test skips there; here it opens the GUI test plugin's window on # Xvfb and checks its size, its timer and fd callbacks, and closing. - name: Engine — plugin editor windows on Xvfb (Linux) - if: runner.os == 'Linux' && steps.changes.outputs.skip != 'true' && !cancelled() + if: runner.os == 'Linux' && steps.changes.outputs.engine == 'true' && !cancelled() run: xvfb-run -a ctest --test-dir engine/build-ci --output-on-failure -R "^plugin_editor_window_x11" - - # The Rust steps run even when the engine tests failed (both results - # are reported; the job still fails). cargo test compiles every crate - # of the workspace, so there is no separate debug `cargo build`: it - # compiled the app a second time for nothing (2 to 4.5 minutes a job). + # Runs even when the engine tests failed (both results are reported; the + # job still fails). cargo test compiles every crate of the workspace, so + # there is no separate debug `cargo build`. - name: Rust workspace — test - if: steps.changes.outputs.skip != 'true' && !cancelled() + if: steps.changes.outputs.rust == 'true' && !cancelled() run: cargo test --workspace --locked --no-fail-fast + - name: ccache statistics + if: steps.changes.outputs.run == 'true' && !cancelled() + run: ccache --show-stats + - name: Save cargo registry and target + if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request' && steps.changes.outputs.run == 'true' && !cancelled() + uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: | + ~/.cargo/registry/index + ~/.cargo/registry/cache + ~/.cargo/git/db + target + key: ${{ steps.cargo-cache.outputs.cache-primary-key }} + - name: Save ccache + if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request' && steps.changes.outputs.run == 'true' && !cancelled() + uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: .ccache + key: ${{ steps.ccache-cache.outputs.cache-primary-key }} - # App smoke tests without a GPU or audio device: a reference-state - # screenshot and the scripted self-test (--no-audio), on a software - # renderer (Linux: Xvfb + Mesa lavapipe Vulkan; Windows: WARP via DX12). - # The self-test prints "self-test: P / N checks passed"; P must equal N - # (skipped checks are not counted in N). Release build: the build users - # run. - - name: App — release build (smoke tests) - if: steps.changes.outputs.skip != 'true' && !cancelled() + app: + name: app (${{ matrix.name }}) + strategy: + fail-fast: false + matrix: + include: + - name: macOS + runner: macos-latest + - name: Windows + runner: windows-latest + - name: Linux + runner: ubuntu-latest + runs-on: ${{ matrix.runner }} + # The Windows self-test under WARP alone takes about 30 minutes. + timeout-minutes: 90 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + fetch-depth: 2 # the merge commit and its base, for the change check + - name: What this change needs + id: changes + shell: bash + run: | + if [ "$GITHUB_EVENT_NAME" = pull_request ]; then + git diff --name-only HEAD^1 HEAD > "$RUNNER_TEMP/changed.txt" + cat "$RUNNER_TEMP/changed.txt" + tools/ci/changes.sh < "$RUNNER_TEMP/changed.txt" > "$RUNNER_TEMP/needs.txt" + else + printf 'engine=true\nrust=true\napp=true\n' > "$RUNNER_TEMP/needs.txt" + fi + cat "$RUNNER_TEMP/needs.txt" + if grep -q '^app=true' "$RUNNER_TEMP/needs.txt"; then run=true; else run=false; echo "Nothing in this change reaches the app: skipping its build and smoke tests."; fi + echo "run=$run" | tee -a "$GITHUB_OUTPUT" + - name: Rust toolchain (stable) + if: steps.changes.outputs.run == 'true' + shell: bash + run: | + rustup toolchain install stable --profile minimal --no-self-update + rustup default stable + - name: Install build tools and system packages (Linux) + if: runner.os == 'Linux' && steps.changes.outputs.run == 'true' + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends ninja-build pkg-config ccache \ + xvfb mesa-vulkan-drivers \ + libpipewire-0.3-dev libasound2-dev \ + libxkbcommon-dev libxkbcommon-x11-dev libwayland-dev libx11-dev libx11-xcb-dev \ + libxcursor-dev libxrandr-dev libxi-dev libfontconfig1-dev libfreetype-dev libudev-dev \ + pipewire pipewire-bin wireplumber dbus + - name: Install Ninja and ccache (macOS) + if: runner.os == 'macOS' && steps.changes.outputs.run == 'true' + run: brew install ninja ccache + - name: Install ccache (Windows) + if: runner.os == 'Windows' && steps.changes.outputs.run == 'true' + shell: pwsh + run: | + $v = '4.12.1' + $zip = Join-Path $env:RUNNER_TEMP 'ccache.zip' + Invoke-WebRequest "https://github.com/ccache/ccache/releases/download/v$v/ccache-$v-windows-x86_64.zip" -OutFile $zip + $hash = (Get-FileHash $zip -Algorithm SHA256).Hash + if ($hash -ne '98AEA520D66905B8BA7A8E648A4CC0CA941D5E119D441F1E879A4A9045BF18F6') { throw "ccache zip checksum mismatch: $hash" } + Expand-Archive $zip -DestinationPath $env:RUNNER_TEMP + Join-Path $env:RUNNER_TEMP "ccache-$v-windows-x86_64" | Out-File -FilePath $env:GITHUB_PATH -Append -Encoding utf8 + - name: MSVC developer environment (Windows) + if: runner.os == 'Windows' && steps.changes.outputs.run == 'true' + shell: pwsh + run: | + $vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" + $vs = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath + if (-not $vs) { throw "no Visual Studio with the C++ x64 tools" } + cmd /c "`"$vs\VC\Auxiliary\Build\vcvars64.bat`" >nul && set" | ForEach-Object { + if ($_ -match '^([^=]+)=(.*)$') { "$($Matches[1])=$($Matches[2])" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 } + } + - name: Tool versions and build directories + if: steps.changes.outputs.run == 'true' + shell: bash + run: | + cmake --version | head -1 + ninja --version + ccache --version | head -1 + rustc --version + if [ "$RUNNER_OS" = "Windows" ]; then cl 2>&1 | head -1; else c++ --version | head -1; fi + echo "CARGO_TARGET_DIR=$GITHUB_WORKSPACE/target" >> "$GITHUB_ENV" + - name: Restore cargo registry and target + if: steps.changes.outputs.run == 'true' + id: cargo-cache + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: | + ~/.cargo/registry/index + ~/.cargo/registry/cache + ~/.cargo/git/db + target + key: cargo-v3-${{ runner.os }}-app-${{ hashFiles('Cargo.lock') }}-${{ github.sha }} + restore-keys: | + cargo-v3-${{ runner.os }}-app-${{ hashFiles('Cargo.lock') }}- + cargo-v3-${{ runner.os }}-app- + cargo-v2-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}- + cargo-v2-${{ runner.os }}- + - name: Restore ccache + if: steps.changes.outputs.run == 'true' + id: ccache-cache + uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: .ccache + key: ccache-v2-${{ runner.os }}-app-${{ github.sha }} + restore-keys: | + ccache-v2-${{ runner.os }}-app- + ccache-${{ runner.os }}- + - name: Prepare the restored build + if: steps.changes.outputs.run == 'true' + shell: bash + run: tools/ci/prepare-target.sh release + # Release build: the build users run. + - name: App — release build + if: steps.changes.outputs.run == 'true' run: cargo build --release -p dstudio-app --locked - name: ccache statistics - if: steps.changes.outputs.skip != 'true' && !cancelled() + if: steps.changes.outputs.run == 'true' && !cancelled() run: ccache --show-stats # Saved before the smoke tests, so a hung self-test cannot lose them. - name: Save cargo registry and target - if: github.event_name == 'push' && runner.environment == 'github-hosted' && steps.changes.outputs.skip != 'true' && !cancelled() - uses: actions/cache/save@v6 + if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request' && steps.changes.outputs.run == 'true' && !cancelled() + uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.cargo/registry/index @@ -278,56 +409,56 @@ jobs: target key: ${{ steps.cargo-cache.outputs.cache-primary-key }} - name: Save ccache - if: github.event_name == 'push' && runner.environment == 'github-hosted' && steps.changes.outputs.skip != 'true' && !cancelled() - uses: actions/cache/save@v6 + if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request' && steps.changes.outputs.run == 'true' && !cancelled() + uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: .ccache key: ${{ steps.ccache-cache.outputs.cache-primary-key }} + + # App smoke tests without a GPU or audio device: a reference-state + # screenshot and the scripted self-test, on a software renderer (Linux: + # Xvfb + Mesa lavapipe Vulkan; Windows: WARP via DX12). The self-test + # prints "self-test: P / N checks passed"; P must equal N (skipped checks + # are not counted in N), checked by tools/ci/self-test.sh. - name: App — self-test (macOS) - if: runner.os == 'macOS' && steps.changes.outputs.skip != 'true' && !cancelled() + if: runner.os == 'macOS' && steps.changes.outputs.run == 'true' env: RUST_BACKTRACE: "1" - run: | - status=0 - "$CARGO_TARGET_DIR/release/device-studio" --self-test --no-audio > selftest-macos.txt 2> selftest-macos-stderr.txt || status=$? - echo "self-test exit status: $status" - grep -E "^(FAIL|SKIP)" selftest-macos.txt || true - line=$(grep -E "^self-test: [0-9]+ / [0-9]+ checks passed" selftest-macos.txt) || { tail -30 selftest-macos-stderr.txt; exit 1; } - echo "$line" - [ "$(echo "$line" | sed -E 's#self-test: ([0-9]+) / ([0-9]+).*#\1#')" = "$(echo "$line" | sed -E 's#self-test: ([0-9]+) / ([0-9]+).*#\2#')" ] - if [ "$status" -ne 0 ]; then tail -30 selftest-macos-stderr.txt; exit "$status"; fi - - name: App — screenshot and self-test (Linux, Xvfb + lavapipe) - if: runner.os == 'Linux' && steps.changes.outputs.skip != 'true' && !cancelled() + run: tools/ci/self-test.sh macos "$CARGO_TARGET_DIR/release/device-studio" --self-test --no-audio + # Two self-tests: --no-audio (offline rendering and export, like macOS and + # Windows) and --self-test-audio (plays, records and exports through the + # headless PipeWire graph: a real output stream). Each takes about five + # minutes under lavapipe and they share nothing (each run has its own + # scratch folder), so they run side by side on two X servers. + - name: Headless PipeWire graph (Linux) + if: runner.os == 'Linux' && steps.changes.outputs.run == 'true' + run: tools/ci/headless-pipewire.sh | sed -e "s/^export //" -e "s/'//g" >> "$GITHUB_ENV" + - name: App — screenshot and self-tests (Linux, Xvfb + lavapipe) + if: runner.os == 'Linux' && steps.changes.outputs.run == 'true' env: WGPU_BACKEND: vulkan SLINT_WGPU_CPU: "1" # Slint refuses CPU adapters (lavapipe) unless allowed RUST_BACKTRACE: "1" run: | - xvfb-run -a -s "-screen 0 1600x1000x24" "$CARGO_TARGET_DIR/release/device-studio" --state reference-default --screenshot app-linux.png + app="$CARGO_TARGET_DIR/release/device-studio" + xvfb-run -n 90 -s "-screen 0 1600x1000x24" "$app" --state reference-default --screenshot app-linux.png test -s app-linux.png - status=0 - xvfb-run -a -s "-screen 0 1600x1000x24" "$CARGO_TARGET_DIR/release/device-studio" --self-test --no-audio > selftest-linux.txt 2> selftest-linux-stderr.txt || status=$? - echo "self-test exit status: $status" - grep -E "^(FAIL|SKIP)" selftest-linux.txt || true - line=$(grep -E "^self-test: [0-9]+ / [0-9]+ checks passed" selftest-linux.txt) || { tail -30 selftest-linux-stderr.txt; exit 1; } - echo "$line" - [ "$(echo "$line" | sed -E 's#self-test: ([0-9]+) / ([0-9]+).*#\1#')" = "$(echo "$line" | sed -E 's#self-test: ([0-9]+) / ([0-9]+).*#\2#')" ] - if [ "$status" -ne 0 ]; then tail -30 selftest-linux-stderr.txt; exit "$status"; fi - # Again with the audio device: the app plays, records and exports - # through the headless PipeWire graph (a real output stream). - status=0 - xvfb-run -a -s "-screen 0 1600x1000x24" "$CARGO_TARGET_DIR/release/device-studio" --self-test-audio > selftest-linux-audio.txt 2> selftest-linux-audio-stderr.txt || status=$? - echo "self-test (audio) exit status: $status" - grep -E "^(FAIL|SKIP)" selftest-linux-audio.txt || true - line=$(grep -E "^self-test: [0-9]+ / [0-9]+ checks passed" selftest-linux-audio.txt) || { tail -30 selftest-linux-audio-stderr.txt; exit 1; } - echo "$line" - [ "$(echo "$line" | sed -E 's#self-test: ([0-9]+) / ([0-9]+).*#\1#')" = "$(echo "$line" | sed -E 's#self-test: ([0-9]+) / ([0-9]+).*#\2#')" ] - if [ "$status" -ne 0 ]; then tail -30 selftest-linux-audio-stderr.txt; exit "$status"; fi - # Windows smoke tests are informational (continue-on-error): see the - # exit codes they print. + tools/ci/self-test.sh linux xvfb-run -n 91 -s "-screen 0 1600x1000x24" "$app" --self-test --no-audio > "$RUNNER_TEMP/selftest.log" 2>&1 & + plain=$! + tools/ci/self-test.sh linux-audio xvfb-run -n 92 -s "-screen 0 1600x1000x24" "$app" --self-test-audio > "$RUNNER_TEMP/selftest-audio.log" 2>&1 & + audio=$! + plain_status=0; wait "$plain" || plain_status=$? + audio_status=0; wait "$audio" || audio_status=$? + echo "::group::self-test --no-audio"; cat "$RUNNER_TEMP/selftest.log"; echo "::endgroup::" + echo "::group::self-test --self-test-audio"; cat "$RUNNER_TEMP/selftest-audio.log"; echo "::endgroup::" + grep -h -E "^(self-test|::error)" "$RUNNER_TEMP/selftest.log" "$RUNNER_TEMP/selftest-audio.log" + [ "$plain_status" -eq 0 ] && [ "$audio_status" -eq 0 ] + # Windows exits with code 0x87A after a run that passed (not yet + # explained, see engine/README.md), so the exit code is printed but not + # checked; a missing screenshot or self-test summary, or a failed check, + # fails the step. - name: App — screenshot (Windows, WARP) - if: runner.os == 'Windows' && steps.changes.outputs.skip != 'true' && !cancelled() - continue-on-error: true + if: runner.os == 'Windows' && steps.changes.outputs.run == 'true' timeout-minutes: 5 shell: pwsh env: @@ -340,18 +471,17 @@ jobs: Get-Content shot-windows-stderr.txt "screenshot run exit code: 0x{0:X8}" -f $p.ExitCode if (-not (Test-Path app-windows.png)) { throw "no screenshot" } - # The full self-test under WARP is slow (about 28 minutes: the software + # The full self-test under WARP takes about 30 minutes (the software # renderer is capped at two frames in flight so it no longer runs out of - # memory), so running it on every pull request would hold the Windows - # check for half an hour. It runs on pushes to main, and on a pull - # request only when the pull request has the `windows-self-test` label - # (add the label, then push). + # memory), so pull requests skip it unless they carry the + # `windows-self-test` label (add the label, then push). Pushes to main, + # the nightly run and manual runs always run it, and a failure fails the + # job. - name: App — self-test (Windows, WARP) if: >- - runner.os == 'Windows' && steps.changes.outputs.skip != 'true' && !cancelled() && - (github.ref == 'refs/heads/main' || contains(github.event.pull_request.labels.*.name, 'windows-self-test')) - continue-on-error: true - timeout-minutes: 40 + runner.os == 'Windows' && steps.changes.outputs.run == 'true' && + (github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'windows-self-test')) + timeout-minutes: 60 shell: pwsh env: WGPU_BACKEND: dx12 @@ -359,8 +489,9 @@ jobs: RUST_BACKTRACE: "1" run: | $exe = Join-Path $env:CARGO_TARGET_DIR "release\device-studio.exe" + $start = Get-Date $p = Start-Process -FilePath $exe -ArgumentList '--self-test','--no-audio' -NoNewWindow -Wait -PassThru -RedirectStandardOutput selftest-windows.txt -RedirectStandardError selftest-windows-stderr.txt - "self-test exit code: 0x{0:X8}" -f $p.ExitCode + "self-test exit code: 0x{0:X8} after {1:N0} s" -f $p.ExitCode, ((Get-Date) - $start).TotalSeconds Select-String -Path selftest-windows.txt -Pattern '^(FAIL|SKIP)' | ForEach-Object { $_.Line } Select-String -Path selftest-windows-stderr.txt -Pattern 'frames waited for the GPU' | ForEach-Object { $_.Line } $line = (Select-String -Path selftest-windows.txt -Pattern '^self-test: (\d+) / (\d+) checks passed').Matches | Select-Object -First 1 @@ -368,8 +499,8 @@ jobs: $line.Value if ($line.Groups[1].Value -ne $line.Groups[2].Value) { throw "self-test failures" } - name: Upload app screenshots and self-test logs - if: steps.changes.outputs.skip != 'true' && !cancelled() - uses: actions/upload-artifact@v7 + if: steps.changes.outputs.run == 'true' && !cancelled() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: app-${{ runner.os }} path: | @@ -377,3 +508,28 @@ jobs: selftest-*.txt shot-*.txt if-no-files-found: ignore + retention-days: 14 + + # The required checks. One per system so branch protection keeps the names it + # already requires; each passes only when every job above passed (a job with + # nothing to do for a change passes too). + build-test: + name: build-test (${{ matrix.name }}) + needs: [lint, tests, app] + if: ${{ !cancelled() }} + strategy: + matrix: + name: [macOS, Windows, Linux] + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Every CI job passed + env: + LINT: ${{ needs.lint.result }} + TESTS: ${{ needs.tests.result }} + APP: ${{ needs.app.result }} + run: | + echo "lint: $LINT" + echo "tests (all systems): $TESTS" + echo "app (all systems): $APP" + [ "$LINT" = success ] && [ "$TESTS" = success ] && [ "$APP" = success ] diff --git a/.github/workflows/perf.yml b/.github/workflows/perf.yml index e82fe6ab..3833a851 100644 --- a/.github/workflows/perf.yml +++ b/.github/workflows/perf.yml @@ -65,17 +65,21 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 90 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false fetch-depth: 0 # the base commit is built from a worktree - - uses: dtolnay/rust-toolchain@stable + - name: Rust toolchain (stable) + shell: bash + run: | + rustup toolchain install stable --profile minimal --no-self-update + rustup default stable - name: Install build tools and Valgrind run: | sudo apt-get update sudo apt-get install -y --no-install-recommends ninja-build pkg-config ccache valgrind \ libpipewire-0.3-dev libasound2-dev libx11-dev - - uses: actions/cache@v6 + - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.cache/ccache @@ -86,7 +90,7 @@ jobs: restore-keys: perf-instructions- - name: Compare with the base run: python3 tools/perf/bench_gate.py all --base "$BASE_REF" --mode instructions --work build/perf --deps .perf-deps - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 if: ${{ !cancelled() }} with: name: perf-instructions @@ -111,11 +115,15 @@ jobs: # Setup (tools, caches) plus the 75-minute comparison, with room left for the warning step. timeout-minutes: 110 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false fetch-depth: 0 - - uses: dtolnay/rust-toolchain@stable + - name: Rust toolchain (stable) + shell: bash + run: | + rustup toolchain install stable --profile minimal --no-self-update + rustup default stable - name: Install build tools (Linux) if: runner.os == 'Linux' run: | @@ -134,7 +142,7 @@ jobs: cmd /c "`"$vs\VC\Auxiliary\Build\vcvars64.bat`" >nul && set" | ForEach-Object { if ($_ -match '^([^=]+)=(.*)$') { "$($Matches[1])=$($Matches[2])" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8 } } - - uses: actions/cache@v6 + - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.cache/ccache @@ -180,7 +188,7 @@ jobs: msg="The timing comparison did not finish on ${{ matrix.name }} (a build or benchmark failed, or the runner timed out). This is a warning only; see the step log." echo "::warning title=Timing run failed on ${{ matrix.name }}::$msg" echo "**Timing run failed on ${{ matrix.name }}**: $msg" >> "$GITHUB_STEP_SUMMARY" - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 if: ${{ !cancelled() }} with: name: perf-time-${{ matrix.name }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 9439cc60..63c9b5bd 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -56,7 +56,7 @@ jobs: outputs: version: ${{ steps.v.outputs.version }} steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false fetch-depth: 0 # the tag's ancestry is checked against main @@ -89,15 +89,16 @@ jobs: needs: version runs-on: macos-15 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: dtolnay/rust-toolchain@stable - with: - targets: aarch64-apple-darwin, x86_64-apple-darwin + - name: Rust toolchain (stable, both macOS targets) + run: | + rustup toolchain install stable --profile minimal --no-self-update --target aarch64-apple-darwin --target x86_64-apple-darwin + rustup default stable - name: Install Ninja run: brew install ninja - - uses: actions/cache@v6 + - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.cargo/registry/index @@ -121,7 +122,7 @@ jobs: ditto -c -k --keepParent "build/macos/Device Studio.app" build/macos/DeviceStudio-app.zip # The unsigned app goes to the signing job (tags only); every other run makes an # ad hoc signed .dmg here, with no secrets in reach. - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 if: github.event_name == 'push' && github.ref_type == 'tag' with: name: macos-app @@ -134,7 +135,7 @@ jobs: - name: Smoke test (the packaged app starts and renders) if: env.IS_RELEASE != 'true' run: bash tools/release/smoke-macos.sh dist/*.dmg - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 if: env.IS_RELEASE != 'true' with: name: package-macos @@ -154,10 +155,10 @@ jobs: env: HAS_SIGNING_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE != '' }} steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: actions/download-artifact@v8 + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 with: name: macos-app path: build/macos @@ -180,6 +181,8 @@ jobs: security unlock-keychain -p "$password" "$keychain" security import "$RUNNER_TEMP/cert.p12" -k "$keychain" -P "$MACOS_CERTIFICATE_PASSWORD" -T /usr/bin/codesign security set-key-partition-list -S apple-tool:,apple: -s -k "$password" "$keychain" + # The search list is the new keychain followed by the existing ones (word splitting intended). + # shellcheck disable=SC2046 security list-keychains -d user -s "$keychain" $(security list-keychains -d user | tr -d '"') rm "$RUNNER_TEMP/cert.p12" if [ -n "$APPLE_API_KEY" ]; then @@ -202,7 +205,7 @@ jobs: rm -f "$RUNNER_TEMP/notary-key.p8" - name: Smoke test (the packaged app starts and renders) run: bash tools/release/smoke-macos.sh dist/*.dmg - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: package-macos path: dist/*.dmg @@ -213,10 +216,14 @@ jobs: needs: version runs-on: windows-2025 steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: dtolnay/rust-toolchain@stable + - name: Rust toolchain (stable) + shell: bash + run: | + rustup toolchain install stable --profile minimal --no-self-update + rustup default stable - name: MSVC developer environment shell: pwsh run: | @@ -230,7 +237,7 @@ jobs: shell: pwsh run: | if (-not (Test-Path "${env:ProgramFiles(x86)}\Inno Setup 6\ISCC.exe")) { choco install innosetup --no-progress -y } - - uses: actions/cache@v6 + - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.cargo/registry/index @@ -259,7 +266,7 @@ jobs: $p = Start-Process -FilePath (Join-Path $dir 'device-studio.exe') -ArgumentList '--state', 'reference-default', '--screenshot', $shot -NoNewWindow -Wait -PassThru "exit code: 0x{0:X8}" -f $p.ExitCode if (-not (Test-Path $shot)) { throw "the installed app did not render" } - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: package-windows path: | @@ -280,10 +287,14 @@ jobs: runner: ubuntu-24.04-arm runs-on: ${{ matrix.runner }} steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - - uses: dtolnay/rust-toolchain@stable + - name: Rust toolchain (stable) + shell: bash + run: | + rustup toolchain install stable --profile minimal --no-self-update + rustup default stable - name: Install build packages run: | sudo apt-get update @@ -291,7 +302,7 @@ jobs: libpipewire-0.3-dev libasound2-dev \ libxkbcommon-dev libxkbcommon-x11-dev libwayland-dev libx11-dev libx11-xcb-dev \ libxcursor-dev libxrandr-dev libxi-dev libfontconfig1-dev libfreetype-dev libudev-dev - - uses: actions/cache@v6 + - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: | ~/.cargo/registry/index @@ -311,12 +322,13 @@ jobs: sudo apt-get install -y --no-install-recommends xvfb mesa-vulkan-drivers mkdir -p "$RUNNER_TEMP/unpack" tar -C "$RUNNER_TEMP/unpack" -xzf dist/*.tar.gz - "$RUNNER_TEMP"/unpack/*/install.sh "$RUNNER_TEMP/prefix" + install_sh=("$RUNNER_TEMP"/unpack/*/install.sh) + "${install_sh[0]}" "$RUNNER_TEMP/prefix" test -x "$RUNNER_TEMP/prefix/lib/device-studio/dstudio_plugin_scanner" xvfb-run -a -s "-screen 0 1600x1000x24" "$RUNNER_TEMP/prefix/bin/device-studio" \ --state reference-default --screenshot "$RUNNER_TEMP/linux.png" test -s "$RUNNER_TEMP/linux.png" - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: package-linux-${{ matrix.arch }} path: dist/*.tar.gz @@ -330,7 +342,7 @@ jobs: permissions: contents: write # create the release and upload the packages steps: - - uses: actions/download-artifact@v8 + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 with: path: dist pattern: package-* diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5f32a907..b29ee8c2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -31,10 +31,10 @@ cargo run --release -p dstudio-app -- --self-test --no-audio # drive the real ``` CI runs the tests and the self-test (not the interactive `--demo`) on macOS, Windows and Linux for every pull request. -Its runners have no audio device, so the self-test runs with `--no-audio` and anything that needs audio hardware has to -be tested by hand. On Windows the self-test step is informational -for now (`continue-on-error`), so a green Windows job does not mean the self-test passed: open that step's output to -check. The engine is built with warnings as errors on +Its runners have no audio device, so the self-test runs with `--no-audio` (Linux also runs it against a headless +PipeWire server) and anything that needs audio hardware has to be tested by hand. The Windows self-test takes about +30 minutes under the software renderer, so pull requests run it only with the `windows-self-test` label; main and +the nightly run always run it, and a failure fails the job. The engine is built with warnings as errors on clang, MSVC and GCC, so please build on more than one compiler if you can, or check the CI result. ## Pull requests diff --git a/docs/building.md b/docs/building.md index 6a64e2a9..49cc519b 100644 --- a/docs/building.md +++ b/docs/building.md @@ -151,16 +151,49 @@ issue in wgpu-core's debug-only lock tracing). ## Continuous integration CI ([`.github/workflows/ci.yml`](../.github/workflows/ci.yml)) runs on every push -to main and every pull request, including pull requests from forks. It has one -job per system, all on GitHub-hosted runners (`macos-latest`, `windows-latest`, -`ubuntu-latest`). Each job: - -1. configures and builds the engine with warnings as errors, and runs ctest; -2. builds and tests the Rust workspace (`--locked`); -3. builds the app in release mode and runs `device-studio --self-test - --no-audio`, which must report every check passed (Linux under Xvfb with - Mesa lavapipe; Windows under WARP, informational for now); -4. uploads the screenshots and self-test logs as an artifact. +to main, every pull request (including pull requests from forks), nightly on +main and on demand (Actions › CI › Run workflow). All jobs run on GitHub-hosted +runners (`macos-latest`, `windows-latest`, `ubuntu-latest`). Each system has two +jobs that run side by side: + +- **tests (system)** configures and builds the engine with warnings as errors + and runs ctest, then builds and tests the Rust workspace (`cargo test + --workspace --locked`, debug). +- **app (system)** builds the app in release mode and runs its smoke tests: a + reference-state screenshot and `device-studio --self-test`, which must report + every check passed (`tools/ci/self-test.sh`). macOS runs `--no-audio`; Linux + runs `--no-audio` and `--self-test-audio` side by side under Xvfb with Mesa + lavapipe; Windows takes the screenshot under WARP on every run, and runs the + full self-test (about 30 minutes under WARP) on pushes to main, the nightly + run, manual runs, and pull requests labelled `windows-self-test`. The + screenshots and self-test logs are uploaded as an artifact. + +A **lint** job checks the workflows (actionlint, which also runs shellcheck on +their scripts), the scripts in `tools/`, the change detection below and the +agent skills, in under a minute. + +The required checks are **build-test (macOS)**, **build-test (Windows)** and +**build-test (Linux)**. Each passes only when lint and every tests and app job +passed, so a failure on one system turns all three red; the failing job is +named in the check list. + +Pull requests run only what their changes can reach +([`tools/ci/changes.sh`](../tools/ci/changes.sh), tested by +`tools/ci/test-changes.sh`): documentation alone builds nothing; Rust or UI +changes skip the standalone engine build; engine tests or benchmarks alone skip +the Rust tests and the app; packaging and benchmark tooling have their own +workflows. Anything the script does not know (`Cargo.toml`, `Cargo.lock`, the +CI workflow, `tools/ci/`, a new folder) runs everything, and so do pushes to +main and the nightly run. A job with nothing to do still runs and passes, so +the required checks always report. + +Caches: the cargo registry and target dir (one per system and job, so debug and +release are restored separately) and the ccache store are restored on every run +and saved only by main and the nightly run, so a pull request never writes a +cache another run uses. Every C/C++ compile goes through ccache. Before +building, [`tools/ci/prepare-target.sh`](../tools/ci/prepare-target.sh) makes +every source newer than the restored build (so nothing restored is reused for +changed code) and reconfigures the engine's CMake tree from scratch. The hosted runners have no audio device and no GPU. On macOS and Windows the engine's device tests skip there. The Linux job starts a headless PipeWire @@ -175,8 +208,9 @@ ignored by default and gated on an environment variable: DSTUDIO_TEST_REAL_INPUT=1 cargo test -p dstudio-engine --test real_loopback -- --ignored --nocapture --test-threads=1 ``` -The workflow never uses a self-hosted runner, has no secrets and gives its -token read-only access, so running it on a stranger's pull request is safe. +The workflow never uses a self-hosted runner, has no secrets, gives its token +read-only access and pins every action to a commit, so running it on a +stranger's pull request is safe. ### Performance checks diff --git a/engine/README.md b/engine/README.md index c3f8c782..ae334ee0 100644 --- a/engine/README.md +++ b/engine/README.md @@ -133,9 +133,11 @@ without a GPU or audio device: on Linux under Xvfb with Mesa lavapipe (software `SLINT_WGPU_CPU=1`), a `--state reference-default` screenshot and the full `--self-test --no-audio` (125 checks pass; the 21 checks of the Audio Unit devices section are reported as skipped off macOS), as a required step; on Windows with WARP -(DX12 software adapter) the same, as an informational step: the screenshot -renders and the self-test passes (125, 21 skipped), but the process exits -with code 0x87A after the run (not yet explained; macOS/Linux exit 0). +(DX12 software adapter) the same, also failing the job when a check fails (the +full self-test runs on main, nightly and labelled pull requests; see +docs/building.md): the screenshot renders and the self-test passes, but the +process exits with code 0x87A after the run (not yet explained; macOS/Linux +exit 0), so CI prints that exit code and checks the summary line instead. Debug builds of the app abort at exit on every OS (pre-existing: wgpu-core's debug-only lock tracing runs in a TLS destructor when Slint's thread-local backend drops the wgpu queue), so CI uses a release build. Screenshots and logs are uploaded as artifacts. diff --git a/tools/ci/changes.sh b/tools/ci/changes.sh new file mode 100755 index 00000000..47068b41 --- /dev/null +++ b/tools/ci/changes.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# Decides which parts of CI a pull request needs, from the paths it changes. +# +# git diff --name-only HEAD^1 HEAD | tools/ci/changes.sh +# +# Prints three lines, each `true` or `false`: +# +# engine= the standalone engine build (CMake, warnings as errors) and ctest +# rust= `cargo test` over the workspace (which also builds the engine through build.rs) +# app= the app's release build and its smoke tests (screenshot, self-test) +# +# A part is skipped only when every changed path is known not to reach it. Anything not +# listed below (Cargo.toml, Cargo.lock, the CI workflow, tools/ci/, a new top-level folder) +# runs everything. Pushes to main and scheduled runs do not call this: they run everything. +# +# documentation (*.md, docs/, specs/, .agents/, LICENSE, +# issue templates, dependabot.yml) nothing +# engine/tests/, engine/bench/, engine/tools/ engine (build.rs builds none of them) +# engine/ (anything else) engine, rust, app +# crates/*/tests/, crates/*/benches/ rust (not part of the app binary) +# crates/ (anything else) rust, app +# tools/perf/, tools/release/, tools/art/, +# tools/agents/, tools/macos-bundle.sh, +# .github/workflows/{perf,release}.yml, +# ui-spike/ nothing (their own workflows, or no CI) +# +# Test it with tools/ci/test-changes.sh. +set -euo pipefail + +engine=false +rust=false +app=false +any=false +while IFS= read -r path || [ -n "$path" ]; do + path="${path%$'\r'}" + [ -n "$path" ] || continue + any=true + case "$path" in + *.md | docs/* | specs/* | .agents/* | LICENSE* | .github/ISSUE_TEMPLATE/* | .github/dependabot.yml) ;; + engine/tests/* | engine/bench/* | engine/tools/*) engine=true ;; + engine/*) engine=true rust=true app=true ;; + crates/*/tests/* | crates/*/benches/*) rust=true ;; + crates/*) rust=true app=true ;; + tools/perf/* | tools/release/* | tools/art/* | tools/agents/* | tools/macos-bundle.sh) ;; + .github/workflows/perf.yml | .github/workflows/release.yml | ui-spike/*) ;; + *) engine=true rust=true app=true ;; + esac +done + +# No list at all (the diff failed or was empty): run everything rather than nothing. +if [ "$any" = false ]; then + engine=true rust=true app=true +fi + +echo "engine=$engine" +echo "rust=$rust" +echo "app=$app" diff --git a/tools/ci/prepare-target.sh b/tools/ci/prepare-target.sh new file mode 100755 index 00000000..1ba7ff7e --- /dev/null +++ b/tools/ci/prepare-target.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# Prepares the cargo target dir restored from the CI cache for this job's build (CI only): +# +# tools/ci/prepare-target.sh debug|release +# +# $CARGO_TARGET_DIR must be set. What it does, and why: +# +# 1. Keeps only this job's profile. The tests job builds debug, the app job release; a target +# dir restored from the older shared cache holds both, and saving the other profile again +# would only grow the cache. +# +# 2. Makes every tracked source newer than the restored build. Cargo (path crates, build +# scripts' rerun-if-changed) and Ninja decide what to rebuild by modification time. +# Checkout writes every file before the restore, so the sources are normally newer anyway, +# but a main run that saves its cache between this job's checkout and its restore would +# make its outputs look newer than the checkout, and cargo would reuse them for different +# sources. Touching after the restore rules that out: every workspace crate and the +# engine are always rebuilt from this commit's sources (C++ objects come back from ccache +# when their preprocessed source, flags and compiler match). +# +# 3. Removes CMake caches that were made in another checkout folder (a renamed repository +# moves the workspace), which CMake refuses to reuse. Not on Windows, whose paths CMake +# spells differently from Git Bash. +# +# 4. Removes the engine build script's output (`cargo clean -p dstudio-engine`), so its CMake +# build tree is configured from scratch. The rebuild itself does not depend on this (step 2 +# already reruns build.rs), but a restored CMakeCache.txt pins the compiler's full path, +# which a runner image update (a new Xcode or MSVC version) removes, and keeps cache +# variables that build.rs no longer passes. The fresh configure costs about 15 to 30 +# seconds; the compiles are ccache hits. +set -euo pipefail + +profile=${1:?usage: prepare-target.sh debug|release} +target=${CARGO_TARGET_DIR:?CARGO_TARGET_DIR is not set} +case "$profile" in + debug) other=release release_flag=() ;; + release) other=debug release_flag=(--release) ;; + *) echo "unknown profile $profile" >&2; exit 2 ;; +esac +mkdir -p "$target" + +# 1. +if [ -d "$target/$other" ]; then + echo "Removing the restored $other build ($target/$other)" + rm -rf "${target:?}/$other" +fi + +# 2. +git ls-files -z | xargs -0 touch -- +echo "Touched $(git ls-files | wc -l | tr -d ' ') tracked files" + +# 3. +if [ "${RUNNER_OS:-}" != "Windows" ]; then + resolved_target=$(cd "$target" && pwd -P) + dirs=("$resolved_target") + if [ -n "${DSTUDIO_DEPS_DIR:-}" ] && [ -d "$DSTUDIO_DEPS_DIR" ]; then + dirs+=("$(cd "$DSTUDIO_DEPS_DIR" && pwd -P)") + fi + while IFS= read -r cache; do + [ -f "$cache" ] || continue + here=$(cd "$(dirname "$cache")" && pwd -P) + made=$(sed -n 's/^CMAKE_CACHEFILE_DIR:INTERNAL=//p' "$cache") + if [ -d "$made" ]; then made=$(cd "$made" && pwd -P); fi + [ "$made" = "$here" ] && continue + case "$here" in + "$resolved_target"/dstudio-third-party/*) stale="$resolved_target/dstudio-third-party" ;; + "${dirs[1]:-/nonexistent}"/*) stale="${dirs[1]}" ;; + *) stale="$here" ;; + esac + echo "Removing $stale (its CMakeCache.txt was made in $made)" + rm -rf "$stale" + done < <(find "${dirs[@]}" -name CMakeCache.txt 2>/dev/null) +fi + +# 4. +cargo clean -p dstudio-engine --locked "${release_flag[@]}" diff --git a/tools/ci/self-test.sh b/tools/ci/self-test.sh new file mode 100755 index 00000000..1f54a44e --- /dev/null +++ b/tools/ci/self-test.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# Runs one app self-test and checks its result (CI, macOS and Linux): +# +# tools/ci/self-test.sh +# tools/ci/self-test.sh linux xvfb-run -a target/release/device-studio --self-test --no-audio +# +# The run's stdout and stderr go to selftest-.txt and selftest--stderr.txt (CI +# uploads them). It passes when the app printed "self-test: P / N checks passed" with P = N +# (skipped checks are not counted in N) and exited with status 0. FAIL and SKIP lines are +# echoed either way, and the tail of stderr when it fails. +set -uo pipefail + +name=$1 +shift +out="selftest-$name.txt" +err="selftest-$name-stderr.txt" + +start=$SECONDS +status=0 +"$@" > "$out" 2> "$err" || status=$? +echo "self-test ($name): exit status $status after $((SECONDS - start)) s" +grep -E "^(FAIL|SKIP)" "$out" || true + +fail() { + echo "::error title=Self-test failed ($name)::$1" + tail -30 "$err" + exit 1 +} + +line=$(grep -E "^self-test: [0-9]+ / [0-9]+ checks passed" "$out") || fail "no self-test summary (exit status $status)" +echo "$line" +passed=$(echo "$line" | sed -E 's#^self-test: ([0-9]+) / ([0-9]+).*#\1#') +total=$(echo "$line" | sed -E 's#^self-test: ([0-9]+) / ([0-9]+).*#\2#') +[ "$passed" = "$total" ] || fail "$line" +[ "$status" -eq 0 ] || fail "every check passed, but the app exited with status $status" diff --git a/tools/ci/test-changes.sh b/tools/ci/test-changes.sh new file mode 100755 index 00000000..fabb862d --- /dev/null +++ b/tools/ci/test-changes.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# Checks tools/ci/changes.sh: which CI parts each kind of change runs. +set -euo pipefail +cd "$(dirname "$0")" + +fails=0 +check() { + local want=$1 + shift + local got + got=$(printf '%s\n' "$@" | ./changes.sh | tr '\n' ' ') + if [ "$got" != "$want " ]; then + echo "FAIL: $* -> $got (want $want)" + fails=$((fails + 1)) + fi +} + +all="engine=true rust=true app=true" +none="engine=false rust=false app=false" + +check "$none" README.md docs/building.md specs/003-midi/spec.md .agents/skills/x/SKILL.md LICENSE +check "$none" tools/perf/bench_gate.py tools/release/make-dmg.sh .github/workflows/release.yml ui-spike/src/main.rs +check "engine=true rust=false app=false" engine/tests/test_pdc.cpp engine/bench/bench_main.cpp +check "$all" engine/src/engine.cpp +check "$all" engine/include/dstudio_engine.h README.md +check "$all" engine/CMakeLists.txt +check "engine=false rust=true app=false" crates/dstudio-core/tests/core.rs crates/dstudio-engine/tests/pdc.rs +check "engine=false rust=true app=true" crates/dstudio-app/src/main.rs +check "engine=false rust=true app=true" crates/dstudio-app/ui/main.slint docs/x.md +check "engine=false rust=true app=true" crates/dstudio-engine/build.rs +# Build configuration, dependencies and CI itself run everything. +check "$all" Cargo.lock +check "$all" Cargo.toml +check "$all" .github/workflows/ci.yml +check "$all" tools/ci/headless-pipewire.sh +check "$all" some-new-folder/file.txt +# An empty list (a failed diff) runs everything. +check "$all" "" + +if [ "$fails" -ne 0 ]; then + echo "$fails check(s) failed" + exit 1 +fi +echo "changes.sh: all checks passed" From 31cece77c46c3d9a4678fa62ecd63273aee34b2d Mon Sep 17 00:00:00 2001 From: Chris Laupama <42254688+chrislaupama@users.noreply.github.com> Date: Wed, 11 Nov 2026 19:05:42 +0000 Subject: [PATCH 2/4] CI: prepare-target.sh runs on macOS's bash 3.2 --- tools/ci/prepare-target.sh | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tools/ci/prepare-target.sh b/tools/ci/prepare-target.sh index 1ba7ff7e..7a52451f 100755 --- a/tools/ci/prepare-target.sh +++ b/tools/ci/prepare-target.sh @@ -33,8 +33,8 @@ set -euo pipefail profile=${1:?usage: prepare-target.sh debug|release} target=${CARGO_TARGET_DIR:?CARGO_TARGET_DIR is not set} case "$profile" in - debug) other=release release_flag=() ;; - release) other=debug release_flag=(--release) ;; + debug) other=release ;; + release) other=debug ;; *) echo "unknown profile $profile" >&2; exit 2 ;; esac mkdir -p "$target" @@ -72,5 +72,9 @@ if [ "${RUNNER_OS:-}" != "Windows" ]; then done < <(find "${dirs[@]}" -name CMakeCache.txt 2>/dev/null) fi -# 4. -cargo clean -p dstudio-engine --locked "${release_flag[@]}" +# 4. (No empty-array expansion: macOS runs bash 3.2, where that fails under `set -u`.) +if [ "$profile" = release ]; then + cargo clean -p dstudio-engine --locked --release +else + cargo clean -p dstudio-engine --locked +fi From 4665732193165b955ff66b5f7faa57e42538d578 Mon Sep 17 00:00:00 2001 From: Chris Laupama <42254688+chrislaupama@users.noreply.github.com> Date: Wed, 11 Nov 2026 19:28:54 +0000 Subject: [PATCH 3/4] CI: take the Linux screenshot alongside the two self-tests --- .github/workflows/ci.yml | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5888c4b4..d72a0c23 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -427,9 +427,11 @@ jobs: run: tools/ci/self-test.sh macos "$CARGO_TARGET_DIR/release/device-studio" --self-test --no-audio # Two self-tests: --no-audio (offline rendering and export, like macOS and # Windows) and --self-test-audio (plays, records and exports through the - # headless PipeWire graph: a real output stream). Each takes about five + # headless PipeWire graph: a real output stream). Each takes about six # minutes under lavapipe and they share nothing (each run has its own - # scratch folder), so they run side by side on two X servers. + # scratch folder), so they run side by side on their own X servers, with + # the screenshot: 6.3 minutes for both instead of 12.3 back to back on a + # 4-core machine, both passing. - name: Headless PipeWire graph (Linux) if: runner.os == 'Linux' && steps.changes.outputs.run == 'true' run: tools/ci/headless-pipewire.sh | sed -e "s/^export //" -e "s/'//g" >> "$GITHUB_ENV" @@ -441,17 +443,20 @@ jobs: RUST_BACKTRACE: "1" run: | app="$CARGO_TARGET_DIR/release/device-studio" - xvfb-run -n 90 -s "-screen 0 1600x1000x24" "$app" --state reference-default --screenshot app-linux.png - test -s app-linux.png + xvfb-run -n 90 -s "-screen 0 1600x1000x24" "$app" --state reference-default --screenshot app-linux.png > "$RUNNER_TEMP/screenshot.log" 2>&1 & + shot=$! tools/ci/self-test.sh linux xvfb-run -n 91 -s "-screen 0 1600x1000x24" "$app" --self-test --no-audio > "$RUNNER_TEMP/selftest.log" 2>&1 & plain=$! tools/ci/self-test.sh linux-audio xvfb-run -n 92 -s "-screen 0 1600x1000x24" "$app" --self-test-audio > "$RUNNER_TEMP/selftest-audio.log" 2>&1 & audio=$! + shot_status=0; wait "$shot" || shot_status=$? plain_status=0; wait "$plain" || plain_status=$? audio_status=0; wait "$audio" || audio_status=$? + echo "::group::screenshot (exit status $shot_status)"; cat "$RUNNER_TEMP/screenshot.log"; echo "::endgroup::" echo "::group::self-test --no-audio"; cat "$RUNNER_TEMP/selftest.log"; echo "::endgroup::" echo "::group::self-test --self-test-audio"; cat "$RUNNER_TEMP/selftest-audio.log"; echo "::endgroup::" - grep -h -E "^(self-test|::error)" "$RUNNER_TEMP/selftest.log" "$RUNNER_TEMP/selftest-audio.log" + grep -h -E "^self-test" "$RUNNER_TEMP/selftest.log" "$RUNNER_TEMP/selftest-audio.log" + if [ "$shot_status" -ne 0 ] || [ ! -s app-linux.png ]; then echo "::error title=Screenshot failed (Linux)::exit status $shot_status"; exit 1; fi [ "$plain_status" -eq 0 ] && [ "$audio_status" -eq 0 ] # Windows exits with code 0x87A after a run that passed (not yet # explained, see engine/README.md), so the exit code is printed but not From c6688f57ad89af558e1df5b384e8bba9f26a0dee Mon Sep 17 00:00:00 2001 From: Chris Laupama <42254688+chrislaupama@users.noreply.github.com> Date: Wed, 11 Nov 2026 19:31:32 +0000 Subject: [PATCH 4/4] CI: zero the restored ccache statistics before building --- tools/ci/prepare-target.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/tools/ci/prepare-target.sh b/tools/ci/prepare-target.sh index 7a52451f..505ad61c 100755 --- a/tools/ci/prepare-target.sh +++ b/tools/ci/prepare-target.sh @@ -22,7 +22,10 @@ # moves the workspace), which CMake refuses to reuse. Not on Windows, whose paths CMake # spells differently from Git Bash. # -# 4. Removes the engine build script's output (`cargo clean -p dstudio-engine`), so its CMake +# 4. Zeroes the ccache statistics (they are stored in the restored cache), so the job's +# `ccache --show-stats` counts this run's compiles only. +# +# 5. Removes the engine build script's output (`cargo clean -p dstudio-engine`), so its CMake # build tree is configured from scratch. The rebuild itself does not depend on this (step 2 # already reruns build.rs), but a restored CMakeCache.txt pins the compiler's full path, # which a runner image update (a new Xcode or MSVC version) removes, and keeps cache @@ -72,7 +75,12 @@ if [ "${RUNNER_OS:-}" != "Windows" ]; then done < <(find "${dirs[@]}" -name CMakeCache.txt 2>/dev/null) fi -# 4. (No empty-array expansion: macOS runs bash 3.2, where that fails under `set -u`.) +# 4. +if command -v ccache > /dev/null; then + ccache --zero-stats > /dev/null +fi + +# 5. (No empty-array expansion: macOS runs bash 3.2, where that fails under `set -u`.) if [ "$profile" = release ]; then cargo clean -p dstudio-engine --locked --release else