Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 38 additions & 7 deletions .github/workflows/perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,15 @@ name: Performance
#
# Locally: python3 tools/perf/bench_gate.py all --base origin/main

#
# The Callgrind job is a required check on main, so it has to report on every pull request:
# pull requests run this workflow whatever they change, and the `changes` job skips both
# comparisons when nothing they measure changed (a skipped job counts as passed). Pushes to
# main keep their path filter.

on:
pull_request:
types: [opened, synchronize, reopened, labeled, unlabeled]
paths:
- 'engine/**'
- 'crates/dstudio-core/**'
- 'Cargo.lock'
- 'tools/perf/**'
- '.github/workflows/perf.yml'
push:
branches: [main]
paths:
Expand Down Expand Up @@ -60,8 +60,38 @@ env:
PERF_ACCEPT: ${{ contains(github.event.pull_request.labels.*.name, 'perf-regression-accepted') && '1' || '' }}

jobs:
changes:
name: Benchmarked code changed
runs-on: ubuntu-24.04
timeout-minutes: 5
outputs:
perf: ${{ steps.changes.outputs.perf }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
if: github.event_name == 'pull_request'
with:
persist-credentials: false
fetch-depth: 0
- name: Paths the benchmarks depend on
id: changes
env:
EVENT: ${{ github.event_name }}
run: |
# Pushes and manual runs always compare. A pull request compares when it touches
# what the benchmarks build, or when the diff can't be read.
perf=true
if [ "$EVENT" = pull_request ] && files=$(git diff --name-only "$BASE_REF" HEAD); then
if ! printf '%s\n' "$files" | grep -qE '^(engine/|crates/dstudio-core/|Cargo\.lock$|tools/perf/|\.github/workflows/perf\.yml$)'; then
perf=false
fi
fi
echo "perf=$perf" | tee -a "$GITHUB_OUTPUT"

instructions:
name: Instructions (Linux, Callgrind)
needs: changes
# Skipped (which counts as passed) only when the changes job said so; if it failed, compare anyway.
if: ${{ !cancelled() && needs.changes.outputs.perf != 'false' }}
runs-on: ubuntu-24.04
timeout-minutes: 90
steps:
Expand Down Expand Up @@ -102,8 +132,9 @@ jobs:

time:
name: Time (${{ matrix.name }})
needs: changes
# Adding or removing the accept label only matters to the gate above.
if: github.event.action != 'labeled' && github.event.action != 'unlabeled'
if: ${{ !cancelled() && needs.changes.outputs.perf != 'false' && github.event.action != 'labeled' && github.event.action != 'unlabeled' }}
strategy:
fail-fast: false
matrix:
Expand Down
16 changes: 10 additions & 6 deletions docs/building.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,10 +174,12 @@ A **lint** job checks the workflows (actionlint, which also runs shellcheck on
their scripts), the scripts in `tools/`, the change detection below and the
agent skills, in under a minute.

The required checks are **build-test (macOS)**, **build-test (Windows)** and
**build-test (Linux)**. Each passes only when lint and every tests and app job
passed, so a failure on one system turns all three red; the failing job is
named in the check list.
The required checks are **build-test (macOS)**, **build-test (Windows)**,
**build-test (Linux)**, **lint** and the performance gate **Instructions (Linux,
Callgrind)** (below). Each build-test check passes only when lint and every
tests and app job passed, so a failure on one system turns all three red; the
failing job is named in the check list. The Security workflow's checks and the
timing jobs are advisory.

Pull requests run only what their changes can reach
([`tools/ci/changes.sh`](../tools/ci/changes.sh), tested by
Expand Down Expand Up @@ -238,10 +240,12 @@ they are updated.

### Performance checks

[`.github/workflows/perf.yml`](../.github/workflows/perf.yml) runs on pull
[`.github/workflows/perf.yml`](../.github/workflows/perf.yml) compares on pull
requests that touch `engine/`, `crates/dstudio-core/`, `Cargo.lock`,
`tools/perf/` or the workflow itself, and on
pushes to `main`. It builds every engine benchmark (`engine/bench/`) and the
pushes to `main`. It starts on every pull request, because the Instructions
check is required: for any other change its `changes` job skips both
comparisons, and a skipped job counts as passed. It builds every engine benchmark (`engine/bench/`) and the
core's Rust benchmarks (`crates/dstudio-core/benches/`) twice on the same
runner, at the base commit and with the change, and compares them row by row
with [`tools/perf/bench_gate.py`](../tools/perf/bench_gate.py):
Expand Down
Loading