Skip to content

ci: extract the repeated ccache and apt scaffolding from native_release #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leehack merged 3 commits into from
Sep 22, 2026
Merged

ci: extract the repeated ccache and apt scaffolding from native_release #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
df4dd12
ci: extract the repeated ccache and apt scaffolding from native_release
leehack Sep 22, 2026
a334fd9
ci: scope the HIP apt frontend to the install call
leehack Sep 22, 2026
4966ed1
test: pin the shell and container of every ccache lane
leehack Sep 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/actions/ccache-stats/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Report ccache stats
description: Print ccache statistics and append them to the job summary.
inputs:
label:
description: Lane name shown in the job summary heading
required: true
runs:
using: composite
steps:
- name: ccache stats
shell: bash
run: ccache --show-stats || true
- name: Append ccache stats to summary
shell: bash
env:
CCACHE_STATS_LABEL: ${{ inputs.label }}
run: |
{
echo "### ccache stats (${CCACHE_STATS_LABEL})"
echo '```'
ccache --show-stats || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
36 changes: 36 additions & 0 deletions .github/actions/setup-ccache/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Setup ccache
description: Restore a build lane's ccache directory and prime it for a fresh run.
inputs:
cache-dir:
description: ccache directory to cache between runs
required: true
max-size:
description: Ceiling passed to ccache --max-size
required: true
key-prefix:
description: Cache key prefix for this lane; the run id is appended
required: true
extra-restore-keys:
description: Further restore-key prefixes, one per line, most specific first
required: false
default: ''
runs:
using: composite
steps:
- name: Restore ccache
uses: actions/cache@v6
with:
path: ${{ inputs.cache-dir }}
key: ${{ inputs.key-prefix }}-${{ github.run_id }}
restore-keys: |
${{ inputs.key-prefix }}-
${{ inputs.extra-restore-keys }}
- name: Prime ccache
shell: bash
env:
CCACHE_DIR: ${{ inputs.cache-dir }}
CCACHE_MAXSIZE: ${{ inputs.max-size }}
run: |
mkdir -p "${CCACHE_DIR}"
ccache --max-size "${CCACHE_MAXSIZE}"
ccache --zero-stats
191 changes: 43 additions & 148 deletions .github/workflows/native_release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,33 +183,20 @@ jobs:
echo "Using pinned Android NDK: $NDK_DIR"
- name: Install build deps
run: sudo apt-get update && sudo apt-get install -y ninja-build ccache
- name: Restore ccache
uses: actions/cache@v6
- name: Set up ccache
uses: ./.github/actions/setup-ccache
with:
path: ${{ env.CCACHE_DIR }}
key: ccache-android-${{ matrix.android_abi }}-${{ matrix.backend }}-${{ github.run_id }}
restore-keys: |
ccache-android-${{ matrix.android_abi }}-${{ matrix.backend }}-
ccache-android-${{ matrix.android_abi }}-
- name: Prime ccache
run: |
mkdir -p "${CCACHE_DIR}"
ccache --max-size "${CCACHE_MAXSIZE}"
ccache --zero-stats
cache-dir: ${{ env.CCACHE_DIR }}
max-size: ${{ env.CCACHE_MAXSIZE }}
key-prefix: ccache-android-${{ matrix.android_abi }}-${{ matrix.backend }}
extra-restore-keys: ccache-android-${{ matrix.android_abi }}-
- name: Build
run: python3 tools/build.py android --abi ${{ matrix.android_abi }} --backend ${{ matrix.backend }}
- name: ccache stats
- name: Report ccache stats
if: always()
run: ccache --show-stats || true
- name: Append ccache stats to summary
if: always()
run: |
{
echo "### ccache stats (android/${{ matrix.android_abi }}/${{ matrix.backend }})"
echo '```'
ccache --show-stats || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
uses: ./.github/actions/ccache-stats
with:
label: android/${{ matrix.android_abi }}/${{ matrix.backend }}
- name: Validate Android backend artifacts
run: |
set -euo pipefail
Expand Down Expand Up @@ -337,37 +324,24 @@ jobs:
run: |
brew untap aws/tap 2>/dev/null || true
brew install ccache
- name: Restore ccache
uses: actions/cache@v6
- name: Set up ccache
uses: ./.github/actions/setup-ccache
with:
path: ${{ env.CCACHE_DIR }}
key: ccache-apple-${{ matrix.target }}-${{ github.run_id }}
restore-keys: |
ccache-apple-${{ matrix.target }}-
- name: Prime ccache
run: |
mkdir -p "${CCACHE_DIR}"
ccache --max-size "${CCACHE_MAXSIZE}"
ccache --zero-stats
cache-dir: ${{ env.CCACHE_DIR }}
max-size: ${{ env.CCACHE_MAXSIZE }}
key-prefix: ccache-apple-${{ matrix.target }}
- name: Build
run: python3 tools/build.py apple --target ${{ matrix.target }}
- name: Validate Apple wrapper exports
run: |
OUT_LIB="bin/${{ matrix.out_dir }}/libllamadart.dylib"
test -f "$OUT_LIB" || { echo "Missing $OUT_LIB"; exit 1; }
python3 tools/validate_exports.py --format nm --tool nm "$OUT_LIB"
- name: ccache stats
- name: Report ccache stats
if: always()
run: ccache --show-stats || true
- name: Append ccache stats to summary
if: always()
run: |
{
echo "### ccache stats (apple/${{ matrix.target }})"
echo '```'
ccache --show-stats || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
uses: ./.github/actions/ccache-stats
with:
label: apple/${{ matrix.target }}
- name: Upload
uses: actions/upload-artifact@v7
with:
Expand Down Expand Up @@ -433,56 +407,13 @@ jobs:
github_token: ${{ github.token }}
- name: Install build deps
run: |
apt_get_update() {
for attempt in 1 2 3; do
if sudo apt-get -o Acquire::Retries=3 update; then
return 0
fi
if [ "$attempt" = "3" ]; then
return 1
fi
sleep $((attempt * 15))
done
}
apt_get_install() {
for attempt in 1 2 3; do
if sudo apt-get -o Acquire::Retries=3 install -y "$@"; then
return 0
fi
if [ "$attempt" = "3" ]; then
return 1
fi
sleep $((attempt * 15))
done
}
. tools/apt_retry.sh

if [ "${{ matrix.arch }}" = "arm64" ]; then
sudo dpkg --add-architecture arm64
source /etc/os-release
CODENAME="${VERSION_CODENAME}"
{
printf '%s\n' \
"Types: deb" \
"URIs: http://archive.ubuntu.com/ubuntu" \
"Suites: ${CODENAME} ${CODENAME}-updates ${CODENAME}-backports" \
"Components: main universe restricted multiverse" \
"Architectures: amd64" \
"Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg" \
"" \
"Types: deb" \
"URIs: http://security.ubuntu.com/ubuntu" \
"Suites: ${CODENAME}-security" \
"Components: main universe restricted multiverse" \
"Architectures: amd64" \
"Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg" \
"" \
"Types: deb" \
"URIs: http://ports.ubuntu.com/ubuntu-ports" \
"Suites: ${CODENAME} ${CODENAME}-updates ${CODENAME}-backports ${CODENAME}-security" \
"Components: main universe restricted multiverse" \
"Architectures: arm64" \
"Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg"
} | sudo tee /etc/apt/sources.list.d/ubuntu.sources >/dev/null
sed "s/@VERSION_CODENAME@/${VERSION_CODENAME}/g" tools/docker/ubuntu.sources.template \
| sudo tee /etc/apt/sources.list.d/ubuntu.sources >/dev/null
fi
apt_get_update
apt_get_install ccache ninja-build pkg-config make
Expand Down Expand Up @@ -526,33 +457,20 @@ jobs:
exit 1
}
fi
- name: Restore ccache
uses: actions/cache@v6
- name: Set up ccache
uses: ./.github/actions/setup-ccache
with:
path: ${{ env.CCACHE_DIR }}
key: ccache-linux-${{ matrix.arch }}-${{ matrix.backend }}-${{ github.run_id }}
restore-keys: |
ccache-linux-${{ matrix.arch }}-${{ matrix.backend }}-
ccache-linux-${{ matrix.arch }}-
- name: Prime ccache
run: |
mkdir -p "${CCACHE_DIR}"
ccache --max-size "${CCACHE_MAXSIZE}"
ccache --zero-stats
cache-dir: ${{ env.CCACHE_DIR }}
max-size: ${{ env.CCACHE_MAXSIZE }}
key-prefix: ccache-linux-${{ matrix.arch }}-${{ matrix.backend }}
extra-restore-keys: ccache-linux-${{ matrix.arch }}-
- name: Build
run: python3 tools/build.py linux --arch "${{ matrix.arch }}" --backend "${{ matrix.backend }}"
- name: ccache stats
- name: Report ccache stats
if: always()
run: ccache --show-stats || true
- name: Append ccache stats to summary
if: always()
run: |
{
echo "### ccache stats (linux/${{ matrix.arch }}/${{ matrix.backend }})"
echo '```'
ccache --show-stats || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
uses: ./.github/actions/ccache-stats
with:
label: linux/${{ matrix.arch }}/${{ matrix.backend }}
- name: Validate Linux backend artifacts
run: |
OUT_DIR="bin/linux/${{ matrix.arch }}"
Expand Down Expand Up @@ -695,46 +613,23 @@ jobs:
github_token: ${{ github.token }}
- name: Install build deps
run: |
apt_get_install() {
for attempt in 1 2 3; do
if DEBIAN_FRONTEND=noninteractive apt-get -o Acquire::Retries=3 install -y "$@"; then
return 0
fi
if [ "$attempt" = "3" ]; then
return 1
fi
sleep $((attempt * 15))
done
}
apt_get_install build-essential binutils ccache ninja-build pkg-config python3 python3-pip rocblas-dev hipblas-dev
. tools/apt_retry.sh
DEBIAN_FRONTEND=noninteractive apt_get_install build-essential binutils ccache ninja-build pkg-config python3 python3-pip rocblas-dev hipblas-dev
python3 -m pip install --upgrade cmake
cmake --version
- name: Restore ccache
uses: actions/cache@v6
- name: Set up ccache
uses: ./.github/actions/setup-ccache
with:
path: ${{ env.CCACHE_DIR }}
key: ccache-linux-x64-hip-${{ github.run_id }}
restore-keys: |
ccache-linux-x64-hip-
- name: Prime ccache
run: |
mkdir -p "${CCACHE_DIR}"
ccache --max-size "${CCACHE_MAXSIZE}"
ccache --zero-stats
cache-dir: ${{ env.CCACHE_DIR }}
max-size: ${{ env.CCACHE_MAXSIZE }}
key-prefix: ccache-linux-x64-hip
- name: Build
run: python3 tools/build.py linux --arch x64 --backend hip
- name: ccache stats
- name: Report ccache stats
if: always()
run: ccache --show-stats || true
- name: Append ccache stats to summary
if: always()
run: |
{
echo "### ccache stats (linux/x64/hip)"
echo '```'
ccache --show-stats || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
uses: ./.github/actions/ccache-stats
with:
label: linux/x64/hip
- name: Validate Linux HIP artifacts
run: |
OUT_DIR="bin/linux/x64"
Expand Down
Loading
Loading