Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/actions/restore-ccache/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Restore ccache
description: >
Restores the ccache that save-ccache stored under the key, or else the newest one stored under a key that starts with the key and a `_`
(the `_` is there because `macos-15` is a prefix of `macos-15-intel`). The cache of a name is the one of its last run on main.

inputs:
key:
description: Cache key, the same one given to save-ccache
required: true
path:
description: Directory of the ccache
default: .ccache

runs:
using: composite
steps:
- name: Restore the cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ${{ inputs.path }}
key: ${{ inputs.key }}
restore-keys: |
${{ inputs.key }}_
11 changes: 8 additions & 3 deletions .github/actions/save-ccache/action.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Save ccache
description: >
Replaces the cache stored under the key instead of adding an entry per commit, so the
Replaces the cache stored under the key, for the ref that is running, instead of adding an entry per commit, so the
caches of the repository stay below its quota. The job needs the actions: write permission.

inputs:
Expand All @@ -19,8 +19,13 @@ runs:
env:
GH_TOKEN: ${{ github.token }}
CACHE_KEY: ${{ inputs.key }}
# cache keys are immutable, there is nothing to delete the first time
run: gh cache delete "$CACHE_KEY" --repo "$GITHUB_REPOSITORY" || true
# Only the entry of the ref that is running: the API deletes the entries of that key of every ref when it is not given one (`gh cache delete <key>` does not give one), and a run of a
# tag or of a branch would delete the one of main, which is the one that the runs of main and of the pull requests restore, to
# leave one that they can not read. Cache keys are immutable, there is nothing to delete the first time
run: |
ref=$(printf '%s' "$GITHUB_REF" | jq -sRr @uri)
key=$(printf '%s' "$CACHE_KEY" | jq -sRr @uri)
gh api --method DELETE "repos/$GITHUB_REPOSITORY/actions/caches?key=$key&ref=$ref" > /dev/null || true

- name: Save the cache
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
Expand Down
1 change: 1 addition & 0 deletions .github/python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.13
34 changes: 23 additions & 11 deletions .github/workflows/build-deb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,29 +62,38 @@ jobs:
persist-credentials: false
submodules: recursive

# A tag is a release and its packages have the version of pyproject.toml alone. The others are snapshots of a commit, which have
# ~git<commit> after it: it sorts before the release, and the name of the package says what it was built from. The version of the
# module, in pyproject.toml, does not change, so the build does not either and ccache keeps hitting
- name: Determine package version
id: version
env:
# the commit of the pull request, github.sha is the one of its merge with main
COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
REF_TYPE: ${{ github.ref_type }}
run: |
VERSION=$(sed -n 's/^version = "\(.*\)"$/\1/p' pyproject.toml | head -n 1)
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
SNAPSHOT=""
if [ "$REF_TYPE" != tag ]; then
SNAPSHOT="git${COMMIT::7}"
fi
echo "version=${VERSION}${SNAPSHOT:+~$SNAPSHOT}" >> "$GITHUB_OUTPUT"
echo "snapshot=${SNAPSHOT}" >> "$GITHUB_OUTPUT"

- name: Free up disk space
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc || true
docker system prune -af || true

- name: Restore ccache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
uses: ./.github/actions/restore-ccache
with:
path: .ccache
key: ccache-deb-${{ matrix.name }}-${{ matrix.arch }}
restore-keys: |
ccache-deb-${{ matrix.name }}-${{ matrix.arch }}_

# Installs the Build-Depends of debian/control in the container, sets the version of
# debian/changelog from pyproject.toml and runs dpkg-buildpackage. The workspace is
# mounted at /github/workspace in the container.
- name: Build package
- name: Build the deb package (dpkg-buildpackage)
uses: jtdor/build-deb-action@473136f14a42a231124c1f7cefaedfe941203c38 # v1.9.1
env:
CCACHE_DIR: /github/workspace/.ccache
Expand All @@ -94,26 +103,29 @@ jobs:
# would be a job of its own
docker-image: ${{ fromJSON('{"noble":"ubuntu:24.04","jammy":"ubuntu:22.04","trixie":"debian:trixie-slim","bookworm":"debian:bookworm-slim"}')[matrix.name] }}
extra-docker-args: --platform ${{ matrix.arch == 'armhf' && 'linux/arm/v7' || format('linux/{0}', matrix.arch) }}
before-build-hook: sh debian/set-version.sh ${{ matrix.name }}
before-build-hook: sh debian/set-version.sh ${{ matrix.name }} ${{ steps.version.outputs.snapshot }}
buildpackage-opts: --build=binary --no-sign --jobs=auto

- name: Save ccache
# Pull requests only read the cache, pushes to main and manual runs replace it.
if: ${{ !cancelled() && github.event_name != 'pull_request' }}
# Pull requests only read the cache, pushes to main and manual runs replace it. A tag does not save: what it would save is only
# readable by that tag, and the release builds all the legs from the caches of main
if: ${{ !cancelled() && github.event_name != 'pull_request' && github.ref_type != 'tag' }}
uses: ./.github/actions/save-ccache
with:
key: ccache-deb-${{ matrix.name }}-${{ matrix.arch }}

- name: Upload monero-python package
- name: Upload the deb package
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: python3-monero_${{ steps.version.outputs.version }}-1${{ matrix.name }}1_${{ matrix.arch }}
# the packages of the release are attached to its release, and the ones of main and of the pull requests are only looked at for a while
retention-days: 14
path: debian/artifacts/python3-monero_*.deb

# the segments and the exports of the module in the package, and the stack protector and the branch protection in its code. The
# module of the armhf package is read but not disassembled, the objdump of the arm64 runner does not read 32 bit arm.
# After the upload, so that a package that fails can be downloaded and looked at. The release needs this job to succeed
- name: Verify libraries
- name: Check the hardening of the module in the package
run: |
dpkg-deb -x debian/artifacts/python3-monero_*.deb deb-check
bin/check_elf_hardening.sh deb-check/usr/lib/python3/dist-packages/monero*.so
41 changes: 21 additions & 20 deletions .github/workflows/build-linux.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Build Linux Module
name: Build Linux

# Called by build.yml. It builds monero-project, monero-cpp and the module, the last two with the coverage on, and uploads the module
# with the files that the coverage needs. The tests of build.yml run on it, in several jobs. The packages of Linux are the deb ones,
# Called by ci.yml. It builds monero-project, monero-cpp and the module, the last two with the coverage on, and uploads the module
# with the files that the coverage needs. The tests of ci.yml run on it, in several jobs. The packages of Linux are the deb ones,
# which build-deb.yml makes.
on:
workflow_call:
Expand All @@ -26,12 +26,16 @@ jobs:
persist-credentials: false
submodules: recursive

- name: Install dependencies
run: |
bin/install_dependencies_linux.sh
pip3 install -r requirements-build.txt --break-system-packages
- name: Install apt packages
run: bin/install_dependencies_linux.sh

- name: Show the versions of the apt libraries
run: bin/show_versions_linux.sh | bin/job_summary.sh "apt libraries"

- name: Install the Python build requirements (requirements-build.txt)
run: pip3 install -r requirements-build.txt --break-system-packages

- name: Install expat
- name: Install expat 2.7.1 (static)
run: |
wget https://github.com/libexpat/libexpat/releases/download/R_2_7_1/expat-2.7.1.tar.bz2
echo "45c98ae1e9b5127325d25186cf8c511fa814078e9efeae7987a574b482b79b3d expat-2.7.1.tar.bz2" | sha256sum -c
Expand All @@ -42,7 +46,7 @@ jobs:
sudo make install
cd ../

- name: Install unbound
- name: Install unbound 1.22.0 (static)
run: |
wget https://www.nlnetlabs.nl/downloads/unbound/unbound-1.22.0.tar.gz
echo "c5dd1bdef5d5685b2cedb749158dd152c52d44f65529a34ac15cd88d4b1b3d43 unbound-1.22.0.tar.gz" | sha256sum -c
Expand All @@ -55,17 +59,14 @@ jobs:
cd ../

- name: Restore ccache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
uses: ./.github/actions/restore-ccache
with:
path: .ccache
key: ccache-linux-test
restore-keys: |
ccache-linux-test_

- name: Reset ccache statistics
run: ccache --zero-stats

- name: Build monero-project
- name: Build monero-project (static, Release)
run: |
cd external/monero-cpp/external/monero-project
mkdir -p build/release
Expand All @@ -77,7 +78,7 @@ jobs:
make -j3 wallet cryptonote_protocol
cd ../../../../../../

- name: Build monero-cpp libraries
- name: Build monero-cpp (with coverage)
run: |
cd external/monero-cpp
mkdir -p build
Expand All @@ -87,7 +88,7 @@ jobs:
make -j3
cd ../../../

- name: Build monero-python module
- name: Build the monero-python module (with coverage)
run: |
mkdir -p build
cmake -B build -DENABLE_COVERAGE=ON -Dpybind11_DIR="$(python3 -m pybind11 --cmakedir)"
Expand All @@ -103,7 +104,7 @@ jobs:

- name: Show ccache statistics
if: ${{ !cancelled() }}
run: ccache --show-stats
run: ccache --show-stats | bin/job_summary.sh "ccache"

- name: Save ccache
# Pull requests only read the cache, pushes to main and manual runs replace it.
Expand All @@ -113,17 +114,17 @@ jobs:
key: ccache-linux-test

# The module, and the .gcno that lcov reads together with the .gcda that the tests write: the ones of monero-cpp and of the module
# are the only ones, monero-project is not built with the coverage. The tests of build.yml extract it at the root of the
# are the only ones, monero-project is not built with the coverage. The tests of ci.yml extract it at the root of the
# workspace, where the paths of the .gcda are the ones that the build had.
- name: Package the build for the tests
run: |
find build external/monero-cpp/build \( -name 'monero*.so' -o -name '*.gcno' \) -print0 | tar --null -czf linux-build.tar.gz -T -
ls -l linux-build.tar.gz

- name: Upload monero-python build
- name: Upload the Linux build
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: monero-python-linux-x86_64
name: build-linux-x86_64
path: linux-build.tar.gz
# already compressed
compression-level: 0
Expand Down
53 changes: 34 additions & 19 deletions .github/workflows/build-macos.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Build MacOS
name: Build macOS

# Called by build.yml, which has the triggers and the concurrency group: a group here would be the one of the caller, and the
# Called by ci.yml, which has the triggers and the concurrency group: a group here would be the one of the caller, and the
# jobs of the same run would cancel each other.
on:
workflow_call:
Expand Down Expand Up @@ -35,40 +35,54 @@ jobs:
persist-credentials: false
submodules: recursive

- name: Install dependencies
- name: Install brew formulae (boost@1.85, openssl@3 and the other libraries)
run: |
HOMEBREW_NO_AUTO_UPDATE=1 brew install python boost@1.85 hidapi openssl@3 zmq libpgm miniupnpc expat libunwind-headers protobuf ccache
brew unlink boost || true
brew link boost@1.85 --force
# the image of the runner can have another boost linked, which gets in the way of boost@1.85. They are touched only when
# they are there, brew prints an error for a formula that is not installed (the annotations of the run show it as one)
if brew list --versions boost > /dev/null 2>&1; then
brew unlink boost
brew link boost@1.85 --force
fi
# brew links openssl@4 into /opt/homebrew, and the -I/opt/homebrew/include of boost and zeromq comes
# before the one of openssl@3, so its headers win over the ones of OPENSSL_ROOT_DIR
brew unlink openssl@4 || true
if brew list --versions openssl@4 > /dev/null 2>&1; then
brew unlink openssl@4
fi
# brew's openssl is OpenSSL 4 on the newer runner images, which boost 1.85 does not build with.
# monero-project and monero-cpp take brew's openssl unless they are given another one, so
# they get it on the command line, and the module finds it in the environment
echo "OPENSSL_ROOT_DIR=$(brew --prefix openssl@3)" >> "$GITHUB_ENV"

- name: Show the versions of the brew formulae
run: |
{
brew list --versions python boost@1.85 hidapi openssl@3 zmq libpgm miniupnpc expat libunwind-headers protobuf ccache
cmake --version | head -n 1
clang --version | head -n 1
} | bin/job_summary.sh "brew formulae"

- name: Create the Python virtual environment and install the build requirements (requirements-build.txt)
run: |
# a virtual environment: the python of brew has packages that pip can not replace, such as wheel and typing_extensions on the
# Intel runner (they have no RECORD file), and the versions of the requirements files are pinned. The next steps find it first in the PATH
python3 -m venv "$RUNNER_TEMP/venv"
echo "$RUNNER_TEMP/venv/bin" >> "$GITHUB_PATH"
"$RUNNER_TEMP/venv/bin/pip" install -r requirements-build.txt

# static, against openssl@3: the unbound of brew is built with OpenSSL 4
- name: Install unbound (static)
- name: Install unbound 1.22.0 (static)
run: bin/install_unbound_macos.sh

- name: Restore ccache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
uses: ./.github/actions/restore-ccache
with:
path: .ccache
key: ccache-${{ matrix.os }}
restore-keys: |
ccache-${{ matrix.os }}_

- name: Reset ccache statistics
run: ccache --zero-stats

- name: Build monero-project
- name: Build monero-project (static, Release)
run: |
cd external/monero-cpp/external/monero-project
mkdir -p build/release
Expand All @@ -81,7 +95,7 @@ jobs:
make -j3 wallet cryptonote_protocol
cd ../../../../../../

- name: Build monero-cpp libraries
- name: Build monero-cpp
run: |
cd external/monero-cpp
mkdir -p build
Expand All @@ -91,14 +105,14 @@ jobs:
make -j3
cd ../../../

- name: Build monero-python module
- name: Build the monero-python wheel
run: |
mkdir -p build
python3 -m pip wheel . -w dist --no-build-isolation

- name: Show ccache statistics
if: ${{ !cancelled() }}
run: ccache --show-stats
run: ccache --show-stats | bin/job_summary.sh "ccache"

- name: Save ccache
# Pull requests only read the cache, pushes to main and manual runs replace it.
Expand All @@ -107,18 +121,19 @@ jobs:
with:
key: ccache-${{ matrix.os }}

- name: Repair wheel with delocate
- name: Repair the wheel with delocate
run: |
delocate-wheel -w wheelhouse -v dist/*.whl

# the canary and the exports of the module in the wheel that is published
- name: Verify libraries
- name: Check the hardening of the module in the wheel
run: |
python3 -m zipfile -e wheelhouse/*.whl wheel-check
bin/check_macho_hardening.sh wheel-check/monero*.so

- name: Upload monero-python wheel
- name: Upload the macOS wheel
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: monero-${{ matrix.os }}
name: wheel-${{ matrix.arch }}
# the default retention: the release downloads it from the run of main, and a tag can come days after the commit
path: wheelhouse/*.whl
Loading
Loading