Add docker image to use hermetic llvm and sysroot#782
Open
hsharsha wants to merge 8 commits into
Open
Conversation
Gate every caller workflow job by `github.repository_owner` so that forks only run the CI subset they have infrastructure for. Non-ROCm jobs: `github.repository_owner == 'jax-ml'` ROCm jobs: `github.repository_owner == 'jax-ml' || github.repository_owner == 'ROCm'`
job naming, and nightly build workflow Version-tier S3 artifact layout: - Add rocm-release-version tier to S3 artifact paths to prevent wheel collisions across ROCm versions. - Add per-repo/branch/version LATEST pointer for wheel resolution. - Parameterize manylinux container image in build_rocm_artifacts.yml. - Plumb s3_download_uri through bazel_rocm.yml. Standardize CI job naming and matrix schema: - Unify ROCm matrix entries across all caller workflows to use a canonical schema with label, wheel-version, release-version, and tag fields. - Update job names to use matrix.rocm.label. - Add rocm-release-version input to pytest_rocm, bazel_rocm, and download-jax-rocm-wheels to decouple wheel filename matching from S3 path construction. - Add manylinux-image input to build_rocm_artifacts so callers can override the container image. Nightly workflow: - Add a dedicated build-rocm-artifacts job to the nightly workflow that builds wheels with pinned XLA (clone_main_xla: 0) and uploads to a nightly-specific S3 path. - Update nightly pytest and bazel ROCm jobs to depend on the new build.
Extends the wheel test continuous and nightly release workflows to run against TheRock 7.13.0 and TheRock latest builds in addition to ROCm 7.2.0.
hsharsha
force-pushed
the
hhs/add_docker_for_jax_xla
branch
from
5ab7632 to
80c6d34
Compare
hsharsha
requested review from
JehandadKhan,
alekstheod,
charleshofer,
i-chaochen,
leo-automation,
mminutoli and
tsrw2048
Author
|
@mminutoli should we keep docker files here now that rocm-jax would be depricated. Or do we still want to keep all the infra related files there? |
Comment on lines
+56
to
+57
| run: | | ||
| working-directory: ./build/rocm/docker/ |
There was a problem hiding this comment.
Probably a typo, it needs to be its own line. As it is now, the script will fail
| # - jammy | ||
| # - el7 | ||
| # - el8 | ||
| set -x |
There was a problem hiding this comment.
Suggested change
| set -x | |
| set -euo pipefail | |
| set -x |
|
|
||
| jobs: | ||
| build-docker: | ||
| runs-on: ${{ fromJSON(inputs.runner-label) }} |
There was a problem hiding this comment.
Suggested change
| runs-on: ${{ fromJSON(inputs.runner-label) }} | |
| runs-on: ${{ inputs.runner-label }} |
| required: false | ||
| default: 'plugin_wheels' | ||
| type: string | ||
| extra-cr-tag: |
| df -h | ||
| rocm-smi -a || true | ||
| rocminfo | grep gfx || true | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
We should be using v6+ on actions/checkou or pinned hashes
| rocminfo | grep gfx || true | ||
| - uses: actions/checkout@v4 | ||
| - name: Download wheel artifacts | ||
| uses: actions/download-artifact@v4 |
There was a problem hiding this comment.
Same. Needs a relevant version and pinned hash
| ROCM_DEB_REPO=${ROCM_DEB_REPO_HOME}${ROCM_VERS}/ | ||
| AMDGPU_DEB_REPO=${AMDGPU_DEB_REPO_HOME}${AMDGPU_REPO_VERS}/ | ||
|
|
||
| DEBIAN_FRONTEND=noninteractive apt-get --allow-unauthenticated update |
There was a problem hiding this comment.
git diff --check says there's a trailing whitspace
|
do we need to put this to jax upstream? not sure what's sync policy in rocm/jax? |
leo-automation
requested changes
Jun 1, 2026
hsharsha
changed the title
|
We keep our infra in another repository. So maybe we should close this PR and open one there. Nevertheless, the right approach seems to have hermetic root in the build, right @alekstheod ? |
mminutoli
requested changes
Jun 1, 2026
mminutoli
left a comment
mminutoli
left a comment
There was a problem hiding this comment.
As said above, this should be closed IMHO.
psanal35
force-pushed
the
amd-main
branch
2 times, most recently
from
729ed31 to
88e42dc
Compare
mminutoli
force-pushed
the
amd-main
branch
2 times, most recently
from
cb097f7 to
d347a4f
Compare
mminutoli
force-pushed
the
amd-main
branch
2 times, most recently
from
3f9e8b4 to
f25f2b4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add docker image to build and test JAX and XLA using hermetic llvm and sysroot based on ub22 image used for ml-build images