Repository navigation
Builds: the official HIP SDK only, never one from the environment - #239
phantoslayer21 wants to merge 4 commits into
Conversation
- clean/runner/sdk.py: the SDK of every build is AMD's official HIP SDK folder; --rocm may name only that folder (another one needs --force-sdk); a banner names the SDK, compiler and device-library folders in every build log - build.py / k_build.py: --rocm defaults to the official SDK instead of ROCM_PATH; find_clang never falls back to PATH, find_host_cxx not on Windows; children get --force-sdk when the parent had it; a release runtime build still reuses pinned Model L code objects unless --rocm is given - every other script and test that defaulted to ROCM_PATH (or WG_K_TEST_ROCM) uses the official SDK; hip.py's default comes from sdk.py and its LLVM_PATH warning no longer prints the folder - clean/tests/test_sdk.py: the default ignores ROCM_PATH / HIP_PATH / LLVM_PATH / HIP_DEVICE_LIB_PATH, another --rocm is refused without --force-sdk, clang is never taken from PATH, the banner, build.py's device compiles use the official device libraries with ROCM_PATH set elsewhere, and no Python file of clean/ reads ROCM_PATH as a default - BUILDING.md: the SDK rule Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
…d (review fixes) - sdk.py: official installs discovered by version (C:\Program Files\AMD\ROCm\<version>, /opt/rocm-<version>, /opt/rocm), the newest compatible 7.x by default; the compiler canonicalised to the SDK's own (bare and absolute names), anything else approved only with --force-sdk; check() is the boundary; runtime_library() the only HIP runtime source; custom folders logged by id only (banner, refusal, command echo via redact) - build.py: device_command calls sdk.check (every entry point: post_ablate, l_c32_check, k_port_check, fg_d3d12_check, ... pass it); find_clang / find_host_cxx through sdk.compiler; --clang validated; run() redacts - k_build.py: the compiler through sdk.compiler - hip.py: the runtime loaded only from the checked SDK by full path (no bare-name fallback); Hip(rocm_dir) under the same policy; fails closed - test_sdk.py: discovery (newest by numeric version, 7.2 missing, none), compilers, the boundary and the four entry points, build.py / k_build.py refusals, the runtime loader failing closed, redaction, every build.py compile with the SDK's clang and flags (build completes), the SDK-variable scan with negative controls (indexed, getenv, aliases, other variables) and a failing git call failing the test Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
…d compiler output (re-check fixes) - every CLI with --rocm offers --force-sdk and resolves --rocm (and --clang) through the SDK policy right after parsing (sdk.add_force_argument / sdk.apply); children that get --rocm also get --force-sdk - Linux: the newest /opt/rocm-<version> wins; /opt/rocm counts with the version it points to; a standalone /opt/rocm counts with its .info/version (ranked last without one) - build.run / run_post_w64 / post_ablate / post.build_wave64: command echo, compiler output and the post_w64 report go through sdk.redact; redaction matches relative, resolved, 8.3 and home-folder spellings, either separator, any case - test_build_cuid approves its dummy SDK; new tests for all of the above - the native runtime's and installer's HIP runtime lookup stay out of scope (BUILDING.md, issue draft for 0.6.6) Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
8f79f0e to
0a3fded
Compare
…s no HIP SDK (re-check 2) - clean/dx12/gelu_lut/run.py, f16lut/run.py, f16lut/run_in.py: --rocm / --force-sdk through sdk.resolve, the SDK's clang from sdk.compiler, sdk.check before the compile, logs redacted (no fixed 7.2 path) - build.py and k_build.py resolve the HIP SDK only when a device build needs it: --emu-only builds the host emulator with the host compiler alone, also with no SDK installed - tests: emu-only builds without an SDK (and the device build still refused); a scan that every file running the HIP compiler itself goes through the policy Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
|
Some notes from building WrongGPU's HIP side on Linux, in case they help with the SDK policy here. This is about developers who build the HIP kernels or use the HIP diagnostic executor; since #225 players need no ROCm at all. What Linux developers have today (checked 2026-11-17; corrections welcome):
So for now most Linux setups are on a distro 7.1-7.2 with distro paths, and 10.x is only available as AMD's relocatable tarball. With this PR as drafted (only ROCm 10.1 (AMD clang 24) with current main: Happy to help test a policy change on Fedora and with a relocated ROCm 10.1. |
Milestone 0.6.6 (not merged before the 0.6.5 release). Draft. Every HIP build and every HIP runtime load of the Python tools uses an official AMD HIP SDK install, checked in one place. Nothing is taken from the environment or from
PATH.Out of scope: the native runtime's (
clean/runtime/device.cpp) and the installer's (clean/installer/src/wg_core/checks.cpp) own HIP runtime lookup. Changing it changes product behaviour, so it has its own issue for 0.6.6, #242: System32 first, then an official SDK by full path, no bare-name search,HIP_PATHonly when it resolves to an official install, and the installer probe under the same rules.Before this change,
build.py,k_build.pyand about 30 other scripts and tests usedROCM_PATHas their default--rocm.find_clangandfind_host_cxxfell back toPATH, andhip.pyfell back to a bare runtime library name that the loader searches for.The policy (
clean/runner/sdk.py)C:\Program Files\AMD\ROCm\<version>on Windows,/opt/rocm-<version>or/opt/rocmelsewhere, with a compatible 7.x version. Without--rocm, the newest one in numeric version order./opt/rocmcounts with the version it points to (so a newer/opt/rocm-<version>still wins); a standalone/opt/rocmcounts with its.info/version, or ranks last without one. NeverROCM_PATH,HIP_PATH,LLVM_PATH,HIP_DEVICE_LIB_PATH,HIP_CLANG_PATHorPATH.--clang clangmeans the SDK'sbin\clang.exe; an absolute path must lie inside the SDK.--force-sdk, which approves exactly that folder or file for the process. Every script with--rocmhas it (sdk.add_force_argument/sdk.apply:--rocmand--clangare resolved right after parsing), and passes it on to the child processes it gives--rocm. The LUT oracle builders that run the HIP compiler themselves (clean/dx12/gelu_lut/run.py,clean/dx12/f16lut/run.py,clean/dx12/f16lut/run_in.py) take--rocm/--force-sdkthe same way, the SDK's clang fromsdk.compiler, and callsdk.checkbefore compiling (no fixed 7.2 path).build.py --emu-onlyandk_build.py --emu-onlybuild the CPU emulator with the host compiler alone; the HIP SDK is resolved only when a device build (or the runtime library) needs it.device_commandcallssdk.check(), so every HIP compile passes it, whichever entry point started it (build.py,k_build.py,post_ablate.py,l_c32_check.py,k_port_check.py,fg_d3d12_check.py, ...).find_clangandfind_host_cxxgo throughsdk.compiler().hip.pyloads the HIP runtime library only from the checked SDK, by full path, with no bare-name fallback, and fails closed.Hip(rocm_dir)is under the same policy.build.run,run_post_w64and its report file,post_ablate,post.build_wave64). Redaction matches the path as given (also relative), absolute, resolved, the 8.3 short form and the~form, with either separator and any case.ROCM_PATH(orWG_K_TEST_ROCM) uses the official SDK.BUILDING.mdstates the rule.Tests (
clean/tests/test_sdk.py, CPU, fake SDK folders, no compiler started)--force-sdkand accepted with it.device_commandandfind_clangrefuse an unapproved SDK or compiler. The entry pointspost_ablate.build_all,l_c32_check.compile_kernels,k_port_check.prepareandfg_d3d12_check.preparerefuse a custom SDK (also with an explicit compiler), andbuild.py/k_build.pyrefuse a custom--rocmor--clangwithout--force-sdk. No compiler is started in any of these./opt/rocmpointing to 7.1 next torocm-7.2(7.2 wins; the link carries 7.1), pointing to the newest, a real symlink (skipped where the system has none), standalone with and without.info/version, standalone 6.x refused.--force-sdk:post_ablate.py --rocm <custom>is refused without it and reaches the build with it. A scan of every tracked CLI with--rocm(61 files) checks that each one offers--force-sdkand resolves--rocmthrough the policy.--emu-only:build.pyandk_build.pybuild the emulator with no SDK installed (only the host compiler runs); without--emu-onlythey still refuse.--offload-arch) goes throughsdk.resolve,sdk.compilerandsdk.check(build.py:sdk.checkindevice_command), with no fixed SDK path.build.py,k_build.pyandgelu_lut/run.pyback).run/run_post_w64compiler output and report (failing and passing), orpost_ablate's failure, in any of the spellings above; other names next to it stay untouched.--rocm-pathand--rocm-device-lib-path, every Model M source is compiled, and the build completes (no exception swallowed).clean/ortools/reads an SDK variable byos.environ[...],os.getenv,.get()or an alias. The scanner has negative-control fixtures; a failinggit ls-filesfails the test; fewer than 200 files scanned fails it.test_build_cuid.pyapproves its dummy SDK for each test (plus a check that the boundary refuses it without that), andtest_build_motion_define.pypasses--force-sdk.Whole CPU suite (
python -m unittest discover -s clean/tests -t ., at this head, SDK variables removed from the environment): 279 tests run, 213 passed, 66 skipped, 0 failures or errors. The skips: 61 need Linux (lane emulator), 2 need clang++/g++, 2 needWG_NO_K_BUILD, 1 is the GPU smoke test. Every script with--force-sdkalso starts and lists it in--help(61 files).Provenance of the shipped HIP code objects (checked for this PR)
l_post.co: identical.k_block.co: identical.swin.co: the code is identical. Only the compilation-unit id symbol (__hip_cuid_*) differs, because 0.5.5-rc1'sbuild.pydid not yet pass a fixed-cuid.Not tested