Skip to content

RR runtime build: -j N, --shader-timeout and --max-wall - #244

Draft
phantoslayer21 wants to merge 1 commit into
mainfrom
rr-f/build-jobs
Draft

phantoslayer21 wants to merge 1 commit into
mainfrom
rr-f/build-jobs

Conversation

@phantoslayer21

Copy link
Copy Markdown
Owner

Speeds up and bounds the RR runtime library build (python -m clean.rr_f.runtime.tools.build), which compiled most of its ~150 kernels one dxc process at a time and could sit on one slow shader with no limit.

  • -j N (default WG_RR_JOBS or 4): a dry pass of kernels() records every build_dx12.compile_hlsl call instead of running it (every builder computes its cached DXIL path before it compiles), then N workers compile the recorded calls into the same cache; the normal pass then finds every DXIL cached. Same calls, same paths, so the same DXIL bytes. The existing stage precompile pools get N too.
  • --shader-timeout S (default 1200): every dxc call of the build is stopped after S seconds and the kernel is named; the other kernels still compile, then the build stops with the list.
  • --max-wall S (default off): no new compile starts after S seconds; the build stops and the same command continues from the cache, so a full build can run as bounded sessions.

One file changed: clean/rr_f/runtime/tools/build.py. No kernel source changes.

Tested

  • DXIL bytes: a fresh -j 4 build against a serial build of the same commit, compared file by file (name + SHA-256): 38 of 38 kernels built so far identical (30 in the first session, 8 large exact stage kernels in a --max-wall 600 session).
  • --shader-timeout: with 300 s, the 8 largest exact stage kernels were stopped and named while the others kept compiling; they take 840-850 s each with 4 compiles at once, hence the 1200 s default.
  • --max-wall 600: the session stopped starting compiles after 600 s, finished the ones running, and reported the kernels left; the next run continued from the cache.
  • Local gates: leak scan 0 hits on the change; secrets / personal / licence / chunks pass.

Not tested

  • A complete full-library build with -j end to end (link of the runtime DLL after the prebuild): not run yet, compile slots were busy.
  • -j values other than 4.

-j N: a dry pass of kernels() records every build_dx12.compile_hlsl call (each builder computes its cached DXIL path
before compiling, so the dry pass returns it without compiling); N workers then compile the recorded calls into the
same cache, and the real pass finds every DXIL cached. Same calls and paths, so the same DXIL bytes as the serial
build (30 of 30 and later 8 more large kernels compared byte for byte). The stage precompile pools get N too.
--shader-timeout S (default 1200): every dxc call of the build is stopped after S seconds and the kernel named;
the others still compile, then the build stops with the list. --max-wall S: no new compile starts after S seconds,
the build stops and the same command continues from the cache, so a full build can run as bounded sessions.

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as draft November 15, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant