FEAT: Cache numeric Python UDF kernels - #24025
Draft
vyasr wants to merge 3 commits into
Draft
Conversation
Contributor
Author
|
/ok to test |
@vyasr, there was an error processing your request: See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/1/ |
vyasr
added
improvement
vyasr
force-pushed
the
codex/cache-udf-kernels
branch
from
September 7, 2026 19:05
ef8146f to
2ef4032
Compare
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Contributor
Author
|
/ok to test |
vyasr
force-pushed
the
codex/cache-udf-kernels
branch
from
September 7, 2026 19:08
2ef4032 to
42f23b5
Compare
Contributor
Author
|
/ok to test |
vyasr
force-pushed
the
codex/cache-udf-kernels
branch
from
September 7, 2026 19:48
42f23b5 to
1ee5aa9
Compare
Contributor
Author
|
/ok to test |
1 similar comment
Contributor
Author
|
/ok to test |
vyasr
force-pushed
the
codex/cache-udf-kernels
branch
from
September 7, 2026 20:46
1ee5aa9 to
f784670
Compare
Contributor
Author
|
/ok to test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Enable Numba's native on-disk CUDA cache for numeric
Series.applyandDataFrame.applykernels, including scalar arguments. The generated numeric kernels retain a persistent source location and carry their UDF identity in the cache key, allowing specializations to be reused by fresh Python processes.String, NRT-using, and GroupBy UDF paths retain the existing linked-shim implementation because Numba cannot serialize kernels that link
UDF_SHIM_FILE.Persist the cache for the Conda and wheel cuDF Python-test jobs. The GitHub Actions key is scoped to CUDA, Python, architecture, Linux, GPU, and a hash of the cuDF Python sources/tests plus
dependencies.yaml; a compatible previous cache is restored to carry unchanged specializations forward.This temporarily pins the reusable workflow revision from rapidsai/shared-workflows#629. Once that dependency merges, the references can return to
@main.Testing
pytest -n 8 --dist=worksteal python/cudf/cudf/tests/series/methods/test_apply.py(94 passed, 22 xfailed)pytest -n 8 --dist=worksteal python/cudf/cudf/tests/dataframe/methods/test_apply.py -k test_apply_return_literal(3 passed)actionlintfor the shared-workflows changes; cuDF's existing unrelated custom-runner warnings remainChecklist