Repository navigation
V1.5.0 ofi - #12
Open
skaulintel wants to merge 9 commits into
Open
V1.5.0 ofi#12skaulintel wants to merge 9 commits into
skaulintel wants to merge 9 commits into
Conversation
Bring src/plugins/ofi, its unit test, examples and install script over
from v0.6.0_OFI and wire OFI into the 1.5 build: add it to all_plugins,
build it under enabled_plugins.get('OFI'), register the static plugin
and follow the nixl_common_deps -> nixl_common_dep rename. Fix the
static entry point name to match NIXL_REGISTER_STATIC_PLUGIN (OFI).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since NIXL 1.0 the backend wheel is nixl_cu12 and the version vLLM checks comes from the separate PyPI nixl meta wheel, so install both. Install system liburing-dev, exclude prometheus-cpp libcore/libpull from auditwheel, build without isolation, only reuse a repaired (manylinux) cached wheel, and rebuild over an older installed NIXL. Drop the nixl/*.so lookup that only fed a commented-out patchelf. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NIXL 1.5 lets nixlBackendInitParams::customParams be null and copies it into the engine (as an empty map when null), so read from there instead of dereferencing the raw pointer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Installing liburing-dev for NIXL 1.5 made libfabric's configure enable io_uring. The libfabric copied into the wheel after auditwheel is not repaired, so liburing.so.2 was missing on any image that installed the cached wheel and the OFI plugin failed to load. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
modi_mem_addr was only set when the buffer was not page-aligned, so an aligned HPU tensor registered with an uninitialized dmabuf base_addr and fi_mr_regattr failed with EINVAL. Also on v0.6.0_OFI (since e1fc5a9). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
postXfer returned NIXL_SUCCESS right after posting. The agent treats that as a finished transfer and never calls checkXfer, so a transfer read as DONE before its completions arrived and releaseReqH found hundreds of operations still pending. Return NIXL_IN_PROG while any are outstanding. Also on v0.6.0_OFI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prepXfer called postXfer, so every transfer was posted at prep time and again at post time. All requests share one CQ, yet checkXfer credited whatever completions it read to the request being checked, and the progress paths dropped completions without crediting anyone, so releaseReqH found operations still pending. prepXfer now only validates and allocates the handle. Each posted operation carries a context naming its request, and every CQ reader goes through one reaper that credits that request, including error completions. postXfer reports IN_PROG while operations are outstanding. A request released with operations still in flight is freed by its last completion instead of while still referenced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CQ is opened with FI_CQ_FORMAT_CONTEXT but was read into fi_cq_data_entry arrays, so every entry after the first was taken from the wrong offset. Crediting those stale contexts double-completed operations and freed requests that were still in use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
👋 Hi skaulintel! Thank you for contributing to intel-staging/nixl. Your PR reviewers will review your contribution then trigger the CI to test your changes. 🚀 |
Author
- Remove the nixlbench OFI Docker recipes and the root nixl_api_test.py, which nothing references; examples/python/ofi/nixl_api.py remains. - Drop the CUDA and ZE HMEM paths, which were never exercised: this plugin serves Gaudi, and only SynapseAI and system memory remain. - Build the plugin entry with nixlBackendPluginCreator like the other 1.5 plugins, and drop the 'provider' option the engine never read (it takes FI_PROVIDER from the environment). - Remove the unused getStringParam/getSizeTParam helpers. - Check out UCX_BRANCH instead of repeating its value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Forward-ports the OFI plugin (Gaudi/SynapseAI over libfabric,
VLLM_NIXL_BACKEND=OFI) fromv0.6.0_OFIto NIXLv1.5.0. vLLM now requires NIXL >= 1.5.0 (vllm-project/vllm#61032), sovllm-gaudi disabled its libfabric PD test (vllm-project/vllm-gaudi#1900); this lets it be re-enabled.
Changes
install_nixl.py; register OFI in the 1.5 build.install_nixl.py: 1.5 packaging (nixl_cu12+nixlmeta wheel), and libfabric built--with-uring=noso the cached wheel loads on a clean image.v0.6.0_OFI:NIXL 1.5 strided descriptors need no plugin change: the core expands them before
prepXfer.Testing (2x Gaudi3, verbs over RoCE)
check_nixl_version("1.5.0").Known limitation
OFI notifications are still no-op stubs (as in 0.6), so vLLM's prefill frees KV only on timeout.
🤖 Generated with Claude Code