feat(storage): enable the upstream EIC (eic://) remote KV connector on Ascend

[cklxx]

What

The EIC remote KV connector already exists in LMCache core (LMCache/LMCache#1930, shipped since v0.3.10): EICConnectorAdapter registers the eic:// URL scheme and EICConnector implements RemoteConnector under lmcache/v1/storage_backend/connector/. It currently cannot run on Ascend NPU.

This PR does not add a new connector. It makes the existing one importable/usable on Ascend through the lmcache_ascend import-time patch framework, plus an example config, docs, and tests that need no live EIC cluster.

Why it fails on NPU today

EICConnector.__init__ eagerly loads CUDA at construction time (LMCache v0.4.4, eic_connector.py:135-145):

self.cuda_lib = ctypes.CDLL("libcudart.so")
self.cuda_lib.cudaMemcpy.argtypes = [...]
self.cuda_lib.cudaMemcpy.restype = self.cudaError_t

CANN images do not ship libcudart, so merely selecting/importing the connector raises OSError before the eic:// adapter can be used. cudaMemcpy is only called on the CUDA GDR receive path; the NPU deployment uses RDMA (eic_trans_type: 2) and never enters that path.

Changes

• lmcache_ascend/v1/storage_backend/connector/eic_npu.py: an import-time patch with two modes selected by a core capability marker. Core PR fix(storage/eic): portable init, correct batched get/put and multi-group ptr for EIC connector LMCache#5141 already loads libcudart defensively (cuda_lib stays None when the library is absent, RDMA works, only explicit TRANSPORT_GDR is rejected) and marks _LMCACHE_EIC_CUDART_OPTIONAL; against that core this patch is a no-op, because installing a truthy CDLL stand-in would make core take its load-success branch and defeat the new cuda_lib is None GDR guard. Against older core (including the v0.4.4 baseline pinned here) that still loads libcudart eagerly, it installs a guarded ctypes proxy: when the library is present behavior is unchanged; when absent CDLL("libcudart.so") yields a shim whose cudaMemcpy takes the existing argtypes/restype binding and raises "not supported on Ascend NPU" only if the GDR path is ever entered. The proxy is installed on the connector module only, never on process-global ctypes.
• Wired into lmcache_ascend/__init__.py alongside the other storage-backend patches. It is a no-op when the connector module or the vendor eic package is absent, so non-EIC deployments are unaffected.
• examples/eic/lmcache-eic-config.yaml: RDMA-oriented config with the existing eic_* keys.
• docs/eic_remote_backend.md: usage and constraints.
• tests/v1/storage_backend/test_eic_npu.py: exercises the package boundary with throwaway dummy connector modules and function-scoped monkeypatch (no reload of the real vendor-bound module, no global sys.modules/base-class/env leakage). Covers the no-op against marker-carrying core, proxy application + idempotency on legacy core, the shim being confined to the cudart lookup, and a genuine ImportError skip when the connector/vendor package is absent. The dummy is bound on both sys.modules and the parent-package attribute, because import a.b.c as x resolves to a bound parent attribute first.

No connector is forked and no config keys change; selection still happens from remote_url: eic://... via core adapter discovery.

Validation

• Tests run against LMCache v0.4.4 (the version pinned on main), 4/4 passing on both x86_64 and an aarch64 Linux host without CUDA/CANN libraries.
• On the aarch64 host I also confirmed the exact failure/fix directly: unpatched __init__ raises OSError: libcudart.so; patched init succeeds with the shim installed; an accidental GDR call raises the explicit error; other system libraries load normally.
• End-to-end data-plane behavior with a real Atlas + EIC cluster is deployment-specific and is not part of this CI.

Compatibility

• Baseline: LMCache v0.4.4 pinned by this repository (legacy shim path). On core that includes #5141 the patch self-disables and core's own defensive load plus GDR guard take over.
• The eic client is not on public PyPI and ships in the vendor image, as documented in core; tests skip/stub when it is absent.
• CUDA deployments are unchanged: the guarded proxy prefers the real library whenever it loads, and the post-#5141 path leaves core entirely untouched.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[matthewygf]

I found two merge blockers in the current head (06a126e).

1. High — the supplied config requests approximately 477 EiB of local memory, so the documented deployment cannot start. The example sets max_local_cpu_size: 512000000000 and labels it bytes, but LMCache v0.4.4 defines this field in GB and computes int(cpu_size * 1024**3) in the local CPU backend. Parsing the example therefore produces 549755813888000000000 bytes (about 477 EiB), not 512,000,000,000 bytes. The current allocator path also warns that lazy allocation is unavailable and falls back to a full allocation. Please express the intended capacity in GB (for example, 512 for 512 GiB) and add a config regression that bounds the parsed capacity.

2. Medium — concurrent EIC construction can still raise the original missing-libcudart error. patched_init replaces the process-wide ctypes.CDLL and restores it in finally without synchronization. I reproduced this deterministically with two constructors: both enter the wrapper, constructor A loads the shim and then exits/restores the original CDLL; constructor B then reaches its CDLL("libcudart.so") call and receives OSError: missing libcudart. It can also transiently affect unrelated ctypes.CDLL callers in other threads. Please avoid mutating the global ctypes module—e.g. give the EIC module a proxy that delegates all ctypes attributes but guards only its CDLL lookup—and add a two-constructor interleaving regression.

A low-severity documentation issue is also present: from docs/eic_remote_backend.md, the example link should be ../examples/eic/lmcache-eic-config.yaml; the current ../../examples/... escapes the branch segment and is broken.

Static validation here: all changed Python compiles, the YAML parses, and git diff --check passes. Runtime pytest is unavailable in this environment. GitHub Code Quality currently reports action_required with no executed jobs, and the PR has no Atlas/EIC end-to-end result.

[cklxx]

Thanks @matthewygf — all three are addressed on the latest head (2c70b0f), alongside a coordination fix with the matching core PR LMCache/LMCache#5141.

1. High — max_local_cpu_size units (477 EiB). Fixed: the example now sets max_local_cpu_size: 512 and comments the field as GIB (LMCache multiplies by 1024**3), so it means ~512 GiB/process, not bytes.

2. Medium — concurrent construction / global ctypes swap. Re-architected. The old patched_init swapped process-wide ctypes.CDLL with a finally restore is gone. The patch now installs, once, a module-level _GuardedCtypes proxy on the connector module only; it delegates every attribute to real ctypes and intercepts only the libcudart.so lookup. There is no global mutation and no restore window, so the interleaving you described and unrelated CDLL callers cannot observe it. Added the requested concurrent regression (test_guarded_ctypes_concurrent_cudart_lookups_never_touch_global, 8 threads at a barrier) asserting every lookup gets the shim and ctypes.CDLL is real_cdll throughout.

One additional layer matters here: the core PR #5141 now loads libcudart defensively itself (cuda_lib stays None without the library, RDMA works, explicit TRANSPORT_GDR is rejected) and marks _LMCACHE_EIC_CUDART_OPTIONAL. This package detects that marker and no-ops, because a truthy CDLL stand-in would otherwise make core take its load-success branch and defeat core’s new cuda_lib is None GDR guard. The proxy is installed only on older core (incl. the v0.4.4 baseline pinned here). Tests cover both modes.

Low — doc link. Fixed to ../examples/eic/lmcache-eic-config.yaml.

Test notes: the suite is rewritten to function-scoped monkeypatch with throwaway connector dummies (the previous module-scoped reload leaked stubbed sys.modules entries into later tests), and the dummy is bound on both sys.modules and the parent-package attribute because import a.b.c as x resolves to the bound parent attribute first. Real Atlas+EIC end-to-end remains deployment-specific and is not run in this CI.