docs(adrs): close cross-process VRAM sharing with ComfyUI (ADR-0011) - #182
Open
laurigates wants to merge 2 commits into
Open
docs(adrs): close cross-process VRAM sharing with ComfyUI (ADR-0011)#182laurigates wants to merge 2 commits into
laurigates wants to merge 2 commits into
Conversation
A proposal to have loractl train against Krea 2 weights already resident in ComfyUI's VRAM — exported by `shared-tensor` over a UDS and imported with `cudaIpcOpenMemHandle` — was spiked at its most-likely-to-fail rung first: can burn/cubecl adopt a device pointer it did not allocate? It cannot, and this records the mechanism at pinned versions rather than asserting it. cubecl 0.10's `ComputeStorage` (storage/base.rs:74-95) admits memory only through `alloc(size: u64)`; `GpuStorage`'s pointer map is private (gpu.rs:19) and its sole insertion sits inside `alloc` after an unconditional `malloc_async`/`malloc_sync`; `AllocationKind` has no non-owning variant; and a hand-built `StorageHandle` panics in `get` (gpu.rs:148-152). A grep for every adoption-shaped name across cubecl-runtime and cubecl-cuda returns zero hits. Had a pointer been forced in, `perform_deallocations` (gpu.rs:63-80) would free it out from under ComfyUI — silent corruption in another process. This upgrades ADR-0008's and ADR-0010's asserted "verified" to demonstrated, with quoted source. It is version-scoped: re-check on the burn 0.22 migration (#79). The budget fails independently of the mechanism. The only shareable copy is ComfyUI's ~13.1 GB scaled-fp8 base — larger than the ~10.1 GB int4 base loractl already uses — so even a perfect zero-copy import lands at ~26 GB against a 24 GB card. Co-tenancy stays a time-division problem. Reading ComfyUI's model *directory* was never blocked and remains shipped, as does the disk-based reverse channel for in-progress adapters. No probe example, cudarc dependency, or GPU hours were spent: the spike stopped at its first rung by design. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QNa5Xn2QW7fimm5XHokguK
laurigates
marked this pull request as ready for review
August 1, 2026 10:58
…n use Decision 4 cited the scaled-fp8 repack (~13.1 GB). The checkpoint in use is `Comfy-Org/Krea-2` @ 7b75ff3 `diffusion_models/krea2_turbo_int8_convrot.safetensors`. Measured from its safetensors header: 13,492,686,496 bytes, 878 tensors — 224 `I8` weights (12.16 GB) each with an F32 `weight_scale` and a U8 `comfy_quant` marker, 76 `BF16` left unquantized (1.33 GB), 354 F32 norm scales. The budget conclusion is unchanged and now rests on a measured number rather than an assumed one: ~13.5 GB shared base + loractl's ~9.3 GB non-base is already ~23 GB before ComfyUI's own encoder, VAE and generation activations. Adds Decision 5, found while measuring the above and the only actionable item in the ADR — unrelated to IPC. The int8 repack carries no `F8_E4M3`, so `is_fp8_checkpoint` is false and it routes to the plain `load_module` path, where nothing pairs a weight with its scale. Three layers decline to catch it: burn-store maps `Dtype::I8 => DType::I8` without error (store.rs:1055); `CastFloatsAdapter` tests `is_float` and returns non-float snapshots unchanged; and the applier validates shape but not dtype (applier.rs:228-240), so the same-shaped `I8` weight raises no ShapeMismatch. The 224 `weight_scale` keys land as "Unused Tensors". Supported denoiser inputs remain bf16/f32 or a scaled-fp8 repack. Decisions 1-3 are untouched — the cubecl storage blocker is dtype-agnostic. Also records that a same-named community mirror (AX1Y2JP/Krea-2-Turbo-INT8-ConvRot) is a materially different artifact — 16.17 GB, 816 tensors, 192 quantized sites, 4.16 GB left in BF16 — so sizes quoted for this scheme need their source pinned. Read from the load path at pinned versions; not executed (no GPU on this host). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QNa5Xn2QW7fimm5XHokguK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
docs/adrs/0011-comfyui-cross-process-vram-sharing.mdand a pointerparagraph in
docs/roadmap.md. Docs only — no Rust touched.ADR-0011 records a spike into a proposed architecture: run
shared-tensorinside ComfyUIas a "VRAM model server", have loractl import the exported
cudaIpcMemHandle_twith
cudarc, and wrap the device pointer in a burn/cubecl buffer so trainingreads ComfyUI's already-resident Krea 2 weights instead of allocating its own.
The spike was deliberately ordered to test the most-likely-to-fail rung first —
foreign-pointer adoption, which needs no ComfyUI, no socket and no
shared-tensor— and it stopped there.Why
The load-bearing step was "wrap the pointer for CubeCL/Burn". ADR-0008 and
ADR-0010 both already asserted the negative ("cubecl exposes no managed-memory
allocation path (verified)"; "burn 0.21 exposes no pinned-allocation API to
call"), but neither quoted source, so nothing in the repo could settle it. This
upgrades both from asserted to demonstrated, at pinned versions.
Decision 1 — cubecl 0.10 cannot hold a pointer it did not allocate.
ComputeStorage(cubecl-runtime-0.10.0/src/storage/base.rs:74-95) has fivemethods and admits memory only through
fn alloc(&mut self, size: u64)— asize, not a pointer. In
cubecl-cuda-0.10.0/src/compute/storage/gpu.rs,GpuStorage's pointer map is private (:19), its only public constructor isnew(mem_alignment, stream)(:50), and the sole insertion sits insidealloc(
:169) after an unconditionalmalloc_async/malloc_sync.AllocationKind(
:9-12) is private with exactly two variants, both meaning "cubecl owns this".The back door is shut too: a hand-built
StorageHandlepanics inget(
:148-152). A grep forregister|adopt|from_ptr|from_raw|from_device_ptr| external|import|wrap|attachacross all of cubecl-runtime and cubecl-cudareturns zero hits.
Decision 2 — forcing one in would corrupt ComfyUI.
perform_deallocations(gpu.rs:63-80) frees every pointer in the mapunconditionally, by
AllocationKind. There is no ownership flag to consultbecause there is no variant meaning "not mine". This answered the spike's
second rung without hardware.
Decision 3 — the reverse direction is separately unavailable. cubecl's
allocprefersmalloc_async; NVIDIA documents sharing stream-orderedallocations through a different API family than
cudaIpcGetMemHandle, requiringIPC capability at pool creation, which
GpuStorage::newnever requests. Costsnothing — the reverse channel already ships over disk
(
diffusion_trainer.rs:1650-1663, keys pinned bytests/krea2_lora_keys.rs).Decision 4 — the budget fails independently of the mechanism. Measured from
the safetensors header of the checkpoint actually in use (
Comfy-Org/Krea-2@7b75ff3,diffusion_models/krea2_turbo_int8_convrot.safetensors):13,492,686,496 bytes, 878 tensors — 224
I8weights (12.16 GB) each with anF32
weight_scaleand a U8comfy_quantmarker, 76BF16left unquantized(1.33 GB). So the only shareable copy is ~13.5 GB, larger than the ~10.1 GB
int4 base loractl already uses; even a perfect zero-copy import is ~23 GB before
ComfyUI's own encoder, VAE and generation activations. loractl trains int4 Q4S
(packed
u32, block 32) quantized on-device — aliasing either ComfyUI scheme asthat is not a cast.
Decision 5 — the int8 repack is not a loadable loractl input today, over any
transport. The only actionable item here, and unrelated to IPC. The file
carries no
F8_E4M3, sois_fp8_checkpointis false and it routes to the plainload_modulepath, where nothing pairs a weight with its scale. Three layersdecline to catch it: burn-store maps
Dtype::I8 => DType::I8without error(
store.rs:1055);CastFloatsAdaptertestsis_floatand returns non-floatsnapshots unchanged; and the applier validates shape but not dtype
(
applier.rs:228-240), so the same-shapedI8weight raises noShapeMismatch.The 224
weight_scalekeys land as "Unused Tensors". Supported denoiser inputsremain bf16/f32 or a scaled-fp8 repack.
Net: the architecture is closed, co-tenancy stays a time-division problem, and
reading ComfyUI's model directory was never the blocked part
(
config/examples/krea2-comfyui.yaml). No probe example, nocudarcdependency, and no GPU hours were spent.
Verification
cargo fmt --all --checkpassesjust lint/just test— green in CI onfeb7b36(fmt + clippy + test,clippy (opt-in feature paths),cargo-deny,surf lint + checkall passed). They could not be run locally (
justis not installed inthis environment) and the diff is markdown-only, so CI is the gate of
record; re-running on
f0c7602.just audit/just denynot applicablepinned versions (cubecl
0.10.0, the burn0.21fork) or measureddirectly from a safetensors header, with byte counts given
Scope of evidence, stated plainly: all findings come from reading the load
path and upstream source at pinned versions, plus HTTP Range reads of
safetensors headers. Nothing was executed — this host has no GPU and no CUDA
toolkit. Decision 5's final materialization behavior (
Tensor::from_datawith adtype-mismatched snapshot) is therefore called out in the ADR as
backend-dependent and untested; what is certain is that the scale is never
applied.
The ADR is explicitly version-scoped, not a permanent law: it asks for a
re-check on the burn 0.22 migration (#79), which moves cubecl.
Linked issue
None. This ADR closes a question rather than opening work. Decision 5 is worth
an issue if anyone is pointing
model.denoiserat an int8 ComfyUI checkpoint.