[novkien]

Scope

Addresses #3. The fork owner approved the physical-pair backend after native preflight, then explicitly requested merge, native/RPC build, Qwen-27B deployment and acceptance of issues #1/#3 on 2026-09-30 in the Codex task.

This extends the existing documentation draft with the implemented GGML_CUDA_P2P_PAIRS option. Runtime source changes are confined to ggml/src/ggml-cuda/ggml-cuda.cu (114 additions, 3 deletions). The branch includes the earlier local PR #2 merge in its ancestry so the host/guest checkouts can advance without discarding local commits. Its cache tree matches merged PR #2; the PR diff against current master contains only the selective transport and documentation pack.

Implementation

• Parse one immutable physical-pair matrix; reject malformed/range/self-pair input and old-P2P/managed-memory conflicts before enabling peers.
• Validate both capabilities before enabling exactly the requested directions; handle already-enabled peers and stop on an unexpected enable failure.
• Show full UUID/PCI mapping and selected pairs at startup.
• Filter actual VMM owner/reader grants even in an NCCL build, using the physical count independently of logical aliases.
• Decline excluded distinct-physical edges in both CUDA copy callbacks before enqueue; retain the existing explicit host fallback, local D2D and selected peer/event paths.
• Reject unsupported selective collective initialization before NCCL/internal/meta setup. Legacy behavior is retained with the new key absent; HIP/MUSA attempted opt-in is unsupported.

No staging pool, layer-placement, precision, RPC transport or NVML runtime change is added. The required launcher key validation is merged separately in llama-proxy PR #484, tracked by #470.

Verification

Actual four-V100 native coverage and exact candidate hashes are documented in NATIVE-RESULTS.md: 192 malloc + 192 actual-VMM pool cases, 576 copy-callback cases, 2,688 physical-alias cases, eight four-stage scheduler graphs, 25 startup cases, four additional initialization/physical-reader cases and 60 maintained backend operation checks. Zero raw-copy mismatches and no forbidden observer operations were reported in the tested scope. Reverse capability/enable failures were simulated API returns, explicitly identified.

Selected 16 MiB transfer probes measured approximately 126-132 GB/s; eight excluded pageable-host directions measured 3.374-4.382 GB/s. These are copy results, not a Qwen token-speed claim. Copy-slot rotation, cancellation and uncompiled platform variants remain unproven. Historical host-only preflight stays explicitly historical.

The combined server build also passed 33 native PR #2 cache checks and 14 focused HTTP cache/idle tests with the official tiny model. Real Qwen/Nex cache checks, matching published native/RPC manifests, effective managed child verification, Qwen baseline/selective/rollback and four-slot requests follow deployment. Issues remain open until applicable acceptance passes. Optional SSD spill is not implemented.

Delivery and rollback

Publish this reviewed candidate, build the same clean merged source on the GPU host and RPC guest, verify their complete build manifests, activate the reviewed launcher/bridge dependency and change Qwen-27B through revision-checked route APIs. Preserve the current production DFlash settings; test the historical MTP fixture separately with matched configurations. Do not enable the known unsafe global P2P condition. Retain the original route/backend/environment for rollback.

Authorship

Codex implemented and tested this candidate at the private fork owner's explicit request. This is a private-fork delivery and does not claim upstream approval.

[novkien]

Pre-merge handoff

Owner and scope: Codex root task owns fork issue #3 and live acceptance; PR #4 reviewed head ed1d2b9826fa181060e4dc68ca099c43f18cdeb0. The owner explicitly authorized deployment/build plus Qwen route change and acceptance of issues #1/#3 in chat on 2026-09-30.

Contribution and review: the PR contains the selective CUDA source (SHA256 cb675a0ceacc4c2ba3eb5fdff32eb13e529892ab9f84c3002203492a61296c23) and its documentation pack. Its PR #2 cache tree matches current master, so the diff against master adds only P2P code/docs. Main authored/reviewed the documentation and inspected the complete source diff; independent review is recorded before merge. Delivery preserves these bytes.

Validation: Release CUDA/NCCL/RPC server build; actual four-V100 native pool/callback/alias/scheduler/startup cases and 60 maintained backend checks passed within NATIVE-RESULTS scope. The combined cache build passed 33 native and 14 focused HTTP checks. git diff --check passed. Simulated API errors, uncompiled platform guards and unrun concurrency/cancellation/zero-size cases are labeled.

Remaining and next: normal non-force merge of this exact head, then build/install the same published clean source on native/RPC, verify manifests/effective managed child and run live Qwen/Nex acceptance. Proxy/bridge validation merged in novkien/llama-proxy#484; its activation is tracked in ggml-org#470. Root owns route mutations, tests and evidence-based closure. Merge does not close either issue.

[novkien]

Work report - backend delivered; live acceptance in progress

Owner and scope: Codex root task owns fork issues #1/#3. The owner explicitly requested build/deployment, Qwen-27B backend change and tests/closure in this task on 2026-09-30. Later steering explicitly authorized proxy hard-restart recovery, required CUDA order starting with the Gen3 x8 GPU, and requested reducing context/rebalancing the split so pipeline parallelism remains enabled. The resulting configuration is intended for their continued use.

Done and evidence: PR #2 merged at 7865d7f6b3761cd21b4ab7dbfc91c3d78b83164a; its combined native tests passed 33 checks and 14 focused HTTP cases. PR #4 merged at 95e6565970621aa508e80fd805dcf73af413093c after main review and independent review found no actionable defect in the exact CUDA source. Native and RPC guest checkouts advanced cleanly, both builds passed and all installed executable/library hashes were verified. Both manifests identify 95e6565970621aa508e80fd805dcf73af413093c:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855. Native uses CUDA 12.9/sm70/NCCL/RPC; guest uses CUDA 13.3/sm120/RPC. Guest retained NCCL=ON configuration but the package was unavailable and NCCL was not linked there. The supported RPC helper started and verified the fork daemon on port 50052.

Launcher validation merged in novkien/llama-proxy#484. The managed activation stalled on retirement of four long-lived Godot observer streams. Under the owner's subsequent explicit hard-restart authorization, only the proxy service underwent cold recovery; the job was not called safely completed. Those streams reconnected. The reviewed bridge code is active and its authenticated child readback matches the fork executable hash cd5d9185c1e781e9716cbd323fb6295ff6b89c14aabc8010ac75f424bfb0ac78. The old global P2P key is absent.

Configuration steering: actual hardware is four V100-SXM2 16 GB, not 32 GB. This corrects the capacity typo in NATIVE-RESULTS; the UUIDs, native test bytes and transport results are unchanged. A documentation correction accompanies the final acceptance report. Current runtime order is PCI 08:00 (Gen3 x8) -> NV6 -> 0c:00 -> PIX -> 0d:00 -> NV6 -> 09:00, matching UUID order 5ca3,b2e,e149,d4d. Startup and effective child environment both confirm that order.

The original -c 400000 -ts 28,28,29,15 DFlash route loaded but its main scheduler retried without pipeline parallelism after compute-buffer OOM. The owner's new request supersedes that context/split constraint. The current trial uses -c 200000 -ts 29,29,27,15, preserves four GPUs/four slots and the current DFlash2 draft, and keeps GGML_CUDA_NO_PINNED=1. Native allocates an aligned 200192-token unified context; startup reports pipeline parallelism enabled, approximately 4431 MiB main compute buffers per GPU and no fallback. Small four-slot requests left approximately 1901-2873 MiB free per GPU. Different context/split profiles are not used as a P2P-only performance comparison.

Remaining and next: finish fresh same-profile selector-off/on/rollback requests and historical MTP-method comparison under the new owner-selected memory profile; run meaningful long-context Qwen/Nex RAM retention, borrowed snapshot, cache opt-out and media checks; report actual outputs/latency/memory and retain the verified usable Qwen profile. Optional SSD spill is not implemented. Both fork issues stay open until their applicable live checks pass.