Ticket 7: scaffold virtio-accel-vulkan (wayfinder map #154) - #175
Merged
Conversation
- Adopt graph-shaped data plane as permanent (ADR 0001) - Bind Vulkan via ash/loaded; build.rs enumerates host targets (ADR 0002) - Lower via checked-in per-operator SPIR-V + specialization constants at load (ADR 0003) - Declare FP32 base tier + provisional INT8 candidates (ADR 0004) - Workspace, release-policy, portability, README, and CONTRIBUTING plumbing
This was referenced Sep 3, 2026
SnowCheetos
added
the
area: backend
SnowCheetos
marked this pull request as ready for review
September 3, 2026 08:38
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
There are a few concrete correctness/documentation issues in the newly added scaffold (notably a broken ADR link and an example that will panic once the native backend starts initializing).
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR scaffolds the new virtio-accel-vulkan host-native backend crate and lands the first set of Vulkan-wayfinder ADRs (tickets 1–4) that define the long-term design constraints for the backend, while updating workspace documentation and release/publish plumbing to include the new crate.
Changes:
- Add ADRs 0001–0004 capturing Vulkan backend design decisions (data-plane posture,
ashbinding strategy, shader/lowering approach, and tier candidates). - Introduce
crates/virtio-accel-vulkanas a build-gated scaffold crate (admission constants + placeholder API, initial tests, example, and safety/audit skeleton). - Update workspace/release/publish documentation and CI policy to include an 18th published crate and the new publish order position.
File summaries
| File | Description |
|---|---|
| README.md | Updates workspace README to describe the Vulkan scaffold and add it to the crate table/graph. |
| docs/release-policy.md | Updates published-crate count and publish-order table to include virtio-accel-vulkan. |
| docs/portability.md | Adds Vulkan to the host-native tier and documents the target-enumeration gate semantics. |
| docs/adr/0001-graph-shaped-data-plane.md | ADR ratifying the graph-shaped data plane as permanent for GPU-class backends. |
| docs/adr/0002-vulkan-binding-ash.md | ADR selecting ash (loaded/runtime discovery) and defining the build-gate semantics. |
| docs/adr/0003-tosa-lowering-specialized-shaders.md | ADR specifying checked-in SPIR-V with specialization at load_program. |
| docs/adr/0004-first-tier-candidates.md | ADR declaring FP32 base + INT8 candidate tiers and deferring FP16. |
| crates/virtio-accel-vulkan/Cargo.toml | New crate manifest wiring core/tosa deps and conformance dev-dep. |
| crates/virtio-accel-vulkan/build.rs | Build-time cfg gate (va_vulkan) based on target OS + env override. |
| crates/virtio-accel-vulkan/src/lib.rs | Scaffold public API (targets re-export, placeholder accelerator, InitError). |
| crates/virtio-accel-vulkan/src/lower.rs | Defines the two scaffold Target constants for Vulkan tier candidates. |
| crates/virtio-accel-vulkan/tests/vulkan.rs | Integration tests for target constants and placeholder error behavior. |
| crates/virtio-accel-vulkan/examples/tosa_vulkan.rs | Example entrypoint demonstrating the scaffold placeholder behavior. |
| crates/virtio-accel-vulkan/README.md | Crate-level README describing scaffold status, gate, and planned tiers. |
| crates/virtio-accel-vulkan/SAFETY.md | Scaffold safety/audit plan for future unsafe/FFI work. |
| crates/virtio-accel-vulkan/LICENSE-MIT | New per-crate MIT license copy for packaging/publishing policy. |
| crates/virtio-accel-vulkan/LICENSE-APACHE | New per-crate Apache-2.0 license copy for packaging/publishing policy. |
| CONTRIBUTING.md | Adds the Vulkan example command to the local validation/gate list. |
| ci/publication.py | Adds virtio-accel-vulkan to the canonical publish order. |
| ci/check-release-policy.py | Adds virtio-accel-vulkan to the explicit release-policy allowlist. |
| Cargo.toml | Adds the new crate as a workspace member and workspace dependency. |
| Cargo.lock | Records the new workspace package entry for virtio-accel-vulkan. |
Review details
Suppressed comments (1)
crates/virtio-accel-vulkan/src/lib.rs:55
- The doc comment for
available_devices()says the scaffold “enumerate[s] nothing”, but the implementation returnsErr(InitError::RuntimeUnavailable). Either return an empty list or update the doc comment so callers aren’t misled.
/// Enumerate nothing in the scaffold; the native path reports one entry per
/// `ash`-enumerated physical device.
pub fn available_devices() -> Result<Vec<String>, InitError> {
Err(InitError::RuntimeUnavailable)
}
- Files reviewed: 21/22 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+1
to
+8
| fn main() { | ||
| match virtio_accel_vulkan::VulkanAccelerator::new() { | ||
| Ok(_) => unreachable!("the scaffold placeholder never initializes a backend"), | ||
| Err(error) => { | ||
| eprintln!("virtio-accel-vulkan is scaffolded but not yet executing: {error}"); | ||
| } | ||
| } | ||
| } |
Comment on lines
+6
to
+7
| //! [Vulkan wayfinder map](https://github.com/MicroPerceptron/virtio-accel/issues/154). The design | ||
| //! decisions ratified by this scaffold live in [`docs/adr/`](../../../../docs/adr/) (ADRs 0001–0004). |
Comment on lines
+153
to
157
| Eighteen packages are published to crates.io. Publication is ordered: a crate cannot be published | ||
| before | ||
| every crate it depends on, and that includes development dependencies, because a published crate's | ||
| versioned dev-dependencies must resolve from the registry for `cargo test` to run on the packaged | ||
| source. |
This was referenced Sep 3, 2026
micro-perceptron
approved these changes
Sep 3, 2026
micro-perceptron
added a commit
that referenced
this pull request
Sep 3, 2026
Stacked on #175. Resolves wayfinder ticket #176 (research; agent-grounded against Mesa source and gpuinfo). Decides: **Vulkan 1.3 floor** (synchronization2, dynamicRendering, maintenance4, shaderIntegerDotProduct core; feature-enable still required); **memory-domain map** (Host→HOST_VISIBLE|HOST_COHERENT persistently mapped; Device→DEVICE_LOCAL staging confined to read/write — first backend advertising DEVICE_LOCAL_MEMORY; Shared→DEVICE_LOCAL|HOST_VISIBLE advertised only on ReBAR/UMA probe); **allocation count** assumed 4096 (NVIDIA/WDDM binds), dedicated allocations with honest low limits, suballocation permitted while staying DIRECT_BINDING-legal; **alignment reported per device**, no RADV assumption. All findings verified against Mesa sources and gpuinfo as of 2026-09-03.
micro-perceptron
added a commit
that referenced
this pull request
Sep 3, 2026
…apipe) (#182) Stacked on #179. Resolves wayfinder ticket 8 (#181) of map #154: the `ash` FFI, the full `Accelerator` lifecycle, and FP32 IDENTITY executing end-to-end on a real GPU and a software ICD. ## Why The scaffold (#175) and design ADRs (#177, #179) had no execution behind them. Ticket 8 is the proof that the graph-shaped data plane (ADR 0001), runtime-loaded `ash` (ADR 0002), checked-in specialized SPIR-V (ADR 0003), the 1.3 baseline and memory-domain map (ADR 0005), and the no-worker-thread fence-poll execution model (ADR 0006) hold on metal. ## What - **`native.rs`** — Vulkan 1.3 through pinned `ash 0.38.0` (`loaded`, `std`): instance/device enumeration with honest `DeviceIdentity` (device UUID, vendor/device id, `GPU` class; `OTHER` for CPU ICDs), compute-queue selection (compute-only family preferred), per-backend device with `synchronization2` and `bufferDeviceAddress` (the latter only to *measure* allocation alignment). Dedicated `VkBuffer`+`VkDeviceMemory` per buffer, bound directly as storage buffers. `Host`/`Shared` are persistently mapped host-coherent memory (no flush path needed or issued); `Device` is device-local memory reached only through a bounded 4 MiB staging copy inside `write_buffer`/`read_buffer`. Per-context ring of 64 (command buffer, fence, descriptor set) triples; `vkQueueSubmit2` success is the admission boundary; `poll_event` is one `vkGetFenceStatus`; finite timeouts rejected pre-admission; `VK_ERROR_DEVICE_LOST` poisons the instance; parents refuse release while children or in-flight events reference them. Diagnostics: `direct_binding_admissions`, `explicit_transfer_bytes`, `live_resources`. - **`lower.rs`** — hardware-free admission: FP32 target only, one region/block, exactly one boundary-to-boundary IDENTITY, static positive shapes, u32 element domain. `VULKAN_TOSA_CAPABILITY` names exactly that boundary. The integer target stays declared, unadvertised. - **`shader.rs`** — the checked-in SPIR-V 1.3 word-copy kernel, assembled instruction by instruction with its disassembly beside it (no toolchain, no `rspirv`), validated structurally by unit tests and offline with naga's SPIR-V front end. Element count is specialization constant 0. - **Finding recorded in ADR 0005:** ANV exposes memory types a storage buffer may *not* use; the heap list alone mis-selects. The memory-domain map is now chosen against a probe buffer's `memoryTypeBits`. - **SAFETY.md** — real audit: `ash` pin and the exact entry-point inventory, handle lifetime, mappings, ring, device loss, shaders, evidence. Registered in `ci/check-release-policy.py` (`UNSAFE_AUDITS`) and `docs/release-policy.md` (sixth exception). `deny.toml` allows ISC for `libloading`. - **CI** — `vulkan-lavapipe-test` lane mirroring `openvino-host-test`: distribution Mesa, ICD pinned via `VK_DRIVER_FILES`, `VIRTIO_ACCEL_VULKAN=1`, and `VIRTIO_ACCEL_VULKAN_REQUIRE_DEVICE=1` so a missing ICD fails instead of skipping. Ticket 10 still owns the full lane hardening. - **Docs** — README, crate README, `docs/portability.md`, `docs/architecture.md`; ADR 0005/0006 gain a ticket 8 evidence section and the review-flagged wording fixes (`vkGetPhysicalDeviceFeatures2`, the SAFETY.md quote); `docs/release-policy.md` line break, the crate doc link, and the example's `unreachable!` from the #175 review are fixed. ## Validation (2026-09-03, Intel Arc 140V / Lunar Lake, Mesa 26.0.8 ANV, Vulkan 1.4.335; and llvmpipe on the same host) - `cargo test -p virtio-accel-vulkan`: 11 unit + 7 host-independent + 12 native tests pass on **both** devices — lifecycle in every advertised domain (`Host`, `Device`, `Shared` on both), `IDENTITY_EDGES_FP32` bit-exact including NaN payload and subnormal, offset bindings with untouched neighbors, segmented staging to device-local memory, binding/timeout rejection, 16 overlapping read-only submissions in flight with `Busy` on the shared input, ring exhaustion as `ResourceLimit`, parent-release refusal, and the standard conformance suite with accounting and copy-path hooks in every domain. - `cargo run -p virtio-accel-vulkan --example tosa_vulkan` on ANV and pinned to lavapipe: `3.25` round-trips. - Warm FP32 identity (release, 200 samples): ANV admission p50 16.6 µs / submit-to-complete p50 259 µs; llvmpipe 4.3 µs / 58 µs. - `cargo fmt --check`, `cargo clippy --workspace --all-targets --all-features -D warnings`, `python3 ci/check-release-policy.py`, `cargo deny --workspace --all-features check`, `cargo test --workspace --all-targets --all-features` (one pre-existing OpenVINO failure on this host, `executes_boolean_boundaries_on_every_available_device`, reproduces on `main` and is unrelated), placeholder build (`VIRTIO_ACCEL_VULKAN=0`) clippy/tests/example, `cargo doc -D warnings`. ## Not in this ticket Operator tiers beyond IDENTITY (ticket 9), FP16/INT8 gating (ticket 5), full CI lane hardening and the numerical corpus (ticket 10), performance doc and manual hardware commands (ticket 11). Grid-stride dispatch for tensors beyond `maxComputeWorkGroupCount[0] × 64` elements is rejected as `ResourceLimit` at load until ticket 9. Closes #181 once merged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of wayfinder map #154 ticket 7 (scaffold). Resolves map tickets 1, 2, and 4 as ADRs; declares ticket 5's scaffold constants as provisional candidates.
Why
The Vulkan backend can't grow without a landing foundation: build-time gate, placeholder, admission constants, and release plumbing. ADRs 1–4 resolve the map's first four grilling tickets in one reviewable change; the scaffold proves-by-building that the foundation holds.
What
docs/adr/0001–0004):ashvialoadedfeature (dlopens Vulkan loader at runtime; no SDK at build time); the convention question resolved —va_vulkanemit-by-target-enumeration (Linux/Android/Windows/macOS) with loud force-on failure on unsupported targets; runtime loader discovery reportsInitError::RuntimeUnavailable.load_program;VkShaderModule/VkComputePipelineatload_program, neversubmit; retained pipelines/CBs/descriptor pools charged againstArtifactRef::resident_bytes;rspirvfallback recorded.rspirv); FP8 rejected at admission.crates/virtio-accel-vulkan:<core,tosa>deps,<conformance>dev-dep, no features; build.rs gated enumeration probe;#![forbid(unsafe_code)]placeholder returningInitError::RuntimeUnavailable; emptyTosaCapabilityProvider;lower.rstwoTargetconsts.PUBLISHED_PACKAGES+PUBLISH_ORDER(vulkan at 17, facade 18), README/crate table/dependency graph/Vulkan section,docs/portability.mdrow + target-enumeration note (ADR 0002),docs/release-policy.mdcount bump,CONTRIBUTINGgate list.Validation
cargo fmt --check,python3 ci/check-release-policy.py(18 packages), clippy, workspace tests excludingvirtio-accel-coreml(known environment failures, low-level), crate tests: 6 integration + 2 unit pass.ci/publish-dry-run.pyvalidates vulkan at position 12/18 pre-reorder (then reverted); the full ordered run is blocked upstream by the pre-existing coreml failures.