Repository navigation
kernel: TPU, IVE, DWA, RGN and DDR bandwidth drivers, loaded on demand - #361
Conversation
The SG2002's engines beside the CPU get their drivers, built and installed but not loaded until something uses them. No NanoKVM OS feature uses them yet; docs/ACCELERATORS.md lists them with what was measured. - TPU: kernel patches 0047/0048 are Armbian's mainline port of the vendor driver (drivers/soc/sophgo/tpu, armbian/build#10760) and its devicetree node, applied unchanged. CONFIG_SOPHGO_CV181X_TPU=m gives /dev/cvi-tpu0. CONFIG_DMABUF_HEAPS_CMA=y makes the video CMA region a dma-buf heap (/dev/dma_heap/ion), which the TPU, having no IOMMU, needs for contiguous buffers. The runtime (Sophgo cviruntime, no licence upstream) is not part of the image. Kernel release r6. - IVE: cv181x_ive is built. osdrv 0012 resets the engine after an operation that times out: before, one unsupported size (3840x2160 hangs it) left every later two-input operation reading a stale address until the board restarted. - DWA (dewarp, rotation) and RGN (overlays) are built again. The media library still does not open them (cvi_mpi 0012 unchanged). - cv181x_aximon, a small driver of our own for the AXI monitors in front of the DDR controller (register layout from the osdrv mon driver), gives /proc/ddr_bandwidth: bytes read and written per port. udev would load the IVE, TPU, RGN and AXI monitor drivers at boot by device-tree alias. /etc/modprobe.d/nanokvm-accelerators.conf blacklists all five, which only stops that; nanokvm-modules keeps loading the media set only. A user loads one with modprobe. Measured on a NanoKVM PCIe at 1920x1080, 90 Hz, on the earlier test build: - IVE SAD 16x16 at 1920x1080: 5.5 ms, bit-exact against the CPU. - TPU: yolov8n 640x640 INT8 34.5 ms; 1080p90 H.265 kept 90 fps while yolov5s ran. - DDR: idle capture 373 MB/s; H.265 through VPSS 1.6 GB/s; plus yolov5s 2.3 GB/s. This build, booted once on the stand (r6 from dbg.sd): none of the five loaded after boot and none of their devices existed; modprobe of each created /dev/cvi-ive, /dev/cvi-tpu0, /dev/cvi-dwa, /dev/cvi-rgn and /proc/ddr_bandwidth (capture writing 442 MB/s at 2560x1440, 60 Hz); rmmod of each worked; the VI kept 60 fps throughout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dormancygrace
left a comment
There was a problem hiding this comment.
Found two TPU lifetime/initialization issues in the imported driver. I am preparing follow-up fixes and regression checks in this PR, keeping the imported upstream patches unchanged.
dormancygrace
left a comment
There was a problem hiding this comment.
Published fixes for the two findings in f3361c2, as a separate follow-up patch 0049; the original Armbian imports remain unchanged. No further actionable findings in this review.
Verified locally: actual TPU mapping-lifetime/PIO/clock-failure helpers under ASan/UBSan (including a failing negative control with job retention removed), platform-config, platform-build and resource-trim tests; clean application of kernel patches 0047/0048/0049; RISC-V compilation of TPU and AXI monitor modules against the existing Linux 7.2.10 r5 build headers. The new regression test is included in CI.
Limitations: I did not boot a device or test the r5-to-r6 package upgrade, and did not complete a full kernel/image rebuild locally. CI for f3361c2 has started; its full result is not yet verified. The user explicitly asked not to wait for the full CI run and separately asked NOT to merge. PR remains open; both finding threads retain Resolve buttons.
dormancygrace
left a comment
There was a problem hiding this comment.
Final merge check: head remains the reviewed/fixed f3361c2. All 14 completed CI checks passed, including the new TPU regression test in Init scripts and tooling; only the full build/hash job is still running. The author has additionally confirmed device testing of patch 0049: OCR execution, five SIGKILL cases during queued work, subsequent successful jobs/CMA recovery, VI at 60 fps, and module unload (comments 4337399747/4337400231). These are author-reported hardware results, separate from my local verification. No new actionable concerns were added. The user has lifted the merge hold and explicitly authorized merging without waiting for the full build.
Build outputsCompared with the build of main at b98b25d. 30 changed, 97 added, 92 removed of 556 outputs.
|
The SG2002 has engines beside the CPU: the TPU, the IVE, DWA, RGN and the AXI monitors in front of the DDR controller. This PR adds their drivers to the image. It adds no feature that uses them: no screen change map, no OCR and no MCP tools. Nothing loads the drivers until something asks for one.
What
drivers/soc/sophgo/tpu, sophgo-sg200x: add TPU support for the Milk-V Duo S (mainline port of the vendor driver) armbian/build#10760) and its devicetree node, applied unchanged.CONFIG_SOPHGO_CV181X_TPU=mgives/dev/cvi-tpu0.CONFIG_DMABUF_HEAPS_CMA=ymakes the video CMA region a dma-buf heap,/dev/dma_heap/ion. The TPU has no IOMMU and needs contiguous buffers from it.cv181x_iveis built. osdrv 0012 resets the engine after an operation that times out. Without it, one unsupported size (3840×2160 hangs the engine) left every later two-input operation reading a stale address, with no error, until the board restarted.cv181x_aximonis a small driver of our own for the AXI monitors; the register layout comes from the osdrvmondriver. It gives/proc/ddr_bandwidth: bytes read and written per port since sampling started./etc/modprobe.d/nanokvm-accelerators.confblacklists all five, which stops only that automatic load.nanokvm-modulesstill loads only the media set, and a user loads an accelerator driver withmodprobe. An idle engine holds no memory and has no device nodes.docs/ACCELERATORS.mdcovers the engines, how to load them and what was measured.platform/README.mdnow says r6, with 65 in-tree and 18 out-of-tree modules.Stand (NanoKVM PCIe, LT6911UXC)
This build was booted once as r6 from
dbg.sd, with the VI from #349 in its modules:modprobeof each created/dev/cvi-ive,/dev/cvi-tpu0,/dev/cvi-dwa,/dev/cvi-rgnand/proc/ddr_bandwidth. The capture port wrote 442 MB/s, which is 2560 × 1440 × 2 bytes × 60.rmmodof each worked./dev/dma_heap/ionand the 144 MiB CMA region were present.Measured on the earlier test build with the same drivers (1920 × 1080 at 90 Hz):
Tests
Not tested
Merge order
#359 (Linux 7.2.12) also changes the kernel release string. Whichever merges second renames it.
🤖 Generated with Claude Code