Skip to content

kernel: TPU, IVE, DWA, RGN and DDR bandwidth drivers, loaded on demand - #361

Merged
dormancygrace merged 2 commits into
mainfrom
kernel/tpu-ive-support
Oct 21, 2026
Merged

dormancygrace merged 2 commits into
mainfrom
kernel/tpu-ive-support

Conversation

@dormancygrace

Copy link
Copy Markdown
Owner

The SG2002 has engines beside the CPU: the TPU, the IVE, DWA, RGN and the AXI monitors in front of the DDR controller. This PR adds their drivers to the image. It adds no feature that uses them: no screen change map, no OCR and no MCP tools. Nothing loads the drivers until something asks for one.

What

  • TPU. Kernel patches 0047 and 0048 are Armbian's mainline port of the vendor driver (drivers/soc/sophgo/tpu, sophgo-sg200x: add TPU support for the Milk-V Duo S (mainline port of the vendor driver) armbian/build#10760) and its devicetree node, applied unchanged.
    • CONFIG_SOPHGO_CV181X_TPU=m gives /dev/cvi-tpu0.
    • CONFIG_DMABUF_HEAPS_CMA=y makes the video CMA region a dma-buf heap, /dev/dma_heap/ion. The TPU has no IOMMU and needs contiguous buffers from it.
    • The runtime, Sophgo cviruntime, is not in the image: it has no licence upstream.
    • The kernel release becomes r6.
  • IVE. cv181x_ive is built. osdrv 0012 resets the engine after an operation that times out. Without it, one unsupported size (3840×2160 hangs the engine) left every later two-input operation reading a stale address, with no error, until the board restarted.
  • DWA and RGN (dewarp/rotation and overlays) are built again. The media library still does not open them; cvi_mpi 0012 is unchanged.
  • DDR bandwidth. cv181x_aximon is a small driver of our own for the AXI monitors; the register layout comes from the osdrv mon driver. It gives /proc/ddr_bandwidth: bytes read and written per port since sampling started.
  • Loaded on demand. udev would load the IVE, TPU, RGN and AXI monitor drivers at boot by device-tree alias. /etc/modprobe.d/nanokvm-accelerators.conf blacklists all five, which stops only that automatic load. nanokvm-modules still loads only the media set, and a user loads an accelerator driver with modprobe. An idle engine holds no memory and has no device nodes.
  • Docs. docs/ACCELERATORS.md covers the engines, how to load them and what was measured. platform/README.md now says r6, with 65 in-tree and 18 out-of-tree modules.

Stand (NanoKVM PCIe, LT6911UXC)

This build was booted once as r6 from dbg.sd, with the VI from #349 in its modules:

  • After boot: none of the five was loaded, and none of their devices existed.
  • modprobe of each created /dev/cvi-ive, /dev/cvi-tpu0, /dev/cvi-dwa, /dev/cvi-rgn and /proc/ddr_bandwidth. The capture port wrote 442 MB/s, which is 2560 × 1440 × 2 bytes × 60.
  • rmmod of each worked.
  • Video: the VI kept 60 fps throughout. /dev/dma_heap/ion and the 144 MiB CMA region were present.

Measured on the earlier test build with the same drivers (1920 × 1080 at 90 Hz):

Engine Result
IVE SAD 16×16 at 1920 × 1080 in 5.5 ms, bit-exact against the CPU
TPU yolov8n 640 × 640 INT8 in 34.5 ms; 1080p90 H.265 kept 90 fps while yolov5s ran
DDR idle capture 373 MB/s; H.265 through VPSS 1.6 GB/s; with yolov5s added, 2.3 GB/s

Tests

  • Local: all 53 Python and 11 shell script tests of the CI job pass. The kernel, the modules and the boot images build.
  • CI: the release build compares outputs with main. It will show the module directory moving from r5 to r6.

Not tested

  • The r5 → r6 kernel package upgrade on the stand. It is the same path as every earlier revision bump.
  • Speed figures on this exact build. The numbers in the table come from the earlier test build.

Merge order

#359 (Linux 7.2.12) also changes the kernel release string. Whichever merges second renames it.

🤖 Generated with Claude Code

The SG2002's engines beside the CPU get their drivers, built and installed
but not loaded until something uses them. No NanoKVM OS feature uses them
yet; docs/ACCELERATORS.md lists them with what was measured.

- TPU: kernel patches 0047/0048 are Armbian's mainline port of the
  vendor driver (drivers/soc/sophgo/tpu, armbian/build#10760) and its
  devicetree node, applied unchanged. CONFIG_SOPHGO_CV181X_TPU=m gives
  /dev/cvi-tpu0. CONFIG_DMABUF_HEAPS_CMA=y makes the video CMA region a
  dma-buf heap (/dev/dma_heap/ion), which the TPU, having no IOMMU,
  needs for contiguous buffers. The runtime (Sophgo cviruntime, no
  licence upstream) is not part of the image. Kernel release r6.
- IVE: cv181x_ive is built. osdrv 0012 resets the engine after an
  operation that times out: before, one unsupported size (3840x2160
  hangs it) left every later two-input operation reading a stale
  address until the board restarted.
- DWA (dewarp, rotation) and RGN (overlays) are built again. The media
  library still does not open them (cvi_mpi 0012 unchanged).
- cv181x_aximon, a small driver of our own for the AXI monitors in
  front of the DDR controller (register layout from the osdrv mon
  driver), gives /proc/ddr_bandwidth: bytes read and written per port.

udev would load the IVE, TPU, RGN and AXI monitor drivers at boot by
device-tree alias. /etc/modprobe.d/nanokvm-accelerators.conf blacklists
all five, which only stops that; nanokvm-modules keeps loading the
media set only. A user loads one with modprobe.

Measured on a NanoKVM PCIe at 1920x1080, 90 Hz, on the earlier test
build:
- IVE SAD 16x16 at 1920x1080: 5.5 ms, bit-exact against the CPU.
- TPU: yolov8n 640x640 INT8 34.5 ms; 1080p90 H.265 kept 90 fps while
  yolov5s ran.
- DDR: idle capture 373 MB/s; H.265 through VPSS 1.6 GB/s; plus
  yolov5s 2.3 GB/s.

This build, booted once on the stand (r6 from dbg.sd): none of the five
loaded after boot and none of their devices existed; modprobe of each
created /dev/cvi-ive, /dev/cvi-tpu0, /dev/cvi-dwa, /dev/cvi-rgn and
/proc/ddr_bandwidth (capture writing 442 MB/s at 2560x1440, 60 Hz);
rmmod of each worked; the VI kept 60 fps throughout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@dormancygrace dormancygrace left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found two TPU lifetime/initialization issues in the imported driver. I am preparing follow-up fixes and regression checks in this PR, keeping the imported upstream patches unchanged.

Comment thread platform/kernel/0047-soc-sophgo-cv181x-tpu-driver.patch
Comment thread platform/kernel/0047-soc-sophgo-cv181x-tpu-driver.patch

@dormancygrace dormancygrace left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Published fixes for the two findings in f3361c2, as a separate follow-up patch 0049; the original Armbian imports remain unchanged. No further actionable findings in this review.

Verified locally: actual TPU mapping-lifetime/PIO/clock-failure helpers under ASan/UBSan (including a failing negative control with job retention removed), platform-config, platform-build and resource-trim tests; clean application of kernel patches 0047/0048/0049; RISC-V compilation of TPU and AXI monitor modules against the existing Linux 7.2.10 r5 build headers. The new regression test is included in CI.

Limitations: I did not boot a device or test the r5-to-r6 package upgrade, and did not complete a full kernel/image rebuild locally. CI for f3361c2 has started; its full result is not yet verified. The user explicitly asked not to wait for the full CI run and separately asked NOT to merge. PR remains open; both finding threads retain Resolve buttons.

@dormancygrace dormancygrace left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final merge check: head remains the reviewed/fixed f3361c2. All 14 completed CI checks passed, including the new TPU regression test in Init scripts and tooling; only the full build/hash job is still running. The author has additionally confirmed device testing of patch 0049: OCR execution, five SIGKILL cases during queued work, subsequent successful jobs/CMA recovery, VI at 60 fps, and module unload (comments 4337399747/4337400231). These are author-reported hardware results, separate from my local verification. No new actionable concerns were added. The user has lifted the merge hold and explicitly authorized merging without waiting for the full build.

@dormancygrace
dormancygrace merged commit 856ebfa into main Oct 21, 2026
15 checks passed
@dormancygrace
dormancygrace deleted the kernel/tpu-ive-support branch October 21, 2026 13:30
@github-actions

Copy link
Copy Markdown

Build outputs

Compared with the build of main at b98b25d.

30 changed, 97 added, 92 removed of 556 outputs.

directory files
changed . Image
changed boot 18 files
changed boot-fit boot-fit/alpha.sd
boot-fit/beta.sd
boot-fit/detect.sd
boot-fit/lite.sd
boot-fit/pcie.sd
changed dtb dtb/alpha.dtb
dtb/beta.dtb
dtb/detect.dtb
dtb/lite.dtb
dtb/pcie.dtb
changed opensbi opensbi/build-manifest.json (records the hash of platform/build.sh)
added lib 97 files
removed lib 92 files
Hashes: outputs-sha256 artifact of this run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant