Skip to content

apple: program the NHI interrupt throttle via ring->interval_nsec - #75

Open
afrog33k wants to merge 1 commit into
hellas-ai:mainfrom
afrog33k:apple-nhi-throttle
Open

afrog33k wants to merge 1 commit into
hellas-ai:mainfrom
afrog33k:apple-nhi-throttle

Conversation

@afrog33k

Copy link
Copy Markdown

What this fixes

On an Apple NHI the interrupt throttle register lives at 0xd004c, not the
stock 0x38c00 that tb_ring_throttling() programs. The kernel's own ring
activation writes the Apple one from ring->interval_nsec, and skips the
write when that field is zero
— which it always was, because this module
never set it.

So on Apple hosts the throttle was silently skipped, the register kept its
firmware default, and the link carried a latency floor that did not move with
message size: 2 B through 4 KB all measured ~65.3 us (ib_send_lat across a
USB4 link).

The change

  • Set ring->interval_nsec for both rings before tb_ring_start() on Apple
    hosts. This is not gated on the peer's backend: the NHI is Apple's
    regardless of who is on the other end of the cable, so native-backend rings
    toward a Linux peer need it just as much. (tbv_path_is_apple() is
    backend-based and would skip exactly the leg that needs this.)
  • Log the before/after value so the write is observable.
  • Guard the field access behind a Makefile compile probe
    (TBV_HAVE_RING_INTERVAL_NSEC), so stock kernels — where struct tb_ring
    has no such member — still build.

Measured

Before/after on the same link, same harness (ib_send_lat, 2 B, warm floor):

typical
before 65.3 us, flat from 2 B to 4 KB
after 14.8 us, scaling with size

The stock-NHI half of the same fix (writing 0x38c00 directly on a host whose
kernel does not export tb_ring_throttling()) is deliberately not in this
PR — it needs a nhi_throttle_direct-style opt-in and its own discussion.

Verified: builds against the Asahi 7.1.13 USB4 tree with no new warnings.

Context: found while bringing up RDMA over a USB4 cable between an Asahi
host and a stock-kernel AMD host. Write-up:
https://boracode.ai/blog/thunderbolt-no-incarnation

On an Apple NHI the interrupt throttle register (0xd004c) is not the one
tb_ring_throttling() programs (0x38c00); the kernel's own ring activation
writes it from ring->interval_nsec and skips the write when that field is
zero. This module never set the field, so on Apple hosts the throttle was
silently skipped and the register kept its firmware default -- measurable as
a latency floor that does not move with message size (2 B through 4 KB all
~65.3 us on a USB4 link).

Set ring->interval_nsec before activation for every ring on an Apple host
(including native-backend rings toward a Linux peer: the NHI is Apple's
regardless of the peer), log the before/after value, and guard the field
access behind a Makefile compile probe so stock kernels still build.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@hellasbot

hellasbot commented Sep 14, 2026

Copy link
Copy Markdown

Hydra reported failures for this PR.

Failed builds:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants