Skip to content

bench: decode beside a display loop on the Mac (#81) - #161

Merged
with-developer merged 3 commits into
mainfrom
bench/display-loop-mac
Nov 6, 2026
Merged

with-developer merged 3 commits into
mainfrom
bench/display-loop-mac

Conversation

@with-developer

@with-developer with-developer commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Purpose

Issue #81 asks how long a decode takes while a client's display loop draws at the screen's refresh. This is the Mac's half; the iPhone's needs the phone.

Changes

  • scripts/bench_app_on_mac.sh -display (and -short): a borderless window covers a screen while a CVDisplayLink and a CAMetalLayer draw the last decoded picture at the refresh. emberwave (Y, Cb, Cr textures and RGB) and HEVC (NV12) decode as frames arrive, each way without the loop and with a 60 and a 120 Hz loop in the same run. The report adds the time from a frame's arrival until the display presented it.
  • The shared benchmark code gets a PictureSink type, a rate argument for FrameClock and entry points that decode a prepared HEVC stream paced with a sink. The iOS app's behaviour does not change; the shared files typecheck for the iOS Simulator, and scripts/bench_app_on_mac.sh -autorun passes.
  • BENCHMARKS.md section "In a display loop, on the Mac", decision D72, and the READMEs' roadmap note.
  • A review (Sonnet) of the first version led to a second commit: whole beats of the stream's drift against a 60 Hz loop, a black window in the rows without a loop, a guard that stops the run where the window never reaches the screen (a sleeping or locked screen), and a few counting fixes.

Result (two runs of the first version, M3 Max, 2026-11-06)

  • Into the planes, emberwave decodes as quickly as or up to 0.8 ms quicker than HEVC into NV12 in the loop: 3.25 to 3.55 against 4.00 to 4.07 ms at 4K.
  • The loop costs emberwave 0.2 to 0.5 ms and HEVC nothing; it does not keep the GPU's clock up.
  • The time to the screen is 17 to 21 ms for every way and does not tell them apart.

Limits

  • The runs were made with the first version. Its mean time to the screen at 60 Hz depends on the start phase by about 1 ms, and its rows without a loop ran with no window on the screen; BENCHMARKS.md says so. The decode times are not affected by the first. The runs are to be repeated with the changed program once the Mac's screen is unlocked (it was locked when this was finished).
  • The stream arrives at 59.5 frames a second so that frames drift across the refreshes. The Mac's 60 Hz external monitor took no window, so the 60 Hz loop draws every second refresh of the 120 Hz screen. The Mac was not quiet (load average 4 to 7).

Testing

scripts/bench_app_on_mac.sh -display twice, in full, with the first version; -display -short while developing; scripts/bench_app_on_mac.sh -autorun after the last commit. CI builds the iOS app and runs -autorun; it never runs -display.

Refs #81

Issue #81 asks how long a decode takes inside a client's display loop,
where a real client draws the last picture at the screen's refresh while
frames arrive at a stream's pace. The benchmark app's Mac front-end gains
a -display mode: a borderless window over a screen, a CVDisplayLink and a
CAMetalLayer draw the last decoded picture at the refresh, while emberwave
(RGB texture, Y/Cb/Cr textures) and HEVC (NV12) decode as frames arrive.

Each way is also run without the loop in the same run, so the loop's cost
is a difference within one run. The report adds the time from a frame's
arrival until the display presented it (the drawable's presentedTime).

The stream runs at 59.5 frames a second so that frames drift across the
refreshes, as a stream's do against a display; frames locked to the
refresh would all show one latency. CADisplayLink stays at 60 Hz on a
120 Hz screen for an app that is not in front, so a CVDisplayLink is used.

The shared benchmark code gains only a PictureSink type, a settable frame
rate in FrameClock, and entry points that decode an HEVC stream paced with
a sink; the iOS app is not changed.

Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42
BENCHMARKS.md gets the measurements of 2026-11-06: decode times with and
without a display loop at 60 and 120 Hz, and the time to the screen, for
emberwave and HEVC. D72 records how the loop is built and why, and what was
not done (the iPhone half, a real client's pipeline). The READMEs' roadmap
notes the Mac's half of #81 as done.

Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42
A review of the display-loop benchmark found that the mean time to the screen
of the 60 Hz loop depends on the run's start phase, that the rows without a
loop ran in an application with no window on the screen, and that nothing
stopped a run on a sleeping or locked screen from printing rows as if it had
drawn. The program now times whole beats of its stream's drift against a 60 Hz
loop (119 frames, 357 for a run), keeps a black window on the screen for the
rows without a loop, keeps the displays awake, and stops where the window never
reaches the screen. The display link's statistics start with the first timed
frame, frames not drawn are counted as timed offers minus timed draws, frames
the display never presented are counted, and the HEVC stream's failures other
than "unavailable" fail the run. FrameClock takes its rate as an argument.

The docs say what the first runs were and what is to be repeated.

Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42
@with-developer
with-developer merged commit b5f9c5f into main Nov 6, 2026
7 checks passed
@with-developer
with-developer deleted the bench/display-loop-mac branch November 6, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant