[linson007]

Local evaluation — 2026-10-23

Closing after testing a working prototype against main 3135ab45b3da525a8a2dbb8a1aa72ce095dda8a9 on an M4 MacBook Air, 24 GB, macOS 27.0.1, using the installed Ternary Bonsai PQ2_0 27B target and matching DFlash2 draft. The API can be integrated, but these tests did not establish a useful performance improvement.

The prototype joins/starts a per-thread AudioWorkInterval around Runtime::prefillAsync / decodeAsync graph preparation and submission, then finishes/leaves before the explicit GPU wait. Its deadline is 10 ms. Kernels, arithmetic, protocol and weights are identical between modes. Apple documents this API for real-time audio work; successful calls do not establish guaranteed P-core placement. The tested native processes already requested QOS_CLASS_USER_INTERACTIVE.

Four off/audio/audio/off process rounds collected 31 samples per B1–B4 width per round; output token hashes matched across all four rounds. Thermal drift made B1–B3 process-round cycle timings inconclusive. A tighter same-process test then alternated off/audio/audio/off every four cycles, collecting 32 samples per mode per width:

Width	Host control, ms	Host WorkInterval, ms	Whole-cycle median change
B1	0.701	3.147	+1.16%
B2	0.720	2.679	-0.04%
B3	0.763	1.027	-0.65%
B4	0.792	2.151	+0.18%

Positive change means slower. Host submission became 1.35–4.49× slower, with no consistent whole-cycle gain. Host submission time, GPU spans and gaps between GPU commands were measured separately.

Four matched HTTP server rounds also tested 64-token generation, cold 512-token prefill, cached replay and four concurrent requests. Sequential response hashes matched; every burst completed and every server shut down cleanly, with zero WorkInterval failures or remaining active scopes. HTTP ABBA spreads were 10–26%, above the repository's 5% limit, so those timings are inconclusive, not evidence of a speedup or a quantified regression. Memory admission limited burst concurrency, so the native runs provide the fixed-width B1–B4 evidence.

Normal exit, nested scopes and exception cleanup passed in off/audio/qos modes. Production and instrumented benchmark builds and patch checks passed. The full test/sanitizer suites were not run because no production patch is proposed.

The prototype and raw measurements remain local; no implementation or default change is being published. These results do not justify a replacement performance PR on this machine. Issue #193 remains open for future investigation under other hardware or launch conditions.

---

Original RFC description and historical validation:

Commit attribution was corrected to linson007 with a metadata-only history rewrite. File trees are unchanged; historical benchmark source hashes and CI links below retain their original recorded scope.

This is a documentation-only WorkInterval RFC against main 1482f6bbc2abf12375332725302019594150b6cb, related to #193. No scheduling API, active environment flag or performance improvement is implemented in this PR.

Main already reduces per-submit work, batches bindings and provides asynchronous command/host-step execution. Its host-step worker requests QOS_CLASS_USER_INTERACTIVE; an engine-thread WorkInterval scope remains a separate proposal. The RFC now reflects those current paths and the maintainer's request to measure after the existing host improvements.

The unused SPLASH_ENGINE_WORK_INTERVAL configuration header and string-only tests are removed from the proposal. The net diff is one documentation file, covering bounded scope/cleanup, blocking waits, ownership, matched controls, host/GPU timing separation and serving-level validation. Third-party timings are not presented as measurements of current Splash, and a scheduling hint is not presented as guaranteed P-core placement or exact mixed-serving output reproducibility.

Merge commit 7ccef7bc contains current main without rewriting the original branch. Ahead/behind checks show zero commits behind main. Local documentation links and full-patch whitespace checks pass. Fresh current-head CI passes all four ordinary jobs: production/CPU runtime, every native build and sanitizers, and Python 3.12, 3.13 and 3.14 tests/source checks. The opt-in release hardware gate is skipped. The Python 3.14 job runs 1,318 tests with four skips; lint and formatting pass. This RFC changes no runtime, kernel, protocol, memory plan or product default; it is independent of #487's adaptive MoE verification policy.