Skip to content

feat(html-local): add GPU video export (DO NOT MERGE) - #437

Draft
rponeawa wants to merge 15 commits into
mainfrom
html-gpu-export
Draft

rponeawa wants to merge 15 commits into
mainfrom
html-gpu-export

Conversation

@rponeawa

@rponeawa rponeawa commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

HTML video export currently captures PNGs and extracts source frames through FFmpeg. This adds an opt-in Electron path that decodes source video in Chromium, transfers composed GPU textures and encodes H.264 with WebCodecs.

DRAFT — DO NOT MERGE. The user requires this PR to remain Draft. Do not mark it ready, enable auto-merge or merge it.

Implementation

  • Select GPU export with renderer: "electron". Runtime preparation installs pinned Electron 45.0.0 for the host platform, checks the official archive SHA-256, stages extraction, checks the executable version and publishes the installation under a shared cache lock. Healthy installations are reused. Explicit executables must match the pinned release.
  • Package Mediabunny 1.62.1 as a browser bundle with its original source and MPL-2.0 license. Independent packages include the worker scripts and browser assets without exporting ambient WebCodecs types to consumers.
  • Give each active video layer an independent decoder cursor while sharing input/demux state and a bounded sample cache. Divide reverse windows across active layers and close inactive cursors. Preserve reverse playback, holds, loops, fractional rates and rational timeline sampling.
  • Check a renderer-owned frame identifier before accepting shared textures. Serialize native texture import handshakes by ready ordinal; receiver snapshot copying and encoding remain concurrent. Feed an ordered bounded queue and reserve admission for the next required frame.
  • Default retained source samples to 2.25 GiB and pending-frame admission to 128 MiB, with explicit overrides. Video projects use one GPU scene automatically; pure MG retains automatic worker selection. Sample allocation, RSS, physical footprint and total device memory are distinct quantities.
  • Release source samples after capture and GPU windows after encoder flush, before muxing and full output decode validation. Preserve per-request cancellation, deadline, output-limit and process-tree cleanup.
  • On macOS, prepare a locally signed background renderer bundle with LSUIElement=true, set accessory activation before readiness and probe the version in Node mode. Explicit macOS executables must also use background bundles.
  • Add an unattended benchmark with isolated jobs, interleaved repetitions, timeout/cleanup, a single-runner lock, disk-space checks, frame-ID verification, atomic checkpoints and incremental reports. Reports separate fixtures. Resume fingerprints cover actual source, harness, fixture inputs and host, allowing documentation-only commits to preserve completed work.

Completed macOS validation

Apple M1 Pro, 32 GiB RAM, macOS 27.0, Electron 45.0.0. The renderer implementation and dependencies were verified unchanged between the interrupted and resumed measurement sessions. The completed observations span those sessions with normal desktop applications open.

  • 76 benchmark jobs passed, with 80 successful exports and 2 expected injected failures. 31,200 independent timeline/source frame observations matched, with zero mismatches and no observed descendants after completed jobs.
  • Coverage includes eight staggered/reversed layers, aligned layers, mixed-rate two-video output, complex MG, 4K output composition, source/queue overrides, worker screening from 1 through 8, an automatic-worker baseline, two/four concurrent exports, cancellation, renderer termination alongside a successful peer, and recovery.
  • Default and fastest-candidate outputs passed source/timeline identifiers. Worker candidates and automatic MG passed timeline identifiers. A 32 MiB source-cache case passed all 1,080 observations across 120 frames; four workers with a one-byte queue passed all 120 MG frame IDs.
  • The earlier automatic eight-worker shared-texture timeout did not recur in this suite. Its initiating cause remains unconfirmed.
  • Repository type checking passed. The initial local full suite had 1,121 passes, two process-test timeouts and 24 skips. Both timeouts passed isolated sequential reruns together with the Provider suite (64 passed, 11 skipped). Their initial cause is unconfirmed. Existing assertions were corrected to expect the selected memory defaults.
  • With live Chromium enabled, all 83 existing Provider tests passed with zero skips. Resuming after the report commit reused all 76 jobs without starting exports; changing the repetition count was rejected.
  • All CI checks passed at feefcbb9. The 39753c21 Linux check exhausted its 20-minute deadline while the Azure Ubuntu mirror was still downloading FFmpeg dependencies, before tests started. Both Linux installation steps now use the official Ubuntu archive with required FFmpeg dependencies. All final-head checks at 2ed24dbd passed, including both full platform checks, independent package verification, release build and CodeQL: CI run. Linux FFmpeg installation completed in 21 seconds.

Export times include Provider preparation, capture, encoding, muxing, full output decode validation and storage. Independent frame-ID checks run afterward. Memory is sampled every 250 ms across the export host and its descendants, including independent Electron process groups.

Source and queue configuration, eight staggered layers

600 frames, 1080p. Each configuration has five completed observations.

Source / pending budget Median seconds Range, seconds Median peak RSS, MiB Decoder iterator starts
1 GiB / 256 MiB 23.114 20.046 to 29.098 1049 53
2.25 GiB / 128 MiB 17.835 15.490 to 23.682 996 14
3 GiB / 128 MiB 20.835 15.709 to 37.089 930 14
4 GiB / 128 MiB 18.651 15.686 to 25.486 993 14

The current default median is 22.8% lower than the previous 1 GiB / 256 MiB configuration. The 2.25/3/4 GiB cases each used 14 iterator starts. These measurements support retaining the selected defaults; overlapping timing ranges and desktop-load variation prevent a universal-optimum claim.

Pending-frame queue, 1080p MG with four workers

Pending budget Observations Median seconds Median peak physical footprint, MiB
128 MiB 5 16.044 1840
256 MiB 5 16.259 1893
512 MiB 5 16.150 1849

MG workers

Workers Observations Median seconds Range, seconds Median peak RSS, MiB
1 1 26.855 26.855 to 26.855 896
2 1 21.917 21.917 to 21.917 1300
3 1 16.295 16.295 to 16.295 1530
4 5 20.377 14.456 to 24.258 1584
5 1 16.053 16.053 to 16.053 2085
6 5 18.265 15.083 to 21.226 2030
7 1 15.687 15.687 to 15.687 2512
8 1 17.077 17.077 to 17.077 2284
auto 5 16.690 16.103 to 18.608 2738

Automatic selection used eight scenes on this host. Finalists were selected from screening observations; the table preserves the single-observation counts for the other configurations. Retain automatic selection. Two/four workers for the staggered video case took 28.974/48.136 seconds, supporting its existing one-scene automatic strategy.

Additional output and concurrency cases

Case Frames / exports Seconds Peak process-tree RSS, MiB
video 600 frames 12.132 962
aligned 600 frames 13.349 963
mg4k 600 frames 46.725 2715
uhd 300 frames 19.586 988
concurrent-2 2 exports 28.753 1444
concurrent-4 4 exports 30.303 2702
concurrent-faults 3 exports 17.159 1637
recovery 1 exports 14.624 959

The 4K video control composes 1080p sources into 300 output frames. Scaled 4K MG exports 600 frames and retains authored Canvas raster resolutions. Injected cancellation and termination are expected failures; the peer export and recovery matched every source/timeline identifier.

Full results: benchmark report. Raw logs, case inputs, environment snapshots and memory samples remain local under ignored dist/autonomous-gpu-benchmark-v2/.

Earlier validation retained for review

  • Before the independent-cursor correction, the eight-layer fixture took 283.65 seconds at 256 MiB and 106.81 seconds at 1 GiB, with 3,304/1,283 iterator starts. The correction reduced those observations to 27.85/19.01 seconds and 190/53 starts, respectively.
  • A ten-minute 1080p30 export completed 18,000 frames and matched every timeline and both source identifiers. Repeated sequential-export and concurrent fault-isolation workloads completed; post-GC host RSS remained bounded in the corrected observer run.
  • A 300-frame 30000/1001 run with four workers and a one-byte pending budget passed 3,000 drawing/readiness observations across DOM, Web Animations, Canvas, WebGL, SVG, backgrounds, images, iframes, masks and async updates.
  • An independent npm consumer installed the actual tarballs, prepared a fresh Electron cache, completed the full two-video SVML build and matched all 600 timeline/source identifiers. Native macOS monitoring observed zero regular Dock registrations and no activation; its Runtime was stopped.
  • Installer checks covered shared-cache preparation, healthy-cache reuse, corrupt archive rejection, wrong-version rejection and staging cleanup.

Review limits

  • GPU hardware validation covers Apple Silicon macOS. Windows/Linux package CI does not validate target GPU/driver behavior; Intel Macs and other GPU combinations remain unverified.
  • Electron shared-texture APIs are experimental. Version/checksum upgrades require renewed compatibility validation.
  • Earlier isolated-consumer and external desktop-Chrome runs triggered exit watchdog cleanup. Outputs passed their checks, later runs did not reproduce the isolated-consumer incident, and the initiating cause remains unconfirmed.
  • Native encoder buffers, demuxer caches, compositor surfaces and final output storage require memory beyond logical sample/queue budgets. Process-tree RSS and physical-footprint sums do not establish exclusive device-memory consumption.
  • Chromium PNG and WebCodecs H.264 paths use different rate-control modes. Their earlier speed measurements are not bitrate/quality-normalized.

Keep this PR Draft and do not merge.

Enable opt-in Electron capture through electronPath, reuse deterministic frame preparation and source sampling, and encode ordered shared textures with WebCodecs before FFmpeg muxing. Close request-owned HTTP connections and file streams after capture.

Validated type checking, 82 existing tests, alpha fixtures with explicit browser paths, 600-frame MG sequencing, source sampling, cancellation, limits, fractional rates, and packed-code export on Apple Silicon. Windows and Linux hardware validation remains pending.
Reuse browser video decoders and bounded sample caches for forward, reverse, held, and looping sources. Composite VideoSamples through Canvas layers before shared-texture encoding. Preserve source color metadata for planar alpha samples.

Validated the 600-frame video project with independent timeline and source-frame barcodes, transparent media composition, tiny-cache reverse sampling, existing provider checks, and an unpacked package render.
Snapshot completed surfaces before ordered encoding, reserve queue capacity for the next required frame, and reuse one decoder scene for video exports while keeping pure motion graphics parallel.

Dispose source decoders explicitly and align decoder selection with browser capability checks. Verify macOS frame synchronization, source sampling, alpha, cancellation, limits, and packed exports.
Keep the exiting owner alive until signaled descendants stop, accepting zombies awaiting reaping. Bound the synchronous wait and preserve cleanup diagnostics.

Validated with the existing 15 process lifecycle checks and the provider regression suite on macOS.
SanHsien added a commit to SanHsien/hypit that referenced this pull request Oct 26, 2026
…full set

collect_new_tickets asked `gh pr|issue list` for --limit 50 and kept the items
above the watermark. The listing is newest-first, so when more than 50 tickets
had appeared since the last review the oldest unreviewed ones were dropped
without a trace: the report said "N new items" for a larger N, and raising
`reviewed_*_through` afterwards then marked the dropped ones as triaged. This
fork reviews a whole release at a time (63 commits, PRs hypit-ai#375-hypit-ai#437 in the last
one), so a long gap between reviews is the normal case, not a corner.

The limit is now 1000, and a listing that fills the limit while still entirely
above the watermark returns None ("not checked", exit 2) like every other case
where gh cannot give a complete answer.

Reproduction: tools/tests/test_check_upstream_updates.py
(test_a_listing_cut_off_above_the_watermark_is_reported_as_not_checked failed
before this change: 50 of 4900 new items were returned as the full set).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant