Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -498,7 +498,7 @@ The page paces frames by the stream's frame rate, keeping the remainder of each

Without a file the page plays a demo stream it makes itself (`crates/emberwave-web/src/demo.rs`, decision D46): 60 pictures of 960x540 at 30 frames a second, two seconds that loop, of a glowing wavelet whose crests travel and embers that rise from it. The page draws each picture and encodes it with the CPU encoder within 40 kB, one picture at a time, and shows its progress in between. It then plays the PWV1 file this makes like any other. In headless Chrome on an Apple M3 Max the 60 pictures took about 2 s. A file opened or dropped meanwhile, or a sample, takes over, and `?play=NAME` starts that sample instead of the demo.

The player also decodes a stream packet by packet, as a client of a live stream receives it (#109, D55). `Player.open_stream` sets up a stream of a known size without a file, `push_packet` takes a packet, a start of frame header with blocks or blocks alone, in any order, `is_ready` says whether the frame can be drawn, whole or, if asked, the part that arrived, `draw_assembled` draws it through `Decoder::submit_assembled_to_texture`, which uploads through the queue and does not wait, and `drop_packets` lets go of what arrived of a frame and of its sequence number. The bitstream's sequence number has three bits: a packet one to three frames newer starts that frame; one four to seven frames newer, or one to four frames older, is ignored; one five to seven frames older passes for a newer frame, and one eight frames off for the current frame. So a transport keeps frame numbers of its own: it ignores packets of frames older than the one being assembled, and before it pushes the first packet of a newer frame it draws the current one if it is ready, then calls `drop_packets`, never after the push, which would drop that packet too. The colours of such a stream follow each frame's start of frame header unless `set_colors` sets the matrix and the range, as for upstream's encoder, which signals full range whatever the pictures are; the chroma siting follows the header still. A loaded file's colours, siting included, are those of its PWV1 header, for `draw_next` and `draw_assembled` alike. `draw_next`, which plays a loaded file, lets go of each frame it assembled, so that packets pushed after a switch to them start afresh. `packet_ends` says where a frame splits into packets at its blocks, as `emberwave_frame_packets` does in the C API, and `frames` and `frame` hand out the loaded file's frames that can be played. With "Play as packets" (`?packets`, and `?loss=0.05` for a share lost, from 0 to 1) the page plays its stream that way: it takes each frame from the player, cuts it into packets of up to 1200 or 8000 bytes (a larger block goes alone), shuffles them, drops the share chosen, and draws what arrived. A frame of which every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced arrived is drawn with the lost finer blocks blank, and other frames are skipped, the last picture staying; `set_partial_share` sets another share than 90 percent, and the page offers 70 and 50 percent (`?partial=0.7`, D59). With `set_concealment(true)`, off by default, the lost finer blocks keep the last frame's detail where the picture is still instead of going blank (D66). Until the header arrives the count is of all the picture's blocks, so a frame that lost its header is skipped unless its encoder left out few blocks. An encoder that leaves out empty blocks of the coarsest level, as upstream's does and emberwave's did before decision D58, leaves such frames nothing to draw in part. No loss reaches the next frame. In headless Chrome on the M3 Max the demo's frames took 38 packets each; with 5 % of them lost, about one frame in six was drawn whole, half in part and the rest skipped.
The player also decodes a stream packet by packet, as a client of a live stream receives it (#109, D55). `Player.open_stream` sets up a stream of a known size without a file, `push_packet` takes a packet, a start of frame header with blocks or blocks alone, in any order, `is_ready` says whether the frame can be drawn, whole or, if asked, the part that arrived, `received_blocks` and `total_blocks` count what arrived of how much, `draw_assembled` draws it through `Decoder::submit_assembled_to_texture`, which uploads through the queue and does not wait, and `drop_packets` lets go of what arrived of a frame and of its sequence number. The bitstream's sequence number has three bits: a packet one to three frames newer starts that frame; one four to seven frames newer, or one to four frames older, is ignored; one five to seven frames older passes for a newer frame, and one eight frames off for the current frame. So a transport keeps frame numbers of its own: it ignores packets of frames older than the one being assembled, and before it pushes the first packet of a newer frame it draws the current one if it is ready, then calls `drop_packets`, never after the push, which would drop that packet too. The colours of such a stream follow each frame's start of frame header unless `set_colors` sets the matrix and the range, as for upstream's encoder, which signals full range whatever the pictures are; the chroma siting follows the header still. A loaded file's colours, siting included, are those of its PWV1 header, for `draw_next` and `draw_assembled` alike. `draw_next`, which plays a loaded file, lets go of each frame it assembled, so that packets pushed after a switch to them start afresh. `packet_ends` says where a frame splits into packets at its blocks, as `emberwave_frame_packets` does in the C API, and `frames` and `frame` hand out the loaded file's frames that can be played. With "Play as packets" (`?packets`, and `?loss=0.05` for a share lost, from 0 to 1) the page plays its stream that way: it takes each frame from the player, cuts it into packets of up to 1200 or 8000 bytes (a larger block goes alone), shuffles them, drops the share chosen, and draws what arrived. A frame of which every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced arrived is drawn with the lost finer blocks blank, and other frames are skipped, the last picture staying; `set_partial_share` sets another share than 90 percent, and the page offers 70 and 50 percent (`?partial=0.7`, D59). With `set_concealment(true)`, off by default, the lost finer blocks keep the last frame's detail where the picture is still instead of going blank (D66). Until the header arrives the count is of all the picture's blocks, so a frame that lost its header is skipped unless its encoder left out few blocks. An encoder that leaves out empty blocks of the coarsest level, as upstream's does and emberwave's did before decision D58, leaves such frames nothing to draw in part. No loss reaches the next frame. In headless Chrome on the M3 Max the demo's frames took 38 packets each; with 5 % of them lost, about one frame in six was drawn whole, half in part and the rest skipped.

`web/bench.html` times the decoding on the device it runs on, a phone included, with the code of `crates/emberwave-web/src/bench.rs`. It makes test streams at 1080p, 1440p and 4K, four pictures each of a synthetic scene encoded by the CPU encoder in the page at the budgets of the native benchmarks, then decodes them in turn on the player's path, to RGB in GPU memory, and waits for each frame through `onSubmittedWorkDone` before the next: 300 frames after at least 30 and a second of warm-up, as `emberwave-cli bench --output rgb` does natively. Then it decodes them again 60 a second, the warm-up too, on a schedule that the page's timers keep, as the benchmark app does natively at a stream's pace (#95). wgpu's callback for that has to be `Send`, which a JavaScript function is not, so it sets a flag and wakes a future that `wasm_bindgen_futures` turns into the page's promise. Browsers coarsen `performance.now()`, to a millisecond in some, so the mean comes from the time of the whole run and the median and 99th percentile, which are per frame, are as fine as the timer, whose step the report states. The times include what WebGPU adds over native wgpu: the browser's checks, its process boundary and the delay of the report. A GPU error the browser reports during a size fails that size rather than give times for frames that were not decoded, a run during which the page was hidden is marked as not valid, and the benchmark lets go of its device when the run ends.

Expand All @@ -519,9 +519,9 @@ A decoder or an encoder holds the device it was made on, so it may outlive its c
| Function | What it does |
|---|---|
| `emberwave_probe_pwv1`, `emberwave_probe_packet` | Picture size, chroma format and colour signalling from a PWV1 file header or from a packet that starts with a start of frame header |
| `emberwave_context_create`, `_destroy`, `_adapter` | Choose a GPU (backends, software only) and name it |
| `emberwave_context_create`, `_destroy`, `_adapter`, `_info` | Choose a GPU (backends, software only), name it, and describe it: backend, kind of device, PCI vendor and device, and which call made the context |
| `emberwave_decoder_create`, `_destroy`, `_info` | One decoder per picture size and chroma format; the sizes of its planes |
| `emberwave_decoder_push_packet`, `_is_ready`, `_clear`, `_set_partial_share` | Feed packets as they arrive; ask whether the frame, or enough of it, is there; set the share of its blocks a frame decoded in part needs (D59) |
| `emberwave_decoder_push_packet`, `_is_ready`, `_reception`, `_clear`, `_set_partial_share` | Feed packets as they arrive; ask whether the frame, or enough of it, is there; set the share of its blocks a frame decoded in part needs (D59); how many blocks of the frame arrived, of how many |
| `emberwave_decoder_decode_yuv`, `_decode_rgb` | Decode and copy the planes, or RGB, into the caller's memory, row by row with the caller's strides |
| `emberwave_texture_create`, `_destroy`, `emberwave_decoder_decode_texture` | A texture of the picture's size, RGBA or BGRA, on the decoder's device, and a decode that draws the RGB picture into it and waits |
| `emberwave_texture_create_plane`, `emberwave_decoder_decode_plane_textures` | A one-channel texture for the Y, the Cb or the Cr plane, and a decode that puts the planes into three and waits: on Metal it stores them there, elsewhere it copies them in (D52) |
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

All notable changes to emberwave. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Added

- What a client can ask (#152): `emberwave_decoder_reception` in the C API, `Decoder::received_blocks` and `total_blocks` in Rust, and the browser player's `received_blocks` and `total_blocks` count the blocks of the frame being assembled that arrived, of how many, for a transport that reports its losses. `emberwave_context_info` describes a context's GPU: its backend, the kind of device, its PCI vendor and device, and which call made the context, with the new `EMBERWAVE_DEVICE_*` and `EMBERWAVE_CONTEXT_KIND_*` constants. Both fill structs that carry their size (decision D69).

## [0.17.0] - 2026-10-31

The C and Rust APIs, readied to stay compatible from 1.0 on, and broken once here for that. The C API's structs carry their size, so that a later version can add a member without breaking programs; its planes carry their capacity, which the calls check; it has statuses for a call the system lacks and for a buffer too small, and versions to compare the header with the library. The Rust API's enums and structs can grow without breaking callers. The changes below say what each caller changes.
Expand Down
14 changes: 14 additions & 0 deletions crates/emberwave-core/src/gpu/decoder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -973,6 +973,20 @@ impl Decoder {
self.assembler.start_of_frame()
}

/// How many distinct blocks of the frame being received have arrived:
/// for a transport that reports its losses, or sets the share of
/// [`set_partial_share`](Self::set_partial_share) from them.
pub fn received_blocks(&self) -> u32 {
self.assembler.received_blocks()
}

/// How many blocks the frame being received has: as many as its start of
/// frame header announced, or, until that arrives, as many as the
/// picture can hold.
pub fn total_blocks(&self) -> u32 {
self.assembler.total_blocks()
}

/// Decodes the frame assembled from [`push_packet`](Self::push_packet)
/// calls. Blocks that did not arrive decode as zeros, or are concealed
/// ([`set_concealment`](Self::set_concealment)).
Expand Down
2 changes: 1 addition & 1 deletion crates/emberwave-ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -541,7 +541,7 @@ CFRelease(bgra);
- **Waiting.** `emberwave_decoder_decode_yuv` and `emberwave_decoder_decode_rgb` wait for the GPU and copy the picture to the CPU. On an Apple M3 Max a 4K frame takes 1.2 ms to planes and 2.6 to 2.7 ms to RGB that way, against 0.9 and 1.2 ms when the picture stays on the GPU. A call that fails may have written some rows. `emberwave_decoder_decode_texture` and `emberwave_decoder_decode_plane_textures` wait as well, and leave the picture on the GPU, in textures for Metal (see [Pictures on the GPU: Metal](#pictures-on-the-gpu-metal)). On a context of the client's own Metal queue, no call that waits may be made inside a completion handler of a command buffer of that queue: the wait never ends there. That includes the destroy functions, which wait when they let go of the last hold on the queue.
- **Panics.** A panic inside emberwave is caught and reported as `EMBERWAVE_STATUS_INTERNAL`; destroy the object it happened in.
- **Packets.** `emberwave_decoder_push_packet` takes one packet or several back to back, in any order, and ignores a block that arrived already. Frames are told apart by the bitstream's sequence number, which has three bits: a packet of a frame one to three frames newer discards what was received of the older one and starts its own; one four to seven frames newer, or one to four frames older, is ignored; one five to seven frames older passes for a newer frame, and one eight frames off for the same frame. So a transport keeps frame numbers of its own: it drops packets of frames older than the one being received, and before it gives the first packet of a newer frame it decodes the current one if it is ready, then calls `emberwave_decoder_clear`, not after.
- **Partial frames.** `emberwave_decoder_is_ready` with `allow_partial_frame` accepts a frame that lost packets once every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced have arrived, for a picture despite the loss; `emberwave_decoder_set_partial_share` sets another share, lower to draw frames that lost more, with more of their detail blank, or 1 for whole frames only (decision D59). `emberwave_decoder_set_concealment(decoder, 1)` fills the blocks of detail such a frame lacks with the last frame's where the picture is still, so that a still picture keeps its detail, and `0` turns it off again, as it is by default (decision D66). Until that header arrives the count is of all the picture's blocks. emberwave's encoder sends every block of the coarsest level for this; upstream's leaves out the empty ones, so a frame of it that lost any packet may never be ready in part (decision D58).
- **Partial frames.** `emberwave_decoder_is_ready` with `allow_partial_frame` accepts a frame that lost packets once every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced have arrived, for a picture despite the loss; `emberwave_decoder_set_partial_share` sets another share, lower to draw frames that lost more, with more of their detail blank, or 1 for whole frames only (decision D59). `emberwave_decoder_reception` says how many blocks of the frame have arrived, of how many, for a transport that reports its losses or sets the share from them. `emberwave_decoder_set_concealment(decoder, 1)` fills the blocks of detail such a frame lacks with the last frame's where the picture is still, so that a still picture keeps its detail, and `0` turns it off again, as it is by default (decision D66). Until that header arrives the count is of all the picture's blocks. emberwave's encoder sends every block of the coarsest level for this; upstream's leaves out the empty ones, so a frame of it that lost any packet may never be ready in part (decision D58).
- **Encoding.** `emberwave_encoder_encode` reads planes with a stride of at least their width (`emberwave_encoder_info` gives the sizes), `emberwave_encoder_encode_rgb` a picture in RGB with a stride of at least four bytes a pixel, and `emberwave_encoder_encode_textures` reads either from textures on the GPU (see [Pictures to encode on the GPU](#pictures-to-encode-on-the-gpu)), as `_vulkan` and `_d3d12` do with fences or semaphores. Both write the frame, at most `budget` bytes, into a buffer of that many bytes the caller provides, and wait for the GPU. A budget below the smallest frame is `EMBERWAVE_STATUS_BUFFER_TOO_SMALL` with the smallest frame's size in `size`, and encodes nothing. The frame's sequence number, a three-bit counter from 1 that wraps around, advances only with frames that were encoded. `emberwave_frame_packets` splits a frame at its blocks; a block larger than the packet size gets a packet of its own.
- **Signalling.** `EmberwaveSignalling` sets what every frame says about its colours: matrix (BT.709 or BT.2020; the format cannot signal BT.601), range, chroma siting, primaries and transfer function (sRGB or PQ). `NULL` means BT.709, video range, centred chroma and sRGB.
- **Colours.** With a `NULL` colour, `emberwave_decoder_decode_rgb` converts the way the frame's start of frame header says. Upstream's Metal encoder writes BT.709, full range and centred chroma there whatever the pictures are, so pass what you know, such as the colours of a PWV1 file's header from `emberwave_probe_pwv1`.
Expand Down
Loading
Loading