From 8b996ed0d1f4be028bf7d423bd825754772913a8 Mon Sep 17 00:00:00 2001 From: harry Date: Sat, 31 Oct 2026 10:31:07 +0900 Subject: [PATCH] feat: what a client can ask a decoder and a context The additive half of #152's list: a transport that reports its losses, or sets the share of frames drawn in part from them, could not ask how many blocks of a frame had arrived; and a program could learn a context's GPU only from a string meant for people. - Decoder::received_blocks and total_blocks in Rust, the browser player's received_blocks and total_blocks, and emberwave_decoder_reception in the C API, which also says whether the start of frame header arrived. - emberwave_context_info: the backend, the kind of device, the PCI vendor and device, and which call made the context, with EMBERWAVE_DEVICE_* and EMBERWAVE_CONTEXT_KIND_* constants. Both C calls fill structs that carry their size (D69), whose first sizes the test of them pins. Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42 --- ARCHITECTURE.md | 6 +- CHANGELOG.md | 6 + crates/emberwave-core/src/gpu/decoder.rs | 14 +++ crates/emberwave-ffi/README.md | 2 +- crates/emberwave-ffi/include/emberwave.h | 86 +++++++++++++ crates/emberwave-ffi/src/lib.rs | 151 +++++++++++++++++++++++ crates/emberwave-ffi/tests/api.rs | 86 +++++++++++++ crates/emberwave-ffi/tests/metal.rs | 8 ++ crates/emberwave-web/src/lib.rs | 17 +++ scripts/check_web.mjs | 2 + 10 files changed, 374 insertions(+), 4 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d0fc53f..d260ef4 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -498,7 +498,7 @@ The page paces frames by the stream's frame rate, keeping the remainder of each Without a file the page plays a demo stream it makes itself (`crates/emberwave-web/src/demo.rs`, decision D46): 60 pictures of 960x540 at 30 frames a second, two seconds that loop, of a glowing wavelet whose crests travel and embers that rise from it. The page draws each picture and encodes it with the CPU encoder within 40 kB, one picture at a time, and shows its progress in between. It then plays the PWV1 file this makes like any other. In headless Chrome on an Apple M3 Max the 60 pictures took about 2 s. A file opened or dropped meanwhile, or a sample, takes over, and `?play=NAME` starts that sample instead of the demo. -The player also decodes a stream packet by packet, as a client of a live stream receives it (#109, D55). `Player.open_stream` sets up a stream of a known size without a file, `push_packet` takes a packet, a start of frame header with blocks or blocks alone, in any order, `is_ready` says whether the frame can be drawn, whole or, if asked, the part that arrived, `draw_assembled` draws it through `Decoder::submit_assembled_to_texture`, which uploads through the queue and does not wait, and `drop_packets` lets go of what arrived of a frame and of its sequence number. The bitstream's sequence number has three bits: a packet one to three frames newer starts that frame; one four to seven frames newer, or one to four frames older, is ignored; one five to seven frames older passes for a newer frame, and one eight frames off for the current frame. So a transport keeps frame numbers of its own: it ignores packets of frames older than the one being assembled, and before it pushes the first packet of a newer frame it draws the current one if it is ready, then calls `drop_packets`, never after the push, which would drop that packet too. The colours of such a stream follow each frame's start of frame header unless `set_colors` sets the matrix and the range, as for upstream's encoder, which signals full range whatever the pictures are; the chroma siting follows the header still. A loaded file's colours, siting included, are those of its PWV1 header, for `draw_next` and `draw_assembled` alike. `draw_next`, which plays a loaded file, lets go of each frame it assembled, so that packets pushed after a switch to them start afresh. `packet_ends` says where a frame splits into packets at its blocks, as `emberwave_frame_packets` does in the C API, and `frames` and `frame` hand out the loaded file's frames that can be played. With "Play as packets" (`?packets`, and `?loss=0.05` for a share lost, from 0 to 1) the page plays its stream that way: it takes each frame from the player, cuts it into packets of up to 1200 or 8000 bytes (a larger block goes alone), shuffles them, drops the share chosen, and draws what arrived. A frame of which every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced arrived is drawn with the lost finer blocks blank, and other frames are skipped, the last picture staying; `set_partial_share` sets another share than 90 percent, and the page offers 70 and 50 percent (`?partial=0.7`, D59). With `set_concealment(true)`, off by default, the lost finer blocks keep the last frame's detail where the picture is still instead of going blank (D66). Until the header arrives the count is of all the picture's blocks, so a frame that lost its header is skipped unless its encoder left out few blocks. An encoder that leaves out empty blocks of the coarsest level, as upstream's does and emberwave's did before decision D58, leaves such frames nothing to draw in part. No loss reaches the next frame. In headless Chrome on the M3 Max the demo's frames took 38 packets each; with 5 % of them lost, about one frame in six was drawn whole, half in part and the rest skipped. +The player also decodes a stream packet by packet, as a client of a live stream receives it (#109, D55). `Player.open_stream` sets up a stream of a known size without a file, `push_packet` takes a packet, a start of frame header with blocks or blocks alone, in any order, `is_ready` says whether the frame can be drawn, whole or, if asked, the part that arrived, `received_blocks` and `total_blocks` count what arrived of how much, `draw_assembled` draws it through `Decoder::submit_assembled_to_texture`, which uploads through the queue and does not wait, and `drop_packets` lets go of what arrived of a frame and of its sequence number. The bitstream's sequence number has three bits: a packet one to three frames newer starts that frame; one four to seven frames newer, or one to four frames older, is ignored; one five to seven frames older passes for a newer frame, and one eight frames off for the current frame. So a transport keeps frame numbers of its own: it ignores packets of frames older than the one being assembled, and before it pushes the first packet of a newer frame it draws the current one if it is ready, then calls `drop_packets`, never after the push, which would drop that packet too. The colours of such a stream follow each frame's start of frame header unless `set_colors` sets the matrix and the range, as for upstream's encoder, which signals full range whatever the pictures are; the chroma siting follows the header still. A loaded file's colours, siting included, are those of its PWV1 header, for `draw_next` and `draw_assembled` alike. `draw_next`, which plays a loaded file, lets go of each frame it assembled, so that packets pushed after a switch to them start afresh. `packet_ends` says where a frame splits into packets at its blocks, as `emberwave_frame_packets` does in the C API, and `frames` and `frame` hand out the loaded file's frames that can be played. With "Play as packets" (`?packets`, and `?loss=0.05` for a share lost, from 0 to 1) the page plays its stream that way: it takes each frame from the player, cuts it into packets of up to 1200 or 8000 bytes (a larger block goes alone), shuffles them, drops the share chosen, and draws what arrived. A frame of which every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced arrived is drawn with the lost finer blocks blank, and other frames are skipped, the last picture staying; `set_partial_share` sets another share than 90 percent, and the page offers 70 and 50 percent (`?partial=0.7`, D59). With `set_concealment(true)`, off by default, the lost finer blocks keep the last frame's detail where the picture is still instead of going blank (D66). Until the header arrives the count is of all the picture's blocks, so a frame that lost its header is skipped unless its encoder left out few blocks. An encoder that leaves out empty blocks of the coarsest level, as upstream's does and emberwave's did before decision D58, leaves such frames nothing to draw in part. No loss reaches the next frame. In headless Chrome on the M3 Max the demo's frames took 38 packets each; with 5 % of them lost, about one frame in six was drawn whole, half in part and the rest skipped. `web/bench.html` times the decoding on the device it runs on, a phone included, with the code of `crates/emberwave-web/src/bench.rs`. It makes test streams at 1080p, 1440p and 4K, four pictures each of a synthetic scene encoded by the CPU encoder in the page at the budgets of the native benchmarks, then decodes them in turn on the player's path, to RGB in GPU memory, and waits for each frame through `onSubmittedWorkDone` before the next: 300 frames after at least 30 and a second of warm-up, as `emberwave-cli bench --output rgb` does natively. Then it decodes them again 60 a second, the warm-up too, on a schedule that the page's timers keep, as the benchmark app does natively at a stream's pace (#95). wgpu's callback for that has to be `Send`, which a JavaScript function is not, so it sets a flag and wakes a future that `wasm_bindgen_futures` turns into the page's promise. Browsers coarsen `performance.now()`, to a millisecond in some, so the mean comes from the time of the whole run and the median and 99th percentile, which are per frame, are as fine as the timer, whose step the report states. The times include what WebGPU adds over native wgpu: the browser's checks, its process boundary and the delay of the report. A GPU error the browser reports during a size fails that size rather than give times for frames that were not decoded, a run during which the page was hidden is marked as not valid, and the benchmark lets go of its device when the run ends. @@ -519,9 +519,9 @@ A decoder or an encoder holds the device it was made on, so it may outlive its c | Function | What it does | |---|---| | `emberwave_probe_pwv1`, `emberwave_probe_packet` | Picture size, chroma format and colour signalling from a PWV1 file header or from a packet that starts with a start of frame header | -| `emberwave_context_create`, `_destroy`, `_adapter` | Choose a GPU (backends, software only) and name it | +| `emberwave_context_create`, `_destroy`, `_adapter`, `_info` | Choose a GPU (backends, software only), name it, and describe it: backend, kind of device, PCI vendor and device, and which call made the context | | `emberwave_decoder_create`, `_destroy`, `_info` | One decoder per picture size and chroma format; the sizes of its planes | -| `emberwave_decoder_push_packet`, `_is_ready`, `_clear`, `_set_partial_share` | Feed packets as they arrive; ask whether the frame, or enough of it, is there; set the share of its blocks a frame decoded in part needs (D59) | +| `emberwave_decoder_push_packet`, `_is_ready`, `_reception`, `_clear`, `_set_partial_share` | Feed packets as they arrive; ask whether the frame, or enough of it, is there; set the share of its blocks a frame decoded in part needs (D59); how many blocks of the frame arrived, of how many | | `emberwave_decoder_decode_yuv`, `_decode_rgb` | Decode and copy the planes, or RGB, into the caller's memory, row by row with the caller's strides | | `emberwave_texture_create`, `_destroy`, `emberwave_decoder_decode_texture` | A texture of the picture's size, RGBA or BGRA, on the decoder's device, and a decode that draws the RGB picture into it and waits | | `emberwave_texture_create_plane`, `emberwave_decoder_decode_plane_textures` | A one-channel texture for the Y, the Cb or the Cr plane, and a decode that puts the planes into three and waits: on Metal it stores them there, elsewhere it copies them in (D52) | diff --git a/CHANGELOG.md b/CHANGELOG.md index 328d5e1..1f78218 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ All notable changes to emberwave. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow [Semantic Versioning](https://semver.org/). +## [Unreleased] + +### Added + +- What a client can ask (#152): `emberwave_decoder_reception` in the C API, `Decoder::received_blocks` and `total_blocks` in Rust, and the browser player's `received_blocks` and `total_blocks` count the blocks of the frame being assembled that arrived, of how many, for a transport that reports its losses. `emberwave_context_info` describes a context's GPU: its backend, the kind of device, its PCI vendor and device, and which call made the context, with the new `EMBERWAVE_DEVICE_*` and `EMBERWAVE_CONTEXT_KIND_*` constants. Both fill structs that carry their size (decision D69). + ## [0.17.0] - 2026-10-31 The C and Rust APIs, readied to stay compatible from 1.0 on, and broken once here for that. The C API's structs carry their size, so that a later version can add a member without breaking programs; its planes carry their capacity, which the calls check; it has statuses for a call the system lacks and for a buffer too small, and versions to compare the header with the library. The Rust API's enums and structs can grow without breaking callers. The changes below say what each caller changes. diff --git a/crates/emberwave-core/src/gpu/decoder.rs b/crates/emberwave-core/src/gpu/decoder.rs index 2172947..49bfc06 100644 --- a/crates/emberwave-core/src/gpu/decoder.rs +++ b/crates/emberwave-core/src/gpu/decoder.rs @@ -973,6 +973,20 @@ impl Decoder { self.assembler.start_of_frame() } + /// How many distinct blocks of the frame being received have arrived: + /// for a transport that reports its losses, or sets the share of + /// [`set_partial_share`](Self::set_partial_share) from them. + pub fn received_blocks(&self) -> u32 { + self.assembler.received_blocks() + } + + /// How many blocks the frame being received has: as many as its start of + /// frame header announced, or, until that arrives, as many as the + /// picture can hold. + pub fn total_blocks(&self) -> u32 { + self.assembler.total_blocks() + } + /// Decodes the frame assembled from [`push_packet`](Self::push_packet) /// calls. Blocks that did not arrive decode as zeros, or are concealed /// ([`set_concealment`](Self::set_concealment)). diff --git a/crates/emberwave-ffi/README.md b/crates/emberwave-ffi/README.md index 8eb0f5f..86e3c91 100644 --- a/crates/emberwave-ffi/README.md +++ b/crates/emberwave-ffi/README.md @@ -541,7 +541,7 @@ CFRelease(bgra); - **Waiting.** `emberwave_decoder_decode_yuv` and `emberwave_decoder_decode_rgb` wait for the GPU and copy the picture to the CPU. On an Apple M3 Max a 4K frame takes 1.2 ms to planes and 2.6 to 2.7 ms to RGB that way, against 0.9 and 1.2 ms when the picture stays on the GPU. A call that fails may have written some rows. `emberwave_decoder_decode_texture` and `emberwave_decoder_decode_plane_textures` wait as well, and leave the picture on the GPU, in textures for Metal (see [Pictures on the GPU: Metal](#pictures-on-the-gpu-metal)). On a context of the client's own Metal queue, no call that waits may be made inside a completion handler of a command buffer of that queue: the wait never ends there. That includes the destroy functions, which wait when they let go of the last hold on the queue. - **Panics.** A panic inside emberwave is caught and reported as `EMBERWAVE_STATUS_INTERNAL`; destroy the object it happened in. - **Packets.** `emberwave_decoder_push_packet` takes one packet or several back to back, in any order, and ignores a block that arrived already. Frames are told apart by the bitstream's sequence number, which has three bits: a packet of a frame one to three frames newer discards what was received of the older one and starts its own; one four to seven frames newer, or one to four frames older, is ignored; one five to seven frames older passes for a newer frame, and one eight frames off for the same frame. So a transport keeps frame numbers of its own: it drops packets of frames older than the one being received, and before it gives the first packet of a newer frame it decodes the current one if it is ready, then calls `emberwave_decoder_clear`, not after. -- **Partial frames.** `emberwave_decoder_is_ready` with `allow_partial_frame` accepts a frame that lost packets once every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced have arrived, for a picture despite the loss; `emberwave_decoder_set_partial_share` sets another share, lower to draw frames that lost more, with more of their detail blank, or 1 for whole frames only (decision D59). `emberwave_decoder_set_concealment(decoder, 1)` fills the blocks of detail such a frame lacks with the last frame's where the picture is still, so that a still picture keeps its detail, and `0` turns it off again, as it is by default (decision D66). Until that header arrives the count is of all the picture's blocks. emberwave's encoder sends every block of the coarsest level for this; upstream's leaves out the empty ones, so a frame of it that lost any packet may never be ready in part (decision D58). +- **Partial frames.** `emberwave_decoder_is_ready` with `allow_partial_frame` accepts a frame that lost packets once every block of the coarsest level and more than 90 percent of the blocks its start of frame header announced have arrived, for a picture despite the loss; `emberwave_decoder_set_partial_share` sets another share, lower to draw frames that lost more, with more of their detail blank, or 1 for whole frames only (decision D59). `emberwave_decoder_reception` says how many blocks of the frame have arrived, of how many, for a transport that reports its losses or sets the share from them. `emberwave_decoder_set_concealment(decoder, 1)` fills the blocks of detail such a frame lacks with the last frame's where the picture is still, so that a still picture keeps its detail, and `0` turns it off again, as it is by default (decision D66). Until that header arrives the count is of all the picture's blocks. emberwave's encoder sends every block of the coarsest level for this; upstream's leaves out the empty ones, so a frame of it that lost any packet may never be ready in part (decision D58). - **Encoding.** `emberwave_encoder_encode` reads planes with a stride of at least their width (`emberwave_encoder_info` gives the sizes), `emberwave_encoder_encode_rgb` a picture in RGB with a stride of at least four bytes a pixel, and `emberwave_encoder_encode_textures` reads either from textures on the GPU (see [Pictures to encode on the GPU](#pictures-to-encode-on-the-gpu)), as `_vulkan` and `_d3d12` do with fences or semaphores. Both write the frame, at most `budget` bytes, into a buffer of that many bytes the caller provides, and wait for the GPU. A budget below the smallest frame is `EMBERWAVE_STATUS_BUFFER_TOO_SMALL` with the smallest frame's size in `size`, and encodes nothing. The frame's sequence number, a three-bit counter from 1 that wraps around, advances only with frames that were encoded. `emberwave_frame_packets` splits a frame at its blocks; a block larger than the packet size gets a packet of its own. - **Signalling.** `EmberwaveSignalling` sets what every frame says about its colours: matrix (BT.709 or BT.2020; the format cannot signal BT.601), range, chroma siting, primaries and transfer function (sRGB or PQ). `NULL` means BT.709, video range, centred chroma and sRGB. - **Colours.** With a `NULL` colour, `emberwave_decoder_decode_rgb` converts the way the frame's start of frame header says. Upstream's Metal encoder writes BT.709, full range and centred chroma there whatever the pictures are, so pass what you know, such as the colours of a PWV1 file's header from `emberwave_probe_pwv1`. diff --git a/crates/emberwave-ffi/include/emberwave.h b/crates/emberwave-ffi/include/emberwave.h index ff44332..0bdfe1f 100644 --- a/crates/emberwave-ffi/include/emberwave.h +++ b/crates/emberwave-ffi/include/emberwave.h @@ -78,6 +78,36 @@ // Only a software adapter will do: Mesa's lavapipe, or WARP on Windows. #define EMBERWAVE_CONTEXT_SOFTWARE 1 +// A GPU of a kind wgpu does not name. +#define EMBERWAVE_DEVICE_OTHER 0 + +// A GPU that shares the CPU's memory, such as Apple silicon's. +#define EMBERWAVE_DEVICE_INTEGRATED 1 + +// A GPU with memory of its own. +#define EMBERWAVE_DEVICE_DISCRETE 2 + +// A virtual GPU, such as a virtual machine's. +#define EMBERWAVE_DEVICE_VIRTUAL 3 + +// A software adapter on the CPU, such as lavapipe or WARP. +#define EMBERWAVE_DEVICE_CPU 4 + +// A context of `emberwave_context_create`, on a device and queue of its own. +#define EMBERWAVE_CONTEXT_KIND_OWN 0 + +// A context of `emberwave_context_create_metal`, on the caller's Metal +// device and queue. +#define EMBERWAVE_CONTEXT_KIND_METAL 1 + +// A context of `emberwave_context_create_vulkan`, on the caller's Vulkan +// device and queue. +#define EMBERWAVE_CONTEXT_KIND_VULKAN 2 + +// A context of `emberwave_context_create_d3d12`, on a device of its own on +// the adapter the caller named, for textures it shares. +#define EMBERWAVE_CONTEXT_KIND_D3D12 3 + // The ITU-R BT.601 matrix. #define EMBERWAVE_MATRIX_BT601 0 @@ -490,6 +520,25 @@ typedef struct EmberwaveD3D12Sync { uint64_t signal_value; } EmberwaveD3D12Sync; +// The GPU a context runs on, from `emberwave_context_info`. +typedef struct EmberwaveContextInfo { + // `sizeof(EmberwaveContextInfo)`, set by the caller before the call + // (decision D69). + uint32_t struct_size; + // `EMBERWAVE_BACKEND_VULKAN`, `EMBERWAVE_BACKEND_METAL` or + // `EMBERWAVE_BACKEND_DX12`. + uint32_t backend; + // An `EMBERWAVE_DEVICE_*` value: what kind of GPU it is. + uint32_t device_type; + // The adapter's PCI vendor ID, or 0 where the backend gives none, as + // Metal does. + uint32_t vendor_id; + // The adapter's PCI device ID, or 0 where the backend gives none. + uint32_t device_id; + // An `EMBERWAVE_CONTEXT_KIND_*` value: which call made the context. + uint32_t kind; +} EmberwaveContextInfo; + // The sizes of the planes a decoder writes or an encoder reads. typedef struct EmberwaveFrameInfo { // `sizeof(EmberwaveFrameInfo)`, set by the caller before the call @@ -507,6 +556,21 @@ typedef struct EmberwaveFrameInfo { uint32_t chroma_height; } EmberwaveFrameInfo; +// What a decoder has received of the frame it is assembling, from +// `emberwave_decoder_reception`. +typedef struct EmberwaveReception { + // `sizeof(EmberwaveReception)`, set by the caller before the call + // (decision D69). + uint32_t struct_size; + // Distinct blocks of the frame that have arrived. + uint32_t received_blocks; + // Blocks the frame has: as many as its start of frame header announced, + // or, until that arrives, as many as the picture can hold. + uint32_t total_blocks; + // 1 once the frame's start of frame header has arrived, 0 before. + uint32_t has_start_of_frame; +} EmberwaveReception; + // Where a decoder writes one plane or picture. typedef struct EmberwavePlane { // The first byte of the first row. @@ -1195,6 +1259,15 @@ void emberwave_context_destroy(struct EmberwaveContext *context); // `context` is `NULL` or a live context. const char *emberwave_context_adapter(const struct EmberwaveContext *context); +// Describes the GPU `context` runs on, and which call made the context. +// +// # Safety +// +// `context` is `NULL` or a live context, and `info` points to an +// `EmberwaveContextInfo` to fill in, with its `struct_size` set. +enum EmberwaveStatus emberwave_context_info(const struct EmberwaveContext *context, + struct EmberwaveContextInfo *info); + // Creates a decoder on `context` for pictures of `width` x `height` pixels // in the chroma format `chroma`, an `EMBERWAVE_CHROMA_*` value. Width and // height are from 1 to 16384, and even for 4:2:0; other sizes give @@ -1278,6 +1351,19 @@ enum EmberwaveStatus emberwave_decoder_push_packet(struct EmberwaveDecoder *deco bool emberwave_decoder_is_ready(const struct EmberwaveDecoder *decoder, uint32_t allow_partial_frame); +// Describes what has arrived of the frame being assembled: how many of its +// blocks, of how many, and whether its start of frame header. For a +// transport that reports its losses, or sets the share of +// `emberwave_decoder_set_partial_share` from them. +// +// # Safety +// +// `decoder` is `NULL` or a live decoder that no other thread is using, and +// `reception` points to an `EmberwaveReception` to fill in, with its +// `struct_size` set. +enum EmberwaveStatus emberwave_decoder_reception(const struct EmberwaveDecoder *decoder, + struct EmberwaveReception *reception); + // Sets the share of a frame's announced blocks that must have arrived, with // every block of the coarsest level, for `emberwave_decoder_is_ready` with // `allow_partial_frame` to accept a frame that lost packets: 0.9 by diff --git a/crates/emberwave-ffi/src/lib.rs b/crates/emberwave-ffi/src/lib.rs index 291daf8..1261ba1 100644 --- a/crates/emberwave-ffi/src/lib.rs +++ b/crates/emberwave-ffi/src/lib.rs @@ -146,6 +146,29 @@ pub const EMBERWAVE_BACKEND_DX12: u32 = 4; /// Only a software adapter will do: Mesa's lavapipe, or WARP on Windows. pub const EMBERWAVE_CONTEXT_SOFTWARE: u32 = 1; +/// A GPU of a kind wgpu does not name. +pub const EMBERWAVE_DEVICE_OTHER: u32 = 0; +/// A GPU that shares the CPU's memory, such as Apple silicon's. +pub const EMBERWAVE_DEVICE_INTEGRATED: u32 = 1; +/// A GPU with memory of its own. +pub const EMBERWAVE_DEVICE_DISCRETE: u32 = 2; +/// A virtual GPU, such as a virtual machine's. +pub const EMBERWAVE_DEVICE_VIRTUAL: u32 = 3; +/// A software adapter on the CPU, such as lavapipe or WARP. +pub const EMBERWAVE_DEVICE_CPU: u32 = 4; + +/// A context of `emberwave_context_create`, on a device and queue of its own. +pub const EMBERWAVE_CONTEXT_KIND_OWN: u32 = 0; +/// A context of `emberwave_context_create_metal`, on the caller's Metal +/// device and queue. +pub const EMBERWAVE_CONTEXT_KIND_METAL: u32 = 1; +/// A context of `emberwave_context_create_vulkan`, on the caller's Vulkan +/// device and queue. +pub const EMBERWAVE_CONTEXT_KIND_VULKAN: u32 = 2; +/// A context of `emberwave_context_create_d3d12`, on a device of its own on +/// the adapter the caller named, for textures it shares. +pub const EMBERWAVE_CONTEXT_KIND_D3D12: u32 = 3; + /// The ITU-R BT.601 matrix. pub const EMBERWAVE_MATRIX_BT601: u32 = 0; /// The ITU-R BT.709 matrix. @@ -270,6 +293,44 @@ pub struct EmberwaveFrameInfo { pub chroma_height: u32, } +/// The GPU a context runs on, from `emberwave_context_info`. +#[repr(C)] +#[derive(Debug, Clone, Copy)] +pub struct EmberwaveContextInfo { + /// `sizeof(EmberwaveContextInfo)`, set by the caller before the call + /// (decision D69). + pub struct_size: u32, + /// `EMBERWAVE_BACKEND_VULKAN`, `EMBERWAVE_BACKEND_METAL` or + /// `EMBERWAVE_BACKEND_DX12`. + pub backend: u32, + /// An `EMBERWAVE_DEVICE_*` value: what kind of GPU it is. + pub device_type: u32, + /// The adapter's PCI vendor ID, or 0 where the backend gives none, as + /// Metal does. + pub vendor_id: u32, + /// The adapter's PCI device ID, or 0 where the backend gives none. + pub device_id: u32, + /// An `EMBERWAVE_CONTEXT_KIND_*` value: which call made the context. + pub kind: u32, +} + +/// What a decoder has received of the frame it is assembling, from +/// `emberwave_decoder_reception`. +#[repr(C)] +#[derive(Debug, Clone, Copy)] +pub struct EmberwaveReception { + /// `sizeof(EmberwaveReception)`, set by the caller before the call + /// (decision D69). + pub struct_size: u32, + /// Distinct blocks of the frame that have arrived. + pub received_blocks: u32, + /// Blocks the frame has: as many as its start of frame header announced, + /// or, until that arrives, as many as the picture can hold. + pub total_blocks: u32, + /// 1 once the frame's start of frame header has arrived, 0 before. + pub has_start_of_frame: u32, +} + /// Where a decoder writes one plane or picture. #[repr(C)] #[derive(Debug, Clone, Copy)] @@ -778,6 +839,11 @@ versioned!( EmberwaveVulkanImage { first_ends_with: user: *mut c_void, ends_with: user: *mut c_void }, EmberwaveVulkanSync { first_ends_with: signal_value: u64, ends_with: signal_value: u64 }, EmberwaveD3D12Sync { first_ends_with: signal_value: u64, ends_with: signal_value: u64 }, + EmberwaveContextInfo { first_ends_with: kind: u32, ends_with: kind: u32 }, + EmberwaveReception { + first_ends_with: has_start_of_frame: u32, + ends_with: has_start_of_frame: u32 + }, ); /// The `struct_size` of the struct at `ptr`, `what` to the caller, checked @@ -4068,6 +4134,60 @@ pub unsafe extern "C" fn emberwave_context_adapter( } } +/// Describes the GPU `context` runs on, and which call made the context. +/// +/// # Safety +/// +/// `context` is `NULL` or a live context, and `info` points to an +/// `EmberwaveContextInfo` to fill in, with its `struct_size` set. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn emberwave_context_info( + context: *const EmberwaveContext, + info: *mut EmberwaveContextInfo, +) -> EmberwaveStatus { + run(|| { + // SAFETY: as the caller promises. + let Some(context) = (unsafe { context.as_ref() }) else { + return Err(Failure::argument("context is NULL")); + }; + let adapter = context.context.adapter_info(); + let described = EmberwaveContextInfo { + backend: match adapter.backend { + wgpu::Backend::Vulkan => EMBERWAVE_BACKEND_VULKAN, + wgpu::Backend::Metal => EMBERWAVE_BACKEND_METAL, + wgpu::Backend::Dx12 => EMBERWAVE_BACKEND_DX12, + other => { + return Err(Failure( + EmberwaveStatus::Internal, + format!("a context on {other:?}, which emberwave does not use"), + )); + } + }, + device_type: match adapter.device_type { + wgpu::DeviceType::IntegratedGpu => EMBERWAVE_DEVICE_INTEGRATED, + wgpu::DeviceType::DiscreteGpu => EMBERWAVE_DEVICE_DISCRETE, + wgpu::DeviceType::VirtualGpu => EMBERWAVE_DEVICE_VIRTUAL, + wgpu::DeviceType::Cpu => EMBERWAVE_DEVICE_CPU, + wgpu::DeviceType::Other => EMBERWAVE_DEVICE_OTHER, + }, + vendor_id: adapter.vendor, + device_id: adapter.device, + kind: if context.shared_queue { + EMBERWAVE_CONTEXT_KIND_METAL + } else if context.vulkan { + EMBERWAVE_CONTEXT_KIND_VULKAN + } else if context.d3d12 { + EMBERWAVE_CONTEXT_KIND_D3D12 + } else { + EMBERWAVE_CONTEXT_KIND_OWN + }, + ..Default::default() + }; + // SAFETY: as the caller promises. + unsafe { write_struct(info, "info", described) } + }) +} + /// Creates a decoder on `context` for pictures of `width` x `height` pixels /// in the chroma format `chroma`, an `EMBERWAVE_CHROMA_*` value. Width and /// height are from 1 to 16384, and even for 4:2:0; other sizes give @@ -4226,6 +4346,35 @@ pub unsafe extern "C" fn emberwave_decoder_is_ready( } } +/// Describes what has arrived of the frame being assembled: how many of its +/// blocks, of how many, and whether its start of frame header. For a +/// transport that reports its losses, or sets the share of +/// `emberwave_decoder_set_partial_share` from them. +/// +/// # Safety +/// +/// `decoder` is `NULL` or a live decoder that no other thread is using, and +/// `reception` points to an `EmberwaveReception` to fill in, with its +/// `struct_size` set. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn emberwave_decoder_reception( + decoder: *const EmberwaveDecoder, + reception: *mut EmberwaveReception, +) -> EmberwaveStatus { + run(|| { + // SAFETY: as the caller promises. + let decoder = &unsafe { live(decoder) }?.decoder; + let described = EmberwaveReception { + received_blocks: decoder.received_blocks(), + total_blocks: decoder.total_blocks(), + has_start_of_frame: u32::from(decoder.start_of_frame().is_some()), + ..Default::default() + }; + // SAFETY: as the caller promises. + unsafe { write_struct(reception, "reception", described) } + }) +} + /// Sets the share of a frame's announced blocks that must have arrived, with /// every block of the coarsest level, for `emberwave_decoder_is_ready` with /// `allow_partial_frame` to accept a frame that lost packets: 0.9 by @@ -5790,6 +5939,8 @@ mod tests { assert_eq!(EmberwaveVulkanImage::FIRST_SIZE, 48); assert_eq!(EmberwaveVulkanSync::FIRST_SIZE, 40); assert_eq!(EmberwaveD3D12Sync::FIRST_SIZE, 40); + assert_eq!(EmberwaveContextInfo::FIRST_SIZE, 24); + assert_eq!(EmberwaveReception::FIRST_SIZE, 16); } #[test] diff --git a/crates/emberwave-ffi/tests/api.rs b/crates/emberwave-ffi/tests/api.rs index 37b2a0d..890832f 100644 --- a/crates/emberwave-ffi/tests/api.rs +++ b/crates/emberwave-ffi/tests/api.rs @@ -1312,3 +1312,89 @@ fn structs_carry_their_size_and_planes_theirs() { emberwave_context_destroy(context); } } + +#[test] +fn a_decoder_tells_what_arrived_and_a_context_its_gpu() { + let Some(context) = context() else { + return; + }; + // SAFETY: a live context, NULL where the test means it, and structs as + // large as their sizes say. + unsafe { + let mut info = EmberwaveContextInfo::default(); + assert_eq!( + emberwave_context_info(context, &mut info), + EmberwaveStatus::Ok + ); + assert!( + [ + EMBERWAVE_BACKEND_VULKAN, + EMBERWAVE_BACKEND_METAL, + EMBERWAVE_BACKEND_DX12 + ] + .contains(&info.backend), + "{info:?}" + ); + assert!(info.device_type <= EMBERWAVE_DEVICE_CPU, "{info:?}"); + assert_eq!(info.kind, EMBERWAVE_CONTEXT_KIND_OWN); + assert_eq!( + emberwave_context_info(ptr::null(), &mut info), + EmberwaveStatus::InvalidArgument + ); + } + + let bytes = testdata("gradient_200x120_420.pwv1"); + let decoder = decoder_for(context, &bytes); + let frame = frames(&bytes)[0]; + // The frame's packets: the start of frame header and its first blocks, + // then the rest. + let ends = emberwave_core::encode::packets(frame, 64); + assert!(ends.len() > 2, "{}", ends.len()); + // SAFETY: a live decoder, packets of its frame, and structs as large as + // their sizes say. + unsafe { + let mut reception = EmberwaveReception::default(); + assert_eq!( + emberwave_decoder_reception(decoder, &mut reception), + EmberwaveStatus::Ok + ); + assert_eq!( + (reception.received_blocks, reception.has_start_of_frame), + (0, 0) + ); + let header_first = ends[0]; + assert_eq!( + emberwave_decoder_push_packet(decoder, header_first.as_ptr(), header_first.len()), + EmberwaveStatus::Ok + ); + assert_eq!( + emberwave_decoder_reception(decoder, &mut reception), + EmberwaveStatus::Ok + ); + assert_eq!(reception.has_start_of_frame, 1); + let first = reception.received_blocks; + assert!(first < reception.total_blocks, "{reception:?}"); + for packet in &ends[1..] { + assert_eq!( + emberwave_decoder_push_packet(decoder, packet.as_ptr(), packet.len()), + EmberwaveStatus::Ok + ); + } + assert_eq!( + emberwave_decoder_reception(decoder, &mut reception), + EmberwaveStatus::Ok + ); + assert_eq!(reception.received_blocks, reception.total_blocks); + assert!(reception.received_blocks > first); + let mut short = EmberwaveReception { + struct_size: 4, + ..Default::default() + }; + assert_eq!( + emberwave_decoder_reception(decoder, &mut short), + EmberwaveStatus::InvalidArgument + ); + emberwave_decoder_destroy(decoder); + emberwave_context_destroy(context); + } +} diff --git a/crates/emberwave-ffi/tests/metal.rs b/crates/emberwave-ffi/tests/metal.rs index 965dd0d..3a91802 100644 --- a/crates/emberwave-ffi/tests/metal.rs +++ b/crates/emberwave-ffi/tests/metal.rs @@ -379,6 +379,14 @@ fn decodes_on_the_app_queue_need_no_wait() { ptr::eq(on.cast_const().cast(), Retained::as_ptr(&device)), "the context is on the app's device" ); + let mut info = EmberwaveContextInfo::default(); + // SAFETY: a live context and a struct as large as its size says. + let status = unsafe { emberwave_context_info(context, &mut info) }; + assert_eq!(status, EmberwaveStatus::Ok, "{}", last_error()); + assert_eq!( + (info.backend, info.kind), + (EMBERWAVE_BACKEND_METAL, EMBERWAVE_CONTEXT_KIND_METAL) + ); // 192 pixels wide, the picture is copied into its texture; 200, drawn // (decision D42). for name in ["noise_192x136_420.pwv1", "gradient_200x120_420.pwv1"] { diff --git a/crates/emberwave-web/src/lib.rs b/crates/emberwave-web/src/lib.rs index 293eb8b..6e0bbf9 100644 --- a/crates/emberwave-web/src/lib.rs +++ b/crates/emberwave-web/src/lib.rs @@ -448,6 +448,23 @@ impl Player { .is_some_and(|stream| stream.decoder.is_ready(allow_partial)) } + /// How many distinct blocks of the frame being assembled have arrived, + /// 0 without a stream: for a page that reports its losses. + pub fn received_blocks(&self) -> u32 { + self.stream + .as_ref() + .map_or(0, |stream| stream.decoder.received_blocks()) + } + + /// How many blocks the frame being assembled has: as many as its start + /// of frame header announced, or, until that arrives, as many as the + /// picture can hold; 0 without a stream. + pub fn total_blocks(&self) -> u32 { + self.stream + .as_ref() + .map_or(0, |stream| stream.decoder.total_blocks()) + } + /// Sets the share of a frame's announced blocks that must have arrived, /// with every block of the coarsest level, for `is_ready(true)` and /// `draw_assembled` to take a frame that lost packets: 0.9 by default, diff --git a/scripts/check_web.mjs b/scripts/check_web.mjs index 6410dd7..368ae5c 100644 --- a/scripts/check_web.mjs +++ b/scripts/check_web.mjs @@ -21,6 +21,8 @@ for (const [name, value] of [ ["Player.max_file_bytes", Player?.max_file_bytes], ["Player.prototype.set_partial_share", Player?.prototype.set_partial_share], ["Player.prototype.set_concealment", Player?.prototype.set_concealment], + ["Player.prototype.received_blocks", Player?.prototype.received_blocks], + ["Player.prototype.total_blocks", Player?.prototype.total_blocks], ["Bench.create", Bench?.create], ["Demo", Demo], ["Demo.frames", Demo?.frames],