diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 81dc01e..415e6f1 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,6 +1,6 @@ # Architecture -> **Status (v0.15.0).** The bitstream format is documented below and implemented by `emberwave_core::bitstream`, with a slow CPU reference decoder in `emberwave_core::cpu`, the wgpu/WGSL decoder in `emberwave_core::gpu` and the colour conversion in `emberwave_core::color`. The encoder runs on the CPU (`emberwave_core::encode`) and on the GPU (`emberwave_core::gpu`). `emberwave-cli` decodes and encodes files with them, `emberwave-web` plays them in a browser, also packet by packet as a stream arrives, and `emberwave-ffi` offers the decoder and the GPU encoder to C, also as an XCFramework for iPhone, iPad and Mac apps, and hands Metal clients the decoded picture in a texture, or its planes in three, also into the client's own textures and on its own queue without waiting, where calls take turns at its command buffers; it runs on a client's own Vulkan device and images as well, shares textures and fences with Direct3D 12 and 11 devices on Windows, and encodes a host's pictures straight from its textures, in RGB as captures hold them too; on Metal the planes are stored straight into the textures, and on Apple silicon the frames are written where the shaders read them. Decode and encode times are in `BENCHMARKS.md`. The decision log at the end is kept up to date. +> **Status (v0.16.0).** The bitstream format is documented below and implemented by `emberwave_core::bitstream`, with a slow CPU reference decoder in `emberwave_core::cpu`, the wgpu/WGSL decoder in `emberwave_core::gpu` and the colour conversion in `emberwave_core::color`. The encoder runs on the CPU (`emberwave_core::encode`) and on the GPU (`emberwave_core::gpu`). `emberwave-cli` decodes and encodes files with them, `emberwave-web` plays them in a browser, also packet by packet as a stream arrives, and `emberwave-ffi` offers the decoder and the GPU encoder to C, also as an XCFramework for iPhone, iPad and Mac apps, and hands Metal clients the decoded picture in a texture, or its planes in three, also into the client's own textures and on its own queue without waiting, where calls take turns at its command buffers; it runs on a client's own Vulkan device and images as well, shares textures and fences with Direct3D 12 and 11 devices on Windows, and encodes a host's pictures straight from its textures, in RGB as captures hold them too; on Metal the planes are stored straight into the textures, and on Apple silicon the frames are written where the shaders read them. A decoder can keep the last frame's detail in the blocks a frame drawn in part lacks, where the picture is still. Decode and encode times are in `BENCHMARKS.md`. The decision log at the end is kept up to date. ## Overview diff --git a/CHANGELOG.md b/CHANGELOG.md index 8870e9d..28fa468 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,9 @@ All notable changes to emberwave. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow [Semantic Versioning](https://semver.org/). -## [Unreleased] +## [0.16.0] - 2026-10-31 + +Still pictures keep their detail through lost packets, and the wavelet transforms take less GPU time. A decoder that conceals, which a client turns on, fills the blocks of detail that a frame drawn in part lacks with the last frame's, wherever the coarsest level shows that the picture has not moved: in Rust, through the C API and in the browser player. The transforms read their rows straight from the bands, with the same output to the byte: on an Apple M3 Max a 4K frame's GPU work takes 0.47 ms instead of 0.54 ms. ### Added diff --git a/Cargo.lock b/Cargo.lock index f83828b..c42c018 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -288,7 +288,7 @@ dependencies = [ [[package]] name = "emberwave-cli" -version = "0.15.0" +version = "0.16.0" dependencies = [ "clap", "emberwave-core", @@ -299,7 +299,7 @@ dependencies = [ [[package]] name = "emberwave-core" -version = "0.15.0" +version = "0.16.0" dependencies = [ "bytemuck", "naga", @@ -311,7 +311,7 @@ dependencies = [ [[package]] name = "emberwave-ffi" -version = "0.15.0" +version = "0.16.0" dependencies = [ "ash", "cbindgen", @@ -325,7 +325,7 @@ dependencies = [ [[package]] name = "emberwave-web" -version = "0.15.0" +version = "0.16.0" dependencies = [ "emberwave-core", "wasm-bindgen", diff --git a/Cargo.toml b/Cargo.toml index b6b9110..b035e57 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -8,7 +8,7 @@ members = [ ] [workspace.package] -version = "0.15.0" +version = "0.16.0" edition = "2024" license = "MIT" repository = "https://github.com/with-developer/emberwave" @@ -17,7 +17,7 @@ rust-version = "1.87" publish = false [workspace.dependencies] -emberwave-core = { path = "crates/emberwave-core", version = "0.15.0" } +emberwave-core = { path = "crates/emberwave-core", version = "0.16.0" } thiserror = "2" proptest = "1" # Native backends only: GLES would need EGL and the browser backend is for the diff --git a/README.ko.md b/README.ko.md index 8a8c3de..ee2e2e7 100644 --- a/README.ko.md +++ b/README.ko.md @@ -142,7 +142,7 @@ iPhone 17의 Safari에서는 [벤치마크 페이지](https://with-developer.git ## 로드맵 -첫 릴리스까지의 마일스톤 M0~M5([#1](https://github.com/with-developer/emberwave/issues/1)~[#6](https://github.com/with-developer/emberwave/issues/6))를 마쳤고, 그 뒤로 CPU·GPU 인코더([#14](https://github.com/with-developer/emberwave/issues/14)), 브라우저 플레이어와 벤치마크 페이지([#15](https://github.com/with-developer/emberwave/issues/15)), 클라이언트와 호스트를 위한 C API([#16](https://github.com/with-developer/emberwave/issues/16), [#42](https://github.com/with-developer/emberwave/issues/42)), Metal 텍스처를 포함해 GPU 메모리의 그림을 주고받는 기능([#27](https://github.com/with-developer/emberwave/issues/27), [#33](https://github.com/with-developer/emberwave/issues/33)), iPhone·iPad·Mac 앱용 XCFramework([#52](https://github.com/with-developer/emberwave/issues/52))를 만들었습니다. 0.12에서는 Apple GPU에서 프레임마다 하던 복사 두 가지를 없앴습니다. Metal에서는 평면을 텍스처에 바로 쓰고, Apple silicon에서는 프레임을 셰이더가 읽는 메모리에 바로 씁니다([#86](https://github.com/with-developer/emberwave/issues/86)). 0.13에서는 스트림을 도착하는 대로 디코딩합니다. 브라우저 플레이어가 스트림을 패킷 단위로 받고, 모든 프레임에 가장 거친 레벨의 블록을 빠짐없이 담아 패킷을 잃은 프레임도 일부를 디코딩할 수 있게 했습니다. 패킷을 잃은 프레임을 블록이 얼마나 모였을 때 그릴지는 클라이언트가 정합니다. 클라이언트의 Metal 큐를 여러 스레드가 함께 써도 멈추지 않도록, emberwave의 호출은 큐의 명령 버퍼를 차례대로 씁니다([#109](https://github.com/with-developer/emberwave/issues/109), [#116](https://github.com/with-developer/emberwave/issues/116), [#123](https://github.com/with-developer/emberwave/issues/123), [#126](https://github.com/with-developer/emberwave/issues/126)). 0.14에서는 모든 플랫폼에서 클라이언트·호스트와 GPU를 함께 씁니다. 호출자의 Vulkan 장치 위에서 컨텍스트를 만들어 그 이미지에 세마포어와 함께 디코딩하고, Windows에서는 Direct3D 12·11 장치와 텍스처·펜스를 NT 핸들로 공유합니다. 호스트의 인코더는 그림이 담긴 텍스처를 그대로 읽으며, NV12 배치도 받습니다. 릴리스마다 XCFramework를 Swift 패키지가 받는 형태의 압축 파일로도 올립니다([#115](https://github.com/with-developer/emberwave/issues/115), [#133](https://github.com/with-developer/emberwave/issues/133), [#137](https://github.com/with-developer/emberwave/issues/137)). 0.15에서는 호스트가 화면을 캡처한 그대로 RGB 그림을 인코딩합니다. 인코더가 Metal·Vulkan·Direct3D의 BGRA·RGBA 텍스처나 CPU 메모리에서 그림을 읽으면서 Y'CbCr로 바꾸며, 정수로 계산하므로 어느 GPU에서나 같은 바이트가 나옵니다([#142](https://github.com/with-developer/emberwave/pull/142)). 릴리스별 변경 사항은 [`CHANGELOG.md`](CHANGELOG.md)에 있습니다. +첫 릴리스까지의 마일스톤 M0~M5([#1](https://github.com/with-developer/emberwave/issues/1)~[#6](https://github.com/with-developer/emberwave/issues/6))를 마쳤고, 그 뒤로 CPU·GPU 인코더([#14](https://github.com/with-developer/emberwave/issues/14)), 브라우저 플레이어와 벤치마크 페이지([#15](https://github.com/with-developer/emberwave/issues/15)), 클라이언트와 호스트를 위한 C API([#16](https://github.com/with-developer/emberwave/issues/16), [#42](https://github.com/with-developer/emberwave/issues/42)), Metal 텍스처를 포함해 GPU 메모리의 그림을 주고받는 기능([#27](https://github.com/with-developer/emberwave/issues/27), [#33](https://github.com/with-developer/emberwave/issues/33)), iPhone·iPad·Mac 앱용 XCFramework([#52](https://github.com/with-developer/emberwave/issues/52))를 만들었습니다. 0.12에서는 Apple GPU에서 프레임마다 하던 복사 두 가지를 없앴습니다. Metal에서는 평면을 텍스처에 바로 쓰고, Apple silicon에서는 프레임을 셰이더가 읽는 메모리에 바로 씁니다([#86](https://github.com/with-developer/emberwave/issues/86)). 0.13에서는 스트림을 도착하는 대로 디코딩합니다. 브라우저 플레이어가 스트림을 패킷 단위로 받고, 모든 프레임에 가장 거친 레벨의 블록을 빠짐없이 담아 패킷을 잃은 프레임도 일부를 디코딩할 수 있게 했습니다. 패킷을 잃은 프레임을 블록이 얼마나 모였을 때 그릴지는 클라이언트가 정합니다. 클라이언트의 Metal 큐를 여러 스레드가 함께 써도 멈추지 않도록, emberwave의 호출은 큐의 명령 버퍼를 차례대로 씁니다([#109](https://github.com/with-developer/emberwave/issues/109), [#116](https://github.com/with-developer/emberwave/issues/116), [#123](https://github.com/with-developer/emberwave/issues/123), [#126](https://github.com/with-developer/emberwave/issues/126)). 0.14에서는 모든 플랫폼에서 클라이언트·호스트와 GPU를 함께 씁니다. 호출자의 Vulkan 장치 위에서 컨텍스트를 만들어 그 이미지에 세마포어와 함께 디코딩하고, Windows에서는 Direct3D 12·11 장치와 텍스처·펜스를 NT 핸들로 공유합니다. 호스트의 인코더는 그림이 담긴 텍스처를 그대로 읽으며, NV12 배치도 받습니다. 릴리스마다 XCFramework를 Swift 패키지가 받는 형태의 압축 파일로도 올립니다([#115](https://github.com/with-developer/emberwave/issues/115), [#133](https://github.com/with-developer/emberwave/issues/133), [#137](https://github.com/with-developer/emberwave/issues/137)). 0.15에서는 호스트가 화면을 캡처한 그대로 RGB 그림을 인코딩합니다. 인코더가 Metal·Vulkan·Direct3D의 BGRA·RGBA 텍스처나 CPU 메모리에서 그림을 읽으면서 Y'CbCr로 바꾸며, 정수로 계산하므로 어느 GPU에서나 같은 바이트가 나옵니다([#142](https://github.com/with-developer/emberwave/pull/142)). 0.16에서는 패킷을 잃어도 멈춰 있는 화면의 세부가 남습니다. 일부만 받은 프레임에서 빠진 세부 블록을, 화면이 움직이지 않은 곳이면 직전 프레임의 것으로 채웁니다. 클라이언트가 켤 때만 동작합니다. 웨이블릿 변환의 GPU 시간도 5분의 1 줄었습니다([#129](https://github.com/with-developer/emberwave/issues/129), [#147](https://github.com/with-developer/emberwave/pull/147)). 릴리스별 변경 사항은 [`CHANGELOG.md`](CHANGELOG.md)에 있습니다. 앞으로 할 일입니다. 준비되는 대로 각각 릴리스합니다. diff --git a/README.md b/README.md index efd7568..71d114b 100644 --- a/README.md +++ b/README.md @@ -142,7 +142,7 @@ Out of scope: network transport, RTP and FEC. ## Roadmap -Done: the milestones to the first release, M0 to M5 ([#1](https://github.com/with-developer/emberwave/issues/1) to [#6](https://github.com/with-developer/emberwave/issues/6)); since then the encoder, on the CPU and on the GPU ([#14](https://github.com/with-developer/emberwave/issues/14)), the browser player and its benchmark page ([#15](https://github.com/with-developer/emberwave/issues/15)), the C API for clients and hosts ([#16](https://github.com/with-developer/emberwave/issues/16), [#42](https://github.com/with-developer/emberwave/issues/42)), pictures to and from GPU memory, Metal textures included ([#27](https://github.com/with-developer/emberwave/issues/27), [#33](https://github.com/with-developer/emberwave/issues/33)), and the XCFramework for iPhone, iPad and Mac apps ([#52](https://github.com/with-developer/emberwave/issues/52)). 0.12 took two copies out of a frame's work on Apple's GPUs: the planes are stored straight into textures on Metal, and on Apple silicon the frames are written where the shaders read them ([#86](https://github.com/with-developer/emberwave/issues/86)). 0.13 decodes streams as they arrive: the browser player takes a stream packet by packet, every frame carries the blocks that a decode in part needs, a client chooses how much of a frame it draws in part, and calls on a client's own Metal queue take turns at its command buffers ([#109](https://github.com/with-developer/emberwave/issues/109), [#116](https://github.com/with-developer/emberwave/issues/116), [#123](https://github.com/with-developer/emberwave/issues/123), [#126](https://github.com/with-developer/emberwave/issues/126)). 0.14 shares the GPU with clients and hosts on every platform: a context on the caller's own Vulkan device decodes into its images with semaphores, Direct3D 12 and 11 devices on Windows share textures and fences through NT handles, and a host's encoder reads its pictures straight from its textures, NV12's layout included. Each release also has the XCFramework in the archive a Swift package takes ([#115](https://github.com/with-developer/emberwave/issues/115), [#133](https://github.com/with-developer/emberwave/issues/133), [#137](https://github.com/with-developer/emberwave/issues/137)). 0.15 encodes pictures in RGB, as hosts' screen captures hold them: the encoder converts them to Y'CbCr as it reads them, from a BGRA or RGBA texture of Metal, Vulkan or Direct3D or from CPU memory, in integers that make the same bytes on every GPU ([#142](https://github.com/with-developer/emberwave/pull/142)). [`CHANGELOG.md`](CHANGELOG.md) has each release. +Done: the milestones to the first release, M0 to M5 ([#1](https://github.com/with-developer/emberwave/issues/1) to [#6](https://github.com/with-developer/emberwave/issues/6)); since then the encoder, on the CPU and on the GPU ([#14](https://github.com/with-developer/emberwave/issues/14)), the browser player and its benchmark page ([#15](https://github.com/with-developer/emberwave/issues/15)), the C API for clients and hosts ([#16](https://github.com/with-developer/emberwave/issues/16), [#42](https://github.com/with-developer/emberwave/issues/42)), pictures to and from GPU memory, Metal textures included ([#27](https://github.com/with-developer/emberwave/issues/27), [#33](https://github.com/with-developer/emberwave/issues/33)), and the XCFramework for iPhone, iPad and Mac apps ([#52](https://github.com/with-developer/emberwave/issues/52)). 0.12 took two copies out of a frame's work on Apple's GPUs: the planes are stored straight into textures on Metal, and on Apple silicon the frames are written where the shaders read them ([#86](https://github.com/with-developer/emberwave/issues/86)). 0.13 decodes streams as they arrive: the browser player takes a stream packet by packet, every frame carries the blocks that a decode in part needs, a client chooses how much of a frame it draws in part, and calls on a client's own Metal queue take turns at its command buffers ([#109](https://github.com/with-developer/emberwave/issues/109), [#116](https://github.com/with-developer/emberwave/issues/116), [#123](https://github.com/with-developer/emberwave/issues/123), [#126](https://github.com/with-developer/emberwave/issues/126)). 0.14 shares the GPU with clients and hosts on every platform: a context on the caller's own Vulkan device decodes into its images with semaphores, Direct3D 12 and 11 devices on Windows share textures and fences through NT handles, and a host's encoder reads its pictures straight from its textures, NV12's layout included. Each release also has the XCFramework in the archive a Swift package takes ([#115](https://github.com/with-developer/emberwave/issues/115), [#133](https://github.com/with-developer/emberwave/issues/133), [#137](https://github.com/with-developer/emberwave/issues/137)). 0.15 encodes pictures in RGB, as hosts' screen captures hold them: the encoder converts them to Y'CbCr as it reads them, from a BGRA or RGBA texture of Metal, Vulkan or Direct3D or from CPU memory, in integers that make the same bytes on every GPU ([#142](https://github.com/with-developer/emberwave/pull/142)). 0.16 keeps a still picture's detail through lost packets: where the picture has not moved, a frame drawn in part fills the blocks of detail it lacks with the last frame's, when a client asks for it. Its wavelet transforms also take a fifth less GPU time ([#129](https://github.com/with-developer/emberwave/issues/129), [#147](https://github.com/with-developer/emberwave/pull/147)). [`CHANGELOG.md`](CHANGELOG.md) has each release. Next, each in a release when it is ready: diff --git a/crates/emberwave-ffi/README.md b/crates/emberwave-ffi/README.md index 23a2f8b..9136030 100644 --- a/crates/emberwave-ffi/README.md +++ b/crates/emberwave-ffi/README.md @@ -81,7 +81,7 @@ A Swift package names the XCFramework as a binary target. From a release, it tak // In the app's Package.swift, with the version of the release: .binaryTarget( name: "Emberwave", - url: "https://github.com/with-developer/emberwave/releases/download/v0.15.0/Emberwave.xcframework.zip", + url: "https://github.com/with-developer/emberwave/releases/download/v0.16.0/Emberwave.xcframework.zip", checksum: "..." // the release's SHA256SUMS line for Emberwave.xcframework.zip ), .target(