Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions docs/adr/v4-02-gpu-decision.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# ADR V4-02: GPU decoder decision remains blocked

Status: **NOT_RUN / no decoder selected**
Date: 2026-08-02

## Context

V4 requires one DXGI adapter for hardware decode, processing, composition, shared surfaces, and eventual programme encoding. Media Foundation must expose `IMFDXGIBuffer`; FFmpeg must expose `AV_PIX_FMT_D3D11`. Software output, ordinary GPU-to-CPU-to-GPU frame routes, cross-adapter texture paths, and silent fallback are forbidden.

This spike added isolated C++20 probes under `spikes/gpu`. Both candidates receive an explicitly selected adapter LUID and fail closed if a decoded resource is not a D3D11 texture owned by that device. The Media Foundation probe additionally exercises D3D11 video-processor scale, NV12 colour conversion, 90-degree rotation, and 2x2 composition without a CPU frame upload or readback. These are probes only; no engine or product compositor was implemented.

## Evidence obtained

On the target i5-13420H/RTX 4050 laptop, five-second smoke runs covered H.264 and HEVC for four 1080p30 streams and two 1080p60 streams. Every Media Foundation output exposed `IMFDXGIBuffer/ID3D11Texture2D`; every FFmpeg output exposed `AV_PIX_FMT_D3D11/ID3D11Texture2D`. All textures belonged to NVIDIA adapter LUID `0x0:0x16933`. Instrumented upload, readback, ordinary CPU frame-copy, and FFmpeg hardware-frame-transfer counters remained zero. Raw hashes and counts are in `docs/evidence/v4-02-gpu-smoke.json`.

This is smoke evidence, not performance or copy-ledger acceptance. Five seconds cannot establish CPU/GPU/memory/latency distributions or memory slope, and counters without ETW/PIX corroboration cannot prove the full copy ledger.

## Decision

Do not choose Media Foundation or FFmpeg yet. V4-02 remains blocked and downstream production GPU decode/compositor work must not treat either backend as selected.

Adapter policy is nevertheless reaffirmed: the caller supplies one DXGI LUID; missing adapters fail visibly; there is no automatic adapter selection, cross-adapter texture path, or silent switch. Software fallback remains disabled by default, requires explicit operator consent and a visible warning, and is excluded from hardware claims.

Device-removal policy remains the architecture policy: stop GPU consumers, preserve independent ingress and compressed ISO, rebuild the full graph only on the same LUID when still present, otherwise remain in a visible recoverable failure. The policy was not validated by this run.

## Incomplete acceptance cells

- One-hour candidate × codec × workload matrix: `NOT_RUN` under the lead timebox. The drive had about 1.95 GB free before corpus generation; probes now support long-lived in-process looping of a small hashed corpus for a later run.
- CPU/GPU/private-memory/latency comparison and memory slope: `NOT_RUN`; short-lived wrapper sampling was incomplete and is not used for a decision.
- ETW/PIX copy-ledger corroboration: `NOT_RUN`; WPR is available but no elevated trace and review were completed.
- Genuine device removal with independent ingress/ISO continuity: `NOT_RUN`; Graphics Tools/dxcap was absent. A logical teardown was explicitly rejected as a substitute.
- One hardware programme encode: `NOT_RUN`; no encoder was added to this decoder/composition spike.

## Completion command

After installing a pinned MSVC-compatible FFmpeg development package and Windows Graphics Tools, run:

```powershell
cmake -S spikes/gpu -B out/gpu-build -G "Visual Studio 17 2022" -A x64 -DOPENSTREAM_FFMPEG_ROOT=C:/ffmpeg-dev
cmake --build out/gpu-build --config Release
spikes/gpu/run-benchmark.ps1 -CorpusRoot out/gpu-corpus -OutputDirectory out/gpu-results -GenerateCorpus -CorpusSeconds 10 -AdapterLuid 92467 -DurationSeconds 3600 -WarmupSeconds 60
```

Capture and review clean plus device-removal ETW traces, add an isolated hardware-encode probe, and publish the complete raw JSONL before revisiting this ADR.

## Rollback

Delete `spikes/gpu`, `tests/test_v4_gpu_spike.py`, this ADR, and its smoke-evidence file. No product or legacy path depends on the spike.
55 changes: 55 additions & 0 deletions docs/evidence/v4-02-gpu-smoke.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"schema_version": 1,
"status": "SMOKE_ONLY",
"commit_tested": "6c65476a7245e479a0cc9c2bfa6bd167d6cbd875",
"environment": {
"os": "Microsoft Windows 11 Home Single Language 10.0.26200 build 26200",
"cpu": "13th Gen Intel(R) Core(TM) i5-13420H",
"logical_processors": 12,
"adapter": "NVIDIA GeForce RTX 4050 Laptop GPU",
"adapter_luid_decimal": 92467,
"adapter_luid_hex": "0x0:0x16933",
"nvidia_driver": "32.0.15.9282",
"secondary_adapter": "Intel(R) UHD Graphics",
"secondary_driver": "32.0.101.5542",
"power_plan": "Balanced",
"runtime_ffmpeg": "8.0.1-full_build-www.gyan.dev",
"native_probe_ffmpeg_sdk": "BtbN ffmpeg-master-latest-win64-gpl-shared, downloaded 2026-08-02"
},
"command": "spikes/gpu/run-benchmark.ps1 -CorpusRoot out/gpu-corpus -OutputDirectory out/gpu-smoke-results -MediaFoundationProbe out/gpu-ninja/openstream_mf_gpu_probe.exe -FfmpegProbe out/gpu-ninja/openstream_ffmpeg_d3d11_probe.exe -AdapterLuid 92467 -DurationSeconds 5 -WarmupSeconds 1",
"workloads": [
{
"id": "four-1080p30-h264",
"corpus_sha256": ["CA92B26DDAA387FE8FA3FA01A80CE4F290CB14A52301288292F6375A56EADD5E", "1633BF9B19809AF250FF3C2FFDC11CE8ABD8B0670D24B05D039B9A5CBD62489F", "0B6659626082284FE4FE5159EE40B1BF0E0530373D740C986EC5C711F1E53998", "7C0A847783CDCDB306FBEA2D0E1AB029BEC3390A751635934B8284D8BF2C8C34"],
"media_foundation": {"result": "SMOKE_PASS", "frames": 604, "composition_frames": 152, "layout": "2x2", "texture_type": "IMFDXGIBuffer/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0},
"ffmpeg_d3d11": {"result": "SMOKE_PASS", "frames": 596, "texture_type": "AV_PIX_FMT_D3D11/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0, "av_hwframe_transfer_data_calls": 0}
},
{
"id": "four-1080p30-hevc",
"corpus_sha256": ["690BCA5931E1664591D14C96D7D8FA2BE98F12B88BB07DF54354746B49DF577B", "91BE40F42413EDA6F7CC2E5EE9FA9173E829A795C155A464917A62B1DDDAD0D2", "EEA7439A446EEDFA1B0A06C8CA5E937D821B71B1D8C2EB3EC8BDC379558A7334", "66C1D29D1E926166CACF48892CDFCFA5ED73C560D21CEB53DCA7915171536FFC"],
"media_foundation": {"result": "SMOKE_PASS", "frames": 604, "composition_frames": 152, "layout": "2x2", "texture_type": "IMFDXGIBuffer/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0},
"ffmpeg_d3d11": {"result": "SMOKE_PASS", "frames": 596, "texture_type": "AV_PIX_FMT_D3D11/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0, "av_hwframe_transfer_data_calls": 0}
},
{
"id": "two-1080p60-h264",
"corpus_sha256": ["719BAA7C2F5CCC6F5634F0BB13DFB387E76C8D754DDA6E352A655D23167B6744", "3901065CB71E1154754AA7218545AB6D7660247140DFF3A70E320E078FF867B2"],
"media_foundation": {"result": "SMOKE_PASS", "frames": 602, "composition_frames": 302, "layout": "2x1", "texture_type": "IMFDXGIBuffer/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0},
"ffmpeg_d3d11": {"result": "SMOKE_PASS", "frames": 598, "texture_type": "AV_PIX_FMT_D3D11/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0, "av_hwframe_transfer_data_calls": 0}
},
{
"id": "two-1080p60-hevc",
"corpus_sha256": ["0A751D426776566E896BF2E41C3D9C14F050670942094F6557DA9B19D363B1D7", "754772E2842050C956A8C0E2010242D2453C633FFE83FBA7D7B0BA662203DF03"],
"media_foundation": {"result": "SMOKE_PASS", "frames": 602, "composition_frames": 302, "layout": "2x1", "texture_type": "IMFDXGIBuffer/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0},
"ffmpeg_d3d11": {"result": "SMOKE_PASS", "frames": 596, "texture_type": "AV_PIX_FMT_D3D11/ID3D11Texture2D", "ordinary_cpu_frame_copies": 0, "av_hwframe_transfer_data_calls": 0}
}
],
"reporting_correction": "The original two-stream probe summary mislabeled its two-input layout as 2x2. The evidence records the actual 2x1 input layout, and the probe now emits layout dynamically.",
"not_run": {
"one_hour_matrix": "NOT_RUN: lead timebox stopped execution after native smoke; only 1.95 GB was free before corpus generation, so large per-cell corpora were not materialized. The probes support in-process corpus looping for a future long-lived run.",
"cpu_gpu_memory_latency_decision_metrics": "NOT_RUN: five-second smoke samples are not statistically usable and the wrapper returned incomplete private-byte/GPU samples for several short-lived processes.",
"etw_copy_corroboration": "NOT_RUN: no elevated ETW capture/review was completed; WPR commands are checked in, but counter-only copy claims are not promoted to acceptance evidence.",
"device_removal_ingress_iso": "NOT_RUN: Graphics Tools/dxcap was absent and no genuine device-removal injection was performed. Logical teardown was rejected as insufficient evidence.",
"hardware_programme_encode": "NOT_RUN: the spike did not add a programme encoder; no product compositor or encoder was implemented.",
"decoder_selection": "NOT_RUN: required soak, trace, device-loss, and resource gates are incomplete."
}
}
60 changes: 60 additions & 0 deletions spikes/gpu/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
cmake_minimum_required(VERSION 3.30)

# This probe is deliberately standalone. It never adds FFmpeg to the product
# dependency graph: point OPENSTREAM_FFMPEG_ROOT at a pinned FFmpeg *development*
# package with include/ and lib/ directories.
project(openstream_v4_gpu_decision_probe LANGUAGES CXX)

if(NOT WIN32 OR NOT CMAKE_SIZEOF_VOID_P EQUAL 8)
message(FATAL_ERROR "The FFmpeg D3D11 probe supports Windows x64 only")
endif()

set(OPENSTREAM_FFMPEG_ROOT "$ENV{OPENSTREAM_FFMPEG_ROOT}" CACHE PATH
"Pinned FFmpeg development package root (include/ and lib/)")
if(NOT OPENSTREAM_FFMPEG_ROOT)
message(FATAL_ERROR
"Set OPENSTREAM_FFMPEG_ROOT to a pinned FFmpeg development package; runtime ffmpeg.exe is insufficient")
endif()

find_path(OPENSTREAM_AVCODEC_INCLUDE_DIR libavcodec/avcodec.h
HINTS "${OPENSTREAM_FFMPEG_ROOT}/include" NO_DEFAULT_PATH REQUIRED)
find_library(OPENSTREAM_AVCODEC_LIBRARY NAMES avcodec avcodec-62
HINTS "${OPENSTREAM_FFMPEG_ROOT}/lib" NO_DEFAULT_PATH REQUIRED)
find_library(OPENSTREAM_AVFORMAT_LIBRARY NAMES avformat avformat-62
HINTS "${OPENSTREAM_FFMPEG_ROOT}/lib" NO_DEFAULT_PATH REQUIRED)
find_library(OPENSTREAM_AVUTIL_LIBRARY NAMES avutil avutil-60
HINTS "${OPENSTREAM_FFMPEG_ROOT}/lib" NO_DEFAULT_PATH REQUIRED)

foreach(ffmpeg_library IN ITEMS OPENSTREAM_AVCODEC_LIBRARY OPENSTREAM_AVFORMAT_LIBRARY OPENSTREAM_AVUTIL_LIBRARY)
if(NOT "${${ffmpeg_library}}" MATCHES "\\.lib$")
message(FATAL_ERROR "${ffmpeg_library} must be an MSVC-compatible .lib, not ${${ffmpeg_library}}")
endif()
endforeach()

add_executable(openstream_ffmpeg_d3d11_probe src/ffmpeg_d3d11_probe.cpp)
target_compile_features(openstream_ffmpeg_d3d11_probe PRIVATE cxx_std_20)
target_compile_definitions(openstream_ffmpeg_d3d11_probe PRIVATE NOMINMAX WIN32_LEAN_AND_MEAN)
target_compile_options(openstream_ffmpeg_d3d11_probe PRIVATE /permissive- /W4 /WX /utf-8)
target_include_directories(openstream_ffmpeg_d3d11_probe SYSTEM PRIVATE "${OPENSTREAM_AVCODEC_INCLUDE_DIR}")
target_link_libraries(openstream_ffmpeg_d3d11_probe PRIVATE
"${OPENSTREAM_AVFORMAT_LIBRARY}"
"${OPENSTREAM_AVCODEC_LIBRARY}"
"${OPENSTREAM_AVUTIL_LIBRARY}"
d3d11
dxgi
psapi)
file(GLOB OPENSTREAM_FFMPEG_RUNTIME_DLLS CONFIGURE_DEPENDS "${OPENSTREAM_FFMPEG_ROOT}/bin/*.dll")
add_custom_command(TARGET openstream_ffmpeg_d3d11_probe POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${OPENSTREAM_FFMPEG_RUNTIME_DLLS} $<TARGET_FILE_DIR:openstream_ffmpeg_d3d11_probe>
COMMAND_EXPAND_LISTS)

add_executable(openstream_mf_gpu_probe src/mf_gpu_probe.cpp)
target_compile_features(openstream_mf_gpu_probe PRIVATE cxx_std_20)
target_compile_definitions(openstream_mf_gpu_probe PRIVATE NOMINMAX WIN32_LEAN_AND_MEAN)
target_compile_options(openstream_mf_gpu_probe PRIVATE /permissive- /W4 /WX /utf-8)
target_link_libraries(openstream_mf_gpu_probe PRIVATE
d3d11 dxgi mf mfplat mfreadwrite mfuuid ole32 shlwapi)

include(CTest)
add_test(NAME ffmpeg_gpu_probe_help COMMAND openstream_ffmpeg_d3d11_probe --help)
add_test(NAME mf_gpu_probe_help COMMAND openstream_mf_gpu_probe --help)
25 changes: 25 additions & 0 deletions spikes/gpu/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# V4 decoder/GPU decision spike

This directory contains benchmark probes, not the product compositor. The checked-in workload definition covers both codecs at four 1080p30 and two 1080p60. Media files are generated locally, hashed into raw results, and excluded from Git.

The hardware gate is deliberately fail-closed. Media Foundation output must expose `IMFDXGIBuffer`; FFmpeg output must be `AV_PIX_FMT_D3D11`. Both must resolve to `ID3D11Texture2D` on the requested adapter LUID. Software frames, cross-adapter resources, readback, upload, and `av_hwframe_transfer_data` reject a hardware run. `--allow-software-fallback` only records explicit operator consent; this probe still rejects software output and does not implement fallback.

## Build

Use an x64 Visual Studio 2022 developer shell. Configure the FFmpeg development package root as described by the CMake diagnostic.

```powershell
cmake -S spikes/gpu -B out/gpu-build -G "Visual Studio 17 2022" -A x64 -DOPENSTREAM_FFMPEG_ROOT=C:/ffmpeg-dev
cmake --build out/gpu-build --config Release
ctest --test-dir out/gpu-build -C Release --output-on-failure
```

## Benchmark

Generate the deterministic real-media corpus once, then run the full one-hour matrix. Generation uses NVENC unless the operator explicitly opts into a warned software encoder; software-generated corpus is valid input but its generation is not counted as a hardware-path measurement.

```powershell
spikes/gpu/run-benchmark.ps1 -CorpusRoot out/gpu-corpus -OutputDirectory out/gpu-results -GenerateCorpus -CorpusSeconds 10 -AdapterLuid <decimal-luid> -DurationSeconds 3600 -WarmupSeconds 60
```

The runner writes per-second CPU/private-memory/process-GPU samples, environment and driver fingerprints, queue declarations, exact corpus hashes, and probe summaries to `metrics.jsonl`. Capture an ETW trace alongside at least one clean and one device-removal run using the commands in `trace/README.md`; large ETL files remain external and are represented by hashes in checked-in evidence.
Loading
Loading