GoWebGPU Video Rendering Engine

A high-performance, hybrid video rendering architecture combining a Go backend (headless WebGPU export engine) and an HTML5 frontend editor (real-time WebGPU preview).

This engine is designed to power modern SaaS video editors (like Canva Video or CapCut Web) by enabling instant 60fps browser-side editing and offloading heavy production rendering to GPU-accelerated server backends.

---

🚀 Key Features

• Headless WebGPU Backend: Compiles WGSL shaders and runs rendering pipelines inside headless Go containers on the server GPU.
• Chroma Key Shader (WGSL): Multi-parameter chroma keying utilizing Euclidean color distance and smoothstep transition interpolation.
• Hybrid Real-Time Preview: A client-side HTML5 page implementing the identical WGSL shader, offering real-time 60fps interactive controls, color picker, and green screen test pattern generator.
• Structured JSON Timeline: Declares video formats, clips, and effect configurations dynamically inside a JSON schema (timeline.json).
• Automated Benchmark: Built-in comparison suite showing performance benchmarks against standard CPU rendering.

---

🛠️ Tech Stack & Architecture

1. Frontend (Preview Layer): HTML5 Canvas, WebGPU API, and Vanilla CSS glassmorphic UI.
2. Backend (Export Layer):
   • Go & WebGPU Wrapper (github.com/go-webgpu/webgpu): Manages VRAM resource allocation (Texture Views, Buffers, Samplers) and manual garbage cleanup.
   • FFmpeg Pipes: Seamlessly decodes input streams to raw RGBA buffers and pipes rendered RGBA buffers back to FFmpeg for H.264 high-bitrate encoding.

---

📦 Getting Started

Prerequisites

• OS: Windows (AMD64)
• Go: 1.20 or newer
• FFmpeg: ffmpeg.exe and ffprobe.exe must be placed in the project root directory or added to your system PATH.
• Hardware Acceleration: A WebGPU-compliant GPU (Nvidia/AMD/Intel) with modern graphics drivers.

Setup Instructions

1. Download the required native WebGPU libraries:

   go run github.com/go-webgpu/webgpu/cmd/setup@latest

   (If blocked by UAC/elevation limits, compile using custom filename like go build -o get_wgpu.exe github.com/go-webgpu/webgpu/cmd/setup and run .\get_wgpu.exe)

2. Resolve Go modules dependencies:

   go mod tidy

---

🖥️ Usage Guide

1. Run the Video Export Pipeline

Execute the main compiler to parse timeline.json, apply the WebGPU chroma key shader, and generate output.mp4:

$env:GOARCH="amd64"
go run cmd/main.go

2. Launch the Web Preview UI

Start a local web server (to bypass browser local-file CORS blocks):

python -m http.server 8000

Open http://localhost:8000/web/preview.html in Chrome/Edge. Drag-and-drop your custom green screen video and adjust the sliders to key out colors in real-time.

3. Run Performance Benchmark

Verify processing throughput (FPS) between CPU-bound filters and WebGPU:

$env:GOARCH="amd64"
go run cmd/benchmark/main.go

---

📊 Benchmark Analysis

A standard run on a 5-second 1280x720 @ 24fps test video yields:

Engine Mode	Total Time	FPS
CPU (FFmpeg colorkey)	3.09s	38.80
GPU Pipeline (End-to-End)	6.25s	19.18
GPU Pure Shader Stage	2.39s	50.04

• Why the Pure Shader is faster (50.04 FPS): Calculation of pixel transformations is done in parallel across thousands of GPU cores.
• Why the End-to-End GPU Pipeline is currently slower: Medium bandwidth transfer overhead (copying 3.68MB raw RGBA frames per frame between host memory and GPU memory via FFmpeg pipes).

---

🗺️ Future Roadmap & Implementation Plan

To evolve this prototype into a production-grade commercial platform, the following plans will be implemented:

🔴 Phase 1: Native GPU Hardware Decoding/Encoding (Zero-Copy VRAM)

• Goal: Eliminate CPU-GPU memory copy overhead (which is currently capping performance at 19 FPS).
• Implementation:
  • Integrate hardware-accelerated decoders (like Nvidia NVDEC or DXVA2/D3D11 on Windows) directly into the Go backend.
  • Decode video frames straight into GPU Texture memory, bypass CPU system RAM altogether, and pass it directly to WebGPU.
  • Render and encode using hardware encoders (like Nvidia NVENC). This will scale rendering performance to 100+ FPS.

🟡 Phase 2: Multi-Pass Shader Render Graph

• Goal: Support chaining multiple visual effects on a single clip.
• Implementation:
  • Build a multi-pass render pipeline where the output texture of the first shader pass (e.g. Chroma Key) becomes the input texture of the second pass (e.g., Blur, Color Correction LUT, Text Overlays).
  • Consolidate all steps into a single GPU command buffer submission.

🟢 Phase 3: Multi-Track Timeline & Audio Mixing

• Goal: Support multiple overlapping tracks, transitions, and audio resampling.
• Implementation:
  • Parse complex overlapping layers from timeline.json.
  • Perform GPU alpha blending to composite multiple tracks on top of each other.
  • Decode audio streams, resample, apply gain, mix them using an audio library, and merge the mixed audio back into the final video container.

🔵 Phase 4: High-Concurrency SaaS Infrastructure

• Goal: Deploy the engine in the cloud to serve millions of render requests.
• Implementation:
  • Dockerize the Go engine with GPU pass-through drivers (nvidia-container-toolkit).
  • Create a distributed task queue (using Redis/RabbitMQ) with Go worker instances consuming JSON jobs and uploading final MP4s to S3 buckets.