[Video] Add native H3 request-scoped TeaCache and Cache-DiT

[yangzhuxinyzx]

Purpose

Native H3 previously rejected request caches and quality=high. Add explicit TeaCache and Cache-DiT deployment policies to the existing Python/CLI/video HTTP API, with the pinned official polynomial, Pattern_3 implementation, high-quality profile, TaylorSeer calibrator, predefined SCM masks and once/repeat refresh hints. Every request creates and tears down its own residual state, including failure paths; lossless requests run without a cache. TP ranks agree on whether to execute the block stack.

Stacked on #583 (#578 / #571 beneath it). Open H3 PRs were checked on 2026-09-09 and none implements these request-cache policies. This is the cache integration portion of the accepted broader SM70 campaign. It uses vLLM-Omni 7be014bce6374f06c95b703763bdbac4c6198f31 and its exact cache-dit==1.5.0 distribution. TeaCache remains FL2VA-only, caches are mutually exclusive, and both are disabled by default. TaylorSeer and predefined SCM profiles are exposed explicitly, with their official defaults unchanged. quality=high disables both.

Actual executed block identities, cache hits, skipped blocks and modulation-probe overhead are recorded. Only executed useful matrix/attention work enters throughput; skipped blocks and the extra cache decision projection do not inflate FLOPs. Refresh hints are validated against actual sigma intervals, preserving LightX2V and FlashGen/FastH3 step conventions.

Test Plan

In the owned Python 3.12.13 / Torch 2.10.0+cu128 / CUDA 12.8.93 environment:

.venv/bin/python -m pytest tests/video/test_h3_request_cache.py -q
.venv/bin/python -m pytest tests/video/test_h3_request_cache.py tests/video/test_h3_acceptance.py tests/video/test_h3_omni_api.py tests/video/test_h3_service.py tests/video/test_h3_workflows.py tests/video/test_h3_work_counter.py -q -k 'not measured and not gpu and not real'
.venv/bin/python -m pytest tests/video/test_h3_acceptance.py tests/video/test_h3_request_cache.py tests/video/test_h3_service.py tests/video/test_h3_omni_api.py -q
.venv/bin/python -m torch.distributed.run --standalone --nproc-per-node=1 tests/video/test_h3_request_cache_gpu.py
.venv/bin/python -m torch.distributed.run --standalone --nproc-per-node=2 tests/video/test_h3_request_cache_gpu.py
.venv/bin/python -m torch.distributed.run --standalone --nproc-per-node=4 tests/video/test_h3_request_cache_gpu.py

GPU commands used the native lease wrapper and immutable task-built binary manifest on physical V100 GPUs 0–3. Full native lifecycle requests (cached, lossless, cached) have completed for both backends on TP4 at 256x448 / 107 internal frames / 49 intervals, with original FP16 execution weights and FP32 residuals. These are compatibility and isolation checks, not formal speed measurements.

Test Result

• 18 cache policy/lifecycle CPU checks pass, including real Cache-DiT repeated requests and once/repeat hints.
• 120 affected CPU checks pass, 2 GPU cases deselected. The strengthened accounting/API set has 105 passing checks.
• Real small full H3 forward passes TP1, TP2 and TP4 with original FP16 weights and FP32 residual sharding. All ranks reproduce the same execution patterns across requests: forced-compute 6/6/6/6, TeaCache 6/2/2/6, Cache-DiT 6/3/3/6. Outputs remain within 1e-5 for identical-input checks.
• Full native TeaCache requests record 5 / 0 / 5 cache hits on every rank; Cache-DiT records 34 / 0 / 34. All six requests pass basic media and strict work checks. First/third video and audio latents and RGB frames match bitwise; decoded PCM relative L2 is 1.685e-6 / 2.066e-6 respectively, with all declared repeatability gates passing. This establishes request isolation, not approximate-cache agreement with lossless or official full-model quality.
• All commit hooks pass. The first GPU fixture omitted its reduced-size final AdaLN dimension and was rejected during construction; the corrected fixture passes. No production algorithm was changed to accommodate the fixture error.
• cache-dit==1.5.0 has internal __version__=1.3.13.dev9; provenance records the distribution version plus Python source hashes. Hub 1.30.0 resolves Diffusers 0.40.0's get_cached_repo_tree import requirement, in the owned venv only.

Evidence is under /data/minimax-h3/sm70-general-20260909/ (cache-*.log, teacache-native-summary.json, cachedit-native-summary.json and both *-repeat-quality.json files). Behavior and limitations are documented in docs/design/minimax_h3/REQUEST_CACHES.md.

This remains Draft: independent official full-model quality, human review, full workflow coverage and >80 useful TFLOP/s/card acceptance remain incomplete. No AUTO configuration is qualified and no approximate-cache quality threshold is relaxed. The separate VSA FP32 math diagnostic fails the required final latent/video gates and is not claimed as accepted quality.

Optional-policy validation at f5e4340: 42 policy/lifecycle CPU checks pass; 87 other acceptance/service/API checks also pass. Three initial test failures incorrectly expected reuse when a repeated refresh resets SCM before its first cache slot; tests now assert the official all-compute pattern. Real small H3 GPU forwards pass TP1/TP2/TP4 with TaylorSeer order 2 and both dynamic/static fast SCM, including repeated outputs and execution counts 6/6/3/6. Full original-weight TP4 cached/lossless/cached video generation with order-1 TaylorSeer, dynamic fast SCM and shared pageable VAE weights records 23/0/23 hits on all ranks. Final latent/RGB repeats are bitwise; PCM relative L2 is 1.735e-6, spectral cosine >0.999999999999 and RMS ratio 0.999999975445. All media/actual-work/repeatability gates pass. These captured 256x448/107-frame/49-interval requests are lifecycle checks, not independent official full-model quality or primary performance acceptance. See cache-scm-native-summary.json, cache-scm-repeat-quality.json and cache-scm-gpu-tp*.log.

AI assistance: implementation and validation were assisted by OpenAI Codex. Human review is required before promotion or merge.

Shared operator/layer-residency integration at c8e9bf2431 inherits #583/#581/#578/#571 and passes 126 affected CPU checks. The real small H3 cache script additionally passes --layer-offload on TP1/TP2/TP4 for forced compute, TeaCache, Cache-DiT and both TaylorSeer-SCM policies. Two consecutive requests preserve resident controls within the existing 1e-5 check, agree on executed blocks and return every parameter to its host master. This validates integration and lifecycle only; no new full-model quality, primary performance or AUTO acceptance is claimed.

At 174bf69, the distributed cache/layer-residency fixture accepts an explicit FA or FI backend. The new register-probability FI binary passes TP1/TP2/TP4 with all five cache policies and two consecutive requests; resident-output and request-isolation gates are unchanged (cache-register-gpu.json). The associated CPU integration has 103 passes and 7 GPU skips. This extends backend integration, not full cached-model quality or >80 acceptance.

Shared dependency update: this branch now includes the explicit calibrated SM70 TP4 row-reduction interface from kernel commit 6d2a44b8d0. It remains unused by default and does not alter the VSA or cache algorithm. Generic SM70 binary provenance includes sparse Attention and the new communication library. Merge f2ad4f7512 passes 97 targeted CPU checks with one GPU skip. Existing cache lifecycle evidence does not become official quality or >80 performance acceptance.

Native API dependency integration at f7dbba3721 passes 110 targeted CPU checks with one GPU skip. Fixture 567c060fa8 passes TP4 FA and FI with explicit peer reduction, real layer offload, all five cache policies and two consecutive requests; outputs, executed blocks and cross-rank communication counts pass, with zero native fallback calls. Evidence: cache-peer-gpu.json, cache-peer-fa.log, cache-peer-fi.log. This extends integration evidence only.