Repository navigation
[Renderer] Build virtualized meshlet geometry with automatic LOD and streaming #131
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Parent: #126
Depends on: #28, #27, #130, and #136
Problem
Traditional per-mesh LOD swaps and CPU submission cannot provide film-scale source geometry with stable frame cost. #28 and #27 establish GPU-visible geometry and visibility shading, but Bloom still needs an offline/runtime virtualized-geometry system to approach Nanite-class density.
Outcome
A Nanite-inspired, not Nanite-identical static-geometry path:
- imported meshes are partitioned into meshlets/clusters with bounds and error metrics;
- clusters form a hierarchical LOD DAG/tree produced by the cooker;
- GPU culling selects clusters by frustum, backface cone, projected error, and Hi-Z occlusion;
- indirect draws/visibility IDs render selected resident clusters;
- required pages stream under a fixed memory/IO budget;
- unsupported, skinned, deforming, translucent, and custom-vertex content stays on a documented compatibility path.
Offline format
The cooker must emit a versioned, endian-defined format containing:
- cluster vertex/index payloads with bounded size (choose and document limits after GPU/cache experiments);
- object-space AABB/sphere, normal cone, geometric error, parent/child links, and material ranges;
- page grouping optimized for locality plus content hashes and dependency records;
- a coarse always-resident fallback hierarchy;
- source-to-cooked provenance for deterministic rebuilds.
Do not lock the public API to an internal cluster size.
Runtime architecture
- Register virtual meshes and allocate stable mesh/page IDs.
- Maintain residency/page table and request queue with per-frame upload/eviction budgets.
- Cull instances, then traverse cluster hierarchy on GPU to projected-error targets.
- Test selected clusters against previous-frame Hi-Z; preserve conservative behavior for newly revealed content.
- Emit indirect commands/visibility IDs consumed by perf ticket 009: GPU-driven rendering — indirect multi-draw + GPU cull #28/perf ticket 008: visibility buffer (Nanite-style) replaces 4-MRT G-buffer #27.
- Feed missing-page requests back asynchronously; render the nearest resident ancestor until data arrives.
- Expose counters/debug views: selected LOD, rejected reason, page residency, requests, fallback clusters, triangles submitted/rasterized.
Quality behavior
- LOD error is expressed in pixels and scaled by quality tier/render resolution.
- Transitions must not crack. Prefer seamless hierarchy topology; use temporal/dither transitions only where necessary and test ghosting.
- Camera cuts and fast motion may reduce occlusion efficiency but may not make geometry disappear.
- Material boundaries and UV/tangent continuity must survive cluster construction.
Acceptance criteria
- Import and render a static stress asset with at least 10 million source triangles under a configurable residency budget.
- Camera motion produces no holes, cracks, missing-page flashes, or unbounded temporal trails in the fixed motion corpus.
- Projected geometric error remains at or below the selected pixel threshold in an automated image/geometry test.
- GPU culling and submission time scale primarily with candidate/visible clusters, not source triangle count; attach uncapped timings and counters.
- Peak geometry residency respects its configured budget within one page plus explicitly measured staging overhead.
- Skinned/translucent/deforming content is routed to the compatibility path with correct depth/composition and an inspectable reason.
- Asset corruption/version mismatch fails with a useful message and never reads arbitrary offsets.
- Existing immediate-mode and ordinary glTF models still render unchanged.
Likely files
tools/bloom-cook/- new cooked geometry modules under
native/shared/src/ native/shared/src/renderer/scene_pass.rs,hiz.rs,graph.rs, shadersnative/shared/src/models*.rsand scene data ownershipsrc/models/and world asset metadata as needed
Verification
Attach cooker determinism hashes, residency/culling statistics, fixed-camera stills, motion-sequence diffs, and before/after GPU captures on at least one discrete and one integrated GPU.
Non-goals
- Skinned-mesh virtualization.
- Runtime arbitrary topology editing.
- Making mesh shaders mandatory; use them only as an optional backend after feature/correctness qualification.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity
proggeramlug commentedon Jul 28, 2026
Issue #131 cooked-geometry v1 evidence
This checkpoint establishes Bloom's opt-in offline meshlet/page artifact at
revision
59c8df7. It does not enable a virtualized runtime path or changeaccepted pixels.
Static geometry proof
Two independent release cooks of
examples/renderer-test/assets/DamagedHelmet.glbare byte-identical:df089b23324fe8a8e00842a80b44894fd27b276c9124b8ff81dd77f8cf7b2cd2;67b0352a54d3d669d76d688f94899bcb8a665c561587123ea85d5d6a2b1061b0.The default 64-vertex/124-triangle leaf limits and 64 KiB page budget
therefore hold for a canonical textured glTF asset. Material identity and all
static shading vertex attributes remain in the artifact.
Compatibility proof
examples/test-gltf-watch/assets/Fox.glbproduces a valid 176-bytemetadata-only artifact:
skinnedreason;5e4125b6ba31649f5ebbd694d79242e19fdb0c561073beee08f490c12be063e2.The cooker does not reinterpret skinned geometry as static or silently omit
the reason. Equivalent records exist for morph targets, BLEND materials, and
non-triangle topology.
Integrity and determinism gates
Thirteen release tests plus strict Clippy pass. They cover:
truncated data, payload corruption, invalid indices, invalid hierarchy
links, bad counts/stride/bounds, and NaN/Inf data.
The repository's quick
quality-contractcomponent runs format, Clippy, andrelease tests for
bloom-cook; it passed in one second from a warm build.geometry-inspectvalidates the full file before reporting any records.Runtime neutrality
This commit changes only
tools/bloom-cook, documentation, and its CI gate:The current artifact contains leaf clusters only. Runtime loading, hierarchical
LOD construction, coarse fallback clusters, residency, GPU traversal, and
streaming remain later #131 milestones and must be qualified before any
end-state acceptance box is checked.
The detailed format and compatibility contract is in
docs/virtualized-geometry.md.proggeramlug commentedon Jul 28, 2026
Atomic hierarchy cooker checkpoint is now pushed in
845ff9cwith qualification evidence inf8308d1(PR #147).What this adds:
Damaged Helmet (
--hierarchy-levels 8): 15,452 leaf triangles / 247 leaf clusters -> 4,510 triangles / 100 level-3 coarse roots (70.8% triangle and 60.2% raw-payload reductions). Two independent artifacts are byte-identical:57f8cc200da84c7ab63cdcd7bf02cb0c8336a96777a2ccb7fc49beaa0af3fd10.Regression controls:
df089b23...);skinnedcompatibility artifact (5e4125b6...);I checked only the corruption/version-mismatch acceptance box. The unchanged-rendering box stays open until runtime integration is exercised end-to-end; stress asset, projected-pixel error, residency, GPU scaling, motion, and compatibility composition also remain open.
Evidence:
docs/evidence/issue-131-atomic-hierarchy-v1.mdand machine-readable JSON.proggeramlug commentedon Jul 28, 2026
Coarse-page placement checkpoint is pushed in
fe92915, with evidence inc22d89e.The qualified hierarchy is now remapped coarse-first without breaking atomic parent/child ranges. Root/streamable classes and LOD levels cannot share a page, roots must be the page-table prefix, and the strict reader rejects violations.
Damaged Helmet: all 100 roots occupy the first 8 pages; 468,570 raw root bytes become 469,360 logical page bytes (790 bytes packing overhead; 524,288-byte upper bound if a runtime allocates fixed 64 KiB slots). Total geometry payload is unchanged; page separation adds one 64-byte page record. Two outputs are byte-identical (
da0f68d7...), the default leaf artifact remains byte-identical, and 17 release tests plus strict Clippy pass.No runtime acceptance box is checked: this makes the coarse upload boundary exact, but runtime allocation/staging, eviction, ancestor fallback, stress residency, and GPU traversal still require their own evidence.
Evidence:
issue-131-coarse-page-prefix-v1.mdand JSON.proggeramlug commentedon Jul 29, 2026
Quantized static-vertex checkpoint is pushed in
2c51d09, with qualification evidence ina9f0135(PR #147).What this adds:
--vertex-format quantized32output as strict.bgeoversion 2;Damaged Helmet with
--hierarchy-levels 8: 2,937,040 -> 1,363,968 payload bytes (53.56% reduction), 48 -> 23 pages, and 469,360 -> 216,544 root-prefix bytes (53.86%). Position error is at most 0.00001532 object units, normal error 0.00342 degrees, and UV error 0.0004883. Two artifacts are byte-identical:6c8f924e....Sponza separately exercises real tangents: 0.00361 degrees maximum tangent error and 53.06% payload reduction. Its large tiled UVs expose a 0.01381 maximum binary16 error in the report; packed output remains explicit opt-in so asset policy can reject unsuitable error instead of silently changing pixels.
Regression controls:
df089b23...;da0f68d7...;No #131 acceptance box is checked: this is an offline storage/IO/residency prerequisite. Runtime loading, budget enforcement, traversal, fallback, image quality, and GPU timing remain open.
Evidence: Markdown and JSON.
proggeramlug commentedon Jul 29, 2026
Dependency progress from #136 / PR #147:
289ba31and16c58b0now place the qualified.bgeohierarchy behind a deterministic recipe key, immutable content-addressed chunk, and logical manifest. Hierarchy levels, meshlet/page limits, vertex format, source closure, and recipe revision all participate in the key; strict inspection validates the complete chunk before a cache hit.This removes artifact-identity ambiguity for later #131 runtime lookup and proves a zero-write incremental hit, but it does not enable runtime loading, residency, traversal, missing-page fallback, or pixels. No #131 acceptance box changes.
proggeramlug commentedon Jul 29, 2026
Further #136 dependency progress in
2effe1e/9ad5ca9: validated geometry manifests now compile into one deterministicbloom-asset-index-v1table. A future #131 loader can resolve a logical virtual mesh without scanning the manifest tree, andasset-index-inspectfails if any manifest/chunk changed without rebuilding the index.This remains offline-only and does not add residency, traversal, fallback, GPU work, or pixels. No #131 acceptance box changes.
proggeramlug commentedon Aug 9, 2026
Runtime archive/residency checkpoint is pushed in c2a0bd2 with evidence in 10ccfc2 on PR #147.\n\nWhat landed:\n- one bloom-geometry-format reader shared by the cooker and runtime, replacing the duplicated interpretation risk;\n- immutable runtime archive ownership with page slices exposed only after full structural, range, hash, vertex, hierarchy, and page-class validation;\n- indexed loading that additionally verifies complete-file length/hash, format version, payload hash, and source hash against the #136 identity;\n- a fixed-budget residency planner that pins the validated coarse-root page prefix;\n- atomic cluster-group transitions, deterministic LRU eviction with page-index tie-breaking, and nearest fully resident ancestor fallback;\n- telemetry for pinned/resident bytes and pages, uploads, evictions, exact/fallback resolutions, and unresolved requests.\n\nEnd-to-end runtime proof:\n- a valid encoded 3-level hierarchy is loaded through the production reader;\n- 448 root bytes stay pinned under a 1,120-byte budget;\n- fallback progresses deterministically from LOD 2 to 1 to exact LOD 0 as pages become resident;\n- the cache reaches exactly 1,120 bytes and never exceeds the budget;\n- a two-page 448-byte atomic group is rejected when only 224 streamable bytes are available, with state/counters unchanged;\n- corrupt magic and wrong index identity fail before an asset is exposed.\n\nDefault-path control:\n- no EngineState, Renderer, ModelManager, frame-graph, shader, or FFI owner was added;\n- the normal no-default-features renderer benchmark graph contains neither the new format crate nor SHA-256 code;\n- three interleaved detached-HEAD versus working-tree runs produced identical renderer-path telemetry in all six captures;\n- candidate medians were favorable on wall, CPU mean/p95, and GPU mean/p95. The magnitude is treated as machine noise, not a speedup, but rules out a regression.\n\nGates:\n- 382 shared unit tests passed, 1 existing ignore;\n- 59 golden tests passed, 2 hardware-only ignores;\n- 4 render-target tests and all visibility GPU parity/runtime tests passed;\n- 29 cooker tests and 39 quality-governance tests passed;\n- strict production lint, format, contracts, FFI parity, file ratchet, wasm, native models3d, and wasm models3d checks passed.\n\nEvidence:\n- https://github.com/Bloom-Engine/engine/blob/codex/rendering-wip-integrated-20260727/docs/evidence/issue-131-runtime-residency-v1.md\n- https://github.com/Bloom-Engine/engine/blob/codex/rendering-wip-integrated-20260727/docs/evidence/issue-131-runtime-residency-v1.json\n\nNo additional acceptance box is checked yet: this is logical CPU-side residency planning, not measured GPU allocation/staging. Next is stable runtime mesh/page IDs plus fixed GPU page allocation/upload, then projected-error traversal and visibility submission.
proggeramlug commentedon Aug 9, 2026
Fixed GPU page-residency checkpoint is pushed in
6dbbb02, with qualification evidence in8368cdcon PR #147.What landed:
Vertex3D/u32compatibility arena;VirtualMeshIds and generation-protectedVirtualPageIds;Real Metal readback proof on the 32-core Apple M1 Max:
Negative proofs:
Regression gates:
models3dand wasmweb,models3dchecks pass;bloom-geometry-formatand SHA-256;Evidence:
No acceptance box changes yet. Driver-internal
Queue::write_bufferstaging overhead still needs discrete/integrated capture, and the pool is not yet consumed by GPU hierarchy traversal or visibility submission. Next is projected-pixel-error selection with conservative frustum/cone behavior, followed by Hi-Z and missing-page feedback.proggeramlug commentedon Aug 9, 2026
GPU hierarchy traversal checkpoint is pushed in b863d12, with qualification evidence in 5010f3e on PR #147.
What landed:
Real Metal readback on the Apple M1 Max matches an independent CPU oracle exactly for selected records, page requests, and all counters. It covers LOD-0/middle/root thresholds, full frustum rejection, missing leaf pages, selected/request overflow, front/back cone behavior, non-uniform transforms, and a large camera cut plus opposing fast instance motion. The missing-page case keeps both resident middle clusters visible while requesting both leaf pages; missing-current-page and invalid-record counters remain zero.
The fixed pool readback now also proves the 4,096-byte cluster table byte-exact. The small fixture owns 12,288 physical bytes + 256 page-table bytes + 96 mesh-table bytes + 4,096 cluster-table bytes = exactly 16,736 bytes.
Regression boundary:
Evidence:
No acceptance checkbox changes yet. Next is bounded indirect visibility emission and raw cooked-page decode. Any selected-output overflow must route the whole affected virtual batch through compatibility rather than render partial geometry. Conservative previous-frame Hi-Z and asynchronous request feedback follow that correctness checkpoint.
proggeramlug commentedon Aug 9, 2026
Virtual draw/decode checkpoint landed on PR #147.
Code: 92daf0e
Evidence: https://github.com/Bloom-Engine/engine/blob/74d114b/docs/evidence/issue-131-virtual-draw-emission-v1.md
Machine-readable evidence: https://github.com/Bloom-Engine/engine/blob/74d114b/docs/evidence/issue-131-virtual-draw-emission-v1.json
What this adds:
Qualification on the exact code tree: governed quick lane passed in 51 seconds; 401 shared tests passed / 1 existing ignore, 59 goldens passed / 2 hardware ignores, all render-target and visibility GPU tests passed, plus strict lint, FFI/schema parity, file governance, wasm, 39 quality-contract tests, 3 visual fault tests, and 29 cooker tests. Native models3d and wasm web,models3d checks also pass. The no-default dependency boundary remains clean.
No acceptance checkbox changes yet. This is still explicitly ownerless and cannot affect shipping pixels or frame cost. Remaining activation work includes current/previous transform and material-remap ABI, #27 visibility/PBR composition, a bounded non-indirect-count path, previous-frame Hi-Z, asynchronous request feedback/streaming, crack and temporal corpora, per-MRT parity, 10M-triangle stress/timing, and integrated/discrete qualification.
proggeramlug commentedon Aug 9, 2026
PR #147 now includes the qualified temporal/material ABI checkpoint for virtual geometry.
Code: 1b4d828
Evidence/docs: f2b71d6
What is now proven:
Qualification: focused virtual suite 24/24; governed quick lane passed in 72 s with 403 shared unit tests plus the negotiated-device test, 59 goldens, visibility parity/runtime, wasm, strict lint/contracts, quality governance, fault tests, and cooker tests. Native models3d and wasm web+models3d checks also passed; the no-default dependency boundary is unchanged.
Evidence: docs/evidence/issue-131-temporal-material-abi-v1.md
No acceptance box changes yet. The next slice is opt-in visibility-ID/raster/PBR integration with exact per-MRT parity and a separately qualified bounded path for adapters without indirect-count support.
proggeramlug commentedon Aug 9, 2026
Virtual visibility raster checkpoint is qualified and pushed.
Code:
6d1183d(collision-free compatibility/virtual ID namespaces),688db93(raw resident-page ID/depth raster),0ab0629(contract coverage). Evidence/docs:1fce846,docs/evidence/issue-131-virtual-visibility-raster-v1.{md,json}.The real Metal oracle selects/emits/rasterizes four fixture clusters into
Rg32Uint; every covered pixel decodes as virtual draw 0..3 / primitive 0, background remains unambiguous, and both foreground/background are non-empty. The governed quick lane passed in 59s (406 shared tests plus the device, golden, visibility, cooker, governance, Clippy/format/contracts/wasm/native gates).Safety boundary: this remains unattached to
Renderer, adds zero default-path allocation/work/pixel change, routes masked content and unsupported devices to compatibility, and has no shipping fixed-count fallback that could execute stale commands. Production submission requires indirect-count today.No acceptance checkbox changes yet. Next gated slice is virtual PBR reconstruction with exact current/previous clip, normal/tangent/tint/material/face state, and all established MRTs; then explicit compatibility composition and a bounded non-count path.
proggeramlug commentedon Aug 9, 2026
Virtual PBR reconstruction checkpoint is qualified and pushed.
Code:
9b3130fEvidence:
docs/evidence/issue-131-virtual-visibility-pbr-v1.{md,json}Evidence commit:
311b6ffWhat landed:
shade_main_sceneand returns the production four MRTs;Governed quick lane: PASS in 122 s, including 410 shared tests + negotiated device, 59 golden tests, visibility parity/runtime, wasm, strict Clippy/contracts, cooker and governance. Three hardware scenes built/captured; visual comparison is explicitly unqualified because approved portable baselines are absent.
No acceptance checkbox changes yet: this consumer remains explicit and unattached. Next is production ownership/routing and four-MRT composition, followed by a bounded no-indirect-count submission path.
proggeramlug commentedon Aug 9, 2026
Bounded no-indirect-count submission checkpoint implemented and pushed in 44d53ec (draft PR #147).
Problem closed by this slice: Apple M1/Metal exposes primitive index and indirect first instance but not MULTI_DRAW_INDIRECT_COUNT. The virtual visibility raster could be constructed there, yet production submission required an indirect count or a test-only exact draw count.
Fallback design:
Cost and capability behavior:
Apple M1 Max / Metal verification:
3 remaining items
proggeramlug commentedon Aug 26, 2026
Stable checkpoint pushed: 660190b implements bounded asynchronous GPU missing-page feedback and in-memory page streaming; 1461545 records the qualification evidence.
Verified on the exact committed tree:
Evidence: docs/evidence/issue-131-async-page-feedback-v1.md and .json.
No acceptance checkbox changed yet. Remaining work includes store/index IO behind this boundary, conservative previous-frame virtual Hi-Z, Bistro motion/parity, 10M-source-triangle stress, and cross-backend qualification.
proggeramlug commentedon Aug 26, 2026
Conservative previous-frame virtual Hi-Z is now implemented and qualified on branch codex/rendering-wip-integrated-20260727.
Code checkpoint: e930a5e
Evidence checkpoint: 9e0d2ad
Evidence: docs/evidence/issue-131-conservative-virtual-hiz-v1.md and .json
Verified contract:
Metal oracle: stable captured instance selected 0 clusters and culled exactly 2 groups; camera motion and a new instance each stayed visible and selected all 4 fine clusters. Production four-MRT integration reported 2 submitted captures and 1 history instance without touching unrelated pixels.
Qualification: 458 shared tests passed / 1 existing ignore; full GPU golden corpus 77 passed / 2 hardware-specific ignores; 38 focused virtual tests; production renderer Metal integration; native no-default; wasm web+models3d; format and strict quick lint all passed. File-size failures remain limited to the same 3 pre-existing grandfathered files.
No acceptance checkbox changed yet. Remaining gates are async #136 store/index IO, approved Bistro motion/parity, 10M-source-triangle stress, and integrated/discrete Metal/Vulkan/D3D12 timing and quality evidence.
proggeramlug commentedon Aug 27, 2026
Detailed Bistro virtual-geometry parity and camera-return qualification is now verified and pushed.
Code checkpoint: 5c26c6a
Evidence checkpoint: d09d5e8
Evidence:
The real 1,176-placement Bistro exposed and now guards four independent faults: incomplete multi-level atomic replacement ranges, atomic groups falsely culled against different frustum planes, virtual opaque back-face ownership diverging from Bloom's ordinary scene path, and cooked vertex colors omitting the runtime material base-color factor.
Final 640x360 Metal gate after 180 streaming warmup frames:
The gate is regression-sensitive: the original hierarchy failed at 10.7369 mean RGB / 0.76937 SSIM, and hierarchy/face fixes without the cooked color correction still failed at 9.3034 / 0.79258.
Qualification: 459 shared tests passed / 1 existing ignore, 39 focused virtual GPU tests, 77 GPU goldens passed / 2 hardware/manual ignores, 31 cooker tests, strict shared/cooker Clippy, wasm web, native no-default, formatting, and diff checks passed. The patch adds no file-size overage; the repository ratchet remains red only on the same three unrelated pre-existing files.
No acceptance checkbox is changed by this comment: the Bistro endpoint/motion gate is now closed, but the issue's broader 10M stress and full motion-corpus criteria remain. Next is the 10-million-source-triangle fixed-residency stress and uncapped timing/counter qualification, followed by asynchronous #136 store/index IO and cross-backend evidence.
proggeramlug commentedon Aug 27, 2026
10M-source-triangle fixed-residency stress is now qualified on
codex/rendering-wip-integrated-20260727.Checkpoints:
741cc5a— deterministic 10M source generator + runtime gate, source-bounded root dispatch, conservative group-union Hi-Z, production profiler coverage19b77e8— permanent evidence and roadmap updateQualified Metal workload (Apple M1 Max, 640x360):
The gate enforces 16.67 ms wall mean, 8 ms GPU mean, 12 ms GPU p95, 3 ms selector mean, and 0.5 ms draw-emission mean.
Validation: shared 459 passed / 1 existing ignored; GPU golden 77 passed / 2 expected ignored; cooker 32 passed with strict Clippy; quality governance 41 passed; shared strict Clippy, wasm
web, native no-default, formatting, and example inventory passed. The line ratchet remains red only for the same three unrelated pre-existing files; this checkpoint adds none.Evidence:
docs/evidence/issue-131-10m-stress-v1.{md,json}.Remaining #131 work is asynchronous #136 store/index-backed archive I/O behind page feedback, then discrete/cross-backend timing and quality evidence. Runtime activation remains opt-in.
proggeramlug commentedon Aug 27, 2026
Native #136 store/index-backed archive I/O behind #131 GPU feedback is now qualified and pushed.
Checkpoints:
c601626— strict bounded async store/index resolver1cc6f1a— bounded file-page worker and atomic residency uploadc300880— evidenceThe same deterministic 10M-source-triangle artifact now runs without retaining its complete 582,052,704-byte archive. The runtime holds metadata + 50 coarse-root pages, then demand-read 76,226,352 bytes through 370 completed atomic requests with zero failures. It settled at 955 pages in the hard 64 MiB physical pool and ended with zero pending, reserved, ready, or in-flight I/O.
Moving-camera Metal gate (180 warmup + 120 measured at 640x360): 6.7390 ms wall mean, 4.4761 ms GPU mean, 7.8572 ms GPU p95, 1.6054 ms hierarchy selection, 0.1115 ms draw emission. Maximum caller-side request/poll duration was 0.1239 ms. There were zero fallback groups, missing-current pages, selected/request overflows, invalid records, or depth-limit fallbacks.
Qualification: focused virtual geometry 34/34; shared library 465 passed / 1 expected ignored; GPU goldens 77 passed / 2 expected ignored; strict shared Clippy; wasm
web,models3d; format and diff checks. Evidence: Markdown and JSON.I marked the independently verified 10M-under-configurable-residency acceptance item complete. The broader motion corpus, geometric-error oracle, scaling proof, staging-overhead accounting, complete compatibility proof, and discrete/cross-backend evidence remain open. The next #131 qualification slice is discrete-GPU and cross-backend timing/quality.
proggeramlug commentedon Aug 27, 2026
Projected-error geometry qualification is now verified and pushed.
Checkpoints:
6ea4c5b— independent source-vertex-to-parent-surface BVH bound, projected-pixel oracle, geometry recipe v2428f8b6— permanent evidenceThe previous cooker could understate an accepted hierarchy edge: independently measured deviation was
0.00053679076, while the stored simplifier error was0.00040672. Recipe v2 retains the simplifier metric but takes the maximum with an independent one-sided positional measurement and accumulates that conservative value through the hierarchy. The automated oracle walks every LOD0 descendant for every parent group and verifies both the stored object-space bound and the exact one-pixel projected threshold.Two clean 10M cooks were byte-identical: 582,052,704 bytes, 245,500 clusters, 8,496 pages, artifact
13ab0d15..., index5c69a9a6.... Cook wall times were 92.35 and 100.27 seconds versus the prior 121.41-second reference.The corrected file-backed 10M Metal gate passed with the same 955 resident pages in the fixed 64 MiB pool and zero fallback/missing/overflow/invalid/depth-limit/pending events. Timings: 6.0585 ms wall mean, 4.3993 ms GPU mean, 7.7775 ms GPU p95, 1.6000 ms hierarchy selection, 0.1091 ms draw emission.
Evidence:
I marked only the projected-geometric-error acceptance item complete. Discrete/cross-backend qualification remains queued at https://github.com/Bloom-Engine/engine/actions/runs/33046639665 because the repository currently reports no online self-hosted runners; the remaining #131 acceptance items stay open.
proggeramlug commentedon Aug 27, 2026
Candidate/visible-cluster scaling is now qualified and pushed.
Checkpoints:
219344b— reusable 1/10/100-instance scaling sweep on one immutable 10M archivedea1add— permanent evidenceAll points retain the same 10,000,000-source-triangle, 245,500-cluster, 8,496-page artifact. Only the submitted placement set changes. Results on Apple M1 Max Metal:
Instances grew 100x, candidate groups 110x, and selected clusters 143.94x, while hierarchy-selection GPU mean grew only 1.53x. The one-instance point visits four groups despite the unchanged complete archive, guarding source-filtered root dispatch against archive-wide triangle/cluster scans. The full point remained at 955 pages in 64 MiB with zero fallback/missing/overflow/invalid/depth-limit/pending events; GPU mean/p95 were 4.1692/6.9526 ms and wall mean was 6.0774 ms.
The clean cross-backend driver now enforces this sweep for Metal, Vulkan, and DX12 after one cook. Strict renderer lint and all 58 quality-tool tests passed.
Evidence:
I marked only the GPU culling/submission scaling acceptance item complete. Hardware cross-backend, motion-corpus, staging-overhead, complete compatibility, and ordinary-model invariance work remains open.
proggeramlug commentedon Aug 27, 2026
Peak residency and streaming staging are now explicitly measured, regression-gated, and pushed.
Checkpoints:
2fabbaf— lifetime peak residency/I/O telemetry and exact 10M stress assertions27731df— permanent evidence and runtime contract documentationApple M1 Max Metal, file-backed 10M archive, 100 placements, 180 warmup + 120 measured frames:
The gate now requires exact page-stride arithmetic, lifetime peak residency within the pool, exact metadata sums, exact feedback allocation from public record sizes, CPU peak within its configured budget, and combined staging equal to GPU readback plus CPU peak. The run retained zero fallback/missing/overflow/invalid/depth-limit/pending events and all established timing limits passed.
Validation: 45 focused virtual-geometry release tests, enabled/disabled capability-report real-GPU tests, the file-backed 10M stress, strict renderer Clippy, formatting, and diff checks passed. No render pass, allocation, shader branch, budget, or accepted pixel changed.
Evidence:
I marked only the peak-residency/staging acceptance item complete. Motion-corpus, complete compatibility composition, ordinary-model invariance, and discrete cross-backend evidence remain open.
proggeramlug commentedon Aug 28, 2026
Branched hierarchy traversal and motion-residency stabilization are verified and pushed.
Checkpoints:
5b6c5c3— bounded DFS traversal of every authored child replacement range, finite projected-error priority, priority-aware streaming admission, selected-group page-use feedback, and full ancestor-closure protection371ba36— expanded Bistro motion/path-return and exact clear-background leak gates plus the qualified 10M runtime gateRoot cause: the selector previously followed only the first cluster child range in an atomic group. Detailed Bistro groups can branch into several independent child ranges, so sibling branches disappeared as refinement progressed. The GPU and CPU selectors now visit every branch with a 32-entry bounded stack; overflow retains the complete resident parent. Uniform groups retain the cheap single-range fast path. Page-use feedback is emitted only for final selected groups, and CPU archive metadata reconstructs the complete atomic ancestor closure so asynchronous streaming cannot evict a live sibling branch between readbacks.
Clean-tree qualification on Apple M1 Max Metal:
The selector cost is higher than the earlier incorrect single-branch path because it now renders the authored sibling branches, but the established 3 ms selector, 8 ms GPU mean, 12 ms GPU p95, 16.67 ms wall, and 0.5 ms draw gates all pass. The rejected linked-header experiment was removed because it did not improve timing.
I am leaving the broad motion acceptance checkbox open until this exact checkpoint also receives interactive Bistro confirmation and the remaining very subtle SSGI ground-line observation is handled separately.
proggeramlug commentedon Aug 28, 2026
Verified and pushed checkpoint 201b492 (
fix(renderer): pack pinned virtual roots across lods).Root cause: mesh-first coarse-root ordering can cross hierarchy levels, but the cooker still opened a new 64 KiB physical page for every root LOD transition. Detailed Bistro packed the unchanged 58,229,152 root payload bytes into 1,593 pinned slots, leaving only 455 of 2,048 slots for streamed detail and producing virtual-only fallback geometry.
Correction: root and streamable clusters remain strictly separated; streamable pages remain isolated by LOD; pinned roots may now share a page across LOD because they have the same lifetime/residency policy. Geometry recipe is bumped to v3; wire format remains v2. No runtime pass, allocation, shader branch, or per-frame work was added.
Detailed Bistro clean-cook result:
593a4b6b31da0498873986ae9f9488779dee4f7db5734533e12a0fd25eca1c29)Performance qualification also passed on Apple M1 Max/Metal with the fixed 64 MiB stress pool at 1/10/100 placements of the 10M-source-triangle corpus. The 100-placement run measured 5.265658 ms mean GPU, 8.426207 ms p95 GPU, 2.138291 ms hierarchy selection, and 0.116362 ms draw emission, with zero fallback/missing/overflow/IO-failure events.
Regression coverage now gates <=920 detailed-Bistro root pages and >=0.95 SSIM / <=3 mean RGB for direct and moving parity. Full shared/models3d tests, cooker 35/35, strict cooker/format Clippy, determinism, and diff/format checks passed.
Evidence:
docs/evidence/issue-131-root-page-packing-v1.{md,json}.proggeramlug commentedon Aug 29, 2026
Checkpoint 9a30270 adds a real-GPU mixed virtual/ordinary compatibility oracle.
Evidence:
Validation on native Metal / native-full:
This advances the compatibility/depth/composition evidence, but I am leaving the acceptance checkbox open until the skinned/deforming compatibility path is exercised end-to-end as well. The fixed camera-motion criterion also remains open.
proggeramlug commentedon Aug 29, 2026
Skinned/deforming compatibility fix is pushed in checkpoint 2893929.
Root cause fixed:
Real-GPU Metal/native-full oracle now proves:
Validation:
Translucent and skeletal-deforming compatibility are now exercised end-to-end. I am leaving the broad acceptance checkbox open while I audit whether morph-target deformation needs a separate runtime contract; the cooker already reports morph-targets, but the ordinary importer currently has no morph animation owner.
proggeramlug commentedon Aug 29, 2026
Compatibility acceptance is now checked after the code and evidence checkpoints 9a30270, 2893929, and 5f732b9.
Permanent evidence:
Scope boundary: morph-targets remains a versioned inspectable cooker exclusion and is never virtualized as static geometry. Implementing ordinary glTF morph animation is a separate model/animation ownership feature; #131 now proves the virtual path preserves the established compatibility owner, shared depth/composition, live skin palettes, and explicit failure where the palette contract cannot be maintained.
The camera-motion and ordinary-glTF-invariance acceptance lines remain open.
proggeramlug commentedon Aug 29, 2026
Ordinary glTF / immediate-mode invariance is now qualified at
405d383bcb800ab0b1e819f70cfd1b4d339141de(evidence commitf4824e6).load_model_with_textures_from_source_pathloads the checked-inDamagedHelmet.glb, then the established cached-model path draws it.Permanent evidence: ordinary glTF invariance v1 (machine-readable JSON).
The ordinary-model acceptance box is checked. The fixed cross-backend camera-motion corpus remains open.
proggeramlug commentedon Aug 29, 2026
Complete
bistrox.gltfintegrated-GPU motion qualification is fixed, verified, and pushed.Checkpoints:
60249af— source sanitation/material routing and exact single-sided virtual face ownershipb4e7c92— permanent complete-scene evidenceThe full source contains 2,909 placements (2,404 virtual + 505 compatibility) and 1,736,174 eligible source triangles. The cooker now maps 847 malformed optional tangent vertices to the established missing-tangent sentinel, routes 42 active-transmission plus three alpha-blend primitives to inspectable compatibility ownership, and reproduces the same eight-level payload hash across two cooks:
e379cdb4...e5f.Root cause of the late indoor failure: virtual visibility rendered back faces for single-sided opaque glTF materials even though the ordinary retained-scene pipeline culls them. At motion steps 25/30, the reverse side of an interior wall covered the view. Restoring exact glTF face ownership removes the occluder; this was not a streaming, Hi-Z, missing-page, history, or overflow failure.
Apple M1 Max / Metal, 640x360, 180-frame warmup + 30 outbound + 30 return + 30 settle:
Validation: complete release motion gate passed; shared library 481 passed / 1 existing ignored; cooker 48/48; geometry format 1/1; strict cooker/format Clippy; formatting and diff checks.
Evidence: Markdown and JSON.
I am leaving the camera-motion acceptance checkbox open. This completes the integrated-GPU half, while #131's verification contract still requires the same enforced corpus on a discrete GPU/backend; no threshold will be weakened for that run.