research(053b): lazy GT crop warps — storage fix, equivalence-proven + scaled deployment run - #139
Open
dompm wants to merge 6 commits into
Open
research(053b): lazy GT crop warps — storage fix, equivalence-proven + scaled deployment run#139dompm wants to merge 6 commits into
dompm wants to merge 6 commits into
Conversation
…op duplicates - crop_sim.py writes only cropped photo sheets + detail patches + the homography in meta.json; warp_channel() exported as the single warp convention. - dataset.py GlassDelightDataset(crop_view=True) lazily warps photo + all GT channels at load time, in FILE space (before nonlinear decodes) — exact match to the old materialized crops. - verify_lazy_crop_053b.py: equivalence test on the 68 dual-representation pilot samples — 68/68 OK, 986 channel files BIT-EXACT (float EXR maxdiff 0.0), loader wiring maxdiff 0.0. Run before slimming (exit 0 gate). - Retro-slim executed: pilot 12G -> 4.0G (918 redundant files deleted). - 053 report storage claim amended (render-only figure); 053b report supersedes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying vitrai with
|
| Latest commit: |
68bd49f
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://1d89b960.vitrai.pages.dev |
| Branch Preview URL: | https://research-delighting-053b.vitrai.pages.dev |
CTO catch: crop_sim's patches were emitted but dataset.py never referenced them. GlassDelightDataset(patch_prob=0.2) now serves a native-resolution registered detail patch on a fraction of draws (fine texture the 768 work-res destroys); reflect-pad to crop size with valid=0 in the pad (uniform batch shapes, no loss from invented content). Patch inherits its sample's holdout split. Unit-tested: registration bit-exact, loads at shape, split respected both directions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ports a5577a2+36bc550+f6bd8c1 from research/delighting-037 (drift-aware: 8-channel sigma_s AuxHead + phone-ISP loader preserved): latent-space T loss (MSE vs z_T_hat, shadow map adaptive-pooled), need_T opt-in decode, conf gated on decoded steps. test_grad_flow.py ported + extended to 7 heads and wired UNCONDITIONALLY at train_loop top. Acceptance: all heads nonzero grad (T 0.60), mutation-verified (detached z_T_hat fails naming ['T']), 20-step smoke T loss moves 0.1241->0.1003->0.1065 (was bit-frozen under the bug). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…et draws _cache was unbounded (~10-19GB RAM over a full pilot); now an OrderedDict LRU capped at 64 (~2.2GB). Sheet-path sample_crop now guarantees exactly self.crop via reflect-pad + valid=0 (non-square crop_view grids could crash np.stack). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t device row eval_foundation ran only clean linear renders; now every test photo also runs through each ISP preset (deterministic rng), metrics per-preset with wide_edge (held-out device) broken out, clean row kept as headline for continuity. Verified on the smoke adapter: 172 test samples x 6 conditions. Report updated with the full pre-flight fix section + acceptance evidence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…boards refreshed Disk guard stopped the farm cleanly at 348 samples (68 original + 280 net new, one disk-guard-interrupted partial deleted) -- 74% of the planned 468, per the lead's decision not to resume rendering. 17/17 recipes, 23 HDRIs, shadow 31%/ front-light 66% (near target), storage 60.6MB/sample mean (confirms the 053b lazy-crop fix holds at scale). Per-stage timing reconstructed from the raw [TIMING] log (dump_timings never ran -- process was killed by the guard): gt_render dominates at 68.6% of render time, the cost lever for cloud pricing. Boards refreshed over the full set (overview capped to 40 representative samples for legibility). Backup to gdrive in progress. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Post-merge audit follow-up to #138: the 053 storage claim measured renders only —
crop_simmaterialized ~150MB/sample of cropped GT duplicates (true cost ~176MB/sample, 12G for the 68-sample pilot). Full write-up:reports/053b-crop-storage-fix-scaled-run.md.Fix — GT crops are lazy
crop_sim.pynow writes only the cropped photo sheet(s) + detail patches + the 3×3 homography/capture_geometryin meta.json;warp_channel()exported as THE warp convention (LINEAR continuous / NEAREST labels / BORDER_REPLICATE).foundation/dataset.py:GlassDelightDataset(crop_view=True)lazily warps photo + every GT channel at load time from the stored homography, applied in FILE space (before nonlinear decodes) so it reproduces the old materialized files exactly.Equivalence proven BEFORE deletion (committed test)
verify_lazy_crop_053b.pyon the original 68 dual-representation pilot samples: 68/68 OK, 986 channel files bit-exact (float EXR maxdiff 0.000e+00; labels exact), loader wiring maxdiff 0.0. The spuriously-warpedcrop/hand_mask.exr(caster texture, not a GT channel) is documented + excluded.Two-view training diet (CTO catch)
crop_sim's detail patches were emitted but never consumed.GlassDelightDataset(patch_prob=0.2)now serves native-resolution registered detail patches on a fraction of draws (reflect-pad to crop size,valid=0in the pad); patches inherit their sample's holdout split. Registration bit-exact, split respected both directions (unit-tested).Pre-flight fixes (lead's foundation/ review, before any training run)
backbone.decode()'storch.no_grad()was severing T's own supervision (the LoRA only got gradient via the aux path). Latent-space T loss +need_Topt-in decode +test_grad_flow.pywired unconditionally attrain_loop's top. All 7 heads pass with nonzero grad, mutation-verified, 20-step smoke shows T loss moving (was bit-frozen under the bug).eval_foundationnow runs every test photo through each ISP preset (not just clean renders) — per-preset metrics with the held-outwide_edgedevice broken out.self.crop(reflect-pad + valid-mask) — a non-squarecrop_viewgrid could crashnp.stack.The scaled run — 348 samples (final)
Seeds 500–739 (68 original + 280 net new). Disk guard stopped the farm cleanly at 1.8GB free on 2026-07-19; per the lead's decision, rendering was NOT resumed — 348 is 74% of the originally planned 468 and judged an adequate deployment-pilot size. 17/17 recipes, 23 HDRIs, storage 60.6MB/sample mean (confirms the lazy-crop fix holds at scale), holdout partition covers every reserved family. Per-stage timing reconstructed from the raw render log for future cloud-run pricing (
gt_render= 68.6% of render time, the dominant cost lever). Boards refreshed over the full set. NOT 1k locally, plainly — 20k/1k remain cloud jobs.Full
pilot_053_out(20G, 348 samples) backed up togdrive:vitrai-lab-backup/pilot_053_deploymentvia rclone.Do not merge — lead review requested. This closes out 053b; the train-once pilot is now a live decision for the lead/CTO.
🤖 Generated with Claude Code