research(053): deployment-capture realism — phone pipelines, finite-depth scenes, crop sim, holdouts + 1k pilot - #138
Merged
dompm merged 5 commits intoJul 17, 2026
Conversation
Scope A: foundation/phone_pipeline.py — 5 named COMPLETE device ISP pipelines (neutral_mid, bright_window_hdr, low_light_handheld, android_punchy, wide_edge) applied in physical order (AWB/lens-shading/CA/blur/local-HDR+halos/denoise/ sharpen/sat/rescale/JPEG-HEIC), correlated params per device. dataset._augment_photo now calls it instead of the 4 independent knobs. Scope C: multi-axis holdout in dataset.py (EVAL_PROTOCOL §3b-ext) — reserve whole texture-generator families (ring-mottle, confetti-shard, cathedral-red, dark-slate), ~20% of HDRIs by hash, one camera preset (wide_edge), and perspective-rectified geometry, all test-only. holdout_reason() deterministic from meta.json. Scope E: eval_foundation.py scores σ_s (048-owed) — σ_s-MAE + structured-checker relight L1 vs GT-map relight (045/046 methodology). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…warding Scope B (generate_synthetic.py, all gated behind --deploy-scene; --validate untouched): - SHADOW OVERHAUL: generate_shadow_mask draws VARIED silhouettes (hand at angle, forearm, phone edge, blob cluster) from a random side/scale/tilt, replacing the 4 hardcoded axis-aligned rectangle sets. Shadow PRESENCE is a sampled probability (deploy 0.3), not a forced pair for every identity; --shadow-pairs restores dense pairs for the shadow-supervision path. - MIXED FRONT/BACK LIGHTING AS NORM: add_deploy_lighting_and_background adds a soft front area light (behind camera, never in frame) + deploy-default specular + dim reflective interior. White-marker self-emission hack drops 0.6->0.15 (verified: white marks render under the new front light). - FINITE-DEPTH TEXTURED BACKGROUNDS at sampled 10cm/50cm/2m (or HDRI-only), with a nearer shelf-edge/mullion bar at a different depth (depth discontinuity). GT emission renders defensively hide the new objects; gt_B captures them correctly. - --cover-recipes cycles all 17 recipes (kills the stale --count==5 special case). Scope D (render_farm.py): build_cmd now forwards --gt-b --gt-aov --no-tex-dump --exr-codec --specular --fixed-ev --no-marks + all 053 scene flags (previously silently dropped, which would omit gt_B/AOV supervision and blow the storage budget). Scope F infra: crop_sim.py (post-pass user-crop homography applied identically to photo+all GT channels; emits cropped sheet + registered detail patches; stamps capture_geometry+transform into meta) and build_boards_053.py (contact-sheet boards). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- docs/external/053-dataset-capture-review.md: CTO-commissioned review of trunk 8c8877a, verbatim, with provenance frontmatter (lead-verified 5 code claims). - EVAL_PROTOCOL.md -> v1.2: §3b-ext synthetic FAMILY-level holdout (generator families, taxa, HDRIs, camera presets, capture geometries reserved test-only); frozen REAL set §3c unchanged. - reports/053-deployment-capture-realism.md: full write-up (scopes A-F, validate gate, honest limitations, deferred items, scale commands). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deployment pilot rendered through render_farm with full production flags (--deploy-scene --cover-recipes --gt-b --gt-aov --no-tex-dump --exr-codec DWAA). Boards committed (renders gitignored): overview / crop-workflow / shadows. <=80MB/sample; shadow 25%, front-light 71%, bg depths 0.1/0.5/2/inf. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying vitrai with
|
| Latest commit: |
bb379cc
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://8c456eb3.vitrai.pages.dev |
| Branch Preview URL: | https://research-delighting-053-capt.vitrai.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Response to the CTO-commissioned external dataset review of trunk @ 8c8877a (committed verbatim as
docs/external/053-dataset-capture-review.md): make the synthetic dataset faithful to the deployment capture process, so a model can't score by learning "Blender glass grammar". Full write-up:reports/053-deployment-capture-realism.md.A — Phone-ISP presets (
foundation/phone_pipeline.py)5 named COMPLETE device pipelines (neutral_mid, bright_window_hdr, low_light_handheld, android_punchy, wide_edge) in physical ISP order — AWB error, lens shading, CA, motion/defocus blur, local HDR/tonemap with halos, denoise, sharpen with overshoot, saturation/per-channel clip, rescale, JPEG/HEIC — with CORRELATED per-preset parameters, replacing the 4 independent knobs in
dataset._augment_photo. Scene-linear in/out (report 025). Preset strip:results/053/phone_pipeline_strip.jpg.B — Scene realism (
generate_synthetic.py,--deploy-scene)--shadow-pairskeeps the pair path).--cover-recipescycles all 17 recipes (kills the stale--count==5case).C — Holdouts (
foundation/dataset.py+ EVAL_PROTOCOL v1.2 §3b-ext)Whole FAMILIES reserved test-only, deterministically: generator families/taxa (ring-mottle, confetti-shard, cathedral-red, dark-slate), ~20% of HDRIs by hash, one camera preset (wide_edge), perspective-rectified capture geometry. Frozen REAL set §3c untouched. Final gate (untouched real user captures) is a CTO action item — noted, out of scope.
D — render_farm.py fixes
build_cmdnow forwards--gt-b --gt-aov --no-tex-dump --exr-codec --specular --fixed-ev --no-marks+ all 053 flags (previously silently dropped — a 20k run would have lost gt_B/AOV supervision and blown the storage budget).E — σ_s eval (048-owed)
eval_foundation.pyscores σ_s-MAE + structured-checker relight L1 (pred-σ_s relight vs GT-σ_s relight, 045/046 methodology), rows in eval.json + baseline_ladder.md. Unit-tested (perfect→0, no-scatter→full penalty).F — Deployment pilot
68 samples (34 identities × 2 lightings, 17/17 recipes, 23 HDRIs) rendered THROUGH the farm with production flags (
--no-tex-dump --exr-codec DWAA --gt-b --gt-aov), ≤80MB/sample. Crop-sim post-pass (registered homography + detail patches, transform in meta.json). Renders gitignored; committed boards:results/053/boards/(overview / crop-workflow / shadows).Validate gate
17/17 recipes pass uniform-backlight T-agreement (MAE 0.002–0.030, historical band); deploy features are fully gated out of
--validate(re-render agreement ~1e-5).No training launched. Modal/payment untouched. Do not merge — lead review requested.
🤖 Generated with Claude Code