Repository navigation
test(dpt): use reproducible native-scale hash references - #44
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SMP DPT's full-width and reduced numerical references now use a counter-based SplitMix64 hash of each parameter's name and flat index, scaled to the standard deviation of its pinned native initializer. Fixed BatchNorm calibration uses separate integer-generated inputs. Head and logits meet 1e-4 against float32 PyTorch; every intermediate's mirror error against float64 PyTorch is no larger than float32 PyTorch's own. No strict expected failure remains.
The three reference designs, in order:
dpt-integer-offline.jsonremains unchanged as history.dpt-fan-in-offline.json, committed at ce619d9, remains unchanged as history.dpt-hash-offline.jsonis committed unchanged, with per-parameter initializer/std/fan-in metadata and per-output scale/error/decision measurements. Five known hash answers and bit-identical second tensor builds are asserted.Each design was declared before it ran and its evidence measurement ran once. The final recipe was not adjusted after seeing results. The native-init records also remain unchanged. The periodic pattern's coherent low-rank matrices caused amplification even after fan-in scaling; the final hash removes that repeated structure while matching native initializer scales.
As approved by the maintainer, CONTRIBUTING now specifies name-and-position integer hashes scaled to each native initializer's standard deviation for model numerical references. AGENTS corrects the cross-reference to the bold “Model references follow the same rule” paragraph under “Add a block”. The graph fixtures'
_fixed_weightsimplementation stays unchanged.Base revision: 127f66c.
Changed files by layer:
reference.py,diagnostics.py, newinitialization.py, andtest_vit_b16_dpt224.py.vit_b16_dpt224_numerics.pyand the compact fixture's description invit_dpt.py.vit-b16-dpt224.md,converter.md,api.md,model.md; the approved CONTRIBUTING sentence and AGENTS cross-reference.Validation on clean merge head 08406f7, including main at 127f66c (two threads; heavy jobs serialized):
graph_identity.py --check, usingl00-golden-graphs.json: 36 fixture graphs equal; 9 trained graphs not compared because checkpoints are unavailable.graph_identity.py --check-ext, usingx00-g1ext-golden-graphs.json: 19/19 equal. SegFormer asset-backed raw inputs differ from the golden environment; all compared graphs remain byte-identical.block_catalogue.py --check: in sync.live_fixtures.py --offline vit_dpt_reduced: passed, zero cloud calls. Largest absolute mirror/PyTorch errors: reassemble0 3.4924596548080444e-10; reassemble1 2.9103830456733704e-10; reassemble2 5.820766091346741e-10; reassemble3 5.238689482212067e-10; fusion0/1/2 9.313225746154785e-10 each; fusion3 1.862645149230957e-9; head/logits 9.313225746154785e-10 each.live_fixtures.py --plan vit_dpt_reduced: 7 requests, 0 assets/tasks, largest payload 1,944,662 bytes and 31,360 values; within budget. Fixture weights remain unchanged, so no new live run was needed.test_vit_b16_dpt224.py,test_smp_dpt_converter.py,test_timm_vit_converter.py: 92 passed. Includes DPT converter boundaries at both widths, all seven pinned reduced live hashes, and all 68 Prithvi preservation graphs. The reduced inline converter gate compares native SMP literals on both sides, independently of the hash numerical references.sitecustomize: 14 passed, 8 skipped, no failures. No packages changed.Final one-shot numerical result:
All 20 intermediates per width meet the float64 ordering rule. Full-width fusion3 peaks at 51.930015563964844 (float32 step 3.814697265625e-6); mirror/float32 error is 0.00018787384033203125, mirror/float64 error 4.644907568440715e-5, and float32/float64 error 0.00018706553028202677. This is not a 1e-4 intermediate agreement claim against float32. Reduced fusion3 peaks at 28.775081634521484 (step 1.9073486328125e-6).
Graph identity, offline mirror agreement, reduced live agreement, and model accuracy remain separate. The new numerical references have offline evidence only. The compact reduced live fixture remains unchanged. Full-width live execution remains deferred; model accuracy is unmeasured. No weights, inputs, calibration or tolerances were adjusted after the final comparison.