From ccea06fc9925e97223931cd20dec441d89f878f5 Mon Sep 17 00:00:00 2001 From: Noel Ivan Ulloa <127597880+noelEOS@users.noreply.github.com> Date: Tue, 3 Nov 2026 21:31:35 +0800 Subject: [PATCH] Scope the 1e-4 target to block fixtures and model outputs The 1e-4 absolute target against float32 PyTorch now states what it applies to: every block fixture's outputs and a model's final outputs. A model's intermediate outputs are recorded and checked against a float64 run of the same PyTorch model: the mirror must be no farther from it than float32 PyTorch is. Large intermediate values put 1e-4 out of reach of any float32 implementation, PyTorch's included. Every comparison records its scale: the largest value, the float32 step there, the errors against float32 and float64 PyTorch, and argmax flips. Random-weight model references take integer-generated weights, as fixtures already do, with BatchNorm running statistics calibrated from fixed integer-generated inputs by a procedure fixed in advance and run once. Native initialization is not bit-identical across platforms and can let activations grow far beyond a trained model's. Records made with native initialization stay as history. --- AGENTS.md | 11 ++++++++--- CONTRIBUTING.md | 37 +++++++++++++++++++++++++++++++------ 2 files changed, 39 insertions(+), 9 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 90387b2..92af896 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -103,7 +103,9 @@ output. outside the validated records ([guide](docs/converter.md)). Any other new model goes in `tensor_ee/models//`, composed from blocks by hand like `swin_t224`. It imports only library layers, uses the reference model's `state_dict` names, takes NumPy arrays or - `ServerArray`s as weights, and is not loaded by `import tensor_ee`. `compile_model` stays the + `ServerArray`s as weights, and is not loaded by `import tensor_ee`. A random-weight model's + numerical reference uses integer-generated weights with calibrated BatchNorm (CONTRIBUTING, + "Model references follow the same rule"). `compile_model` stays the CNN path (`torch.fx` onto the R blocks): attention models go through the converter, which is extended with recognizers, never by routing them through `compile_model`. - **A new recognizer** follows CONTRIBUTING's "Add a recognizer": @@ -129,8 +131,11 @@ output. - Keep four claims apart: graph identity, offline mirror agreement, live agreement and model accuracy. One does not establish another. -- The 1e-4 absolute target is unchanged. Report the largest absolute error, never only a - relative one. Never relax a check or rewrite a recorded result. +- The 1e-4 absolute target against float32 PyTorch is unchanged and applies to block + fixtures and to a model's final outputs. A model's intermediate outputs are checked + against float64 PyTorch instead: the mirror no farther than float32 PyTorch + (CONTRIBUTING, "Numerical comparisons"). Report the largest absolute error, never + only a relative one. Never relax a check or rewrite a recorded result. ## Never diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 94d7f65..2639915 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -106,6 +106,18 @@ Use `dyadic()` in [`tools/fixtures/core.py`](tools/fixtures/core.py) for arrays model. [`tests/test_graph_identity.py`](tests/test_graph_identity.py) refuses torch's normal generators while the G1 fixtures build. +**Model references follow the same rule.** A random-weight model's numerical reference, +the PyTorch model its mirror is compared with, takes its weights from integer arithmetic +(`_fixed_weights`), never from a library's native random initialization. Native +initialization is not bit-identical across platforms, and it can let activations grow +far beyond a trained model's. Its BatchNorm running statistics are then calibrated from +a fixed set of integer-generated inputs, separate from the evaluation input: a fixed +number of batches, `model.train()` for that pass only under `torch.no_grad()` with +`momentum=None`, then `model.eval()`. Only the running statistics change. Fix the +procedure before looking at any comparison and run it once; never adjust weights, +inputs or calibration until a check passes. A trained model is compared with its +trained weights. Records made with native initialization stay as history. + ## Add a recognizer The converter (`tensor_ee.converter.convert`, [guide](docs/converter.md)) turns a PyTorch @@ -249,12 +261,25 @@ research inputs are not uploaded to CI. ## Numerical comparisons -- Keep three claims apart: graph identity, numerical agreement on recorded inputs, - and model accuracy. One does not establish another. Graph construction alone is - not numerical equivalence. Describe live evidence separately from offline checks. -- Keep the 1e-4 absolute logit target and every recorded exception. Never relax a - check silently or rewrite a historical result; the known U-Net impulse failure - stays documented. +- Keep four claims apart: graph identity, offline mirror agreement, live agreement and + model accuracy. One does not establish another. Graph construction alone is not + numerical equivalence. Describe live evidence separately from offline checks. +- **The 1e-4 absolute target, against float32 PyTorch, applies to every block + fixture's outputs and to a model's final outputs** (its logits, and its + probabilities where it produces them). Keep it and every recorded exception. Never + relax a check silently or rewrite a historical result; the known U-Net impulse + failure stays documented. +- **A model's intermediate outputs** (encoder features, decoder stages, anything + before the final output) are recorded but not held to 1e-4. Each is checked against + a float64 run of the same PyTorch model, with the same float32 weights and input + converted by `model.double()`: the mirror's largest absolute error against float64 + must be no larger than float32 PyTorch's own. Large intermediate values put 1e-4 + out of reach of any float32 implementation; this check shows the library is at + least as accurate as PyTorch there. +- **Record the scale with every comparison**, for each output: its largest absolute + value, the float32 step at that value (`np.spacing`), the largest absolute error + against float32 PyTorch and, for a model, against float64 PyTorch, and argmax flips + where the output has classes. - Retain the actual prepared input pixels and their IEEE float32 identity, the ordered bands, the affine grid and coordinate orientation, and the raw mask channels.