diff --git a/README.md b/README.md index 52347e5..41071da 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,7 @@ shown alongside it uses a separately deployed model endpoint. | Fixed Swin-T/UPerNet, 224 × 224 | [Three-stage workflow](https://github.com/noelEOS/tensor-ee/blob/main/docs/swin.md) | Trained EE weights are public; this workflow also needs a local checkpoint, not yet public | | Prithvi Sen1Floods11, 224 × 224 crops | [Hand-composed crop API](https://github.com/noelEOS/tensor-ee/blob/main/docs/api.md#prithvi-sen1floods11) | The released Sen1Floods11 checkpoint runs live in staged exports, one crop or a region of whole crops, and matches PyTorch within 1.5e-05, on synthetic crops and on a real Sentinel-2 crop (water IoU 0.911 against its hand label, one crop). Accuracy is not established. | | SMP SegFormer-B0 or UPerNet with timm's Swin encoder | [Attention converter](https://github.com/noelEOS/tensor-ee/blob/main/docs/converter.md) | Conversion examples use random weights; the report identifies settings outside the validated records | +| SMP DPT with a timm ViT-B/16 encoder, 224 × 224 | [Attention converter](https://github.com/noelEOS/tensor-ee/blob/main/docs/converter.md#smp-050-dpt-at-224--224) | Converts to graphs byte-identical to the hand-composed targets, full width and reduced. Full width is offline only and misses the 1e-4 absolute target; live agreement covers only the reduced encoder twin (rank-at-most-4 projection weights), the structured reduced DPT twin (7 requests, sampled head/logit readouts), and individual blocks' recorded settings. Random weights only; no accuracy claim. [Evidence and limits](https://github.com/noelEOS/tensor-ee/blob/main/docs/converter.md#smp-dpt-vit-b16-evidence-and-scope). | You can also compose a model from the [block catalogue](https://github.com/noelEOS/tensor-ee/blob/main/docs/building-blocks.md). It has 88 blocks: 23 in `tensor_ee/blocks/raster` and 65 in `tensor_ee/blocks/window`. diff --git a/docs/README.md b/docs/README.md index 3fad37d..d7d0d83 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,7 +10,7 @@ documentation; the records after them hold the evidence the guides cite. | [Getting started](getting-started.md) | Install, the offline example, both demos and a first conversion, with expected output | | [Code Editor demo](../CODE_EDITOR.md) | The trained U-Net in the Earth Engine Code Editor: no install, no checkpoint | | [Browser demo](../DEMO.md) | A local web app that runs either trained model in Earth Engine, with an offline mode | -| [ViT-B/16 encoder target](vit-b16-encoder224.md) | Hand-composed DPT encoder target, offline evidence and reduced live plan | +| [ViT-B/16 encoder target](vit-b16-encoder224.md) | Hand-composed DPT encoder target, offline evidence and reduced live agreement on rank-at-most-4 projection weights ([record](evidence/live-fixtures/20261103T022505Z-vit_encoder_reduced.json)) | | [Models and their evidence](model.md) | Models built from the blocks, what each was checked against, and the trained checkpoints | | [U-Net single-chip walkthrough](water-chip.md) | The trained U-Net from Python: upload its weights, export one chip, check it against PyTorch | | [Release validation](validation.md) | The walkthrough's verified run, which the README figure was rendered from | @@ -18,7 +18,7 @@ documentation; the records after them hold the evidence the guides cite. | [API guide](api.md) | The CNN API: entry points, supported forms, numerical and validity rules | | [SMP CNNs](smp-models.md) | Compile standard segmentation-models-pytorch CNNs with `adapt_smp_model`: contract, evidence, live check | | [Your own CNN over a region](own-model.md) | A trained SMP Unet/resnet34 from checkpoint to a tiled, batch-exported region, with its halo cost | -| [Converter guide](converter.md) | Convert, report, plan, build and run SegFormer-B0 and the Swin-T/UPerNet | +| [Converter guide](converter.md) | Convert, report, plan and build SegFormer-B0, Swin-T/UPerNet and SMP DPT with timm ViT-B/16; runner evidence and limits | | [Block catalogue](building-blocks.md) | Every block's layout, locality, precision policy, status and evidence; the layers and the public API | | [Live fixtures](live-fixtures.md) | The harness that compares a block's Earth Engine result and its NumPy mirror with PyTorch | | [Library contract](library-contract.md) | The rules the library is built to: layers, tensor types, policies, metadata, gates | diff --git a/docs/api.md b/docs/api.md index 84ac748..7ff14b2 100644 --- a/docs/api.md +++ b/docs/api.md @@ -248,6 +248,30 @@ The historical research runners are archived at the tag `research-archive-2026-10-12` ([history](history/README.md)). +## SMP DPT with a timm ViT-B/16 encoder + +tensor-ee converts SMP DPT models with a timm ViT-B/16 encoder at 224 × 224. +Use `tensor_ee.converter.convert.convert`, with SMP 0.5.0, timm 1.0.30, +evaluation mode, unfused attention, one class token and cat readout; the +[converter example and refusals](converter.md#smp-050-dpt-at-224--224) +show the supported form. The result is a weight-free block graph and conversion +report; `build` binds weights and produces dense logits on the abstract input grid. +The caller prepares the crop and handles georeferenced relabeling. + +The [evidence summary](converter.md#smp-dpt-vit-b16-evidence-and-scope) keeps graph +identity, offline mirror agreement, live agreement and model accuracy separate. +Random-weight full-width and reduced conversions are byte-identical to their +hand-composed targets. Full width is offline only: the DPT offline NumPy mirror +misses the 1e-4 absolute target against both float32 and float64 PyTorch, with +0 argmax flips. Float32 PyTorch itself differs from float64 PyTorch by more +([numbers and dtype record](vit-b16-dpt224.md#numerical-scope)). Live agreement +covers only the reduced encoder twin with rank-at-most-4 projection weights, +the structured reduced DPT twin (`vit_dpt_reduced`, 7 requests, sampled head/logit +readouts), and individual blocks' recorded settings. The full-width live run +needs the staged runner and remains a follow-up. Random weights establish no +model accuracy. ViT-S, ViT-L, DeiT-III, DINOv2 and patch 8 are later work; +fused attention, register tokens and other readouts are outside the claim. + ## Class rows beside spatial windows `tensor_ee.blocks.window.merge.concat_rows(ops, parts, *, rows, channels)` diff --git a/docs/converter.md b/docs/converter.md index cd46cc7..b3e44bf 100644 --- a/docs/converter.md +++ b/docs/converter.md @@ -21,7 +21,8 @@ PyTorch model ──convert──▶ block graph ──plan──▶ stages ─ This guide begins with offline conversions of the two original models: SegFormer-B0, `Segformer("mit_b0")` of segmentation-models-pytorch (SMP) 0.5.0, and SMP's `UPerNet` with timm's Swin-T encoder, both at 224 × 224 with two input bands and two classes. -The timm ViT-B/16 encoder and SMP DPT recognizers are described in section 6. +tensor-ee converts SMP DPT models with a timm ViT-B/16 encoder at 224 × 224. +The recognizers, refused settings and evidence limits are described in section 6. Two terms recur below. The **hand-built graphs** are the 19 SegFormer-B0 and Swin-T/UPerNet graphs composed by hand from the blocks and run live before the converter existed; the @@ -42,7 +43,7 @@ Four kinds of evidence apply, and they are kept apart: conversion that succeeds says the library can express the model, not that it was validated. - **Offline mirror agreement**: NumPy mirrors against PyTorch, per block, in the [live fixture harness](live-fixtures.md). The converter adds none. -- **Live agreement.** One run so far ([record of the run](evidence/x06-live.json)): the converted SegFormer-B0, under the planner's default four +- **Live agreement.** For the original converted models, one runner run so far ([record of the run](evidence/x06-live.json)): the converted SegFormer-B0, under the planner's default four stages, run by the library runner on one chip with random weights, agreed with PyTorch within the 1e-4 absolute target at every saved boundary (largest absolute error 3.58e-6; logits 2.98e-6, 0 argmax disagreements). C1–C3 and the logits are bit-identical to the hand-staged run, and the @@ -50,6 +51,8 @@ Four kinds of evidence apply, and they are kept apart: EECU-seconds. - **Model accuracy**: not measured. The weights here are random. +For ViT-B/16 and SMP DPT, see the separate [four-part evidence summary](#smp-dpt-vit-b16-evidence-and-scope) below; the SegFormer runner result does not validate them. + ## Before you start Install the `swin` extra (SMP 0.5.0 and timm 1.0.30; [Getting started, Install](getting-started.md#1-install)). @@ -600,11 +603,55 @@ no numerical execution, live run or model-accuracy claim. Individual blocks outs their validated settings remain outside in the report, including W14 convolutions; the hand model's reduced composition evidence has its own narrower scope. +### SMP DPT ViT-B/16: evidence and scope + +**tensor-ee converts SMP DPT models with a timm ViT-B/16 encoder at 224 × 224**. +This claim uses SMP 0.5.0 and timm 1.0.30, evaluation mode, unfused attention, +one class token and cat readout, as in the example above. Four claims stay separate: + +- **Graph identity.** The [timm encoder converter gate](../tests/test_timm_vit_converter.py) + and [SMP DPT converter gate](../tests/test_smp_dpt_converter.py) convert random-weight + models to graphs byte-identical to the hand-composed targets, at full width and + reduced width. Both builds bind the same symbolic table reads; reduced inline + gates also compare random-weight literals. These gates establish graph identity, + not numerical execution. +- **Offline mirror agreement.** The [encoder evidence](vit-b16-encoder224.md) + and [DPT numerical evidence](vit-b16-dpt224.md#numerical-scope) compare NumPy + mirrors with PyTorch. The [reduced DPT fixture record](evidence/dpt-reduced-offline.json) + has largest absolute error 1.86e-9. Full width is **offline only** and does + **not meet the unchanged 1e-4 absolute target**: against float32 PyTorch, + fusion3 differs by 6.03e-4 and logits by 3.51e-4; against float64 PyTorch, + fusion3 differs by 1.18e-4 and logits by 1.50e-4. Activations reach about + 190, where one float32 step is about 1.5e-5. Float32 PyTorch itself differs + from float64 PyTorch by more (fusion3 6.13e-4, logits 3.70e-4); there are + **0 argmax flips**. The [dtype record](evidence/dpt-dtype-offline.json) + retains these failures; neither rounding scale nor unchanged decisions relaxes + the absolute target. +- **Live agreement.** It exists only for the + [reduced ViT encoder twin](evidence/live-fixtures/20261103T022505Z-vit_encoder_reduced.json) + with **rank-at-most-4 projection weights**, the + [reduced DPT twin](evidence/live-fixtures/20261103T041356Z-vit_dpt_reduced.json) + (`vit_dpt_reduced`, **7 requests**, structured rank-at-most-4 projection weights + and sampled head/logit readouts), plus the individual blocks' live records + linked in the [catalogue](building-blocks.md). Both twins matched their mirrors + bit for bit; the largest absolute EE-versus-PyTorch errors were 5.72e-6 for + the encoder and 1.86e-9 for DPT. This does not validate unrestricted weights + or promote settings outside individual block records. The full-width live run + has not been done; it needs the staged runner and remains a follow-up + ([proposal and limits](vit-b16-dpt224.md#offline-request-sizing-and-deferred-full-width-live-run)). +- **Model accuracy.** None is claimed: random weights only, no accuracy evaluation. + +The claim does not cover ViT-S, ViT-L, DeiT-III, DINOv2 or patch 8; these are +later work. Fused attention, register tokens and other readouts are also outside +this claim and are refused by these recognizers. Parameterized settings that +convert outside a block's validated record gain no validation from acceptance. + ## Limits - Families: SMP SegFormer (keys and values one 7 × 7 window after the spatial reduction, so `mit_b0` at 224 × 224) and SMP UPerNet with timm's Swin. Only the settings in each block's record - are validated. The root timm ViT encoder returns one atomic pair; SMP DPT consumes four intermediate pairs and returns dense logits. + are validated. SMP DPT with a timm ViT-B/16 encoder at 224 × 224 also converts, + under the evidence limits above. The root timm ViT encoder returns one atomic pair; SMP DPT consumes four intermediate pairs and returns dense logits. - One input and one output; batch execution only. The planner cuts by rules and metadata: interactive memory and recomputation cost are not modeled, and its `extent` default was read off one hand plan (calibrating costs is deferred). diff --git a/docs/model.md b/docs/model.md index 5e70fe3..6e1f969 100644 --- a/docs/model.md +++ b/docs/model.md @@ -182,18 +182,38 @@ or an accuracy benchmark. The [hand-composed ViT encoder target](vit-b16-encoder224.md) exposes every raw block's spatial/class pair and the final norm, with SMP DPT encoder parameter -names. Reduced and full-width random-weight timm comparisons pass offline; the -reduced interactive live attempt failed before encoder readback (client recursion). -Only the input echo returned exactly. No encoder live agreement, converter -equivalence, DPT decoder, or model accuracy is claimed. +names. Reduced and full-width random-weight timm comparisons pass offline. +The [first reduced live attempt](evidence/live-fixtures/20261102T191233Z-vit_encoder_reduced.json) +failed before encoder readback because of client recursion; only its input echo +returned exactly. The [passing rerun](evidence/live-fixtures/20261103T022505Z-vit_encoder_reduced.json) +matched the mirror bit for bit at all encoder outputs, with largest absolute +EE-versus-PyTorch error 5.72e-6. This live agreement covers only the reduced +twin's recorded inputs and rank-at-most-4 projection weights. Full width remains +offline only; no model accuracy is claimed. + +The [timm ViT converter gate](../tests/test_timm_vit_converter.py) converts +random-weight full-width and reduced encoders to graphs byte-identical to this +hand-composed target. That is graph identity, separate from offline mirror +agreement and the reduced twin's live evidence. The [converter guide](converter.md#smp-dpt-vit-b16-evidence-and-scope) +states the claim and its limits. ## ViT-B/16 + DPT target, 224 × 224 The [hand-composed DPT target](vit-b16-dpt224.md) reuses the merged ViT encoder and existing library blocks, with SMP 0.5.0 defaults, six bands and two classes. -Its notes state the layout chain, offline comparisons, reduced composition -fixture and measured export proposal. Full-width live agreement and model -accuracy are not established. No DPT recognizer is added by this target. -The reduced twin carries the 1e-4 claim. The native full-width absolute check is -a documented strict expected failure, with a separate float64 ordering guard; -full-width live validation is deferred to a follow-up and is not met here. +The [SMP DPT recognizer](converter.md#smp-050-dpt-at-224--224) now converts this +model; its [converter gate](../tests/test_smp_dpt_converter.py) compares +random-weight full-width and reduced conversions byte for byte with the target. +This establishes graph identity, not numerical execution. + +The [reduced DPT live record](evidence/live-fixtures/20261103T041356Z-vit_dpt_reduced.json) +passed 7 requests, matching the mirror bit for bit at all ten readouts, with +largest absolute EE-versus-PyTorch error 1.86e-9. This covers only the structured +rank-at-most-4 reduced twin and its sampled head/logit readouts, not unrestricted +weights. The reduced twin meets the unchanged 1e-4 absolute target. The offline +NumPy mirror at full width misses that target against both float32 and float64 +PyTorch, with 0 argmax flips; float32 PyTorch itself differs from float64 by more +([numerics and dtype record](vit-b16-dpt224.md#numerical-scope)). Its absolute +check remains a strict expected failure, with a separate float64 ordering guard. +Full-width live agreement has not been established: the staged runner is a +follow-up. Random weights establish no model accuracy. diff --git a/docs/vit-b16-dpt224.md b/docs/vit-b16-dpt224.md index 3d60b47..1e8a94e 100644 --- a/docs/vit-b16-dpt224.md +++ b/docs/vit-b16-dpt224.md @@ -1,8 +1,11 @@ # ViT-B/16 + DPT target at 224 -`tensor_ee.models.vit_b16_dpt224` is the hand-composed target for a later SMP -DPT recognizer. It reuses `vit_b16_encoder224` and existing library blocks. -It adds no block or converter support and is not loaded by `import tensor_ee`. +`tensor_ee.models.vit_b16_dpt224` is the hand-composed target for the +[SMP DPT recognizer](converter.md#smp-050-dpt-at-224--224). It reuses +`vit_b16_encoder224` and existing library blocks and is not loaded by +`import tensor_ee`. The [converter gate](../tests/test_smp_dpt_converter.py) +checks byte-identical graphs at full and reduced width: graph identity only, +separate from the offline and reduced live numerical evidence below. Weights are native NumPy float32 arrays or native/flattened `ServerArray`s, using SMP 0.5.0's `state_dict` names. Constructors build graphs only. diff --git a/docs/vit-b16-encoder224.md b/docs/vit-b16-encoder224.md index 70c0e11..1124a92 100644 --- a/docs/vit-b16-encoder224.md +++ b/docs/vit-b16-encoder224.md @@ -1,11 +1,14 @@ # ViT-B/16 encoder target at 224 -`tensor_ee.models.vit_b16_encoder224` is a hand-composed target for a later -converter recognizer. It is not imported by `import tensor_ee`. It uses existing -blocks, with no converter or new block. It has offline composition and -numerical evidence, plus live agreement for the reduced twin on the recorded -rank-at-most-4 projection weights. Full width has offline evidence only. No -converter equivalence, trained-model accuracy, or DPT decoder is claimed. +`tensor_ee.models.vit_b16_encoder224` is the hand-composed target for the +[timm ViT converter recognizer](converter.md#timm-vit-b16-encoder-at-224--224). +It is not imported by `import tensor_ee` and uses existing library blocks. +The [converter gate](../tests/test_timm_vit_converter.py) checks byte-identical +graphs at full and reduced width; this is graph identity only. Separately, the +target has offline composition and numerical evidence, plus live agreement for +the reduced twin on the recorded rank-at-most-4 projection weights. Full width +has offline evidence only. No trained-model accuracy is claimed. The DPT decoder +has its [own target and evidence](vit-b16-dpt224.md). ## Contract and usage diff --git a/tests/test_dpt_registry.py b/tests/test_dpt_registry.py index 1c937eb..a5df7b4 100644 --- a/tests/test_dpt_registry.py +++ b/tests/test_dpt_registry.py @@ -69,23 +69,36 @@ def test_registry_holds_library_functions_and_resize_cost(): resample.window_resize_values(14, 14, 2, align_corners=True, window=14, out_window=7), 0) -def test_dpt_additions_preserve_main_registry_entries_by_identity_and_order(): - from tensor_ee.converter import operations - main = operations._MAIN_OPERATIONS - main_keys = ( - "relabel_to_windows", "overlap_embedding", "mit_block", "channel_norm", "project_pyramid", - "fuse_head", "relabel_from_windows", "patch_embed", "swin_block", "shifted_swin_block", - "patch_merge", "psp_forward", "fpn_layer_norm", "fpn_lateral", "fpn_merge", "fpn_conv_block", - "fusion_forward", "head_logits", "head_dense") +def test_dpt_additions_preserve_main_function_metadata_and_key_order(): + """Pin main a988d79's 19 function/metadata references and key order, not adapters.""" + from tensor_ee.blocks.window import patch, segformer, swin, upernet + from tensor_ee.layouts import boundaries + + # Written from a988d79's operations.py, independently of its import-time snapshot. + main = { + "relabel_to_windows": (boundaries.relabel_to_windows, None), + "overlap_embedding": (segformer.overlap_embedding, segformer.METADATA["overlap_embedding"]), + "mit_block": (segformer.mit_block, segformer.METADATA["mit_block"]), + "channel_norm": (segformer.channel_norm, segformer.METADATA["channel_norm"]), + "project_pyramid": (segformer.project_pyramid, segformer.METADATA["project_pyramid"]), + "fuse_head": (segformer.fuse_head, segformer.METADATA["fuse_head"]), + "relabel_from_windows": (boundaries.relabel_from_windows, None), + "patch_embed": (patch.patch_embed, patch.METADATA["patch_embed"]), + "swin_block": (swin.swin_block, swin.METADATA["swin_block"]), + "shifted_swin_block": (swin.shifted_swin_block, swin.METADATA["shifted_swin_block"]), + "patch_merge": (patch.patch_merge, patch.METADATA["patch_merge"]), + "psp_forward": (upernet.psp_forward, upernet.METADATA["psp_forward"]), + "fpn_layer_norm": (upernet.fpn_layer_norm, upernet.METADATA["fpn_layer_norm"]), + "fpn_lateral": (upernet.fpn_lateral, upernet.METADATA["fpn_lateral"]), + "fpn_merge": (upernet.fpn_merge, upernet.METADATA["fpn_merge"]), + "fpn_conv_block": (upernet.fpn_conv_block, upernet.METADATA["fpn_conv_block"]), + "fusion_forward": (upernet.fusion_forward, upernet.METADATA["fusion_forward"]), + "head_logits": (upernet.head_logits, upernet.METADATA["head_logits"]), + "head_dense": (upernet.head_dense, upernet.METADATA["head_dense"]), + } additions = ("linear", "conv3x3_window", "batch_norm", "concat", "gelu", "relu", "add", "window_depth_to_space", "window_resize") - assert tuple(name for name, _ in main) == main_keys - assert tuple(operations._DPT_OPERATIONS) == additions - assert not set(main_keys).intersection(operations._DPT_OPERATIONS) - assert tuple(OPERATIONS) == main_keys + additions - assert tuple(OPERATIONS.items())[:len(main)] == main - for name, original in main: - current = OPERATIONS[name] - assert current is original, name - for field in ("function", "metadata", "cost", "constants", "infer", "call", "roles"): - assert getattr(current, field) is getattr(original, field), (name, field) + assert tuple(OPERATIONS) == tuple(main) + additions + for name, (function, metadata) in main.items(): + assert OPERATIONS[name].function is function, name + assert OPERATIONS[name].metadata is metadata, name