Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ shown alongside it uses a separately deployed model endpoint.
| Fixed Swin-T/UPerNet, 224 × 224 | [Three-stage workflow](https://github.com/noelEOS/tensor-ee/blob/main/docs/swin.md) | Trained EE weights are public; this workflow also needs a local checkpoint, not yet public |
| Prithvi Sen1Floods11, 224 × 224 crops | [Hand-composed crop API](https://github.com/noelEOS/tensor-ee/blob/main/docs/api.md#prithvi-sen1floods11) | The released Sen1Floods11 checkpoint runs live in staged exports, one crop or a region of whole crops, and matches PyTorch within 1.5e-05, on synthetic crops and on a real Sentinel-2 crop (water IoU 0.911 against its hand label, one crop). Accuracy is not established. |
| SMP SegFormer-B0 or UPerNet with timm's Swin encoder | [Attention converter](https://github.com/noelEOS/tensor-ee/blob/main/docs/converter.md) | Conversion examples use random weights; the report identifies settings outside the validated records |
| SMP DPT with a timm ViT-B/16 encoder, 224 × 224 | [Attention converter](https://github.com/noelEOS/tensor-ee/blob/main/docs/converter.md#smp-050-dpt-at-224--224) | Converts to graphs byte-identical to the hand-composed targets, full width and reduced. Full width is offline only and misses the 1e-4 absolute target; live agreement covers only the reduced encoder twin (rank-at-most-4 projection weights), the structured reduced DPT twin (7 requests, sampled head/logit readouts), and individual blocks' recorded settings. Random weights only; no accuracy claim. [Evidence and limits](https://github.com/noelEOS/tensor-ee/blob/main/docs/converter.md#smp-dpt-vit-b16-evidence-and-scope). |

You can also compose a model from the [block catalogue](https://github.com/noelEOS/tensor-ee/blob/main/docs/building-blocks.md).
It has 88 blocks: 23 in `tensor_ee/blocks/raster` and 65 in `tensor_ee/blocks/window`.
Expand Down
4 changes: 2 additions & 2 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@ documentation; the records after them hold the evidence the guides cite.
| [Getting started](getting-started.md) | Install, the offline example, both demos and a first conversion, with expected output |
| [Code Editor demo](../CODE_EDITOR.md) | The trained U-Net in the Earth Engine Code Editor: no install, no checkpoint |
| [Browser demo](../DEMO.md) | A local web app that runs either trained model in Earth Engine, with an offline mode |
| [ViT-B/16 encoder target](vit-b16-encoder224.md) | Hand-composed DPT encoder target, offline evidence and reduced live plan |
| [ViT-B/16 encoder target](vit-b16-encoder224.md) | Hand-composed DPT encoder target, offline evidence and reduced live agreement on rank-at-most-4 projection weights ([record](evidence/live-fixtures/20261103T022505Z-vit_encoder_reduced.json)) |
| [Models and their evidence](model.md) | Models built from the blocks, what each was checked against, and the trained checkpoints |
| [U-Net single-chip walkthrough](water-chip.md) | The trained U-Net from Python: upload its weights, export one chip, check it against PyTorch |
| [Release validation](validation.md) | The walkthrough's verified run, which the README figure was rendered from |
| [Swin-T/UPerNet workflow](swin.md) | The fixed trained Swin-T/UPerNet as three saved-asset stages |
| [API guide](api.md) | The CNN API: entry points, supported forms, numerical and validity rules |
| [SMP CNNs](smp-models.md) | Compile standard segmentation-models-pytorch CNNs with `adapt_smp_model`: contract, evidence, live check |
| [Your own CNN over a region](own-model.md) | A trained SMP Unet/resnet34 from checkpoint to a tiled, batch-exported region, with its halo cost |
| [Converter guide](converter.md) | Convert, report, plan, build and run SegFormer-B0 and the Swin-T/UPerNet |
| [Converter guide](converter.md) | Convert, report, plan and build SegFormer-B0, Swin-T/UPerNet and SMP DPT with timm ViT-B/16; runner evidence and limits |
| [Block catalogue](building-blocks.md) | Every block's layout, locality, precision policy, status and evidence; the layers and the public API |
| [Live fixtures](live-fixtures.md) | The harness that compares a block's Earth Engine result and its NumPy mirror with PyTorch |
| [Library contract](library-contract.md) | The rules the library is built to: layers, tensor types, policies, metadata, gates |
Expand Down
24 changes: 24 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,30 @@ The historical
research runners are archived at the tag `research-archive-2026-10-12`
([history](history/README.md)).

## SMP DPT with a timm ViT-B/16 encoder

tensor-ee converts SMP DPT models with a timm ViT-B/16 encoder at 224 × 224.
Use `tensor_ee.converter.convert.convert`, with SMP 0.5.0, timm 1.0.30,
evaluation mode, unfused attention, one class token and cat readout; the
[converter example and refusals](converter.md#smp-050-dpt-at-224--224)
show the supported form. The result is a weight-free block graph and conversion
report; `build` binds weights and produces dense logits on the abstract input grid.
The caller prepares the crop and handles georeferenced relabeling.

The [evidence summary](converter.md#smp-dpt-vit-b16-evidence-and-scope) keeps graph
identity, offline mirror agreement, live agreement and model accuracy separate.
Random-weight full-width and reduced conversions are byte-identical to their
hand-composed targets. Full width is offline only: the DPT offline NumPy mirror
misses the 1e-4 absolute target against both float32 and float64 PyTorch, with
0 argmax flips. Float32 PyTorch itself differs from float64 PyTorch by more
([numbers and dtype record](vit-b16-dpt224.md#numerical-scope)). Live agreement
covers only the reduced encoder twin with rank-at-most-4 projection weights,
the structured reduced DPT twin (`vit_dpt_reduced`, 7 requests, sampled head/logit
readouts), and individual blocks' recorded settings. The full-width live run
needs the staged runner and remains a follow-up. Random weights establish no
model accuracy. ViT-S, ViT-L, DeiT-III, DINOv2 and patch 8 are later work;
fused attention, register tokens and other readouts are outside the claim.

## Class rows beside spatial windows

`tensor_ee.blocks.window.merge.concat_rows(ops, parts, *, rows, channels)`
Expand Down
53 changes: 50 additions & 3 deletions docs/converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ PyTorch model ──convert──▶ block graph ──plan──▶ stages ─
This guide begins with offline conversions of the two original models: SegFormer-B0,
`Segformer("mit_b0")` of segmentation-models-pytorch (SMP) 0.5.0, and SMP's `UPerNet` with
timm's Swin-T encoder, both at 224 × 224 with two input bands and two classes.
The timm ViT-B/16 encoder and SMP DPT recognizers are described in section 6.
tensor-ee converts SMP DPT models with a timm ViT-B/16 encoder at 224 × 224.
The recognizers, refused settings and evidence limits are described in section 6.

Two terms recur below. The **hand-built graphs** are the 19 SegFormer-B0 and Swin-T/UPerNet
graphs composed by hand from the blocks and run live before the converter existed; the
Expand All @@ -42,14 +43,16 @@ Four kinds of evidence apply, and they are kept apart:
conversion that succeeds says the library can express the model, not that it was validated.
- **Offline mirror agreement**: NumPy mirrors against PyTorch, per block, in the
[live fixture harness](live-fixtures.md). The converter adds none.
- **Live agreement.** One run so far ([record of the run](evidence/x06-live.json)): the converted SegFormer-B0, under the planner's default four
- **Live agreement.** For the original converted models, one runner run so far ([record of the run](evidence/x06-live.json)): the converted SegFormer-B0, under the planner's default four
stages, run by the library runner on one chip with random weights, agreed with PyTorch within the
1e-4 absolute target at every saved boundary (largest absolute error 3.58e-6; logits 2.98e-6, 0
argmax disagreements). C1–C3 and the logits are bit-identical to the hand-staged run, and the
logits to the single-graph run, which exported nothing else. The four stages took 92.2 batch
EECU-seconds.
- **Model accuracy**: not measured. The weights here are random.

For ViT-B/16 and SMP DPT, see the separate [four-part evidence summary](#smp-dpt-vit-b16-evidence-and-scope) below; the SegFormer runner result does not validate them.

## Before you start

Install the `swin` extra (SMP 0.5.0 and timm 1.0.30; [Getting started, Install](getting-started.md#1-install)).
Expand Down Expand Up @@ -600,11 +603,55 @@ no numerical execution, live run or model-accuracy claim. Individual blocks outs
their validated settings remain outside in the report, including W14 convolutions;
the hand model's reduced composition evidence has its own narrower scope.

### SMP DPT ViT-B/16: evidence and scope

**tensor-ee converts SMP DPT models with a timm ViT-B/16 encoder at 224 × 224**.
This claim uses SMP 0.5.0 and timm 1.0.30, evaluation mode, unfused attention,
one class token and cat readout, as in the example above. Four claims stay separate:

- **Graph identity.** The [timm encoder converter gate](../tests/test_timm_vit_converter.py)
and [SMP DPT converter gate](../tests/test_smp_dpt_converter.py) convert random-weight
models to graphs byte-identical to the hand-composed targets, at full width and
reduced width. Both builds bind the same symbolic table reads; reduced inline
gates also compare random-weight literals. These gates establish graph identity,
not numerical execution.
- **Offline mirror agreement.** The [encoder evidence](vit-b16-encoder224.md)
and [DPT numerical evidence](vit-b16-dpt224.md#numerical-scope) compare NumPy
mirrors with PyTorch. The [reduced DPT fixture record](evidence/dpt-reduced-offline.json)
has largest absolute error 1.86e-9. Full width is **offline only** and does
**not meet the unchanged 1e-4 absolute target**: against float32 PyTorch,
fusion3 differs by 6.03e-4 and logits by 3.51e-4; against float64 PyTorch,
fusion3 differs by 1.18e-4 and logits by 1.50e-4. Activations reach about
190, where one float32 step is about 1.5e-5. Float32 PyTorch itself differs
from float64 PyTorch by more (fusion3 6.13e-4, logits 3.70e-4); there are
**0 argmax flips**. The [dtype record](evidence/dpt-dtype-offline.json)
retains these failures; neither rounding scale nor unchanged decisions relaxes
the absolute target.
- **Live agreement.** It exists only for the
[reduced ViT encoder twin](evidence/live-fixtures/20261103T022505Z-vit_encoder_reduced.json)
with **rank-at-most-4 projection weights**, the
[reduced DPT twin](evidence/live-fixtures/20261103T041356Z-vit_dpt_reduced.json)
(`vit_dpt_reduced`, **7 requests**, structured rank-at-most-4 projection weights
and sampled head/logit readouts), plus the individual blocks' live records
linked in the [catalogue](building-blocks.md). Both twins matched their mirrors
bit for bit; the largest absolute EE-versus-PyTorch errors were 5.72e-6 for
the encoder and 1.86e-9 for DPT. This does not validate unrestricted weights
or promote settings outside individual block records. The full-width live run
has not been done; it needs the staged runner and remains a follow-up
([proposal and limits](vit-b16-dpt224.md#offline-request-sizing-and-deferred-full-width-live-run)).
- **Model accuracy.** None is claimed: random weights only, no accuracy evaluation.

The claim does not cover ViT-S, ViT-L, DeiT-III, DINOv2 or patch 8; these are
later work. Fused attention, register tokens and other readouts are also outside
this claim and are refused by these recognizers. Parameterized settings that
convert outside a block's validated record gain no validation from acceptance.

## Limits

- Families: SMP SegFormer (keys and values one 7 × 7 window after the spatial reduction, so
`mit_b0` at 224 × 224) and SMP UPerNet with timm's Swin. Only the settings in each block's record
are validated. The root timm ViT encoder returns one atomic pair; SMP DPT consumes four intermediate pairs and returns dense logits.
are validated. SMP DPT with a timm ViT-B/16 encoder at 224 × 224 also converts,
under the evidence limits above. The root timm ViT encoder returns one atomic pair; SMP DPT consumes four intermediate pairs and returns dense logits.
- One input and one output; batch execution only. The planner cuts by rules and metadata: interactive
memory and recomputation cost are not modeled, and its `extent` default was read off one hand plan
(calibrating costs is deferred).
Expand Down
40 changes: 30 additions & 10 deletions docs/model.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,18 +182,38 @@ or an accuracy benchmark.

The [hand-composed ViT encoder target](vit-b16-encoder224.md) exposes every raw
block's spatial/class pair and the final norm, with SMP DPT encoder parameter
names. Reduced and full-width random-weight timm comparisons pass offline; the
reduced interactive live attempt failed before encoder readback (client recursion).
Only the input echo returned exactly. No encoder live agreement, converter
equivalence, DPT decoder, or model accuracy is claimed.
names. Reduced and full-width random-weight timm comparisons pass offline.
The [first reduced live attempt](evidence/live-fixtures/20261102T191233Z-vit_encoder_reduced.json)
failed before encoder readback because of client recursion; only its input echo
returned exactly. The [passing rerun](evidence/live-fixtures/20261103T022505Z-vit_encoder_reduced.json)
matched the mirror bit for bit at all encoder outputs, with largest absolute
EE-versus-PyTorch error 5.72e-6. This live agreement covers only the reduced
twin's recorded inputs and rank-at-most-4 projection weights. Full width remains
offline only; no model accuracy is claimed.

The [timm ViT converter gate](../tests/test_timm_vit_converter.py) converts
random-weight full-width and reduced encoders to graphs byte-identical to this
hand-composed target. That is graph identity, separate from offline mirror
agreement and the reduced twin's live evidence. The [converter guide](converter.md#smp-dpt-vit-b16-evidence-and-scope)
states the claim and its limits.

## ViT-B/16 + DPT target, 224 × 224

The [hand-composed DPT target](vit-b16-dpt224.md) reuses the merged ViT encoder
and existing library blocks, with SMP 0.5.0 defaults, six bands and two classes.
Its notes state the layout chain, offline comparisons, reduced composition
fixture and measured export proposal. Full-width live agreement and model
accuracy are not established. No DPT recognizer is added by this target.
The reduced twin carries the 1e-4 claim. The native full-width absolute check is
a documented strict expected failure, with a separate float64 ordering guard;
full-width live validation is deferred to a follow-up and is not met here.
The [SMP DPT recognizer](converter.md#smp-050-dpt-at-224--224) now converts this
model; its [converter gate](../tests/test_smp_dpt_converter.py) compares
random-weight full-width and reduced conversions byte for byte with the target.
This establishes graph identity, not numerical execution.

The [reduced DPT live record](evidence/live-fixtures/20261103T041356Z-vit_dpt_reduced.json)
passed 7 requests, matching the mirror bit for bit at all ten readouts, with
largest absolute EE-versus-PyTorch error 1.86e-9. This covers only the structured
rank-at-most-4 reduced twin and its sampled head/logit readouts, not unrestricted
weights. The reduced twin meets the unchanged 1e-4 absolute target. The offline
NumPy mirror at full width misses that target against both float32 and float64
PyTorch, with 0 argmax flips; float32 PyTorch itself differs from float64 by more
([numerics and dtype record](vit-b16-dpt224.md#numerical-scope)). Its absolute
check remains a strict expected failure, with a separate float64 ordering guard.
Full-width live agreement has not been established: the staged runner is a
follow-up. Random weights establish no model accuracy.
9 changes: 6 additions & 3 deletions docs/vit-b16-dpt224.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# ViT-B/16 + DPT target at 224

`tensor_ee.models.vit_b16_dpt224` is the hand-composed target for a later SMP
DPT recognizer. It reuses `vit_b16_encoder224` and existing library blocks.
It adds no block or converter support and is not loaded by `import tensor_ee`.
`tensor_ee.models.vit_b16_dpt224` is the hand-composed target for the
[SMP DPT recognizer](converter.md#smp-050-dpt-at-224--224). It reuses
`vit_b16_encoder224` and existing library blocks and is not loaded by
`import tensor_ee`. The [converter gate](../tests/test_smp_dpt_converter.py)
checks byte-identical graphs at full and reduced width: graph identity only,
separate from the offline and reduced live numerical evidence below.
Weights are native NumPy float32 arrays or native/flattened `ServerArray`s,
using SMP 0.5.0's `state_dict` names. Constructors build graphs only.

Expand Down
15 changes: 9 additions & 6 deletions docs/vit-b16-encoder224.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# ViT-B/16 encoder target at 224

`tensor_ee.models.vit_b16_encoder224` is a hand-composed target for a later
converter recognizer. It is not imported by `import tensor_ee`. It uses existing
blocks, with no converter or new block. It has offline composition and
numerical evidence, plus live agreement for the reduced twin on the recorded
rank-at-most-4 projection weights. Full width has offline evidence only. No
converter equivalence, trained-model accuracy, or DPT decoder is claimed.
`tensor_ee.models.vit_b16_encoder224` is the hand-composed target for the
[timm ViT converter recognizer](converter.md#timm-vit-b16-encoder-at-224--224).
It is not imported by `import tensor_ee` and uses existing library blocks.
The [converter gate](../tests/test_timm_vit_converter.py) checks byte-identical
graphs at full and reduced width; this is graph identity only. Separately, the
target has offline composition and numerical evidence, plus live agreement for
the reduced twin on the recorded rank-at-most-4 projection weights. Full width
has offline evidence only. No trained-model accuracy is claimed. The DPT decoder
has its [own target and evidence](vit-b16-dpt224.md).

## Contract and usage

Expand Down
Loading
Loading