diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ebe7056..d1b61e9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -81,7 +81,9 @@ tag `research-archive-2026-10-12` ([history](docs/history/README.md)). `NOT_LIVE` for "Fixed — offline only". Name settings by the parameters of the block, its cost functions or its constants builder (the record's `sources`), and add the block to [`tests/test_x02b_validated_settings.py`](tests/test_x02b_validated_settings.py)'s - allowance tables only with a reason; + allowance tables only with a reason. A new setting on an existing block enters that + block's record at its default value, the one value its evidence ran, until a live run + varies it; a call at any other value is then reported outside the record; - `per_setting`, where a radius or chip-edge dependence depends on a setting (`window_resize`, `patch_embed`). 4. **Add offline tests.** Build the block's graph against the offline signatures and diff --git a/docs/api.md b/docs/api.md index 0e33a06..11b0a87 100644 --- a/docs/api.md +++ b/docs/api.md @@ -54,7 +54,27 @@ full window (9), as PyTorch's zero padding does; `False` divides by the number o in-bounds pixels. Convolution uses odd same-padded kernels, stride 1/2, dilation 1 and zero padding -mode. Groups=1 retains the existing `conv2d` block. Groups greater than one route +mode. A convolution is an `nn.Conv2d` module or `F.conv2d` (`torch.conv2d`) whose weight +and bias are the model's own parameters or buffers, read only by that call. Zero padding +directly ahead of it, as `nn.ZeroPad2d`, `nn.ConstantPad2d` with value 0 or `F.pad` with +a static four-integer pad, mode `"constant"` and value 0 or None, adds to the +convolution's own padding; an `nn.Identity` directly ahead of `F.conv2d` is absorbed too. +On each axis the total padding before the input must be `kernel // 2 - phase` with +`0 <= phase < stride`, and PyTorch's output count at the input size must be +`ceil((n - phase) / stride)`. Phase 0 is the same-padded convolution and compiles to its +existing graph; a nonzero phase (for example `(0, 1, 0, 1)` ahead of a 3 × 3 stride-2 +convolution) runs the same block, sampling input pixel `stride * i + phase` +(the blocks' keyword-only `phase`, an integer or a `(phase_y, phase_x)` pair). A +nonzero phase is validated live on small fixtures at stride 2: an ordinary 3 × 3 +convolution at phase 1, and depthwise 3 × 3 at phases 1 and `(1, 0)` and 5 × 5 at +phase 1; other kernels and phases are not validated. Refused: padding computed from a +tensor's shape, negative pads, other `F.pad` lengths, nonzero values, reflection, +replication and circular padding, a padding node with any other use, a total padding +whose phase is outside `[0, stride)` or whose output count is not +`ceil((n - phase) / stride)` (an unpadded stride-1 convolution, for example), and +functional weights that are computed or shared. An unpadded 3 × 3 stride-2 convolution +on an odd-sized map is the phase-1 form and compiles; on an even-sized map it is +refused. Groups=1 retains the existing `conv2d` block. Groups greater than one route to `grouped_conv2d`, including depthwise convolutions and channel multipliers; groups must divide both channel counts. The grouped block is validated live for depthwise 3 x 3 at strides 1/2 with optional bias, depthwise 5 x 5 at stride 1, and diff --git a/docs/building-blocks.md b/docs/building-blocks.md index 6bc3742..532e282 100644 --- a/docs/building-blocks.md +++ b/docs/building-blocks.md @@ -132,8 +132,8 @@ their own. | [`eval_dropout`](../tensor_ee/blocks/raster/activations.py) | R → R | pointwise | `float32_operands` | no | Ready | [evidence/t02-operators.json](evidence/t02-operators.json); [evidence/t05-primary.json](evidence/t05-primary.json) | | [`common_score_mask`](../tensor_ee/blocks/raster/activations.py) | R → R | pointwise | `float32_operands` | no | Ready | [evidence/t03-outputs.json](evidence/t03-outputs.json) | | [`softmax`](../tensor_ee/blocks/raster/activations.py) | R → R | pointwise | `float32_operands` | no | Ready | [evidence/t03-outputs.json](evidence/t03-outputs.json); [evidence/t05-primary.json](evidence/t05-primary.json) | -| [`grouped_conv2d`](../tensor_ee/blocks/raster/conv.py) | R → R | k×k, radius 2 | `float32_operands` | no | Ready — depthwise 3×3 stride 1 or 2, 5×5 stride 1; 3×3 stride 1 with 2 channels per group or channel multiplier 2 | [evidence/raster-grouped-conv-offline.json](evidence/raster-grouped-conv-offline.json); [evidence/live-fixtures/20261029T125301Z-grouped_conv.json](evidence/live-fixtures/20261029T125301Z-grouped_conv.json) | -| [`conv2d`](../tensor_ee/blocks/raster/conv.py) | R → R | k×k, radius 3 | `float32_operands` | no | Ready | [cnn/results/parity.json](history/cnn/results/parity.json); [cnn/results/primitives_float32.json](history/cnn/results/primitives_float32.json); [cnn/results/resnet_primitives.json](history/cnn/results/resnet_primitives.json); [cnn/results/resnet18_full64_float32_policy.json](history/cnn/results/resnet18_full64_float32_policy.json); [cnn/results/segwater_js_parity_1x1.json](history/cnn/results/segwater_js_parity_1x1.json); [cnn/results/tiled_resnet18_2x2_parity.json](history/cnn/results/tiled_resnet18_2x2_parity.json) | +| [`grouped_conv2d`](../tensor_ee/blocks/raster/conv.py) | R → R | k×k, radius 3 | `float32_operands` | no | Ready — depthwise 3×3 stride 1 or 2, 5×5 stride 1 or 2; 3×3 stride 1 with 2 channels per group or channel multiplier 2 | [evidence/raster-grouped-conv-offline.json](evidence/raster-grouped-conv-offline.json); [evidence/live-fixtures/20261029T125301Z-grouped_conv.json](evidence/live-fixtures/20261029T125301Z-grouped_conv.json); [evidence/raster-padded-conv-offline.json](evidence/raster-padded-conv-offline.json); [evidence/live-fixtures/20261112T185127Z-padded_conv.json](evidence/live-fixtures/20261112T185127Z-padded_conv.json) | +| [`conv2d`](../tensor_ee/blocks/raster/conv.py) | R → R | k×k, radius 3 | `float32_operands` | no | Ready | [cnn/results/parity.json](history/cnn/results/parity.json); [cnn/results/primitives_float32.json](history/cnn/results/primitives_float32.json); [cnn/results/resnet_primitives.json](history/cnn/results/resnet_primitives.json); [cnn/results/resnet18_full64_float32_policy.json](history/cnn/results/resnet18_full64_float32_policy.json); [cnn/results/segwater_js_parity_1x1.json](history/cnn/results/segwater_js_parity_1x1.json); [cnn/results/tiled_resnet18_2x2_parity.json](history/cnn/results/tiled_resnet18_2x2_parity.json); [evidence/raster-padded-conv-offline.json](evidence/raster-padded-conv-offline.json); [evidence/live-fixtures/20261112T185127Z-padded_conv.json](evidence/live-fixtures/20261112T185127Z-padded_conv.json) | | [`concatenate`](../tensor_ee/blocks/raster/merge.py) | R → R | pointwise | `float32_operands` | no | Ready | [cnn/results/parity.json](history/cnn/results/parity.json); [cnn/results/resnet18_full64_float32_policy.json](history/cnn/results/resnet18_full64_float32_policy.json); [cnn/results/segwater_js_parity_1x1.json](history/cnn/results/segwater_js_parity_1x1.json) | | [`residual_add`](../tensor_ee/blocks/raster/merge.py) | R → R | pointwise | `float32_operands` | no | Ready | [cnn/results/resnet_primitives.json](history/cnn/results/resnet_primitives.json); [cnn/results/resnet18_full64_float32_policy.json](history/cnn/results/resnet18_full64_float32_policy.json); [cnn/results/segwater_js_parity_1x1.json](history/cnn/results/segwater_js_parity_1x1.json) | | [`multiply`](../tensor_ee/blocks/raster/merge.py) | R → R | pointwise | `float32_operands` | no | Ready — same grid on a 5 × 7 map; one-pixel gate on 5 × 7 and 12 × 20 maps | [evidence/raster-global-pool-multiply-offline.json](evidence/raster-global-pool-multiply-offline.json); [evidence/live-fixtures/20261112T151417Z-global_pool_multiply.json](evidence/live-fixtures/20261112T151417Z-global_pool_multiply.json) | @@ -450,7 +450,7 @@ generated list cannot drift from the code. | Core | [`tensor_ee.core.precision`](../tensor_ee/core/precision.py) | Precision policies, and how float32 values enter an Earth Engine graph. | `validate_policy`, `exact_scalar`, `encode_float32`, `constant_value` | | Core | [`tensor_ee.core.tensor`](../tensor_ee/core/tensor.py) | Tensor carriers: R (`Raster`) and W (`WindowTensor`), plus `Feature`. | `band_names`, `ServerArray`, `Raster`, `WindowTensor`, `Feature`, `same_grid` | | Layouts | [`tensor_ee.layouts.boundaries`](../tensor_ee/layouts/boundaries.py) | Stage boundaries: how saved images enter a stage's graph and how its outputs leave it. | `grid_view`, `pin_boundary`, `saved_image`, `boundary_windows`, `boundary_raster`, `relabel_to_windows`, `relabel_from_windows` | -| Layouts | [`tensor_ee.layouts.raster`](../tensor_ee/layouts/raster.py) | R-layout converters: pinning, zero extension, stride sampling, embedded fixtures. | `project`, `finish`, `zero_extended`, `stride_sample`, `from_image`, `stride_sample_phase`, `embedded_raster` | +| Layouts | [`tensor_ee.layouts.raster`](../tensor_ee/layouts/raster.py) | R-layout converters: pinning, zero extension, stride sampling, embedded fixtures. | `project`, `finish`, `zero_extended`, `phase_pair`, `canonical_phase`, `stride_sample`, `sample_grids`, `from_image`, `stride_sample_phase`, `embedded_raster` | | Layouts | [`tensor_ee.layouts.shift`](../tensor_ee/layouts/shift.py) | W-layout translation, Swin's cyclic shift, periodic extension and domain masks. | `timm_shift`, `projection`, `translate`, `materialize`, `cyclic_neighbor`, `roll_axis`, `roll_windows`, `shift_windows`, `unshift_windows`, `periodic_extension`, `domain_mask` | | Layouts | [`tensor_ee.layouts.window`](../tensor_ee/layouts/window.py) | R <-> W converters, halos and im2col. | `as_row`, `flatten_row`, `stride_samples`, `stack_tokens`, `to_windows`, `from_dense`, `depth_to_space`, `to_dense`, `zeros`, `reshape`, `zero_outside`, `check_crop`, `same_crop`, `neighbours`, `halo`, `taps`, `im2col` | | Weights | [`tensor_ee.weights.assets`](../tensor_ee/weights/assets.py) | The FeatureCollection weight-asset format and its provider. | `ConvolutionWeights`, `convolution_weights`, `feature_records`, `FeatureCollectionWeightProvider` | diff --git a/docs/evidence/live-fixtures/20261112T185127Z-padded_conv.json b/docs/evidence/live-fixtures/20261112T185127Z-padded_conv.json new file mode 100644 index 0000000..26ebc3c --- /dev/null +++ b/docs/evidence/live-fixtures/20261112T185127Z-padded_conv.json @@ -0,0 +1,1274 @@ +{ + "schema_version": 1, + "kind": "live_fixture_run", + "status": "passed", + "command": "python tools/live_fixtures.py --run padded_conv --project tensor-ee --budget requests=12", + "selection": [ + "raster/conv2d/pad0101-k3-s2-h8-w10", + "raster/conv2d/pad0101-k3-s2-h7-w9", + "raster/grouped_conv2d/depthwise-pad0101-k3-s2", + "raster/grouped_conv2d/depthwise-pad1212-k5-s2", + "raster/grouped_conv2d/depthwise-pad2222-k5-s2", + "raster/grouped_conv2d/depthwise-pad1101-k3-s2" + ], + "project": "tensor-ee", + "credentials": "adc", + "revision": "d162897f734788e1dd8f845eb4288108e5e3f134", + "dirty": false, + "tensor_ee": "tensor_ee/__init__.py", + "versions": { + "python": "3.14.6", + "earthengine-api": "1.7.45", + "torch": "2.14.0", + "timm": "1.0.30", + "segmentation-models-pytorch": "0.5.0", + "numpy": "2.5.3" + }, + "platform": { + "system": "Darwin", + "machine": "arm64" + }, + "started_utc": "2026-11-12T18:51:27+00:00", + "budget": { + "requests": 12, + "raster_side": 64, + "window_side": 8, + "values": 1000000, + "request_bytes": 2000000, + "in_flight": 4, + "assets": 0, + "tasks": 0 + }, + "plan": { + "fixtures": 6, + "requests": 12, + "values": 2557, + "payload_bytes": 10228, + "request_bytes": 181763, + "max_values": 400, + "max_request_bytes": 22716, + "max_bands": 5, + "embedded_constants": 402, + "inline_values": 3132, + "assets": 0, + "tasks": 0, + "in_flight": 1 + }, + "max_in_flight": 1, + "errors": [], + "fixtures": [ + { + "spec": { + "id": "raster/conv2d/pad0101-k3-s2-h8-w10", + "lane": "raster", + "block": "conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 1, + "padding": [ + 0, + 1, + 0, + 1 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 2 + ], + "output": [ + 4, + 5, + 3 + ] + }, + "seed": 81, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((0, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 1)", + "mirror": null, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(0, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); No NumPy mirror of the dense convolution's matrixMultiply exists: compared with PyTorch only." + }, + "inputs_sha256": "d0180c8411c71d4c28486837accfe4bf9ab9652a74071e88b3384d75b022bec6", + "weights_sha256": "9baaa16c75f7ede1f533f709f804f25b1ff0a8b3f38db5ab4bbb0c6d11f42b64", + "status": "passed", + "wall_seconds": 1.501, + "failures": [], + "offline": { + "out": { + "mirror_vs_pytorch": null, + "note": "no mirror" + } + }, + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 3 + } + ], + "bands": 4, + "values": 80, + "payload_bytes": 320, + "request_bytes": 17447, + "graph_sha256": "c658b3cb4403bd7a21b7b7b6c4acc1f933b969665520edc9ec73ce136dcf3d8b", + "graph_bytes": 17219, + "embedded_constants": 57, + "inline_values": 160, + "status": "ok", + "wall_seconds": 1.194 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 2 + } + ], + "bands": 3, + "values": 240, + "payload_bytes": 960, + "request_bytes": 8065, + "graph_sha256": "32df73f4874ce8fd89813ef47470ad748ec1b54e28683d4f9a983180760356d5", + "graph_bytes": 7836, + "embedded_constants": 0, + "inline_values": 160, + "status": "ok", + "wall_seconds": 0.306 + } + ], + "outputs": { + "out": { + "kind": "output", + "request": 0, + "shape": [ + 4, + 5, + 3 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 60, + "max_abs": 2.9802322387695312e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 0, + "col": 0, + "channel": 2, + "actual": -0.3765730559825897, + "expected": -0.37657302618026733 + } + } + }, + "input": { + "kind": "input", + "request": 1, + "shape": [ + 8, + 10, + 2 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "input": { + "exact": true, + "bound": null, + "values": 160, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + } + } + }, + { + "spec": { + "id": "raster/conv2d/pad0101-k3-s2-h7-w9", + "lane": "raster", + "block": "conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 1, + "padding": [ + 0, + 1, + 0, + 1 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 7, + 9, + 2 + ], + "output": [ + 3, + 4, + 3 + ] + }, + "seed": 82, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((0, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 1)", + "mirror": null, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(0, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); No NumPy mirror of the dense convolution's matrixMultiply exists: compared with PyTorch only." + }, + "inputs_sha256": "45ff7845069d5cbbe766ef26d188dee198ba789c4d7ff22742e2d4592b5b50d3", + "weights_sha256": "9d9afa7f7b72e58fecdae1a41aa2e21bdebe97be687a787bb617d244b7788f45", + "status": "passed", + "wall_seconds": 0.775, + "failures": [], + "offline": { + "out": { + "mirror_vs_pytorch": null, + "note": "no mirror" + } + }, + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 4, + "height": 3 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 4, + "height": 3, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 3 + } + ], + "bands": 4, + "values": 48, + "payload_bytes": 192, + "request_bytes": 16990, + "graph_sha256": "6c9275df4150c694d69c874cd6a708abd8e811a7bac3e5095395d0ed0529edb0", + "graph_bytes": 16762, + "embedded_constants": 57, + "inline_values": 126, + "status": "ok", + "wall_seconds": 0.417 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 9, + "height": 7 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 9, + "height": 7, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 2 + } + ], + "bands": 3, + "values": 189, + "payload_bytes": 756, + "request_bytes": 7636, + "graph_sha256": "6a2a08285c30df3302b9687d23ea85279e6ab41dfcf315b8a0a4e1433938c74d", + "graph_bytes": 7408, + "embedded_constants": 0, + "inline_values": 126, + "status": "ok", + "wall_seconds": 0.357 + } + ], + "outputs": { + "out": { + "kind": "output", + "request": 0, + "shape": [ + 3, + 4, + 3 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 36, + "max_abs": 5.960464477539063e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 2, + "col": 1, + "channel": 2, + "actual": 0.3490458130836487, + "expected": 0.3490457534790039 + } + } + }, + "input": { + "kind": "input", + "request": 1, + "shape": [ + 7, + 9, + 2 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "input": { + "exact": true, + "bound": null, + "values": 126, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + } + } + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad0101-k3-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 4, + "padding": [ + 0, + 1, + 0, + 1 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 83, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((0, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(0, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "7191642021ea6b2ad313f4a97f5fd0c13fdcbcd87944eaed9240560cc830a7a1", + "weights_sha256": "d816913c5045aa7daa44db6a395ee99b316b5e1e397f2bbb77931a4fb4f290b2", + "status": "passed", + "wall_seconds": 2.463, + "failures": [], + "offline": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 5.960464477539063e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 1, + "col": 0, + "channel": 1, + "actual": 0.27021902799606323, + "expected": 0.27021896839141846 + } + } + } + }, + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 20578, + "graph_sha256": "6f866eab05fc4673d2562beb5eaffcb3280ba6a8f59db70d12eed2a2193bdc64", + "graph_bytes": 20350, + "embedded_constants": 40, + "inline_values": 320, + "status": "ok", + "wall_seconds": 1.246 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11270, + "graph_sha256": "f2614088951209d21e3a2452285d1e143573ba8c1ae23768bd73c3a4a699fb5c", + "graph_bytes": 11041, + "embedded_constants": 0, + "inline_values": 320, + "status": "ok", + "wall_seconds": 1.216 + } + ], + "outputs": { + "out": { + "kind": "output", + "request": 0, + "shape": [ + 4, + 5, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 5.960464477539063e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 1, + "col": 0, + "channel": 1, + "actual": 0.27021902799606323, + "expected": 0.27021896839141846 + } + }, + "mirror": { + "exact": true, + "bound": null, + "values": 80, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + }, + "input": { + "kind": "input", + "request": 1, + "shape": [ + 8, + 10, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "input": { + "exact": true, + "bound": null, + "values": 320, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + } + } + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad1212-k5-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 5, + "stride": 2, + "groups": 4, + "padding": [ + 1, + 2, + 1, + 2 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 84, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((1, 2, 1, 2)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(1, 2, 1, 2) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "60a9febe42ac1b26072859284bebf7b3b59f6960b71af6583ff129caa45871a5", + "weights_sha256": "79f4b91b98c46eb9840e14ce3da557db83b0069638b7e4b3acc2d9062a819991", + "status": "passed", + "wall_seconds": 1.453, + "failures": [], + "offline": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 5.960464477539063e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 2, + "col": 0, + "channel": 3, + "actual": 0.3108431100845337, + "expected": 0.3108430504798889 + } + } + } + }, + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 22716, + "graph_sha256": "39a5a6f297ef4ab14bda450241724f0e5fad8256006dcdb7e0e8430426d176f2", + "graph_bytes": 22488, + "embedded_constants": 104, + "inline_values": 320, + "status": "ok", + "wall_seconds": 1.189 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11269, + "graph_sha256": "dbda4ceac7a86f6aaeb50ce17f785753f321c67921738925e0d7db2002cfd4bd", + "graph_bytes": 11040, + "embedded_constants": 0, + "inline_values": 320, + "status": "ok", + "wall_seconds": 0.263 + } + ], + "outputs": { + "out": { + "kind": "output", + "request": 0, + "shape": [ + 4, + 5, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 5.960464477539063e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 2, + "col": 0, + "channel": 3, + "actual": 0.3108431100845337, + "expected": 0.3108430504798889 + } + }, + "mirror": { + "exact": true, + "bound": null, + "values": 80, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + }, + "input": { + "kind": "input", + "request": 1, + "shape": [ + 8, + 10, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "input": { + "exact": true, + "bound": null, + "values": 320, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + } + } + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad2222-k5-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 5, + "stride": 2, + "groups": 4, + "padding": [ + 2, + 2, + 2, + 2 + ], + "phase": 0, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 85, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((2, 2, 2, 2)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(2, 2, 2, 2) absorbed into F.conv2d, so the block runs at phase (0, 0) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "f44951870074a5da1f80cf9259a1cd1b3127f2f64237c3760c535cea1be68ff3", + "weights_sha256": "4c5d36839c919d0522d7fe32346fdabf2d4011756ecacbc700bc28ee3dc59baf", + "status": "passed", + "wall_seconds": 1.54, + "failures": [], + "offline": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 4.470348358154297e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 2, + "col": 3, + "channel": 1, + "actual": 0.20211467146873474, + "expected": 0.20211462676525116 + } + } + } + }, + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 22701, + "graph_sha256": "d74118084276cd3c425a75e5ce21a454053d41f015aee7278bd788689912ebe4", + "graph_bytes": 22473, + "embedded_constants": 104, + "inline_values": 320, + "status": "ok", + "wall_seconds": 1.317 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11250, + "graph_sha256": "cbddcc996820ff74adb79b4905b9862dd2e55ddbfcd212dd065a3adc4a12291f", + "graph_bytes": 11021, + "embedded_constants": 0, + "inline_values": 320, + "status": "ok", + "wall_seconds": 0.223 + } + ], + "outputs": { + "out": { + "kind": "output", + "request": 0, + "shape": [ + 4, + 5, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 4.470348358154297e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 2, + "col": 3, + "channel": 1, + "actual": 0.20211467146873474, + "expected": 0.20211462676525116 + } + }, + "mirror": { + "exact": true, + "bound": null, + "values": 80, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + }, + "input": { + "kind": "input", + "request": 1, + "shape": [ + 8, + 10, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "input": { + "exact": true, + "bound": null, + "values": 320, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + } + } + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad1101-k3-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 4, + "padding": [ + 1, + 1, + 0, + 1 + ], + "phase": [ + 1, + 0 + ], + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 86, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((1, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(1, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 0) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "a594aa74b13a6c6e528d301ab83cca568ebb937a27a38247d77048cf4d4feb81", + "weights_sha256": "df6fa2afbafaea008d49f1fe9d9a223d0ec3b868131078e07d3041e3b9189d3e", + "status": "passed", + "wall_seconds": 1.533, + "failures": [], + "offline": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 1.4901161193847656e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 0, + "col": 0, + "channel": 3, + "actual": 0.18980781733989716, + "expected": 0.18980783224105835 + } + } + } + }, + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 20579, + "graph_sha256": "d69fb69902c5ff4c58cf8bf0dd7d663ae04fdbb53a2f39354c55620dc066a48e", + "graph_bytes": 20351, + "embedded_constants": 40, + "inline_values": 320, + "status": "ok", + "wall_seconds": 1.22 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11262, + "graph_sha256": "5a441c878fdc60be35c4da5b619331af353545fbcbe066cc546c52e4cb300062", + "graph_bytes": 11033, + "embedded_constants": 0, + "inline_values": 320, + "status": "ok", + "wall_seconds": 0.312 + } + ], + "outputs": { + "out": { + "kind": "output", + "request": 0, + "shape": [ + 4, + 5, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 1.4901161193847656e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 0, + "col": 0, + "channel": 3, + "actual": 0.18980781733989716, + "expected": 0.18980783224105835 + } + }, + "mirror": { + "exact": true, + "bound": null, + "values": 80, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + }, + "input": { + "kind": "input", + "request": 1, + "shape": [ + 8, + 10, + 4 + ], + "dtypes": [ + "float32" + ], + "float32": true, + "invalid_pixels": 0, + "input": { + "exact": true, + "bound": null, + "values": 320, + "differing": 0, + "max_abs": 0.0, + "first": [], + "pass": true + } + } + } + } + ], + "cloud_calls": { + "requests": 12, + "computePixels": 12, + "algorithms_list": 1, + "refused": [] + }, + "finished_utc": "2026-11-12T18:51:40+00:00", + "wall_seconds": 12.136 +} diff --git a/docs/evidence/raster-padded-conv-offline.json b/docs/evidence/raster-padded-conv-offline.json new file mode 100644 index 0000000..a449e38 --- /dev/null +++ b/docs/evidence/raster-padded-conv-offline.json @@ -0,0 +1,862 @@ +{ + "schema_version": 1, + "kind": "offline_fixture_check", + "cloud_calls": 0, + "status": "passed", + "revision": "1a378121c2a44e4df12cbde1d37e20353d1652be", + "dirty": false, + "tensor_ee": "tensor_ee/__init__.py", + "versions": { + "python": "3.14.6", + "earthengine-api": "1.7.45", + "torch": "2.14.0", + "timm": "1.0.30", + "segmentation-models-pytorch": "0.5.0", + "numpy": "2.5.3" + }, + "platform": { + "system": "Darwin", + "machine": "arm64" + }, + "plan": { + "totals": { + "fixtures": 6, + "requests": 12, + "values": 2557, + "payload_bytes": 10228, + "request_bytes": 181763, + "max_values": 400, + "max_request_bytes": 22716, + "max_bands": 5, + "embedded_constants": 402, + "inline_values": 3132, + "assets": 0, + "tasks": 0, + "in_flight": 1 + }, + "violations": [] + }, + "fixtures": [ + { + "spec": { + "id": "raster/conv2d/pad0101-k3-s2-h8-w10", + "lane": "raster", + "block": "conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 1, + "padding": [ + 0, + 1, + 0, + 1 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 2 + ], + "output": [ + 4, + 5, + 3 + ] + }, + "seed": 81, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((0, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 1)", + "mirror": null, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(0, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); No NumPy mirror of the dense convolution's matrixMultiply exists: compared with PyTorch only." + }, + "inputs_sha256": "d0180c8411c71d4c28486837accfe4bf9ab9652a74071e88b3384d75b022bec6", + "weights_sha256": "9baaa16c75f7ede1f533f709f804f25b1ff0a8b3f38db5ab4bbb0c6d11f42b64", + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 3 + } + ], + "bands": 4, + "values": 80, + "payload_bytes": 320, + "request_bytes": 17447, + "graph_sha256": "c658b3cb4403bd7a21b7b7b6c4acc1f933b969665520edc9ec73ce136dcf3d8b", + "graph_bytes": 17219, + "embedded_constants": 57, + "inline_values": 160 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 2 + } + ], + "bands": 3, + "values": 240, + "payload_bytes": 960, + "request_bytes": 8065, + "graph_sha256": "32df73f4874ce8fd89813ef47470ad748ec1b54e28683d4f9a983180760356d5", + "graph_bytes": 7836, + "embedded_constants": 0, + "inline_values": 160 + } + ], + "errors": [], + "seconds": 0.016, + "outputs": { + "out": { + "mirror_vs_pytorch": null, + "note": "no mirror" + } + }, + "status": "passed" + }, + { + "spec": { + "id": "raster/conv2d/pad0101-k3-s2-h7-w9", + "lane": "raster", + "block": "conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 1, + "padding": [ + 0, + 1, + 0, + 1 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 7, + 9, + 2 + ], + "output": [ + 3, + 4, + 3 + ] + }, + "seed": 82, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((0, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 1)", + "mirror": null, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(0, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); No NumPy mirror of the dense convolution's matrixMultiply exists: compared with PyTorch only." + }, + "inputs_sha256": "45ff7845069d5cbbe766ef26d188dee198ba789c4d7ff22742e2d4592b5b50d3", + "weights_sha256": "9d9afa7f7b72e58fecdae1a41aa2e21bdebe97be687a787bb617d244b7788f45", + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 4, + "height": 3 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 4, + "height": 3, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 3 + } + ], + "bands": 4, + "values": 48, + "payload_bytes": 192, + "request_bytes": 16990, + "graph_sha256": "6c9275df4150c694d69c874cd6a708abd8e811a7bac3e5095395d0ed0529edb0", + "graph_bytes": 16762, + "embedded_constants": 57, + "inline_values": 126 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 9, + "height": 7 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 9, + "height": 7, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 2 + } + ], + "bands": 3, + "values": 189, + "payload_bytes": 756, + "request_bytes": 7636, + "graph_sha256": "6a2a08285c30df3302b9687d23ea85279e6ab41dfcf315b8a0a4e1433938c74d", + "graph_bytes": 7408, + "embedded_constants": 0, + "inline_values": 126 + } + ], + "errors": [], + "seconds": 0.016, + "outputs": { + "out": { + "mirror_vs_pytorch": null, + "note": "no mirror" + } + }, + "status": "passed" + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad0101-k3-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 4, + "padding": [ + 0, + 1, + 0, + 1 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 83, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((0, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(0, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "7191642021ea6b2ad313f4a97f5fd0c13fdcbcd87944eaed9240560cc830a7a1", + "weights_sha256": "d816913c5045aa7daa44db6a395ee99b316b5e1e397f2bbb77931a4fb4f290b2", + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 20578, + "graph_sha256": "6f866eab05fc4673d2562beb5eaffcb3280ba6a8f59db70d12eed2a2193bdc64", + "graph_bytes": 20350, + "embedded_constants": 40, + "inline_values": 320 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11270, + "graph_sha256": "f2614088951209d21e3a2452285d1e143573ba8c1ae23768bd73c3a4a699fb5c", + "graph_bytes": 11041, + "embedded_constants": 0, + "inline_values": 320 + } + ], + "errors": [], + "seconds": 0.016, + "outputs": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 5.960464477539063e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 1, + "col": 0, + "channel": 1, + "actual": 0.27021902799606323, + "expected": 0.27021896839141846 + } + } + } + }, + "status": "passed" + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad1212-k5-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 5, + "stride": 2, + "groups": 4, + "padding": [ + 1, + 2, + 1, + 2 + ], + "phase": 1, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 84, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((1, 2, 1, 2)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(1, 2, 1, 2) absorbed into F.conv2d, so the block runs at phase (1, 1) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "60a9febe42ac1b26072859284bebf7b3b59f6960b71af6583ff129caa45871a5", + "weights_sha256": "79f4b91b98c46eb9840e14ce3da557db83b0069638b7e4b3acc2d9062a819991", + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 22716, + "graph_sha256": "39a5a6f297ef4ab14bda450241724f0e5fad8256006dcdb7e0e8430426d176f2", + "graph_bytes": 22488, + "embedded_constants": 104, + "inline_values": 320 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11269, + "graph_sha256": "dbda4ceac7a86f6aaeb50ce17f785753f321c67921738925e0d7db2002cfd4bd", + "graph_bytes": 11040, + "embedded_constants": 0, + "inline_values": 320 + } + ], + "errors": [], + "seconds": 0.015, + "outputs": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 5.960464477539063e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 2, + "col": 0, + "channel": 3, + "actual": 0.3108431100845337, + "expected": 0.3108430504798889 + } + } + } + }, + "status": "passed" + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad2222-k5-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 5, + "stride": 2, + "groups": 4, + "padding": [ + 2, + 2, + 2, + 2 + ], + "phase": 0, + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 85, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((2, 2, 2, 2)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(2, 2, 2, 2) absorbed into F.conv2d, so the block runs at phase (0, 0) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "f44951870074a5da1f80cf9259a1cd1b3127f2f64237c3760c535cea1be68ff3", + "weights_sha256": "4c5d36839c919d0522d7fe32346fdabf2d4011756ecacbc700bc28ee3dc59baf", + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 22701, + "graph_sha256": "d74118084276cd3c425a75e5ce21a454053d41f015aee7278bd788689912ebe4", + "graph_bytes": 22473, + "embedded_constants": 104, + "inline_values": 320 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11250, + "graph_sha256": "cbddcc996820ff74adb79b4905b9862dd2e55ddbfcd212dd065a3adc4a12291f", + "graph_bytes": 11021, + "embedded_constants": 0, + "inline_values": 320 + } + ], + "errors": [], + "seconds": 0.016, + "outputs": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 4.470348358154297e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 2, + "col": 3, + "channel": 1, + "actual": 0.20211467146873474, + "expected": 0.20211462676525116 + } + } + } + }, + "status": "passed" + }, + { + "spec": { + "id": "raster/grouped_conv2d/depthwise-pad1101-k3-s2", + "lane": "raster", + "block": "grouped_conv2d", + "parts": [], + "settings": { + "kernel_size": 3, + "stride": 2, + "groups": 4, + "padding": [ + 1, + 1, + 0, + 1 + ], + "phase": [ + 1, + 0 + ], + "clip_footprints": false + }, + "policy": "float32_operands", + "shapes": { + "input": [ + 8, + 10, + 4 + ], + "output": [ + 4, + 5, + 4 + ] + }, + "seed": 86, + "tolerance": 0.0001, + "requests": 2, + "reference": "nn.ZeroPad2d((1, 1, 0, 1)) then F.conv2d(x, weight, bias, 2, 0, 1, 4)", + "mirror": { + "source": "reference.raster_conv.grouped_conv2d", + "exact": true, + "bound": null + }, + "history": "New fixture: not yet run live.", + "calibration": false, + "notes": "compile_model on a stand-in module: ZeroPad2d(1, 1, 0, 1) absorbed into F.conv2d, so the block runs at phase (1, 0) (output pixel i is input pixel 2 i + phase on each axis); The mirror applies the same zero padding and PyTorch's output count." + }, + "inputs_sha256": "a594aa74b13a6c6e528d301ab83cca568ebb937a27a38247d77048cf4d4feb81", + "weights_sha256": "df6fa2afbafaea008d49f1fe9d9a223d0ec3b868131078e07d3041e3b9189d3e", + "requests": [ + { + "index": 0, + "grid": { + "dimensions": { + "width": 5, + "height": 4 + }, + "affineTransform": { + "scaleX": 20, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -20, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 5, + "height": 4, + "readouts": [ + { + "name": "out", + "kind": "output", + "channels": 4 + } + ], + "bands": 5, + "values": 100, + "payload_bytes": 400, + "request_bytes": 20579, + "graph_sha256": "d69fb69902c5ff4c58cf8bf0dd7d663ae04fdbb53a2f39354c55620dc066a48e", + "graph_bytes": 20351, + "embedded_constants": 40, + "inline_values": 320 + }, + { + "index": 1, + "grid": { + "dimensions": { + "width": 10, + "height": 8 + }, + "affineTransform": { + "scaleX": 10, + "shearX": 0, + "translateX": 1000, + "shearY": 0, + "scaleY": -10, + "translateY": 2000 + }, + "crsCode": "EPSG:3857" + }, + "width": 10, + "height": 8, + "readouts": [ + { + "name": "input", + "kind": "input", + "channels": 4 + } + ], + "bands": 5, + "values": 400, + "payload_bytes": 1600, + "request_bytes": 11262, + "graph_sha256": "5a441c878fdc60be35c4da5b619331af353545fbcbe066cc546c52e4cb300062", + "graph_bytes": 11033, + "embedded_constants": 0, + "inline_values": 320 + } + ], + "errors": [], + "seconds": 0.015, + "outputs": { + "out": { + "mirror_vs_pytorch": { + "tolerance": 0.0001, + "comparison": "absolute", + "values": 80, + "max_abs": 1.4901161193847656e-08, + "over_tolerance": 0, + "pass": true, + "worst": { + "row": 0, + "col": 0, + "channel": 3, + "actual": 0.18980781733989716, + "expected": 0.18980783224105835 + } + } + } + }, + "status": "passed" + } + ] +} diff --git a/docs/smp-models.md b/docs/smp-models.md index c5eec87..769cfcf 100644 --- a/docs/smp-models.md +++ b/docs/smp-models.md @@ -170,6 +170,19 @@ offline graph construction with symbolic weights, and exact PyTorch adapter probe equality. These checks do not establish Earth Engine numerical agreement for either whole model. +SMP's own EfficientNet encoders write each convolution as a padding module (or an +`Identity`) followed by `F.conv2d` with the module's weights, and fix that padding for +the encoder's configured image size; some stride-2 layers are padded `(0, 1, 0, 1)` +(3 × 3) or `(1, 2, 1, 2)` (5 × 5). The compiler recognises these as convolutions at a +sampling phase ([API guide](api.md#spatial-and-numerical-rules)). With random weights, +three input channels and two classes for `efficientnet-b0`, and eight input channels and +three classes for `efficientnet-b3`, a Unet at 256 × 256 is `supported` by +`check_compatibility` after `adapt_smp_model`, whose probe is equal (max_abs 0). Without the +adapter, the decoder's 10 in-place ReLUs are the only blockers. This is a trace status +only: no graph of either model has been evaluated, the sampling phase is validated live +on small fixtures and not on these models, and there is no numerical or accuracy claim +for these encoders. + Run the full matrix explicitly; the default suite skips it: ```sh diff --git a/tensor_ee/blocks/raster/conv.py b/tensor_ee/blocks/raster/conv.py index 6e312ca..597ad99 100644 --- a/tensor_ee/blocks/raster/conv.py +++ b/tensor_ee/blocks/raster/conv.py @@ -37,6 +37,28 @@ group), and a 4 -> 8 channel multiplier with 4 groups (both 3 x 3, stride 1, with bias). Kernel, stride and groups are settings of the PyTorch module, so the record names only clip_footprints and the status names the module settings. + +**Sampling phase** (both blocks, keyword-only ``phase``, default 0). An integer or a +pair ``(phase_y, phase_x)``, each ``0 <= phase < stride``. On an axis with input size +n, kernel k, stride s and phase p, output pixel i is the centred convolution at input +pixel ``s i + p``:: + + y[i] = b + sum_t w[t] * x[s i + p - k // 2 + t], t = 0 .. k - 1, x = 0 outside [0, n) + +which is PyTorch's convolution of a map zero-padded ``k // 2 - p`` pixels before the +input, with ``ceil((n - p) / s)`` outputs. The convolution itself is unchanged (the same +neighbourhood bands, operands, accumulation and one float32 rounding by the caller); +only ``stride_sample`` picks other pixels. Phase 0 is today's graph. Its reach from an +output's anchor ``s i`` is ``k // 2 + p``. ``canonical_phase`` gives the one form a phase +is reported in. + +Validated live at stride 2 with clip_footprints=False, through ``compile_model`` on zero +padding ahead of ``F.conv2d``: dense 3 x 3 at phase 1 on 8 x 10 and 7 x 9 inputs (within +6e-8 of float32 PyTorch; the dense block has no NumPy mirror); depthwise 3 x 3 at phase 1 +and at phase (1, 0), and depthwise 5 x 5 at phase 1 and, through a symmetric pad, at +phase 0, on 8 x 10 inputs (Earth Engine equal to the NumPy mirror in every value). Other +kernels and phases, such as a dense 5 x 5 or 7 x 7 at a nonzero phase or a dense mixed +phase, are not validated. """ from __future__ import annotations @@ -56,6 +78,7 @@ # the names conv2d looks up (tests patch _inline_float32_array here). from ...layouts.raster import project as _project, stride_sample as _stride_sample from ...layouts.raster import zero_extended as _zero_extended +from ...layouts.raster import canonical_phase, phase_pair from ...weights.constants import inline_float32_array as _inline_float32_array @@ -148,12 +171,19 @@ def _patch_bands(bands, kh, kw): def conv2d(raster: Raster, module: nn.Conv2d, parameters: tuple[ee.Array, ee.Array | None] | None = None, *, execution_dtype: str = "float32", clip_footprints: bool = False, - policy: str = FLOAT32_OPERANDS) -> Raster: - """PyTorch cross-correlation via ordered neighborhood bands and matrixMultiply.""" + policy: str = FLOAT32_OPERANDS, phase=0) -> Raster: + """PyTorch cross-correlation via ordered neighborhood bands and matrixMultiply. + + ``phase`` (keyword-only, default 0; an integer or a pair ``(phase_y, phase_x)``, + each ``0 <= phase < stride``) samples the centred convolution at input pixel + ``stride * i + phase`` on each axis: PyTorch's convolution with zero padding + ``kernel // 2 - phase`` before the input. Validated live at kernel 3, stride 2, phase 1. + """ validate_policy(policy, allowed=FLOAT32_OPERANDS) if execution_dtype != "float32": raise ValueError("execution_dtype must be float32 (maximum supported precision)") (kh, kw), stride_value = conv_options(module) + phase = phase_pair(phase, stride_value) stride = (stride_value, stride_value) if module.in_channels != len(raster.bands): raise UnsupportedOperation("Conv2d input channel count does not match raster") @@ -187,19 +217,23 @@ def conv2d(raster: Raster, module: nn.Conv2d, # redundant reproject in the long residual/decoder graph. if stride[0] > 1: result = _project(result, raster.grid, clip_footprints) - return _stride_sample(result, bands, raster.grid, stride[0], clip_footprints) + return _stride_sample(result, bands, raster.grid, stride[0], clip_footprints, phase=phase) def grouped_conv2d(raster: Raster, module: nn.Conv2d, parameters: tuple[ee.Array, ee.Array | None] | None = None, - *, clip_footprints: bool = False, policy: str = FLOAT32_OPERANDS) -> Raster: + *, clip_footprints: bool = False, policy: str = FLOAT32_OPERANDS, + phase=0) -> Raster: """Grouped cross-correlation: one repeated patch, double product and tap reduction. ``parameters`` follows conv2d's asset layout, with the grouped weight's actual input width: [out, in_per_group * kh * kw], bias [out, 1] or None. + ``phase`` is ``conv2d``'s sampling phase (validated live for depthwise 3 x 3 and 5 x 5 + at stride 2). """ validate_policy(policy, allowed=FLOAT32_OPERANDS) (kh, kw), stride = grouped_conv_options(module) + phase = phase_pair(phase, stride) if module.in_channels != len(raster.bands): raise UnsupportedOperation("Conv2d input channel count does not match raster") groups = module.groups @@ -227,7 +261,7 @@ def grouped_conv2d(raster: Raster, module: nn.Conv2d, result = result.arrayProject([0]).arrayFlatten([list(bands)]) if stride > 1: result = _project(result, raster.grid, clip_footprints) - return _stride_sample(result, bands, raster.grid, stride, clip_footprints) + return _stride_sample(result, bands, raster.grid, stride, clip_footprints, phase=phase) # --------------------------------------------------------------------------- @@ -268,23 +302,31 @@ def grouped_conv2d_arrays(module: nn.Conv2d) -> int: "grouped_conv2d": BlockMetadata( name="grouped_conv2d", layout_in="R", layout_out="R", locality=Locality.KERNEL, policy=FLOAT32_OPERANDS, embedded_constants=grouped_conv2d_values, - asset_arrays=grouped_conv2d_arrays, radius=2, - status="Ready — depthwise 3×3 stride 1 or 2, 5×5 stride 1; 3×3 stride 1 with 2 channels per group or channel multiplier 2", + asset_arrays=grouped_conv2d_arrays, radius=3, + status="Ready — depthwise 3×3 stride 1 or 2, 5×5 stride 1 or 2; 3×3 stride 1 with 2 channels per group or channel multiplier 2", evidence=("docs/evidence/raster-grouped-conv-offline.json", - "docs/evidence/live-fixtures/20261029T125301Z-grouped_conv.json"), + "docs/evidence/live-fixtures/20261029T125301Z-grouped_conv.json", + "docs/evidence/raster-padded-conv-offline.json", + "docs/evidence/live-fixtures/20261112T185127Z-padded_conv.json"), + # Radius 3: the 5 x 5 kernel at phase 1 reaches k // 2 + 1. The live runs varied + # phase (0, 1 and (1, 0)), so the record does not name it. validated=Validated(({"clip_footprints": (False,)},))), # Radius k // 2 for a k x k kernel (receptive_field._window_axis with - # padding k // 2). The static field records the largest validated kernel, - # 7 (radius 3); kernels 1 and 3 have radius 0 and 1. + # padding k // 2). The static field records the largest validated reach: kernel + # 7 at phase 0 (radius 3); kernels 1 and 3 have radius 0 and 1. At a nonzero phase the + # window reaches k // 2 + phase on its far side: 2 for the validated kernel 3 at phase 1. # Validated: any accepted call. Kernels 1, 3 and 7 at strides 1 and 2, and - # clip_footprints both ways (True in resnet18_full64_float32_policy.json), were run; - # the unqualified "Ready" covers them, and other forms are refused. + # clip_footprints both ways (True in resnet18_full64_float32_policy.json), were run, + # and kernel 3 at stride 2 and phase 1; the unqualified "Ready" covers them, and other + # forms are refused. The live runs varied phase, so the record does not name it. "conv2d": BlockMetadata( name="conv2d", layout_in="R", layout_out="R", locality=Locality.KERNEL, policy=FLOAT32_OPERANDS, embedded_constants=_conv_coefficients, asset_arrays=_conv_asset_arrays, radius=3, status="Ready", evidence=("docs/history/cnn/results/parity.json", "docs/history/cnn/results/primitives_float32.json", "docs/history/cnn/results/resnet_primitives.json", "docs/history/cnn/results/resnet18_full64_float32_policy.json", - "docs/history/cnn/results/segwater_js_parity_1x1.json", "docs/history/cnn/results/tiled_resnet18_2x2_parity.json"), + "docs/history/cnn/results/segwater_js_parity_1x1.json", "docs/history/cnn/results/tiled_resnet18_2x2_parity.json", + "docs/evidence/raster-padded-conv-offline.json", + "docs/evidence/live-fixtures/20261112T185127Z-padded_conv.json"), validated=ANY_SETTINGS), }) diff --git a/tensor_ee/compatibility.py b/tensor_ee/compatibility.py index 96a82e7..6570e48 100644 --- a/tensor_ee/compatibility.py +++ b/tensor_ee/compatibility.py @@ -11,7 +11,7 @@ from torch.fx import GraphModule, Node, symbolic_trace from .compiler import UnsupportedOperation, _ROOT_MODULES -from .semantics import (SemanticUnknown, TensorMetadata, diagnostic_convolution, +from .semantics import (EMITS_NOTHING, SemanticUnknown, TensorMetadata, diagnostic_convolution, infer_metadata, semantic_call, specialize_plumbing) @@ -152,7 +152,7 @@ def finish() -> CompatibilityReport: # Static semantics first: invalid options remain blockers even # when a predecessor's shape could not be inferred. call = semantic_call(node, graph) - if call.kind == "batchnorm_constant": + if call.kind in EMITS_NOTHING: continue if call.kind in ("conv", "grouped_conv"): if not diagnostic_convolution(node, graph): diff --git a/tensor_ee/compiler.py b/tensor_ee/compiler.py index 847f5d0..a05ac22 100644 --- a/tensor_ee/compiler.py +++ b/tensor_ee/compiler.py @@ -7,7 +7,8 @@ batch size one; channels are EE bands and spatial dimensions are the Grid. Semantics are intentionally strict: finite rectangles, missing input values -filled with zero, odd same-padded stride-one/two convolutions, aligned k x k +filled with zero, odd stride-one/two convolutions, same-padded or zero-padded at +a sampling phase (``semantics.convolution``), aligned k x k pooling (k = 2, 3, 4), padded 3x3 max pooling (stride one or two) and stride-one average pooling, integer nearest/bilinear upsampling, residual addition, channel concatenation, global average pooling onto the map's one-pixel grid and elementwise @@ -246,7 +247,10 @@ def compile_model(model: nn.Module, image: ee.Image, input_bands: Sequence[str], clip_footprints: bool = False) -> CompiledModel: """Convert an eval-mode model into an EE graph; do not evaluate or export it. - Supports Conv2d (including grouped/depthwise), ReLU, AvgPool2d, MaxPool2d, nearest and bilinear Upsample, + Supports Conv2d (including grouped/depthwise) and F.conv2d with the model's own weight and + bias, each with any zero padding directly ahead of it (ZeroPad2d, ConstantPad2d with 0, + F.pad with 0) that leaves kernel // 2 - phase pixels before the input with + 0 <= phase < stride (a nonzero phase is validated live at stride 2 for 3 x 3 kernels and a depthwise 5 x 5), ReLU, AvgPool2d, MaxPool2d, nearest and bilinear Upsample, Sigmoid, Identity, LeakyReLU, SiLU, exact GELU, Tanh, ReLU6, Hardswish, Hardsigmoid, eval Dropout/Dropout2d, channel Softmax, eval BatchNorm2d and F.batch_norm with fixed buffer/parameter statistics, torch.cat, @@ -317,14 +321,15 @@ def compile_model(model: nn.Module, image: ee.Image, input_bands: Sequence[str], placeholders = 0 output: Raster | None = None - def convolve(key: str, raster: Raster, module: nn.Conv2d) -> Raster: + def convolve(key: str, raster: Raster, module: nn.Conv2d, phase) -> Raster: parameters = None if weight_provider is None else weight_provider(key, module) if module.groups != 1: - return grouped_conv2d(raster, module, parameters, clip_footprints=clip_footprints) + return grouped_conv2d(raster, module, parameters, clip_footprints=clip_footprints, phase=phase) return conv2d(raster, module, parameters, execution_dtype=execution_dtype, - clip_footprints=clip_footprints) + clip_footprints=clip_footprints, phase=phase) - from .semantics import (TensorMetadata, diagnostic_convolution, infer_metadata, semantic_call) + from .semantics import (EMITS_NOTHING, TensorMetadata, diagnostic_convolution, infer_metadata, + semantic_call) metadata: dict[Node, TensorMetadata] = {} for node in traced.graph.nodes: try: @@ -344,7 +349,7 @@ def convolve(key: str, raster: Raster, module: nn.Conv2d) -> Raster: continue else: call = semantic_call(node, traced) - if call.kind == "batchnorm_constant": + if call.kind in EMITS_NOTHING: continue metadata[node] = infer_metadata(call, tuple(metadata[x] for x in call.inputs)) kind = call.kind @@ -352,9 +357,10 @@ def convolve(key: str, raster: Raster, module: nn.Conv2d) -> Raster: if not include_unfused_nodes and diagnostic_convolution(node, traced): environment[node] = None continue - raster = convolve(node.name, environment[call.inputs[0]], call.module) + raster = convolve(node.name, environment[call.inputs[0]], call.module, call.options["phase"]) elif kind == "batchnorm": - raster = convolve(node.name, environment[call.options["source"]], call.module) + raster = convolve(node.name, environment[call.options["source"]], call.module, + call.options["phase"]) folded.append(node.name) else: inputs = tuple(environment[x] for x in call.inputs) diff --git a/tensor_ee/layouts/raster.py b/tensor_ee/layouts/raster.py index 8636fcc..bb6c2c2 100644 --- a/tensor_ee/layouts/raster.py +++ b/tensor_ee/layouts/raster.py @@ -14,11 +14,13 @@ from __future__ import annotations +from numbers import Integral from typing import Sequence import ee import numpy as np +from ..core.errors import reject from ..core.grid import FeatureGrid, Grid from ..core.tensor import Raster, band_names @@ -44,32 +46,74 @@ def zero_extended(raster: Raster) -> ee.Image: return raster.image.unmask(0, sameFootprint=False).reproject(raster.grid.projection()) +def phase_pair(phase, stride: int) -> tuple[int, int]: + """A sampling phase as ``(phase_y, phase_x)``: an integer ``p`` means ``(p, p)``. + + Each phase is an integer with ``0 <= phase < stride``; a bool, a non-integer, a + sequence of another length or a value outside that range is refused.""" + values = tuple(phase) if isinstance(phase, (tuple, list)) else (phase, phase) + if (len(values) != 2 or any(isinstance(p, bool) or not isinstance(p, Integral) for p in values) + or any(not 0 <= p < stride for p in values)): + reject(f"phase must be an integer or a pair (phase_y, phase_x) of integers with " + f"0 <= phase < stride {stride}; received {phase!r}") + return int(values[0]), int(values[1]) + + +def canonical_phase(phase, stride: int) -> int | tuple[int, int]: + """The one form a phase is recorded and reported in: the integer ``p`` when both axes + have phase ``p``, else the pair ``(phase_y, phase_x)``. ``0``, ``(0, 0)`` and + ``[0, 0]`` are all ``0``.""" + y, x = phase_pair(phase, stride) + return y if y == x else (y, x) + + def stride_sample(image: ee.Image, bands: Sequence[str], grid: Grid, - stride: int, clip_footprints: bool = False) -> Raster: - """Sample indices 0,s,2s,... and retain a canonical tensor grid. + stride: int, clip_footprints: bool = False, *, phase=0) -> Raster: + """Sample indices phase, phase+s, phase+2s,... and retain a canonical tensor grid. EE's same-corner coarse grid would sample centers at input indices 1,3,... for stride two. Move its corner by half an input pixel to sample 0,2,..., then ``changeProj`` moves those *values* onto canonical tensor coordinates. A plain reproject at the second step would resample and lose the alignment. + ``phase`` (keyword-only, default 0) is an integer or a pair ``(phase_y, phase_x)``, + each ``0 <= phase < stride`` (``phase_pair``). On an axis with phase ``p`` the sample + grid's corner moves by ``(2 p - (s - 1)) / 2`` input pixels, so output pixel ``i`` is + input pixel ``s i + p``, sampled at that pixel's centre, and the axis has + ``ceil((n - p) / s)`` pixels. The canonical grid keeps the same upper-left corner. + An axis with phase 0 keeps today's corner expression, so the default emits today's + graph. Validated live at stride 2 with phases 1 and (1, 0), through the convolution blocks. + The stride is a parameter, but only strides 1 and 2 are evaluated. """ + phase = phase_pair(phase, stride) if stride == 1: return finish(image, bands, grid, clip_footprints) - width = (grid.width + stride - 1) // stride - height = (grid.height + stride - 1) // stride - canonical = grid.resized(stride, width, height) - a, b, c, d, e, f = canonical.transform - sample = Grid(grid.crs, - (a, b, c - (stride - 1) * grid.transform[0] / 2, - d, e, f - (stride - 1) * grid.transform[4] / 2), - width, height) + sample, canonical = sample_grids(grid, stride, phase) sampled = finish(image, bands, sample, clip_footprints) relabeled = sampled.image.changeProj(sample.projection(), canonical.projection()) return finish(relabeled, bands, canonical, clip_footprints) +def sample_grids(grid: Grid, stride: int, phase=0) -> tuple[Grid, Grid]: + """``stride_sample``'s two grids at ``stride`` > 1: the sample grid, whose pixel + ``(i, j)`` is centred on input pixel ``(s i + phase_y, s j + phase_x)``, and the + canonical grid (same upper-left corner as ``grid``, pixel ``s``) the values move to.""" + phase_y, phase_x = phase_pair(phase, stride) + width = (grid.width - phase_x + stride - 1) // stride + height = (grid.height - phase_y + stride - 1) // stride + if width < 1 or height < 1: + reject(f"A {grid.height} x {grid.width} input sampled at stride {stride} and phase " + f"({phase_y}, {phase_x}) has no output pixels", "input_shape") + canonical = grid.resized(stride, width, height) + a, b, c, d, e, f = canonical.transform + x0 = (c - (stride - 1) * grid.transform[0] / 2 if phase_x == 0 + else c + (2 * phase_x - (stride - 1)) * grid.transform[0] / 2) + y0 = (f - (stride - 1) * grid.transform[4] / 2 if phase_y == 0 + else f + (2 * phase_y - (stride - 1)) * grid.transform[4] / 2) + return Grid(grid.crs, (a, b, x0, d, e, y0), width, height), canonical + + def _float32(raster: Raster) -> Raster: return Raster(raster.image.toFloat(), raster.bands, raster.grid) diff --git a/tensor_ee/receptive_field.py b/tensor_ee/receptive_field.py index 0c43df3..91d1eb4 100644 --- a/tensor_ee/receptive_field.py +++ b/tensor_ee/receptive_field.py @@ -220,7 +220,8 @@ def _union_axis(sources: list[AxisSupport]) -> AxisSupport: def analyze_receptive_field(model: nn.Module | GraphModule) -> ReceptiveFieldReport: """Trace supported inference operations without executing any tensor forward. - Supports the compiler's odd same-padded convolutions, its pools (aligned + Supports the compiler's odd convolutions (same-padded, or zero-padded at a sampling + phase: the window starts at the call's total left and top padding), its pools (aligned k x k average/max for k in 2, 3, 4; padded 3x3 max at stride 1 or 2; padded 3x3 stride-1 average), fixed integer nearest/bilinear upsampling, inference BatchNorm, ReLU/Sigmoid/Identity, residual addition, channel concatenation, the elementwise @@ -247,7 +248,7 @@ def source(node: Any) -> SpatialSupport: raise UnsupportedReceptiveField("Expected a previously supported tensor node") return supports[node] - from .semantics import semantic_call + from .semantics import EMITS_NOTHING, semantic_call for node in graph.graph.nodes: try: if node.op == "placeholder": @@ -261,15 +262,21 @@ def source(node: Any) -> SpatialSupport: continue else: call = semantic_call(node, graph) - if call.kind == "batchnorm_constant": + if call.kind in EMITS_NOTHING: continue support = source(call.inputs[0]) if call.kind in ("conv", "grouped_conv", "pool"): module = call.module kernel = _pair(module.kernel_size, name="Window kernel") stride = _pair(module.stride or module.kernel_size, name="Window stride") - padding = ((kernel[0] // 2, kernel[1] // 2) if module.padding == "same" - else _pair(module.padding, name="Window padding")) + if "padding" in call.options: + # A convolution's total zero padding before the input (its own and + # any zero padding directly ahead): top and left, per axis. + left, _, top, _ = call.options["padding"] + padding = (top, left) + else: + padding = ((kernel[0] // 2, kernel[1] // 2) if module.padding == "same" + else _pair(module.padding, name="Window padding")) support = SpatialSupport(_window_axis(support.y, kernel[0], stride[0], padding[0]), _window_axis(support.x, kernel[1], stride[1], padding[1])) elif call.kind == "nearest": diff --git a/tensor_ee/semantics.py b/tensor_ee/semantics.py index 7e9f7da..cae29a7 100644 --- a/tensor_ee/semantics.py +++ b/tensor_ee/semantics.py @@ -26,6 +26,13 @@ from .blocks.raster.conv import _coefficient_field, conv_options, finite_coefficients, grouped_conv_options from .blocks.raster.pooling import aligned_size_message, pool_config, pool_options from .blocks.raster.resample import nearest_factor +from .layouts.raster import canonical_phase, phase_pair + + +EMITS_NOTHING = ("batchnorm_constant", "conv_constant", "absorbed") +"""Call kinds that emit no tensor: coefficient reads that are operands of the one call +using them, and nodes absorbed into the convolution that reads them (zero padding, and an +``Identity`` directly ahead of a functional convolution).""" @dataclass(frozen=True) @@ -91,6 +98,14 @@ def _method_mul_arguments(input, other): pass +def _conv2d_arguments(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1): + """Signature only: ``F.conv2d`` (the same object as ``torch.conv2d``).""" + + +def _pad_arguments(input, pad, mode="constant", value=None): + """Signature only: ``F.pad``.""" + + _GLOBAL_DIMS = ((2, 3), (3, 2), (-2, -1), (-1, -2)) """The spatial axes of an NCHW mean that pool the whole map, as ``dim`` may name them.""" @@ -314,7 +329,7 @@ def needs_specialization(node): node.args, node.kwargs = (options.pop("input"),), options changed = True call = semantic_call(node, graph) - if call.kind == "batchnorm_constant": + if call.kind in EMITS_NOTHING: continue if all(x in metadata for x in call.inputs): metadata[node] = infer_metadata(call, tuple(metadata[x] for x in call.inputs)) @@ -322,6 +337,20 @@ def needs_specialization(node): if plumbing or size_call: errors[node] = error if not isinstance(error, TypeError) else UnsupportedOperation( f"Invalid call arguments: {error}") + # A pad read from a tensor's shape stays refused, even where a shape element would + # resolve to a literal at this input size: zero padding must be written statically. + for node in graph_copy.nodes: + if node.op == "call_function" and node.target is F.pad and node not in errors: + try: + pad = _bind_options(_pad_arguments, node.args, node.kwargs)["pad"] + except TypeError: + continue + if any(x in values for x in _nodes_in(pad)): + try: + _reject("F.pad padding must be static integers; padding computed from a tensor's shape " + "is unsupported") + except UnsupportedOperation as error: + errors[node] = error # Replace scalar/tuple uses only after their provenance has been checked. from torch.fx.node import map_arg for node in graph_copy.nodes: @@ -382,27 +411,296 @@ def _module_batchnorm_options(module: nn.BatchNorm2d) -> dict[str, Any]: return options -def batchnorm_predecessor(node: Node, graph: GraphModule) -> tuple[Node, Node, nn.Conv2d] | None: - """The unchanged convolution fold, or None for a standalone inference BatchNorm.""" +def _batchnorm_fold(node: Node, graph: GraphModule) -> tuple[Node, Convolution, nn.Conv2d] | None: + """The convolution fold with the recognised convolution it folds, or None.""" module = graph.get_submodule(str(node.target)) _module_batchnorm_options(module) predecessor = _bind_options(_input_arguments, node.args, node.kwargs)["input"] - if (not isinstance(predecessor, Node) or predecessor.op != "call_module" or len(predecessor.users) != 1): + if not isinstance(predecessor, Node) or len(predecessor.users) != 1: return None - conv = graph.get_submodule(str(predecessor.target)) - if not isinstance(conv, nn.Conv2d): + if predecessor.op == "call_module": + conv = graph.get_submodule(str(predecessor.target)) + if not isinstance(conv, nn.Conv2d): + return None + if conv.training or module.training: + _reject("Set model.eval() before compilation", "training_mode") + if module.num_features != conv.out_channels: + _reject("BatchNorm running statistics must match convolution output channels") + recognised = convolution(predecessor, graph) + elif predecessor.op == "call_function" and predecessor.target is torch.conv2d: + recognised = convolution(predecessor, graph) + if module.num_features != recognised.module.out_channels: + _reject("BatchNorm running statistics must match convolution output channels") + else: return None - if conv.training or module.training: - _reject("Set model.eval() before compilation", "training_mode") - if module.num_features != conv.out_channels: - _reject("BatchNorm running statistics must match convolution output channels") - (conv_options if conv.groups == 1 else grouped_conv_options)(conv) # Preserve the compiler's existing folding operation and coefficient identity. - fused = torch.nn.utils.fuse_conv_bn_eval(conv, module) + fused = torch.nn.utils.fuse_conv_bn_eval(recognised.module, module) finite_coefficients(fused) - source = _bind_options(_input_arguments, predecessor.args, predecessor.kwargs)["input"] + _inputs((recognised.source,)) + return predecessor, recognised, fused + + +def batchnorm_predecessor(node: Node, graph: GraphModule) -> tuple[Node, Node, nn.Conv2d] | None: + """The unchanged convolution fold, or None for a standalone inference BatchNorm. + + The convolution is an ``nn.Conv2d`` module or a functional ``F.conv2d``, with any zero + padding directly ahead of it (``convolution``); the source returned is what that + convolution reads, the padding's input when a padding node is absorbed.""" + folded = _batchnorm_fold(node, graph) + if folded is None: + return None + predecessor, recognised, fused = folded + return predecessor, recognised.source, fused + + +# --------------------------------------------------------------------------- +# Convolutions: nn.Conv2d or F.conv2d, with the zero padding directly ahead of them +# --------------------------------------------------------------------------- +@dataclass(frozen=True) +class Convolution: + """A recognised convolution call. + + ``source``: the tensor it reads (the input of an absorbed padding node or Identity); + ``module``: the ``nn.Conv2d`` the blocks run, the model's own module where today's form + applies, else a stand-in holding the call's own weight and bias Parameters with padding + ``kernel // 2``; ``phase``: ``canonical_phase`` of ``(phase_y, phase_x)``; + ``padding``: the total zero padding ``(left, right, top, bottom)``, the padding node's + plus the convolution's own; ``path``: the module path of its weight (state-dict names).""" + + source: Node + module: nn.Conv2d + phase: Any + padding: tuple[int, int, int, int] + path: str + + +_OTHER_PADDING = tuple(getattr(nn, name) for name in ( + "ReflectionPad2d", "ReplicationPad2d", "CircularPad2d") if hasattr(nn, name)) +_PADDING_MESSAGE = ("Zero padding is supported only directly ahead of the one convolution that reads it " + "(nn.Conv2d or F.conv2d), as that convolution's padding") +_CONV_ATTRIBUTE_MESSAGE = ("Functional conv2d weight and bias must be get_attr parameters or buffers used only " + "as this call's operands") + + +def _is_convolution(node: Node, graph: GraphModule) -> bool: + if node.op == "call_module": + return isinstance(graph.get_submodule(str(node.target)), nn.Conv2d) + return node.op == "call_function" and node.target is torch.conv2d + + +def _convolution_operands(node: Node) -> dict[str, Any]: + if node.op == "call_module": + return _bind_options(_input_arguments, node.args, node.kwargs) + return _bind_options(_conv2d_arguments, node.args, node.kwargs) + + +def _read_by_one_convolution(node: Node, graph: GraphModule, *, functional: bool = False) -> bool: + """Whether ``node``'s only use is as the input operand of one convolution call (a + functional one only, with ``functional``). The padding node and the convolution both + ask this, so their verdicts cannot disagree.""" + if len(node.users) != 1: + return False + user = next(iter(node.users)) + if not _is_convolution(user, graph) or (functional and user.op != "call_function"): + return False + try: + operands = _convolution_operands(user) + except TypeError: + return False + return operands["input"] is node and not any( + node in _nodes_in(value) for key, value in operands.items() if key != "input") + + +def _static_padding(values, spelling: str) -> tuple[int, int, int, int]: + if not isinstance(values, (tuple, list)) or any(isinstance(x, Node) for x in values): + _reject(f"{spelling} padding must be static integers; padding computed from a tensor's shape " + f"is unsupported (received {values!r})") + if len(values) != 4: + _reject(f"{spelling} must pad the two spatial axes with four integers (left, right, top, bottom); " + f"received {values!r}") + if any(isinstance(x, bool) or not isinstance(x, Integral) for x in values): + _reject(f"{spelling} padding must be static integers; received {values!r}") + if any(x < 0 for x in values): + _reject(f"{spelling} with negative padding (cropping) is unsupported; received {tuple(values)!r}") + return tuple(int(x) for x in values) + + +def zero_padding(node: Node, graph: GraphModule) -> tuple[Node, tuple[int, int, int, int]] | None: + """A zero-padding node's input and ``(left, right, top, bottom)``; None for any other node. + + Spellings: ``nn.ZeroPad2d``, ``nn.ConstantPad2d`` with value 0, and ``F.pad`` with a + static four-integer pad, mode ``"constant"`` and value 0 or None. Refused: other values + and modes, reflection, replication and circular padding, negative pads, pads computed + from a tensor's shape, ``F.pad`` tuples of another length, and a padding node with any + other use than as the input of the one convolution directly after it.""" + if node.op == "call_module": + module = graph.get_submodule(str(node.target)) + if isinstance(module, _OTHER_PADDING): + _reject(f"{type(module).__name__} is unsupported: only zero padding directly ahead of a convolution", + "unsupported_operator") + if not isinstance(module, nn.ConstantPad2d): + return None + spelling, value = type(module).__name__, module.value + source = _bind_options(_input_arguments, node.args, node.kwargs)["input"] + padding = _static_padding(module.padding, spelling) + elif node.op == "call_function" and node.target is F.pad: + spelling = "F.pad" + options = _bind_options(_pad_arguments, node.args, node.kwargs) + if options["mode"] != "constant": + _reject(f"F.pad mode {options['mode']!r} is unsupported: only zero ('constant') padding " + "directly ahead of a convolution") + value = 0 if options["value"] is None else options["value"] + source = options["input"] + padding = _static_padding(options["pad"], spelling) + else: + return None + if (isinstance(value, bool) or not isinstance(value, Real) or value != 0 + or math.copysign(1.0, value) < 0): + _reject(f"{spelling} value must be 0 (zero padding); received {value!r}") + if not _read_by_one_convolution(node, graph): + _reject(f"{_PADDING_MESSAGE}; this {spelling} has another use") _inputs((source,)) - return predecessor, source, fused + return source, padding + + +def _absorbed_identity(node: Node, graph: GraphModule) -> bool: + """An ``nn.Identity`` whose only use is as the input of one functional convolution. + Elsewhere, and ahead of an ``nn.Conv2d``, an Identity keeps its own node.""" + return (node.op == "call_module" and isinstance(graph.get_submodule(str(node.target)), nn.Identity) + and _read_by_one_convolution(node, graph, functional=True)) + + +def _conv_attribute(node: Any, user: Node, graph: GraphModule): + """A functional convolution's weight or bias: a get_attr parameter or buffer used only + as that call's weight or bias operand.""" + message = _CONV_ATTRIBUTE_MESSAGE + if not isinstance(node, Node) or node.op != "get_attr" or set(node.users) != {user}: + _reject(message) + operands = _bind_options(_conv2d_arguments, user.args, user.kwargs) + if any(node in _nodes_in(value) for key, value in operands.items() if key not in ("weight", "bias")): + _reject(message) + try: + return graph.get_parameter(str(node.target)) + except AttributeError: + try: + return graph.get_buffer(str(node.target)) + except AttributeError: + _reject(message) + + +def conv_constant(node: Node, graph: GraphModule) -> bool: + """Allow get_attr only as the weight or bias of the one functional convolution using it.""" + if node.op != "get_attr" or len(node.users) != 1: + return False + user = next(iter(node.users)) + if user.op != "call_function" or user.target is not torch.conv2d: + return False + try: + operands = _bind_options(_conv2d_arguments, user.args, user.kwargs) + if operands["weight"] is not node and operands["bias"] is not node: + return False + _conv_attribute(node, user, graph) + except (TypeError, UnsupportedOperation): + return False + return True + + +def _static_pair(value, name: str) -> tuple[int, int]: + pair = tuple(value) if isinstance(value, (tuple, list)) else (value, value) + if len(pair) != 2 or any(isinstance(x, bool) or not isinstance(x, Integral) for x in pair): + _reject(f"Functional conv2d {name} must be a static integer or pair of integers; received {value!r}") + return int(pair[0]), int(pair[1]) + + +def _stand_in(weight, bias, *, stride, dilation, groups, padding_mode="zeros") -> nn.Conv2d: + """An ``nn.Conv2d`` with padding ``kernel // 2`` holding these weight and bias tensors + themselves (no copy, no initialization and no RNG draw), for the blocks' unchanged checks.""" + if not isinstance(weight, torch.Tensor) or weight.dim() != 4: + _reject("Conv2d weight must be a four-dimensional [out, in / groups, kh, kw] tensor", "coefficient_shape") + out, per_group, kh, kw = (int(x) for x in weight.shape) + try: + conv = torch.nn.utils.skip_init( + nn.Conv2d, per_group * groups, out, (kh, kw), stride=stride, padding=(kh // 2, kw // 2), + dilation=dilation, groups=groups, bias=bias is not None, padding_mode=padding_mode) + except (TypeError, ValueError, RuntimeError) as error: + _reject(f"Unsupported Conv2d settings: {error}") + conv.weight = weight if isinstance(weight, nn.Parameter) else nn.Parameter(weight, requires_grad=False) + if bias is not None: + conv.bias = bias if isinstance(bias, nn.Parameter) else nn.Parameter(bias, requires_grad=False) + return conv.eval() + + +def convolution(node: Node, graph: GraphModule) -> Convolution | None: + """The convolution a node calls, with the zero padding directly ahead of it; None for a + node that calls no convolution. + + An ``nn.Conv2d`` with padding ``kernel // 2`` (or ``"same"``) and no padding node ahead + is today's form: the module itself, its own checks and messages, phase 0. Otherwise, on + each axis, the total zero padding before the input (the padding node's plus the + convolution's own) must be ``kernel // 2 - phase`` with ``0 <= phase < stride``; the + output count is checked against PyTorch's at the proven input size + (``infer_metadata``). The blocks then run the centred convolution sampled at that phase. + """ + if not _is_convolution(node, graph): + return None + operands = _convolution_operands(node) + source = operands["input"] + if node.op == "call_module": + module = graph.get_submodule(str(node.target)) + path, own = str(node.target), module.padding + kh, kw = _pair(module.kernel_size) + padded = _quiet_zero_padding(source, graph) + if padded is None and (own == "same" or own == (kh // 2, kw // 2)): + (conv_options if module.groups == 1 else grouped_conv_options)(module) + return Convolution(source, module, 0, (kw // 2, kw // 2, kh // 2, kh // 2), path) + stand_in = _stand_in(module.weight, module.bias, stride=module.stride, dilation=module.dilation, + groups=module.groups, padding_mode=module.padding_mode) + else: + weight = _conv_attribute(operands["weight"], node, graph) + bias = None if operands["bias"] is None else _conv_attribute(operands["bias"], node, graph) + path = str(operands["weight"].target).rpartition(".")[0] or str(operands["weight"].target) + own = operands["padding"] + if own not in ("same", "valid"): + own = _static_pair(own, "padding") + groups = operands["groups"] + if isinstance(groups, bool) or not isinstance(groups, Integral) or groups < 1: + _reject(f"Functional conv2d groups must be a static positive integer; received {groups!r}") + stand_in = _stand_in(weight, bias, stride=_static_pair(operands["stride"], "stride"), + dilation=_static_pair(operands["dilation"], "dilation"), groups=int(groups)) + kh, kw = _pair(stand_in.kernel_size) + padded = _quiet_zero_padding(source, graph) + if padded is None and isinstance(source, Node) and _absorbed_identity(source, graph): + source = _bind_options(_input_arguments, source.args, source.kwargs)["input"] + # The blocks' unchanged checks and messages: odd kernel, stride 1 or 2, dilation 1, + # groups, zero padding mode, float32 and finite coefficients. + _, stride = (conv_options if stand_in.groups == 1 else grouped_conv_options)(stand_in) + own = (kh // 2, kw // 2) if own == "same" else (0, 0) if own == "valid" else _pair(own) + if any(isinstance(x, bool) or not isinstance(x, Integral) or x < 0 for x in own): + _reject(f"Conv2d padding must be non-negative integers; received {own!r}") + if padded is not None: + source, (left, right, top, bottom) = padded + else: + left = right = top = bottom = 0 + left, right, top, bottom = left + own[1], right + own[1], top + own[0], bottom + own[0] + phase = (kh // 2 - top, kw // 2 - left) + if not all(0 <= p < stride for p in phase): + _reject(f"Conv2d must use same padding (kernel_size // 2), or zero padding that starts " + f"kernel_size // 2 - phase pixels before the input on each axis with 0 <= phase < stride: " + f"padding (left {left}, right {right}, top {top}, bottom {bottom}), the convolution's own plus " + f"any zero padding directly ahead, gives phase {phase} at kernel {(kh, kw)} and stride {stride}") + _inputs((source,)) + return Convolution(source, stand_in, canonical_phase(phase, stride), (left, right, top, bottom), path) + + +def _quiet_zero_padding(source: Any, graph: GraphModule): + """``zero_padding`` for a convolution's input; a refused padding node is not absorbed + (the padding node itself reports why).""" + if not isinstance(source, Node): + return None + try: + return zero_padding(source, graph) + except (UnsupportedOperation, TypeError): + return None _BATCHNORM_CONSTANTS = ("running_mean", "running_var", "weight", "bias") @@ -468,11 +766,9 @@ def semantic_call(node: Node, graph: GraphModule) -> SemanticCall: source = _bind_options(_input_arguments, node.args, node.kwargs)["input"] inputs = _inputs((source,)) if isinstance(module, nn.Conv2d): - if module.groups != 1: - grouped_conv_options(module) - return SemanticCall("grouped_conv", inputs, module=module) - conv_options(module) - return SemanticCall("conv", inputs, module=module) + return _convolution_call(node, graph) + if zero_padding(node, graph) is not None: + return SemanticCall("absorbed", ()) if isinstance(module, (nn.AvgPool2d, nn.MaxPool2d)): return SemanticCall("pool", inputs, {"config": pool_config(module)}, module) if isinstance(module, nn.Upsample): @@ -487,12 +783,16 @@ def semantic_call(node: Node, graph: GraphModule) -> SemanticCall: if isinstance(module, nn.Sigmoid): return SemanticCall("sigmoid", inputs) if isinstance(module, nn.Identity): + if _absorbed_identity(node, graph): + return SemanticCall("absorbed", ()) return SemanticCall("identity", inputs) if isinstance(module, nn.BatchNorm2d): - folded = batchnorm_predecessor(node, graph) + folded = _batchnorm_fold(node, graph) if folded is not None: - predecessor, source, fused = folded - return SemanticCall("batchnorm", inputs, {"source": source, "predecessor": predecessor}, fused) + predecessor, recognised, fused = folded + return SemanticCall("batchnorm", inputs, { + "source": recognised.source, "predecessor": predecessor, "phase": recognised.phase, + "padding": recognised.padding, "path": recognised.path}, fused) return SemanticCall("batch_norm", inputs, _module_batchnorm_options(module)) if isinstance(module, nn.AdaptiveAvgPool2d): global_pool_size(module.output_size) @@ -500,6 +800,11 @@ def semantic_call(node: Node, graph: GraphModule) -> SemanticCall: _reject(f"Unsupported module {type(module).__name__}", "unsupported_operator") if node.op == "call_function": target = node.target + if target is torch.conv2d: + return _convolution_call(node, graph) + if target is F.pad: + zero_padding(node, graph) + return SemanticCall("absorbed", ()) if target is F.batch_norm: options = _bind_options(F.batch_norm, node.args, node.kwargs) if options["training"] is not False: @@ -568,11 +873,24 @@ def semantic_call(node: Node, graph: GraphModule) -> SemanticCall: return SemanticCall(str(node.target), _inputs(values)) if batchnorm_constant(node, graph): return SemanticCall("batchnorm_constant", ()) + if conv_constant(node, graph): + return SemanticCall("conv_constant", ()) + if node.op == "get_attr" and any(user.op == "call_function" and user.target is torch.conv2d + for user in node.users): + _reject(_CONV_ATTRIBUTE_MESSAGE) _reject(f"Unsupported FX operation {node.op}: {node.target}", "unsupported_operator") except TypeError as error: _reject(f"Invalid call arguments: {error}") +def _convolution_call(node: Node, graph: GraphModule) -> SemanticCall: + recognised = convolution(node, graph) + kind = "conv" if recognised.module.groups == 1 else "grouped_conv" + return SemanticCall(kind, _inputs((recognised.source,)), + {"phase": recognised.phase, "padding": recognised.padding, "path": recognised.path}, + recognised.module) + + def infer_metadata(call: SemanticCall, inputs: tuple[TensorMetadata, ...]) -> TensorMetadata: first = inputs[0] n, c, h, w = first.shape @@ -582,8 +900,21 @@ def infer_metadata(call: SemanticCall, inputs: tuple[TensorMetadata, ...]) -> Te if c != call.module.in_channels: _reject("Conv2d input channel count does not match raster", "input_shape") stride = _pair(call.module.stride)[0] - return TensorMetadata((n, call.module.out_channels, (h + stride - 1) // stride, - (w + stride - 1) // stride), first.scale * stride) + phase_y, phase_x = phase_pair(call.options.get("phase", 0), stride) + height, width = (h - phase_y + stride - 1) // stride, (w - phase_x + stride - 1) // stride + if "padding" in call.options: + # PyTorch's output count must be ceil((n - phase) / stride) on each axis: a + # padding that drops or adds an output is refused. + left, right, top, bottom = call.options["padding"] + kh, kw = _pair(call.module.kernel_size) + torch_height = (h + top + bottom - kh) // stride + 1 + torch_width = (w + left + right - kw) // stride + 1 + if (torch_height, torch_width) != (height, width) or height < 1 or width < 1: + _reject(f"Zero padding (left {left}, right {right}, top {top}, bottom {bottom}) on a {h} x {w} " + f"input gives PyTorch {torch_height} x {torch_width} outputs; the convolution sampled at " + f"phase ({phase_y}, {phase_x}) and stride {stride} gives ceil((n - phase) / stride), " + f"{height} x {width}", "input_shape") + return TensorMetadata((n, call.module.out_channels, height, width), first.scale * stride) if call.kind == "pool": config = call.options["config"] if config.aligned and (h % config.kernel or w % config.kernel): diff --git a/tensor_ee/weight_assets.py b/tensor_ee/weight_assets.py index b68375c..6bee0e7 100644 --- a/tensor_ee/weight_assets.py +++ b/tensor_ee/weight_assets.py @@ -54,10 +54,12 @@ def effective_convolutions(model: nn.Module) -> list[ConvolutionWeights]: call = semantic_call(node, traced) except UnsupportedOperation as error: raise UnsupportedOperation(f"At FX node {node.name} ({node.target}): {error}") from error + # The source names the weight's module path (a state-dict prefix): the module + # for nn.Conv2d, the weight parameter's owner for a functional convolution. if call.kind in ("conv", "grouped_conv") and not diagnostic_convolution(node, traced): - result.append(convolution_weights(node.name, call.module, str(node.target))) + result.append(convolution_weights(node.name, call.module, call.options["path"])) elif call.kind == "batchnorm": - source = str(call.options["predecessor"].target) + "+" + str(node.target) + source = call.options["path"] + "+" + str(node.target) result.append(convolution_weights(node.name, call.module, source)) if not result: raise ValueError("Model contains no deployable convolutions") diff --git a/tests/reference/raster_conv.py b/tests/reference/raster_conv.py index 87af9d9..5f862a0 100644 --- a/tests/reference/raster_conv.py +++ b/tests/reference/raster_conv.py @@ -3,24 +3,32 @@ Inputs/outputs are HWC; weights keep PyTorch's [out, in_per_group, kh, kw] order. Reference-only loops apply each group's actual channels without zero-filled block-diagonal weights. Tap accumulation follows channel, row, column order. + +``padding`` is PyTorch's zero padding ``(left, right, top, bottom)``; the default is the +symmetric ``kernel // 2`` on each axis. The output has PyTorch's count on each axis, +``(n + before + after - k) // stride + 1``, which is ``ceil(n / stride)`` at the default. """ import numpy as np -def grouped_conv2d(x, weight, bias=None, *, groups, stride=1): +def grouped_conv2d(x, weight, bias=None, *, groups, stride=1, padding=None): x = np.asarray(x, np.float32).astype(np.float64) weight = np.asarray(weight, np.float32).astype(np.float64) out, inputs, kh, kw = weight.shape h, w, _ = x.shape - padded = np.pad(x, ((kh // 2, kh // 2), (kw // 2, kw // 2), (0, 0))) - result = np.zeros(((h + stride - 1) // stride, (w + stride - 1) // stride, out), np.float64) + left, right, top, bottom = (kw // 2, kw // 2, kh // 2, kh // 2) if padding is None else padding + padded = np.pad(x, ((top, bottom), (left, right), (0, 0))) + rows = (h + top + bottom - kh) // stride + 1 + columns = (w + left + right - kw) // stride + 1 + result = np.zeros((rows, columns, out), np.float64) for group in range(groups): outputs = slice(group * (out // groups), (group + 1) * (out // groups)) for c in range(inputs): for ky in range(kh): for kx in range(kw): - values = padded[ky:ky + h:stride, kx:kx + w:stride, group * inputs + c] + values = padded[ky:ky + stride * (rows - 1) + 1:stride, + kx:kx + stride * (columns - 1) + 1:stride, group * inputs + c] result[..., outputs] += values[..., None] * weight[outputs, c, ky, kx] if bias is not None: result += np.asarray(bias, np.float32).astype(np.float64) diff --git a/tests/test_live_fixtures.py b/tests/test_live_fixtures.py index cda2cfc..08e400e 100644 --- a/tests/test_live_fixtures.py +++ b/tests/test_live_fixtures.py @@ -903,6 +903,59 @@ def test_global_pool_multiply_fixture_ids_and_data_are_stable(): assert all(np.count_nonzero(value == 0) == 0 for value in data.inputs.values()), fixture.id +def test_padded_conv_fixture_ids_data_and_graphs_are_stable(): + """Six distinct graphs. The phase-0 5 x 5 fixture is today's graph of the same + nn.Conv2d(padding=2), by recognition; each phase-1 fixture differs from its + nn.Conv2d(padding=kernel // 2) graph.""" + import tensor_ee + from tools.fixtures.core import inline_raster + from tools.fixtures.raster import _PADDED, _padded_cost_module, _rect_grid + + expected = { + "raster/conv2d/pad0101-k3-s2-h8-w10": ( + "d0180c8411c71d4c28486837accfe4bf9ab9652a74071e88b3384d75b022bec6", + "9baaa16c75f7ede1f533f709f804f25b1ff0a8b3f38db5ab4bbb0c6d11f42b64", 1), + "raster/conv2d/pad0101-k3-s2-h7-w9": ( + "45ff7845069d5cbbe766ef26d188dee198ba789c4d7ff22742e2d4592b5b50d3", + "9d9afa7f7b72e58fecdae1a41aa2e21bdebe97be687a787bb617d244b7788f45", 1), + "raster/grouped_conv2d/depthwise-pad0101-k3-s2": ( + "7191642021ea6b2ad313f4a97f5fd0c13fdcbcd87944eaed9240560cc830a7a1", + "d816913c5045aa7daa44db6a395ee99b316b5e1e397f2bbb77931a4fb4f290b2", 1), + "raster/grouped_conv2d/depthwise-pad1212-k5-s2": ( + "60a9febe42ac1b26072859284bebf7b3b59f6960b71af6583ff129caa45871a5", + "79f4b91b98c46eb9840e14ce3da557db83b0069638b7e4b3acc2d9062a819991", 1), + "raster/grouped_conv2d/depthwise-pad2222-k5-s2": ( + "f44951870074a5da1f80cf9259a1cd1b3127f2f64237c3760c535cea1be68ff3", + "4c5d36839c919d0522d7fe32346fdabf2d4011756ecacbc700bc28ee3dc59baf", 0), + "raster/grouped_conv2d/depthwise-pad1101-k3-s2": ( + "a594aa74b13a6c6e528d301ab83cca568ebb937a27a38247d77048cf4d4feb81", + "df6fa2afbafaea008d49f1fe9d9a223d0ec3b868131078e07d3041e3b9189d3e", (1, 0)), + } + fixtures = select(["padded_conv"]) + assert [fixture.id for fixture in fixtures] == list(expected) + specs = {f"raster/{spec.block}/{spec.name}": spec for spec in _PADDED} + case = ApiTestCase() + case.setUp() + try: + graphs = set() + for fixture in fixtures: + data = fixture.data(fixture.seed) + assert (data.inputs_sha256, data.weights_sha256, fixture.settings["phase"]) == expected[fixture.id] + assert fixture.requests == 2 and (fixture.mirror is None) == (fixture.block == "conv2d") + out = fixture.build(ee, data)[0] + graphs.add(out.image.serialize()) + spec = specs[fixture.id] + grid = _rect_grid(10, *spec.shape) + bands = ("b0", "b1", "b2", "b3")[:spec.inputs] + plain = tensor_ee.compile_model(_padded_cost_module(spec, data), + inline_raster(ee, grid.projection(), data.inputs["x"], bands), bands, grid) + assert (out.image.serialize() == plain.image.serialize()) == (fixture.settings["phase"] == 0), fixture.id + assert list(out.shape[:2]) == fixture.shapes["output"][:2] + assert len(graphs) == len(fixtures) + finally: + case.tearDown() + + def test_only_the_128_global_pool_fixture_raises_the_raster_grid(): """The maintainer's allowance is one fixture's own request-grid side (128); the budget's default stays 64 for every other fixture, and a lowered --budget applies to that one too.""" diff --git a/tests/test_raster_blocks.py b/tests/test_raster_blocks.py index eac46d7..01d7ed3 100644 --- a/tests/test_raster_blocks.py +++ b/tests/test_raster_blocks.py @@ -256,10 +256,17 @@ def _parameters(function): return [(p.name, p.kind, p.default) for p in inspect.signature(function).parameters.values()] +ADDED_AFTER_POLICY = { # keyword-only settings added later, after policy=, each defaulting to today's graph + "conv2d": [("phase", KEYWORD, 0)], + "grouped_conv2d": [("phase", KEYWORD, 0)], +} + + @pytest.mark.parametrize("name", sorted(BLOCKS)) def test_block_signature_adds_only_a_keyword_only_policy(name): block = getattr(BLOCK_MODULES[BLOCKS[name]], name) - assert _parameters(block) == BASE_PARAMETERS[name] + [("policy", KEYWORD, POLICIES.get(name, P.FLOAT32_OPERANDS))] + assert _parameters(block) == (BASE_PARAMETERS[name] + [("policy", KEYWORD, POLICIES.get(name, P.FLOAT32_OPERANDS))] + + ADDED_AFTER_POLICY.get(name, [])) @pytest.mark.parametrize("name", FACADES) @@ -377,8 +384,8 @@ def test_compile_model_keeps_the_per_node_float32_cast(offline_ee): "Ready — maps 5 × 7, 8 × 8, 12 × 20, 64 × 64 and 128 × 128"), "multiply": (M.Locality.POINTWISE, None, "Ready — same grid on a 5 × 7 map; one-pixel gate on 5 × 7 and 12 × 20 maps"), - "grouped_conv2d": (M.Locality.KERNEL, 2, - "Ready — depthwise 3×3 stride 1 or 2, 5×5 stride 1; 3×3 stride 1 with 2 channels per group or channel multiplier 2"), + "grouped_conv2d": (M.Locality.KERNEL, 3, + "Ready — depthwise 3×3 stride 1 or 2, 5×5 stride 1 or 2; 3×3 stride 1 with 2 channels per group or channel multiplier 2"), "bilinear_upsample": (M.Locality.KERNEL, 1, "Ready — factors 2, 3, 4 (align_corners=False), 2, 4 (align_corners=True)"), "batch_norm": (M.Locality.POINTWISE, None, "Ready"), "standardize": (M.Locality.POINTWISE, None, "Ready — 6 bands with scale 1e-4; 3 bands without scale"), diff --git a/tests/test_raster_global_pool_multiply.py b/tests/test_raster_global_pool_multiply.py index 6de3e7b..1ea1ff4 100644 --- a/tests/test_raster_global_pool_multiply.py +++ b/tests/test_raster_global_pool_multiply.py @@ -349,10 +349,15 @@ def test_compatibility_reports_the_stand_ins_supported(offline_ee): def test_efficientnet_b3_unet_pools_and_products_are_no_longer_blocked(): - """Trace only, no forward: SMP 0.5.0's EfficientNet-B3 U-Net at 256 px keeps its other - blockers (functional conv2d with get_attr weights, ZeroPad2d, in-place ReLU, ...), but its 26 - squeeze-excitation pools and products are no longer among them.""" + """SMP 0.5.0's EfficientNet-B3 U-Net at 256 px. Its 26 squeeze-excitation pools and + products are supported. Traced as built, its only blockers are the decoder's 10 in-place + ReLUs (the functional convolutions and their zero padding are recognised, + tests/test_raster_padded_conv.py). Through ``adapt_smp_model``, which clears the copied + in-place flags and checks its trace against the model with one PyTorch forward, the + whole model is supported.""" smp = pytest.importorskip("segmentation_models_pytorch") + from tensor_ee.smp import adapt_smp_model + model = smp.Unet("efficientnet-b3", encoder_weights=None, in_channels=8, classes=3).eval() report = check_compatibility(model, (1, 8, 256, 256)).to_dict() nodes = {node["name"]: node for node in report["nodes"]} @@ -363,4 +368,9 @@ def test_efficientnet_b3_unet_pools_and_products_are_no_longer_blocked(): assert not blocked & {n["name"] for n in pools + products} assert all(n["status"] != "unsupported" for n in pools + products) assert not any("adaptive_avg_pool2d" in b["message"] or "mul" in b["message"].split() for b in report["blockers"]) - assert report["status"] == "unsupported" # the functional convolutions remain + assert report["status"] == "unsupported" + assert [b["message"] for b in report["blockers"]] == ["In-place ReLU is unsupported"] * 10 + adapted = adapt_smp_model(model, input_shape=(1, 8, 256, 256)) + report = check_compatibility(adapted, (1, 8, 256, 256)).to_dict() + assert report["status"] == "supported", report["blockers"] + report["unknowns"] + assert report["output_shape"] == [1, 3, 256, 256] diff --git a/tests/test_raster_grouped_conv.py b/tests/test_raster_grouped_conv.py index 7edf2b9..a1c129c 100644 --- a/tests/test_raster_grouped_conv.py +++ b/tests/test_raster_grouped_conv.py @@ -133,7 +133,7 @@ def test_grouped_block_and_legacy_block_keep_separate_acceptance_and_status(offl C.conv_options(module) assert C.METADATA["conv2d"].validated.configurations == ((),) assert C.METADATA["grouped_conv2d"].status.startswith("Ready — depthwise") - assert C.METADATA["grouped_conv2d"].radius == 2 + assert C.METADATA["grouped_conv2d"].radius == 3 # 5 x 5 at phase 1: k // 2 + 1 def inverted_residual(): @@ -273,7 +273,8 @@ def test_values_group_order_stride_and_parameters_match_mirror_and_pytorch(monke monkeypatch.setattr(K, "ee", fake) monkeypatch.setattr(C, "_zero_extended", lambda raster: raster.image) monkeypatch.setattr(C, "_project", lambda image, grid, clip: image) - def sample(image, bands, g, stride, clip): + def sample(image, bands, g, stride, clip, *, phase=(0, 0)): + assert phase == (0, 0) # the blocks pass the default sampling phase as a pair return Raster(Image(image.values[::stride, ::stride], bands), bands, g.resized(stride, (g.width + stride - 1) // stride, (g.height + stride - 1) // stride)) monkeypatch.setattr(C, "_stride_sample", sample) @@ -292,3 +293,43 @@ def sample(image, bands, g, stride, clip): with torch.no_grad(): torch_out = module(torch.from_numpy(x.transpose(2, 0, 1)[None])).numpy()[0].transpose(1, 2, 0) assert np.max(np.abs(a - torch_out)) <= 1e-4 + + +@pytest.mark.parametrize("kernel,padding", [(3, (0, 1, 0, 1)), (5, (1, 2, 1, 2)), (3, (1, 1, 0, 1)), + (3, (0, 1, 1, 1)), (5, (2, 2, 2, 2))]) +@pytest.mark.parametrize("height,width", [(8, 10), (7, 9)]) +def test_a_sampling_phase_matches_the_padded_mirror_and_pytorch(monkeypatch, kernel, padding, height, width): + """The block's own array graph, interpreted in NumPy, at the phase a zero padding gives + (kernel // 2 - left, kernel // 2 - top): equal to the mirror with that padding, and to + PyTorch's padded convolution within 1e-4. The sampler is modelled as taking input pixels + stride * i + phase, the rule ``stride_sample`` implements (tests/test_raster_padded_conv.py + checks its grids).""" + fake = SimpleNamespace(Array=Array, Image=Image, + Kernel=SimpleNamespace(fixed=lambda kw, kh, *args: (kh, kw)), + Reducer=SimpleNamespace(sum=lambda: "sum")) + monkeypatch.setattr(C, "ee", fake) + monkeypatch.setattr(K, "ee", fake) + monkeypatch.setattr(C, "_zero_extended", lambda raster: raster.image) + monkeypatch.setattr(C, "_project", lambda image, grid, clip: image) + def sample(image, bands, g, stride, clip, *, phase=(0, 0)): + py, px = phase + return Raster(Image(image.values[py::stride, px::stride], bands), bands, + g.resized(stride, (g.width - px + stride - 1) // stride, (g.height - py + stride - 1) // stride)) + monkeypatch.setattr(C, "_stride_sample", sample) + left, right, top, bottom = padding + phase = (kernel // 2 - top, kernel // 2 - left) + spec = _Grouped(f"depthwise-k{kernel}-s2", kernel=kernel, stride=2) + data = _grouped_data(spec)(61) + module = _grouped_module(spec, data) + x = ((np.arange(height * width * 4).reshape(height, width, 4) % 97 - 48) / 64).astype(np.float32) + raster = Raster(Image(x, tuple(f"z{i}" for i in range(4))), tuple(f"z{i}" for i in range(4)), + Grid("EPSG:3857", (20, 0, 1000, 0, -20, 2000), width, height)) + a = C.grouped_conv2d(raster, module, phase=phase).image.values.astype(np.float32) + expected = mirror(x, data.weights["weight"], data.weights["bias"], groups=4, stride=2, padding=padding) + np.testing.assert_array_equal(a, expected) + with torch.no_grad(): + torch_out = torch.nn.functional.conv2d( + torch.nn.functional.pad(torch.from_numpy(x.transpose(2, 0, 1)[None]), padding), + module.weight, module.bias, 2, 0, 1, 4).numpy()[0].transpose(1, 2, 0) + assert torch_out.shape == a.shape == ((height - phase[0] + 1) // 2, (width - phase[1] + 1) // 2, 4) + assert np.max(np.abs(a - torch_out)) <= 1e-4 diff --git a/tests/test_raster_padded_conv.py b/tests/test_raster_padded_conv.py new file mode 100644 index 0000000..83cfca7 --- /dev/null +++ b/tests/test_raster_padded_conv.py @@ -0,0 +1,663 @@ +"""Zero padding ahead of a convolution: the sampling phase of ``conv2d`` and +``grouped_conv2d``, and the compiler's recognition of the padded forms. Offline only.""" +from __future__ import annotations + +from dataclasses import replace +import json +import re + +import ee +from ee.apitestcase import ApiTestCase +import pytest +import torch +from torch import nn +from torch.fx import symbolic_trace +from torch.nn import functional as F + +from tensor_ee import Grid, Raster, UnsupportedOperation, check_compatibility, compile_model +from tensor_ee.blocks.raster import conv as C +from tensor_ee.layouts import raster as L +from tensor_ee.receptive_field import UnsupportedReceptiveField, analyze_receptive_field +from tensor_ee.semantics import batchnorm_predecessor, convolution, semantic_call +from tensor_ee.weight_assets import effective_convolutions +from tensor_ee.weight_mapping import MatchedWeightProvider + + +@pytest.fixture +def offline_ee(): + case = ApiTestCase() + case.setUp() + try: + yield + finally: + case.tearDown() + + +def _grid(width, height, scale=20): + return Grid("EPSG:3857", (scale, 0, 1000, 0, -scale, 2000), width, height) + + +def _base_stride_sample(image, bands, grid, stride, clip_footprints=False): + """``stride_sample`` as it stood before the phase setting, verbatim.""" + if stride == 1: + return L.finish(image, bands, grid, clip_footprints) + width = (grid.width + stride - 1) // stride + height = (grid.height + stride - 1) // stride + canonical = grid.resized(stride, width, height) + a, b, c, d, e, f = canonical.transform + sample = Grid(grid.crs, + (a, b, c - (stride - 1) * grid.transform[0] / 2, + d, e, f - (stride - 1) * grid.transform[4] / 2), + width, height) + sampled = L.finish(image, bands, sample, clip_footprints) + relabeled = sampled.image.changeProj(sample.projection(), canonical.projection()) + return L.finish(relabeled, bands, canonical, clip_footprints) + + +# --------------------------------------------------------------------------- +# The sampler: phase 0 is today's graph; a phase moves the sample grid exactly +# --------------------------------------------------------------------------- +@pytest.mark.parametrize("stride", [1, 2]) +@pytest.mark.parametrize("size", [(8, 8), (10, 8), (7, 5), (9, 9)]) +@pytest.mark.parametrize("clip", [False, True]) +@pytest.mark.parametrize("phase", [0, (0, 0), [0, 0]]) +def test_the_sampler_at_phase_zero_is_todays_graph(offline_ee, stride, size, clip, phase): + grid = _grid(*size) + image = ee.Image.constant([0.5, -1.0]).rename(["c0", "c1"]) + base = _base_stride_sample(image, ("c0", "c1"), grid, stride, clip) + default = L.stride_sample(image, ("c0", "c1"), grid, stride, clip) + stated = L.stride_sample(image, ("c0", "c1"), grid, stride, clip, phase=phase) + assert base.image.serialize() == default.image.serialize() == stated.image.serialize() + assert base.grid == default.grid == stated.grid and base.bands == stated.bands + + +@pytest.mark.parametrize("size", [(8, 10), (7, 9), (10, 8), (5, 7)]) +@pytest.mark.parametrize("phase", [(0, 0), (1, 1), (0, 1), (1, 0)]) +def test_each_sample_pixel_is_centred_on_its_input_pixel(size, phase): + """Pixel (i, j) of the sample grid is centred on input pixel (2 i + phase_y, + 2 j + phase_x), on a non-square grid with the y spacing negative; the canonical grid + keeps the input's corner and has ceil((n - phase) / 2) pixels on each axis.""" + width, height = size + grid = Grid("EPSG:3857", (20, 0, 1000, 0, -30, 2000), width, height) + sample, canonical = L.sample_grids(grid, 2, phase) + py, px = phase + a, _, c, _, e, f = grid.transform + sa, _, sc, _, se, sf = sample.transform + assert (sa, se) == (2 * a, 2 * e) + assert (canonical.width, canonical.height) == (-(-(width - px) // 2), -(-(height - py) // 2)) + assert (sample.width, sample.height) == (canonical.width, canonical.height) + assert canonical.transform == (2 * a, 0, c, 0, 2 * e, f) + for i in range(sample.height): + for j in range(sample.width): + centre = (sc + (j + 0.5) * sa, sf + (i + 0.5) * se) + assert centre == (c + (2 * j + px + 0.5) * a, f + (2 * i + py + 0.5) * e) + + +@pytest.mark.parametrize("phase", [1, (1, 0), (0, 1), (1, 1), [1, 1]]) +def test_a_phase_changes_only_the_sample_grid(offline_ee, phase): + grid = _grid(10, 8) + image = ee.Image.constant([0.5]).rename(["c0"]) + shifted = L.stride_sample(image, ("c0",), grid, 2, phase=phase) + assert shifted.image.serialize() != L.stride_sample(image, ("c0",), grid, 2).image.serialize() + py, px = L.phase_pair(phase, 2) + assert shifted.grid == grid.resized(2, (10 - px + 1) // 2, (8 - py + 1) // 2) + + +@pytest.mark.parametrize("bad", [True, 1.0, 2, -1, (0, 2), (0,), (0, 0, 0), "1", None, (True, 0)]) +def test_invalid_phases_are_refused(bad): + with pytest.raises(UnsupportedOperation, match="phase must be an integer or a pair"): + L.phase_pair(bad, 2) + + +def test_stride_one_accepts_phase_zero_only(): + assert L.phase_pair(0, 1) == (0, 0) + with pytest.raises(UnsupportedOperation, match="0 <= phase < stride 1"): + L.phase_pair(1, 1) + + +def test_a_phase_with_no_output_pixels_is_refused(): + with pytest.raises(UnsupportedOperation, match="no output pixels"): + L.sample_grids(_grid(1, 4), 2, (0, 1)) + + +def test_the_canonical_phase(): + assert [L.canonical_phase(p, 2) for p in (0, (0, 0), [0, 0], 1, (1, 1), (0, 1), [1, 0])] == [ + 0, 0, 0, 1, 1, (0, 1), (1, 0)] + + +# --------------------------------------------------------------------------- +# The blocks: the setting, its default and its refusals +# --------------------------------------------------------------------------- +def _dense(stride=2, kernel=3): + module = nn.Conv2d(2, 3, kernel, stride=stride, padding=kernel // 2).eval() + with torch.no_grad(): + module.weight.copy_(torch.arange(module.weight.numel(), dtype=torch.float32) + .reshape(module.weight.shape) / 64) + module.bias.copy_(torch.tensor([0.5, -0.25, 0.125])) + return module + + +def _depthwise(stride=2, kernel=3): + module = nn.Conv2d(4, 4, kernel, stride=stride, padding=kernel // 2, groups=4).eval() + with torch.no_grad(): + module.weight.copy_(torch.arange(module.weight.numel(), dtype=torch.float32) + .reshape(module.weight.shape) / 64) + module.bias.copy_(torch.tensor([0.5, -0.25, 0.125, 1.0])) + return module + + +def _raster(channels, grid): + bands = tuple(f"c{i}" for i in range(channels)) + return Raster(ee.Image.constant(list(range(1, channels + 1))).rename(list(bands)), bands, grid) + + +@pytest.mark.parametrize("block,module,channels", [ + (C.conv2d, _dense, 2), (C.grouped_conv2d, _depthwise, 4)]) +@pytest.mark.parametrize("stride", [1, 2]) +@pytest.mark.parametrize("clip", [False, True]) +def test_the_blocks_at_phase_zero_are_todays_graph(offline_ee, block, module, channels, stride, clip): + raster = _raster(channels, _grid(10, 8)) + conv = module(stride) + today = block(raster, conv, clip_footprints=clip) + for phase in (0, (0, 0), [0, 0]): + stated = block(raster, conv, clip_footprints=clip, phase=phase) + assert stated.image.serialize() == today.image.serialize() and stated.grid == today.grid + + +@pytest.mark.parametrize("block,module,channels", [ + (C.conv2d, _dense, 2), (C.grouped_conv2d, _depthwise, 4)]) +@pytest.mark.parametrize("phase", [1, (0, 1), (1, 0)]) +def test_the_blocks_differ_from_today_only_in_the_sampler(offline_ee, block, module, channels, phase): + """A phase reuses the block's convolution expression: only the sample grid changes.""" + raster = _raster(channels, _grid(10, 8)) + calls = [] + original = C._stride_sample + + def record(image, bands, grid, stride, clip_footprints=False, *, phase=0): + calls.append((image.serialize(), bands, grid, stride, clip_footprints, phase)) + return original(image, bands, grid, stride, clip_footprints, phase=phase) + + with pytest.MonkeyPatch.context() as patch: + patch.setattr(C, "_stride_sample", record) + today = block(raster, module(2)) + shifted = block(raster, module(2), phase=phase) + (image0, *rest0, phase0), (image1, *rest1, phase1) = calls + assert image0 == image1 and rest0 == rest1 + assert (phase0, phase1) == ((0, 0), L.phase_pair(phase, 2)) + py, px = L.phase_pair(phase, 2) + assert shifted.grid == _grid(10, 8).resized(2, (10 - px + 1) // 2, (8 - py + 1) // 2) + assert shifted.image.serialize() == L.stride_sample( + ee.deserializer.fromJSON(image1), rest1[0], rest1[1], 2, phase=phase).image.serialize() + assert today.image.serialize() != shifted.image.serialize() + + +@pytest.mark.parametrize("block,module,channels", [ + (C.conv2d, _dense, 2), (C.grouped_conv2d, _depthwise, 4)]) +def test_a_stride_one_phase_and_bad_values_are_refused(offline_ee, block, module, channels): + raster = _raster(channels, _grid(10, 8)) + for bad in (1, (0, 1), True, 0.0): + with pytest.raises(UnsupportedOperation, match="phase must be"): + block(raster, module(1), phase=bad) + for bad in (2, (0, 2), -1): + with pytest.raises(UnsupportedOperation, match="phase must be"): + block(raster, module(2), phase=bad) + + +# --------------------------------------------------------------------------- +# The records: the live run varied the phase, so neither record names it +# --------------------------------------------------------------------------- +@pytest.mark.parametrize("name", ["conv2d", "grouped_conv2d"]) +def test_the_records_do_not_name_the_phase_after_the_live_run(name): + """A setting enters a record at its default until a live run varies it. The live run + ran phases 0, 1 and (1, 0), so ``phase`` left both records and a call at any phase is + inside them, in either spelling.""" + from tensor_ee.core.metadata import check_settings, describe_validated + + entry = C.METADATA[name] + base = {} if name == "conv2d" else {"clip_footprints": False} + assert describe_validated(entry.validated) == ("any accepted call" if name == "conv2d" + else "{clip_footprints False}") + assert "phase" not in entry.validated.names + for spelling in (0, (0, 0), 1, (1, 1), [1, 1], (0, 1), (1, 0), [1, 0]): + assert check_settings(entry, {**base, "phase": L.canonical_phase(spelling, 2)}).inside + assert check_settings(entry, base).inside + assert entry.radius == 3 # conv2d: kernel 7; grouped: 5 x 5 at phase 1 + assert sum(path.endswith("padded_conv.json") or path.endswith("padded-conv-offline.json") + for path in entry.evidence) == 2 + + +# --------------------------------------------------------------------------- +# Recognition: zero padding in its spellings, directly ahead of a convolution in either spelling +# --------------------------------------------------------------------------- +def _values(shape, salt=0): + """Integer-generated dyadic values (no RNG).""" + count = 1 + for n in shape: + count *= n + return ((torch.arange(count, dtype=torch.int64) * 7 + salt) % 23 - 11).to(torch.float32).reshape(shape) / 32 + + +def _batch_norm(channels): + bn = nn.BatchNorm2d(channels) + with torch.no_grad(): + bn.running_mean.copy_(_values((channels,), 3)) + bn.running_var.copy_(_values((channels,), 5).abs() + 0.5) + bn.weight.copy_(_values((channels,), 7) + 1) + bn.bias.copy_(_values((channels,), 9)) + return bn.eval() + + +class PaddedConv(nn.Module): + """A padding node in one spelling, then a convolution in one spelling (the stand-in for + a convolution with static "same" padding written as a padding module and ``F.conv2d`` + with the module's own weight and bias), optionally followed by a BatchNorm.""" + + def __init__(self, cin, cout, kernel, *, stride=1, groups=1, bias=True, pad=(1, 1, 1, 1), + spelling="ZeroPad2d", call="F.conv2d", own=0, bn=False): + super().__init__() + self.spelling, self.call, self.stride, self.groups, self.own = spelling, call, stride, groups, own + self.pad_values = tuple(pad) if pad is not None else None + if spelling == "ZeroPad2d": + self.pad = nn.ZeroPad2d(pad) + elif spelling == "ConstantPad2d": + self.pad = nn.ConstantPad2d(pad, 0.0) + elif spelling == "Identity": + self.pad = nn.Identity() + weight = _values((cout, cin // groups, kernel, kernel), 1) + bias_value = _values((cout,), 2) if bias else None + if call == "nn.Conv2d": + self.conv = nn.Conv2d(cin, cout, kernel, stride, own, groups=groups, bias=bias) + with torch.no_grad(): + self.conv.weight.copy_(weight) + if bias: + self.conv.bias.copy_(bias_value) + else: + self.weight = nn.Parameter(weight) + self.bias = nn.Parameter(bias_value) if bias else None + self.bn = _batch_norm(cout) if bn else None + + def forward(self, x): + if self.spelling == "F.pad": + x = F.pad(x, self.pad_values) + elif self.spelling == "F.pad-value": + x = F.pad(x, self.pad_values, "constant", 0.0) + elif self.spelling in ("ZeroPad2d", "ConstantPad2d", "Identity"): + x = self.pad(x) + if self.call == "nn.Conv2d": + x = self.conv(x) + elif self.call == "F.conv2d": + x = F.conv2d(x, self.weight, self.bias, self.stride, self.own, 1, self.groups) + else: + x = torch.conv2d(x, self.weight, self.bias, stride=self.stride, padding=self.own, groups=self.groups) + return x if self.bn is None else self.bn(x) + + +def _reference(cin, cout, kernel, *, stride=1, groups=1, bias=True, bn=False): + """The same convolution as an ``nn.Conv2d`` with padding kernel // 2.""" + conv = nn.Conv2d(cin, cout, kernel, stride, kernel // 2, groups=groups, bias=bias) + with torch.no_grad(): + conv.weight.copy_(_values((cout, cin // groups, kernel, kernel), 1)) + if bias: + conv.bias.copy_(_values((cout,), 2)) + return nn.Sequential(conv, *([_batch_norm(cout)] if bn else [])).eval() + + +def _compile(model, channels, width=10, height=8, **kwargs): + bands = tuple(f"b{i}" for i in range(channels)) + image = ee.Image.constant(list(range(1, channels + 1))).rename(list(bands)) + return compile_model(model.eval(), image, bands, _grid(width, height), **kwargs) + + +SYMMETRIC_SPELLINGS = ( # (padding spelling, convolution spelling, own padding): total kernel // 2 + ("ZeroPad2d", "F.conv2d", 0), ("ConstantPad2d", "F.conv2d", 0), ("F.pad", "torch.conv2d", 0), + ("F.pad-value", "F.conv2d", 0), ("ZeroPad2d", "nn.Conv2d", 0), ("Identity", "F.conv2d", None), + ("none", "F.conv2d", None), ("none", "torch.conv2d", None)) + + +@pytest.mark.parametrize("groups", [1, 4], ids=["dense", "depthwise"]) +@pytest.mark.parametrize("kernel", [1, 3, 5]) +@pytest.mark.parametrize("stride", [1, 2]) +@pytest.mark.parametrize("bn", [False, True], ids=["conv", "conv-bn"]) +@pytest.mark.parametrize("spelling,call,own", SYMMETRIC_SPELLINGS, ids=lambda v: str(v)) +def test_symmetric_padding_compiles_to_the_nn_conv2d_graph(offline_ee, groups, kernel, stride, bn, spelling, call, own): + """Zero padding of kernel // 2 ahead of a convolution, in any spelling, with the + convolution as F.conv2d (get_attr weights) or as an nn.Conv2d behind the pad, compiles + byte for byte to today's graph of nn.Conv2d(padding=kernel // 2), with and without a + folded BatchNorm.""" + p = kernel // 2 + model = PaddedConv(4, 4, kernel, stride=stride, groups=groups, bias=not bn, pad=(p, p, p, p), spelling=spelling, + call=call, own=p if own is None else own, bn=bn) + reference = _reference(4, 4, kernel, stride=stride, groups=groups, bias=not bn, bn=bn) + for unfused in (True, False): + a = _compile(model, 4, include_unfused_nodes=unfused) + b = _compile(reference, 4, include_unfused_nodes=unfused) + assert a.image.serialize() == b.image.serialize() and a.grid == b.grid + assert len(a.folded_batchnorms) == len(b.folded_batchnorms) == int(bn) + + +def _block_graph(module, phase, channels=4, width=10, height=8): + """The compiled graph of ``module``'s block at ``phase``, as compile_model emits it: the + input materialized once, the block, and its float32 materialization.""" + from tensor_ee.compiler import _finish + + bands = tuple(f"b{i}" for i in range(channels)) + image = ee.Image.constant(list(range(1, channels + 1))).rename(list(bands)) + grid = _grid(width, height) + raster = _finish(image.select(list(bands)).unmask(0, sameFootprint=False), C._names(channels), grid) + raster = Raster(raster.image.toFloat(), raster.bands, raster.grid) + block = C.conv2d if module.groups == 1 else C.grouped_conv2d + out = block(raster, module, phase=phase) + return out.image.toFloat().serialize(), out.grid + + +PHASED_SPELLINGS = ( # (phase, kernel, [(padding spelling, pad, convolution spelling, own padding), ...]) + (1, 3, [("ZeroPad2d", (0, 1, 0, 1), "F.conv2d", 0), ("F.pad", (0, 1, 0, 1), "nn.Conv2d", 0), + ("ConstantPad2d", (0, 1, 0, 1), "torch.conv2d", 0), ("F.pad", (0, 2, 0, 2), "F.conv2d", 0)]), + (1, 5, [("ZeroPad2d", (1, 2, 1, 2), "F.conv2d", 0), ("F.pad", (0, 1, 0, 1), "nn.Conv2d", 1)]), + ((1, 0), 3, [("F.pad", (1, 1, 0, 1), "F.conv2d", 0), ("ZeroPad2d", (0, 0, 0, 1), "F.conv2d", (0, 1)), + ("ZeroPad2d", (1, 1, 0, 1), "nn.Conv2d", 0)]), + ((0, 1), 3, [("F.pad", (0, 1, 1, 1), "torch.conv2d", 0), ("ZeroPad2d", (0, 1, 0, 0), "nn.Conv2d", (1, 0))]), +) + + +@pytest.mark.parametrize("groups", [1, 4], ids=["dense", "depthwise"]) +@pytest.mark.parametrize("phase,kernel,spellings", PHASED_SPELLINGS, ids=["p1-k3", "p1-k5", "p10-k3", "p01-k3"]) +@pytest.mark.parametrize("size", [(10, 8), (9, 7)], ids=["even", "odd"]) +def test_each_padded_spelling_is_the_block_at_its_phase(offline_ee, groups, phase, kernel, spellings, size): + """Spellings of one padded convolution compile to one graph: the block at the phase the + padding gives (left and top pads kernel // 2 - phase), on even and odd inputs. A right or + bottom pad that gives PyTorch another output count at the size is refused instead.""" + width, height = size + expected, grid = _block_graph(_reference(4, 4, kernel, stride=2, groups=groups)[0], phase, + width=width, height=height) + compiled = 0 + for spelling, pad, call, own in spellings: + model = PaddedConv(4, 4, kernel, stride=2, groups=groups, pad=pad, spelling=spelling, call=call, own=own) + own_y, own_x = own if isinstance(own, tuple) else (own, own) + counts = ((height + pad[2] + pad[3] + 2 * own_y - kernel) // 2 + 1, + (width + pad[0] + pad[1] + 2 * own_x - kernel) // 2 + 1) + if counts != (grid.height, grid.width): + with pytest.raises(UnsupportedOperation, match="gives PyTorch"): + _compile(model, 4, width, height) + continue + out = _compile(model, 4, width, height) + assert out.image.serialize() == expected and out.grid == grid, (spelling, pad, call, own) + compiled += 1 + assert compiled >= 2 + + +def test_a_bare_strided_convolution_without_padding_follows_the_same_rule(offline_ee): + """``nn.Conv2d(3, stride=2, padding=0)`` reads input pixels 2i .. 2i + 2: phase 1. On an + odd input PyTorch's count is ceil((n - 1) / 2), so it compiles (the valid stride-2 + convolution of an odd map); on an even input it gives one output fewer and is refused.""" + model = PaddedConv(4, 4, 3, stride=2, pad=None, spelling="none", call="nn.Conv2d", own=0) + expected, grid = _block_graph(_reference(4, 4, 3, stride=2)[0], 1, width=9, height=7) + out = _compile(model, 4, 9, 7) + assert out.image.serialize() == expected and out.grid == grid == _grid(9, 7).resized(2, 4, 3) + with pytest.raises(UnsupportedOperation, match=r"gives PyTorch 3 x 4 outputs.*4 x 5"): + _compile(model, 4, 10, 8) + # A bare unpadded convolution at stride 1 shrinks the map: refused with today's words. + with pytest.raises(UnsupportedOperation, match=r"same padding \(kernel_size // 2\)"): + _compile(PaddedConv(4, 4, 3, pad=None, spelling="none", call="nn.Conv2d", own=0), 4) + + +def test_folding_reaches_through_a_recognised_convolution_with_its_phase(offline_ee): + model = PaddedConv(4, 6, 3, stride=2, bias=False, pad=(0, 1, 0, 1), bn=True) + reference = _reference(4, 6, 3, stride=2, bias=False, bn=True) + fused = torch.nn.utils.fuse_conv_bn_eval(reference[0], reference[1]) + expected, grid = _block_graph(fused, 1) + for unfused in (True, False): + out = _compile(model, 4, include_unfused_nodes=unfused) + assert out.image.serialize() == expected and out.grid == grid + assert out.folded_batchnorms == ("bn",) + assert ("conv2d" in out.nodes) is unfused + traced = symbolic_trace(model) + call = semantic_call(next(n for n in traced.graph.nodes if n.name == "bn"), traced) + assert call.kind == "batchnorm" and call.options["phase"] == 1 and call.options["padding"] == (0, 1, 0, 1) + assert call.options["source"].name == "x" + predecessor, source, _ = batchnorm_predecessor(next(n for n in traced.graph.nodes if n.name == "bn"), traced) + assert (predecessor.name, source.name) == ("conv2d", "x") + + +def test_an_identity_is_absorbed_only_ahead_of_a_functional_convolution(offline_ee): + functional = _compile(PaddedConv(4, 4, 3, pad=None, spelling="Identity", call="F.conv2d", own=1), 4) + module = _compile(PaddedConv(4, 4, 3, pad=None, spelling="Identity", call="nn.Conv2d", own=1), 4) + assert "pad" not in functional.nodes and "pad" in module.nodes # the Identity keeps its node + assert functional.image.serialize() != module.image.serialize() # today's extra materialization + + +def test_recognition_draws_no_random_numbers(): + model = PaddedConv(4, 4, 3, stride=2, pad=(0, 1, 0, 1)).eval() + traced = symbolic_trace(model) + state = torch.random.get_rng_state() + node = next(n for n in traced.graph.nodes if n.op == "call_function" and n.target is torch.conv2d) + recognised = convolution(node, traced) + assert torch.equal(torch.random.get_rng_state(), state) + assert recognised.module.weight is model.weight and recognised.module.bias is model.bias + assert recognised.module.padding == (1, 1) and recognised.phase == 1 and recognised.path == "weight" + buffered = PaddedConv(4, 4, 3, pad=(1, 1, 1, 1)).eval() + weight = buffered.weight.detach().clone() + del buffered.weight + buffered.register_buffer("weight", weight) + traced = symbolic_trace(buffered) + node = next(n for n in traced.graph.nodes if n.op == "call_function" and n.target is torch.conv2d) + assert torch.equal(convolution(node, traced).module.weight, weight) + + +# --------------------------------------------------------------------------- +# Refusals, each with its message +# --------------------------------------------------------------------------- +class ShapePad(nn.Module): + """Padding computed from the input's shape, as dynamic "same" padding does.""" + + def __init__(self, form): + super().__init__() + self.form = form + self.weight = nn.Parameter(_values((4, 4, 3, 3), 1)) + + def forward(self, x): + if self.form == "arithmetic": + pad = (0, x.shape[-1] % 2 + 1, 0, x.shape[-2] % 2 + 1) + else: # a shape element itself (batch 1) + pad = (0, x.shape[0], 0, x.shape[0]) + return F.conv2d(F.pad(x, pad), self.weight, None, 2) + + +class Reused(nn.Module): + def __init__(self, use): + super().__init__() + self.use = use + self.pad = nn.ZeroPad2d(1) + self.weight = nn.Parameter(_values((4, 4, 3, 3))) + self.pool = nn.MaxPool2d(3, 1) + + def forward(self, x): + padded = self.pad(x) + if self.use == "pool": + return self.pool(padded) + return F.conv2d(padded, self.weight) + padded[:, :, 1:-1, 1:-1] + + +class Functional(nn.Module): + def __init__(self, form): + super().__init__() + self.form = form + self.weight = nn.Parameter(_values((4, 4, 3, 3))) + + def forward(self, x): + if self.form == "computed": + return F.conv2d(x, self.weight * 2, None, 1, 1) + if self.form == "shared": + return F.conv2d(x, self.weight, None, 1, 1) + F.conv2d(x, self.weight, None, 1, 1) + if self.form == "dilation": + return F.conv2d(x, self.weight, None, 1, 2, 2) + return F.conv2d(x, self.weight, None, 3, 1) + + +class Explicit(nn.Module): + def __init__(self, form): + super().__init__() + self.form = form + self.weight = nn.Parameter(_values((4, 4, 3, 3))) + pads = {"nonzero value": nn.ConstantPad2d(1, 0.5), "reflection": nn.ReflectionPad2d(1), + "replication": nn.ReplicationPad2d(1)} + self.pad = pads.get(form, nn.Identity()) + + def forward(self, x): + if self.form == "F.pad nonzero value": + x = F.pad(x, (1, 1, 1, 1), value=1.0) + elif self.form == "F.pad reflect": + x = F.pad(x, (1, 1, 1, 1), mode="reflect") + elif self.form == "F.pad replicate": + x = F.pad(x, (1, 1, 1, 1), mode="replicate") + else: + x = self.pad(x) + return F.conv2d(x, self.weight) + + +REFUSALS = [ + ("pad from shape arithmetic", lambda: ShapePad("arithmetic"), "padding computed from a tensor's shape"), + ("negative pad", lambda: PaddedConv(4, 4, 3, pad=(-1, 1, 1, 1)), r"negative padding \(cropping\)"), + ("F.pad of two", lambda: PaddedConv(4, 4, 3, pad=(1, 1), spelling="F.pad"), "four integers"), + ("F.pad of six", lambda: PaddedConv(4, 4, 3, pad=(1, 1, 1, 1, 0, 0), spelling="F.pad"), "four integers"), + ("nonzero value", lambda: Explicit("nonzero value"), r"ConstantPad2d value must be 0"), + ("F.pad nonzero value", lambda: Explicit("F.pad nonzero value"), r"F.pad value must be 0"), + ("reflection", lambda: Explicit("reflection"), "ReflectionPad2d is unsupported"), + ("replication", lambda: Explicit("replication"), "ReplicationPad2d is unsupported"), + ("F.pad reflect", lambda: Explicit("F.pad reflect"), "F.pad mode 'reflect' is unsupported"), + ("F.pad replicate", lambda: Explicit("F.pad replicate"), "F.pad mode 'replicate' is unsupported"), + ("pad before a pool", lambda: Reused("pool"), "ZeroPad2d has another use"), + ("pad read twice", lambda: Reused("twice"), "ZeroPad2d has another use"), + ("phase beyond the stride", lambda: PaddedConv(4, 4, 3, pad=(0, 0, 0, 0)), + r"gives phase \(1, 1\) at kernel \(3, 3\) and stride 1"), + ("padding that enlarges", lambda: PaddedConv(4, 4, 3, pad=(2, 2, 2, 2)), r"gives phase \(-1, -1\)"), + ("extra output", lambda: PaddedConv(4, 4, 3, stride=2, pad=(0, 3, 0, 3)), r"gives PyTorch 5 x 6 outputs"), + ("weight computed", lambda: Functional("computed"), "get_attr parameters or buffers used only as this call's operands"), + ("weight shared", lambda: Functional("shared"), "get_attr parameters or buffers used only as this call's operands"), + ("dilation", lambda: Functional("dilation"), "dilation=1"), + ("stride 3", lambda: Functional("stride3"), "stride=1 or 2"), +] + + +ARITHMETIC_FIRST = {"pad from shape arithmetic": "Unsupported function ", + "weight computed": "Unsupported FX operation get_attr"} +"""compile_model stops at the first refused node: here the arithmetic ahead of the padding or +the convolution, itself refused. check_compatibility reports every node.""" + + +@pytest.mark.parametrize("name,model,message", REFUSALS, ids=[r[0] for r in REFUSALS]) +def test_each_refusal_fires_with_its_message(offline_ee, name, model, message): + model = model().eval() + with pytest.raises(UnsupportedOperation, match=re.escape(ARITHMETIC_FIRST.get(name, "")) or message): + _compile(model, 4) + report = check_compatibility(model, (1, 4, 8, 10)).to_dict() + assert report["status"] == "unsupported" + assert any(re.search(message, blocker["message"]) for blocker in report["blockers"]), report["blockers"] + + +def test_a_pad_read_from_the_shape_is_refused_even_where_it_would_resolve(offline_ee): + """``specialize_plumbing`` would resolve a bare shape element into a literal at the + compiled input size (as for an interpolation size), but a pad read from a tensor's shape + stays refused: zero padding must be written statically. The raw trace (receptive field) + refuses the shape read itself.""" + message = "padding computed from a tensor's shape" + model = ShapePad("element").eval() + with pytest.raises(UnsupportedOperation, match=message): + _compile(model, 4) + report = check_compatibility(model, (1, 4, 8, 10)).to_dict() + assert report["status"] == "unsupported" + assert [b["node"] for b in report["blockers"] if message in b["message"]] == ["pad"] + for form in ("element", "arithmetic"): + with pytest.raises(UnsupportedReceptiveField, match="getattr"): + analyze_receptive_field(ShapePad(form).eval()) + + +# --------------------------------------------------------------------------- +# Receptive field, weight collection and matching +# --------------------------------------------------------------------------- +@pytest.mark.parametrize("pad,kernel,bounds", [ + ((0, 1, 0, 1), 3, ((0, 2), (2, 4))), ((1, 1, 1, 1), 3, ((-1, 1), (1, 3))), + ((1, 2, 1, 2), 5, ((-1, 3), (1, 5))), ((2, 2, 2, 2), 5, ((-2, 2), (0, 4)))]) +def test_the_receptive_field_uses_the_left_and_top_pads(pad, kernel, bounds): + report = analyze_receptive_field(PaddedConv(4, 4, kernel, stride=2, pad=pad).eval()) + assert report.x.phase_bounds == (bounds[0],) and report.x.bounds(1) == bounds[1] + assert report.y.phase_bounds == (bounds[0],) + mixed = analyze_receptive_field(PaddedConv(4, 4, 3, stride=2, pad=(1, 1, 0, 1)).eval()) + assert mixed.y.bounds(0) == (0, 2) and mixed.x.bounds(0) == (-1, 1) + + +class Stem(nn.Module): + """A functional stem with a folded BatchNorm, then a biased functional 1 x 1 convolution + behind an Identity: weight paths ``stem`` and ``head``.""" + + def __init__(self): + super().__init__() + self.stem = PaddedConv(4, 6, 3, stride=2, bias=False, pad=(0, 1, 0, 1), bn=True) + self.act = nn.SiLU() + self.head = PaddedConv(6, 2, 1, pad=None, spelling="Identity") + + def forward(self, x): + return self.head(self.act(self.stem(x))) + + +def test_weights_are_collected_and_matched_by_their_state_dict_paths(offline_ee): + model = Stem().eval() + parameters = effective_convolutions(model) + assert [(p.key, p.source) for p in parameters] == [("stem_bn", "stem+stem.bn"), ("conv2d_1", "head")] + fused = torch.nn.utils.fuse_conv_bn_eval(_reference(4, 6, 3, stride=2, bias=False)[0], model.stem.bn) + assert torch.equal(torch.from_numpy(parameters[0].weights), fused.weight.detach().reshape(6, -1)) + assert {"stem.weight", "head.weight", "head.bias"} <= set(model.state_dict()) + layers = {} + for index, parameter in enumerate(parameters): + saved = replace(parameter, key=f"saved_{index}") + layers[saved.key] = {"shape": list(saved.shape), "dtype": "float32", "bias": saved.bias is not None, + "sha256": saved.sha256, "assets": ["projects/example/assets/weights"]} + manifest = {"format": "native_ee_featurecollection_weights_v2", "status": "ready", "layers": layers} + provider = MatchedWeightProvider(model, manifest) + assert provider.mapping == {"stem_bn": "saved_0", "conv2d_1": "saved_1"} + out = _compile(model, 4, weight_provider=provider, include_unfused_nodes=False) + assert provider.used_asset_keys == {"saved_0", "saved_1"} and not provider.fallback_keys + assert out.grid == _grid(10, 8).resized(2, 5, 4) and out.folded_batchnorms == ("stem_bn",) + + +# --------------------------------------------------------------------------- +# SMP's EfficientNet encoders (trace and the adapter's one PyTorch probe; no evaluation) +# --------------------------------------------------------------------------- +STRIDED = { # encoder -> {weight path: (kernel, groups > 1, total padding (l, r, t, b), phase)} at 256 px + "efficientnet-b0": { + "encoder._conv_stem": (3, False, (0, 1, 0, 1), 1), + "encoder._blocks.1._depthwise_conv": (3, True, (0, 1, 0, 1), 1), + "encoder._blocks.3._depthwise_conv": (5, True, (1, 2, 1, 2), 1), + "encoder._blocks.5._depthwise_conv": (3, True, (0, 1, 0, 1), 1), + "encoder._blocks.11._depthwise_conv": (5, True, (1, 2, 1, 2), 1)}, + "efficientnet-b3": { + "encoder._conv_stem": (3, False, (0, 1, 0, 1), 1), + "encoder._blocks.2._depthwise_conv": (3, True, (0, 1, 0, 1), 1), + "encoder._blocks.5._depthwise_conv": (5, True, (2, 2, 2, 2), 0), + "encoder._blocks.8._depthwise_conv": (3, True, (0, 1, 0, 1), 1), + "encoder._blocks.18._depthwise_conv": (5, True, (2, 2, 2, 2), 0)}, +} + + +@pytest.mark.parametrize("encoder,channels,classes", [("efficientnet-b0", 3, 2), ("efficientnet-b3", 8, 3)]) +def test_smp_efficientnet_unets_are_supported_through_the_adapter(encoder, channels, classes): + """Each stride-2 convolution carries the phase its padding gives. ``efficientnet-b0`` + pads its 5 x 5 stride-2 layers (1, 2, 1, 2), phase 1; ``efficientnet-b3`` pads its + (2, 2, 2, 2), phase 0 (SMP fixes the padding for the encoder's configured image size).""" + smp = pytest.importorskip("segmentation_models_pytorch") + from tensor_ee.smp import adapt_smp_model + + shape = (1, channels, 256, 256) + model = smp.Unet(encoder, encoder_weights=None, in_channels=channels, classes=classes).eval() + adapted = adapt_smp_model(model, input_shape=shape) + report = check_compatibility(adapted, shape).to_dict() + assert report["status"] == "supported", report["blockers"] + report["unknowns"] + assert report["output_shape"] == [1, classes, 256, 256] + strided = {} + for node in adapted.graph.nodes: + if node.op in ("call_module", "call_function"): + recognised = convolution(node, adapted) + if recognised is not None and tuple(recognised.module.stride) == (2, 2): + strided[recognised.path] = (recognised.module.kernel_size[0], recognised.module.groups > 1, + recognised.padding, recognised.phase) + assert strided == STRIDED[encoder] diff --git a/tools/fixtures/raster.py b/tools/fixtures/raster.py index 4d6acaa..a534097 100644 --- a/tools/fixtures/raster.py +++ b/tools/fixtures/raster.py @@ -2,9 +2,10 @@ (calibration), at additional pooling and nearest-resize settings (the selection ``p02``) and the bounded activations (``relu6``, ``hardsigmoid``, ``hardswish``; ``hard_activations``), standalone inference BatchNorm (``batch_norm``), integer bilinear resizing (``bilinear``), -grouped/depthwise convolution (``grouped_conv``), and global average pooling with the +grouped/depthwise convolution (``grouped_conv``), global average pooling with the elementwise multiply, alone and as a squeeze-excitation gate through ``compile_model`` -(``global_pool_multiply``). +(``global_pool_multiply``), and zero padding ahead of a convolution, which runs the +convolution at a sampling phase, through ``compile_model`` (``padded_conv``). Every fixture runs the library block in ``tensor_ee/blocks/raster`` plus float32 materialization (through the public facade where available, as ``compile_model`` @@ -1055,6 +1056,161 @@ def _gate_fixtures(cost) -> list[Fixture]: return fixtures +# --------------------------------------------------------------------------- +# Zero padding ahead of a convolution: the sampling phase (selection padded_conv) +# --------------------------------------------------------------------------- +@dataclass(frozen=True) +class _Padded: + """One padded convolution through ``compile_model``: ``nn.ZeroPad2d(padding)`` then + ``F.conv2d`` with the module's own weight and bias at stride 2. Dense 2 -> 3 channels + (``groups`` 1) or depthwise on 4 channels. Only forms with distinct graphs are fixtures; + spellings of one graph are offline byte-identity tests (tests/test_raster_padded_conv.py).""" + + block: str + name: str + seed: int + padding: tuple # (left, right, top, bottom) + kernel: int = 3 + groups: int = 1 + shape: tuple = (8, 10) # (H, W) + + @property + def inputs(self) -> int: + return 2 if self.groups == 1 else self.groups + + @property + def outputs(self) -> int: + return 3 if self.groups == 1 else self.groups + + @property + def phase(self) -> tuple: + left, _, top, _ = self.padding + return self.kernel // 2 - top, self.kernel // 2 - left + + @property + def output(self) -> tuple: + (h, w), (py, px) = self.shape, self.phase + return (h - py + 1) // 2, (w - px + 1) // 2 + + +_PADDED = ( + # Phase 1: the 3 x 3 stride-2 convolution padded (0, 1, 0, 1), on an even and an odd map. + _Padded("conv2d", "pad0101-k3-s2-h8-w10", 81, (0, 1, 0, 1)), + _Padded("conv2d", "pad0101-k3-s2-h7-w9", 82, (0, 1, 0, 1), shape=(7, 9)), + _Padded("grouped_conv2d", "depthwise-pad0101-k3-s2", 83, (0, 1, 0, 1), groups=4), + # Phase 1 at 5 x 5: padded (1, 2, 1, 2), the "same" padding of an even map; reach 3. + _Padded("grouped_conv2d", "depthwise-pad1212-k5-s2", 84, (1, 2, 1, 2), kernel=5, groups=4), + # Phase 0 at 5 x 5 stride 2 through a symmetric pad: today's graph by recognition. + _Padded("grouped_conv2d", "depthwise-pad2222-k5-s2", 85, (2, 2, 2, 2), kernel=5, groups=4), + # Different phases on the two axes: (1, 0), padded (1, 1, 0, 1). + _Padded("grouped_conv2d", "depthwise-pad1101-k3-s2", 86, (1, 1, 0, 1), groups=4), +) + +_PADDED_BANDS = ("b0", "b1", "b2", "b3") + + +def _padded_data(spec: _Padded): + def data(seed: int) -> FixtureData: + weights = dyadic(seed, "padded.weight", (spec.outputs, spec.inputs // spec.groups, spec.kernel, spec.kernel), + scale=0.125) + return FixtureData(inputs={"x": dyadic(seed, "x", (*spec.shape, spec.inputs))}, weights={ + "weight": weights, "bias": dyadic(seed, "padded.bias", (spec.outputs,), scale=0.125)}) + return data + + +def _padded_module(spec: _Padded, data: FixtureData): + """The stand-in: a padding module, then ``F.conv2d`` reading the module's own weight and + bias (static "same" padding as some encoders write it), with no model package imported.""" + import torch + from torch import nn + from torch.nn import functional as F + + class StaticPaddingConv(nn.Module): + def __init__(self): + super().__init__() + self.static_padding = nn.ZeroPad2d(spec.padding) + self.weight = nn.Parameter(torch.from_numpy(data.weights["weight"].copy())) + self.bias = nn.Parameter(torch.from_numpy(data.weights["bias"].copy())) + + def forward(self, x): + return F.conv2d(self.static_padding(x), self.weight, self.bias, 2, 0, 1, spec.groups) + + return StaticPaddingConv().eval() + + +def _padded_cost_module(spec: _Padded, data: FixtureData): + """The same convolution as an ``nn.Conv2d``, for the block's cost function.""" + import torch + from torch import nn + + module = nn.Conv2d(spec.inputs, spec.outputs, spec.kernel, 2, spec.kernel // 2, groups=spec.groups) + with torch.no_grad(): + module.weight.copy_(torch.from_numpy(data.weights["weight"])) + module.bias.copy_(torch.from_numpy(data.weights["bias"])) + return module.eval() + + +def _padded_build(spec: _Padded): + def build(ee, data: FixtureData) -> list: + import tensor_ee + + grid = _rect_grid(10, *spec.shape) + bands = _PADDED_BANDS[:spec.inputs] + image = inline_raster(ee, grid.projection(), data.inputs["x"], bands) + compiled = tensor_ee.compile_model(_padded_module(spec, data), image, bands, grid) + return [Readout("out", compiled.image, spec.outputs, raster_grid(compiled.grid)), + Readout("input", compiled.nodes["x"].image, spec.inputs, raster_grid(grid), "input", "x")] + return build + + +def _padded_reference(spec: _Padded): + def reference(data: FixtureData) -> dict: + import torch + + with torch.no_grad(): + return {"out": hwc(_padded_module(spec, data)(nchw(data.inputs["x"])))} + return reference + + +def _padded_mirror(spec: _Padded): + def mirror(data: FixtureData) -> dict: + from reference.raster_conv import grouped_conv2d + + return {"out": grouped_conv2d(data.inputs["x"], data.weights["weight"], data.weights["bias"], + groups=spec.groups, stride=2, padding=spec.padding)} + return mirror + + +def _padded_fixtures(cost) -> list[Fixture]: + import tensor_ee + from torch import nn + from tensor_ee.blocks.raster.conv import conv2d, grouped_conv2d + from tensor_ee.layouts.raster import canonical_phase + + fixtures = [] + for spec in _PADDED: + dense = spec.groups == 1 + fixtures.append(Fixture( + id=f"raster/{spec.block}/{spec.name}", lane=LANE, block=spec.block, + settings={"kernel_size": spec.kernel, "stride": 2, "groups": spec.groups, "padding": spec.padding, + "phase": canonical_phase(spec.phase, 2), "clip_footprints": False}, policy=POLICY, + shapes={"input": [*spec.shape, spec.inputs], "output": [*spec.output, spec.outputs]}, + seed=spec.seed, data=_padded_data(spec), build=_padded_build(spec), reference=_padded_reference(spec), + reference_source=f"nn.ZeroPad2d({spec.padding}) then F.conv2d(x, weight, bias, 2, 0, 1, {spec.groups})", + mirror=None if dense else MirrorSpec(_padded_mirror(spec), True, "reference.raster_conv.grouped_conv2d"), + requests=2, + constants=cost(spec.block, lambda data, spec=spec: (_padded_cost_module(spec, data),)), + history=("docs/evidence/live-fixtures/20261112T185127Z-padded_conv.json " + + ("(passed; within 1e-4 of PyTorch, no mirror)" if dense + else "(passed; Earth Engine equal to the mirror)")), + accepts=(tensor_ee.compile_model, nn.ZeroPad2d, nn.Conv2d, conv2d if dense else grouped_conv2d), + notes=(f"compile_model on a stand-in module: ZeroPad2d{spec.padding} absorbed into F.conv2d, so the " + f"block runs at phase {spec.phase} (output pixel i is input pixel 2 i + phase on each axis); " + + ("No NumPy mirror of the dense convolution's matrixMultiply exists: compared with PyTorch " + "only." if dense else "The mirror applies the same zero padding and PyTorch's output count.")))) + return fixtures + + SELECTIONS = { # raster-setting validation: every new setting, plus today's pooling and nearest settings as regression checks # (three calibration fixtures). @@ -1071,6 +1227,7 @@ def _gate_fixtures(cost) -> list[Fixture]: "global_pool_multiply": (*(f"raster/global_avg_pool2d/{spec.name}" for spec in _GLOBAL_POOLS), "raster/multiply/same-grid-h5-w7", "raster/multiply/gate-h5-w7", "raster/multiply/squeeze-excitation-h12-w20"), + "padded_conv": tuple(f"raster/{spec.block}/{spec.name}" for spec in _PADDED), } """This lane's named selections (``tools.fixtures.select``).""" @@ -1146,4 +1303,4 @@ def cost(block, args): ] return (calibration + _p02_fixtures(cost) + _hard_fixtures(cost) + _batch_norm_fixtures(cost) + _bilinear_fixtures(cost) + _grouped_fixtures(cost) + _standardize_fixtures(cost) - + _gate_fixtures(cost)) + + _gate_fixtures(cost) + _padded_fixtures(cost))