feat(optim): add Conv channel-affine folding#1167
Conversation
Simple exampleAssume a Conv produces 10. The graph then multiplies it by 2 and adds 3: Instead, ModelKit changes the Conv's stored numbers: The modified Conv directly produces 23, so the separate |
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
Reviewed the Conv channel-affine folding. The core algebra is correct and nicely conservative — I checked direct chains, Split/Slice routing, channel-preserving views, nested slices, partial (single-branch) folds, shared-weight Convs, multi-consumer terminals, and chained Conv→Mul→Conv, and they all stay numerically exact; the 17 tests pass. A few non-blocking notes below.
DingmaomaoBJTU
left a comment
There was a problem hiding this comment.
Re-reviewed 4209ab2 (fix: preserve affine precision and metadata). Both earlier points are resolved and verified:
- Affine precision —
calculation_dtype = np.result_type(weight_dtype, np.float32)is now threaded through_collect_affine_chain/_collect_routed_affine_candidates, so the accumulators no longer truncate to float32. My repro (float64scale = 1 + 2**-30) now folds to1.0000000009313226exactly instead of1.0, andtest_float64_affine_values_preserve_weight_precisionlocks it in. float16 still accumulates in float32 as expected. - Stale value_info —
_prune_stale_value_infocleanly drops metadata for tensors no longer produced (only removes names absent from inputs/initializers/node outputs), and the newassert not transformed.graph.value_infoguards it.
All 18 algebraic tests pass and my direct/routed/view/nested-slice/shared-weight/chained probes stay numerically exact. The one remaining note (the per-fold _GraphIndex.build) is a low-priority perf item and safe to leave. LGTM from my side.
Summary
Add an opt-in
conv-channel-affine-foldingalgebraic rewrite that folds safe scalar or channel-wise constantMul/Addoperations into Conv weights and bias.The rewrite supports direct affine chains and channel routing through static
Split/Sliceand channel-preservingReshape/Squeeze/Unsqueezeoperations. It conservatively rejects dynamic shapes and constants, shared or public intermediate tensors, overlapping channel ranges, unsupported broadcasts, and non-floating Conv parameters. Multiple independent matches are handled, generated parameters use collision-safe names, dead constants and replaced initializers are pruned, and repeated optimization is idempotent.BatchNormalization folding from the source change is intentionally excluded.
Why this is not a pattern rewrite
The current pattern rewriter replaces a fixed, removable source subgraph with a fixed target topology whose source and target schemas must match. Conv channel-affine folding requires different behavior:
Split,Slice, and shape-only view branches.Mul/Addnodes are removed and router outputs are rewired.These requirements are match-parameterized parameter surgery over partially retained topology, not fixed subgraph substitution. Supporting them as a pattern would require broader framework capabilities for mutable initializer transforms, variable topology, partial-match retention, and cross-branch aggregation. Therefore this remains a focused algebraic rewrite.