feat(inference): support direct ONNX inpainting models

[ssss141414]

Summary

This adds generic direct-ONNX LaMa inpainting support for opencv/inpainting_lama, including bare Hugging Face repository resolution, an explicit persisted image+mask runtime contract, and CPU recipes for fp32 and fp16. The contribution ships at Effort/Outcome L2 and reaches the committed Goal ceiling of L1 PASS for both required CPU tuples. The published graph is already INT8-prequantized, so “fp32” and “fp16” identify the precision of its remaining floating initializers rather than pure-float models.

Model metadata

What the model does

LaMa performs image inpainting: it accepts a 512 x 512 RGB image and a 512 x 512 single-channel binary mask, synthesizes replacement RGB content for the masked region, and returns an inpainted 512 x 512 RGB image (verified).

Evidence: the model card pinned to revision aee6d22f0a13e5e35af1c9a1c3afd62841fc6f3f, pinned lama.py Lama.infer() wrapper, and published ONNX image, mask, and output contract.

Primary user stories

• A user supplies a photograph and a painted binary mask to obtain a completed image with the masked object or damaged region filled in (verified; pinned demo and model card).

Supported tasks

• inpainting — checkpoint support surface (verified). The pinned model card and demo explicitly define image-inpainting behavior. The repository has no config.json, pipeline_tag, Transformers class, or Optimum registration; no such surface is claimed.

Model architecture

Lama (repository inference wrapper; not a torch.nn.Module)
├── Runtime preprocessing: RGB image blob + thresholded binary mask (512 x 512)
├── Published ONNX LaMa generator
│   ├── Image/mask composition and padded FFC convolution/downsampling stages (model.0-4)
│   ├── FFC residual blocks x 18 (model.5-22)
│   │   ├── Local↔global convolution branches
│   │   └── Fourier unit transforms and residual additions
│   ├── Upsampling decoder: ConvTranspose + BatchNorm + ReLU x 3 (model.23-32)
│   └── Padded RGB Conv + Sigmoid output (model.33-35)
└── Runtime postprocessing: HWC uint8 conversion and aspect-ratio resize

• Source/confidence: pinned lama.py and published ONNX graph scopes, I/O, and topology (mapped).

Validation and support evidence

Baseline

The replacement charter freezes WinML winml, version 0.2.0 at current main commit 4daf0f19097d03e92aeb3c4f9713b7d037e0be3d. The repaired feature head is f03add820f384ce725e83400d53d108ff43aad0a, based directly on that commit; tester r5 refreshed revision, semantic-contract, analyze, component/op, and quality evidence at this head and retained earlier build/perf/eval measurements only where artifact-integrity and affected-scope checks established valid carryover.

The baseline below is intentionally run in a detached clean worktree at the frozen current-main SHA. It distinguishes the canonical bare repository-ID failure from the successful direct-ONNX diagnostic; the direct-file result proves generic graph ingestion/execution only and does not upgrade bare-ID support on current main.

git fetch origin main
$BASELINE_REPO = Join-Path (Get-Location) '.winml-repro/current-main-4daf0f1'
git worktree add --detach $BASELINE_REPO 4daf0f19097d03e92aeb3c4f9713b7d037e0be3d
Push-Location $BASELINE_REPO
uv sync --frozen

& .\.venv\Scripts\winml.exe --version
# winml, version 0.2.0

git rev-parse HEAD
# 4daf0f19097d03e92aeb3c4f9713b7d037e0be3d

$BASELINE_OUT = Join-Path (Get-Location) '.winml-repro/bare-repository-id'
& .\.venv\Scripts\winml.exe build -m 'opencv/inpainting_lama' -o $BASELINE_OUT --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild --no-color
# exit 2 after 23.170 s; no model artifact:
# Unrecognized model in opencv/inpainting_lama (the repository has no config.json model_type).

$PINNED_ONNX = Join-Path (Get-Location) '.winml-repro/inpainting_lama_2025jan.onnx'
Invoke-WebRequest -UseBasicParsing -Uri 'https://huggingface.co/opencv/inpainting_lama/resolve/aee6d22f0a13e5e35af1c9a1c3afd62841fc6f3f/inpainting_lama_2025jan.onnx' -OutFile $PINNED_ONNX
if ((Get-FileHash $PINNED_ONNX -Algorithm SHA256).Hash.ToLowerInvariant() -ne '7df918ac3921d3daf0aae1d219776cf0dc4e4935f035af81841b40adcf74fdf2') { throw 'Pinned ONNX SHA-256 mismatch' }
$DIRECT_OUT = Join-Path (Get-Location) '.winml-repro/direct-onnx-diagnostic'
& .\.venv\Scripts\winml.exe build -m $PINNED_ONNX -o $DIRECT_OUT --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild --no-color
# exit 0; Build complete; model.onnx is 92,591,623 bytes and retains the pinned SHA-256.

Pop-Location

• Bare repository build: FAIL (exit 2 after 23.170 s). No model artifact was produced. The repository has no config.json model_type, so current main reports Unrecognized model in opencv/inpainting_lama.
• Starting auto-config behavior: direct-ONNX winml config generated a recipe with skip_optimize: true, export: null, empty optim, quant: null, compile: null, and loader task inpainting; it had no runtime contract. The diagnostic direct-ONNX build above succeeded and preserved the published 92,591,623-byte graph, but this does not upgrade the failing bare-repository workflow.
• Perf floor: the diagnostic direct-ONNX CPU run exited 0 after 36.239 s with mean/p50 2594.816/2591.605 ms, 0.39 samples/s, and RSS total delta 654.06 MB. The coarse detector reported w8a8 because the published graph is prequantized.
• Eval floor: exit 1 before dataset/model execution because inpainting is unsupported by winml eval. The pinned mattmdjaga/human_parsing_dataset revision db120bb5c18c146a8fbd2160f7575a288269fe7d train sample is schema-probe-only; it has no authoritative image+mask+restored-target inpainting metric, and no metric validity is claimed.
• Optimum probe: UNREGISTERED. No vendor registration, post-WinML registration, or WinML-added registration was found.
• The baseline Goal floor was L0.

Goal

• Effort: L2 — generalized class-of-models code support.
• Committed Goal ceiling: L1.
• Outcome: L2.
• Success definition: both exact CPU fp32/fp16 tuples freshly build and execute semantic image+mask inference through the public runtime with concrete performance evidence.
• This replacement charter re-issued the baseline after main moved from 38767add6f91c7b10b6394fae3af6f437e02effd to 4daf0f19097d03e92aeb3c4f9713b7d037e0be3d; the Effort, Goal ceiling, Outcome, and success definition did not change.

Outcome

• Shipped tier: L2.
• Highest reached Goal verdict: L1 PASS; the ceiling was reached and not downgraded.
• Coverage: full; deferred tuples: none.
• Checked-in recipes: examples/recipes/opencv_inpainting_lama/cpu/cpu/inpainting_fp32_config.json and examples/recipes/opencv_inpainting_lama/cpu/cpu/inpainting_fp16_config.json.
• Shipped code covers deterministic bare-repository ONNX resolution with immutable revision provenance and ambiguity rejection; offline/connectivity fallback for optional discovery; recipe-free/export-null direct ONNX builds; consistent prequantized/FP16 stage behavior; and generic, persisted, fail-closed inpainting runtime dispatch.
• Recipe-free bare-ID acceptance passed in 6.066265 s, producing the pinned inpainting_lama_2025jan.onnx artifact and recording the repository ID, selected artifact, and immutable Hub revision in the build manifest.
• The public inpainting caller contract is canonical: nonzero mask pixels identify holes (the replaced region). Persisted runtime.options.mask_semantics does not redefine caller input; it declares the model-side ONNX mask tensor polarity. After binarizing the caller mask, the adapter preserves it for nonzero-is-hole (hole → 1, background → 0) and inverts it for zero-is-hole (hole → 0, background → 1), preserving the same semantic hole region.
• Fresh tester-r5 semantic-contract evidence exercised one fixed caller mask through the public pipeline adapter under both model-side polarities. In each case, decoded model-side hole coordinates exactly matched caller nonzero coordinates [[0,1],[1,1],[1,2]]; the focused semantic-contract regressions passed 3 tests, the complete task/inpainting files passed 28 tests, and the broader affected suite passed 369 tests.
• The refreshed learner audit confirms existing findings lama-001 through lama-004, _task-inpainting-001, and _meta-085 through _meta-088 without adding, retiring, or rewriting finding IDs. The inpainting-eval gap remains tracked by microsoft/winml-cli#1158; skill-only evidence remains isolated in Draft PR gim-home/ModelKitArtifacts#170.

The L1 ceiling remains the formal result. Supplementary L2 used the published ONNX artifact at revision aee6d22f0a13e5e35af1c9a1c3afd62841fc6f3f because the checkpoint publishes no PyTorch weights or loadable PyTorch implementation; no PyTorch result is fabricated.

Candidate	Reference	Verdict	Cosine similarity	Max abs	Mean abs	RMSE	PSNR (dB)	Unmasked max abs
mixed INT8+fp32	pinned published ONNX	PASS	1.0	0.0	0.0	0.0	∞	0.0
mixed INT8+fp16	pinned published ONNX	PASS	0.9847547323069161	134.32603454589844	9.53465276117015	28.621334963655865	18.997005881260385	0.0623779296875

The public runtime adapter exactly matched direct candidate output for both tuples: fp32 cosine 0.9999999999999999, max abs 0.0; fp16 cosine 1.0, max abs 0.0. Against the pinned source ONNX through that adapter, fp32 remained exact and fp16 measured cosine 0.9847192758836978, max abs 135.0, mean abs 9.529802958170572, and RMSE 28.638184333305173.

Quality results

Tester r5 validated the current head f03add820f384ce725e83400d53d108ff43aad0a. Build/artifact, CLI perf, semantic runtime perf, supplementary ONNX-reference, and eval evidence was retained only under the tester's explicit carryover checks; semantic-contract, analyze, component/op, quality, and provenance evidence was freshly rerun or refreshed.

Current-head check	Result
Canonical caller-mask and both model-side polarity regressions	PASS — 3 passed in 1.27 s
Complete task + inpainting test files	PASS — 28 passed in 1.31 s
Broader affected suite	PASS — 369 passed in 18.87 s
mypy -p winml.modelkit	PASS — no issues in 407 source files
Ruff on all 27 changed Python files	PASS — all checks passed
Ruff format check on all 27 changed Python files	PASS — 27 files already formatted
git diff --check origin/main...HEAD	PASS

The current repair changed public contract documentation/error wording and focused tests, not _prepare_mask() executable conversion, recipes, model artifacts, perf/session code, or eval support. Tester r5 reverified source/recipe/artifact hashes and affected scope before retaining the concrete build, perf, runtime, ONNX-reference, and eval evidence below.

Per-EP/device/precision results — including perf and eval data

Goal ladder

Tier	Verdict	Evidence
L0	PASS	Both exact required tuples freshly built and passed ONNX structural, named-I/O, initializer-transition, runtime-contract, and immutable-provenance validation.
L1	PASS	Both tuples completed CLI perf and semantic image+mask inference through InferenceEngine → persisted manifest contract → WinMLInpaintingPipeline → CPU model → PNG postprocessing, with exact adapter/direct parity.

Build and CLI perf

CLI perf used 3 warmups and 20 measured iterations with the same named semantic inputs.

Tier	EP / Device	Precision	Verdict	Build time	Graph semantics	Mean	p50	Throughput	RAM Δ	VRAM Δ	Task metric
L0/L1	CPUExecutionProvider / cpu	fp32	PASS	7.0649837 s	606 FLOAT + 598 INT8 + 299 INT64 initializers; FLOAT I/O; 18,001 nodes	2569.827 ms	2555.584 ms	0.39 samples/s	+656.16 MB total; +183.07 MB model load	+0.0 MB local / +0.0 MB shared	—
L0/L1	CPUExecutionProvider / cpu	fp16	PASS	10.0799193 s	606 FLOAT16 + 598 INT8 + 299 INT64 initializers; FLOAT I/O; 18,436 nodes	2589.314 ms	2582.497 ms	0.39 samples/s	+728.93 MB total; +193.34 MB model load	+0.0 MB local / +0.0 MB shared	—

Both artifacts retain the source graph’s 598 INT8 initializers. The coarse CLI precision detector consequently reports w8a8; tuple identity is established by requested precision plus the audited residual FLOAT→FLOAT16 transition.

Supplementary public-runtime measurements used 2 warmups and 5 measured iterations:

EP / Device	Precision	Runtime mean	Runtime p50	Runtime throughput	Runtime RAM Δ	Output
CPUExecutionProvider / cpu	fp32	2594.686 ms	2601.28 ms	0.3853442811112959 samples/s	+1103.91796875 MB	PNG, 512 x 512, 42,667 bytes
CPUExecutionProvider / cpu	fp16	2668.932 ms	2679.08 ms	0.37464229996198944 samples/s	+1178.15625 MB	PNG, 512 x 512, 49,083 bytes

Eval support evidence

Tier	EP / Device	Precision	Verdict	Dataset / revision / subset	Split / samples	Task metric	Blocker
L3 supplementary	CPUExecutionProvider / cpu	fp32	CLI-BLOCKED	mattmdjaga/human_parsing_dataset / db120bb5c18c146a8fbd2160f7575a288269fe7d / none	train / 1	none	Exit 1 after 0.6550235000322573 s: Task 'inpainting' is not supported; no output was produced. Schema probe exit 2 has the same unsupported-task registry gap.
L3 supplementary	CPUExecutionProvider / cpu	fp16	CLI-BLOCKED	mattmdjaga/human_parsing_dataset / db120bb5c18c146a8fbd2160f7575a288269fe7d / none	train / 1	none	Exit 1 after 0.6559123999904841 s: Task 'inpainting' is not supported; no output was produced. Schema probe exit 2 has the same unsupported-task registry gap.

The dataset is a schema probe only and does not establish an authoritative restored-target contract or canonical inpainting metric. This is an evaluator-registry plus metric-dataset-contract gap, not an EP, export, precision, build, or runtime failure.

Delta

Recipes and configuration

Relative to the replacement charter’s generated baseline recipe, both checked-in recipes add the explicit /runtime contract; fp16 additionally changes /quant from null to an explicit FP16 conversion configuration.

Recipe	JSON pointer	Baseline value	Shipped value
fp32 and fp16	/runtime	null	{"pipeline":"inpainting","options":{"image_input_name":"image","mask_input_name":"mask","output_name":"output","image_color_order":"bgr","image_value_range":[0,1],"mask_semantics":"nonzero-is-hole","output_color_order":"bgr","output_value_range":[0,255]}}
fp16 only	/quant	null	explicit mode: fp16 configuration below

{
  "mode": "fp16",
  "samples": 10,
  "calibration_method": "minmax",
  "weight_type": "uint8",
  "activation_type": "uint8",
  "per_channel": false,
  "symmetric": false,
  "weight_symmetric": null,
  "activation_symmetric": null,
  "save_calibration": false,
  "distribution": "uniform",
  "seed": null,
  "calibration_load_path": null,
  "calibration_save_path": null,
  "op_types_to_quantize": null,
  "nodes_to_exclude": null,
  "fp16_keep_io_types": true,
  "fp16_op_block_list": null
}

The public task schema defines one caller-side convention: nonzero mask pixels are holes. The persisted runtime contract separately declares exact image/mask/output tensor roles, BGR image [0,1], model-side ONNX mask tensor polarity, and BGR output [0,255]. For the checked-in LaMa recipes, mask_semantics: "nonzero-is-hole" means the canonical caller mask is passed through after binarization. A different model declaring zero-is-hole receives the inverted ONNX tensor while the caller-designated hole region remains unchanged. Both schemas validated, actual builds persisted the contract in resolved configuration and manifests, and reducibility is consistent with the charter. The fp32 recipe has no additional delta; the fp16 recipe converts only the remaining floating tensors while preserving prequantized INT8 weights. The production recipe README remains unchanged.

The actual public pipeline passed all four fail-closed contract checks: missing contract, unsupported image range, absent/non-distinct tensor roles, and an unknown model-specific option all raised their expected ValueError rather than guessing semantics.

Code

Path	Symbols	Class-wide behavior change
src/winml/modelkit/loader/onnx_hub.py	ResolvedHubOnnx, resolve_hf_repo_onnx()	Discovers exactly one ONNX sibling for config-less Hub repositories, pins resolution/download, rejects ambiguity, and falls back safely on optional-preflight connectivity failures.
src/winml/modelkit/utils/model_input.py	ModelInput, resolve_model_input()	Adds opt-in bare-repository ONNX discovery and carries repository, artifact, and revision provenance.
src/winml/modelkit/commands/build.py	build(), _run_quantize_stage(), _build_onnx_pipeline()	Routes recipe-free/export-null repositories through direct ONNX, records provenance/runtime metadata, and preserves explicit FP16 conversion for prequantized graphs.
src/winml/modelkit/config/build.py	WinMLRuntimeConfig, WinMLBuildConfig, generate_onnx_build_config()	Adds validated serializable runtime metadata and suppresses incompatible requantization while retaining explicit FP16 conversion.
src/winml/modelkit/build/common.py	ensure_pre_quantized_stamped()	Clears only incompatible quantization modes and preserves mode=fp16.
src/winml/modelkit/build/onnx.py	build_onnx_model()	Skips optimization for prequantized inputs while still converting remaining floating tensors when FP16 is explicit.
src/winml/modelkit/build/hf.py	build_hf_model()	Applies the same prequantized/FP16 semantics in the Python HF path.
src/winml/modelkit/utils/cli.py	normalize_model_arg()	Resolves an unambiguous bare direct-ONNX Hub repository.
src/winml/modelkit/models/auto.py	WinMLAutoModel.from_pretrained()	Recognizes bare direct-ONNX Hub repositories and retains runtime metadata.
src/winml/modelkit/inference/engine.py	InferenceEngine.load(), InferenceEngine.load_schema_only()	Resolves repositories for normal inference, restores manifest runtime metadata, and avoids artifact download for explicit-task schema-only loading.
src/winml/modelkit/inference/tasks.py	TASK_REGISTRY, _postprocess_inpainting()	Registers the generic two-image inpainting contract and JSON-safe PNG output; the public mask description fixes caller polarity as nonzero-is-hole.
src/winml/modelkit/inference/pipeline.py	_CUSTOM_PIPELINE_FACTORIES, create_pipeline()	Dispatches registered non-Transformers task pipelines and passes persisted runtime metadata.
src/winml/modelkit/inference/inpainting.py	WinMLInpaintingPipeline, _prepare_mask()	Implements a fail-closed metadata-driven adapter: canonical caller nonzero holes are encoded into the persisted model-side ONNX mask polarity without changing the semantic hole region.
src/winml/modelkit/utils/manifest.py	WinMLManifest	Persists optional runtime metadata while retaining schema-version-1 compatibility.
src/winml/modelkit/models/winml/base.py	WinMLPreTrainedModel.runtime_config	Carries the runtime contract on programmatic wrappers.

The committed diff also adds or updates focused tests for build paths, config generation, inference, task registration, Hub resolution, auto-model loading, manifests, and model-input resolution.

Analyze summary — component level and op level

ANALYZE-PARTIAL-SUCCESS: both literal tester-owned analyze --ep all --device all processes exited 1 because OpenVINO plugin registration failed with Error 126 (onnxruntime_providers_shared.dll was missing). Each process nevertheless emitted complete fail-closed JSON with 11 unique nonempty classification rows, including all six required public rule-backed targets, complete graph metadata, and 100% component mapping. This is static rule analysis, not runtime execution; every emitted EP row has runtime_support=false.

Component-level summary

Artifact	Architecture coverage	Mapping	Confidence	Actionable EP findings
fp32	mask/composite; encoder scopes 0-4; 18 FFC residual scopes 5-22; decoder scopes 23-33; output scopes 34-35	18,001 mapped; 0 partial; 0 unmapped	verified	QNN NPU/GPU partial: Shape, Transpose. OpenVINO NPU partial: Sqrt; unsupported: Add, Cast, Div. OpenVINO GPU/CPU unsupported: Add, Cast, Div, Sqrt.
fp16	mask/composite; encoder scopes 0-4; 18 FFC residual scopes 5-22; decoder scopes 23-33; output scopes 34-35	18,436 mapped; 0 partial; 0 unmapped	verified	QNN NPU/GPU partial: Shape, Transpose. OpenVINO NPU partial: Sqrt; unsupported: Add, Cast, Div. OpenVINO GPU/CPU unsupported: Add, Cast, Div, Sqrt.

There are no component-mapping gaps. The nonzero process exits are retained; complete component/op payloads do not establish runtime compatibility.

Op-level summary

Artifact	Graph	Dominant ops	EP roll-up
fp32	18,001 ops / 30 types	Constant 7,054; Reshape 1,570; Concat 1,360; Slice 1,322; Shape 1,188; Cast 782; Transpose 638; Unsqueeze 504	11 nonempty rows, six rule-backed. QNN NPU/GPU partial: Shape, Transpose. OpenVINO NPU partial: Sqrt; unsupported: Add, Cast, Div. OpenVINO GPU/CPU unsupported: Add, Cast, Div, Sqrt. Other rows retain unknown operations; no runtime-support claim.
fp16	18,436 ops / 30 types	Constant 7,054; Reshape 1,570; Concat 1,360; Slice 1,322; Cast 1,217; Shape 1,188; Transpose 638; Unsqueeze 504	11 nonempty rows, six rule-backed. QNN NPU/GPU partial: Shape, Transpose. OpenVINO NPU partial: Sqrt; unsupported: Add, Cast, Div. OpenVINO GPU/CPU unsupported: Add, Cast, Div, Sqrt. Other rows retain unknown operations; no runtime-support claim.

Reproduce commands

The block starts with tester-owned portable build/perf/eval commands. The rules setup downloads the official v0.2.0 release asset verified through the public GitHub release API: asset rules-v0.2.0.zip, published SHA-256 6dcabbe7f5493fbc2bd23de195ab6e35b3578d510f18c2e220bc5538a1f232cc, six expected EP directories, and 1,746 Parquet files. The pinned package expands those EP directories directly at the selected root, so WINMLCLI_RULES_DIR is set deterministically to that resolved expansion path. The two tester-owned analyze invocations include literal --ep all --device all; on the tester host each exited 1 for the OpenVINO Error 126 above while still emitting the complete 11-row static payload.

$OUT='temp/opencv-inpainting-lama-repro'
$env:OUT=$OUT; python -c "import os,numpy as np; from pathlib import Path; y,x=np.mgrid[0:512,0:512]; image=np.empty((1,3,512,512),dtype=np.float32); image[0,0]=x/511; image[0,1]=y/511; image[0,2]=((x+y)%256)/255; mask=np.zeros((1,1,512,512),dtype=np.float32); mask[0,0,160:352,176:336]=1; p=Path(os.environ['OUT']); p.mkdir(parents=True,exist_ok=True); np.savez(p/'semantic_inputs.npz',image=image,mask=mask)"
winml build -c examples/recipes/opencv_inpainting_lama/cpu/cpu/inpainting_fp32_config.json -m opencv/inpainting_lama -o $OUT/fp32 --ep cpu --device cpu --precision fp32 --no-analyze --no-compile --rebuild --no-color
winml build -c examples/recipes/opencv_inpainting_lama/cpu/cpu/inpainting_fp16_config.json -m opencv/inpainting_lama -o $OUT/fp16 --ep cpu --device cpu --precision fp16 --no-analyze --no-compile --rebuild --no-color

$RULES_ASSET_URL='https://github.com/microsoft/winml-cli/releases/download/v0.2.0/rules-v0.2.0.zip'
$RULES_ZIP=Join-Path $OUT 'rules-v0.2.0.zip'
$RULES_EXPANDED=Join-Path $OUT 'rules-v0.2.0'
Invoke-WebRequest -UseBasicParsing -Uri $RULES_ASSET_URL -OutFile $RULES_ZIP
$EXPECTED_RULES_SHA256='6dcabbe7f5493fbc2bd23de195ab6e35b3578d510f18c2e220bc5538a1f232cc'
$ACTUAL_RULES_SHA256=(Get-FileHash $RULES_ZIP -Algorithm SHA256).Hash.ToLowerInvariant()
if ($ACTUAL_RULES_SHA256 -ne $EXPECTED_RULES_SHA256) { throw "rules-v0.2.0.zip SHA-256 mismatch: $ACTUAL_RULES_SHA256" }
if (Test-Path $RULES_EXPANDED) { Remove-Item $RULES_EXPANDED -Recurse -Force }
Expand-Archive -Path $RULES_ZIP -DestinationPath $RULES_EXPANDED -Force
$EXPECTED_EP_DIRS=@('NvTensorRTRTXExecutionProvider_GPU','OpenVINOExecutionProvider_CPU','OpenVINOExecutionProvider_GPU','OpenVINOExecutionProvider_NPU','QNNExecutionProvider_GPU','QNNExecutionProvider_NPU')
$ACTUAL_EP_DIRS=@(Get-ChildItem $RULES_EXPANDED -Directory | Sort-Object Name | Select-Object -ExpandProperty Name)
if (($ACTUAL_EP_DIRS -join "`n") -ne (($EXPECTED_EP_DIRS | Sort-Object) -join "`n")) { throw "Unexpected rules package layout: $($ACTUAL_EP_DIRS -join ', ')" }
$RULE_FILES=@(Get-ChildItem $RULES_EXPANDED -Recurse -File -Filter '*.parquet')
if ($RULE_FILES.Count -ne 1746) { throw "Unexpected rules file count: $($RULE_FILES.Count)" }
$env:WINMLCLI_RULES_DIR=(Resolve-Path $RULES_EXPANDED).Path
winml analyze --model $OUT/fp32/model.onnx --ep all --device all --output $OUT/analyze_fp32.json --overwrite --format json --no-color
winml analyze --model $OUT/fp16/model.onnx --ep all --device all --output $OUT/analyze_fp16.json --overwrite --format json --no-color
Remove-Item Env:WINMLCLI_RULES_DIR

winml perf -m $OUT/fp32/model.onnx --ep cpu --device cpu --precision fp32 --input-data $OUT/semantic_inputs.npz --iterations 20 --warmup 3 --memory --output $OUT/perf_fp32.json --overwrite --format json --no-color
winml perf -m $OUT/fp16/model.onnx --ep cpu --device cpu --precision fp16 --input-data $OUT/semantic_inputs.npz --iterations 20 --warmup 3 --memory --output $OUT/perf_fp16.json --overwrite --format json --no-color
winml eval -m $OUT/fp32/model.onnx --model-id opencv/inpainting_lama --task inpainting --dataset mattmdjaga/human_parsing_dataset --dataset-revision db120bb5c18c146a8fbd2160f7575a288269fe7d --split train --samples 1 --no-shuffle --device cpu --ep cpu --output $OUT/eval_fp32.json --overwrite --format json --no-color
winml eval -m $OUT/fp16/model.onnx --model-id opencv/inpainting_lama --task inpainting --dataset mattmdjaga/human_parsing_dataset --dataset-revision db120bb5c18c146a8fbd2160f7575a288269fe7d --split train --samples 1 --no-shuffle --device cpu --ep cpu --output $OUT/eval_fp16.json --overwrite --format json --no-color

[ssss141414]

REQUEST_CHANGES — independent reviewer verdict

PR: #1159
Reviewed head: 135f982c522bd9552b2d3e23656c9bbdc86f6545
Current origin/main / frozen baseline / merge base: 38767add6f91c7b10b6394fae3af6f437e02effd (git rev-list --count <baseline>..origin/main = 0)

Delivery status: posting was attempted as reviewer identity shzhen_microsoft through both GraphQL/CLI and REST. GitHub rejected formal review and fallback issue-comment mutations with HTTP 403: Unauthorized: As an Enterprise Managed User, you cannot access this content. The verdict is preserved here for manual maintainer/orchestrator delivery; the producer identity was not used as a substitute.

Blocking findings

1. [producer] Required CI type checking fails on PR-introduced code. The completed Lint/lint job fails uv run mypy -p winml.modelkit with two diagnostics: src/winml/modelkit/commands/build.py:949 accesses .export on WinMLBuildConfig | list[WinMLBuildConfig] without narrowing, and src/winml/modelkit/inference/pipeline.py:74 passes WinMLPreTrainedModel | WinMLCompositeModel to a factory accepting only WinMLPreTrainedModel. Fix both and rerun the required lint job. CI is not pending: 8 checks passed and this 1 required check failed.

2. [producer] Optional bare-repository discovery breaks the established offline/cached-HF fallback. resolve_hf_repo_onnx() catches only HfHubHTTPError at src/winml/modelkit/loader/onnx_hub.py:59-73, while all newly changed general entry points enable discovery. An independent probe patched HfApi.model_info to raise OfflineModeIsEnabled; resolve_model_input("microsoft/resnet-50", discover_repo_onnx=True) propagated it (offline_fallback_probe_exit=7) instead of falling through to the existing loader/cache path. Catch the Hub offline/connection preflight failures that are safe to defer, preserve ambiguity errors, and add regression coverage proving a cached/normal HF ID remains usable offline.

3. [producer] load_schema_only() now downloads model weights despite its public contract. Its docstring promises “without downloading model weights,” but src/winml/modelkit/inference/engine.py:393-403 calls resolve_model_input(..., discover_repo_onnx=True), whose single-ONNX path downloads the full artifact. For this model that is a ~92.6 MB graph merely to display the already explicit inpainting schema. Split metadata discovery from artifact download or bypass download when an explicit task is sufficient; add a test asserting schema-only does not invoke the ONNX downloader.

4. [producer; planner only if the fix changes the chartered class-wide scope] The globally registered inpainting adapter silently assumes an OpenCV/LaMa-specific data contract that ONNX metadata cannot establish. src/winml/modelkit/inference/inpainting.py:23-29,105-129 derives only names/shapes, then unconditionally applies BGR [0,1], nonzero-mask semantics, and BGR [0,255] output handling. src/winml/modelkit/inference/pipeline.py:43-49 registers that adapter task-wide for every inpainting model. A same-shaped RGB, [-1,1], inverted-mask, or differently scaled model would execute without an error and return wrong pixels. Make preprocessing/postprocessing semantics explicit and data-driven (or constrain/reject models whose contract cannot be proven) and test at least one non-LaMa contract or explicit rejection path.

5. [explainer] The PR body is not yet self-contained/reproducible per the reviewer contract. The Baseline section reports the version/commit/result but omits the exact baseline winml build -m opencv/inpainting_lama ... command and the required winml --version and git rev-parse HEAD commands. The reproduce block also sets $INPUTS to a prose placeholder (<named semantic NPZ ...>), so the published perf commands cannot run as written. Add the exact baseline commands/results and a portable command that creates the deterministic named NPZ (or reference a committed/public input artifact). Do not expose internal temp/ handoff paths.

Independent evidence and passed gates

• Fresh checkout: reviewed a new clone at C:\repo\winml-cli-pr1159-independent-review; the dirty producer tree was not used or modified. Diff is 23 files / 750 insertions / 48 deletions, limited to the two recipes, generalized source, and tests. Production recipe README and skill/agent files are unchanged.
• Conversation gate: enumerated 0 line comments, 0 issue comments, 0 submitted reviews, and 0 review threads; open threads = 0; GraphQL hasNextPage=false.
• Shipment gate: PR remains Draft, author is ssss141414, label model-scale-by-skill is present, head matches the reviewed commit, and mergeability is MERGEABLE. This review does not promote it.
• Frozen handoffs: read planner charter, producer deliverable, tester verdict table, learner findings, model-breakdown JSON, both tester build configs/manifests, and analyze/component/op evidence. The model-breakdown identity/revision/architecture and reported 18,001/18,436-node component coverage agree with the handoffs. Issue Add inpainting evaluator, dataset schema, and canonical metric #1158 is open; Lane A methodology PR P0-REL-E: Release Execution — Tag, PyPI Publish, Announce, Internal Docs Archive #170 is open, Draft, and labeled.
• Build/structure: independently rebuilt recipe-free bare-ID and fp16 paths from PR head; both exit 0 and contain Build complete. Recipe-free SHA-256 is 7df918ac...fdf2 with FLOAT=606, INT8=598, INT64=299. FP16 SHA-256 is e47df3c...0d21, size 77,140,951 bytes, and the log shows an executed FP16 stage; FLOAT16=606, INT8=598, INT64=299. Both have IR 8, opset 21, named image/mask inputs and output with the claimed shapes, and immutable Hub revision aee6d22f...6f3f in the manifest.
• Perf/eval: independent fp16 semantic named-input CPU perf exits 0: mean 2588.404 ms, p50 2624.924 ms, throughput 0.39 samples/s, RSS delta 727.83 MB; detector reports w8a8, consistent with the preserved INT8 source weights. Independent eval --schema --task inpainting exits 2 with the claimed unsupported-task blocker.
• Tests/lint: focused affected suite passes (286 passed in 108.49s). Ruff passes all 21 changed Python files. Full-repo Ruff reports the same six unrelated diagnostics on clean current origin/main, so they are recorded as pre-existing and are not charged to this PR. The required mypy/CI failure above is PR-introduced and blocking.
• Evidence/body positives: required section hierarchy, metadata tree, CPU fp32/fp16 tuple rows, mixed-precision disclosure, perf/memory values, eval blocker/dataset revision, recipe delta, component/op summaries, methodology declaration, and exact Draft/label state are present.

Ownership routing

• Producer: findings 1–4; add/adjust focused tests, rerun changed-file Ruff, uv run mypy -p winml.modelkit, focused pytest, bare-repo build, mixed INT8+FP16 artifact audit, and schema-only/offline probes.
• Explainer: finding 5 only; update the portable PR body after producer evidence is refreshed. Keep Draft and retain model-scale-by-skill.
• Tester: after producer fixes, independently reverify both CPU tuples and refresh any changed build/perf evidence; do not inherit producer results.
• Learner: update findings only if the fixes reveal new methodology/task-family knowledge; otherwise retain the frozen Lane A record.
• Planner: re-issue the charter only if resolving finding 4 changes the generalized fix class or Goal/Outcome scope.
• Reviewer: re-enter from a new checkout only after all routed actions land and all review threads plus required CI are green.

[ssss141414]

REQUEST_CHANGES — independent reviewer re-review

PR: #1159
Reviewed head: 7859d8e024c8cb9f718593c9a16376176a0bd87c
Fresh detached worktree: C:\repo\winml-cli-pr1159-independent-review-r2

Blocking finding

1. [planner/tester/explainer] The frozen baseline is stale against current origin/main. The PR body and frozen handoffs cite baseline 38767add6f91c7b10b6394fae3af6f437e02effd, but the final mandatory refresh fetched current main 4daf0f19097d03e92aeb3c4f9713b7d037e0be3d; git rev-list --count 38767add6f91c7b10b6394fae3af6f437e02effd..origin/main returned 1. The new commit is ci(e2e): move daily E2E Test schedule to 02:00 (UTC+8) (#1160) and changes only .pipelines/Modelkit E2E Test.yml, but the reviewer contract _meta-052 is explicit: any nonzero drift requires rebase onto current main and a refreshed baseline. Rebase, rerun the exact recipe-free baseline and required tuple evidence, refresh the frozen handoffs/PR body to the new main SHA, and let CI rerun. Keep the PR Draft and retain model-scale-by-skill.

Prior REQUEST_CHANGES disposition

All five prior findings are independently verified fixed at the reviewed head:

1. Mypy: fixed. Fresh python -m mypy -p winml.modelkit exits 0: Success: no issues found in 407 source files; changed-file Ruff and format checks also exit 0. GitHub Lint/lint is COMPLETED/SUCCESS.
2. Offline fallback: fixed. Optional direct-ONNX discovery now catches Hub offline/connectivity failures while preserving multi-artifact ambiguity. The exact resolve_model_input(..., discover_repo_onnx=True) offline regression and resolver ambiguity tests pass.
3. Schema-only no download: fixed. An explicit task bypasses repository artifact discovery/download; focused tests assert both explicit Hub-artifact and bare-repository paths do not call the downloader.
4. Persisted fail-closed runtime semantics: fixed. Recipes explicitly declare tensor roles, RGB/BGR order, input/output ranges, and mask polarity. Fresh builds persist the same contract in both winml_build_config.json and build_manifest.json; build-directory loading restores it. Missing/malformed/unknown contracts fail closed, and an alternate RGB/[-1,1]/inverted-mask contract is covered.
5. Exact baseline commands and portable public reproduction: fixed. The live body includes winml --version, git rev-parse HEAD, the exact no-recipe baseline build command/result, deterministic NPZ creation, both builds, raw perf, public runtime-adapter execution, downloadable public rules, analyze, and eval commands. Its 13-command reproduce block is byte-for-byte the tester-owned public_reproducible_commands join and contains no private handoff path or unresolved placeholder.

Independent evidence

• Discussion gate: enumerated 0 line comments, 1 conversation comment (the prior structured verdict), 0 submitted reviews, and 0 review threads; unresolved threads = 0; GraphQL hasNextPage=false. The prior verdict is answered by commit 7859d8e0, refreshed handoffs/body, and the independent checks above.
• Scope/design: 29 changed files: two recipes plus generalized direct-ONNX/build/runtime code and focused tests; production recipe README and skill/agent files are unchanged. The inpainting adapter is now metadata-driven rather than model-ID-driven, rejects absent/unknown contracts, validates declared names against graph I/O, and keeps task-wide dispatch generic.
• Focused tests: 83 independently selected prior-finding/config/manifest/programmatic-loading tests pass in 13.20 s.
• Fresh builds: recipe-free, fp32 recipe, and fp16 recipe all exit 0 and print Build complete. Recipe-free/fp32 SHA-256 is 7df918ac3921d3daf0aae1d219776cf0dc4e4935f035af81841b40adcf74fdf2 (92,591,623 bytes); fp16 is e47df3c0866782a9c2f22c0635071110f025e26e15172749189841c09ed30d21 (77,140,951 bytes) with an executed FP16 stage.
• Artifact audit: both pass onnx.checker, use IR 8/opset 21, and expose named FLOAT image [batch,3,512,512], mask [batch,1,512,512], and output [batch,3,512,512]. fp32 has FLOAT=606/INT8=598/INT64=299 initializers; fp16 has FLOAT16=606/INT8=598/INT64=299.
• Public runtime probes: both fresh build directories load through InferenceEngine using the persisted contract and execute semantic image+mask inference on CPU, returning 512×512 PNG output.
• CI at reviewed head: CodeQL, Lint, all five WinML CLI CI shards, policy CodeQL, and CLA are COMPLETED/SUCCESS.
• Shipment state: head still matches 7859d8e0; PR is Draft and has model-scale-by-skill. It was not promoted or edited.

This is a single freshness blocker only. Re-review is required after the rebase and refreshed baseline evidence.

[ssss141414]

The stale-baseline request has been addressed without changing the committed scope or Goal ceiling. The branch is rebased onto current main at 4daf0f19097d03e92aeb3c4f9713b7d037e0be3d; the refreshed feature head is 28815d61b548f3b255fa6be169c69e62020676a0, with that current-main commit as its merge base.

The replacement charter, recipe-free baseline, both required CPU fp32/fp16 build and perf tuples, supplementary ONNX-reference comparison, eval blockers, analyze summaries, and five quality checks were refreshed against that exact head/base pair. The PR description now transcribes those owning artifacts, including the new baseline/per-tuple values and portable reproduction commands. The PR remains Draft with model-scale-by-skill; this update reports the evidence refresh and does not claim reviewer approval.

[ssss141414]

The explainer-owned PR prose has been refreshed for repaired head eca1e3b5896adedc2e93d7ed6016364399febe2f without changing tester-owned measurements.

• The unresolved rules placeholder is gone. The reproduction block now downloads the official v0.2.0 release asset rules-v0.2.0.zip, checks its published SHA-256 (6dcabbe7f5493fbc2bd23de195ab6e35b3578d510f18c2e220bc5538a1f232cc), verifies the six expected EP directories and 1,746 Parquet files, deterministically sets WINMLCLI_RULES_DIR to the resolved expansion root, and runs both analyze commands. The asset name, URL, digest, and expanded layout were verified against the public GitHub release API and a fresh download.
• The Baseline section now includes literal public PowerShell commands for the frozen current-main checkout, WinML version, git rev-parse HEAD, the recipe-free bare repository-ID build, and the pinned direct-ONNX diagnostic build. The prose and command comments preserve the distinction: bare opencv/inpainting_lama fails on current main; the pinned direct ONNX succeeds diagnostically but does not upgrade bare-ID support.
• The body now identifies the repaired head and records the contract-neutral public mask schema plus the post-repair quality/provenance checks. Pre-repair perf, eval, L2 comparison, component, and op evidence remains unchanged only where the tester verified artifact/evidence identity.

The PR remains Draft with model-scale-by-skill. This reports the requested prose fixes and requests re-review; it does not claim reviewer approval.

[ssss141414]

Both final-r4 blockers have been addressed at current head f03add820f384ce725e83400d53d108ff43aad0a.

1. The public contract is now explicit and directional: caller mask nonzero pixels are always holes (the replaced region), while persisted runtime.options.mask_semantics controls model-side ONNX tensor polarity/conversion. The canonical mask is passed through for nonzero-is-hole and inverted for zero-is-hole, preserving the same semantic hole coordinates. Fresh tester-r5 evidence passed the 3 focused semantic-contract regressions, all 28 task/inpainting tests, and the 369-test broader affected suite; full ModelKit mypy, Ruff, Ruff format, and diff checks also pass.
2. Both public analyze reproductions now use the tester-owned literal shape --ep all --device all while retaining the verified v0.2.0 rules download, SHA-256, six-directory, 1,746-Parquet, and deterministic WINMLCLI_RULES_DIR setup. Fresh fp32/fp16 runs each retained exit 1 because OpenVINO plugin registration failed with Error 126, but each emitted complete fail-closed component/op JSON with 11 unique nonempty rows, including all six required public rule-backed targets. The body labels this ANALYZE-PARTIAL-SUCCESS and does not treat static classifications as runtime compatibility.

The PR description has been refreshed from producer-r5/tester-r5, replacement charter r2, and learner-r2. Concrete build/perf/runtime/eval values remain unchanged only where tester r5 explicitly validated carryover. The PR remains Draft with model-scale-by-skill; please re-review both fixes. This update does not claim reviewer approval.