Problem
winml eval does not register the canonical inpainting task, although runtime inference can use an image + binary mask contract.
Both eval --schema --task inpainting and artifact evaluation reject the task before dataset loading. This prevents task-metric evidence for direct-ONNX inpainting models such as opencv/inpainting_lama.
Requested capability
- Register an
inpainting evaluator and schema.
- Define the required image, mask, and ground-truth restored-target columns.
- Document a canonical metric or metric set for inpainting quality.
- Keep input preprocessing aligned with the runtime image + mask contract.
Reproduction
winml eval --schema --task inpainting
Observed with WinML CLI 0.2.0:
Task 'inpainting' is not supported by winml eval.
A public image+mask dataset can exercise input shape, but a valid evaluator also needs authoritative restored targets and a documented metric; scores must not be inferred from an input-only dataset.
Problem
winml evaldoes not register the canonicalinpaintingtask, although runtime inference can use an image + binary mask contract.Both
eval --schema --task inpaintingand artifact evaluation reject the task before dataset loading. This prevents task-metric evidence for direct-ONNX inpainting models such asopencv/inpainting_lama.Requested capability
inpaintingevaluator and schema.Reproduction
winml eval --schema --task inpaintingObserved with WinML CLI 0.2.0:
Task 'inpainting' is not supported by winml eval.A public image+mask dataset can exercise input shape, but a valid evaluator also needs authoritative restored targets and a documented metric; scores must not be inferred from an input-only dataset.