recipe(cross-encoder): add MiniLM L2 text-classification recipes

[kujin66]

Summary

Adds recipe coverage for cross-encoder/ms-marco-MiniLM-L2-v2 as a BERT sentence-pair text-classification cross-encoder. This is an L0 recipe-only contribution: current origin/main already builds the model, and the checked-in recipes make the verified CPU and DML GPU fp32/fp16 tuples discoverable and reproducible. The refreshed Goal ladder reaches L2 on CPU; L3 task-metric eval is blocked by dataset/label-semantics mismatch for this MS MARCO single-logit relevance checkpoint.

Model metadata

What the model does

cross-encoder/ms-marco-MiniLM-L2-v2 scores the relevance of a query/document text pair using a BERT-family sequence-classification head. Evidence/confidence: Hugging Face config resolves architectures=['BertForSequenceClassification'], model_type='bert', task text-classification; confidence: verified from checkpoint config and WinML inspection/build behavior.

Primary user stories

• A retrieval or ranking application supplies a query plus candidate passage and receives a relevance logit for reranking. Evidence/confidence: model id and checkpoint class are MS MARCO cross-encoder sequence classification; confidence: mapped.

Supported tasks

• text-classification, with sentence-pair inputs input_ids, attention_mask, and token_type_ids, output logits [1, 1]. Support surfaces: checkpoint, transformers, Optimum ONNX, WinML. Evidence/confidence: Optimum probe for bert includes text-classification; WinML config/build emitted AutoModelForSequenceClassification; confidence: verified.

Model architecture

BertForSequenceClassification
|-- BertModel
|   |-- BertEmbeddings
|   |-- BertEncoder x 2 layers
|   |   |-- Self-attention
|   |   |-- Intermediate GELU feed-forward
|   |   `-- Output projection + LayerNorm
|   `-- BertPooler
`-- Sequence-classification head -> logits [1, 1]

• Source/confidence: checkpoint config and HTP export metadata reported 15,615,745 parameters, 51 modules, 19 traced modules, and 137 tagged export nodes before optimization; confidence: verified.

Validation and support evidence

Baseline

• Base refreshed to current origin/main: 5deebd422e95f28fe8fd912ee50ce874710187b3.
• PR head after fix: d70a4d0d4bb5788b98deaf16ca881437067ccdf4.
• WinML version: 0.2.0.
• Baseline build without recipe passed on CPU: winml build -m cross-encoder/ms-marco-MiniLM-L2-v2 --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild completed with Build complete in 12.7s; model I/O was input_ids, attention_mask, token_type_ids [1, 512] int32 -> logits [1, 1].
• Starting auto-config recipe: winml config -m cross-encoder/ms-marco-MiniLM-L2-v2 emitted a BERT text-classification config with the same I/O, loader.model_class=AutoModelForSequenceClassification, loader.model_type=bert, and optim.clamp_constant_values=true.
• Optimum coverage probe for bert: vendor and after-WinML tasks are feature-extraction, fill-mask, multiple-choice, question-answering, text-classification, token-classification; added_by_winml=[], verdict VENDOR-ONLY.

Goal

• Effort: L0 recipe-only.
• Goal ceiling: L2, because current main already provides L0 build support and CPU can reach L1 perf and L2 numeric comparison.
• Outcome: L0 checked-in recipe coverage for exact PASS tuples.
• Target precision plan: CPU/cpu fp32, CPU/cpu fp16, DML/gpu fp32, DML/gpu fp16.

Outcome

• Highest reached Goal verdict: L2 PASS on CPU.
• Coverage: full for targeted tuples CPU/cpu/fp32, CPU/cpu/fp16, DML/gpu/fp32, and DML/gpu/fp16.
• Shipped recipe paths:
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/cpu/cpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/cpu/cpu/text-classification_fp16_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/dml/gpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/dml/gpu/text-classification_fp16_config.json
• examples/recipes/README.md remains untouched by this PR.
• Methodology friction observed: none. Existing skill guidance covered the README scope issue, baseline refresh, fp16 CLI flag, perf memory capture, and analyze rule setup.

Per-EP/device/precision results

Providers snapshot: ['DmlExecutionProvider', 'CPUExecutionProvider'].

Tier	EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM delta	VRAM delta	Task metric
L0	CPUExecutionProvider / cpu	fp32	PASS	-	-	-	-	-	Build complete in 89.7s; fp16 initializers 0/60; artifact 62,489,557 bytes
L0	CPUExecutionProvider / cpu	fp16	PASS	-	-	-	-	-	Build complete in 93.7s; fp16 initializers 44/60; artifact 31,258,173 bytes
L0	DmlExecutionProvider / gpu	fp32	PASS	-	-	-	-	-	Build complete in 85.8s; fp16 initializers 0/60; artifact 62,489,557 bytes
L0	DmlExecutionProvider / gpu	fp16	PASS	-	-	-	-	-	Build complete in 92.8s; fp16 initializers 44/60; artifact 31,258,173 bytes
L1	CPUExecutionProvider / cpu	fp32	PASS	35.56 ms	35.68 ms	28.12 samples/s	+146.2 MB total	n/a	-
L1	CPUExecutionProvider / cpu	fp16	PASS	44.95 ms	44.89 ms	22.25 samples/s	+139.6 MB total	n/a	-
L1	DmlExecutionProvider / gpu	fp32	PASS	52.61 ms	59.71 ms	19.01 samples/s	+195.3 MB total	+68.9/+148.5 MB local/shared total	-
L1	DmlExecutionProvider / gpu	fp16	PASS	33.25 ms	32.72 ms	30.07 samples/s	+97.9 MB total	+70.9/+54.8 MB local/shared total	-
L2	CPUExecutionProvider / cpu	fp32	PASS	-	-	-	-	-	PyTorch vs ONNX named-input comparison: cosine 1.0, max_abs 0.0
L2	CPUExecutionProvider / cpu	fp16	PASS	-	-	-	-	-	PyTorch vs ONNX named-input comparison: cosine 1.0, max_abs 0.0017414093
L3	CPUExecutionProvider / cpu	fp32	CLI-BLOCKED-DATA-SEMANTICS	-	-	-	-	-	Attempted glue/mrpc validation, 5 samples, sentence-pair columns; eval failed during label alignment: dataset label equivalent does not align to this checkpoint's MS MARCO single-logit relevance label space.

Delta

Compared with the refreshed auto-config recipe from winml config -m cross-encoder/ms-marco-MiniLM-L2-v2:

JSON pointer	Auto-config value	Shipped value	Reason
/optim/clamp_constant_values	true	absent in checked-in recipes; build autoconf applies gelu_fusion and matmul_add_fusion	Checked-in recipe stays minimal; build fills optimization defaults.
/quant for fp16 recipes	null	mode=fp16, fp16_keep_io_types=true	Precision-specific coverage artifact; build command also uses --precision fp16, and artifact validation confirms fp16 initializers and half-size model.
path bucket	default config selected DML/GPU on this host	explicit cpu/cpu and dml/gpu recipe copies	Recipe paths are exact verified EP/device/precision coverage claims.

Reducibility: recipe-only remains appropriate. The model family is already vendor-covered and WinML builds it without source changes; these files are verified coverage artifacts for exact EP/device/precision tuples, not a class-wide code fix.

Analyze summary

Static rule analysis completed with populated v0.2.0 runtime rules and emitted JSON. The command returned non-zero while still writing complete per-EP records; treated as ANALYZE-PARTIAL-SUCCESS because rule-less EP rows are all-unknown, while rule-backed rows are complete.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP findings
fp32 optimized graph	embeddings; 2x BERT encoder attention/FFN; pooler/classification output	137 export nodes tagged before optimization; optimized graph has 78 ops / 16 types	TensorRT/QNN/OpenVINO: all 78 ops supported; CUDA/MIGraphX/DML: no rule data, all 78 unknown in static rules

Op-level summary

Artifact	Graph	Dominant ops	EP roll-up
fp32 optimized graph	78 ops / 16 unique types	Reshape 20; Gemm 14; Add 8; Transpose 8; LayerNormalization 5; MatMul 4; Mul 4	NvTensorRTRTX, QNN, OpenVINO: 78 supported / 0 partial / 0 unsupported / 0 unknown. CUDA, MIGraphX, DML: 0 supported / 0 partial / 0 unsupported / 78 unknown due to missing rule data.

Reproduce commands

$OUT='temp/pr1143-repro'
uv run winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/cpu/cpu/text-classification_fp32_config.json -m cross-encoder/ms-marco-MiniLM-L2-v2 -o $OUT/cpu_fp32 --ep cpu --device cpu --rebuild
uv run winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/cpu/cpu/text-classification_fp16_config.json -m cross-encoder/ms-marco-MiniLM-L2-v2 -o $OUT/cpu_fp16 --ep cpu --device cpu --precision fp16 --rebuild
uv run winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/dml/gpu/text-classification_fp32_config.json -m cross-encoder/ms-marco-MiniLM-L2-v2 -o $OUT/dml_fp32 --ep dml --device gpu --rebuild
uv run winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/dml/gpu/text-classification_fp16_config.json -m cross-encoder/ms-marco-MiniLM-L2-v2 -o $OUT/dml_fp16 --ep dml --device gpu --precision fp16 --rebuild
uv run winml perf -m $OUT/cpu_fp32/model.onnx --ep cpu --device cpu
uv run winml perf -m $OUT/cpu_fp16/model.onnx --ep cpu --device cpu
uv run winml perf -m $OUT/dml_fp32/model.onnx --ep dml --device gpu
uv run winml perf -m $OUT/dml_fp16/model.onnx --ep dml --device gpu
uv run winml analyze --model $OUT/cpu_fp32/model.onnx --ep all --output $OUT/analyze_all.json
uv run winml eval -m $OUT/cpu_fp32/model.onnx --model-id cross-encoder/ms-marco-MiniLM-L2-v2 --task text-classification --dataset glue --dataset-name mrpc --split validation --samples 5 --column input_column=sentence1 --column second_input_column=sentence2 --column label_column=label --ep cpu --device cpu --output $OUT/eval_cpu_fp32_mrpc.json --overwrite --format json

Quality gates

• uv run ruff check src/ tests/: PASS, All checks passed!

• uv run mypy -p winml.modelkit: PASS, Success: no issues found in 415 source files.

• uv run pytest tests/unit/models tests/unit/loader tests/unit/datasets tests/unit/export --tb=short --no-cov -m "not e2e and not npu and not gpu": PASS, 1351 passed, 6 skipped, 1 xfailed.

• uv run pytest tests/unit/commands tests/unit/config tests/unit/build tests/unit/compiler tests/unit/session tests/unit/eval --tb=short --no-cov -m "not e2e and not npu and not gpu": PASS, 2373 passed, 8 skipped, 2 deselected.

• Rebase freshness note: after rebase from 3ad2863 to d70a4d0, git diff 3ad2863..HEAD -- examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2 was empty, so the full four-tuple build/perf/analyze evidence remains tied to byte-identical recipe files; baseline, ruff, and mypy were rerun on the refreshed head.

[kujin66]

APPROVE (coverage: full for target EPs cpu and dml-gpu)

Reviewer checks performed:

• PR exists and is draft: recipe(cross-encoder): add MiniLM L2 text-classification recipes #1143.
• PR body contains the 9 required hand-off items: recipe paths, README row, build output dirs, build logs, findings, Optimum probe, claimed E/G/O, Goal ladder, and methodology declaration.
• Baseline freshness: git fetch origin main; current origin/main is 238bd386d77381b70e03377295a1f40a69982576; PR body baseline head matches it; git rev-list --count 238bd386d77381b70e03377295a1f40a69982576..origin/main returned 0.
• Diff scope for L0 recipe-only is correct: examples/recipes/README.md plus four JSON files under examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/{cpu/cpu,dml/gpu}/; no source or test code included.
• Recipe schema and layout are consistent with existing multi-EP cross-encoder L4/L6 recipes.
• Evidence spot-check: DML fp16 perf JSON reports DmlExecutionProvider, precision fp16, mean latency 13.416 ms, throughput 74.54 samples/s.
• Artifact validation evidence includes expected I/O and true FP16 initializer counts for fp16 artifacts (46/52).
• Analyze was host-blocked by missing runtime rule parquet files; the PR body cites the exact CLI error and mines export_htp_metadata.json / winml_build_config.json instead, matching reviewer guidance.

No requested changes.