[Maturity KPI Gap] fidelity_gate_pass_rate_pct critical gap

[ErenAri]

KPI metric

fidelity_gate_pass_rate_pct

Current value

0.0

Target value

= 98.0

Severity

critical

Impact

Recommendation and promotion credibility is blocked because fidelity evidence currently fails in generated reports.

Probable root cause

Fidelity configuration and/or calibration is not aligned with current generated trace behavior; baseline reports in outputs/**/*fidelity*_report.json are failing.

Remediation plan

• Owner: ml-systems, due 2026-03-15: produce trace-by-trace failure breakdown and candidate threshold adjustments.
• Owner: simulation-core, due 2026-03-22: verify invariant and queue-series assumptions against failing reports.
• Owner: qa-release, due 2026-03-29: add KPI trend check in milestone review workflow.

[ErenAri]

Progress update (2026-02-26): first remediation slice is merged to main.

Implemented:

• commit 32ffcb6 - stabilize fidelity gating for low-sample traces.
• Added denominator-floor support for aggregate divergence (relative_divergence) to avoid ratio explosions on tiny absolute deltas.
• Added low-sample distribution guard in fidelity gate (sample_size < 30 skips KL/KS checks and reports metadata).
• Added transparency fields in fidelity report: distribution_checks.sample_size, min_sample_size_required, skipped_small_sample.
• Added tests:
  • test_fidelity_gate_small_fixture_skips_distribution_noise
  • test_relative_divergence_respects_denominator_floor

Verification:

• pytest tests/unit/test_fidelity_gate.py tests/unit/test_fidelity_properties.py tests/unit/test_reproducibility_suite.py::test_fidelity_gate_is_deterministic -q
• Result: 10 passed
• Sample-trace baseline fidelity probe now reports status=pass for both policies.

Next actions on this issue:

1. Run fidelity gate across the full reference trace set and publish pass-rate delta.
2. Build a fail-reason histogram artifact for traces still failing.
3. Tune thresholds only where diagnostics show persistent true positives.