Skip to content

fix(ml-worker): replace hard clip in isolation_forest normalization so tail scores stay spread, not pinned at 1.0 (#3169) - #3175

Merged
Xore merged 1 commit into
mainfrom
agent/issue-3169-coder
Sep 12, 2026
Merged

Xore merged 1 commit into
mainfrom
agent/issue-3169-coder

Conversation

@Xore

@Xore Xore commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Clip past p99 collapsed 52.4% of alerts onto identical score 1.0.

Replaces the hard clip with continuous slope-matched asymptotic decay:
threshold and weights untouched, p50 still maps to 0.2, p99 still maps
to 0.95, but tail scores stay spread out past p99 instead of pinning.
Also clamps the batch-path exp overflow.

301 tests pass, including new discriminating saturation tests.

The ~20% ceiling acceptance criterion is a post-deploy field
measurement (Europe/Berlin), tracked as follow-up.

Closes #3169

…o tail scores stay spread, not pinned at 1.0 (#3169)

Clip past p99 collapsed 52.4% of alerts onto identical score 1.0.
Swap for continuous slope-matched asymptotic decay: threshold and
weights untouched, p50 still maps to 0.2, p99 still maps to 0.95, but
tail scores stay spread out past p99 instead of pinning. Also clamps
the batch-path exp overflow.

The ~20% ceiling acceptance criterion is a post-deploy field
measurement, tracked as follow-up.

Closes #3169
@strix-security

strix-security Bot commented Sep 12, 2026 •

Copy link
Copy Markdown

Strix Security Review

No security issues found.

Updated for 2ca9254.


Reviewed by Strix
Re-run review · Configure security review settings

@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@strix-security strix-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed PR #3175, a numerical correction to the isolation-forest score normalization in ml-worker/models/isolation_forest.py (plus its tests). The change replaces the linear extrapolation-with-hard-clip past the p99 anchor with a continuous, slope-matched asymptotic decay toward 1.0, keeping the p50→0.2 and p99→0.95 anchors and the segment slope (0.75) unchanged.

Security review found no vulnerabilities. The modified functions (_percentile_normalize and _percentile_normalize_batch) take only internally-derived model scores and calibration percentiles (p50/p99 from the model's own holdout data), not untrusted web/request input, and return bounded values in [0,1]. The only numerical hazard — exp() overflow in the vectorized np.where path — is explicitly mitigated via np.maximum(frac - 1.0, 0.0) before the exponent, and the scalar path's exponent is provably non-positive on its branch. Division is guarded by the p99 == p50 early return. No auth, authorization, secret, injection, or serialization surfaces are touched. The test file changes add only assertions and the already-imported math module. Static analysis (bandit, gitleaks) reported no findings.


Reviewed by Strix
Configure security review settings

@Xore
Xore merged commit 17402f7 into main Sep 12, 2026
110 checks passed
@Xore
Xore deleted the agent/issue-3169-coder branch September 12, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ml: isolation_forest still 52.4% ceiling-pinned post-#3097 — residual saturation blocks ML_ALERT_THRESHOLD retune

1 participant