feat(ml-worker): add disposition corpus export census - #3297
Merged
Merged
Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Owner
Author
|
The two failing checks on this PR are pre-existing breakage on
Every PR running the backend-service job is red until that is fixed. Not fixed here to keep this diff to its own scope. |
Export closed operator dispositions through one read-only Elasticsearch PIT, redact free-text reasons by default, and report a hashed census with explicit precision-only and calibration gates. Add fixture coverage for denominators, missing fields, class balance, and safe transport errors. Closes #3295
Accept Elasticsearch 9 date-stat bounds returned alongside the aggregation and preserve the exact ISO range in the census.
Handle both Elasticsearch 8 nested stats and Elasticsearch 9 flattened date-stat responses so the live census reports its actual alert window.
Ensure live census reports fixed open and closed status buckets even when Elasticsearch omits zero-count terms.
Clarify that disposition reasons are redacted by default and only exposed through an explicit operator opt-in.
Return the atomic report digest consistently with the existing benchmark convention and document the independent report-file hash.
Keep model grouping and missing-field census honest when every detector abstained but the model_scores object is present.
Compute the census report hash over canonical content with the digest field excluded, store it in the report, and print it with the snapshot digest. Add coverage and usage documentation for the report artifact contract.
Xore
force-pushed
the
issue-3295-coder
branch
from
September 25, 2026 11:44
d3f40db to
0300449
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ml-anomaliesexport/census command with one point-in-time snapshot for status totals and closed dispositionsValidation
python3 -m pytest ml-worker/tests/test_disposition_corpus.py -q— 11 passedpython3 -m pytest ml-worker/tests/ -q— 319 passedopen, zero labels, time range2026-09-05T22:00:04.700Z–2026-09-25T07:12:42.167ZCloses #3295