Repository navigation
Add daily moderation quality statistics with late-event repair - #538
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Persist correction-based TP/TN/FP/FN counts for fast dashboard queries, separating actual pipeline totals, counterfactual ML, and matched ML/LLM comparisons by recorded deployment. Keep abstained, unresolved, and excluded cases visible.
The scheduled job atomically replaces today and the previous 14 UTC days, plus older days affected by edits, reviewer corrections, or unbans. An event-insert trigger queues work in the event transaction; resolving pending IDs into durable dirty days and acknowledging those IDs is atomic. This handles out-of-order commits without a high-water event-ID checkpoint, and failed rebuilds retain their dirty markers. All verdict interpretation remains in F#.
Authenticated POST /quality-history runs the daily refresh; ?day=YYYY-MM-DD rebuilds one day. Explicit ?mode=backfill fills older missing days in resumable batches (default seven), separately from the daily job. Includes service-role grants, dashboard SQL with provisional/freshness fields, and structured runtime logging with a warning above the 30-second daily budget.
Validation: Release build and nine PostgreSQL integration tests passed, covering corrected counts, reversal, unban, old edits, today's provisional counts, backfill boundaries, late commits with lower IDs, rollback, failed-rebuild retry, and endpoint authentication. Mandatory repository hooks passed.
Local synthetic benchmark using the actual job: 20,148 messages / 49,006 events across 15 days; daily refresh with 2,729 pending events took 1.15–1.45 seconds, and processing the full 49,006-event backlog took 1.71 seconds. Repeated aggregates matched the evaluator. These are local measurements; production end-to-end runtime remains to be verified after deployment.