Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What: the classification validator reports macro precision, recall and F1 from a confusion matrix.
Why: it only had top-1/top-5. Lands part of #852 focused on the classifier changes.
ClassifyValidator: per-batch confusion matrix (in-placeindex_add_, cost independent of class count). New keysmetrics/precision,metrics/recall,metrics/f1: unweighted mean over classes present in the validation targets; precision is 0 for a never-predicted class. Same key names as the point validator.fitness(top-1) are unchanged, so best-checkpoint selection and early stopping are unchanged.libreyolo valclassify output gainsprecision,recall,f1in--jsonand in the text line, only when the validator returned them (that branch whitelists keys);results.csvand loggers see three new metric columns.val()docstring, a "Validation metrics" section indocs/classification_training.md(the only classify training doc today), and a CHANGELOG entry under Unreleased.Shared code:
ClassifyValidatoris where classify metrics are computed for every family, so the change has to live there. It is inherited by the CLIP, SigLIP2 and ViT classify validators and used by every classify trainer and exported backend. Keys are additive and nothing enumerates classify keys with a fixed list. Under DDP, classify validation already runs on rank 0 over the whole split, so the matrix needs no reduction.Check: the macro definition in
_macro_precision_recall_f1(classes absent from the targets are excluded, which differs from a union-of-labels macro average on subset splits); the out-of-range target mask in_update_metrics; key names.Verified: full PR gate locally (macOS, CPU), 7437 passed, plus 14 distributed.
Not verified: no GPU run.
Code provenance
Original code written for this PR against LibreYOLO's own first-party code; no third-party code ported, adapted, or introduced; no GPL/AGPL/LGPL/non-commercial/unknown-license material involved.
The PR appears safe to merge, with the previous quadratic-memory problem fully resolved and no new actionable defects identified.
Summary
This PR adds macro precision, recall, and F1 to classification validation while preserving top-1 as the fitness and checkpoint-selection metric.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Classification logits and targets] --> B[Top-1 prediction] B --> C[Valid target-range mask] C --> D[Per-class true positives] C --> E[Per-class prediction counts] C --> F[Per-class target counts] D --> G[Per-class precision recall and F1] E --> G F --> G G --> H[Macro average over classes present in targets] H --> I[Python API logs CLI JSON and text]Reviews (2) · Last reviewed commit: "Accumulate per-class confusion counts in..."