Make the validation sample-plot count configurable (#830)

[EHxuban11]

Closes #830.

What changed

• plot_samples on ValidationConfig and TrainConfig, and on the val and
  train CLI commands. It replaces the hardcoded _N_VAL_SAMPLES = 8 in the
  detection validator.
• 0 disables the sample-image plot, -1 keeps every validated image, default
  stays 8.
• Collection stays bounded by the requested count, so a small budget does not
  hold images in memory for the whole run. -1 on a large val set deliberately
  does; that is what the reporter asked for and the help text says so.

It cannot change the metrics

That was the explicit constraint on this issue, so it is what the tests are
built around. The budget is consulted in exactly one place, inside
_track_plots_data, which is already separate from scoring. The tests drive
the real _track_plots_data at budgets 0 / 1 / 8 / 20 / -1 and assert the
confusion matrix receives an identical call sequence every time, with only the
sample buffer length differing. Every image is still processed at every
setting, including plot_samples=0.

Naming

The reporter suggested n_samples. I used plot_samples instead, because
n_samples reads like "how many images to validate", which is exactly the
misreading this issue warns against. No ecosystem equivalent exists to match
(checked public docs only).

Verification

• 39 tests in tests/unit/test_val_plot_samples.py: accepted and rejected
  values, default equals the old hardcoded 8, budget arithmetic, metric
  invariance as above, a legacy config without the field keeping old behavior,
  and both CLI surfaces forwarding the value.
• pytest tests/unit -k "val or plot or coco or config or cli or trainer":
  1631 passed, 48 skipped.
• Not verified: no real validation run and no plot rendered; this covers the
  budget decision and the plumbing, not the plot image itself.

Contributor note

MARIANOCEREDA offered to take this on 2026-08-27 and was invited to. No PR
appeared in the three weeks since. Close this in favour of theirs if they turn
up with one.

Code provenance

Original code written for this PR. No third-party code was ported, adapted or
derived.

Confidence Score: 5/5

The PR appears safe to merge; the previously outstanding pose-budget behavior is now fixed and no new actionable failures remain.

Findings

1.  Pose ignores sample budget ▶

Summary

The PR replaces the fixed validation sample-plot limit with a validated plot_samples configuration shared by training, standalone validation, detection, and pose workflows.

• Supports bounded counts, 0 to disable collection, and -1 to retain every validated image.
• Preserves the prior default of eight samples and keeps sample collection separate from metric processing.
• Forwards the setting through both CLI commands and the standard and RF-DETR training validation paths.
• Adds unit coverage for configuration validation, CLI forwarding, collection limits, metric invariance, legacy defaults, and pose handling.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    CLI[train / val CLI] --> Config[TrainConfig / ValidationConfig]
    API[Python train / val API] --> Config
    Config --> Validation[Detection or Pose Validator]
    Validation --> Scoring[Score every image]
    Validation --> Budget{plot_samples}
    Budget -->|0| None[Keep no plot samples]
    Budget -->|positive N| Bounded[Keep at most N samples]
    Budget -->|-1| All[Keep every sample]

Loading

_{Reviews (3) · Last reviewed commit: "Merge remote-tracking branch 'origin/dev..."}

[greptile-apps]

Pose ignores sample budget

The shared plot_samples option is accepted for pose validation, but PoseValidator still hardcodes an eight-image limit. As a result, model.val(plot_samples=0) still produces pose sample plots, while values above eight or -1 still collect only eight images. Route the configured budget into pose sample collection or reject the option for that task.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!