Add an identification eval suite with offline fixture images and a coverage gate

[ChelseaKR]

What

Nine suites cover the answer pipeline; sprout identify (ADR-0010) has none. The design promises the visual match only selects a species, is always labelled "a visual match, not a cited fact", abstains when the match is not in the corpus, and shows rejected candidates (R8). Nothing measures those promises.

Add a tenth suite, identification. Fixtures are small committed images (synthetic or CC0), each labelled in-corpus species, out-of-corpus species, or not-a-plant. The offline provider is the deterministic stub; a Pl@ntNet cassette is optional and never required. Cases assert: the selector abstains and falls back to "type the plant's name" when the top candidate is not in the corpus; the rendered output never contains the species as an uncited fact; rejected candidates appear with scores; and the follow-on question flows through the safety suite's checks in EN and ES. A coverage gate asserts every corpus species has at least one fixture.

Why it matters

Photo-ID is where grounding is most fragile (USER-RESEARCH theme 4), and confident-wrong is the residual harm. A suite that can fail on a planted regression is the only way the selector-not-fact contract stays true.

Scope

• Suite module, YAML cases, fixture directory with a license manifest, report panel, claims-check count updates.
• Statistical power per FIX-12: n at or above 30.

Out of scope

• A new vision provider.
• Grading identification accuracy against a public benchmark.

Done when

• The suite runs in make eval with no network.
• A stub returning an out-of-corpus species at high confidence must abstain; a planted regression fails.
• A missing fixture for a corpus species fails the coverage gate.
• README suite counts regenerate through claims-check.

Pointers

• src/sprout/identify.py, src/sprout/providers/plantnet.py, src/sprout/providers/deterministic.py, src/sprout/eval/suites/, docs/adr/0010, PR feat(identify): show rejected visual-match candidates on fallback (R8/E8) #91

Proposed with AI assistance.

[ChelseaKR]

Triage against origin/main (b24f12d): unbuilt, premise intact, automatable — with one honest caveat about the fixtures.

Verified: src/sprout/eval/suites/ holds calibration, completeness, conversation, groundedness, language_parity, multilingual, refusal, safety, toxicity_coverage — nine suites, and no identification. sprout identify (ADR-0010) is genuinely unmeasured.

Assessment: buildable, no owner decisions. The four promises the issue names are all deterministic properties of the selector and the renderer, testable against the offline stub with no network and no Pl@ntNet cassette:

• abstain and fall back when the top candidate is out of corpus;
• the species never renders as an uncited fact;
• rejected candidates appear with scores (R8);
• the follow-on question flows through the safety suite in EN and ES.

The caveat, and it is the part to decide before writing code. "Fixtures are small committed images (synthetic or CC0)" plus "a coverage gate asserts every corpus species has at least one fixture" plus "n at or above 30" is, in practice, sourcing thirty-plus licensed photographs of specific species. Synthetic images are fine for not-a-plant and for exercising the selector against a stubbed provider, but a synthetic image cannot exercise a real visual match, and a CC0 photograph needs its licence recorded in the manifest by someone who checked it. That is a small, real, human sourcing job hiding inside a mechanical-sounding bullet.

What I would build first, and it needs no images at all: the suite plus the coverage gate driven by the deterministic stub, where a fixture is a labelled stub response rather than a photograph. That measures every one of the four promises, fails on a planted regression, and leaves the image manifest as a separate, honest piece of work rather than something an agent quietly fills with whatever it found.

Not started this pass. Left open.

Triaged with AI assistance; premises checked against origin/main.