Add a quote_check gate that verifies cited quotes offline from a source snapshot

[ChelseaKR]

What

real_targets/quotecheck.py fetches a cited public document and looks for the quoted text, so a pack can say the harness found the quote rather than the target said it did. It lives outside the package and only the three adapters call it. Make it a gate any suite can use.

A case declares verify_quotes: true; the target reports quotes: [{citation, url, text}] (directly or via annotations, see the annotations issue). By default the gate looks quotes up in a committed snapshot directory (--sources-dir) whose manifest records URL, sha256, fetched_at, and the tool used. A separate gauntlet snapshot verb populates that directory over the network, outside CI. --fetch on run is explicit opt-in for live lookups.

Why it matters

The 2026-08-28 correction made unverified quotes leave the accepted context. That rule exists only in the adapters, and a team gating its own feature cannot use it. With a snapshot, the gate is deterministic and offline, which is what a merge gate has to be; an unverifiable quote is reported and excluded, never counted as grounded.

Scope

• src/gauntlet/gates/quote_check.py, case fields, and the snapshot manifest format.
• gauntlet snapshot --cases DIR --out DIR (network, records tool: stdlib, curl, pdftotext).
• Pack section per quote: verified, not_found, unverifiable with the reason.
• A paired toy defect (alter_quote_one_word) so the self-test doctrine covers it.

Out of scope

• Paraphrase judgement; that is the judge gate.
• Document formats beyond HTML text and pdftotext output.

Done when

• With a snapshot present the run opens no socket (test blocks socket.socket) and the toy passes.
• A quote altered by one word fails as not_found and names the case.
• A missing snapshot entry is unverifiable, excluded from the pass count, listed in the pack, and a 100% threshold turns the gate red.
• gauntlet inventory counts the new gate and the README block regenerates.

Pointers

• real_targets/quotecheck.py (counts_as_grounded), src/gauntlet/gates/grounding.py, src/gauntlet/gates/base.py
• src/gauntlet/toy/, tests/test_self_test_doctrine.py
• Related: New packs must record quote-check outcomes so a replay verifies rather than skips #31, Quote-check gate: an unverifiable citation is silently accepted as grounded, not excluded like a not-found one #19 (closed)

Proposed with AI assistance.

[ChelseaKR]

Triage against origin/main (8ae3fa2): unbuilt, premise intact, automatable — but larger than it looks.

Verified: src/gauntlet/gates/ holds adversarial, base, false_positive, golden, grounding, readability, refusal — no quote_check. real_targets/quotecheck.py is outside the package and only the three adapters call it. ✔

Assessment: buildable, no owner decisions, and the design in the issue is already the right shape — a committed snapshot directory keeps the gate deterministic and offline, which is what a merge gate has to be, and unverifiable being excluded from the pass count rather than counted as grounded is the correction from #19 applied at gate level.

Two things to get right, both already learned in this repo:

1. GAUNTLET_QUOTE_CHECKS=off currently makes every check unverifiable, and counts_as_grounded accepts only verified. The new gate must inherit that polarity exactly. A gate that treats unverifiable as a pass is the 2026-08-28 defect re-introduced one layer up, and tests/test_real_target_packs.py documents at length how invisible it was.
2. The snapshot manifest must record the tool as well as the URL and sha256 — pdftotext output is not stable across versions, so a manifest that records only a hash will report not_found on a re-fetch and blame the target.

Not started by me this pass. Left open.

Triaged with AI assistance; premises checked against origin/main.

[ChelseaKR]

Still open and genuinely unbuilt as a gate. Checked against origin/main so the next reader does not re-derive it, and to record the half that did land.

Not present:

• src/gauntlet/gates/quote_check.py — the gates package holds adversarial, base, false_positive, golden, grounding, readability, refusal and nothing else.
• The verify_quotes case field, the --sources-dir snapshot directory, and the gauntlet snapshot verb: git grep -n 'verify_quotes\|sources.dir\|"snapshot"' -- src/ returns nothing.
• real_targets/quotecheck.py is still outside the package, imported only by real_targets/narration.py and real_targets/permit_bearings/target.py.

What did land, and it is the harder half of "deterministic and offline": PR #38 made a recording carry its quote-check outcomes. real_targets/quotecheck.py writes every outcome to the run's raw log keyed by document and normalized quote, so a replay reproduces the verification rather than skipping it, and its docstring states the rule this issue would inherit — "under GAUNTLET_QUOTE_CHECKS=off nothing is recorded, because recording 'the harness did not look' as an outcome would put an absence in the log where a measurement belongs." tests/test_quote_verification_contract.py and test_a_recording_carries_its_quote_check_outcomes hold it, including the quote_checks_without_a_recorded_outcome counter.

So the three-outcome vocabulary (verified / not_found / unverifiable), the normalization, the provenance counters and the replay contract already exist and are tested. What a gate needs on top is the case field, the snapshot manifest, a snapshot verb, the pack section, and the paired toy defect.

Prepared with AI assistance; reviewed before submission.