docs(tri-comparison): mandatory recall audit — every GG non-detection assessed & logged

.claude/skills/tri-comparison-ledger-sweep/SKILL.md

@@ -370,6 +370,95 @@ lines that broke `func_start` confirmed the identical false-positive, requiring
 same PR). A shared root cause fixed in only one of two structurally-identical rule keys is a
 half-fixed bug, not a complete one.
 
+## 2.6. The recall audit -- MANDATORY, every function GitGalaxy does not detect gets individually assessed
+
+**This is a validation GATE, not an optional deep-dive. A language's tri-comparison work is not
+"done" -- step 8's capstone must not be written -- until every function tree-sitter OR ctags
+reports that GitGalaxy does not, across the WHOLE corpus, has been individually looked at and
+sorted into exactly one of the two buckets below.** The point is a hard guarantee: *we know
+whether GitGalaxy is missing anything real, per language, and it is written down.*
+
+Why this needs to be its own step and can't be left to per-shape investigation: the ledger's
+shapes are keyed by discrepancy *pattern*. A shape like `<lang>/function/existence/
+agree[tree_sitter]_vs[ctags,gitgalaxy]` can be marked `status: validated` with a verdict of
+"tree-sitter over-detects here" **without anyone having confirmed that every occurrence in that
+bucket is a tree-sitter error rather than a GitGalaxy miss.** A blanket verdict silently hides a
+real recall gap sitting in the same bucket. Confirmed, not hypothetical (2026-08-29, cpp): the
+`agree[tree_sitter]_vs[ctags,gitgalaxy]` bucket had ~164 occurrences; ~96 were `OPCODE(...)`
+bytecode-dispatch-macro hallucinations, ~60 were `_FORCE_INLINE_`-macro mis-parses and bodyless
+`= delete`/`= default` special members -- all tree-sitter's fault -- but **~2 were genuine
+GitGalaxy recall gaps** (`STDAPI DllCanUnloadNow()` macro-return-type form; K&R-style
+`PRIVATE void PlinkPrint(out,plp,tag)` with untyped param declarations on the following lines)
+that the "tree-sitter over-detects" framing would have buried forever.
+
+### How to run it
+
+Do it per language, against BOTH tools, at the OCCURRENCE level (not just name-set diff -- a name
+GitGalaxy finds 3 times and tree-sitter finds 5 times is 2 unassessed non-detections). Use
+`tests/tools/recall_audit.py` (`python tests/tools/recall_audit.py <lang> [<lang> ...]`, or no
+args for the full set) -- it enumerates every `tree-sitter-finds / GitGalaxy-misses` and
+`ctags-only-finds / GitGalaxy-misses` occurrence with its source line, so the only thing left is
+reading source and classifying. It also cross-checks `measure()`'s own `real_functions -
+found_functions` (the number the published accuracy table's recall is computed from) so a
+divergence between the raw name-diff and the audit's counted misses is itself surfaced.
+
+### The two buckets -- every non-detection lands in exactly one
+
+1. **A real GitGalaxy recall gap.** A body-bearing definition, in live (non-dead-code) source,
+   that GitGalaxy's `func_start` genuinely does not match or that `detector.py`'s slicing drops.
+   -> **GitHub issue with an isolated repro** (the exact source form + the minimal failing case),
+   same standing rule as step 4.3's bucket 1. Group multiple occurrences of the *same* form into
+   one issue. If the fix is small and well-scoped (a regex widening with a regression check),
+   ship it in the same pass per bucket 1's extension.
+
+2. **A comparison-tool or audit-tool artifact -- GitGalaxy is correct.** Name the specific
+   mechanism; do not accept a vague "tree-sitter over-detects." The mechanisms seen so far, each
+   with a different correct resolution:
+   - **Macro-invocation hallucination** (`OPCODE(X) {`, `ENTRY(X): {`, `IFACEMETHOD_(...)`) --
+     tree-sitter/ctags read a function-like macro call as a definition. Resolution: the audit's
+     tree-sitter walk must filter it. C already has `_C_KNOWN_MACRO_HALLUCINATIONS`; add/extend
+     the per-language set (`_CPP_KNOWN_MACRO_HALLUCINATIONS`, ...) rather than leaving the audit
+     to count it against GitGalaxy's recall forever.
+   - **Dead preprocessor code** (`#if 0 ... #endif`, `#if FALSE`) -- tree-sitter has no
+     preprocessor model and parses the dead branch. Resolution: detect the guard and drop those
+     tree-sitter functions from ground truth (this is `docs/why_gitgalaxy_beats_ast_here.md`
+     Claim 8's exact shape -- add the concrete citations there too).
+   - **Bodyless declaration** -- a forward declaration / prototype / `= default` / `= delete`
+     special member / abstract signature with no body. GitGalaxy deliberately only counts
+     body-bearing definitions. Resolution: the audit already drops these for perl/typescript;
+     extend the drop to the language in hand.
+   - **Deliberate scope boundary** -- a nested closure GitGalaxy does not record as a peer node
+     (shell `zgrep` inside an `if` guard, a 2-levels-deep lambda). Already an accepted design
+     choice; confirm the verdict says so and move on.
+   - **Naming-convention mismatch** -- both tools found it, they just name it differently
+     (`operator String() const` vs `operator String`; ctags' full ObjC selector `linkTo:` vs
+     GitGalaxy's `linkTo`; tree-sitter naming `- unsigned char foo` by the return type
+     `unsigned`). Resolution: canonicalize the tree-sitter/ctags name to GitGalaxy's convention
+     in the shared reader (`_get_node_name` / `ctags_reader.py`), OR document the mapping. NOT a
+     recall gap -- the function is found.
+   - **Occurrence-alignment fuzz** -- a name defined many times where the rank/line pairing in
+     `_align_occurrences_by_line` leaves a phantom "unmatched real" even though GitGalaxy's own
+     count for that name is correct. Resolution: confirm GitGalaxy's occurrence count matches the
+     real source count for that name, note it, and (if it's inflating a published number) widen
+     the pairing tolerance.
+
+### Log the cleared result
+
+The language's capstone §9 (step 8) must state, explicitly: the recall number, that **every**
+non-detection was individually assessed, the count in each bucket, and -- for bucket 2 -- the
+per-mechanism breakdown with at least one real `file:line` per mechanism. A capstone that just
+says "recall 98.6%" without the assessment breakdown is not finished. If any audit-tool fix from
+bucket 2 changed the published recall/precision number, the capstone and the summary table must
+both reflect the post-fix number, and the baseline is regenerated.
+
+### Cross-language, once
+
+After the per-language passes, run `recall_audit.py` with no args one time and confirm the only
+remaining `tree-sitter-finds / GitGalaxy-misses` occurrences anywhere are the ones already filed
+as issues. That list -- ideally short, every entry pointing at an open issue -- is the standing
+answer to "is GitGalaxy missing anything," and belongs in `docs/self_scan/
+tri_comparison_README.md` so it doesn't have to be re-derived.
+
 ## 3. Dispatch to Gemini -- read-only, no worktree, self-contained prompt
 
 Use `Agent` with `subagent_type: gemini-analyzer`, `run_in_background: true`. One dispatch per
@@ -590,8 +679,12 @@ PR sit open and rot into a conflict is now proactive by default.
 ## 8. Capstone: when a language's backlog clears, write it up before moving on
 
 **Trigger:** every currently-reproducing shape for a language is `status: "validated"` (check with
-the same query step 1 uses, filtered to that language, confirming zero results). Do this BEFORE
-starting the next language, not as a someday follow-up -- the whole reason it's cheap right now is
+the same query step 1 uses, filtered to that language, confirming zero results) **AND step 2.6's
+recall audit for that language is complete -- every function GitGalaxy does not detect has been
+individually assessed and sorted, with the breakdown ready to write into §9.** Both conditions,
+not just the first: a fully-`validated` ledger with an unaudited recall gap is exactly the
+silent-miss failure step 2.6 exists to prevent. Do this BEFORE starting the next language, not as
+a someday follow-up -- the whole reason it's cheap right now is
 that every file:line citation, every confirmed mechanism, and every "is this GitGalaxy's fault"
 verdict is still loaded in this session's context. Reconstructing that same picture from a cold
 read of the ledger later costs real tokens and real judgment a fresh session doesn't have for
@@ -624,6 +717,10 @@ Two things to produce, both while the context is cheap:
    - Summary stats: shapes investigated, occurrences covered, confirmed GitGalaxy engine defects
      found (compare across languages -- `rust` found 2 real ones, `c` found zero, and that
      contrast IS the finding, not a gap in one or the other).
+   - **The step 2.6 recall-audit result**: the recall number, "every non-detection individually
+     assessed", the bucket-1 count (real gaps, each linked to its issue) and the bucket-2
+     per-mechanism breakdown with a real `file:line` each. This is the part that lets a reader
+     trust the recall number instead of taking it on faith.
    - "Where GitGalaxy wins outright" -- the confirmed cases, with real file:line citations, not
      hand-wavy summaries.
    - "Where the other tools have real, documented gaps" -- same standard.

docs/self_scan/tri_comparison_README.md

@@ -69,6 +69,29 @@ same repo: `credit_tools` only makes sense for a shape where ONE tool's claim wa
 crediting a tool that's already in a 2-of-3 agreeing pair double-counts, since that pair already
 gets precision credit from the base reconciliation).
 
+## The recall audit: is GitGalaxy missing anything?
+
+A validated shape verdict says what's true about a *pattern* — it does not, on its own, prove
+that every occurrence in that pattern's bucket is a tree-sitter/ctags error and not a real
+GitGalaxy miss hiding among them (cpp, 2026-08-29: ~164 "misses" in one bucket, ~162 tree-sitter
+macro/parse artifacts, 2 genuine GitGalaxy recall gaps the "tree-sitter over-detects" verdict
+would have buried). So the `tri-comparison-ledger-sweep` skill's **step 2.6** requires a
+separate, occurrence-level pass: every function tree-sitter OR ctags reports that GitGalaxy does
+not is individually read and sorted into either a filed GitHub issue (a real recall gap) or a
+named tool/audit artifact (macro hallucination, `#if 0` dead code, bodyless declaration,
+deliberate scope boundary, naming-convention mismatch, occurrence-alignment fuzz). Run
+`python tests/tools/recall_audit.py` (no args = every language) to enumerate them; a language's
+step 8 capstone is not done until its list is empty of unassessed entries.
+
+**Standing answer — the only real GitGalaxy function-recall gaps across the whole corpus:**
+
+<!-- RECALL_AUDIT:BEGIN -->
+_(populated by the sweep — each entry is `<lang>: <source form> -> #<issue>`)_
+<!-- RECALL_AUDIT:END -->
+
+Everything else `recall_audit.py` prints is a comparison-tool artifact, catalogued per language
+in that language's `docs/language_status/<lang>.md` §9.
+
 ## The catalog: confirmed, evidenced differences between the three tools
 
 This is the part `docs/self_scan/README.md` was missing — not a live list of open questions (that's

tests/tools/recall_audit.py

@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+"""
+recall_audit.py
+
+Enumerates every function tree-sitter OR universal-ctags reports that GitGalaxy does NOT, across
+the whole language-crucible corpus, per language, at the OCCURRENCE level (a name GitGalaxy finds
+3x and tree-sitter finds 5x is 2 non-detections, not 0). For each, prints the source line so the
+only work left is reading it and sorting it into one of the two buckets in the
+`tri-comparison-ledger-sweep` skill's step 2.6:
+
+  1. a real GitGalaxy recall gap        -> file a GitHub issue with an isolated repro
+  2. a comparison/audit-tool artifact   -> name the mechanism, fix the audit or document it
+
+This is the standing answer to "is GitGalaxy missing anything?" -- run it with no arguments for
+the full sweep, or name languages to scope it.
+
+    python tests/tools/recall_audit.py                # every language
+    python tests/tools/recall_audit.py cpp shell      # just these
+
+It is verification-only tooling (needs tree-sitter-language-pack, optionally universal-ctags),
+never imported by anything under gitgalaxy/. Same optional-dependency contract as
+tri_comparison_gatherer.py.
+"""
+
+from __future__ import annotations
+
+import collections
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent))
+
+import tree_sitter_accuracy_audit as tsaa
+from tri_comparison_gatherer import gather_language
+
+# The languages with a tree-sitter baseline (NODE_MAPS) plus the ctags-comparable gg-only set --
+# the exact universe where a "GitGalaxy misses X" question is answerable.
+_TS_LANGS = sorted(tsaa.NODE_MAPS)
+
+
+def _src_line(corpus_dir: Path, relpath: str, lineno: int) -> str:
+    p = corpus_dir / relpath
+    try:
+        lines = p.read_text(encoding="utf-8", errors="replace").splitlines()
+    except OSError:
+        return "<source unavailable>"
+    if 1 <= lineno <= len(lines):
+        return lines[lineno - 1].strip()[:120]
+    return "<line out of range>"
+
+
+def audit_language(lang: str) -> dict:
+    readings = gather_language(lang)
+    corpus_dir = tsaa.ensure_corpus(lang)
+
+    ts_misses: list[tuple[str, str, int]] = []  # (file, name, line)
+    ct_misses: list[tuple[str, str, int]] = []
+
+    for fr in readings:
+        gg_by = collections.Counter(o.name for o in fr.gg_funcs)
+
+        ts_lines: dict[str, list[int]] = collections.defaultdict(list)
+        for o in fr.ts_funcs:
+            ts_lines[o.name].append(o.line)
+        ct_lines: dict[str, list[int]] = collections.defaultdict(list)
+        for o in fr.ctags_funcs:
+            ct_lines[o.name].append(o.line)
+
+        for name, lines in ts_lines.items():
+            surplus = sorted(lines)[gg_by.get(name, 0) :]
+            ts_misses.extend((fr.file_path, name, ln) for ln in surplus)
+
+        ts_names = set(ts_lines)
+        for name, lines in ct_lines.items():
+            if name in ts_names:
+                continue  # already covered by the tree-sitter column
+            surplus = sorted(lines)[gg_by.get(name, 0) :]
+            ct_misses.extend((fr.file_path, name, ln) for ln in surplus)
+
+    return {
+        "lang": lang,
+        "corpus_dir": corpus_dir,
+        "ts_misses": sorted(ts_misses),
+        "ct_misses": sorted(ct_misses),
+    }
+
+
+def _measure_counted_misses(lang: str) -> int | None:
+    """The number the published accuracy table's recall is computed from:
+    real_functions - found_functions from measure()."""
+    if lang not in tsaa.NODE_MAPS:
+        return None
+    try:
+        m = tsaa.measure(lang)
+    except SystemExit:
+        return None
+    return m["real_functions"] - m["found_functions"]
+
+
+def main(argv: list[str]) -> int:
+    langs = argv[1:] or _TS_LANGS
+    grand_ts = 0
+    for lang in langs:
+        try:
+            res = audit_language(lang)
+        except Exception as exc:
+            print(f"\n{'=' * 78}\n{lang}: SKIPPED -- {exc}\n{'=' * 78}")
+            continue
+
+        counted = _measure_counted_misses(lang)
+        n_ts = len(res["ts_misses"])
+        n_ct = len(res["ct_misses"])
+        grand_ts += n_ts
+        print(f"\n{'=' * 78}\n{lang}\n{'=' * 78}")
+        if counted is not None:
+            note = ""
+            if counted != n_ts:
+                note = f"   <-- differs from raw name-diff ({n_ts}); check drop-rules / alignment"
+            print(f"  measure() counted misses (recall denominator gap): {counted}{note}")
+
+        print(f"\n  tree-sitter finds / GitGalaxy misses  ({n_ts}):")
+        for f, name, ln in res["ts_misses"]:
+            print(f"    {f}:{ln}  {name!r}")
+            print(f"        | {_src_line(res['corpus_dir'], f, ln)}")
+
+        if n_ct:
+            print(f"\n  ctags-only finds / GitGalaxy misses  ({n_ct}):")
+            per_file = collections.Counter(f for f, _, _ in res["ct_misses"])
+            shown = collections.Counter()
+            for f, name, ln in res["ct_misses"]:
+                shown[f] += 1
+                if shown[f] <= 8:
+                    print(f"    {f}:{ln}  {name!r}  | {_src_line(res['corpus_dir'], f, ln)}")
+            for f, c in per_file.items():
+                if c > 8:
+                    print(f"    ... {f}: +{c - 8} more")
+
+    print(f"\n{'=' * 78}\nTOTAL tree-sitter-finds / GitGalaxy-misses across {len(langs)} language(s): {grand_ts}")
+    print("Every one must be sorted into skill step 2.6's bucket 1 (issue) or bucket 2 (artifact).")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv))