docs(tri-comparison): mandatory recall audit — every GG non-detection assessed & logged - #2458
Merged
Conversation
…n assessed Adds step 2.6 to the tri-comparison-ledger-sweep skill: a validation GATE requiring that every function tree-sitter OR ctags reports that GitGalaxy does not, across the whole corpus, is individually read and sorted into exactly one of two buckets -- a real GitGalaxy recall gap (GitHub issue with an isolated repro) or a named comparison/audit-tool artifact (macro hallucination, #if 0 dead code, bodyless declaration, deliberate scope boundary, naming-convention mismatch, occurrence-alignment fuzz). Why it can't be left to per-shape investigation: the ledger keys entries by discrepancy PATTERN. A shape verdict of "tree-sitter over-detects here" can be marked validated without anyone confirming every occurrence in that bucket is a tree-sitter error rather than a GitGalaxy miss. Confirmed real (cpp, 2026-08-29): one bucket held ~164 occurrences -- ~162 tree-sitter OPCODE-macro / _FORCE_INLINE_-parse artifacts, 2 genuine GitGalaxy recall gaps (STDAPI macro-return-type; K&R untyped-param defs) the blanket verdict would have buried. - step 8's capstone trigger now also requires the recall audit complete - capstone §9 must carry the recall number + "every non-detection assessed" + the bucket breakdown with a real file:line per mechanism - new `tests/tools/recall_audit.py` enumerates every non-detection with its source line and cross-checks measure()'s own counted misses - tri_comparison_README.md gets a "is GitGalaxy missing anything?" section with a RECALL_AUDIT marker block for the standing answer Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Zm1uVBpVEZJ9SB7bWFR8S
Contributor
Owner
Author
|
Superseded by #2463, which includes this commit plus the execution of step 2.6 (c/cpp/fortran/objc recall corrections + docs). Closing in favor of that one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds step 2.6 to the
tri-comparison-ledger-sweepskill — a validation gate requiringthat every function tree-sitter OR ctags reports that GitGalaxy does not, across the whole
corpus, is individually read and sorted into exactly one of two buckets:
#if 0deadcode, bodyless declaration, deliberate scope boundary, naming-convention mismatch,
occurrence-alignment fuzz), fix the audit or document it
Why
The ledger keys entries by discrepancy pattern. A shape verdict of "tree-sitter over-detects
here" can be marked
validatedwithout anyone confirming every occurrence in that bucket is atree-sitter error rather than a GitGalaxy miss. Confirmed real (cpp, 2026-08-29): one bucket
held ~164 occurrences — ~162 tree-sitter
OPCODE-macro /_FORCE_INLINE_-parse artifacts, but2 genuine GitGalaxy recall gaps (
STDAPI DllCanUnloadNow()macro-return-type form; K&RPRIVATE void PlinkPrint(out,plp,tag)untyped-param defs) that the blanket "over-detects"verdict would have buried forever.
Changes
SKILL.mdstep 2.6 — the gate, how to run it, the two buckets with per-mechanismresolution guidance.
validatedledger (a validated ledger with an unaudited recall gap is exactly the failuremode this prevents).
the bucket-1 count (each linked to its issue), and the bucket-2 per-mechanism breakdown with a
real
file:lineeach.tests/tools/recall_audit.py(new) — enumerates every non-detection vs tree-sitter andctags at the occurrence level, with source lines, and cross-checks
measure()'s ownreal_functions - found_functionsso a divergence between the raw name-diff and the countedmisses is itself surfaced.
tri_comparison_README.md— a "is GitGalaxy missing anything?" section with aRECALL_AUDITmarker block for the standing answer.Follow-up (separate PR)
Executing step 2.6 for the current state: the 8 languages with func recall < 100% break down as
~3 genuine GitGalaxy gaps (shell keyword-as-argument, c K&R defs, cpp macro-return-type) + a
large tail of tree-sitter/ctags artifacts the audit is currently counting against GitGalaxy's
recall. That PR files the issues and corrects the audit.
🤖 Generated with Claude Code
https://claude.ai/code/session_019Zm1uVBpVEZJ9SB7bWFR8S