docs(readme): function-detection target closed; refresh tri-comparison stats - #2476
Merged
Conversation
…n stats The "Structural validation" and "Evidence" sections carried stale numbers (87/180 discrepancies validated, 16 languages on a 2-tool comparison) from early in the tri-comparison sweep. Current: 200/201 shapes validated, 13 on a 2-tool comparison, and the manual-verification set is abap/agc_assembly/ dockerfile/jcl/livecode (yaml gained a comparator; agc_assembly needs manual verification for `args` since no tool emits a signature there). Adds the first closed target: every tree-sitter-comparable language (31) now measures 100.0% function recall and 100.0% validated function precision on the pinned corpus. Framed narrowly on purpose — one structural target (existence), one fixed corpus, small-n languages where the percentage means little, not an "as accurate as an AST" claim — with a pointer to the mandatory recall audit as the process for the next real-world gap. Class and args targets still open. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Zm1uVBpVEZJ9SB7bWFR8S
Contributor
… shell recall is 30/31 The first draft said "class and argument targets still in progress" and "all 31 at 100% function recall" — both overstated in one direction and understated in the other. Corrected against the ledger: - Function precision: 100% validated, all 31 (accurate). - Function recall: 100% for 30 of 31. Shell is 99.8% — one function defined inside an `if` guard that the top-level-only Mode-D shell extractor skips by design (validated as an intentional limitation, ledger shape `shell/function/existence/agree[ctags,tree_sitter]_vs[gitgalaxy]`). The c and haskell `_vs[gitgalaxy]` shapes are validated comparison-tool artifacts (shared macro hallucination; multi-clause tagging) with 0 counted misses. - Classes: GitGalaxy is never the debited tool in any class shape — 100% validated recall and precision wherever a class ground truth exists. - Args: GitGalaxy is never debited in any *validated* args shape; every one is tree-sitter/ctags miscounting or a no-formal-params language. One objc shape (n=2) remains unvalidated. Reframed as "current state" bullets rather than "first target closed". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Zm1uVBpVEZJ9SB7bWFR8S
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs-only. Two spots in
README.mdcarried stale tri-comparison numbers from early in the sweep.Changes
"Structural validation" section:
16 more get two→13 more get two(current tool coverage: 24 all-three / 13 two / 5 manual)abap/dockerfile/jcl/livecode/yaml→abap/agc_assembly/dockerfile/jcl/livecode(matchesmanual_verification.json; yaml gained a comparator, agc_assembly needs manual verification forargswhere no tool emits a signature)87 of 180 discrepancy shapes validated (48%)→200 of 201 (99.5%)"Evidence → Tri-comparison" bullet: same number refresh + one sentence on the function-detection result and "its limits".
No engine or measurement code touched.
🤖 Generated with Claude Code
https://claude.ai/code/session_019Zm1uVBpVEZJ9SB7bWFR8S