fix(groovy): tri-comparison manual verification -- 3 real engine defects fixed - #2532
Merged
Merged
Conversation
…cts fixed
tree-sitter has no usable grammar for Groovy despite tree_sitter_language_pack
nominally loading a "groovy" grammar: it has no class_declaration/method_declaration
node type at all (class Foo { parses into generic command/unit/block soup, 3.2%
ERROR rate on a 30-file sample), and the two node types this repo's NODE_MAPS used
to map it to ("func", "generics_class") are semantically wrong, not just imprecise
-- "func" nodes are call expressions, not definitions, and "generics_class" nodes
are generic type usage, not class declarations. ctags has zero Groovy support at
all. Removed groovy from tree_sitter_accuracy_audit.py's NODE_MAPS (documented
alongside the existing cobol/dockerfile/scheme/yaml exclusions) and routed it
through the gg_only manual-verification treatment instead, same category
abap/dockerfile/jcl/livecode/yaml are already in.
With no automated ground truth available, ran a manual verification instead: an
independent grep-based ground-truth scan across the full 188-file language-crucible
v1.2.0 groovy corpus (sharing no implementation with GitGalaxy's own regexes),
cross-checked against the real pipeline's actual DB output. Found and fixed three
real, independent GitGalaxy engine defects along the way:
- class_start had zero capture groups (pattern.groups() == 0) -- class existence
was detected correctly but every one of 286 named classes was literally called
"Anonymous_Class" instead of its real name. The only language in
_CLASS_START_NAMED_EXTRACTION_LANGS with this defect (checked all 36).
- Prism never shielded groovy's """/''' multi-line strings -- code-shaped fixture
text embedded in one (Gradle integration-test build-script snippets, Spock's own
compiler-smoke-test source samples) leaked into the real code stream and
corrupted extraction for the rest of the file via desynced brace-counting. Fixed
with a new _strip_groovy_triple_quoted_strings in prism.py, same idiom as the
existing PowerShell here-string / Lua long-bracket precedents.
- func_start's zero-prefix branch matched the bare "def" keyword as a function name
once a Spock quoted feature-method name got correctly blanked for brace-safety.
Fixed two ways: excluded "def" from both branches' own exclusion lookahead, and
preserved a quoted name immediately following "def" in detector.py's brace-safety
shield instead of blanking it (mirrors the existing zig `@"..."` exception in the
same function). A related _extract_name defect (silently truncating a quoted
multi-word name down to its last word) was also fixed with an early-return fast
path -- confirmed zero-blast-radius elsewhere since groovy is currently the only
language whose func_start captures a quoted name at all.
Post-fix: zero unexplained recall gaps found across the whole corpus for either
functions or classes. One confirmed, unfixed false-positive class remains --
MarkupBuilder/Jenkins-Pipeline-DSL trailing-closure calls (`button(...) { }`) are
syntactically indistinguishable from a real zero-prefix declaration -- filed as
#2530 rather than patched with a fragile denylist (needs its own design pass).
Verified: 661/718 (92.1%) named functions, 222/222 (100%) named classes.
Full verification chain: standalone regex re-tests, extraction gauntlet + strict
suite, ruff/mypy/dead-key audits, full pytest suite (7145 passed), and the full
~80-repo crucible differential scan (2109 diffs -- 1587 directly groovy-
attributable, the remaining 522 all tiny ripple <1% relative from the shared
force-directed graph layout and global risk-normalization recomputing, confirmed
via a pristine-origin/main control run showing zero diffs). Both golden master
fixtures re-blessed. Ledger shape
groovy/function/existence/agree[gitgalaxy]_vs[tree_sitter] validated with
credit_tools=[gitgalaxy] (tree-sitter's non-corroboration is a confirmed tool
limitation, not evidence against GitGalaxy). Full write-up in
docs/language_status/groovy.md §9; verified counts recorded in
docs/self_scan/manual_verification.json.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
squid-protocol
added a commit
that referenced
this pull request
Aug 31, 2026
…r declarations (#2597) * fix(groovy): func_start no longer mistakes MarkupBuilder/DSL calls for declarations Closes #2530. func_start's zero-prefix branch (needed to match real bare constructors, `MyClass(String arg) {`) was syntactically indistinguishable from Groovy's MarkupBuilder/NodeBuilder DSL idiom -- a method call with a named-argument map plus a trailing closure (`div(class: "x") { ... }`, `button(name: "clear", type: "submit") { ... }`). Pervasive in Jenkins's own `.groovy` view templates and Jenkins Pipeline DSL steps. Fix: a real Groovy declaration's parameter list can never contain a `key:` token (that syntax is call-site named-argument sugar only), so branch 2 now rejects a candidate whose parenthesized argument list contains an identifier immediately followed by `:` with no intervening whitespace -- distinguishing it from a ternary's `cond ? a : b`, which conventionally has a space before the colon. Verified via a full corpus differential scan (galaxyscope --db-only, 201 files): 44 of the 57 misdetections in jenkins_view_groovy/ resolved, zero removals anywhere else in the corpus -- no legitimate bare constructor lost recall, including edge cases like a Map-literal default (`= [:]`) or a ternary default value. ReDoS-scaling probed (linear to n=16000). 13 residual FPs remain, out of scope per the issue: single-positional-arg DSL calls (`stage('Build') { }`, `node('x') { }`, `dir(x) { }`) have no `key:` token to key off of -- same class the issue explicitly deferred (a curated builder-tag-name denylist is fragile/incomplete by construction). Fixing the button(...) false positive also unmasked one new, unrelated func_start defect in a different branch (a paren-less call, `raw _("Dismiss")`, previously silently absorbed as "nested inside button") -- filed separately as #2558 rather than bundled in here. Golden masters re-blessed (both precision modes): 43 net diff (44 direct removals - 1 new #2558 surfacing), rest is the standard topological/ aggregate ripple class already documented in CLAUDE.md's Differential Scan protocol. Verified with a full uncapped diff categorization: zero unexplained/off-target changes anywhere in the 201-file corpus. Also updates docs/language_status/groovy.md, manual_verification.json, and regenerates the tri-comparison chart/ledger to reflect the corrected groovy function precision (661/718 -> 661/675, 92.1% -> 97.9%). Verification: full groovy extraction gauntlet + strict suite (150 passed, new regression test added), cross-language strict registry (21 passed), full pytest suite (7154 passed), audit_check.py clean, tree_sitter_accuracy_audit.py --all --ci clean (groovy correctly excluded as gg_only per #2532), crucible_check.py both modes PASS after re-bless. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * chore: regenerate tri-comparison chart/ledger after merging main Resolves the merge conflict in tri_comparison_ledger.json from main's own auto-update-tri-comparison-data bot commit (#2553) landing while this branch's crucible verification was running. Took main's version as the base and regenerated via tri_comparison_chart.py --all --write on top of the merged code state, so the output reflects both main's other-language refreshes and this branch's groovy fix together. Re-verified clean: audit_check.py, full pytest suite (7154 passed), crucible_check.py both modes PASS. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tree_sitter_language_pack's "groovy" grammar loads without error but can't parse realGroovy declarations -- confirmed via direct inspection (30-file/13K-node sample, 3.2%
ERRORrate, noclass_declaration/method_declarationnode type at all). The two nodetypes this repo's
NODE_MAPSused to map groovy to ("func","generics_class") turnedout to be semantically wrong, not just imprecise:
"func"nodes are call expressions(
getLogger(JiraService.class)), not definitions;"generics_class"nodes are generictype usage (
Map<String, X>), not class declarations.ctagshas zero Groovy support atall. Groovy is now routed through the
gg_onlymanual-verification treatment (same categoryabap/dockerfile/jcl/livecode/yaml are already in) instead of a broken tree-sitter comparison.
With no automated ground truth available, ran a manual verification per the
tri-comparison-ledger-sweepskill's fallback procedure: an independent grep-basedground-truth scan across the full 188-file
language-cruciblev1.2.0 groovy corpus,cross-checked against the real pipeline's DB output. Found and fixed three real,
independent GitGalaxy engine defects:
class_starthad zero capture groups -- class existence was detected correctly,but every one of 286 named classes was literally called
"Anonymous_Class"instead ofits real name. The only language in
_CLASS_START_NAMED_EXTRACTION_LANGSwith thisdefect (checked all 36).
"""/'''multi-line strings -- code-shaped fixturetext embedded in one (Gradle integration-test build-script snippets, Spock's own
compiler-smoke-test source samples) leaked into the real code stream and corrupted
extraction for the rest of the file via desynced brace-counting. New
_strip_groovy_triple_quoted_stringsinprism.py, same idiom as the existingPowerShell here-string / Lua long-bracket precedents.
func_start's zero-prefix branch matched the baredefkeyword as a function nameonce a Spock quoted feature-method name got correctly blanked for brace-safety. Fixed two
ways (excluded
deffrom the exclusion lookahead; preserved a quoted name followingdefin the brace-safety shield instead of blanking it), plus a related_extract_namefix (it silently truncated a quoted multi-word name to its last word) -- confirmed
zero-blast-radius elsewhere since groovy is currently the only language whose
func_startcaptures a quoted name at all.Post-fix: zero unexplained recall gaps across the whole corpus for either functions or
classes. One confirmed, unfixed false-positive class remains -- MarkupBuilder/
Jenkins-Pipeline-DSL trailing-closure calls (
button(...) { }) are syntacticallyindistinguishable from a real zero-prefix declaration -- filed as
#2530 rather than patched with a
fragile denylist (needs its own design pass). Verified: 661/718 (92.1%) named functions,
222/222 (100%) named classes.
Full write-up:
docs/language_status/groovy.md§9.Type of change
gitgalaxy/core/detector.py,language_standards.py,prism.py, or a per-language rule)CI checklist
python tests/tools/audit_check.pypassespython -m pytest tests/passes -- 7145 passedpython tests/tools/crucible_check.pypasses -- both golden masters re-blessed after confirming the diff (2109 changes) traced entirely to the groovy fix plus expected small ripple (see Verification below)python tests/tools/tree_sitter_accuracy_audit.py --all --cipasses -- groovy no longer measured this way (see Summary)Differential Scan target
Not applicable to a single repo -- verified against the full ~80-repo crucible corpus (see Verification).
Verification
Full-corpus differential scan, done twice for control:
golden_diff.deep_compare()(uncapped, unlike the 50-line-truncatedpytest failure message) against the committed golden master with the 3 fixes applied:
2109 diffs. 1587 directly groovy-attributable (
directory_groups/groovy/*mass/exposurerecomputing from corrected counts). The remaining 522 are all tiny (<1% relative, one
5.5% outlier that's a repo-wide aggregate directly downstream of the groovy change) --
topological-coordinate ripple from the shared force-directed graph layout recomputing
when any node's structural mass changes, and global risk-normalization shifting slightly
-- explicitly called out as an acceptable ripple class in CLAUDE.md's Differential Scan
protocol. Zero off-target anomalies in any unrelated file.
origin/main-- zero diffs, confirming the 2109 diffs are entirely attributable tothis PR's fixes, not environment noise.
crucible_check.py --update --yes), then re-rancrucible_check.pyplain -- bothfull_precisionandzero_dependencynow PASS clean.Other changes in this PR
tests/tools/tree_sitter_accuracy_audit.py: removed groovy's brokenNODE_MAPSentry,documented the exclusion (same style as the existing cobol/dockerfile/scheme/yaml bullets),
deleted the now-dead
tests/tree_sitter_accuracy_baseline_groovy.json.tests/tools/tri_comparison_chart.py: added groovy to_GG_ONLY_LANGSso it renderscorrectly as a GitGalaxy-only bar instead of silently disappearing from
--all; fixedstale language-count comments this move affected (45 total unchanged, groovy just moved
buckets -- was mis-stated as 46 in an earlier draft of this PR, corrected).
tests/tools/tri_comparison_reconcile.py,tests/tools/ctags_reader.py: matchingcomment-accuracy fixes for the same bucket move.
docs/self_scan/tri_comparison_ledger.json: validated the pre-existinggroovy/function/existence/agree[gitgalaxy]_vs[tree_sitter]shape(
credit_tools: [gitgalaxy]) -- verified only this one entry's status/verdict changed,everything else is the normal chart-regen timestamp/count refresh.
docs/self_scan/manual_verification.json: new groovy entry (function/class), sameschema as abap/dockerfile/jcl/livecode/yaml.
docs/language_status/groovy.md: new, sections 1-8 built via thelanguage-statusskillfrom primary sources (rules dict, test counts, closed issues,
gitgalaxy-raw-outputevidence), §9 the full tri-comparison write-up.
Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com