chore: bump language-crucible pin to v1.2.0 + regenerate all fixtures - #2478
Merged
Conversation
Corpus grew from 152 data/ folders / 2,344 files (v1.1.0) to 218 / 3,306:
new `sqlite` + `embedded_python` categories; assembly 3->16 folders, groovy
2->16 (was .java-only), cobol 11->15 (~2x, NIST COBOL-85 CCVS suite), tcl 1->9,
html 7->15, css 3->11. Full breakdown: language-crucible RELEASING.md's v1.2.0
section. Engine code is UNCHANGED — every number below moved because the test
corpus got bigger/harder, not because the parser regressed.
Regenerated against language-crucible main @ 77bc85e (= the v1.2.0 tag), per
docs/self_scan/BUMPING_THE_CRUCIBLE_PIN.md:
1. Golden masters (both modes) — near-total rewrite. Per-file *raw* structural
signatures are stable for unchanged files; the churn is scan-global derived
metrics: 3D topological coordinates (whole DAG re-projected), PageRank
popularity/blast-radius, directory-group rollups, and repo-wide risk ratios
(e.g. c/cpython/gc.c Public Exports 354->383, Orphaned Logic 29->0 — both
scan-global "Net Mitigated Signals", not per-file counts). `flag-golden-
master-changes` is non-blocking for exactly this reason.
2. Tri-comparison chart + ledger — 0 removed entries, 0 status reversions,
9 new `unvalidated` discrepancy shapes (css 2, embedded_python 2, sqlite 2,
groovy 1, tcl 1, html 1), all from new corpus content. Candidates for a
future tri-comparison-ledger-sweep.
3. tri_comparison_report.py --write.
4. tree-sitter-accuracy history + chart.
5. tree-sitter-accuracy baselines — 4 drifted (css, groovy, html, tcl), all
from corpus growth. Regenerated via the measure()+write bypass (the
absolute-count regression gate refuses on any category that grew
substantially — see the doc's "Known gap"). Reading the examples first, per
the doc:
- css extra_functions 0->5, precision 100%->83.3%: GitGalaxy's CSS
func_start counts @layer / @Keyframes / @media / @supports at-rules as
functions. Genuine imprecision, newly exposed by Tailwind v4 + Gutenberg
CSS-Modules content. Worth a follow-up issue.
- groovy extra_functions 0->721, recall N/A->0.0%: tree-sitter finds ZERO
real functions across all 188 groovy files — its Groovy grammar provides
no usable ground truth here (the v1.1.0 baseline was all-zeros only
because old groovy corpus was .java, which the groovy standard doesn't
classify). The 721 are unverified GitGalaxy claims, not confirmed false
positives. That said, spot-checking shows real func_start FPs on the
Jenkins builder-DSL (`div`/`span`/`node`/`stage` method-calls-as-markup)
and the bare `def` keyword — worth a follow-up issue, and arguably groovy
should be marked tree-sitter-blind for this audit.
- html extra_functions 0->1: `<script>` counted once in a reveal.js deck.
- tcl extra_functions 1->3, recall 99.3%->99.1%: namespaced procs
(`ns::proc`) + SQLite test-harness procs; per-file rate flat/improved.
Summary table in language_standards.py resynced (--summary-table); the only
non-table change to that file.
6. tests/_crucible_pin.py PINNED_TAG -> v1.2.0, plus the "as of this writing"
pointers in tests/README.md, docs/self_scan/README.md,
docs/self_scan/tri_comparison_README.md, docs/language_status/csharp.md, and
one comment in tree_sitter_accuracy_audit.py. Historical v1.1.0 references
(prior-bump notes, dated evidence citations) left as-is.
7. grep for v1.1.0 — no hardcoded literal in any workflow; all read
${{ vars.LANGUAGE_CRUCIBLE_REF }}.
NOT done here (needs maintainer + coordination with language-crucible):
- cut the language-crucible v1.2.0 tag against 77bc85e + publish the release
- `gh variable set LANGUAGE_CRUCIBLE_REF --body v1.2.0 --repo squid-protocol/gitgalaxy`
CI (`crucible-audit`, `tri-comparison-audit`, `tree-sitter-accuracy-audit`)
will stay red until both of those land, since they clone against the tag/var,
not PINNED_TAG. See the PR body for the merge sequence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
squid-protocol
added a commit
that referenced
this pull request
Aug 30, 2026
….2.0 corpus (#2481) * fix(cobol): three fixed-format func_start false positives from the v1.2.0 corpus The v1.2.0 language-crucible expansion (#2478) added ~600 files of strict fixed-format punched-card COBOL -- the NIST COBOL-85 CCVS conformance suite + cobol-check -- which newly exercised three func_start false-positive shapes. None affect free-format or Area-A/B COBOL; all are column-1-7 handling. 1. CONTINUE (no-op statement) on its own line -- `003200 CONTINUE.` -- matched as a paragraph. 36 occurrences (IF4014/IF4024/IF4034/cobol-sample SAMPLE1). Fixed: CONTINUE added to func_start's reserved-verb shield, same class as LOCAL-STORAGE (#1890). 2. A bare 6-digit sequence number on a lone-period line -- `012100 .` -- captured as a paragraph name. 6 occurrences. Fixed: the captured identifier must now contain >= 1 letter. Digit-led real names (`0000-MAIN`, `100-INIT-ROUTINE`) still match -- they have letters. 3. A column-7 `D` debug-line indicator glued onto the real paragraph name: `064100D` + `DEBUG-LINE-TEST-03-A` was captured as `DDEBUG-LINE-TEST-03-A`. Fixed in two places: - anchor: consume a col-7 `D`/`d` only when a name char immediately follows (`(?<=[0-9])[Dd](?=[A-Za-z])`), and let the identifier start right after a verified 6-digit+indicator prefix via `(?<=[0-9]{6}[ \-Dd])` alongside `\b`. A `D` followed by whitespace (`064000D PASS.`) is left alone so debug-only paragraph *redefinitions* aren't newly counted. - detector.py `_slice_by_labels`: removed the C++ GoogleTest / Boost.Test `BOOST_`/`TEST`/`TEST_F`/`TEST_CASE` -> `match.group(0)` fallback. That heuristic only makes sense for the brace-based extractor (where it still lives); in this label-based Mode A (assembly / COBOL / ABAP) func_start always captures a real identifier and match.group(0) is `<anchor> + <name>`, so a *substring* hit on `DEBUG-LINE-TEST-03-A` only re-mangled it. This also corrects 5 JCL step names -- LGTESTC1 / LGTESTP1-4 were collapsing to `EXEC`. Net: cobol Function/Method Declarations 11744 -> 11701 (-43 FP occurrences); jcl unchanged in count, 5 step names corrected. Only cobol/jcl move in the golden masters -- verified no other language affected. Both golden masters re-blessed; tri-comparison regenerated (cobol [gitgalaxy]_vs[ctags] shape 55 -> 10 occurrences). One residual, filed separately: `066600DDEBUG-LINE-TEST-05-A` in DB1034.2.cbl (a debug-line paragraph bracketed by `*` comment lines) still mangles via a fourth, not-yet-identified extraction path. Verified: cobol/assembly/abap/agc_assembly extraction + strict gauntlets (512 tests), language_standards strict, ruff/mypy (no new findings), crucible_check both modes against the full corpus. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * ledger(cobol): record the v1.2.0 fixed-format func_start fixes (gitgalaxy#2481, residual #2480) cobol/function/existence/agree[gitgalaxy]_vs[ctags]: 55 -> 10 occurrences after the CONTINUE / bare-sequence-number / column-7-D fixes. 9 of the 10 remaining are GitGalaxy-correct SECTION headers (still the #1891 harness {"p"}-kind filter); 1 residual filed as #2480. credit/debit left empty -- still mixed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Aug 30, 2026
…#2491) #2478 bumped the language-crucible pin to v1.2.0 (which added one .jcl file, cics-java-jcics-samples/etc_VSAM_DEFVSAM.jcl) without refreshing manual_verification.json. The jcl record still claimed the v1.1.0 totals (375 func / 117 class), so the chart's staleness guard tripped and jcl's Func Precision and Class Precision panels dropped to a bare 0/376 / 0/118 with no badge. Re-verified the new file and the whole v1.2.0 corpus three ways (primary regex, independent line scanner, galaxyscope --db-only pipeline incl. function_data / class_data row counts) -- exact agreement at 376 / 118, zero per-file mismatches, zero false positives. The new file contributes exactly one EXEC step (//DEFINE EXEC PGM=IDCAMS) and one JOB card (//DEFVSAM JOB); its instream IDCAMS SYSIN control statements are correctly not matched. No engine change, no defect. - manual_verification.json: jcl function 375->376, class 117->118 (2026-08-30), notes rewritten for the v1.2.0 corpus. - tri_comparison_chart.svg regenerated (--all --write): jcl Func/Class Precision restored to 376/376** / 118/118** with GitGalaxy's badge; summary tallies 64->66 comparisons, GitGalaxy-best 27->29. No other language's cells changed. - tri_comparison_ledger.json: last_reconciled_at re-stamp churn only (full regen). - docs/language_status/jcl.md: new dated §9 subsection. Claude-Session: https://claude.ai/code/session_01VTkRyd9CxYQx42EMBdVkGx Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Aug 30, 2026
squid-protocol
added a commit
that referenced
this pull request
Aug 30, 2026
…dit-reader fix (#2507) (#2509) `tree-sitter-accuracy-audit` was red on main for css with a ground-truth drift (`real_functions: baseline=24 current=25`). Not corpus drift: files_scanned is 21 in both the committed baseline and now -- #2478 already regenerated it against the v1.2.0 pin. The 24->25 is #2500 ("css tri-comparison ... audit-reader fixes, no engine change") landing its own `_get_node_name` changes without re-blessing this JSON: the fixed dead-code `keyframes_statement` branch (was always `return None`, so 0 `@keyframes` corpus-wide) adds keyframes; the `at_rule` branch now returning None for the bodyless `@layer a, b;` form removes those. Net +1. #2500 updated the tri-comparison chart/ledger and docs/language_status/css.md but missed this file -- its commit message records the target state ("css Func Found 25*/24*/0* -> 25/25/0"). Zero GitGalaxy engine change, zero corpus change. Regenerated via `--regenerate` (also refreshes the summary table in language_standards.py: css 83.3%/80.0% -> 100%/100%, matching the tri-comparison). All movement is an improvement or the explained audit-reader correction: found_functions 20->25, extra_functions 5->0, args_exact_match 14->19. Also documents the permanent css at-rule `args_comparable` discrepancy in the SCOPE docstring (GitGalaxy reads a param count off the at-rule prelude; `_get_param_count` has no parameters field for at-rule nodes -> real=0). Not an engine defect -- a CSS at-rule has no formal parameter list, and counting at-rules as functions is a ledger-validated design choice (`css/function/existence/agree[gitgalaxy,tree_sitter]_vs[ctags]`, status validated). The gated `args_exact_match` is unaffected. No golden-master re-bless: no engine/regex change. Claude-Session: https://claude.ai/code/session_01Tpu4YmAESTUeSoPQ5LqLWy Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RELEASING.mdstep 2 for language-crucible v1.2.0: regenerate every corpus-pinned fixture against the new corpus, then bump the pin.What moved and why
The corpus grew 152
data/folders / 2,344 files → 218 / 3,306 (newsqlite+embedded_pythoncategories; assembly ×17, groovy ×8 — was.java-only — cobol ×1.9 with the NIST COBOL-85 CCVS suite, tcl/html/css expansions). The engine is unchanged on this branch — every number below moved because the test corpus got bigger and harder, not because the parser regressed.Regenerated per
docs/self_scan/BUMPING_THE_CRUCIBLE_PIN.md, againstlanguage-cruciblemain@77bc85e(= the future v1.2.0 tag), in.crucible_venvs/full_precision+LANGUAGE_CRUCIBLE_PATHpointed at a clean../language-crucibleworktree.flag-golden-master-changesis non-blocking for exactly this.unvalidatedshapes (css 2, embedded_python 2, sqlite 2, groovy/tcl/html 1 each) — all new corpus content.tri_comparison_report.py --writemeasure()+write bypass (the absolute-count gate refuses on any grown category; see the doc's "Known gap"). Detail below.PINNED_TAG→v1.2.0tests/README.md,docs/self_scan/README.md,tri_comparison_README.md,docs/language_status/csharp.md, one comment intree_sitter_accuracy_audit.py. Historical v1.1.0 refs left as-is.v1.1.0${{ vars.LANGUAGE_CRUCIBLE_REF }}.The 4 tree-sitter-accuracy baseline drifts (read the examples, per the doc)
css—extra_functions 0 → 5, precision100% → 83.3%. GitGalaxy's CSSfunc_startcounts@layer/@keyframes/@media/@supportsat-rules as functions. Genuine imprecision, newly exposed by Tailwind v4 + Gutenberg CSS-Modules content. → worth a follow-up issue.groovy—extra_functions 0 → 721, recallN/A → 0.0%. tree-sitter finds ZERO real functions across all 188 groovy files — its Groovy grammar gives no usable ground truth (the v1.1.0 baseline was all-zeros only because old groovy corpus was.java). The 721 are unverified GitGalaxy claims, not confirmed FPs. That said, spot-checking shows realfunc_startFPs on the Jenkins builder-DSL (div/span/node/stagemethod-calls-as-markup) and the baredefkeyword. → worth a follow-up issue; arguablygroovyshould be marked tree-sitter-blind for this audit.html—extra_functions 0 → 1:<script>counted once in a reveal.js deck.tcl—extra_functions 1 → 3, recall99.3% → 99.1%: namespaced procs (ns::proc) + SQLite test-harness procs; per-file rate flat/improved.language_standards.py's summary table resynced (--summary-table) — the only non-table change to that file.Merge sequence — CI is red until steps 2–3 below
crucible-audit/tri-comparison-audit/tree-sitter-accuracy-auditclone against the tag / repo variable, notPINNED_TAG— so they'll fail until:RELEASING.mdstep 3 waits for).language-crucible:git tag -a v1.2.0 -m "…" 77bc85e && git push origin v1.2.0, thengh release create v1.2.0 --notes-file …(draft notes staged).gh variable set LANGUAGE_CRUCIBLE_REF --body v1.2.0 --repo squid-protocol/gitgalaxy.🤖 Generated with Claude Code