fix(cobol): #2480 func_start sequence-area bypass + cobol function precision to 100% - #2489
Merged
Merged
Conversation
…ecision to 100% ## #2480 -- `066600DDEBUG-LINE-TEST-05-A` name mangle `che-che4z_nist_ccvs85/DB1034.2.cbl:665` is a debug-line paragraph (`066600DDEBUG-LINE-TEST-05-A.` -- col-1-6 sequence number, col-7 `D` debug indicator, name glued on) bracketed by `*` comment lines. prism.py blanks those comment lines to empty lines before detector.py runs func_start, leaving a run of `\n` in front of the real line. The func_start anchor's post-sequence-area slot was `[ \t\n]*`. With re.M, `^` could match on one of those blank lines; the optional 6-char sequence-area group then matched empty (a `\n` is not `[0-9a-zA-Z \t]`), and `[ \t\n]*` skipped forward across the newline onto the real content line -- past the sequence-area shield -- so `066600` + `D` were swept into the identifier capture. Fix: narrow that slot to `[ \t]*` (horizontal whitespace only). `^` already re-anchors on the real line under re.M, and the one genuine vertical gap (a paragraph name and its `SECTION` keyword on separate physical lines) is consumed by the section-6 lookahead's own `[ \t\n]+SECTION`, not here. Re-adding a blank-line consumer was tried and rejected -- a second `[ \t]*` after the original `[ \t\n]*`, and a leading `(?:[ \t]*\n)*`, were both measurably catastrophic on a whitespace-heavy pathological input (12s vs 1.3s on the cobol corpus). Verified corpus-wide (541 cobol files): exactly one captured func name changes, `066600DDEBUG-LINE-TEST-05-A` -> `DEBUG-LINE-TEST-05-A`; total func-name count unchanged; no ReDoS. ctags tags `DEBUG-LINE-TEST-05-A` correctly, so GitGalaxy now agrees with ctags on that slot. ## Golden masters -- cobol Start/End line corrections (143 diffs each) The same `[ \t\n]*` -> `[ \t]*` change also corrects a milder, corpus-wide manifestation: for any COBOL paragraph preceded by a comment banner or blank lines, the old anchor reported a Start Line N lines too early (anchored on the preceding blank/comment line the regex then skipped over). Both golden masters re-blessed. Every diff is a Start/End line shift by an identical positive amount (+1..+50) with LOC, branch count, and the function-name set byte-identical -- the slice span is the same, just correctly located. Zero functions gained or lost anywhere (beyond the one #2480 name correction). Downstream `avg_documentation` ripples because per-function doc-exposure keys off the line range. ## cobol function precision 99.90% -> 100% Raw reconcile precision was 10130/10140. Of the 10 uncorroborated GitGalaxy-only occurrences: 1 was #2480 (now agrees with ctags -> 10131); the other 9 (`NUMBER1/2/3` in SG3034.2.cbl and SG4014.2.cbl, `SECT-IC219-0001/0002/0003` in OBIC24.2.cbl) are all segment-numbered SECTION headers (`NUMBER1 SECTION 18.`, `SECT-IC219-0001 SECTION 30.`) -- COBOL-68/74 program segmentation, still accepted by modern compilers. Universal Ctags 5.9.0's Cobol parser cannot parse a segment-numbered section header at all (emits zero tags -- verified directly on all 9); GitGalaxy handles them via its `SECTION(?:[ \t\n]+[0-9]{1,2})?` allowance. This is a textbook `credit_tools` case (mirrors the already-validated sibling `cobol/class/existence/agree[gitgalaxy]_vs[ctags]`). Set `credit_tools: ["gitgalaxy"]` on `cobol/function/existence/agree[gitgalaxy]_vs[ctags]`; ledger-adjusted cobol function precision is now 10140/10140 = 100%, has_open_question False. Corrected the ledger verdict's stale claim that this shape was "still the #1891 harness {"p"} filter" -- #1891 was fixed 2026-08-22 in #2121; `CTAGS_FUNC_KINDS["cobol"]` has carried section kind "s" since. Added the segment-numbered-SECTION limitation to ctags_reader.py's cobol notes and as Claim 12's fourth instance in why_gitgalaxy_beats_ast_here.md. Verification: cobol + Mode-A sibling (assembly/abap/agc_assembly) extraction & strict gauntlets, test_language_standards_strict, ruff/mypy/dead-key/ast-accuracy audits, crucible_check both modes full corpus, tree-sitter-accuracy --all --ci, tri-comparison --all --ci, both golden masters, golden_crucible marker -- all green. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AxT15g1E4ybUDLvUwoZMcj
squid-protocol
added
bug
Contributor
…esolution `test_worker_strips_leading_utf8_bom` passed a raw `tempfile.TemporaryDirectory()` path as `_init_worker`'s `root_str`. On macOS that path is under `/var/...` but resolves to `/private/var/...`, and on Windows CI it comes back as an 8.3 short name (`C:\Users\RUNNER~1\...`) that resolves to `C:\Users\runneradmin\...`. GuideStarLens and ApertureFilter both `.resolve()` their own root internally, so `path_obj.relative_to(self.root)` raised `ValueError: ... is not in the subpath of ...` before the BOM assertions ran -- failing every macos/windows leg of full-suite-matrix while ubuntu (no /tmp symlink indirection) passed. The real orchestrator always resolves its scan target (`Path(args.target).resolve()`), so the fix is to make the test mirror that: `td = str(Path(td).resolve())`. Pre-existing since #2430 added the test; only surfaced now because full-suite-gate runs on PR-label and this branch is the first labeled PR since. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AxT15g1E4ybUDLvUwoZMcj
squid-protocol
added
threat: false-positive
This was referenced Aug 31, 2026
squid-protocol
added a commit
that referenced
this pull request
Aug 31, 2026
…aphs (#2552) Closes #2538. `CEE3DMP.`/`CEEMOUT.`/`CEEDUMP.` (Language Environment runtime diagnostic calls) sitting alone on a line indented into Area B were counted as paragraph headers -- func_start's reserved-word shield had no entry for them, and the post-sequence-area whitespace slot tolerates any horizontal indent to support genuinely free-format source, so it can't distinguish Area A from Area B on its own. Traced #2489 (the previous func_start sequence-area hardening) to confirm this wasn't a deliberate decision -- that PR only fixed vertical-whitespace handling, unrelated to column position. Fix: add CEE3DMP/CEEMOUT/CEEDUMP to the reserved-word shield, the same low-risk pattern already used repeatedly in this file (CONTINUE, LOCAL-STORAGE, SOURCE-COMPUTER/OBJECT-COMPUTER). These three are already a recognized class elsewhere in cobol.py (the `telemetry` rule). A general Area-A column anchor was considered and deferred -- the existing sequence-area consumer can't reliably tell a genuine fixed-format sequence number from free-format leading spaces without file-level fixed/free-format detection, a bigger change than this bug warrants. Verification: reproduced the exact FP (5 matches for 3 real paragraphs, now 3), added cases to test_cobol.py/test_cobol_strict.py, full cobol + cross-language strict suites pass, audit_check.py clean, crucible_check.py both modes PASS against the full corpus with zero diffs (these tokens aren't present in the existing corpus). Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2480.
#2480 —
066600DDEBUG-LINE-TEST-05-Aname mangleche-che4z_nist_ccvs85/DB1034.2.cbl:665is a debug-line paragraph bracketed by*comment lines:prism.pyblanks the two comment lines to empty lines beforedetector.pyrunsfunc_start, leaving\n\n\nin front of line 666. The anchor's post-sequence-area slot was[ \t\n]*. Underre.M,^matched on a blank line; the optional 6-char sequence-area group matched empty (a\nisn't[0-9a-zA-Z \t]); then[ \t\n]*skipped forward across the newline onto the real line — past the sequence-area shield — so066600+Dwere swept into the identifier capture.Fix: narrow that one slot,
[ \t\n]*→[ \t]*(horizontal whitespace only).^already re-anchors on the real line underre.M; the one genuine vertical gap (a name and itsSECTIONkeyword on separate physical lines) is consumed by the section-6 lookahead's own[ \t\n]+SECTION, not here.A blank-line consumer was tried and rejected: a second
[ \t]*after the original[ \t\n]*, and a leading(?:[ \t]*\n)*, were both measurably catastrophic on a whitespace-heavy pathological input (12s vs 1.3s on the cobol corpus). New ReDoS cases intest_cobol.pypin this down.Corpus-wide verification (541 cobol files): exactly one captured func name changes —
066600DDEBUG-LINE-TEST-05-A→DEBUG-LINE-TEST-05-A. Total func-name count unchanged.ctagstagsDEBUG-LINE-TEST-05-Acorrectly, so GitGalaxy now agrees with it on that slot.Golden masters — cobol Start/End line corrections (143 diffs each, both fixtures re-blessed)
The same change also corrects a milder, corpus-wide manifestation of the same bug: for any COBOL paragraph preceded by a comment banner or blank lines, the old anchor reported a
Start LineN lines too early (it anchored on the preceding blank/comment line and then skipped over it).Every golden-master diff is a
Start Line/End Lineshift by an identical positive amount (+1..+50), withLines of Code,Control Flow Branches, and the function-name set byte-identical — the slice span is the same, just correctly located. Zero functions gained or lost anywhere (beyond the one #2480 name correction).avg_documentationripples slightly because per-function doc-exposure keys off the line range.cobol function precision 99.90% → 100%
Raw reconcile precision was
10130/10140. Of the 10 uncorroborated GitGalaxy-only occurrences incobol/function/existence/agree[gitgalaxy]_vs[ctags]:066600DDEBUG-LINE-TEST-05-A(DB1034.2.cbl)NUMBER1/2/3(SG3034.2.cbl)NUMBERn SECTION 18.— segment-numbered SECTION headerNUMBER1/2/3(SG4014.2.cbl)SECT-IC219-0001/0002/0003(OBIC24.2.cbl)SECT-IC219-000n SECTION 30.— sameNAME SECTION nn.is COBOL-68/74 program segmentation (segment-priority number), still accepted by modern compilers. GitGalaxy handles it viafunc_start'sSECTION(?:[ \t\n]+[0-9]{1,2})?allowance. Universal Ctags 5.9.0's Cobol parser cannot parse a segment-numbered section header —ctags --kinds-Cobol='*'emits zero tags (neithersnorp) for all 9 lines, verified directly on all three files.Textbook
credit_toolscase, identical pattern to the already-validated siblingcobol/class/existence/agree[gitgalaxy]_vs[ctags](split-line PROGRAM-ID). Setcredit_tools: ["gitgalaxy"]on the shape → ledger-adjusted cobol function precision is now 10140/10140 = 100%,has_open_questionFalse. The chart SVG updates10130/10140→10140/10140.Also corrected a stale ledger verdict: it claimed this shape was "still the #1891 harness
{"p"}filter" — #1891 was fixed 2026-08-22 in #2121;CTAGS_FUNC_KINDS["cobol"]has carried section kind"s"since. Added the segment-numbered-SECTION limitation toctags_reader.py's cobol notes and as Claim 12's fourth instance inwhy_gitgalaxy_beats_ast_here.md.Verification
test_language_standards_strict,test_tri_comparison_ledger,test_tri_comparison_reconcileaudit_check.py(ruff / mypy / dead-key / ast-accuracy) — all clearcrucible_check.pyboth modes, full corpus — both PASStree_sitter_accuracy_audit.py --all --ci— all OKtri_comparison_chart.py --all --ci— all OK-m golden_crucible— passtests/extraction/sweep — 6277 passed🤖 Generated with Claude Code