fix(core): wire lang_defs into StatisticalAuditor; drop the Low-Sample Guard (#2325) - #2331
Merged
Merged
Conversation
…e Guard (#2325) `Orchestrator.__init__` constructed `StatisticalAuditor(parent_logger=logger)` with no `lang_defs`, so `self.lang_defs == {}` on every real scan. In `audit()`'s per-language loop the Dynamic Auditability Check then took its `else: is_inert = True` branch for *every* language ("Unknown/Undefined languages are inert by default"), and the `if is_inert` bypass short-circuited each group straight into `verified_files`. Net effect: GATE C, GATE D (MAD baseline / Impossible Density Law / Zero-Density floor), and dead-code detection never executed in production -- only the pre-loop Heuristic Extension Consensus triage did. The gate logic is covered by test_statistical_auditor.py, which passes `lang_defs` explicitly, so the tests always exercised code the real pipeline never reached. - `galaxyscope.py` -- pass `lang_defs` to the constructor. GATE D's data-dump / packed-payload / MAD floors and dead-code detection now run. - `statistical_auditor.py` -- DELETE GATE C (the "Low-Sample Threshold Guard"). It reverted any language with <= orphan_threshold files to plaintext unless every file carried a Tier-0 "Absolute Consensus" lock -- but a clean single-extension file locks at Tier 2 "Single Indicator (Ext)", never Tier 0, so activating it would have erased ~13 legitimately-classified languages (css, kotlin, nix, objective-c, scheme, proto, tcl, yacc, ...) from any repo with a small footprint of them. Small-sample hallucination is already handled upstream by the consensus triage and the Tier-4/undeterminable pre-filters. Removed the now-dead `orphan_threshold` computation and `math` import. - `test_statistical_auditor.py` -- replaced the GATE C test with one asserting a small weak-tier population is now kept intact. Corpus impact (verified via crucible_check, both modes): GATE D now relegates exactly ONE file across the whole ~80-repo corpus -- `javascript/threejs/Nodes.js`, a 166-line pure re-export barrel (0 functions, ~140 `export ... from` lines) tripping the Packed Payload Guard at 3.32 signal-hits/line. Every other diff in the golden-master rebless is downstream count / coordinate ripple from that single drop. No language reclassified, no other relegations. Verification: test_statistical_auditor.py (9) + tests/security_auditing + tests/core_engine (717) green; ruff/mypy/dead-key/ast-accuracy clean (ruff baseline: one PERF203 line-shift); crucible_check both modes PASS; tri_comparison_chart --all --ci green. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
The docstring note added in the previous commit shifted the pre-existing try/except from line 330 to 337. Pure line-shift, no new finding. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
squid-protocol
added a commit
that referenced
this pull request
Aug 27, 2026
…#2309) (#2335) `_count_top_level_args` unwraps a signature's outer `(...)` and counts depth-0 commas in the remaining body. For a Dart parameter list whose parameters live in a named group `({a, b, c})` or an optional-positional group `([a, b])`, the group's own `{`/`[` was read as ordinary nesting -- so every comma inside sat at depth 1 and none counted. A Flutter widget constructor like `EditableText({super.key, required this.controller, ...})` measured **1** argument instead of 76; `_Editable`, `_UpdateTextSelectionAction` and ~dozens more were similarly flattened. This is the args half of #2309 (the `func_start` existence half, and the bodyless `this.`/`super.`-forwarding constructor case, were handled by #2071 / #2311). Fix: in `_count_top_level_args`, for `primary_lang_id == "dart"`, when the parameter-list body ends (ignoring trailing whitespace) in `}` or `]`, locate that terminal group's matching opener at depth 0 and blank just those two delimiter characters, so the group's members land at the top level and their separating commas are counted. A `{...}` / `[...]` that is a default-VALUE literal (`{int x = const [1, 2, 3]}`, `{Map m = const {'a': 1}}`) is never flush against the body's end, so it stays nested and its internal commas stay correctly ignored. Scoped to dart -- no other language's count path is touched. Result (name-matched `gather_language("dart")` args diff vs tree-sitter): GitGalaxy under-counts by >=2 (excluding tree-sitter's own same-name reconciliation inflation) drop from ~40 to 2, and those 2 are tree-sitter / reconciler quirks, not GitGalaxy defects (`build` at editable_text.dart:6253 is genuinely 1 param; `copyWith` at theme_data.dart:140 is a bodyless abstract method with 0). `EditableText` (1->76), `_Editable` (1->39), `_UpdateTextSelectionAction` (4->6) now match tree-sitter exactly. Regenerated: - Both golden masters -- 328 diffs, all dart (arg counts + downstream structural-impact / spatial-coordinate ripple); 0 non-dart content change. - `tree_sitter_accuracy_baseline_dart.json` -- `args_exact_match` 1519 -> 1633 (+114); summary table in `language_standards.py` Dart row 99.4% -> 99.5%. - `tree_sitter_accuracy_baseline_javascript.json` -- `files_scanned` 18 -> 17, a stale value left by #2331 (GATE D relegating `threejs/Nodes.js`); the tree-sitter-accuracy workflow doesn't trigger on statistical_auditor.py paths so it was never updated. Included here so this PR's own tree-sitter-accuracy-audit run (which `detector.py` does trigger) is green. Verification: test_detector.py (+ new `_count_top_level_args` dart case) and tests/extraction (6234) green; ruff/mypy/dead-key/ast-accuracy clean; crucible_check both modes PASS; tree_sitter_accuracy_audit --all --ci and tri_comparison_chart --all --ci both green. Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2325.
The bug
Orchestrator.__init__builtStatisticalAuditor(parent_logger=logger)— nolang_defs. Soself.lang_defs == {}on every real scan, and inaudit()'s per-language loop the Dynamic Auditability Check always took itselse: is_inert = Truebranch ("Unknown/Undefined languages are inert by default"). Theif is_inert:bypass then short-circuited every language group straight intoverified_files.Net effect: GATE C, GATE D (MAD baseline / Impossible Density Law / Zero-Density floor), and dead-code detection never executed in production — only the pre-loop Heuristic Extension Consensus triage did.
test_statistical_auditor.pypasseslang_defsexplicitly, so the tests always exercised code the real pipeline never reached.Fix
Per the decision on #2325 (wire it up, but drop GATE C):
galaxyscope.pylang_defsto the constructor — GATE D's data-dump / packed-payload / MAD floors and dead-code detection now runstatistical_auditor.pyorphan_thresholdfiles to plaintext unless every file had a Tier-0 lock — but a clean single-extension file (foo.css,foo.kt,foo.nix) locks at Tier 2 "Single Indicator (Ext)", never Tier 0. Verified: activating it would erase ~13 legitimately-classified languages (css, kotlin, nix, objective-c, scheme, proto, tcl, yacc, dockerfile, …) from GitGalaxy's own corpus. Small-sample hallucination is already handled upstream by the consensus triage + Tier-4/undeterminable pre-filters. Also removed the now-deadorphan_threshold/mathimport.test_statistical_auditor.pyCorpus impact — one file
GATE D, now active, relegates exactly one file across the whole ~80-repo corpus:
javascript/threejs/Nodes.js— a 166-line pure re-export barrel (0 functions, ~140export … fromlines) tripping the Packed Payload Guard at 3.32 signal-hits/line. It's a borderline catch (a hand-written index file, not minified/obfuscated), but it carries no analyzable logic and it's the guard behaving as specified/tested.Everything else in the golden-master rebless is downstream count / coordinate ripple from that single drop — no language reclassified, no other relegations (verified with a structural per-file diff). Both golden masters re-blessed;
crucible_checkPASS both modes.Verification
test_statistical_auditor.py(9) +tests/security_auditing+tests/core_engine— 717 passcrucible_check.py— PASS both modestri_comparison_chart.py --all --ci— green (tree-sitter-accuracy-audit/tri-comparison-auditdon't trigger on these file paths;crucible-audit+full-suiteare the gates)Note
Nodes.jsbeing silently dropped is exactly the kind of thing #2326 (consensus engine silently deletes cleanly-extracted files) is about — if the Packed Payload Guard should distinguish hand-written barrel files from packed content (e.g. gate onis_minifiedor line length), that's a good follow-up there.🤖 Generated with Claude Code