Summary
gitgalaxy/metrics/statistical_auditor.py's Heuristic Extension Consensus (lines ~127-219) sends every artifact with tier >= 4 or "Collision" in proof to ambiguous_artifacts, then in the loop-back deletes any it can't confirm:
# If we reach here, the file was ambiguous and the ecosystem couldn't save it.
# Banish it to unparsable_files immediately to prevent hallucinations.
reason = "Unresolved Ambiguity (Tier 4 Fallback failed Ecosystem Consensus)"
unparsable_files.append(self._format_for_exclusion(artifact, reason))
The only ways to survive: an 80%+ ecosystem vote from confidently-parsed siblings of the same extension, or the .h/.hpp/.inc C-family fallback. Anything else that was ambiguous is dropped from file_data entirely — even when it classified to a real supported language and extracted real functions.
This was the mechanism behind #1926 (every .y grammar deleted, because .y is always a collision and can never accumulate a confident-sibling vote).
Partial mitigation already shipped
#2324 added a "decisive collision-resolution keep" to that branch: a file that came in via a genuine extension collision, resolved with identity confidence >= 0.85, and produced extracted structure is now kept at reduced confidence instead of deleted. Bare Tier-4 lexical guesses (no extension) are still banished.
Remaining question
The broader default — "delete to prevent hallucinations" — still applies to:
- extension collisions that resolve with lower confidence
- bare Tier-4 lexical-discovery files with real extracted structure
Is silent deletion the right call there, vs. keeping at low confidence / surfacing them in a distinct "low-confidence identity" bucket that still reaches file_data? A file GitGalaxy extracted 50 functions from is a strange thing to make completely invisible with no signal to the user beyond a line in the Excluded Artifacts forensic list.
Found via #1926 / #2324. Related: #2325 (the density gates downstream of this triage don't actually run).
Summary
gitgalaxy/metrics/statistical_auditor.py's Heuristic Extension Consensus (lines ~127-219) sends every artifact withtier >= 4 or "Collision" in prooftoambiguous_artifacts, then in the loop-back deletes any it can't confirm:The only ways to survive: an 80%+ ecosystem vote from confidently-parsed siblings of the same extension, or the
.h/.hpp/.incC-family fallback. Anything else that was ambiguous is dropped fromfile_dataentirely — even when it classified to a real supported language and extracted real functions.This was the mechanism behind #1926 (every
.ygrammar deleted, because.yis always a collision and can never accumulate a confident-sibling vote).Partial mitigation already shipped
#2324 added a "decisive collision-resolution keep" to that branch: a file that came in via a genuine extension collision, resolved with identity confidence >= 0.85, and produced extracted structure is now kept at reduced confidence instead of deleted. Bare Tier-4 lexical guesses (no extension) are still banished.
Remaining question
The broader default — "delete to prevent hallucinations" — still applies to:
Is silent deletion the right call there, vs. keeping at low confidence / surfacing them in a distinct "low-confidence identity" bucket that still reaches
file_data? A file GitGalaxy extracted 50 functions from is a strange thing to make completely invisible with no signal to the user beyond a line in the Excluded Artifacts forensic list.Found via #1926 / #2324. Related: #2325 (the density gates downstream of this triage don't actually run).