Part of #2560 — per-language tracking for the keyword-rosetta control corpus (built for #1096). Planted intent is identical in every language, so every deviation below is measured bias vs. the 46-language median, validated in the corpus's deviation ledger. 🔴 = beyond ±50% of the cross-language median, 🟡 = ±25–50%.
Triage order (per the epic): structure detection first, then signal/keyword rules, then risk-score consequences — §3's numbers are downstream of §1–§2 and re-baseline as those fix. Cross-cutting engine issues (e.g. #2535/#2546) fix many languages at once; this issue tracks this language's residual after each cross-cutting fix lands, and closes when the language sits inside the ±25% band on every comparable metric (or every remaining deviation is ledgered as intended morphology).
1. Structure detection (fix first)
In band — no structure-level deviations.
2. Signal / keyword deviations
- 🔴
comment_lines: 52 vs median 30 (+73%)
- 🔴
doc: 2 vs median 1 (+100%)
- 🔴
io: 6 vs median 3 (+100%)
- 🔴
state_mutation: 2 vs median 6 (-67%)
- 🟡
safety: 3 vs median 2 (+50%)
- 🟡
test: 3 vs median 2 (+50%)
3. Risk-exposure consequences (downstream)
- 🔴
risk_cognitive_load: 4.821 vs median 15.1444 (-68%)
- 🟡
risk_api_exposure: 7.35003 vs median 5.31104 (+38%)
Ledgered shapes touching python
| defect |
type |
issue |
evidence |
summary |
api-contextual-baseline-fix |
engine-semantic |
#2536 |
— |
galaxyscope.py ~2145 ('THE CONTEXTUAL BASELINE FIX'): a file imported by others (popularity > 0) has orphaned_logic converted into api before recording |
census-requires-git-tracked |
engine-semantic |
— |
— |
GalaxyScope's census enumerates git-tracked files only; an untracked folder scans as '0 files mapped' with no per-file warning |
assert-overlaps-safety-and-test |
keyword-overlap |
— |
— |
python's test rule includes \bassert\b and the safety rule includes assert: one planted assert counts both |
os-sys-prefix-overlaps-io |
keyword-overlap |
— |
— |
python-family io rule matches the os./sys |
string-literal-selective-shielding |
upstream-question |
#2535 |
— |
Keywords inside one string literal ('if eval fails, try open') count toward branch (+2), safety (+1), io (+1) but eval does NOT count toward high_risk_execution (routes … |
Decoy outcomes (file-level evidence)
- main.py ·
never calls eval and no while loop (comment): stripped: contributed 0 (prism comment stripping works)
- a.py ·
could open a socket in prose only (comment): stripped: contributed 0
- a.py ·
MESSAGE = "if eval fails, try open" (string): PARTIAL: if/try/open COUNT from inside the literal (branch +2, safety +1, io +1, baked into a.py expectations) but eval does NOT (feeds sec_tainted_injection instead) — literal shielding is selective by signal family; candidate upstream issue
- b.py ·
the try keyword stays in prose (comment): stripped: contributed 0
- c.py ·
HACK_LEVEL = level (code): reverse decoy works: identifier did not count, the # HACK comment did (fragile_debt = 1 exactly)
Reproduce / evidence
Part of #2560 — per-language tracking for the keyword-rosetta control corpus (built for #1096). Planted intent is identical in every language, so every deviation below is measured bias vs. the 46-language median, validated in the corpus's deviation ledger. 🔴 = beyond ±50% of the cross-language median, 🟡 = ±25–50%.
Triage order (per the epic): structure detection first, then signal/keyword rules, then risk-score consequences — §3's numbers are downstream of §1–§2 and re-baseline as those fix. Cross-cutting engine issues (e.g. #2535/#2546) fix many languages at once; this issue tracks this language's residual after each cross-cutting fix lands, and closes when the language sits inside the ±25% band on every comparable metric (or every remaining deviation is ledgered as intended morphology).
1. Structure detection (fix first)
In band — no structure-level deviations.
2. Signal / keyword deviations
comment_lines: 52 vs median 30 (+73%)doc: 2 vs median 1 (+100%)io: 6 vs median 3 (+100%)state_mutation: 2 vs median 6 (-67%)safety: 3 vs median 2 (+50%)test: 3 vs median 2 (+50%)3. Risk-exposure consequences (downstream)
risk_cognitive_load: 4.821 vs median 15.1444 (-68%)risk_api_exposure: 7.35003 vs median 5.31104 (+38%)Ledgered shapes touching python
api-contextual-baseline-fixcensus-requires-git-trackedassert-overlaps-safety-and-testos-sys-prefix-overlaps-iostring-literal-selective-shieldingDecoy outcomes (file-level evidence)
never calls eval and no while loop(comment): stripped: contributed 0 (prism comment stripping works)could open a socket in prose only(comment): stripped: contributed 0MESSAGE = "if eval fails, try open"(string): PARTIAL: if/try/open COUNT from inside the literal (branch +2, safety +1, io +1, baked into a.py expectations) but eval does NOT (feeds sec_tainted_injection instead) — literal shielding is selective by signal family; candidate upstream issuethe try keyword stays in prose(comment): stripped: contributed 0HACK_LEVEL = level(code): reverse decoy works: identifier did not count, the # HACK comment did (fragile_debt = 1 exactly)Reproduce / evidence
data/python/—expected_signals.jsoncarries the shape-by-shape accountingGALAXYSCOPE_BIN=<venv>/bin/galaxyscope python tools/verify_language.py pythonin the corpus repo