Strix gate treats findings whose cited path:line does not exist in the scanned tree as blocking vulnerability evidence

[seonghobae]

What happened

.github#1661 head 24005da6, Strix job 101359146403 (run 33981164214, created 2026-09-05T17:30:21Z, sidecar at contextual-orchestrator@2e414d15). Strix completed a scan (1.5 M input tokens) and reported three CRITICAL findings in scripts/ci/noema_review_gate.py at lines 3018, 3244-3249, 3256, 3300, 3400. The file on that head is 2062 lines long. Every cited location is past end-of-file; each PoC block is a # Simulate … sketch rather than an executed probe. Read against the actual module, none of the three vulnerability classes has a sink: the single subprocess wrapper run() (:282-300) takes an argv list with shell=False and scrubs stderr (:296, patterns :264-270) before raising; parse_diff_path() (:558-569) feeds only an in-memory locations set (:545, :550), and the module has no open()/Path()/os.path sink. Details with line references on #1661.

The required check still failed: strix.yml:957 reported_vulnerability_signal='Vulnerabilities[[:space:]]+[1-9]|…severity…:' matched the summary box, and :1012 exited 1 ("Strix reported security findings or failed for a non-backend reason").

Why this is a gate gap, not a one-off

The governance model for the review path already refuses LLM claims that are not bound to the tree: README.md/CLAUDE.md state that deterministic code may repair trusted path:line bindings only on probes that carry an independent proof and a source-line digest, and never invents observed results. The Strix path has no equivalent step — a finding is blocking evidence as soon as the scanner prints a non-zero vulnerability count, whether or not its locations exist. A hallucinated location is the security-scan analogue of the ModelBehaviorError case already handled in docs/doctoring/strix-model-behavior-error.md: the model did not follow the contract, so the scan is not authoritative evidence in either direction.

Constraint that must survive any fix

docs/doctoring/strix-model-behavior-error.md: "Do not remove the vulnerability fail-closed guard." A finding with a bad location must not become passing evidence. The right shape is the one that doc already uses for ModelBehaviorError: classify the run as typed non-passing model evidence and move to the configured cross-model fallback, never a same-model retry, never a silent drop.

Proposed direction (not implemented here)

1. In the trusted gate (scripts/ci/strix_quick_gate.sh / the report parser), after a scan reports findings, resolve each finding's Location N: <path>:<line>[-<line>] against the scanned tree: the path must exist in the PR scope and every cited line must be ≤ the file's line count on the scanned head.
2. If every reported finding fails that check, emit a typed marker (e.g. STRIX_UNBOUND_FINDING_LOCATIONS) in the console log; the outer strix.yml classifier treats that marker like model_behavior_error_signal — typed, non-passing, eligible for cross-model fallback — while a mix of bound and unbound findings stays a plain vulnerability failure.
3. Contract tests in the style of tests/test_strix_model_behavior_error.py: (a) three findings all past EOF → typed non-passing + fallback; (b) one bound finding among unbound ones → fail closed unchanged; (c) a bound finding → fail closed unchanged; (d) Vulnerabilities 0 unaffected.
4. Record the classifier in docs/doctoring/ next to the ModelBehaviorError note, with this run as the incident.

Not in scope

Whether Strix's line numbers were offset by the scope copy (/workspace/strix-pr-scope.*) or invented outright is unknown; either way the cited lines do not exist in the file, so the check in (1) is the same. The noema-review failure on the same head (validator rejected a review that cited an array index as a source line, noema_review_gate.py:594-650) is the same class on the review side and is already fail-closed there — this issue is only about the security-scan side lacking the equivalent binding check.