Skip to content

Security: fail-closed label parsing, explicit reviewer decision, CI secret scan - #22

Open
bunytwirls wants to merge 2 commits into
mainfrom
security/parser-and-decision-fixes
Open

bunytwirls wants to merge 2 commits into
mainfrom
security/parser-and-decision-fixes

Conversation

@bunytwirls

Copy link
Copy Markdown
Collaborator

Summary

Security & verification fixes for two decision-integrity bugs, plus documentation and CI hardening.

Bugs fixed

1. Parser could invert the model's meaning
The non-JSON fallback searched the whole response for a label word, so
"The evidence does not show the claim is SUPPORTED." was recorded as SUPPORTED.
JSON with an invalid label (e.g. {"label": "APPROVED", "rationale": "...UNSUPPORTED..."}) also fell through to word matching.

Fix: plain-text output is only accepted when the label stands on its own line (UNSUPPORTED, Label: SUPPORTED). A label word inside prose, conflicting labels, or an invalid JSON label now fail closed to UNPARSEABLE (parse_status=invalid_label for the JSON case).

2. Reviewer could record a decision without making one
The "Final decision" radio defaulted to SUPPORTED, so an untouched form saved a SUPPORTED decision.

Fix: no default selection (index=None), and reviewer_submission_errors rejects a missing or invalid decision with "Choose a final decision."

Tests

  • 8 new tests (test_parser.py, test_review_state.py, test_ui_security.py)
  • Full suite: 58 passed; 6 of the new tests fail on the previous code
  • Frozen evaluation results are unaffected: every row in results/predictions.csv and results/security_probes.csv has parse_status=json

Documentation

  • docs/SECURITY.md: parser and decision-log controls updated; coverage list includes prose/invalid labels and missing final decision
  • docs/SECURITY_TEST_MATRIX.md: acceptance cases 11–12 added; A03 row notes CI permissions and secret scanning

CI / build hardening

  • ci.yml: permissions: contents: read (read-only CI token)
  • ci.yml: new secret-scan job runs gitleaks v8.21.2 (checksum-verified) over the full Git history on every push and PR
  • .dockerignore: excludes .env.* and local_decisions.csv from the container image

Scan results (2026-11-13)

  • gitleaks (full history): no leaks found, 56 commits scanned (~350 KB)
  • pip-audit (requirements.txt): No known vulnerabilities found

Not verified

Live Google Cloud IAM bindings and the final Cloud Run redeploy were not checked (no project access). Configuration was reviewed from scripts/deploy_cloud_run.sh and docs/GCP_DEPLOYMENT.md only.

How to verify

pip install -r requirements.txt
python -m pytest -q
python scripts/validate_dataset.py

Verified with tests and scans.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant