ci: scan issue and comment bodies — this repo has never scanned one - #78
ci: scan issue and comment bodies — this repo has never scanned one#78yakimoto wants to merge 8 commits into
Conversation
…ment body Measured across all 28 public wave-av repos (claude-workstation#1747, #1794): TWO coverage shapes satisfy the one required check name `Secrets + content policy`. 27 repos triggers: pull_request, push, workflow_dispatch jobs: guard 1 repo triggers: + issues, issue_comment jobs: + body-guard This repo is in the 27. All 28 report the same green check. The outlier is wave-moq-edge, and its own comment says why it matters: "`edited` matters as much as `opened`: a body can be made to leak long after the PR is first raised, and until this workflow covered it, nothing ever re-scanned." A PR/issue/comment BODY is exactly as world-readable as the tree, and until now it was scanned by nothing server-side. That gap was not theoretical on wave-moq-edge: a PR was blocked for naming a private repo in wrangler.toml while the very same name, with more operational detail attached, sat unchallenged in its body. WHAT LANDS HERE — the bundle the workflow's own header names, minus what this repo already has (.gitleaks.toml and content-policy.sh are already vendored): .github/workflows/public-repo-guard.yml replaced (73 -> 163 lines) scripts/public-repo-guard/body-policy.sh new, mode 100755 scripts/public-repo-guard/tests/body-policy.test.sh new, mode 100755 Copied from wave-moq-edge, which has run this shape in production. Modes preserved via the git trees API — the contents API would have created both scripts 100644. HONEST ABOUT WHAT IT CAN DO. On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication. Also inherited from the reference: concurrency moves from workflow-level to PER JOB, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green. The body gate ships with its own fixtures and runs them in CI. Its NEGATIVE cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing. Refs wave-av/claude-workstation#1747. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are limited based on label configuration. 🚫 Excluded labels (none allowed) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_9af9e0c1-18c9-4915-a979-c562aab15a80) |
ApprovabilityVerdict: Needs human review Unable to check for correctness in c82fc3d. This PR introduces a significant new security scanning feature for PR/issue/comment bodies with complex allowlist logic. Unresolved review comments raise potential bypass concerns in the allowlist design that warrant human evaluation. You can customize Macroscope's approvability policy. Learn more. |
|
Running ultrareview automatically — This adds a new merge-gating body-scanning job with regex-based allowlists and per-job concurrency; a subtle false-positive or fail-open bug would block or silently unguard PRs across every repo this guard is vendored to.. I'll post findings when complete. |
PR Summary by QodoCI: scan PR/issue/comment bodies in public-repo-guard
AI Description
Diagram
High-Level Assessment
Files changed (3)
|
Code Review by Qodo
1.
|
Qodo FixerNo findings are within the configured fix scope. To change which findings are fixed, adjust the setting on your Qodo configuration page. |
…ue exit 2 Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
9239ff5 to
178fd84
Compare
…d base ref Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…event a base change fires Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…ckout credentials Three review-driven fixes: - check()'s allowlist filters were '|| true'd: a filter dying with exit >= 2 emptied the match list and reported a DETECTED hit as clean. Each filter now captures its status and exits 2, same as the main scan. - The test fixtures hardcoded real private repo names and credential names; the file is gate-exempt by path, so it published exactly what the gate blocks. All fixture names are now obviously synthetic (acme-*). - All three checkout steps now set persist-credentials: false, matching every other workflow here: these jobs execute scripts from the PR's own tree, and that code must not run beside a token in .git/config. Also documents why a body-guard failure on an issue/comment lands on main's head commit and why the job must stay non-required. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…five The self-test step runs tests/body-policy.test.sh, so an install that copies only the four listed files fails on a step nobody read. List the fixture as part of the install unit. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
… allowlist to prose rules Two review findings on the body gate: - The bootstrap fallback ran the PR's own body-policy.sh whenever the base ref lacked a copy, which is not only the install PR: any PR opened against (or retargeted onto) a branch predating the guard would have its own code judge its own body. The fallback is now the default branch's copy (which a PR cannot control), and if no trusted copy exists anywhere the job fails closed instead of executing untrusted code. - The about-the-control allowlist was applied to every rule, so a line mentioning the gate by name was exempt even when it also carried a live credential. It now applies only to prose rules (internal-marker, private-repo-ops); credential- and infrastructure-format rules stay strict. Fixtures pin both halves. Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…logy rule strict on gate-naming lines Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
| - name: body policy self-test (fixtures) | ||
| run: bash scripts/public-repo-guard/tests/body-policy.test.sh |
There was a problem hiding this comment.
🟨 Tree-scan job executes the pull request's own guard scripts as part of a required check
The required guard job checks out the PR's tree and runs scripts/public-repo-guard/tests/body-policy.test.sh (and, pre-existing, content-policy.sh) straight from that untrusted checkout, so a PR can rewrite the fixtures or the policy script it is being judged by and make the self-test pass unconditionally. The new body-guard job explicitly avoids exactly this (.github/workflows/public-repo-guard.yml:164-192 resolves the scanner from the trusted base/default ref), so the two halves of the same gate apply opposite trust models.
Was this helpful? React with 👍 or 👎 to provide feedback.
This repo's
public-repo-guardhas never scanned a single issue or comment body.Measured across all 28 public wave-av repos (
wave-av/claude-workstation#1747,#1794): two coverage shapes satisfy the one required check nameSecrets + content policy.pull_request, push, workflow_dispatchguardissues,issue_commentbody-guardThis repo is in the 27. All 28 report the same green check — because a required check asserts that something named X passed, never what X examined.
The outlier is
wave-moq-edge, and its own comment says why it matters:That gap was not theoretical there: a PR was blocked for naming a private repo in
wrangler.tomlwhile the very same name, with more operational detail attached, sat unchallenged in its body.What lands
Three files — the bundle the workflow's own header names, minus what this repo already has (
.gitleaks.tomlandcontent-policy.share already vendored, and are checked as prerequisites; a repo missing either is refused rather than half-installed):The workflow's header names four files as the install unit but executes a fifth —
tests/body-policy.test.sh, in its own self-test step. Omitting it installs a workflow that fails on a step nobody read, so the manifest ships it. Modes are preserved via the git trees API; the contents API creates100644regardless, which would silently break running these scripts as executables.Planned by
governance/lib/vendor-bundle.mjs(claude-workstation#1850) against a checked-in manifest, not by ad-hoc shell.One deliberate divergence from the reference, stated rather than silent
The shipped workflow is
wave-moq-edge's withactions/checkoutbumped from v5.0.1 to v7.0.1 (3d3c42e5aac5ba805825da76410c181273ba90b1), the pin already used byclaude-workstation's own gate.Copying verbatim was checked first and rejected on evidence: of the 18 target repos, 17 carry a byte-identical guard, and
wave-realtime-edgealready runs v7.0.0 — so a verbatim copy would have downgraded it, and shipped a stale pin to the other 17. A separate PR brings the reference itself up to the same pin.Honest about what this can and cannot do
On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication.
Also inherited from the reference: concurrency moves from workflow-level to per job, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green.
The body gate ships with its own fixtures and runs them in CI. Its negative cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing.
Refs
wave-av/claude-workstation#1747.Note
Medium Risk
Changes security CI behavior and merge blocking on PR bodies; issue/comment coverage is detection-only after publication. Low runtime risk—mostly vendored guard scripts and workflow logic, not application code.
Overview
Extends
public-repo-guardso world-readable text—not just the published tree—is checked. The workflow now runs onissues,issue_comment, andpull_requestedited, and adds abody-guardjob that materializes title/body from the event payload into a file (no shell interpolation) and runsbody-policy.sh.body-policy.shmirrors many tree rules (credentials, infra IDs, internal markers) but uses a proximity rule for private repos: bare cross-repo mentions stay allowed, while a private repo name within ~140 characters of operational detail (credential names, secret bindings, secret counts) blocks. An about-the-control allowlist andguard:allowkeep security discussion and documented examples from tripping the gate; violations redact matched text in CI output.The existing
guardjob is narrowed so it does not run on issue/comment events or PR body-onlyeditedevents; per-job concurrency replaces workflow-level grouping so rapid body edits do not cancel tree scans and leave stale failed checks.actions/checkoutis bumped to v7.0.1, andbody-policy.test.shfixtures run in CI on the tree job.Reviewed by Cursor Bugbot for commit c80ed0f. Configure here.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is enabled.Note
Scan PR, issue, and comment bodies for credential and infrastructure leaks in CI
body-guardjob to public-repo-guard.yml that triggers on PR, issue, and issue comment events; materializes the title/body from the GitHub event payload and scans it with the new body-policy.sh scanner.rg -P(PCRE2 required) to block credential formats (Stripe, Anthropic, GitHub PATs, AWS AKIDs, embedded private keys), internal-only markers, operator home paths, and private-repo names near operational detail patterns driven byGUARD_PRIVATE_REPOS.guardjob to validate scanner behavior before any real scan.GUARD_PRIVATE_REPOSmust be set in CI; an empty value exits 2 and fails the job.Macroscope summarized c82fc3d.
Review-driven hardening (post-review commits)
(?i)is now scoped to the private-repo name alternation ((?i:...)); a top-level flag leaked intoOPS_DETAILand made the SCREAMING_CASE credential rule match lowercase prose likecache_key. Two fixtures pin both halves (names stay case-insensitive, prose stays clean).body-guardnow executes the scanner from the trusted base ref (second sparse checkout atgithub.event.pull_request.base.sha), so a PR can no longer editbody-policy.shin the same push that leaks in its body and go green. When the base ref predates the guard (a long-lived branch, or a retargeted PR), the fallback is the default branch's copy, never the PR's own checkout; if no trusted copy exists anywhere the job fails closed rather than execute untrusted code.Both jobs probe for a PCRE2-capable ripgrep up front and fail with an error naming the problem instead of an opaque
exit 2.The about-the-control allowlist is scoped to prose rules only; a line naming the gate no longer exempts a credential-format hit on the same line.