Skip to content

ci: scan issue and comment bodies — this repo has never scanned one - #43

Open
yakimoto wants to merge 13 commits into
mainfrom
ci/1747-public-repo-guard-body-scan
Open

ci: scan issue and comment bodies — this repo has never scanned one#43
yakimoto wants to merge 13 commits into
mainfrom
ci/1747-public-repo-guard-body-scan

Conversation

@yakimoto

@yakimoto yakimoto commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

This repo's public-repo-guard has never scanned a single issue or comment body.

Measured across all 28 public wave-av repos (wave-av/claude-workstation#1747, #1794): two coverage shapes satisfy the one required check name Secrets + content policy.

repos triggers jobs
27 pull_request, push, workflow_dispatch guard
1 + issues, issue_comment + body-guard

This repo is in the 27. All 28 report the same green check — because a required check asserts that something named X passed, never what X examined.

The outlier is wave-moq-edge, and its own comment says why it matters:

edited matters as much as opened: a body can be made to leak long after the PR is first raised, and until this workflow covered it, nothing ever re-scanned.

That gap was not theoretical there: a PR was blocked for naming a private repo in wrangler.toml while the very same name, with more operational detail attached, sat unchallenged in its body.

What lands

Three files — the bundle the workflow's own header names, minus what this repo already has (.gitleaks.toml and content-policy.sh are already vendored, and are checked as prerequisites; a repo missing either is refused rather than half-installed):

.github/workflows/public-repo-guard.yml               replaced (72 -> 163 lines)
scripts/public-repo-guard/body-policy.sh              new, mode 100755
scripts/public-repo-guard/tests/body-policy.test.sh   new, mode 100755

The workflow's header names four files as the install unit but executes a fifthtests/body-policy.test.sh, in its own self-test step. Omitting it installs a workflow that fails on a step nobody read, so the manifest ships it. Modes are preserved via the git trees API; the contents API creates 100644 regardless, which would silently break running these scripts as executables.

Planned by governance/lib/vendor-bundle.mjs (claude-workstation#1850) against a checked-in manifest, not by ad-hoc shell.

One deliberate divergence from the reference, stated rather than silent

The shipped workflow is wave-moq-edge's with actions/checkout bumped from v5.0.1 to v7.0.1 (3d3c42e5aac5ba805825da76410c181273ba90b1), the pin already used by claude-workstation's own gate.

Copying verbatim was checked first and rejected on evidence: of the 18 target repos, 17 carry a byte-identical guard, and wave-realtime-edge already runs v7.0.0 — so a verbatim copy would have downgraded it, and shipped a stale pin to the other 17. A separate PR brings the reference itself up to the same pin.

Honest about what this can and cannot do

On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication.

Also inherited from the reference: concurrency moves from workflow-level to per job, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green.

The body gate ships with its own fixtures and runs them in CI. Its negative cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing.

Refs wave-av/claude-workstation#1747.


Note

Medium Risk
Changes merge-gate and branch-protection behavior (required checks, concurrency, bootstrap skip) and expand what content can block merges; logic is security-sensitive but heavily documented and covered by fixtures.

Overview
Extends public-repo-guard so PR, issue, and comment titles/bodies are scanned server-side, not only the published tree. A new body-guard job runs on pull_request, issues, and issue_comment (including edited), materializes event text to a temp file via jq, and runs body-policy.sh from a trusted base ref (sparse checkout) so fork PRs cannot substitute a no-op scanner.

The existing guard job still scans the tree with gitleaks and content-policy.sh, but now also installs a pinned, checksum-verified ripgrep with PCRE2, runs body-policy.test.sh in CI, uses per-job concurrency (tree cancels in progress; body does not), and still runs on edited PRs so a skipped required check cannot mask a failed tree scan.

body-policy.sh mirrors many tree rules (credentials, CF account_id, internal IPs, operator paths, internal markers) with redacted annotations, guard:allow, and an “about the control” allowlist—but private repo names only block when paired with operational detail within ~140 characters (unlike content-policy.sh, which blocks bare names in files). GUARD_PRIVATE_REPOS still drives the name list from org vars.

tests/body-policy.test.sh adds fixture positives/negatives (including bare cross-repo references) and fail-closed paths; actions/checkout is bumped to v7.0.1 in the workflow.

Reviewed by Cursor Bugbot for commit 75f4f7a. Configure here.


View with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is enabled.

Review in cubic

Note

Scan PR, issue, and comment bodies for credential and operational leaks in public-repo-guard

  • Adds body-policy.sh, a new Bash script that scans untrusted body text (PR titles/bodies, issue bodies, comment bodies) using PCRE2-enabled ripgrep for credentials (Stripe, Anthropic, GitHub PATs, AWS AKIDs, Supabase, private keys) and operational leaks (internal markers, private repo names near operational tokens, operator home paths).
  • Adds a new Body content policy job to public-repo-guard.yml triggered on PR edits, issue opens/edits, and comment creates/edits; the job materializes body text from the event payload and runs the policy script.
  • Matched content is redacted in GitHub Actions annotations; guard:allow inline markers exempt specific lines with a required reason.
  • Adds body-policy.test.sh with comprehensive fixtures covering blocks, passes, allowlist exemptions, and fail-closed behavior.
  • Behavioral Change: the workflow now re-runs on PR title/body edits and uses per-job concurrency instead of workflow-level concurrency, which changes how concurrent runs are queued and cancelled.

Macroscope summarized 75f4f7a.

…ment body

Measured across all 28 public wave-av repos (claude-workstation#1747, #1794):
TWO coverage shapes satisfy the one required check name `Secrets + content policy`.

  27 repos  triggers: pull_request, push, workflow_dispatch      jobs: guard
   1 repo   triggers: + issues, issue_comment                    jobs: + body-guard

This repo is in the 27. All 28 report the same green check.

The outlier is wave-moq-edge, and its own comment says why it matters:

  "`edited` matters as much as `opened`: a body can be made to leak long after the
   PR is first raised, and until this workflow covered it, nothing ever re-scanned."

A PR/issue/comment BODY is exactly as world-readable as the tree, and until now it
was scanned by nothing server-side. That gap was not theoretical on wave-moq-edge: a
PR was blocked for naming a private repo in wrangler.toml while the very same name,
with more operational detail attached, sat unchallenged in its body.

WHAT LANDS HERE — the bundle the workflow's own header names, minus what this repo
already has (.gitleaks.toml and content-policy.sh are already vendored):

  .github/workflows/public-repo-guard.yml          replaced (73 -> 163 lines)
  scripts/public-repo-guard/body-policy.sh         new, mode 100755
  scripts/public-repo-guard/tests/body-policy.test.sh  new, mode 100755

Copied from wave-moq-edge, which has run this shape in production. Modes preserved
via the git trees API — the contents API would have created both scripts 100644.

HONEST ABOUT WHAT IT CAN DO. On a PR this PREVENTS the merge. On an issue or comment
the text is already public the moment it posts, so this is DETECTION: it says go
redact, fast. Only a client-side pre-write hook stops that class before publication.

Also inherited from the reference: concurrency moves from workflow-level to PER JOB,
because the two jobs want opposite behaviour. A workflow-level group forced one
policy on both, and rapid body edits cancelled the tree job repeatedly — every
cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while
the live runs were green.

The body gate ships with its own fixtures and runs them in CI. Its NEGATIVE cases are
the load-bearing half: a leak gate that blocks legitimate cross-repo references gets
switched off, and then it protects nothing.

Refs wave-av/claude-workstation#1747.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@yakimoto yakimoto added the rr:skip-coderabbit RF.P1 reviewer routing (#1039) label Aug 6, 2026
@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_c47dab26-9abf-4f10-9a58-a27ba576d46b)

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Excluded labels (none allowed) (1)
  • rr:skip-coderabbit

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f9337aff-07eb-4ba8-9571-0945fbb89316

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@macroscopeapp

macroscopeapp Bot commented Aug 6, 2026

Copy link
Copy Markdown

Approvability

Verdict: Needs human review

Unable to check for correctness in 75f4f7a. This PR adds a new security scanning job that gates PR merges by scanning titles/bodies for sensitive content (~300 lines of new scripts). While the author owns all changed files, the introduction of new merge-blocking workflow logic and significant new scanning rules warrants human review to verify the approach and behavior.

You can customize Macroscope's approvability policy. Learn more.

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Running ultrareview automatically — This adds a new CI security gate scanning untrusted PR/issue/comment bodies with regex-based policy and fail-closed paths, plus workflow event/concurrency changes — a subtle regex or workflow edge case could silently skip scans, leak internal names, or break merge gating.. I'll post findings when complete.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

CI: scan PR/issue/comment bodies with public-repo-guard

✨ Enhancement ⚙️ Configuration changes 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description

• Extend public-repo-guard to run on issue/comment/PR body edits.
• Add a dedicated body-guard job that scans event payload text safely.
• Introduce body policy rules plus fixture tests to prevent false positives.
Diagram

graph TD
  GH{{"GitHub events"}} --> WF["public-repo-guard workflow"] --> G["Job: guard"] --> TREE["Tree scan (content-policy + gitleaks)"]
  G --> TESTS["Body policy fixtures"]
  WF --> BG["Job: body-guard"] --> PAYLOAD[("GITHUB_EVENT_PATH") ] --> BODYS["Body scan (body-policy)"]

  subgraph Legend
    direction LR
    _ext{{"External"}} ~~~ _job["Job"] ~~~ _file["Script/file"] ~~~ _data[("Data")]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Separate workflow dedicated to issues/comments
  • ➕ Simpler per-workflow triggers/concurrency rules
  • ➕ Body scanning failures isolated from tree scanning checks
  • ➖ More files to vendor and keep consistent across repos
  • ➖ More required-check coordination (multiple check names)
2. Use GitHub-native scanning (secret scanning / code scanning)
  • ➕ Less custom policy code to maintain
  • ➕ Potentially better credential detectors
  • ➖ Does not cover the repo-specific "private repo + operational detail" policy shape
  • ➖ Body scanning and policy nuance (allowlists, redaction) may not be expressible

Recommendation: Keep the PR’s approach: a dedicated body-guard job inside the existing workflow, with explicit event types, per-job concurrency, and fail-closed payload-shape validation. It minimizes cost (no tree checkout per comment), avoids pull_request_target security pitfalls, and the included fixture tests address the main operational risk: false positives that cause the guard to be disabled.

Files changed (3) +342 / -5

Enhancement (1) +139 / -0
body-policy.shIntroduce ripgrep-based policy scanner for PR/issue/comment text +139/-0

Introduce ripgrep-based policy scanner for PR/issue/comment text

• Adds a body policy script that scans materialized, untrusted title/body text using ripgrep PCRE rules and explicit allowlisting. Enforces fail-closed behavior on scanner errors, redacts matched content in CI annotations, and supports opt-out via 'guard:allow <reason>'. Implements both credential-format blockers and a proximity-based rule for private repo names plus operational detail, driven by the injected GUARD_PRIVATE_REPOS list.

scripts/public-repo-guard/body-policy.sh

Tests (1) +108 / -0
body-policy.test.shAdd fixture tests for body-policy precision and redaction behavior +108/-0

Add fixture tests for body-policy precision and redaction behavior

• Adds hermetic fixture tests that validate both blocking and non-blocking cases, emphasizing deployability (avoiding false positives on legitimate cross-references). Verifies that blocking output does not echo the matched body text and that the script fails closed (exit 2) when invoked incorrectly. Includes a constructed AWS AKID fixture to avoid accidentally committing credential-shaped literals.

scripts/public-repo-guard/tests/body-policy.test.sh

Other (1) +95 / -5
public-repo-guard.ymlAdd body-guard job and expand triggers to scan body edits +95/-5

Add body-guard job and expand triggers to scan body edits

• Extends workflow triggers to include issues and issue_comment events and to re-run on edited bodies. Adds a new 'Body content policy' job that extracts title/body from the event payload into a file and scans it, while keeping the existing tree scan job scoped to relevant events. Updates concurrency to be per-job (tree cancels in-progress; body does not) and bumps actions/checkout to v7.0.1; also adds a fixture self-test step for the body policy.

.github/workflows/public-repo-guard.yml

@qodo-code-review

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

To customize comments, go to the Qodo configuration screen, or learn more in the docs.

Qodo Logo

@qodo-code-review

Copy link
Copy Markdown

Qodo Fixer

No findings are available for this PR yet. Findings appear here once Qodo has reviewed the PR.

devin-ai-integration[bot]

This comment was marked as resolved.

yakimoto and others added 2 commits August 6, 2026 17:30
…aundering

Address review findings:
- body-guard now checks out the gate scripts from the base branch tip
  (pull_request.base.sha), not the PR merge ref, so a fork PR can no longer
  rewrite body-policy.sh to pass its own body.
- The ABOUT_THE_CONTROL allowlist no longer applies to credential-format
  rules: a line naming the gate can still carry a live key. Only the explicit
  guard:allow marker exempts those. Fixture added.
- Preflight the ripgrep build for PCRE2 (script + both install steps) so a
  non-PCRE2 rg fails with the real cause instead of an opaque scanner error.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…dentifiers

Address review findings:
- The private-repo-ops pattern opens with a global (?i) so repo names match
  case-insensitively, but that flag reached OPS_DETAIL and defeated its
  SCREAMING_CASE requirement: lowercase identifiers like api_key next to a
  repo name blocked clean PRs. The credential-name branch is now (?-i:...).
- Infra identifiers (Cloudflare account_id, Tailscale IP, operator path) now
  ignore the ABOUT_THE_CONTROL allowlist like the credential formats do: a
  real internal IP on a line naming the gate is still a leak.
- Fixtures for both.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
cubic-dev-ai[bot]

This comment was marked as resolved.

…s, not 4)

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

yakimoto and others added 2 commits August 6, 2026 17:40
The trusted-ref checkout means the installing PR's base has no
body-policy.sh yet. That is exactly the pre-install state (no gate), and a
PR head cannot cause it, so skip with a visible warning instead of exiting
127; the gate is live from the first event after merge.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
…s off

- issue_comment runs now scan just the new comment body. Re-scanning the
  parent issue's title/body meant one old violation failed every future
  comment, with line numbers into a concatenated buffer that maps to
  nothing. Each text is judged exactly once, on the event that changes it.
- An empty GUARD_PRIVATE_REPOS (fork runs may not receive org variables)
  now emits a visible warning instead of a quietly green check with the
  highest-value rule off.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

…banners

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

…ures

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_f411b32c-4210-4040-8d71-4f8e64ccb48b)

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 3 new potential issues.

Open in Devin Review

Comment thread .github/workflows/public-repo-guard.yml
Comment on lines +90 to +92
env:
RIPGREP_VERSION: "14.1.1"
RIPGREP_SHA256: "4cf9f2741e6c465ffdb7c26f38056a59e2a2544b51f7cc128ef28337eeae4d8e"

@devin-ai-integration devin-ai-integration Bot Aug 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Pinned ripgrep checksum is unverifiable from the repo

The fallback install pins ripgrep 14.1.1 with SHA-256 4cf9f2...; unlike the gitleaks pin this value appears nowhere else in the repo, so a typo would only surface as a hard failure of both jobs at the moment the preinstalled runner rg loses PCRE2 — i.e. precisely the fallback path this step exists to keep working, and one that never executes on today's runner images. Worth verifying the digest against the upstream release once, since the failure mode is silent until it is urgent.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment thread .github/workflows/public-repo-guard.yml
… the required check

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
… the body gate

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_9a3fb931-0615-4088-a68b-9c131ab05f3e)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rr:skip-coderabbit RF.P1 reviewer routing (#1039)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant