Skip to content

ci: scan issue and comment bodies — this repo has never scanned one - #56

Open
yakimoto wants to merge 6 commits into
mainfrom
ci/1747-public-repo-guard-body-scan
Open

ci: scan issue and comment bodies — this repo has never scanned one#56
yakimoto wants to merge 6 commits into
mainfrom
ci/1747-public-repo-guard-body-scan

Conversation

@yakimoto

@yakimoto yakimoto commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

This repo's public-repo-guard has never scanned a single issue or comment body.

Measured across all 28 public wave-av repos (wave-av/claude-workstation#1747, #1794): two coverage shapes satisfy the one required check name Secrets + content policy.

repos triggers jobs
27 pull_request, push, workflow_dispatch guard
1 + issues, issue_comment + body-guard

This repo is in the 27. All 28 report the same green check — because a required check asserts that something named X passed, never what X examined.

The outlier is wave-moq-edge, and its own comment says why it matters:

edited matters as much as opened: a body can be made to leak long after the PR is first raised, and until this workflow covered it, nothing ever re-scanned.

That gap was not theoretical there: a PR was blocked for naming a private repo in wrangler.toml while the very same name, with more operational detail attached, sat unchallenged in its body.

What lands

Three files — the bundle the workflow's own header names, minus what this repo already has (.gitleaks.toml and content-policy.sh are already vendored, and are checked as prerequisites; a repo missing either is refused rather than half-installed):

.github/workflows/public-repo-guard.yml               replaced (72 -> 163 lines)
scripts/public-repo-guard/body-policy.sh              new, mode 100755
scripts/public-repo-guard/tests/body-policy.test.sh   new, mode 100755

The workflow's header names four files as the install unit but executes a fifthtests/body-policy.test.sh, in its own self-test step. Omitting it installs a workflow that fails on a step nobody read, so the manifest ships it. Modes are preserved via the git trees API; the contents API creates 100644 regardless, which would silently break running these scripts as executables.

Planned by governance/lib/vendor-bundle.mjs (claude-workstation#1850) against a checked-in manifest, not by ad-hoc shell.

One deliberate divergence from the reference, stated rather than silent

The shipped workflow is wave-moq-edge's with actions/checkout bumped from v5.0.1 to v7.0.1 (3d3c42e5aac5ba805825da76410c181273ba90b1), the pin already used by claude-workstation's own gate.

Copying verbatim was checked first and rejected on evidence: of the 18 target repos, 17 carry a byte-identical guard, and wave-realtime-edge already runs v7.0.0 — so a verbatim copy would have downgraded it, and shipped a stale pin to the other 17. A separate PR brings the reference itself up to the same pin.

Honest about what this can and cannot do

On a PR this PREVENTS the merge. On an issue or comment the text is already public the moment it posts, so this is DETECTION: it says go redact, fast. Only a client-side pre-write hook stops that class before publication.

Also inherited from the reference: concurrency moves from workflow-level to per job, because the two jobs want opposite behaviour. A workflow-level group forced one policy on both, and rapid body edits cancelled the tree job repeatedly — every cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while the live runs were green.

The body gate ships with its own fixtures and runs them in CI. Its negative cases are the load-bearing half: a leak gate that blocks legitimate cross-repo references gets switched off, and then it protects nothing.

Refs wave-av/claude-workstation#1747.


Note

Cursor Bugbot is generating a summary for commit 5b4efc0. Configure here.


View with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is enabled.

Note

Scan PR/issue/comment/review bodies for secrets and internal markers in CI

  • Adds a body-guard job to public-repo-guard.yml that runs on PRs, issues, comments, and reviews; it materializes untrusted body text from $GITHUB_EVENT_PATH and scans it with the new body-policy.sh.
  • body-policy.sh checks for credential formats (Stripe, Anthropic, GitHub PATs, AWS keys, PEM blocks), infrastructure identifiers, internal markers, operator home paths, private-repo operational details, and internal Tailscale IPs using PCRE2 ripgrep; BLOCK violations cause a nonzero exit.
  • Allowlisting is supported via guard:allow <reason> inline comments; self-referential prose about the control is filtered for select rules to avoid false positives.
  • A fixture test suite in body-policy.test.sh covers positive, negative, precision, and fail-closed cases; the guard job now runs these tests as a self-test step.
  • The guard job skips purely body-edit and issue/comment events; both jobs install a version-pinned, SHA-256-verified PCRE2-capable ripgrep binary instead of the apt package.

Macroscope summarized 3716e82.

Review in cubic

…ment body

Measured across all 28 public wave-av repos (claude-workstation#1747, #1794):
TWO coverage shapes satisfy the one required check name `Secrets + content policy`.

  27 repos  triggers: pull_request, push, workflow_dispatch      jobs: guard
   1 repo   triggers: + issues, issue_comment                    jobs: + body-guard

This repo is in the 27. All 28 report the same green check.

The outlier is wave-moq-edge, and its own comment says why it matters:

  "`edited` matters as much as `opened`: a body can be made to leak long after the
   PR is first raised, and until this workflow covered it, nothing ever re-scanned."

A PR/issue/comment BODY is exactly as world-readable as the tree, and until now it
was scanned by nothing server-side. That gap was not theoretical on wave-moq-edge: a
PR was blocked for naming a private repo in wrangler.toml while the very same name,
with more operational detail attached, sat unchallenged in its body.

WHAT LANDS HERE — the bundle the workflow's own header names, minus what this repo
already has (.gitleaks.toml and content-policy.sh are already vendored):

  .github/workflows/public-repo-guard.yml          replaced (73 -> 163 lines)
  scripts/public-repo-guard/body-policy.sh         new, mode 100755
  scripts/public-repo-guard/tests/body-policy.test.sh  new, mode 100755

Copied from wave-moq-edge, which has run this shape in production. Modes preserved
via the git trees API — the contents API would have created both scripts 100644.

HONEST ABOUT WHAT IT CAN DO. On a PR this PREVENTS the merge. On an issue or comment
the text is already public the moment it posts, so this is DETECTION: it says go
redact, fast. Only a client-side pre-write hook stops that class before publication.

Also inherited from the reference: concurrency moves from workflow-level to PER JOB,
because the two jobs want opposite behaviour. A workflow-level group forced one
policy on both, and rapid body edits cancelled the tree job repeatedly — every
cancelled check-run stays attached to the commit, so the PR reported UNSTABLE while
the live runs were green.

The body gate ships with its own fixtures and runs them in CI. Its NEGATIVE cases are
the load-bearing half: a leak gate that blocks legitimate cross-repo references gets
switched off, and then it protects nothing.

Refs wave-av/claude-workstation#1747.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@yakimoto yakimoto added the rr:skip-coderabbit RF.P1 reviewer routing (#1039) label Aug 6, 2026
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Excluded labels (none allowed) (1)
  • rr:skip-coderabbit

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 85e8e1c3-de5c-4709-9b9d-7b069a0f2ee0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_11ee7f73-d241-42ec-9bb8-7afe2dd756e1)

@cubic-dev-ai

cubic-dev-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Running ultrareview automatically — This rewires the org's security gate: new body-scanning job, event triggers, concurrency policy, and a fail-closed regex scanner with its own allowlist — a bug here could silently miss leaks or break the required check across repos, so it warrants a deeper pass.. I'll post findings when complete.

@macroscopeapp

macroscopeapp Bot commented Aug 6, 2026

Copy link
Copy Markdown

Approvability

Verdict: Needs human review

Unable to check for correctness in 3716e82. This PR introduces a new body-scanning security gate with significant scope. The author does not own the modified files (owned by wave-av/sdk-owners), and there's an unresolved comment identifying a potential bug where some review comments could silently go unscanned due to concurrency grouping.

You can customize Macroscope's approvability policy. Learn more.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Add issue/comment body scanning to public-repo-guard CI

✨ Enhancement ⚙️ Configuration changes 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Add issues and issue_comment triggers to scan public PR/issue/comment text.
• Split workflow into tree scan and body scan jobs with per-job concurrency policies.
• Add a tested body-policy.sh gate that fails closed and redacts matches in logs.
Diagram

graph TD
  A{{"GitHub event"}} --> B["public-repo-guard.yml"] --> C["guard job (tree)"] --> D["content-policy.sh"]
  B --> E["body-guard job"] --> F["GITHUB_EVENT_PATH"] --> G["body-policy.sh"]
  G --> H[("GUARD_PRIVATE_REPOS var")]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Use a reusable workflow from a central repo
  • ➕ Single source of truth; faster updates across many repos
  • ➕ Less per-repo drift and less vendored script duplication
  • ➖ Requires trusting the referenced workflow repo/versioning model
  • ➖ Can be harder to audit when reviewers expect everything in-repo
2. Rely on GitHub-native secret scanning / GHAS
  • ➕ First-party tooling; fewer custom regexes and bash edge cases
  • ➕ Potentially better detection for common credential formats
  • ➖ May require paid GitHub Advanced Security features for orgs
  • ➖ Doesn’t cover the bespoke “private repo + ops detail proximity” policy
3. Scan bodies via an external webhook/app (server-side)
  • ➕ Can act immediately on issue/comment creation across repos
  • ➕ Can centralize policy enforcement and reporting
  • ➖ Operational overhead: hosting, auth, retries, rate limits
  • ➖ More moving parts than a self-contained workflow job

Recommendation: The PR’s approach (add a dedicated body-guard job that reads from GITHUB_EVENT_PATH, fails closed on unknown payloads, and avoids pull_request_target) is the best fit for a public-repo security gate: it minimizes privilege while closing the “unscanned body text” gap. A reusable workflow is the main viable alternative if the org wants centralized maintenance, but it trades off local auditability and requires stricter version governance.

Files changed (3) +342 / -5

Enhancement (1) +139 / -0
body-policy.shImplement body content policy scanner for PR/issue/comment text +139/-0

Implement body content policy scanner for PR/issue/comment text

• Adds a bash-based scanner using ripgrep PCRE rules to detect credential formats, internal identifiers, and internal-only markers, with explicit allowlisting and redacted reporting. Implements a proximity-based rule for private repo names near operational detail driven by 'GUARD_PRIVATE_REPOS', and fails closed on scanner errors or invalid invocation. Ensures matched content is not echoed back into world-readable CI annotations.

scripts/public-repo-guard/body-policy.sh

Tests (1) +108 / -0
body-policy.test.shAdd fixture tests for body-policy allow/block behavior +108/-0

Add fixture tests for body-policy allow/block behavior

• Introduces hermetic fixture tests covering blocking cases, deployability-focused non-blocking negatives, and fail-closed behavior. Includes a regression check ensuring annotations never print the matched (potentially sensitive) text, and avoids embedding credential-shaped literals directly in source.

scripts/public-repo-guard/tests/body-policy.test.sh

Other (1) +95 / -5
public-repo-guard.ymlAdd body scanning job and broaden event coverage +95/-5

Add body scanning job and broaden event coverage

• Extends triggers to include PR opened/edited flows plus issues and issue comments, and introduces a new 'body-guard' job to scan title/body text extracted from the event payload. Refactors concurrency to be per-job (tree scans cancel-in-progress; body scans do not) and bumps 'actions/checkout' to a newer pinned commit. Adds a workflow self-test step that runs the new fixture tests for the body policy script.

.github/workflows/public-repo-guard.yml

devin-ai-integration[bot]

This comment was marked as resolved.

@qodo-code-review

qodo-code-review Bot commented Aug 6, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Fail-open on filter errors ✓ Resolved 🐞 Bug ⛨ Security
Description
In scripts/public-repo-guard/body-policy.sh, the allowlist filtering pipeline ends with || true,
so ripgrep errors during filtering (exit >=2) are treated the same as “no matches” and the rule can
incorrectly report a pass. This contradicts the script’s stated fail-closed behavior and can let a
violating PR/issue/comment body slip through undetected if filtering fails.
Code

scripts/public-repo-guard/body-policy.sh[R53-56]

+  matches="$(printf '%s' "$raw" \
+    | rg -vN -- 'guard:allow[[:space:]]+[^[:space:]]' \
+    | rg -vNiP -- "$ABOUT_THE_CONTROL" || true)"
+  [[ -z "$matches" ]] && return 0
Evidence
The script explicitly documents that ripgrep errors must fail closed, but the matches assignment
masks all failures in the filtering pipeline with || true, which can hide ripgrep errors and make
matches empty, returning success.

scripts/public-repo-guard/body-policy.sh[41-56]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`body-policy.sh` intends to fail closed on scanner errors, but the post-processing (allowlist) filtering pipeline is wrapped in `|| true`, which suppresses ripgrep failures (exit >=2). This can incorrectly convert a real filtering/scanner error into an empty `matches` result, returning success.

## Issue Context
The `check()` function correctly fails closed on errors from the primary `rg -nP` scan. However, it does not distinguish between ripgrep exit code 1 (no matches after filtering) and exit code >=2 (error) during the filtering phase.

## Fix Focus Areas
- scripts/public-repo-guard/body-policy.sh[37-56]

## Expected fix approach
- Keep allowing “no matches” semantics (exit 1) from the filter rgs.
- Explicitly detect and fail closed on filter errors (exit >=2), e.g.:
 - Run each filter stage separately and check `$?`, or
 - Temporarily disable `pipefail` / capture `PIPESTATUS` and treat only rc==0/1 as non-fatal.
- Avoid re-introducing printing of matched text (keep redaction behavior unchanged).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context used
Review mode: ⚖️ Balanced: Downgraded extended -> standard: change is below the extended eligibility bar (hunks 5/18, lines 347/200; both must reach the floor). Router rationale: This adds a security-sensitive GitHub Actions body-scanning path plus substantial independent shell-policy and fixture logic, creating multiple plausible, easy-to-miss defects across workflow events, untrusted payload handling, regex policy, and fail-closed behavior.

To customize comments, go to the Qodo configuration screen, or learn more in the docs.

Qodo Logo

qodo-code-review[bot]

This comment was marked as resolved.

@qodo-code-review

Copy link
Copy Markdown

Qodo Fixer

🍒 Ready to be cherry-picked — ✅ Merged (0) · ☑ Fixed (1)

Grey Divider

🔗 Fix PR: #57

This fix PR was closed automatically. Its branch is preserved so you can cherry pick the changes into the original PR.

Prompt for coding agent

This is an automated fix prepared on a separate branch (#57). It is NOT applied to this PR.
To use it: review Fix PR #57 (https://github.com/wave-av/sdks/pull/57), evaluate each change critically against your local context, and cherry-pick the changes that are correct into this branch. Do not accept them blindly.
Process — 1 fixed
  • ☑ Fixed: Fail-open on filter errors

yakimoto and others added 2 commits August 6, 2026 17:31
…scoping

Review follow-ups:
- body-guard now checks out the base repo's default branch, so a fork PR can
  no longer ship a neutered body-policy.sh that passes its own body; a loud,
  documented bootstrap skip covers the installing PR itself
- the about-the-control allowlist is scoped to the prose rules
  (internal-marker, private-repo-ops); credential/infra rules are never
  exempted by merely mentioning the gate or SECURITY.md
- (?i) in private-repo-ops is scoped to the repo names only, so lowercase
  prose (auth_token, api_key) near a private repo name no longer blocks
- ripgrep is verified to have PCRE2 and, when missing, installed from the
  pinned checksum-verified release binary instead of apt; body-policy.sh
  fails closed with a clear message on a PCRE2-less build

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
The allowlist filter stages ended in '|| true', so an rg error (>=2) during
filtering was indistinguishable from 'every line allowlisted' and produced a
pass. Capture each stage's exit code: 0/1 proceed, >=2 exits 2, matching the
primary scan's fail-closed contract. Redaction behavior unchanged.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
cubic-dev-ai[bot]

This comment was marked as resolved.

Reviews were the remaining public text channel the body gate did not see:
pull_request_review carries the review summary in .review.body and
pull_request_review_comment carries line comments in .comment.body, and
neither arrives as issue_comment. Subscribe to both, admit them into
body-guard, and extract .review.body in the materialize step.

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 new potential issues.

Open in Devin Review

Comment thread scripts/public-repo-guard/tests/body-policy.test.sh Outdated
Comment thread .github/workflows/public-repo-guard.yml Outdated
Comment on lines +65 to +68
if: >-
(github.event_name == 'pull_request' && github.event.action != 'edited')
|| github.event_name == 'push'
|| github.event_name == 'workflow_dispatch'

@devin-ai-integration devin-ai-integration Bot Aug 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Tree scan is skipped on edited, which changes what the required check reports

With edited added to the pull_request types, every body/title edit now starts a workflow run in which the guard job is skipped, producing a fresh check-run with conclusion skipped on the head commit. GitHub treats skipped as passing for required checks, so this should not block merges, but it does mean the visible status for "Secrets + content policy" on a PR can end up being a skip rather than the earlier green tree scan. Worth confirming against the repo's branch-protection configuration if that check is required.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub treats a skipped check-run as passing for required checks, so the intentional edited skip on the tree job cannot block merges; the behavior is by design and documented in the workflow, and branch-protection configuration is not verifiable or changeable from this repo's code.

Comment thread scripts/public-repo-guard/tests/body-policy.test.sh Outdated
… with synthetic placeholders

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
devin-ai-integration[bot]

This comment was marked as resolved.

…to a path start

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

# already-reviewed copy of the scanner. A PR that changes body-policy.sh
# still has its copy exercised — by the self-test step in `guard`, against
# fixtures on the tree, where the required check reviews it.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 actions/checkout bumped to an unverified v7.0.1 pin, out of step with the rest of the repo

This PR changes the checkout pin from 93cb6efe… # v5.0.1 to 3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 in both jobs, while every other workflow in this repo pins actions/checkout@v4 (see .github/workflows/_checks.yml:23 and the test-*.yml files pinning @v4). The bump is unrelated to the stated purpose of the PR (body scanning) and cannot be verified offline. If the SHA/tag comment do not correspond to a real published release, both jobs fail at the very first step and the whole gate goes red. Worth confirming the SHA against the upstream release, and confirming the version actually supports the sparse-checkout / sparse-checkout-cone-mode inputs used at .github/workflows/public-repo-guard.yml:181-182.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rr:skip-coderabbit RF.P1 reviewer routing (#1039)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant