Repository navigation
Description
Problem
The dogfood CI job "Self-scan (CLI from source)" (dogfood.yml) produces different exit codes on PRs that contain only package.json and pnpm-lock.yaml changes with no modifications to scanner logic, rules, or source files.
Observed pattern (all from June 15, 2026 Dependabot wave):
| PR | Change | Self-scan result |
|---|---|---|
| #89 | turbo bump (root) | ✅ Pass |
| #91 | tailwind bump (root) | ✅ Pass |
| #94 | next bump (root) | ✅ Pass |
| #96 | @types/node bump (root) | ✅ Pass |
| #88 | semantic-release bump (root) | ❌ Fail |
| #90 | next bump (apps/web) | ❌ Fail |
| #92 | tailwind bump (apps/web) | ❌ Fail |
| #93 | @types/node bump (apps/web) | ❌ Fail |
| #95 | @types/node bump (packages/cli) | ❌ Fail |
No rule files, no source files, no action.yml changes. The only differences are package.json and pnpm-lock.yaml contents.
Hypotheses
A (most likely): The false-positive patterns tracked in #78, #79, and #80 are present in the repo source, and the scan result depends on whether Semgrep or Trivy fetches an updated rule DB at runtime. If the DB updates between runs, different PRs trip the FP and others don't.
B: The pnpm-lock.yaml content is being scanned by a native rule. Version bumps in the lock file alter the matched content (e.g., package URLs, integrity hashes) and trigger or suppress a finding.
C: Something in the semantic-release, next, tailwind, or @types/node package installation changes what files are present during the build step, and those files get scanned.
Scope
This is an investigation issue — no code changes in this issue itself.
- Download and compare workflow run log artifacts from one passing PR (e.g. chore(deps)(deps-dev): bump turbo from 2.9.16 to 2.9.18 in the lint-and-format group #89) and one failing PR (e.g. chore(deps)(deps-dev): bump semantic-release from 25.0.3 to 25.0.5 in the semantic-release group across 1 directory #88) — look for the specific rule ID, file path, and matched string that caused the exit code 1 on the failing run.
- Run two local scans on the same SHA (the current main HEAD) back-to-back and confirm whether they produce identical results. If not, the issue is a non-deterministic scanner DB fetch.
- Document the root cause.
- File or link a follow-up issue with the fix.
Acceptance Criteria
- Root cause is documented in a comment on this issue: specific rule ID, file path, and matched string (or confirmation that it's a timing/DB issue)
- Two local scan runs on the same SHA produce the same exit code (determinism confirmed or falsified)
- A follow-up issue is linked that will fix the identified cause, OR one of the existing issues (feat(rules): avoid SQL injection false positives on parameterized tagged template literals #78, feat(rules): reduce A02-HARDCODED-SECRET false positives on type definitions and UI strings #79, feat(rules): reduce A09-SENSITIVE-LOG false positives on configuration/status messages #80) is confirmed as the root cause and this issue is closed with a note
Non-goals
- Fixing the root cause in this issue
- Changing CI workflow structure
QA Checklist
- Identified specific finding(s) that cause exit code 1 on failing PRs
- Confirmed or falsified: two runs on same SHA → same exit code
- Root cause documented in issue comment
- Follow-up issue filed or existing issue linked
Risk
Low — investigation only
Labels
area:ci, area:scanners, reliability:determinism, investigation, risk:low
Activity