Skip to content

chore(security): automated monthly SBOM & VEX report - #199

Open
Gustavohsdp wants to merge 8 commits into
developmentfrom
chore/security-monthly
Open

chore(security): automated monthly SBOM & VEX report#199
Gustavohsdp wants to merge 8 commits into
developmentfrom
chore/security-monthly

Conversation

@Gustavohsdp

@Gustavohsdp Gustavohsdp commented Jul 27, 2026

Copy link
Copy Markdown

What this PR adds

Installs the automated software supply-chain security pipeline for STruCpp (JS / npm; TypeScript → C++ compiler distributed as a self-contained binary). Three flows, all driven by the same data:

  • Monthly SBOM & VEX report.github/workflows/security-monthly.yml. On the 1st of each month: regenerates the SBOM (CycloneDX 1.6 + SPDX 2.3), scans against OSV honoring the osv-scanner.toml VEX baseline, renders a report from security/report-config.json, opens a PR with the dated snapshot under security/<YYYY-MM>/. Runs against the default branch (development).
  • PR security gate.github/workflows/security-pr-gate.yml. On every PR: compares base vs head and fails only when the PR introduces a NEW HIGH/CRITICAL advisory (pre-existing never blocks). Posts a sticky comment with package, version, advisory link, fixed version, and how to resolve.
  • Archive on merge.github/workflows/security-pr-archive.yml. On merge, commits the SBOM + report under security/pr-<number>-<date>/.

Supporting files: scripts/, osv-scanner.toml (VEX baseline), security/report-config.json, security/README.md.

Security posture (assessment)

764 components, but only 1 direct runtime dependency (chevrotain). Of 29 raw advisories: 0 require remediation, 0 mitigated, 29 not applicable (all in build/test tooling not shipped, or in lodash-es whose vulnerable functions are never called). Both critical advisories are dev tooling.

⚠️ Licensing note (material for IP/legal)

STruCpp itself is GPL-3.0-or-later, with a GCC-style Runtime Library Exception (so C++ produced by the compiler is not forced to be GPL). Third-party dependencies are all permissive. This is the product's intended license — flagged for acquisition due diligence.

What to review

  • osv-scanner.toml + security/report-config.json — the VEX triage + the licensing note. Best reviewed by product security (Thiago).
  • .github/workflows/security-*.yml — the three flows.

Notes

  • Not enforced yet: the gate reports pass/fail but does not block merges until set as a required status check.
  • Claude triage authenticates via the org secret CLAUDE_CODE_OAUTH_TOKEN.

Ports the validated supply-chain automation to strucpp (public repo — no AI
triage; the Claude step is skipped gracefully when no token is present, and the
deterministic SBOM + report + PR still run). Regenerates the SBOM (CycloneDX+
SPDX), scans OSV honoring the VEX baseline, renders the report, opens a monthly
PR into security/<YYYY-MM>/. 0 advisories require action; baseline suppresses the
54 known not-affected IDs (build/test tooling + lodash-es via chevrotain).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
…erge

Gate now only scans/reports (contents: read) and posts one actionable sticky PR
comment; per-PR SBOM archived on MERGE by security-pr-archive.yml. Validated on
autonomy-edge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
@Gustavohsdp
Gustavohsdp force-pushed the chore/security-monthly branch from 39e5c29 to ce2204f Compare July 31, 2026 11:38
Gate fails closed on scanner error; MEDIUM no longer blocks as HIGH; CVSS 4.0
handled conservatively; diff keyed by advisory id + package. Report HTML escaped
(+ PDF rendered with JS disabled). cdx-to-spdx dedupes packages and uses OR for
dual licenses. gen-osv-ignores requires a verdict column. Validated on autonomy-edge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
devin-ai-integration Bot added a commit to wattzor/STruCpp that referenced this pull request Aug 4, 2026
Gustavohsdp and others added 3 commits August 4, 2026 10:16
Scan fail-closed; osv v2.4.0 (+SHA256) & cdxgen 11.11.0 pinned; node:test in the
gate; cdx-to-csv.mjs; OSV text; explicit base. Public repo: removed Claude/OAuth
triage step + token guidance.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
…token fallback (public: no AI)

build-report derives headline metrics from the live scan; monthly runs raw+delta;
archive -> push-on-default + orphan security-archive (SBOM & raw scan only);
monthly PR token fallback. Public repo: Claude triage kept out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
@Gustavohsdp

Copy link
Copy Markdown
Author

Review feedback addressed ✅

Thanks for the thorough review. Every point raised has been addressed across follow-up commits — the gate check is green and the branch is up to date.

Correctness / security bugs

  • PR gate now fails closed — a scanner error (missing/invalid JSON, or an exit code other than 0/1) hard-fails the step instead of silently passing (“no new vulnerabilities”) or blocking every advisory.
  • SeverityMEDIUM is normalized to MODERATE (no longer blocks as HIGH); CVSS 4.0 is treated conservatively and flagged; the CVSS 3.1 base-score formula cites the spec (§7.1).
  • Diff keyed by advisory id + package — the same CVE on a new package now counts as introduced; a version bump of an already-vulnerable package does not.
  • Report HTML injection — config strings are escaped (only attribute-less <b>/<code>/… re-enabled) and the PDF renders with --disable-javascript.
  • SPDX — components are de-duplicated by ref (no more two packages sharing one SPDXID); multiple licenses join with OR (dual-licensed), not AND.
  • gen-osv-ignores requires a verdict column and refuses to run without one (can no longer suppress the entire scan).
  • Monthly scan / scan-vulns fail closed too — a broken scan can no longer produce a clean report.

Supply-chain / hardening

  • osv-scanner pinned to v2.4.0 and verified against its SHA256SUMS; cdxgen / cyclonedx-py pinned to exact versions.
  • Inline node -e CSV builder extracted to scripts/cdx-to-csv.mjs.
  • Unit tests (node:test) for the gate diff + SPDX, executed inside the gate.
  • Report methodology corrected to OSV / osv-scanner (was “npm/pnpm/pip audit”).

Report & archive integrity

  • The report’s headline numbers are now derived from the live scan (raw vs. VEX-suppressed vs. surfacing), so it cannot attest stale figures.
  • The per-PR archive moved to a push-on-default-branch trigger (writable token — works for merged fork PRs) that stores SBOM + raw scan only on a dedicated orphan security-archive branch — no repo/history bloat and no false attestation.
  • The monthly PR uses secrets.SECURITY_BOT_TOKEN (with GITHUB_TOKEN fallback) so its own checks can run.

Since this is a public repository, the optional Claude/OAuth triage step and its token guidance were removed — the monthly report is deterministic + human-reviewed.

Two notes

  1. The monthly and archive workflows run on cron / push-to-default, so they execute for the first time after merge — the deterministic pieces are validated locally and the gate is green, but they don’t run on this PR itself.
  2. To make the monthly PR trigger checks, a one-time SECURITY_BOT_TOKEN secret (a GitHub App installation token or a fine-grained PAT with contents + PR write) is needed; without it the PR still opens, just without checks.

Ready for another look 🙏

@marconetsf
marconetsf self-requested a review August 4, 2026 21:35

@marconetsf marconetsf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together — the overall design is sound and several parts are genuinely hard to get right on the first pass. The fail-closed posture is real and consistent (scanner exit-code validation for 0/1 vs. anything else, load() exiting 2 on a shapeless results[], scan-vulns.mjs refusing to write an empty register). Keying the gate on id|package rather than id alone is the correct call and it's covered by a test. contents: read on the gate job is the right choice for something meant to become a required check, and the CycloneDX→SPDX conversion handles bom-ref dedupe and OR for dual-licensed components correctly.

I ran the suite locally (node --test scripts/__tests__/*.test.mjs → 7/7 pass) and exercised build-report.mjs against synthetic scans. I'm requesting changes on three items below; the rest can land as follow-ups.


Blocking

1. The monthly report can auto-publish a false attestation

This is the main concern. With the AI triage step removed, nothing updates security/report-config.json anymore, but build-report.mjs renders from two sources at once:

  • live — component count, license distribution, and the Raw / Suppressed / Surfacing metrics via tally();
  • static from configheadline, section 4 (affected: []**None.**), criticalNote, remediation, mitigated.

So the first month a new advisory lands in the chevrotain runtime subtree, the generated document — stamped Classification: Confidential and "aligned with EO 14028 / CISA VEX guidance" — will say both of these:

| Surfacing after suppression — review | 1 (0 critical · 1 high · 0 moderate · 0 low) |

Headline posture.none require remediationnone are exploitable in the shipped compiler

## 4. Findings Requiring Remediation (Affected)
**None.**

A compliance artifact that contradicts itself is worse than no artifact, and this one is explicitly framed for acquisition due diligence. Two workable fixes:

  • (a) have build-report.mjs exit non-zero when deltaScan is present, delta.total > 0, and cfg.affected.length === 0 — the workflow fails loudly and a human triages before any document exists; or
  • (b) keep generating, but inject a banner at the top of both MD and HTML (⚠️ DRAFT — N advisories awaiting triage; the narrative below is stale) and replace section 4 with the raw delta list.

I'd lean toward (b): the monthly PR still opens (which is the whole human-review trigger), it just can't lie in the body. Either way the invariant worth encoding is live metrics and static narrative must never disagree.

2. Advisory register row count is read from config instead of the scan

scripts/build-report.mjs — both the Markdown (~L146) and HTML (~L221) artifact tables:

md.push(`| \`sbom/vulnerabilities.csv\` | CSV | Full annotated advisory register (${cfg.advisories.total} rows) |`);

Reproduced locally with a synthetic scan of 1 raw / 0 delta:

| Raw advisories detected (this scan · 2026-08) | 1 (1 critical · 0 high · 0 moderate · 0 low) |
| Surfacing after suppression — review          | 0 (0 critical · 0 high · 0 moderate · 0 low) |
| `sbom/vulnerabilities.csv` | CSV | Full annotated advisory register (29 rows) |

The CSV is produced from the delta scan (node scripts/scan-vulns.mjs /tmp/osv.json …), so in practice it will hold ~0 rows while the report claims 29. Use delta.total when the live scan is available, reusing the same if (rawScan && deltaScan) branch that already exists for advisoryRows.

3. The gate executes its own decision script from the PR's HEAD

.github/workflows/security-pr-gate.yml runs node scripts/pr-gate-diff.mjs … from the PR checkout. Because pull_request uses the workflow and the code from the merge ref, a PR can neutralize the gate by editing scripts/pr-gate-diff.mjs (or flipping GATE_THRESHOLD in the YAML). Deleting the job outright would be caught by the required check reporting "expected but never reported" — editing the script would not.

Since the PR description proposes making gate a required status check, this is worth closing before enforcement:

  • run the decision script from the base worktree (node /tmp/base/scripts/pr-gate-diff.mjs /tmp/base.json /tmp/head.json "$GATE_THRESHOLD"), and/or
  • add a CODEOWNERS entry requiring security review on scripts/, .github/workflows/, and osv-scanner.toml.

The CODEOWNERS route also covers edits to the VEX baseline itself, which today are only caught by reviewer diligence.


Non-blocking, but worth addressing

4. --disable-javascript is very likely a no-op. That Chromium switch was removed a long time ago and unknown flags are silently ignored, so the three-line comment justifying it as defense-in-depth doesn't hold. The current equivalent is --blink-settings=scriptEnabled=false. Worth confirming on a manual run and swapping. (The rich() escaping is the real defense and that part is correct — only attribute-less tags are re-enabled.)

5. The osv-scanner integrity check is TOFU, and it's duplicated three times. SHA256SUMS is fetched from the same release as the binary and isn't signature-verified, so it protects against transport corruption, not a compromised release — "verified against the release SHA256SUMS" overstates it. Pin the literal digest instead:

echo "<known-sha256>  /tmp/osv-scanner_linux_amd64" | sha256sum -c -

The same five-line block is copy-pasted across all three workflows; extracting it to .github/actions/setup-osv-scanner/action.yml (or scripts/install-osv-scanner.sh) will keep the version pin from drifting on the first bump.

6. Third-party actions pinned to mutable tags. peter-evans/create-pull-request@v6 and browser-actions/setup-chrome@v1 run in a job holding contents: write + pull-requests: write and, when configured, SECURITY_BOT_TOKEN. In a supply-chain PR specifically, full-SHA pinning is the expected posture.

7. npx --yes @cyclonedx/[email protected] is fetched at runtime. security-pr-archive.yml executes it in a job with contents: write that then pushes. The version is pinned but integrity isn't. A devDependency + npm ci would put it under the lockfile's SRI; otherwise worth documenting the tradeoff.

8. No timeout-minutes on any job. FETCH_LICENSE=true makes cdxgen hit the network per package across ~764 components; a hang falls through to the 6-hour default. Suggest 30 for monthly/archive and 15 for the gate.

9. The new tests are invisible to npm test and npm run lint. Verified against the repo:

  • vitest.config.ts has include: ["tests/**/*.test.ts", "tests/**/*.spec.ts"], so scripts/__tests__/security-scripts.test.mjs never runs under npm test;
  • npm run lint is eslint src/, and scripts/ matches an ignore pattern — npx eslint scripts/*.mjs returns "File ignored because of a matching ignore pattern" for all six new scripts.

That makes the header comment in scripts/cdx-to-csv.mjs inaccurate — it claims extraction was done "so it gets syntax highlighting, lint coverage, and is testable like the sibling scripts". Suggest adding "test:security": "node --test scripts/__tests__/*.test.mjs" to package.json (called from both the workflow and ci.yml) and bringing scripts/ into ESLint. As it stands, the only place these tests run is the very gate they're meant to protect.

10. VEX baseline: 54 suppressions vs. 29 advisories in the config. grep -c IgnoredVulns osv-scanner.toml returns 54, while report-config.json declares "advisories": { "total": 29 } and notAffected sums to 29 (26 + 3). The two artifacts were generated from different scans, and since the TOML header says "Regenerated from a live osv-scanner scan", the config is the stale one — which is exactly what feeds the report narrative in item 1. Three related points:

  • All 54 entries share ignoreUntil = "2026-10-01". On October 1st, 54 advisories resurface at once. It won't block PRs (they'll be present on both base and head), but the October report will read as a massive regression. Either stagger the dates or call this out in security/README.md.
  • [[IgnoredVulns]] matches by ID globally, not per package. A suppression justified as component_not_present (dev tooling) stays in effect if the same GHSA later shows up in the shipped chevrotain subtree — precisely the case the file header says it wants to keep visible ("The real shipped runtime subtree is left UNSUPPRESSED so a future advisory in it surfaces for triage"). If the pinned osv-scanner version supports PackageOverrides, that's the safer construct for the 51 dev-tooling entries.
  • Minor VEX accuracy: GHSA-f23m-r3pf-42rh and GHSA-r5fr-rjxr-66jc carry the lodash-es rationale but are labelled [lodash]. In a document headed for due diligence, the rationale should match the package it's attached to.

Minor / cleanup

  • Dead workflow steps. The repo has package-lock.json (npm), no .gitmodules, and no Python — so corepack enable, setup-python@v5, and submodules: recursive (all three workflows) only add CI time. generate-sbom.sh already auto-detects TYPE=npm correctly.
  • FETCH_LICENSE=true contradicts the "Reproducible SBOM" header — network-resolved licenses make output dependent on external availability. Relatedly, 2>/dev/null on the cdxgen invocation swallows exactly the OOM diagnostics the comment two lines above says it's mitigating.
  • git fetch --no-tags --depth=1 in the gate runs on a checkout made with fetch-depth: 0, which creates .git/shallow for no benefit — the base SHA is already present. The whole fetch can probably go.
  • security/latest symlink — on a Windows checkout without core.symlinks, this materializes as a text file containing 2026-08. Team develops on Windows. A security/latest.txt pointer or a README line would be more portable.
  • Docs out of sync with the implementation. security/README.md still says the workflow "has Claude triage any new advisory and update the report data", and its directory listing shows a 00-READ-ME-FIRST.md that nothing generates. The build-report.mjs header says "which is what Claude updates when a new advisory appears", and the PR description still mentions CLAUDE_CODE_OAUTH_TOKEN. Since AI triage was dropped for this public repo, the "Reviewing the monthly PR" section needs to describe the human flow.
  • Escaping is inconsistent with the file's own threat model. build-report.mjs declares report-config.json untrusted but interpolates f.priority, f.severity, n.count, m.advisories, and r.label (inside badge()) without esc(). Harmless while those are hand-written literals; a hole the moment the config is machine-generated again. Worth applying esc() uniformly.
  • const sevBadge = (s) => s; is dead code.
  • gen-osv-ignores.mjs isn't wired into any workflow — worth saying so in the header. Also, its parseCsvLine splits on \n before parsing quotes, so it breaks on a quoted field containing a newline — and scan-vulns.mjs writes summary via .slice(0, 140) without normalizing newlines, which is its own input.
  • The "Render PDF" step only installs Chrome; the PDF is produced by the next step. Suggest renaming to "Setup Chrome".
  • "version": "0.6.0" in report-config.json duplicates package.json and will drift — derive it the same way NAME already is.
  • A binary PDF committed monthly grows the code repo monotonically. Consider the same treatment as the archive (orphan branch) or a release asset.
  • Token embedded in the git URL (security-pr-archive.yml) ends up in /tmp/arch/.git/config. Ephemeral and masked in logs, but git -c http.extraheader=… is cleaner.
  • Out of scope, but relevant: ci.yml triggers on branches: [main] while the default branch is development, so lint/typecheck/tests don't run on PRs into development at all. The new workflows target development correctly; the existing one doesn't. Probably its own PR.

Test coverage

Seven tests, all passing, hitting the right parts of pr-gate-diff (threshold, id+package keying, fail-closed exit 2, CVSS 3.1 vector) and cdx-to-spdx (SPDXID dedupe, OR vs AND). Gaps, in risk order:

  1. build-report.mjs has no tests at all, and it's where items 1 and 2 live. A test asserting "a delta > 0 with affected: [] never renders a report that says None" would be the highest-value one in the suite.
  2. scan-vulns.mjs fail-closed path — exiting 1 on a missing results[] is the guarantee that keeps an empty register from reading as "no vulnerabilities". It deserves the same test the gate already has.
  3. cdx-to-csv.mjs — comma/quote/newline escaping.
  4. pr-gate-diff — the approx: true path (CVSS 4.0-only, and advisories with no severity data at all → conservative HIGH). That's the branch deciding to block in the absence of data, and it's currently untested.

Summary

Solid foundation, and the parts that are hard to get right are right. Requesting changes on 1, 2, and 3: item 1 because this pipeline's output is an attestation document that can become false with no human in the loop, item 2 because it's a concrete factual error in that same document, and item 3 because the PR explicitly proposes enforcing the gate as a required check. Items 4–10 are fine as follow-ups, with 9 worth prioritizing — otherwise the tests protecting the gate never run anywhere but inside the gate.

Gustavohsdp and others added 2 commits August 6, 2026 14:08
…rom base

Addresses the three blocking items in marconetsf's review of #199.

1. The monthly report can no longer publish a self-contradicting attestation.
   With AI triage removed, report-config.json is not auto-updated, so a live scan
   that surfaces advisories used to sit above a static "§4: None / none require
   remediation" narrative. build-report.mjs now renders a DRAFT banner and a
   "surfacing this scan — awaiting triage" table (from the delta scan) whenever the
   live scan surfaces anything, and never prints "None" while advisories surface.
   Live metrics and static narrative can no longer disagree.

2. The advisory-register row count is read from the delta scan, not the static
   config (which drifts) — it is generated from that same scan.

3. The PR gate runs its decision script AND threshold from the BASE ref, not the
   PR head, so a PR cannot weaken its own gate by editing pr-gate-diff.mjs or
   GATE_THRESHOLD in the same change (falls back to the head copy only on first
   landing, when base has no gate yet).

New tests assert the invariant "a delta > 0 with affected:[] never renders None,
and shows a DRAFT banner + surfacing table" (build-report had no tests before).

ci-sync: build-report.mjs, gen-osv-ignores.mjs and the unit tests are byte-identical
to autonomy-edge #343 / openplc-web #628 (same shared scripts land in all PRs
together). The baseline/report-config count mismatch (review item 10) is
non-blocking and left for a follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
Ports the shared pr-gate-diff.mjs fix from openplc-runtime #158: iterate OSV
groups[] (aliased advisories = one underlying issue) and take the group
max_severity, so a GHSA and its PYSEC/CVE aliases are counted — and can block —
exactly once, at the issue's real severity, instead of the PYSEC twin falling back
to an approximate HIGH and double-blocking. New tests cover the group dedup/severity.

Keeps the shared scripts and unit tests byte-identical across the SBOM PRs (ci-sync).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
@marconetsf
marconetsf self-requested a review August 7, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants