chore(security): automated monthly SBOM & VEX report - #158
Conversation
Ports the supply-chain automation to openplc-runtime (public repo — no AI triage; Claude step skipped gracefully, deterministic SBOM + report + PR still run). Python variant: generate-sbom.sh resolves requirements.txt into a clean venv and builds the CycloneDX SBOM with cyclonedx-py; OSV scans the SBOM honoring the VEX baseline. 0 advisories require action (all 5 confined to test/dev tooling or functions the Flask server never calls). SOEM (EtherCAT C lib) is a submodule tracked apart — no advisory, but GPLv3-or-commercial (material, noted in report). Baseline suppresses the 23 known IDs; the live Flask stack is left visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
…erge (Python) Gate now only scans/reports (contents: read) and posts one actionable sticky PR comment; per-PR SBOM archived on MERGE by security-pr-archive.yml. Validated on autonomy-edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
4fafbf9 to
6a54f18
Compare
Gate fails closed on scanner error; MEDIUM no longer blocks as HIGH; CVSS 4.0 handled conservatively; diff keyed by advisory id + package. Report HTML escaped (+ PDF rendered with JS disabled). cdx-to-spdx dedupes packages and uses OR for dual licenses. gen-osv-ignores requires a verdict column. Validated on autonomy-edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
…ic repo Scan fail-closed; osv v2.4.0 (+SHA256) & cyclonedx-bom 7.3.1 pinned; node:test in the gate; cdx-to-csv.mjs; OSV text; explicit base. Public repo: removed Claude/ OAuth triage step + token guidance (deterministic report + human review only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
…token fallback (Python, public: no AI) build-report from live scan; monthly raw+delta (CDX); archive -> push-on-default + orphan security-archive (SBOM & raw scan only); token fallback. Public repo: Claude triage kept out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
Review feedback addressed ✅Thanks for the thorough review. Every point raised has been addressed across follow-up commits — the Correctness / security bugs
Supply-chain / hardening
Report & archive integrity
Since this is a public repository, the optional Claude/OAuth triage step and its token guidance were removed — the monthly report is deterministic + human-reviewed. Two notes
Ready for another look 🙏 |
marconetsf
left a comment
There was a problem hiding this comment.
Reviewed the full diff plus the actual CI logs for this branch. The engineering is solid and the fail-closed reasoning is above average for this kind of pipeline, but there are a few things I think need to be resolved before merge — the first one only shows up in the run logs, so starting there.
Blocking
1. The entire VEX baseline is dead, and the generated report will contradict itself
From run 30915809962 (the latest gate run on this branch):
Scanned .../sbom/openplc-runtime.cdx.json file and found 41 packages
osv-scanner.toml has unused ignores:
- GHSA-58qw-9mgm-455v
- GHSA-5rjg-fvgr-3xxf
... (all 23 IDs)
23 of 23 suppressions are unused. Because requirements.txt is unpinned, resolution has already moved to fixed versions, so the current scan finds zero advisories across the 41 packages.
That breaks the report internally. build-report.mjs:632-640 computes the "Key metrics" table live from the scan, so it will print 0 raw advisories, while report-config.json still asserts in headline that "its 5 advisories are all either in test tooling or in vulnerable functions the runtime never calls", with notAffected listing 5 and criticalNote restating it. The static advisories.total: 5 also feeds the vulnerabilities.csv row count at build-report.mjs:717, which is generated from the live scan.
A compliance deliverable that contradicts its own live numbers is worse than no deliverable. Either the narrative fields have to be derived from the scan too, or the config has to be regenerated as part of the same run that renders the report.
2. Suppressions have no version constraint, so they fail open silently
[[IgnoredVulns]] matches on advisory ID only. If a future resolution (or an explicit pin) brings back the vulnerable version of python-dotenv, click, pytest or filelock, the advisory stays suppressed until 2026-10-01 — and since the PR gate reads the same TOML, it will not block either. Suggest trimming the baseline to what the scanner actually reports today (apparently nothing) and pinning requirements.txt, which also removes the drift that caused item 1.
3. PYSEC/GHSA aliasing causes false HIGH blocks
severityOf() (scripts/pr-gate-diff.mjs:1193-1212) only reads database_specific.severity. PYSEC records typically carry neither that field nor a CVSS vector, so they fall through to case 4 and return { sev: 'HIGH', approx: true }. With GATE_THRESHOLD=HIGH that blocks the merge.
Worse, osv-scanner emits GHSA and PYSEC as separate vulnerabilities[] entries for the same underlying issue, so a MODERATE advisory is counted twice: once bucketed correctly, once as an approximate HIGH that blocks. scan-vulns.mjs double-counts the same way in the CSV register.
Suggested fix: consume results[].packages[].groups[] (it carries aliases and max_severity) and dedupe per group; and/or treat approx: true as warn-only rather than blocking. A test case for exactly this alias pair would be the highest-value addition to the existing suite.
4. Untrusted pip install runs in jobs that hold a write token
security-pr-archive.yml (contents: write, pushes to main) and security-monthly.yml (contents + pull-requests: write) both run scripts/generate-sbom.sh, which does pip install -r requirements.txt and therefore executes setup.py from any sdist in the resolution.
For a PR whose whole subject is supply chain, worth splitting: one job with no permissions resolves and scans and uploads an artifact, a second minimal job publishes it. --only-binary :all: (or --require-hashes once pinned) removes code execution from the resolve step entirely. The gate already got this right with contents: read — that is the pattern to replicate.
5. Third-party actions pinned to mutable tags
peter-evans/create-pull-request@v6 (v7 is current), browser-actions/setup-chrome@v1, and the actions/* uses. Full-SHA pinning is the baseline expectation in a supply-chain pipeline.
Should fix
6. The SBOM does not cover the Python plugins. There are core/src/drivers/plugins/python/{modbus_master,modbus_slave,opcua}/requirements.txt files, and scripts/manage_plugin_venvs.sh installs those venvs into the runtime. The SBOM only resolves the root requirements.txt, and the gate's manifest-change list (security-pr-gate.yml:291) does not include those paths — so a PR adding a vulnerable dependency to a plugin passes in ~40s with no base resolve and never appears in the inventory. That conflicts with the report's "full transitive graph" claim.
7. requirements.txt mixes runtime and dev deps (pytest, pytest-flask, pre-commit). That is the root cause of roughly 18 of the 23 suppressions ("component_not_present: test/developer tooling"). Splitting out requirements-dev.txt — which the gate already looks for — fixes the baseline at the source and makes the SBOM reflect the deployed surface.
8. Two of the three workflows have never run. gh run list shows only 5 runs, all "Security — PR gate". The whole monthly path (Chrome/PDF, ls | grep, the security/latest symlink, create-pull-request) and the archive/orphan-branch path are unexercised. Worth triggering the monthly via workflow_dispatch before merging.
9. REPORT="$DIR/$(ls "$DIR" | grep -E 'Security-Report\.html$')" (security-monthly.yml:100): without pipefail, a non-matching grep yields "$DIR/" and Chrome fails with an opaque error. build-report.mjs:797 already computes the deterministic basename — have it print the path and consume that.
10. --disable-javascript is not in the modern Chromium switch list (it was removed); the working equivalent is --blink-settings=scriptEnabled=false. Worth confirming on the first manual run, since the comment presents it as defense-in-depth. The escaping in build-report.mjs remains the real control either way.
11. One wasted full resolve per PR (security-pr-gate.yml:329): the trailing generate-sbom.sh "restores" HEAD's sbom/, but nothing after it consumes sbom/ (the archive is a separate workflow). cp -r sbom /tmp/head-sbom beforehand, or just drop the step.
12. Token on the git command line (security-pr-archive.yml:208,223): https://x-access-token:$GH_TOKEN@... is visible in /proc and in any error that echoes the URL. Prefer http.extraheader or GIT_ASKPASS. Also, git clone ... 2>/dev/null swallows auth failures and falls into the "create orphan branch" path for the wrong reason.
13. The gate can suppress itself without mandatory review. It reads osv-scanner.toml from the PR head, so a PR can unblock itself by adding an [[IgnoredVulns]] entry (documented as option 3). Reasonable by design, but it wants CODEOWNERS on osv-scanner.toml and security/report-config.json, otherwise the control is effectively optional.
Conventions and minor
- Emojis violate the project standard. CLAUDE.md: "No emojis anywhere in code, comments, or documentation".
scripts/pr-gate-diff.mjsuses them in the error output and in the PR comment body. - Base branch. CLAUDE.md says feature branches start from and target
development(which exists; releases reachmainby merging it). This PR targetsmain, and all three workflows hardcodemain. Confirm that is intentional. - A Node toolchain in a C/C++/Python repo.
.pre-commit-config.yamlhas no JS hook (onlycheck-yaml/check-json), so the five.mjsfiles get no lint or format coverage — and the preceding commit messages say "(Python)". Either add prettier/eslint to pre-commit, or port to Python (Black/isort/Ruff are already configured). - Tests sit outside the repo suite.
scripts/__tests__/only runs in the gate workflow;scripts/run-pytest.shdoes not reach it. Coverage is partial:scan-vulns.mjs,cdx-to-csv.mjs,gen-osv-ignores.mjsandbuild-report.mjs(the script that produces the deliverable) have no tests at all. - Hand-maintained versions are already stale.
report-config.jsonsays4.1.8,mainis at v4.1.9,pyproject.tomlsays0.1.0. Worth deriving fromVERSION/tag. The PR description says "45 Python components"; the actual run reports 41. - Stale docs.
security/README.mdstill says "has Claude triage any new advisory" and "check Claude's justification", and its tree shows00-READ-ME-FIRST.md, which no script generates. The header comment insecurity-monthly.ymlends mid-sentence ("Auth for the triage step: your Claude"). The PR description states the opposite — no AI triage on this public repo. gen-osv-ignores.mjsis not invoked by any workflow; the header reads as if it were a pipeline step.- Nits:
bucketFromScorehass > 0 ? 'LOW' : 'LOW'; both CSV writers omit a trailing newline (theend-of-file-fixerhook would flag them if committed);security/latestas a symlink plus thedestroyed-symlinkshook plus Windows checkouts (the repo has MSYS2 branches) is a likely broken-symlink commit;SBOM_PIP_ARGSis declared in two workflows but missing from the monthly one, despite the comment at line 49 saying it is set per repo; the osv-scanner install block is duplicated three times and is a good candidate for a composite action.
What works well
Worth saying explicitly, because a lot of this is careful:
- Consistent, deliberate fail-closed behavior: any osv-scanner exit code outside
{0,1}is an error,load()exits 2 on missing or shapeless input, andscan-vulns.mjsrefuses to write an empty register. contents: readon the gate, explicitly justified so it is safe to require in branch protection.- Idempotent sticky comment via an HTML marker, and the guard that skips commenting on fork PRs.
- Escaping plus a tag allowlist for
report-config.jsonstrings in the rendered HTML. SPDXIDdedup andORinstead ofANDinlicenseExpr, both covered by tests —ANDwould invert the legal obligation in a procurement artifact, so that is exactly the right thing to have pinned down.- Keying by
${id}|${package}so a version bump of an already-vulnerable package is not reported as introduced. - Verified checksum on the osv-scanner download; CVSS 3.1 implemented against the FIRST.org spec with the section referenced.
- The SOEM (GPLv3-or-commercial) licensing finding is genuinely material and well documented.
Verdict
Requesting changes on items 1-3 and 6 (and preferably 4-5) — as it stands, merging would ship a compliance report that contradicts its own live numbers, plus a gate with a known false-block mode. Everything else is fine as follow-up.
… PYSEC gate, cover plugins Addresses the blocking items (1, 2, 3, 6) in marconetsf's review of #158. 1 & 2. The report no longer contradicts the live scan, and the baseline no longer fails open. requirements.txt is unpinned and CI resolves it into a clean venv, so the scanned set is the currently-resolved one, which has no known advisory — yet report-config asserted 5, and all 23 osv-scanner.toml suppressions were reported "unused ignores" (a stale suppression can only fail OPEN). The baseline is emptied and report-config reconciled to zero (headline/criticalNote made qualitative), so live metrics and static narrative agree. build-report.mjs derives the counts from the live scan and renders a DRAFT banner + a surfacing table if anything ever surfaces above the baseline. 3. PYSEC/GHSA aliasing no longer causes false HIGH blocks or double counts. pr-gate-diff.mjs now iterates OSV groups[] (aliased advisories = one issue) and takes the group max_severity, so a PYSEC record with no severity of its own is scored at its real (GHSA) severity and counted once, represented by its GHSA id. Verified against a live scan: the four filelock/pygments PYSEC+GHSA pairs collapse to four MODERATE/LOW findings and the gate PASSES instead of falsely blocking. 6. The SBOM now covers the Python plugins. generate-sbom.sh resolves every core/src/drivers/plugins/python/*/requirements.txt into the SBOM env, and the PR gate includes those manifests in its change detection and restores their base versions for the base resolve — so a PR adding a vulnerable plugin dependency is caught instead of passing without a diff. New tests cover the PYSEC+GHSA group dedup/severity and the report DRAFT invariant. ci-sync: build-report.mjs / gen-osv-ignores.mjs / pr-gate-diff.mjs / the unit tests are byte-identical to the other SBOM PRs (the pr-gate PYSEC fix lands in all four). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
…le AI-triage docs Non-blocking follow-ups from marconetsf's review of #158: - Pin the third-party and actions/* uses in the three security workflows to full commit SHAs (create-pull-request, setup-chrome, checkout, setup-node, setup-python) — mutable tags are not an integrity guarantee in a supply-chain pipeline (review item 5). - Render the PDF from build-report.mjs's deterministic report basename instead of `ls | grep` (which, without pipefail, yields "$DIR/" on no match and fails Chrome opaquely); assert the file exists (item 9). - Replace the no-op `--disable-javascript` Chrome switch with the working `--blink-settings=scriptEnabled=false` (item 10). - Drop the stale AI-triage wording from security/README.md and the monthly workflow header (00-READ-ME-FIRST.md, "Claude triage", mid-sentence header) — this public repo has no AI triage; advisories surface in the PR for a human. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AejAiL4tfXCyMhwtjY4vFT
Review feedback — resolutionThanks for the very thorough review. Here is a point-by-point status. Two commits carry the changes: Blocking
Preferably (4–5)
Should fix (7–13)
Conventions & minor
Happy to fold items 4, 11, 12 (and 7/13 once you confirm the policy calls) into a single follow-up if you'd prefer them in this PR instead. |
marconetsf
left a comment
There was a problem hiding this comment.
Approving, conditional on one small edit landing before merge (details at the end).
I re-checked the four blocking items against 6f89344, ran the suite locally (17/17 pass) and verified the pinned SHAs upstream. The follow-up work is good:
- 2 — baseline emptied rather than kept, with the fail-open reasoning written down in the file. Right call.
- 3 —
pr-gate-diff.mjsnow iteratesgroups[]and takesmax_severity, so an aliased PYSEC record is scored at its real severity and counted once. The two new tests pin exactly the pair that would have false-blocked, including the inverse case (a PYSEC-only group with a genuine HIGHmax_severitystill blocks). That is the test I was hoping for. - 6 —
generate-sbom.shresolves the plugin manifests, and the gate both detects them in the change list and restores their base versions for the base resolve. The|| echo "::warning::"per plugin is the right trade-off: a resolution conflict degrades to visible partial coverage instead of silently dropping it. - 1 — the structural half is solved: counts derived from the live scan, DRAFT banner, and a "surfacing this scan" table so the document can never publish a stale all-clear.
registerRowsfrom the delta scan closes the CSV drift. - 5 — all five
uses:on full SHAs across the three workflows. I resolved each SHA against its upstream repo; they check out. - 9 / 10 — deterministic basename with an existence assert, and the working Blink switch.
The deferrals on 4, 7, 8, 11 and 12 are reasonable, and I agree with the reasoning on 4 specifically: --only-binary :all: would break any sdist-only dependency, and the resolve/publish job split deserves its own reviewable change rather than being bolted on here. Please do land it soon though — both workflows that run generate-sbom.sh still hold a write token, and they only start executing after this merges.
Before merge
security/report-config.json:26 still carries the old number, and it renders verbatim into section 7 of the deliverable (build-report.mjs:240 and :321, with no draft conditional):
"No action required for exploitable vulnerabilities - all 5 advisories are not reachable in the running product"
while the Key metrics table in the same document now prints 0 from the live scan. That is the same self-contradiction that blocked the first pass, just reduced to one line. Two neighbours worth fixing in the same edit:
practicesasserts"Automated dependency updates (Dependabot / advisory databases)"as"In place", but there is no.github/dependabot.ymlin this repo. A practice claimed as in place in a compliance artifact should either exist or be markedRecommended."version": "4.1.8", whileVERSIONisv4.1.9. Deriving it fromVERSIONat render time would stop this from recurring.
Follow-up (not blocking)
Residual half of item 3. The gate is fixed, but scan-vulns.mjs:30 and idSet() in build-report.mjs:46 still count per v.id with no alias grouping. Harmless today at zero advisories, but on the first real one the gate and the report will state different totals for the same scan, and the DRAFT table will show the issue twice (once as GHSA, once as PYSEC with severity UNKNOWN). Same groups[] treatment applies.
Item 13 / CODEOWNERS. Nothing exists in the repo yet. Until it does, the suppression path is an optional control. I will get you the security team handle so you can cover osv-scanner.toml, security/report-config.json, scripts/ and .github/workflows/.
Base branch — already discussed offline, nothing needed here.
Emojis in pr-gate-diff.mjs. Accepted for now on the ci-sync argument: keeping the five copies byte-identical is worth more than the convention here. Please fold it into the shared-scripts pass across all repos when that happens, along with the bucketFromScore no-op and the trailing CSV newline.
Approving on the understanding that the report-config.json edit lands before you merge.
What this PR adds
Installs the automated software supply-chain security pipeline for openplc-runtime (hybrid C/C++ core + Python Flask web server). Three flows, all driven by the same data:
.github/workflows/security-monthly.yml. On the 1st of each month: resolvesrequirements.txtinto a clean venv and builds the SBOM with cyclonedx-py (CycloneDX 1.6 + SPDX 2.3), scans against OSV honoring theosv-scanner.tomlVEX baseline, renders a report fromsecurity/report-config.json, opens a PR with the dated snapshot undersecurity/<YYYY-MM>/. Runs against the default branch (main)..github/workflows/security-pr-gate.yml. On every PR: fails only when the PR introduces a NEW HIGH/CRITICAL advisory (pre-existing never blocks). Resolves base vs head SBOMs; skips the base resolve whenrequirements.txtis unchanged. Posts a sticky comment with package, version, advisory link, fixed version, and how to resolve..github/workflows/security-pr-archive.yml. On merge, commits the SBOM + report undersecurity/pr-<number>-<date>/.Supporting files:
scripts/(Python SBOM generator, report renderer, OSV diff/gate),osv-scanner.toml(VEX baseline),security/report-config.json,security/README.md.Security posture (assessment)
45 Python components (the single third-party C library, SOEM, is a pinned git submodule with no known advisory, tracked separately). All 5 Python advisories are not applicable (test/dev tooling, or functions the running Flask server never calls): 0 require remediation, 0 exploitable.
The runtime is MIT, but the optional EtherCAT plugin links SOEM, which is GPLv3-or-commercial (RT-Labs). Under the open-source option the plugin becomes GPLv3; a commercial product likely needs a purchased SOEM license. Affects only the EtherCAT plugin — the MIT core and other plugins are unaffected. Flagged for acquisition due diligence.
What to review
osv-scanner.toml+security/report-config.json— the VEX triage + the SOEM licensing note. Best reviewed by product security (Thiago)..github/workflows/security-*.yml— the three flows.Notes
gatereports pass/fail but does not block merges until set as a required status check.