sbom: add opt-in SBOM generation and SBOM-based vulnerability scanning

[bhouse-nexthop]

Why I did it

SONiC currently ships no Software Bill of Materials. There is no inventory of what is in each .bin, no record of the patches and forks SONiC carries on top of upstream sources, no per-artifact vulnerability surface, and no machine-readable provenance. This makes CVE response, license attribution, supply-chain audits (SLSA, SBOM regulations like EO 14028 / CRA), and reproducibility checks expensive or impossible.

This PR adds opt-in SBOM generation and SBOM-based vulnerability scanning. When ENABLE_SBOM=y is passed at build time, every artifact (.bin, .img, .swi, plus the standalone test-container .gzs) gets a CycloneDX 1.6 SBOM sidecar covering Debian packages, Python wheels, language-ecosystem lockfile contents (Rust, Go, npm, pnpm, yarn), Docker image layers, vendor blobs, and SONiC-built sources. The SBOMs preserve fork-of-upstream provenance (e.g. the FRR submodule pinned to a specific upstream tag plus SONiC's patch set) via pedigree.ancestors[], so a CVE in upstream FRR can be tied back to the SONiC build that carries the corresponding patch. A separate set of standalone scripts consume the SBOMs to produce vulnerability reports (grype + OpenVEX suppressions) and reproducibility diffs.

The default build path is unchanged. With ENABLE_SBOM unset (the default), all SBOM-related work is short-circuited in slave.mk via a make-level $(if) guard, so non-SBOM builds incur zero overhead — no extra subprocesses, no Python startup cost, no apt-cache calls. The 5 added build dependencies (syft, grype, cyclonedx-cli, plus their installer) are only fetched when SBOM is enabled, through the existing sonic-build-hooks wget shim so versions-web tracking still applies.

The Azure Pipelines side replaces the recently-merged Trivy scans (PR #27079) with the new SBOM-based scanner so CI vulnerability reports come from the same data that ships in the artifact, not a separate re-scan of the image tarball.

Comparison with the trivy-based PRs this would supersede

The Azure Pipelines half of this PR replaces / overlaps with two recent trivy PRs:

• PR ci: add Trivy vulnerability scan for docker-ptf and docker-sonic-mgmt #27079 — informational trivy scan of docker-ptf and docker-sonic-mgmt. Already merged. This PR removes those two trivy_scan jobs.
• PR ci: add Trivy vulnerability scan for broadcom and mellanox platform containers and host rootfs #27322 — informational trivy scan of all docker-*.gz containers and the host rootfs for broadcom and mellanox platforms. Not yet merged. This PR addresses the same goal more thoroughly.

Trivy is a perfectly reasonable image scanner. But trivy alone solves only the vulnerability scanning slice of the supply-chain problem, and it does so by re-discovering what's in the artifact at scan time rather than capturing what the build actually put there. Crucially, the SBOM-based approach finds more CVEs, not the same set — because the input components carry richer provenance, the scanner can match them against CVE databases that trivy can't reach from a plain rootfs scan.

	Trivy (PR #27079, #27322)	SBOM + scanner (this PR)
CVE database coverage	NVD + GitHub Advisory DB + distro feeds	NVD + GitHub Advisory DB + distro feeds — same coverage at the DB level
CVEs surfaced for unmodified upstream Debian packages	✓ matches name@version against the trivy DB	✓ matches against grype's equivalent feeds
CVEs surfaced for SONiC-patched / forked upstream packages	✗ false negatives — trivy keys on name@version. SONiC-built debs carry custom version strings (frr_10.5.4-sonic-0, openssh-server_1:9.2p1-2+fips, etc.) that don't match upstream CVE DB entries. CVEs in those packages are silently missed	✓ — pedigree.ancestors[] records the upstream tag/commit. Grype matches against the upstream identity, and OpenVEX statements (auto-extracted from src/*/patches/) suppress the CVEs that SONiC's carried patches have already fixed. So real unfixed CVEs surface; already-patched CVEs don't generate noise
CVEs surfaced for Rust crate transitive deps	✗ false negatives — rustc embeds no crate info; trivy sees only the wrapping .deb and cannot enumerate crates compiled into the binary. Any CVE in a Rust crate (RustSec advisories, GHSA-Rust entries) is unreachable	✓ — slave Dockerfiles install cargo-auditable and route cargo build through a wrapper, so every Rust binary embeds its full resolved Cargo.lock in a .dep-v0 ELF section. Grype's cargo cataloger reads it and CVE-matches every crate. This is the largest CVE-coverage gain in the PR for SONiC's increasingly-Rust-heavy components (sonic-swss, sonic-swss-common, sonic-dash-ha, sonic-host-services, sonic-ctrmgrd-rs, …)
CVEs surfaced for build-time deps that don't survive to the runtime FS	✗ — trivy only sees the assembled rootfs	✓ — collect_version_files harvests lockfiles inside each build container before cleanup phases run, capturing deps that were pulled, used, and discarded during the build
CVEs surfaced for vendor binary blobs (Broadcom/Mellanox/Marvell SDKs)	partial — depends on whether the blob is identifiable to trivy	✓ — recipe-emit fragments label these with pkg:generic/<vendor>/... PURLs and explicit license metadata. Vendors that publish CVE advisories can be matched; for the rest, the component is at least visible in the inventory for triage
False-positive suppression	trivy supports .trivyignore — per-rule, per-line, not signed, detached from the source code that justifies the suppression	OpenVEX v0.2.0 JSON — each suppression carries status, justification, impact_statement, and a reference to the source patch that fixes the CVE. Consumed by grype natively, machine-readable, and the auto-extractor regenerates them from src/*/patches/ on every build (gitignored, can't drift)
Reproducibility check	✗ — trivy output drifts as its CVE DB updates	✓ — the SBOM is a deterministic function of source state; scripts/sbom_diff.py does build-vs-build diffs to detect non-deterministic build inputs
SLSA / in-toto provenance	✗	✓ — unsigned SLSA v1.0 in-toto attestation per .bin (subject SHA-256 + materials). Release engineering signs with cosign later
PURL precision	generic Debian PURLs only	SONiC-specific PURLs (pkg:deb/sonic/... vs pkg:deb/debian/...), plus pkg:github/, pkg:cargo/, pkg:golang/, pkg:npm/, pkg:oci/, pkg:generic/<vendor>/
License attribution	partial — trivy detects licenses but coverage varies	✓ — DEP-5 debian/copyright parser + licensecheck fallback + 105-entry SPDX header→identifier map. Matches the structured format SONiC actually ships
Output for downstream consumers	trivy report (scan artifact only)	CycloneDX 1.6 + SPDX 2.3 + in-toto provenance — shippable alongside the .bin. Customers can run their own scans, do license audits, satisfy EU CRA / EO 14028 reporting
CI cost	downloads multi-GB artifacts (PR #27322 pulls sonic-buildimage.<platform> per platform, extracts squashfs, scans every docker-*.gz in parallel — measured in many minutes per platform)	downloads ~50 MB of *.cdx.json sidecars, scans them in one parallel job. Cheap enough to cover all platforms, not just broadcom + mellanox
Build-time overhead with ENABLE_SBOM=n	n/a (trivy runs post-build)	zero — make-level $(if $(filter y,$(ENABLE_SBOM)),...,:) short-circuit means no Python startup, no apt-cache calls, no extra subprocesses on non-SBOM builds
Build-time overhead with ENABLE_SBOM=y	n/a	~14 minutes of measured SBOM-induced work on a full broadcom build. Breakdown: per-.deb lang-deps extraction ~1m 45s, per-container/slave collect_version_files SBOM harvest ~4m 04s, aggregator × 3 ASIC variants ~7m 30s, VEX patch-scan ~32s. The aggregator's syft scanner_containers step dominates the primary variant (~141s) but the SHA-256-keyed cache drops it to ~15s on variants 2 and 3. Measured during development via temporary per-phase timing instrumentation that was removed before final submission — only the cache itself is shipping
Scope	docker containers + (PR #27322) host rootfs from squashfs extraction	every shipping artifact: .bin, .img, .swi, plus standalone SBOMs for docker-ptf, docker-ptf-sai, docker-sonic-mgmt

TL;DR: trivy answers "what CVEs match the packages I can identify by name+version in this rootfs?" SONiC's SBOM tooling answers a strictly larger question — "what CVEs match the components actually built into this artifact, including SONiC-patched forks (with already-fixed CVEs suppressed via VEX), Rust crates embedded in binaries, and build-time deps that were consumed and discarded?" — at lower CI cost, and produces shippable CycloneDX/SPDX/SLSA artifacts as a side effect.

Work item tracking

• Microsoft ADO (number only):

How I did it

The implementation is two commits on this branch (rebased onto latest master): the SBOM feature itself, plus a small follow-up that stages the CI vulnerability scan in disabled form (see Azure Pipelines integration below). High-level structure:

1. SBOM generation

• Build-flag plumbing: ENABLE_SBOM, SBOM_FORMAT, SBOM_SCAN_TOOL, SBOM_INCLUDE_LICENSES added to rules/config. Makefile.work and build_debian.sh propagate the flag into the slave container and the chroot. prepare_docker_buildinfo.sh injects ENV ENABLE_SBOM into every container's Dockerfile so per-container post-build hooks can branch on it.
• Recipe-emit fragments: A new sbom_emit_fragment helper in slave.mk is invoked from every recipe site that produces an artifact in scope: SONIC_DPKG_DEBS, SONIC_MAKE_DEBS, SONIC_DERIVED_DEBS, SONIC_EXTRA_DEBS, SONIC_ONLINE_DEBS, SONIC_PYTHON_STDEB_DEBS, SONIC_PYTHON_WHEELS, and the docker-image-save step. The helper is wrapped in a make-level $(if $(filter y,$(ENABLE_SBOM)),...,:) short-circuit so non-SBOM builds never invoke Python. Each recipe produces a small per-component CycloneDX fragment; scripts/sbom_fragment.py detects four fork-of-upstream patterns (sonic-net submodules, dget+patches, nested submodule à la FRR, direct-upstream-submodule + sidecar patches) and emits pedigree.ancestors[] accordingly.
• Observation harvest: src/sonic-build-hooks/scripts/collect_version_files is extended (gated on ENABLE_SBOM=y) to emit a 9-column TSV per container (Package, Version, Architecture, Source, SourceVersion, Maintainer, Homepage, Filename, SHA256) using one bulk xargs apt-cache show call plus an awk parser — one fork per container rather than one per package — and to snapshot /usr/share/doc/*/copyright plus any language lockfiles (Cargo.lock, go.sum, package-lock.json, pnpm-lock.yaml, yarn.lock) found in the container.
• Rust crate visibility via cargo-auditable: every slave Dockerfile (sonic-slave-{bookworm,trixie,bullseye,buster}) installs cargo-auditable and drops a wrapper at /usr/local/bin/cargo that routes cargo build through cargo auditable build. The resulting binaries embed the resolved Cargo.lock into a .dep-v0 ELF section. Syft's rust-binary cataloger reads that section at scan time, so the SBOM lists exactly the crates the active feature set actually pulls in — not a Cargo.lock lockfile superset. Zero changes required to any individual debian/rules.
• Lockfile expansion: scripts/sbom_parse_lockfiles.py parses lockfiles harvested from runtime containers and emits pkg:cargo/, pkg:golang/, pkg:npm/ PURLs so transitive dependencies compiled into the image are visible. Mostly relevant for ecosystems other than Rust (which is already covered precisely by cargo-auditable) and Go (already covered natively by syft).
• Image-level aggregator: scripts/build_sbom.py merges recipe fragments + observation TSVs + lockfile components + scanner output. Dedupe runs on three keys (PURL, name+version+arch, name+normalized-version+arch); the normalizer strips epoch prefixes (1:) and suffixes (+fips, +sonic, +bN, +debNuM) so e.g. openssh-server@10.0p1-7 and openssh-server@1:10.0p1-7+fips collapse to one component. The aggregator runs once per .bin/.img/.swi, and again in --container mode to produce standalone sidecars for docker-ptf, docker-ptf-sai, and docker-sonic-mgmt.
• License resolution: scripts/sbom_resolve_licenses.py parses DEP-5 debian/copyright files (the structured format) and falls back to licensecheck heuristic scanning for free-form headers. A bundled scripts/sbom_license_map.json translates 104 common Debian License header strings to SPDX identifiers.
• Outputs: CycloneDX 1.6 JSON is the canonical format; SPDX 2.3 is produced via cyclonedx-cli convert. scripts/sbom_emit_provenance.py produces an unsigned in-toto/SLSA v1.0 provenance attestation per artifact. scripts/sbom_diff.py is provided for two-build reproducibility checks.
• Tool fetch: scripts/install_sbom_tool.sh pins syft 1.44.0, grype 0.112.0, and cyclonedx-cli 0.32.0 with SHA-256 checksums for amd64 and arm64, fetched through the existing wget shim so versions-web records the downloads.

2. Vulnerability scanning + VEX

• Scanner: scripts/sbom_vuln_scan.py runs grype against a CycloneDX SBOM, applies VEX statements under vex/, and emits a CycloneDX VEX-annotated report plus a human-readable table. Policy is configurable (--min-severity, --fail-on, --ignore-unfixed). Standalone Python — not a make target — so engineers and CI can run it against any published SBOM without re-driving the build system.
• Diff: scripts/sbom_vuln_diff.py shows CVE drift between two scan reports (added / fixed / unchanged).
• VEX auto-extraction: scripts/sbom_extract_vex_from_patches.py scans src/*/patches/ for CVE markers (CVE-YYYY-NNNN) in patch headers and emits OpenVEX v0.2.0 JSON statements claiming fixed status with the patch as evidence. The output directory vex/auto/ is gitignored; the extractor runs automatically at the start of scripts/build_sbom.sh so the suppression set always reflects the current state of src/*/patches/.
• Triage workflow: vex/README.md documents the OpenVEX schema, the auto-vs-manual directory split, and the engineer workflow for adding not_affected / false_positive / under_investigation statements with justification and impact_statement fields.

3. Azure Pipelines integration

• .azure-pipelines/azure-pipelines-build.yml: appends ENABLE_SBOM=y to BUILD_OPTIONS so every existing make $BUILD_OPTIONS … line picks it up.
• .azure-pipelines/build-template.yml: same ENABLE_SBOM=y propagation, plus a new inline SBOM vulnerability scan step that iterates over every target/sonic-*.bin.cdx.json and target/docker-*.gz.sbom.cdx.json produced by the build, runs sbom_vuln_scan.py against each, and publishes sbom-vuln-scan-results.<platform>.
• azure-pipelines.yml (top-level): the Test-stage [OPTIONAL] Trivy vulnerability scan (docker-ptf) job from PR ci: add Trivy vulnerability scan for docker-ptf and docker-sonic-mgmt #27079 is replaced with a unified [OPTIONAL] SBOM-based vulnerability scan (all artifacts) job that downloads only the *.cdx.json sidecars from every sonic-buildimage.<platform> artifact (a few MB instead of multi-GB images) and scans them in parallel groups. continueOnError: true mirrors the original — informational, doesn't gate merges.
• .azure-pipelines/docker-sonic-mgmt.yml: same trivy → sbom replacement for the docker-sonic-mgmt pipeline.

All three SBOM vulnerability-scan sites are staged but not active: each carries a condition: false gate (with an inline comment giving the exact predicate to restore). The wiring is in place — sbom_vuln_scan.py, the artifact-download steps, the result-publishing — but the scans don't fire on CI runs yet. SBOM artifact production itself is unaffected: every ENABLE_SBOM=y build still emits the full .cdx.json / .spdx.json / .intoto.json triplet, and scripts/sbom_vuln_scan.py is fully usable as a standalone tool against any published SBOM.

Documentation: README.sbom.md (~150 lines, with table of contents) covers the build flag, the four data sources, dedupe priority, vulnerability scanning quick-start, VEX workflow, reproducibility, and limitations.

How to verify it

Default path — non-SBOM builds are unchanged:

make configure PLATFORM=broadcom
make target/sonic-broadcom.bin
ls target/*.cdx.json target/*.spdx.json 2>&1   # expected: no files

There should be no SBOM artifacts, no extra Python invocations during the build, and no increase in build wall time.

Enable SBOM:

make configure PLATFORM=broadcom ENABLE_SBOM=y
make ENABLE_SBOM=y target/sonic-broadcom.bin
ls target/sonic-broadcom.bin.cdx.json target/sonic-broadcom.bin.spdx.json target/sonic-broadcom.bin.intoto.json
jq '.components | length' target/sonic-broadcom.bin.cdx.json   # expected: thousands

For per-container SBOMs (the test containers not embedded in any .bin):

make ENABLE_SBOM=y target/docker-ptf.gz target/docker-sonic-mgmt.gz
ls target/docker-ptf.gz.sbom.cdx.json target/docker-sonic-mgmt.gz.sbom.cdx.json

Vulnerability scan:

python3 scripts/sbom_vuln_scan.py \
    --vex vex \
    --min-severity medium \
    target/sonic-broadcom.bin.cdx.json

Expected: a table of MEDIUM+ findings with VEX-suppressed CVEs filtered out.

Reproducibility:

# Build twice with identical inputs
python3 scripts/sbom_diff.py build1/sonic-broadcom.bin.cdx.json build2/sonic-broadcom.bin.cdx.json

Expected: an empty diff (any differences indicate non-deterministic build inputs).

VEX auto-extraction:

python3 scripts/sbom_extract_vex_from_patches.py --output vex/auto
ls vex/auto/*.json

Expected: one OpenVEX JSON file per CVE referenced in any src/*/patches/ patch header.

Azure Pipelines: the existing [OPTIONAL] Trivy vulnerability scan (docker-ptf) and [OPTIONAL] Trivy vulnerability scan (docker-sonic-mgmt) jobs are gone from the Test stage; replacement [OPTIONAL] SBOM-based vulnerability scan jobs are wired up but currently gated with condition: false at all three sites (top-level VulnScan stage, per-platform inline scan step, docker-sonic-mgmt scan job). Each gate has an inline comment giving the predicate to restore. To validate the wiring locally, run python3 scripts/sbom_vuln_scan.py --vex vex/ target/<artifact>.cdx.json against any SBOM the build produces.

Which release branch to backport (provide reason below if selected)

• 202305
• 202311
• 202405
• 202411
• 202505
• 202511

This is a feature, not a fix — no backport requested.

Tested branch (Please provide the tested image version)

• master (rebased onto upstream) / sonic-broadcom.bin (verified end-to-end: 56,605/56,600/56,579 components across the 3 asic variants — broadcom/dnx/legacy-th — with ~46 MB CycloneDX, ~27 MB SPDX, ~2 KB in-toto provenance each; exit 0 from a full reset + configure + build cycle.)

Description for the changelog

Add opt-in SBOM generation and SBOM-based vulnerability scanning (ENABLE_SBOM=y); replaces the Trivy CI jobs from PR #27079 with the new scanner.

Link to config_db schema for YANG module changes

N/A — no YANG schema changes.

A picture of a cute animal (not mandatory but encouraged)

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[bhouse-nexthop]

/azpw run

[mssonicbld]

⚠️ Notice: /azpw run only runs failed jobs now. If you want to trigger a whole pipline run, please rebase your branch or close and reopen the PR.
💡 Tip: You can also use /azpw retry to retry failed jobs directly.

Retrying failed(or canceled) jobs...

[mssonicbld]

No Azure DevOps builds found for #27455.

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[bhouse-nexthop]

/azpw run

[mssonicbld]

⚠️ Notice: /azpw run only runs failed jobs now. If you want to trigger a whole pipline run, please rebase your branch or close and reopen the PR.
💡 Tip: You can also use /azpw retry to retry failed jobs directly.

Retrying failed(or canceled) jobs...

[mssonicbld]

No failed(or canceled) stages or jobs found in the most recent build 1118358.

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[securely1g]

Review on behalf of @lguohan:

Summary

Comprehensive SBOM framework for SONiC — CycloneDX 1.6 + SLSA provenance + OpenVEX + vulnerability scanning via grype. This is a significant infrastructure addition.

Architecture Impressions

The layered approach is well-designed:

1. Per-artifact fragments (sbom_fragment.py) emitted at recipe-build-time
2. Aggregator (build_sbom.py) merges fragments + scanner cross-check
3. Post-build vuln scan (sbom_vuln_scan.py) — deliberately decoupled from build
4. VEX for patch-fix suppression — both auto-extracted and curated
5. Lockfile parsing for transitive Rust/Go/npm deps
6. License resolution from DEP-5 copyright files

The separation of SBOM (reproducible, build-time) from vulnerability report (time-dependent, post-build) is the right architectural choice.

Strengths

• Opt-in via ENABLE_SBOM=y — zero overhead for non-SBOM builds
• cargo-auditable integration — embeds resolved Cargo.lock in ELF binaries, then extracts per-.deb
• Multi-source merge with recipe-emit priority > observation > lockfile > scanner — correct precedence
• Fail-soft by default (SBOM_STRICT=n) — SBOM issues never break a build unless explicitly opted in
• sbom_diff.py for reproducibility verification between build hosts
• Thorough documentation in code comments and vex/README.md

Questions / Concerns

1. Build time impact — Even with ENABLE_SBOM=y, how much wall-clock time does this add? The per-.deb dpkg-deb -x + ELF walk + rust-audit-info could be significant across ~120 debs. Any benchmark data?

2. Slave Dockerfile changes across all distros (buster/bullseye/bookworm/trixie) — the cargo-auditable + rust-audit-info + cargo-wrapper addition increases slave image size and build time. Is this gated by ENABLE_SBOM or always installed?

3. cargo-wrapper shim (/usr/local/bin/cargo) — This intercepts ALL cargo invocations. What happens during cargo test, cargo clippy, cargo fmt? Does it only wrap cargo build?

4. Scanner tool pinning (install_sbom_tool.sh) — SHA-256 verification is good. Who owns version bumps? Should these be in a separate versions file for easier tracking?

5. License map completeness — sbom_license_map.json covers common licenses but will inevitably miss exotic ones. Is there a process for community contributions to this map?

6. syft OCI-archive scanning — The comment says "SONiC docker .gz files are OCI archives" — but SONiC uses docker save | gzip, which produces docker-archive format, not OCI. This may cause syft to silently return zero components. Please verify.

7. VEX auto-extraction confidence — sbom_extract_vex_from_patches.py marks loose CVE mentions as under_investigation. How do you prevent false-positive VEX suppressions from leaking into production scans? Is there a review gate?

Minor Issues

• sbom_fragment.py line ~1000: dict[str, Any] type hint requires Python 3.9+ — fine for bookworm+ but breaks on buster (3.7). Consider Dict from typing.
• The prepare_docker_buildinfo.sh change copies cargo-wrapper only for sonic-slave-* images — the condition check looks correct.

Overall

This is production-quality work with excellent documentation. The architecture is sound and extensible. Main concern is verifying build-time overhead and the docker-archive vs OCI-archive distinction for scanner accuracy.

LGTM with the above clarifications addressed.

[bhouse-nexthop]

Thanks for the detailed review. Going through the questions:

1. Build time impact — Good question. We have instrumentation in place (per-.deb lang-deps timing log line in sbom_fragment.py and scripts/build-timing-report.sh for per-package wall-clock from log HEADER/FOOTER markers) but no published A/B benchmark yet. We'll run a back-to-back ENABLE_SBOM=y vs unset build on the same host and post the delta.

2. Slave Dockerfile gating — cargo-auditable@0.6.4 and rust-audit-info@0.5.4 are installed unconditionally, not gated by ENABLE_SBOM. The combined cost is small: ~24 MB against a 13.7 GB slave image (0.17%). We left it unconditional so a slave image rebuilt without ENABLE_SBOM doesn't silently break a subsequent ENABLE_SBOM=y build that reuses it.

3. cargo-wrapper scope — Only wraps cargo build. files/build/cargo-wrapper matches build explicitly in the case and falls through to exec \"\$REAL\" \"\$@\" for everything else (test, clippy, fmt, …). The wrapper docstring documents this.

4. Scanner tool pinning — Versions + SHA-256s are inlined in scripts/install_sbom_tool.sh today. Happy to lift them into a sidecar versions file for easier bumping if that's the preference.

5. License map — README.sbom.md has a "Maintaining `scripts/sbom_license_map.json`" section (lines 318-347) with the 5-step workflow for adding entries and the rationale for keeping it manually curated (false-positive risk on legally-meaningful license-string variants).

6. syft OCI vs docker-archive — SONiC's docker .gz files are OCI archives, not docker-archive. Spot-checked target/docker-database.gz: the inner tar has blobs/sha256/... at the top level (OCI layout), not manifest.json + repositories (docker-archive layout). The oci-archive: scheme is correct; build_sbom.py:518-561 even handles the gzip wrap that syft can't read directly. That said, the last build did show ~6 syft scan of oci-archive:... failed (rc=1) warnings on a subset of containers — final cross-check still recovered 219,587 components so it wasn't catastrophic, but worth a separate look at which containers failed and why.

7. VEX auto-extraction confidence — Two-tier in sbom_extract_vex_from_patches.py: high-confidence CVE regex matches (explicit CVE-in-patch-header or Fixes: pattern) emit status: not_affected (suppresses in grype); loose mentions emit status: under_investigation which is a no-op for suppression — grype still surfaces the finding for human triage. So loose mentions cannot quietly hide a real vulnerability.

Minor — `dict[str, Any]` PEP 585 — Not relevant in practice: only bookworm (3.11) and trixie (3.13) slaves are in active use; buster/bullseye Dockerfile edits were added for completeness but the SBOM emission paths don't run there.

Will follow up with the benchmark numbers.

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[bhouse-nexthop]

Follow-up on the build-time question — we measured the SBOM-induced cost directly via temporary per-phase timing instrumentation added during development. That instrumentation was removed before final submission (only the optimizations themselves are shipping); the numbers below are from those measurement runs.

Per-phase attribution:

Phase	Cost
Per-.deb lang-deps extraction (267 debs: dpkg-deb -x + rust-audit-info + go version -m + dist-info walk)	~1m 45s total
collect_version_files SBOM harvest (~70 invocations: 4 slaves × 2 phases + ~30 containers × 2 phases)	~4m 04s total
Aggregator × 3 ASIC variants	~7m 30s total
VEX patch-scan × 3 invocations	~32s total
Sum of measured SBOM-induced cost	~14 min

Notes:

• Per-.deb extraction is cheap: 267 debs at avg 0.4s/deb. The kernel image alone is 49s (every kernel-module ELF gets walked); the rest are sub-second. Only 11 of 267 actually contained Rust/Go/Python attribution data — most extractions complete in well under 100ms.
• collect_version_files is the per-container/slave bulk apt-cache show + copyrights tar + lockfiles tar. Average 1.12s per invocation. Bulk apt-cache show dominates that step (avg 2.33s) since each container resolves ~1000 packages.
• Aggregator dominates the measurable SBOM cost. Per-variant breakdown for the primary build: scanner_containers 141s, license_resolution 75s, everything else <10s combined. Variants 2 and 3 short-circuit scanner_containers to ~15s via the per-archive SHA-256 cache, dropping their aggregator totals from ~230s to ~110s each.

[mssonicbld]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[bradh352]

/azpw run

[mssonicbld]

⚠️ Notice: /azpw run only runs failed jobs now. If you want to trigger a whole pipline run, please rebase your branch or close and reopen the PR.
💡 Tip: You can also use /azpw retry to retry failed jobs directly.

Retrying failed(or canceled) jobs...

[mssonicbld]

Retrying failed(or canceled) stages in build 1123889:

✅Stage Test:

• Job impacted-area-kvmtest-multi-asic-t1 by Elastictest: retried.