Skip to content

Implement bounded offline OCI, Docker-save and SBOM inventory - #1

Merged
CompleteDotTech merged 7 commits into
mainfrom
feat/offline-inventory
Oct 26, 2026
Merged

CompleteDotTech merged 7 commits into
mainfrom
feat/offline-inventory

Conversation

@CompleteDotTech

@CompleteDotTech CompleteDotTech commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Add optional standalone offline inventory for bounded OCI/Docker-save archives and CycloneDX/SPDX metadata. Verify SHA-256 OCI descriptors and sizes; apply layer whiteouts/replacements in memory; never extract, follow links, execute images/packages or contact registries. Preserve independent image failures and supported siblings. Bound bytes, expansion, paths, members, capture and processing work. Selected metadata links/devices are incomplete; Docker config digests explicitly do not bind layer contents. Emit actual conservative Node2Shell/Sharp/Flowise input profiles or canonical provenance.

Validation: 19 local synthetic tests pass, with hostile archive/whiteout/link/gzip-header/publication cases and actual consumer CLI probes. GitHub Copilot hosted and CLI findings were fixed; the final independent remediation review found no actionable defects on 875ba15. Six exact-head hosted jobs pass; help, diff and checksum validation pass. Supports node2shell-check #12; all assessed exploitability remains not_tested.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved parsing, whiteout, provenance and export-limit defects undermine reliable inventory reporting and consumer compatibility.

4 open issues
What changed in this PR

Implements the optional offline inventory tool tracked by node2shell-check #12, providing container and SBOM metadata to standalone checkers without becoming a runtime dependency.

Changes:

  • Adds bounded OCI, Docker-save and SBOM parsing with digest verification, layer processing and provenance.
  • Adds canonical JSON and Node2Shell, Sharp and Flowise exports with protected output publication.
  • Adds synthetic tests, cross-platform CI, usage documentation and an example SBOM.
File Description
test_inventory.py Adds synthetic parsing, safety, export and publication tests.
SHA256SUMS Records the Python entry-point checksum.
README.md Documents usage, supported formats, limits and uncertainty.
oci_runtime_inventory.py Implements the standalone parser, inventory exports and CLI.
examples/​inventory.sbom.json Provides a synthetic CycloneDX example.
.github/​workflows/​tests.yml Runs tests and CLI help across three operating systems and two Python versions.

Review effort: Balanced

Comment thread oci_runtime_inventory.py Resolved
Comment thread oci_runtime_inventory.py Resolved
Comment thread oci_runtime_inventory.py Resolved
Comment thread oci_runtime_inventory.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Consumer byte validation can allow oversized output, and later layers can hide invalid metadata beneath a nondirectory ancestor.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Validate selected-file ancestors after every layer

oci_runtime_inventory.py:426

A layer containing a symlink app and app/package.json, followed by a layer replacing app with a directory, retains the inaccessible package and reports complete coverage. The final-only ancestor check misses the invalid intermediate state. Validate selected-file ancestors after each layer and reject the image before later layers can hide this condition.

4 resolved since last review

Review effort: Balanced

Comment thread oci_runtime_inventory.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The hostile-archive parser requires human review, with unresolved correctness and resource-handling issues and unverified exact-head hosted CI.

2 open issues
4 previously missed issues

In code that hasn't changed since last review

Medium severity Enforce 1 MiB trailing padding limit on layers

oci_runtime_inventory.py:211

The documented 1 MiB TAR trailing-padding limit is enforced for the outer archive but not for layers. After the layer terminator, this loop accepts zero padding up to the remaining global expanded-byte budget, so a layer with several MiB of padding is marked complete instead of exceeding the advertised limit. Track post-terminator bytes here and reject padding beyond 1 MiB for both plain and gzip layers.

Medium severity Validate schemaVersion for OCI indexes

oci_runtime_inventory.py:304

Root and nested OCI indexes never validate schemaVersion, unlike image manifests below. An index.json with a missing or unsupported schema version is still traversed and can produce complete: true and exit 0 when its descriptors resolve. Validate each index as schema version 2 before processing its descriptors so malformed or unsupported indexes are not reported as fully supported.

Medium severity Recognize held Node.js packages as installed

oci_runtime_inventory.py:391

Matching only install ok installed omits installed Node.js packages with Status: hold ok installed. A held installation therefore exports an empty coverage placeholder instead of its observed version. Check the installation-state field independently of the desired selection, and add a held-package fixture.

Medium severity Exclude namespaced CycloneDX components from unscoped matches

oci_runtime_inventory.py:486

CycloneDX uses group for a package namespace, so a component with group: "@vendor" and name: "sharp" represents @vendor/sharp, not the unscoped sharp package. Matching only name emits the wrong package identity into both canonical and checker inventories; the same issue affects the other target names. Validate the CycloneDX group and exclude nonempty namespaces when matching these unscoped targets, while continuing to traverse nested components.

1 resolved since last review

Review effort: Balanced

Comment thread oci_runtime_inventory.py Resolved
Comment thread oci_runtime_inventory.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Whiteout ancestor validation and Sharp export filtering need fixes, and file-publication guarantees need automated coverage.

No open issues 🎉

3 previously missed issues

In code that hasn't changed since last review

Medium severity Reject whiteout-prefixed ancestor components

oci_runtime_inventory.py:228

A layer containing app/.wh.removed and app/.wh.removed/package.json can emit a package observation and be marked complete because the whiteout marker disappears before ancestor validation. OCI reserves .wh.* names, so reject whiteout-prefixed ancestor components while preserving valid same-layer additions under app/removed/. Refresh SHA256SUMS after updating the entry point.

Medium severity Reject C1 controls in Sharp version evidence

oci_runtime_inventory.py:527

C1 controls (U+0080–U+009F) pass this filter, but Sharp's inventory schema rejects that entire range. A synthetic SBOM with version 0.35.5\u0085 makes --consumer sharp exit 0 and publish a payload that violates the checker's input contract. Reject C1 controls alongside the existing controls, emitting empty version evidence without changing the canonical observation, and refresh SHA256SUMS after updating the entry point.

Medium severity Add tests for file publication success and interruption

oci_runtime_inventory.py:572

Please add synthetic tests for successful and interrupted file publication. Every --output invocation in test_inventory.py:271–278 is rejected by validate_output, so neither new-file publication via os.link nor forced replacement via os.replace is exercised by the CI matrix; the interruption test only covers stdout. Test both successful paths and inject an interruption at file publication, checking destination preservation before commit and that publication is not retried.

2 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Hostile-archive parsing has unresolved correctness and resource-limit issues and still requires final human review.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Isolate per-image planning failures

oci_runtime_inventory.py:430

If an OCI index contains a valid image followed by one declaring an unsupported layer format such as zstd, image_plans() raises before any image is processed, leaving images: []. This loses usable sibling-image observations and omits the failed image's incomplete-coverage record. Isolate planning failures per image so supported siblings are scanned and failed images are recorded as incomplete, while retaining the global limits.

Review effort: Balanced

Comment thread oci_runtime_inventory.py Resolved

Copy link
Copy Markdown
Contributor Author

Final validation: 875ba15 has SUCCESS hosted checks on all six configured jobs. 19 local tests pass (CopyEscape has one platform skip). GitHub Copilot CLI reviewed the final remediation diff and relevant surrounding code with no actionable defect; earlier hosted/CLI findings have code/tests or explicit documented dispositions. No unresolved inline threads remain. Review did not execute tests; local and hosted validation are separate evidence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants