Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ python oci_runtime_inventory.py --sbom examples/inventory.sbom.json
python oci_runtime_inventory.py --archive authorized-image.tar --consumer sharp --output sharp-input.json
python oci_runtime_inventory.py --archive authorized-image.tar --consumer node2shell --output node-input.json
python oci_runtime_inventory.py --sbom examples/inventory.sbom.json --consumer flowise --output flowise-input.json
python oci_runtime_inventory.py --sbom examples/inventory.sbom.json --consumer sharp --output sharp-input.json --receipt sharp-receipt.json
python -m unittest -v
```

Expand All @@ -36,6 +37,24 @@ digests when comparing observations. Neither identity establishes publisher trus
or that an archive can run. Config Env/Cmd/hostnames and image
tags are not reported. SBOM digests identify the SBOM artifact, not an image.

Canonical reports also retain `generated_at` (UTC inventory-generation time),
`tool.version` and `tool.executable_sha256` (SHA-256 of the standalone file's
bytes, including when invoked through a symlink). The code has no separate
advisory/rule table: the executable digest identifies its parser and export
rules. Generation time is captured once per invocation and reused if an
interruption produces a partial report. It is not an artifact build time,
evidence-acquisition time, publisher attestation or deployed-state observation.
No such times are inferred from file/archive modification timestamps.

`content_sha256` hashes the canonical JSON projection with `generated_at` and
`content_sha256` omitted, using sorted keys, ASCII escaping, two-space indentation
and a final newline in UTF-8. This permits stable content comparisons across
different generation times while retaining all tool, coverage and artifact
evidence. Existing artifact/config/layer/SBOM digests remain unchanged. For
deterministic Python fixtures, `main(argv, clock=...)` accepts a callable returning
an aware `datetime`; it is converted to UTC without altering content identities
or confidence. The public CLI uses the current UTC clock.

The virtual filesystem applies layers in order. Whiteouts remove lower-layer
entries; opaque whiteouts retain same-layer additions regardless of TAR ordering.
Replaced/deleted manifests cannot survive as current package observations.
Expand All @@ -59,6 +78,21 @@ not empty evidence presented as a clean scan. Canonical output retains the full
coverage/provenance record; consumer output intentionally carries only each
checker's accepted fields. Preserve the process exit code as well as the output.

Consumer schemas do not accept the canonical generation/tool envelope, so these
fields are never inserted into their records. Use `--receipt PATH` with a consumer
export to retain the canonical inventory alongside it. The receipt includes
`consumer`, `export_sha256` (the SHA-256 of the exact exported UTF-8 JSON bytes,
including the final newline), and `inventory`. It contains no input/output path
or hostname. CLI stdout is written as UTF-8 bytes without platform newline
translation, so a redirected export has the same hash on Windows and POSIX.
The two output paths must be distinct and each uses the same
protected no-clobber/explicit-force policy. The receipt is published first;
publication of two files is not transactional. If interruption or output failure
leaves only the receipt, compare its `export_sha256` with the consumer bytes
actually received before associating them. Neither publication is retried after
either begins. A receipt for incomplete input retains `complete=false`; the
consumer retains its existing unknown placeholder policy and exit code.

## Supported formats and bounds

- Uncompressed outer USTAR/V7 TAR containing OCI image-layout 1.0 or Docker-save
Expand Down
2 changes: 1 addition & 1 deletion SHA256SUMS
Original file line number Diff line number Diff line change
@@ -1 +1 @@
e4bec83a0e16bdfb7935040844535a82164d113b16c3d9aea17c3284242cadd4 oci_runtime_inventory.py
5a9c40db3524f5934ef2d99b2735d77b04de323b0d56c12b60c2fa8c5d3572cb oci_runtime_inventory.py
66 changes: 58 additions & 8 deletions oci_runtime_inventory.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from __future__ import annotations

import argparse
from datetime import datetime, timezone
import gzip
import zlib
import hashlib
Expand Down Expand Up @@ -557,6 +558,28 @@ def export_inventory(report, consumer):
return output


def serialized(data):
return json.dumps(data, ensure_ascii=True, sort_keys=True, indent=2, allow_nan=False) + "\n"


def content_identity(report):
"""Hash canonical evidence, excluding generation time and this digest."""
content = {key: value for key, value in report.items() if key not in {"generated_at", "content_sha256"}}
return hashlib.sha256(serialized(content).encode("utf-8")).hexdigest()


def generation_context(clock=None):
value = (clock or (lambda: datetime.now(timezone.utc)))()
if not isinstance(value, datetime) or value.tzinfo is None or value.utcoffset() is None:
raise InventoryError("generation_clock_requires_aware_datetime")
with regular_input(Path(__file__).resolve())[0] as stream:
# Source identity is bounded independently of potentially large input archives.
source = stream.read(MAX_JSON + 1)
if len(source) > MAX_JSON:
raise InventoryError("executable_identity_size_limit")
return value.astimezone(timezone.utc).isoformat().replace("+00:00", "Z"), hashlib.sha256(source).hexdigest()


def _export_inventory(report, consumer):
rows = [row for image in [*report["images"], *report.get("sboms", [])] if image["coverage_complete"] for row in image["components"]]
def version(row, maximum=512):
Expand Down Expand Up @@ -589,10 +612,17 @@ def validate_output(output, inputs, force):


def publish(data, output, force, state):
text = json.dumps(data, ensure_ascii=True, sort_keys=True, indent=2, allow_nan=False) + "\n"
text = serialized(data)
if output is None:
state["stdout_started"] = True
sys.stdout.write(text)
binary = getattr(sys.stdout, "buffer", None)
if binary is not None:
# Receipt hashes cover these exact UTF-8 bytes, not platform text translation.
binary.write(text.encode("utf-8"))
binary.flush()
else:
# In-memory text streams used by embedders have no byte channel.
sys.stdout.write(text)
state["published"] = True
return
staged = None
Expand All @@ -617,13 +647,14 @@ def publish(data, output, force, state):
staged.unlink(missing_ok=True)


def main(argv=None):
def main(argv=None, *, clock=None):
parser = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
parser.add_argument("--archive", type=Path, help="Existing uncompressed OCI-layout or Docker-save TAR archive")
parser.add_argument("--sbom", action="append", default=[], type=Path, help="Existing CycloneDX 1.4-1.6 or SPDX 2.2/2.3 JSON; repeatable")
parser.add_argument("--consumer", choices=("canonical", "node2shell", "sharp", "flowise"), default="canonical", help="Optional offline consumer inventory; provenance remains unverified")
parser.add_argument("--max-expanded-mib", type=int, default=512, help="Global expanded-byte budget, 16-4096 MiB (default 512)")
parser.add_argument("--output", type=Path, help="New JSON report path; default stdout")
parser.add_argument("--receipt", type=Path, help="Optional canonical receipt for a consumer export, linked by its exact SHA-256")
parser.add_argument("--force", action="store_true", help="Replace an existing report, never a protected input")
parser.add_argument("--version", action="version", version=VERSION)
args = parser.parse_args(argv)
Expand All @@ -633,16 +664,36 @@ def main(argv=None):
parser.error("at most 100 SBOM files")
if not 16 <= args.max_expanded_mib <= 4096:
parser.error("--max-expanded-mib must be 16 through 4096")
if args.receipt and args.consumer == "canonical":
parser.error("--receipt accompanies --consumer node2shell, sharp or flowise")
report = dict(schema_version=SCHEMA_VERSION, tool={"name": "oci-runtime-inventory", "version": VERSION}, complete=True,
images=[], sboms=[], notes=["Offline package metadata only; no runtime, package provenance, patch state, absence or exploitability is established."],
exploitability="not_tested")
report["limits"] = dict(archive_bytes=MAX_ARCHIVE, expanded_bytes=args.max_expanded_mib * 1024**2, compressed_bytes=1024**3,
members=MAX_MEMBERS, capture_bytes=MAX_CAPTURE, images=MAX_IMAGES, layers_per_image=MAX_LAYERS)
budget = Budget(args.max_expanded_mib * 1024**2)
state = {"validated": False, "published": False, "stdout_started": False, "publication_started": False}
state = {"validated": False, "published": False, "stdout_started": False,
"publication_started": False, "pair_started": False}
code = 0
def publish_report():
report["usage"] = {name: getattr(budget, name) for name in ["raw", "expanded", "members", "capture", "work"]}
report["content_sha256"] = content_identity(report)
exported = export_inventory(report, args.consumer)
if args.receipt:
receipt = dict(schema_version="1.0", consumer=args.consumer,
export_sha256=hashlib.sha256(serialized(exported).encode("utf-8")).hexdigest(), inventory=report)
Comment thread
CompleteDotTech marked this conversation as resolved.
Resolved
# Either artifact may survive interruption; never retry a partially published pair.
state["pair_started"] = True
publish(receipt, args.receipt, args.force, {"published": False, "publication_started": False})
publish(exported, args.output, args.force, state)
try:
validate_output(args.output, ([args.archive] if args.archive else []) + args.sbom, args.force)
validate_output(args.receipt, ([args.archive] if args.archive else []) + args.sbom, args.force)
if args.output and args.receipt:
validate_output(args.receipt, [args.output], args.force)
generated_at, executable_sha256 = generation_context(clock)
report["generated_at"] = generated_at
report["tool"]["executable_sha256"] = executable_sha256
state["validated"] = True
try:
if args.archive:
Expand All @@ -655,15 +706,14 @@ def main(argv=None):
except (OSError, EOFError, ValueError, zlib.error):
report["complete"] = False
report["notes"].append("input_invalid_or_unavailable")
report["usage"] = {name: getattr(budget, name) for name in ["raw", "expanded", "members", "capture", "work"]}
code = 0 if report["complete"] else 2
publish(export_inventory(report, args.consumer), args.output, args.force, state)
publish_report()
except KeyboardInterrupt:
if state["validated"] and not any(state.get(key) for key in ["published", "stdout_started", "publication_started"]):
if state["validated"] and not any(state.get(key) for key in ["published", "stdout_started", "publication_started", "pair_started"]):
report["complete"] = False
report["notes"].append("interrupted_completed_images_only")
try:
publish(export_inventory(report, args.consumer), args.output, args.force, state)
publish_report()
except (OSError, ValueError, KeyboardInterrupt):
pass
sys.stderr.write("Interrupted; output may be incomplete or publication indeterminate; no publication retry.\n")
Expand Down
Loading
Loading