Skip to content

Convert versioned checker reports to private SARIF and NDJSON - #1

Merged
CompleteDotTech merged 6 commits into
mainfrom
feat/offline-conversion
Oct 26, 2026
Merged

CompleteDotTech merged 6 commits into
mainfrom
feat/offline-conversion

Conversation

@CompleteDotTech

@CompleteDotTech CompleteDotTech commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Add an optional standalone offline converter for canonical v1.0 reports and six explicitly versioned legacy checker profiles. Produce SARIF 2.1.0 or NDJSON with stable observation fingerprints, conflict-preserving deduplication, counts and completion state. Private strings are projected to hashes by default; relative source locations require opt-in. Arbitrary tool version labels are rejected, and canonical limitations/snapshots contribute to evidence identity.

Validation: 13 local stdlib tests pass; five SARIF variants pass the vendored official OASIS schema. Actual synthetic outputs from the supported checker CLIs were accepted. Copilot hosted findings were addressed, followed by an independent GitHub Copilot CLI review of 0c8e2c5 with no actionable bridge finding. No targets, uploads or runtime assessment. This supports node2shell-check issue #14; each checker remains independently usable.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved conversion issues affect JSON validity, preservation of distinct assessments, stable fingerprints, canonical identifier acceptance, and per-tool completion reporting.

5 open issues
What changed in this PR

Adds an optional, standalone offline bridge for the checker family, converting existing versioned reports into privacy-projected SARIF or NDJSON without adding checker dependencies or uploads.

Changes:

  • Bundles canonical validation and six explicit legacy adapters.
  • Adds privacy projection, deterministic fingerprints, deduplication, protected output publication, and policy exit codes.
  • Adds synthetic tests, the official SARIF schema, documentation, and cross-platform CI.
File Description
tools/​validate_sarif.py Validates synthetic SARIF against the bundled schema.
test_bridge.py Tests conversion, privacy, validation, and output protection.
SHA256SUMS Records the standalone script checksum.
schemas/​sarif-schema-2.1.0.json Bundles the official SARIF schema.
schemas/​OASIS-LICENSE.md Includes upstream schema terms.
scan_results_bridge.py Implements validation, adapters, conversion, and CLI output.
README.md Documents usage, privacy, bounds, and exit policies.
LICENSING.md Clarifies separate upstream schema licensing.
examples/​report.json Provides a synthetic canonical report.
BRAND_ASSETS.md Clarifies trademark and logo permissions.
AGENTS.md Updates repository guidance for offline conversion.
.gitignore Excludes Python caches and virtual environments.
.github/​workflows/​tests.yml Adds cross-platform tests and optional schema validation.

Review effort: Balanced

Comment thread scan_results_bridge.py Resolved
Comment thread scan_results_bridge.py Resolved
Comment thread scan_results_bridge.py Resolved
Comment thread scan_results_bridge.py Resolved
Comment thread scan_results_bridge.py Resolved
Copilot stopped reviewing on behalf of CompleteDotTech due to an error October 25, 2026 23:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Unresolved privacy and conversion correctness issues require correction and human confirmation, with exact-head hosted CI still unverified.

No open issues 🎉

3 previously missed issues

In code that hasn't changed since last review

Medium severity Protect free-form tool versions and completion metadata from disclosure

scan_results_bridge.py:654

The canonical contract allows arbitrary text in tool.version, so a value such as build-token=SYNTHETIC_SECRET bypasses privacy projection and appears verbatim in both default output formats. Hash free-form tool versions or restrict them to an explicitly public format before exporting them. Apply the same handling to the completion keys at lines 765–766, which also populate SARIF tool versions for empty reports.

Medium severity Deduplicate evidence locations before determining unique source position

scan_results_bridge.py:691

Distinct evidence IDs can legitimately refer to the same source path and line. With two such entries, this list has length 2, so the converter drops the unambiguous location and changes the fingerprint even though the asset, component and source position are unchanged. Deduplicate path/line pairs before checking for a single location, and add a regression case with two evidence entries at the same position.

Medium severity Set explicit SARIF evaluation kind for non-finding observations

scan_results_bridge.py:726

Omitting kind makes every result inherit SARIF's default fail, including unknown, fixed_version_reported, and mitigated observations. Setting level to note only changes severity; SARIF consumers still see these as failed rule evaluations unless they understand the custom status property. Set an explicit evaluation kind for non-finding observations, such as review for uncertainty or informational with a compatible level for metadata-only results.

5 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Unresolved privacy and observation-identity issues can expose private metadata or lose meaningful report distinctions.

No open issues 🎉

3 previously missed issues

In code that hasn't changed since last review

Medium severity Prevent Private Tool Versions from Leaking into Published Metadata

scan_results_bridge.py:665

The canonical contract allows arbitrary text in tool.version, but this value bypasses the privacy projection. A valid report whose version is https://private.invalid/?token=SYNTHETIC_SECRET exits successfully and publishes that URL/token unchanged in SARIF driver metadata and NDJSON result rows; even an empty SARIF report exposes it through the completion keys. Validate published versions against an explicitly public format or privacy-project free-form versions consistently in both normalized rows and run/completion metadata before serialization.

Medium severity Deduplicate Evidence Locations Before Determining Location Ambiguity

scan_results_bridge.py:702

Two valid source-evidence entries with different evidence IDs can refer to the same path and line. Counting entries rather than distinct locations treats this unambiguous case as locationless: SARIF omits the location even with path publication enabled, and adding corroborating evidence changes the fingerprint despite the source location remaining unchanged. Accept the location when all valid entries refer to the same path/line.

Medium severity Include Report Limitations in Canonical Evidence Digest

scan_results_bridge.py:717

Report-level limitations are omitted from canonical evidence and its digest. Consequently, complete reports with identical results and snapshots but different coverage limitations normalize identically and are merged as exact duplicates, losing the distinction between their assessment constraints. Include the limitations before privacy projection and hashing so they remain privately represented and observations with different limitations stay distinct.

Review effort: Balanced

Copy link
Copy Markdown
Contributor Author

Final gate: exact head 0c8e2c5 has SUCCESS hosted CI across Linux, Windows and macOS. GitHub Copilot CLI independently reviewed its complete source/test/doc/workflow diff (excluding the vendored SARIF schema), with no actionable bridge finding. The provenance URL was independently checked: CompleTetech-Security is the actual organization. All 13 local tests, five official-schema SARIF probes and diff/checksum validation pass. Privacy hashes are linkable/guessable and numbers remain as documented; no anonymity or production assessment is claimed.

@CompleteDotTech
CompleteDotTech merged commit a9bd1b7 into main Oct 26, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants