Skip to content

RFC: a generic scanner slot, so a fourth verdict source does not need a schema migration #3783

Description

@MoltyCel

Area

Security / trust

Context

Scanner reports are declared as individually named optional fields — skillSpectorAnalysis, aigAnalysis, llmAnalysis — repeated across four artifact tables in convex/schema.ts, then read back by name in convex/lib/scannerReports.ts and rolled up in convex/lib/securityScanPolicy.ts.

Adding a fourth source therefore touches four table declarations, the reader, the dispatcher, the rollup and the audit UI. The cost is not in the analysis; it is in the shape of the storage. That cost falls on every future source equally, which is the part worth fixing once.

Disclosure up front: we are a self-interested party proposing a slot we would like to fill. The RFC is written so that it stands or falls on the generic mechanism, not on who fills it.

Goals

  • Let a new verdict source register without a schema migration.
  • Keep the existing per-scanner report shape, so no stored report needs rewriting.
  • Leave the artifact sha256 binding and the evidence-never-verdict rule exactly as they are.

Non-goals

  • No thresholds, evasion indicators or vendor-private signals are discussed here.
  • No exception to the blocking-verdict rule is requested. Evidence into the rollup, never the verdict.
  • This does not decide which sources should be admitted, only how a source is stored once admitted.

What already works and should not change

Verdicts are keyed by artifact sha256, so a report is bound to the exact bytes that were published rather than to a name or a version string. That binding is the reason external evidence can be trusted at all here, and nothing in this proposal alters it.

The authority model in docs/security-audits.md should stay unchanged:

Skill audits include findings from Tencent Zhuque Lab's A.I.G scanner. A.I.G checks agent instructions for vulnerability patterns and supplies supporting evidence to ClawScan's artifact-wide review. It does not issue ClawHub's final verdict or independently block installation.

Evidence into the rollup, never the blocking verdict. This proposal keeps that rule and asks for no exception to it.

Proposal

Replace the named fields with one map, keeping the existing per-scanner validator shapes as the value type:

scanners: v.optional(v.record(v.string(), scannerReportValidator))

scannerReportValidator is the shape aigAnalysisValidator already defines — status, issueCount, findings[], scannerVersion?, summary?, error?, checkedAt — so no existing report needs rewriting. A migration writes the three current fields into scanners under their present keys and leaves the old fields readable for one release.

A source then registers a key, a display name and a trust tier. The rollup reads the map instead of three names.

Examples

What this is for, stated plainly. MolTrust runs a skill auditor that would register as one such source. It differs from a linter in one respect that is relevant here: it binds a verdict to a signed, on-chain-anchored W3C Verifiable Credential over a canonical skill hash, so a third party can recompute the claim without asking us. A track record only its issuer can compute is a reputation service; one any party can recompute is evidence.

We are proposing the slot, not asking to be privileged in it. If the maintainers conclude the map is not worth the migration, that answer is usable and we will implement against the named-field pattern instead.

User impact

  • Maintainers: one migration now instead of one per future source; the rollup reads a map rather than a growing list of names.
  • Authors and users: no visible change. The audit UI renders whatever keys are present.
  • External contributors: a source can be added without touching four table declarations, which lowers the bar for a contribution that is mostly analysis rather than schema work.
  • Existing reports: unchanged. The three current fields migrate under their present keys and stay readable for one release.

Open questions

  1. Is the migration worth doing now, or should a fourth source follow the existing named-field pattern and the map wait for a fifth?
  2. Should a registering source declare a trust tier, or is a flat "evidence, never verdict" rule for all non-first-party sources simpler and safer?
  3. What should the UI show when two sources disagree? The A.I.G precedent covers agreement; disagreement between an external attestation and a static scan is arguably itself the useful signal.
  4. Is there an appetite for requiring external sources to publish their check list and version, so a verdict can be reproduced later? We publish ours and would be glad to see it required of everyone.

Feedback deadline

2026-10-02

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low-priority cleanup, docs, polish, ergonomics, or speculative work.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:securitySecurity boundary, credential, authz, sandbox, or sensitive-data risk.issue-rating: 🌊 off-meta tidepoolIssue quality rating does not apply to this item.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions