Skip to content

[Product Gap] Ship a source-authoritative detector vertical slice with independent efficacy evidence #938

Description

@seonghobae

Buyer-visible problem

AppGuardrail can inventory and classify evidence supplied by callers, but the current issue-detection foundation does not yet prove that AppGuardrail itself acquires source-authoritative evidence and directly detects the underlying control failure. A buyer cannot distinguish a real detector from a typed wrapper around a Boolean, list, log regex, or signed upstream result.

Scope

Implement one bounded end-to-end detector vertical slice on current develop, using the historical #815/#813/#763 cases as candidate real-world replays. Select the highest-leverage case whose source system and expected outcome can be reproduced without privileged customer data.

The slice must bind:

atomic cause → control obligation → trusted probe/acquirer → independently acquired source identity → typed assessment → independent oracle → persisted evidence → control-plane/API result

Required contracts

  • Add explicit probe_ref and acquirer_ref identifiers; no caller-provided pass/fail value may substitute for acquisition.
  • Record repository/source artifact identity with SHA-256 and relevant revision metadata.
  • Fail closed for unknown detector family, missing source identity, malformed source evidence, ambiguous multi-cause ordering, duplicate evidence, and stale evidence.
  • Keep legacy envelopes readable only through a documented compatibility boundary; new production evidence must use the canonical source-bound schema.
  • Integrate the detector through the same production path used by CLI/control-plane/API consumers.
  • Preserve PII for authorized work rather than indiscriminate masking: use tenant isolation, least-privilege access, encryption, immutable audit, purpose binding, retention policy, and field-level authorization.

Test-first acceptance criteria

  1. A RED test proves the pre-change product can accept or classify caller assertions without acquiring the authoritative source.
  2. A real source fixture and an independently authored oracle prove true-positive, true-negative, malformed, unavailable, stale, duplicate, and adversarial cases.
  3. Production-predicate mutation tests fail when the source acquisition, obligation binding, identity verification, or outcome mapping is inverted or bypassed.
  4. The exact production entrypoint is exercised in a black-box test and a live exact-head audit.
  5. Production statement and branch coverage remain 100%; all public modules, classes, functions, methods, and error types have complete docstrings.
  6. No ignored/skipped efficacy or GPU tests are used to claim completion.
  7. ADR, ARCHITECTURE, threat model, test strategy, operability/runbook, requirement traceability, and CHANGELOG are updated with APA 7th references to the governing standards.
  8. Exact-head Tests, appguardrail-scan, SAST, CodeQL, dependency review, supply-chain checks, and independent review pass before merge.

Source preservation

PR #911 and branch agent/issue-detection-contract preserve the prior inventory/documentation prototype. Reuse only reviewed, current, minimal pieces; do not copy the 20k-line draft wholesale or treat its registry-generated fixtures as an independent oracle.

Exit condition

A buyer can inspect one detector and verify, from immutable evidence and reproducible tests, that AppGuardrail—not a caller assertion—observed the authoritative source condition and produced the decision.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: authAuthentication, authorization, identity, or tenant isolationarea: ci-cdCI, GitHub Actions, checks, release, or supply chainarea: dependenciesDependency or lockfile maintenancearea: securitySecurity boundary, hardening, or vulnerability preventionmaintenancepriority: mediumNormal-priority or P2 workscope: product-gapCustomer-visible product gapstatus: triagedOpen issue has an organization taxonomy assignmenttype: featureNew or expanded product capability

    Type

    No type

    Projects

    • Status
      In progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions