Caveat Drop

Claims can become misleading when their caveats disappear. Caveat Drop is a small local instrument for reviewing whether public claims dropped limitations from the evidence notes that were supposed to bound them.

The category is caveat-retention review: boundary before scale, review before release, and demo stays demo.

Release Status

SOURCE_STATUS: PUBLIC_PACKAGE ACCESS_STATUS: CLEARED_FOR_EXTERNAL_USE UNIT27_POSITION: ADJACENT_CLAIM_REVIEW_UTILITY

This repository is a Unit27 public utility: visible, inspectable, and intended for orientation, testing, and practical use. Controlled protocol materials remain outside this source package.

It answers one narrow question:

Did the public claim drop a limitation that should still be visible?

Failure Mode

Caveat drop happens when evidence notes include important boundaries, but the public claim omits them.

This is one way proofwashing happens: a local checklist, screenshot, toy demo, synthetic test, or one run may be real, but if the caveats disappear, the remaining claim can sound broader than the evidence supports.

What Caveat Drop Does

Caveat Drop reads:

• a Markdown claim document
• structured Markdown caveat/evidence notes

It produces a review-only table with:

• source line
• claim
• matched caveat note
• match basis
• evidence strength
• supported scope
• dropped caveats
• unsupported scope
• risk flags
• bounded wording
• next verification step

What It Does Not Do

Caveat Drop is not a verifier, fact-checker, fraud detector, certification system, compliance system, legal reviewer, medical safety tool, or truth oracle.

It does not prove that a claim is true or false. It does not inspect external sources, validate screenshots, audit code, certify evidence, or decide what you should publish.

It is a heuristic review aid. The output is a prompt for human judgment, not a final authority.

Where It Fits

Caveat Drop sits beside Humility Engine, Claim Drift, Proof Decay, and Evidence Floor as an adjacent claim-review utility.

• Humility Engine asks: "Does this claim outrun its evidence?"
• Claim Drift asks: "Did this claim become stronger between drafts?"
• Proof Decay asks: "Is old proof being reused as current proof?"
• Evidence Floor asks: "Does this claim meet the minimum evidence required for this claim class?"
• Caveat Drop asks: "Did the claim drop a caveat that should still bound it?"

Who It Is For

• builders reviewing public claims before release
• researchers and operators preserving evidence boundaries in public claims
• teams checking whether limitations survived the move from notes to public copy
• anyone trying to avoid turning a limited artifact into a broader proof claim by omission

Quick Demo

Install dependencies and run the synthetic demo scenarios:

npm install
npm run demo

Or run one scenario directly:

npm run review -- examples/scenarios/medical_safety_claim.md \
  --caveats examples/scenarios/medical_safety_caveats.md \
  --output examples/reviews/medical_safety_caveat_review.md \
  --review-only

The output is a review-only caveat-retention table. It does not rewrite the source document.

Before / After Example

Evidence note:

Local checklist and synthetic demo only. No clinical review, no patient data, no hospital validation.

Public claim:

The model is safe for hospitals.

Caveat Drop review result:

Local checklist on synthetic prompts is the supported scope. It should keep the missing caveat visible. This does not establish medical or clinical safety.

Synthetic Demo Scenarios

The repo includes three synthetic scenarios:

• examples/scenarios/portfolio_claim.md
• examples/scenarios/org_readiness_claim.md
• examples/scenarios/medical_safety_claim.md

Generated review-only outputs live in:

• examples/reviews/portfolio_caveat_review.md
• examples/reviews/org_readiness_caveat_review.md
• examples/reviews/medical_safety_caveat_review.md

Structured Caveat Notes

Caveat notes use a small structured Markdown format:

## Caveat: local checklist boundary
- evidence provided: local checklist and synthetic demo only
- evidence strength: artifact-backed
- evidence type: local checklist
- scope supported: local checklist on synthetic prompts
- required caveats: local checklist only; synthetic prompts only; no clinical review
- limitations: no clinical review; no patient data; no external validation
- unsupported scope: medical or clinical safety; hospital safety
- corroboration status: none
- severity: high

Supported evidence-strength values:

• unsupported
• self-attested
• artifact-backed
• externally corroborated
• live-demonstrable

Current Limits

• Claim extraction is sentence-based and may miss claims spread across sections.
• Review-only output is the only supported output shape.
• Caveat matching is keyword-based with an ordered fallback.
• Dropped-caveat detection uses simple phrase matching, not semantic verification.
• The tool does not inspect files, URLs, screenshots, logs, demos, or external sources.
• Suggested bounded wording is for manual editing, not automatic source replacement.

Reliability

CI verifies the unit tests and three review-only demo runs before changes are considered ready.

Verify

npm install
npm test
npm run demo
npm run build

License

MIT