Claim Drift

AI-polished drafts can get stronger without getting truer. Claim Drift is a small local instrument for reviewing how public claims change between versions.

The category is claim-drift review.

Release Status

SOURCE_STATUS: PUBLIC_PACKAGE ACCESS_STATUS: CLEARED_FOR_EXTERNAL_USE UNIT27_POSITION: ADJACENT_CLAIM_REVIEW_UTILITY

This repository is a Unit27 public utility: visible, inspectable, and intended for orientation, testing, and practical use. Controlled protocol materials remain outside this source package.

It answers one narrow question:

Did the claim become more confident, broader, or more proof-like without matching evidence?

Two Failure Modes

Uncertainty laundering is when uncertain, inferred, incomplete, or weakly supported information becomes confident language.

Proofwashing is when thin evidence, such as a screenshot, demo, checklist, one run, or self-attested note, is presented as if it proves a much broader claim.

Claim Drift focuses on how those risks can appear across versions: a cautious draft becomes a stronger public claim even though the evidence record did not move with it.

What Claim Drift Does

Claim Drift compares a before draft and an after draft, checks optional structured evidence notes, and produces a review-only table.

It is designed to help surface:

• confidence_escalation
• scope_expansion
• proof_language_added
• evidence_gap
• scope_mismatch
• possible proofwashing
• suggested bounded wording for manual review
• next verification steps

The working rule is simple: proof before claim, boundary before scale, review before release.

What It Does Not Do

Claim Drift is not a verifier, fact-checker, fraud detector, certification system, compliance system, legal reviewer, medical safety tool, or truth oracle.

It does not prove either draft is true or false. It does not browse the web, inspect external sources, validate artifacts, certify evidence, audit code, or decide what you should publish.

It is a heuristic review aid. The output is a prompt for human judgment, not a final authority.

Where It Fits

Claim Drift sits beside Humility Engine as an adjacent claim-review utility.

• Humility Engine asks: "Does this draft overstate the evidence?"
• Claim Drift asks: "Did this claim become stronger between drafts?"

It is also distinct from Boundary Engine. Boundary Engine checks public repository language against recorded proof artifacts. Claim Drift compares two drafts and highlights when the later language appears to outrun the earlier evidence boundary.

Who It Is For

• builders reviewing portfolio or project claims
• researchers and operators preserving evidence boundaries in public claims
• teams comparing release copy across draft versions
• anyone trying to catch when a demo, artifact, or hunch becomes language that sounds proven

Quick Demo

Run the synthetic portfolio scenario:

python3 run.py examples/scenarios/portfolio_before.md examples/scenarios/portfolio_after.md \
  --evidence examples/scenarios/portfolio_evidence.md \
  --output examples/reviews/portfolio_drift_review.md \
  --review-only

The output is a drift-review table with source lines, evidence notes, risk flags, suggested bounded wording, and next verification steps.

Before / After Example

Earlier draft:

The platform demo runs on sample prompts.

Later draft:

The platform is production-ready and proven for teams.

Evidence available:

Local demo run on synthetic prompts, no production deployment, no external users, no independent evaluation.

Claim Drift review result:

A local demo run on synthetic prompts is supported by the available evidence. This does not establish production or deployment readiness.

Synthetic Demo Scenarios

The repo includes three synthetic scenarios:

• examples/scenarios/portfolio_before.md
• examples/scenarios/portfolio_after.md
• examples/scenarios/org_readiness_before.md
• examples/scenarios/org_readiness_after.md
• examples/scenarios/medical_safety_before.md
• examples/scenarios/medical_safety_after.md

Generated review-only outputs live in:

• examples/reviews/portfolio_drift_review.md
• examples/reviews/org_readiness_drift_review.md
• examples/reviews/medical_safety_drift_review.md

Run Your Own Draft Pair

Requirements: Python 3. No package install is required.

python3 run.py before.md after.md --evidence evidence.md --output review.md --review-only

The tool does not rewrite either source file. Read the drift table and revise manually.

Structured Evidence Notes

Evidence notes use a small structured Markdown format:

## Evidence: local demo
- evidence strength: artifact-backed
- evidence type: local output
- scope supported: local demo run on sample prompts
- limitations: no production deployment; no external users
- corroboration status: none

Supported evidence-strength values:

• unsupported
• self-attested
• artifact-backed
• externally corroborated
• live-demonstrable

The scope fields matter. For example, an artifact-backed local demo can support "a local demo ran on sample prompts," but it does not establish "production-ready and proven for teams."

Drift Table Schema

field	meaning
before line	Source line of the closest prior claim.
after line	Source line of the changed/new claim.
before claim	Closest claim from the earlier draft.
after claim	Claim from the later draft.
drift type	Confidence, scope, proof-language, or wording movement.
risk flag	Possible proofwashing, evidence gap, scope mismatch, weak evidence, or none.
evidence note	Best matching structured evidence note.
suggested bounded wording	Narrower claim language for manual review.
next verification step	What evidence or wording change should happen next.

Current Limits

• Claim extraction is sentence-based and may miss claims spread across sections.
• Review-only output is the only supported output shape.
• The extractor skips common Markdown structures, but it is still a heuristic parser.
• Before/after claim matching is keyword-based and can pair the wrong claims.
• Evidence matching is keyword-based and can choose the wrong note.
• Evidence strength is read from the evidence notes; the tool does not validate artifacts.
• The tool does not inspect files, URLs, screenshots, logs, demos, or external sources.
• Suggested bounded wording is for manual editing, not automatic source replacement.

Reliability

CI verifies the unit tests, Python compile check, and three review-only demo runs before changes are considered ready.

Verify

python3 -m unittest discover -s tests
PYTHONPYCACHEPREFIX=/private/tmp/claim_drift_pycache python3 -m py_compile claim_drift.py run.py

License

MIT