Compliance PoC — PII redaction & data retention (#275)

[illeatmyhat]

Presentation PoC for #275 (Compliance — PII, Retention). For demonstrating the merged hook seam, not a merge candidate — hence draft. This branch is main plus a small set of demo files; every claim is backed by runnable examples. Full walkthrough in DEMO.md.

What changed

The original PoC predated the memory hook seam and imported detectors directly (PIIConfig / NullRedactor / backend.redactor) — all since removed from main and replaced by the seam. This branch has been reset to main + demo files and now shows how to USE the merged seam (PR #295): plugins configured via evolve hooks init → evolve.hooks.yaml (auto-discovered) or code-first HooksConfig(plugins=[HookPluginSpec(...)]), firing at the backend read/write choke points and before every LLM call.

The two #275 asks, plus secrets — three redaction methods on one seam

1. PII must never be saved into memory — two PII methods on memory_pre_write + llm_pre_call:
   • Regex (PIIFilterMemoryPlugin, [pii-regex]) — structured identifiers (email, phone, SSN, card, IP) at precision ~1.00; no NER.
   • Semantic (ReadiSemanticPIIPlugin, [pii-semantic]) — IBM READI NER; catches the names regex cannot.
2. Data-retention policies — age/unused flag-or-delete + session retention that cascade-deletes derived memories via provenance, with a why on every decision. CLI: evolve retention run --policy <file> [--apply] (dry-run by default).
3. Structured secrets (adjacent to Compliance - PII, Retention, etc. #275) — SecretsFilterMemoryPlugin ([secrets]) redacts credentials/tokens (AWS keys, GitHub/Slack tokens, private keys) on write and on LLM egress.

Running all three together is the recommended defence-in-depth posture.

Runnable demos

uv sync --extra hooks --extra pii-regex --extra pii-semantic --extra secrets
uv run --no-sync python examples/pii_redaction_demo.py   # regex + semantic; name caught only by semantic
uv run --no-sync python examples/secrets_demo.py         # AWS key + GitHub token redacted (write + egress)
uv run --no-sync python examples/hooks_demo.py           # normalizer + access-stamp + regex PII together
uv run --no-sync python examples/retention_demo.py       # flag / delete / provenance cascade, with "why"

Measured results (retained — they measure detector/model behavior)

English — ai4privacy, 43,501 records — per-entity recall:

PII type	regex ([pii-regex])	semantic (READI, en_core_web_trf)
Overall recall	0.12	0.47
Precision	1.00	0.99
email / ip	1.00 / 1.00	1.00 / 0.95
phone / ssn / credit card	0.46 / 0.22 / 0.04	0.60 / 0.43 / 0.03
first / last / middle name	0.00	0.92 / 0.97 / 0.94

Japanese — ai4privacy openpii-1.5m, 2,000 records — a language-matched model (one config line) is decisive:

PII type	regex	English NER	Japanese NER (ja_core_news_trf)
Overall recall	0.03	0.15	0.92
given / surname	0.00 / 0.00	0.07 / 0.04	0.82 / 0.96

Regex is high-precision on well-formed structured PII but blind to names and non-Western formats; semantic NER adds names/locations (~4× recall on English) but is language-bound. Hybrid (regex + language-matched NER + secrets) is the production shape.

Reviewer note

Branch = main + demo files only. Diff vs main: examples/pii_redaction_demo.py, examples/secrets_demo.py, DEMO.md (all new). Everything the demos exercise (the seam, the three plugins, retention, hooks_demo.py / retention_demo.py / pii_benchmark.py) is already on main. macOS/Apple-Silicon caveat: READI fails closed on the MPS backend, so the semantic e2e path needs a CPU/CUDA host; pii_redaction_demo.py falls back to regex-only with a printed note on darwin. Reproduce, don't trust the prose — full walkthrough in DEMO.md.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8d4c1796-8101-4238-8b00-a47910edb987

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/275-pii-retention-poc

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[agent-toolkit-owl]

No vulnerabilities found.