PII detection with ProgramAsWeights

Detect and type personally identifiable information locally with one compiled ProgramAsWeights neural program.

Try the live demo

Python

pip install programasweights --extra-index-url https://pypi.programasweights.com/simple/

import json
import programasweights as paw

detect_pii = paw.function("73a0e38b8bbe3427cd1d")

text = "Name: Ada Lovelace\nEmail: ada@example.com\nPIN: 4821"
print(json.loads(detect_pii(text)))

[['Ada Lovelace', 'private_person'], ['ada@example.com', 'private_email'], ['4821', 'secret']]

The program downloads once and then runs locally. See example.py for the complete runnable example.

Output

The function returns a JSON array of [text, type] pairs. Each text is copied exactly from the input, and each type is one of:

private_person  private_email  private_phone  private_address  private_url
private_date    account_number secret         other_pii

Compile it yourself

The public program was compiled from the English specification in spec.txt. compile.py contains the complete compilation script:

from pathlib import Path
import programasweights as paw

program = paw.compile(
    Path("spec.txt").read_text().strip(),
    compiler="paw-ft-bs48",
    public=True,
)

print(program.id)

The published compilation used by this repository is:

73a0e38b8bbe3427cd1d

Compiling requires a PAW account and API key. Loading the published program by ID does not require recompiling it.

Results

On an untouched multilingual set of 512 AI4Privacy documents, the published program reached:

• 0.8464 typed-character F1
• 0.9085 label-agnostic extraction F1
• 93.9% type accuracy on characters where the prediction and annotation overlap

We compared compact JSON, reversed JSON, JSON objects, and TSV outputs on a separate 497-document search set, froze the finalists, and selected the published program on another 522 documents before opening the sealed set. All source groups are disjoint. The original compact [text, type] JSON interface won; simplifying its specification did not generalize.

See RESULTS.md for every specification, compiled program, failed run, and benchmark result.

Reproduce the benchmark

The benchmark implementation is under scripts/, the parser and metrics are under src/paw_pii/, and tests are under tests/. Downloaded datasets, model checkpoints, and generated prediction files are intentionally excluded from Git.

PYTHONPATH=src python -m pytest -q

License

MIT