Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
370d464
Healthcare recognizers
bhargavikalicheti Jul 12, 2026
a5ecee2
Merge branch 'main' into feature/us-healthcare-recognizer
SharonHart Jul 15, 2026
c67e7a3
fix(analyzer): use thresholds for healthcare recognizers
bhargavikalicheti Jul 22, 2026
589c043
docs(analyzer): cite healthcare identifier sources
bhargavikalicheti Jul 22, 2026
36d96de
Merge branch 'main' into feature/us-healthcare-recognizer
omri374 Jul 26, 2026
5c3640b
Merge branch 'main' into feature/us-healthcare-recognizer
omri374 Aug 4, 2026
cef2f77
Update CHANGELOG with new recognizers and features
omri374 Aug 4, 2026
d3ae87c
docs(analyzer): clarify member ID references
bhargavikalicheti Aug 10, 2026
522e3ec
fix(analyzer): lower member ID base confidence
bhargavikalicheti Aug 11, 2026
7ebf31d
fix(analyzer): lower member ID base confidence
bhargavikalicheti Aug 11, 2026
2aa4fb2
fix(analyzer): anchor healthcare IDs on labels
bhargavikalicheti Aug 16, 2026
9fa1d57
fix(analyzer): validate provider EIN prefixes
bhargavikalicheti Aug 16, 2026
2ed83f3
test(analyzer): use approximate context score
bhargavikalicheti Aug 16, 2026
8ed30de
test(analyzer): cover healthcare ID edge cases
bhargavikalicheti Aug 16, 2026
d2fb76b
test(analyzer): use spaCy for healthcare context
bhargavikalicheti Aug 16, 2026
a41f7e7
refactor(analyzer): flatten healthcare recognizers
bhargavikalicheti Aug 16, 2026
f2d4dad
docs(analyzer): clarify healthcare detection
bhargavikalicheti Aug 16, 2026
68c775b
Merge branch 'main' into feature/us-healthcare-recognizer
bhargavikalicheti Aug 17, 2026
b52181a
Updating the yaml with changes that couldve lost during merge
bhargavikalicheti Aug 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ All notable changes to this project will be documented in this file.

### Analyzer
#### Added
- Added a disabled-by-default US health insurance member ID (`US_HEALTH_INSURANCE_MEMBER_ID`) recognizer requiring healthcare or insurance context.
- Added disabled-by-default US healthcare administrative ID recognizers for claim numbers, prior authorization numbers, prescription numbers, provider tax IDs, and referral numbers.
- Added `UuidRecognizer` (generic, entity type `UUID`) to detect UUIDs in the standard 8-4-4-4-12 hyphenated hexadecimal format, covering RFC 4122 versions 1-5 and RFC 9562 versions 6-8. Validates version and variant nibbles and filters the nil UUID to reduce false positives.
- South African ID number (`ZA_ID_NUMBER`) recognizer for the 13-digit national identity number, using pattern matching, context words, birth-date validation, and Luhn checksum validation. Disabled by default.
- South African recognizers for `ZA_PASSPORT`, `ZA_INCOME_TAX_NUMBER`, `ZA_DRIVER_LICENSE`, `ZA_VAT_NUMBER`, `ZA_COMPANY_REGISTRATION`, `ZA_TRAFFIC_REGISTER_NUMBER`, `ZA_LICENSE_PLATE`, `ZA_MOBILE_NUMBER`, and `ZA_TELEPHONE_NUMBER`. All disabled by default.
Expand Down
6 changes: 6 additions & 0 deletions docs/supported_entities.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,15 @@ For more information, refer to the [adding new recognizers documentation](analyz
|US_BANK_NUMBER|A US bank account number is between 8 to 17 digits.|Pattern match and context|
|US_DRIVER_LICENSE|A US driver license according to <https://ntsi.com/drivers-license-format/>|Pattern match and context|
|US_ITIN | US Individual Taxpayer Identification Number (ITIN). Nine digits that start with a "9" and contain a "7" or "8" as the 4 digit.|Pattern match and context|
|US_CLAIM_NUMBER|A US healthcare claim identifier used in billing and claims processing.|Pattern match, context enhancement, and entity threshold|
|US_HEALTH_INSURANCE_MEMBER_ID|A US health insurance member or subscriber identifier printed on an insurance card. Healthcare or insurance context increases detection confidence.|Pattern match, context enhancement, and entity threshold|
|US_MBI|A US Medicare Beneficiary Identifier (MBI) with 11 alphanumeric characters.|Pattern match and context|
|US_NPI|A US National Provider Identifier (NPI) is a 10-digit number issued to healthcare providers by CMS under HIPAA.|Pattern match, context and checksum|
|US_PASSPORT |A US passport number with 9 digits.|Pattern match and context|
|US_PRESCRIPTION_NUMBER|A US prescription or pharmacy order identifier.|Pattern match, context enhancement, and entity threshold|
|US_PRIOR_AUTHORIZATION_NUMBER|A US prior authorization identifier used for treatment or drug approval requests.|Pattern match, context enhancement, and entity threshold|
|US_PROVIDER_TAX_ID|A US provider organization tax identifier (TIN/EIN) used in healthcare billing workflows.|Pattern match, context enhancement, and entity threshold|
|US_REFERRAL_NUMBER|A US healthcare referral identifier, including specialty or infusion referral numbers.|Pattern match, context enhancement, and entity threshold|
|US_SSN|A US Social Security Number (SSN) with 9 digits.|Pattern match and context|

### UK
Expand Down
42 changes: 42 additions & 0 deletions presidio-analyzer/presidio_analyzer/conf/default_recognizers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,48 @@ recognizers:
enabled: false
country_code: us

- name: UsHealthInsuranceMemberIdRecognizer

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enabling these recognizers via YAML silently wipes their score thresholds (applies to all six new entries)

Enabling any of these six recognizers via this config which is the only supported path, since they ship enabled: false — erases the score thresholds the constructors set. RecognizerListLoader.get() runs recognizer.score_thresholds = normalize_score_thresholds(conf.get("score_thresholds")) (recognizers_loader_utils.py:435), which turns the absent key into {} and overwrites the {'US_CLAIM_NUMBER': 0.6}-style defaults from __init__.

Verified with a real AnalyzerEngine: after flipping enabled: true, "Tracking number CLM456789123 is active"US_CLAIM_NUMBER at 0.1, and "v2patch10build7"US_HEALTH_INSURANCE_MEMBER_ID at 0.1 — exactly the false positives the PR's tests assert are suppressed (those tests use direct instantiation + add_recognizer, so they never hit this).

Two possible fixes: declare score_thresholds explicitly on these six yaml entries, or make the loader only assign when the conf actually provides thresholds. The second also fixes this for user-supplied configs that omit the key.

supported_languages:
- en
type: predefined
enabled: false
country_code: us

- name: UsPriorAuthorizationNumberRecognizer
supported_languages:
- en
type: predefined
enabled: false
country_code: us

- name: UsClaimNumberRecognizer
supported_languages:
- en
type: predefined
enabled: false
country_code: us

- name: UsPrescriptionNumberRecognizer
supported_languages:
- en
type: predefined
enabled: false
country_code: us

- name: UsReferralNumberRecognizer
supported_languages:
- en
type: predefined
enabled: false
country_code: us

- name: UsProviderTaxIdRecognizer
supported_languages:
- en
type: predefined
enabled: false
country_code: us

- name: NhsRecognizer
supported_languages:
- en
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,16 @@
from .country_specific.us.medical_license_recognizer import MedicalLicenseRecognizer
from .country_specific.us.us_bank_recognizer import UsBankRecognizer
from .country_specific.us.us_driver_license_recognizer import UsLicenseRecognizer
from .country_specific.us.us_health_insurance_member_id_recognizer import (
UsHealthInsuranceMemberIdRecognizer,
)
from .country_specific.us.us_healthcare_admin_recognizers import (
UsClaimNumberRecognizer,
UsPrescriptionNumberRecognizer,
UsPriorAuthorizationNumberRecognizer,
UsProviderTaxIdRecognizer,
UsReferralNumberRecognizer,
)
from .country_specific.us.us_itin_recognizer import UsItinRecognizer
from .country_specific.us.us_mbi_recognizer import UsMbiRecognizer
from .country_specific.us.us_npi_recognizer import UsNpiRecognizer
Expand Down Expand Up @@ -229,11 +239,17 @@
"SgFinRecognizer",
"UrlRecognizer",
"UsBankRecognizer",
"UsClaimNumberRecognizer",
"UsHealthInsuranceMemberIdRecognizer",
"UsItinRecognizer",
"UsLicenseRecognizer",
"UsMbiRecognizer",
"UsNpiRecognizer",
"UsPassportRecognizer",
"UsPrescriptionNumberRecognizer",
"UsPriorAuthorizationNumberRecognizer",
"UsProviderTaxIdRecognizer",
"UsReferralNumberRecognizer",
"UsSsnRecognizer",
"EsNifRecognizer",
"SpacyRecognizer",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@
from .medical_license_recognizer import MedicalLicenseRecognizer
from .us_bank_recognizer import UsBankRecognizer
from .us_driver_license_recognizer import UsLicenseRecognizer
from .us_health_insurance_member_id_recognizer import (
UsHealthInsuranceMemberIdRecognizer,
)
from .us_healthcare_admin_recognizers import (
UsClaimNumberRecognizer,
UsPrescriptionNumberRecognizer,
UsPriorAuthorizationNumberRecognizer,
UsProviderTaxIdRecognizer,
UsReferralNumberRecognizer,
)
from .us_itin_recognizer import UsItinRecognizer
from .us_mbi_recognizer import UsMbiRecognizer
from .us_npi_recognizer import UsNpiRecognizer
Expand All @@ -15,9 +25,15 @@
"UsItinRecognizer",
"UsBankRecognizer",
"UsLicenseRecognizer",
"UsClaimNumberRecognizer",
"UsHealthInsuranceMemberIdRecognizer",
"UsMbiRecognizer",
"UsNpiRecognizer",
"UsPassportRecognizer",
"UsPrescriptionNumberRecognizer",
"UsPriorAuthorizationNumberRecognizer",
"UsProviderTaxIdRecognizer",
"UsReferralNumberRecognizer",
"AbaRoutingRecognizer",
"UsSsnRecognizer",
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
"""Recognizer for US health insurance member identifiers."""

from typing import Dict, List, Optional

from presidio_analyzer import Pattern, PatternRecognizer


class UsHealthInsuranceMemberIdRecognizer(PatternRecognizer):
"""Recognize US health insurance member/subscriber IDs with context.

US health insurance member identifiers are payer-specific and do not have a
single universal checksum or format. To avoid broad matching of generic
alphanumeric IDs, this recognizer requires both:
- a plausible alphanumeric member ID pattern, and
- nearby healthcare/insurance context.

CMS consumer guidance explicitly labels the payer-assigned member number on
a sample insurance card. Medicaid T-MSIS defines MEMBER-ID as the value shown
on the insurance carrier's card and permits up to 20 characters. These
sources establish the identifier and upper bound, not a universal syntax;
the default regex is therefore a conservative, replaceable heuristic.
Presidio applies ``re.IGNORECASE`` through its default global regex flags,
so the uppercase character classes also match lowercase and mixed-case IDs.

CMS card reference: https://www.cms.gov/files/document/11818-sample-insurance-card-english.pdf
Medicaid data reference: https://www.medicaid.gov/tmsis/dataguide/v4/data-elements/tpl003036/

:param patterns: List of patterns to be used by this recognizer
:param context: List of context words which increase detection confidence
:param supported_language: Language this recognizer supports
:param supported_entity: The entity this recognizer can detect
:param score_thresholds: Optional default and entity-specific score thresholds
"""

COUNTRY_CODE = "us"

PATTERNS = [
Pattern(
Comment thread
omri374 marked this conversation as resolved.
"Health insurance member ID (weak)",
r"\b(?=[A-Z0-9-]{6,20}\b)(?=[A-Z0-9-]*[A-Z])"
r"(?=[A-Z0-9-]*\d)[A-Z]{1,5}-?[A-Z0-9]{5,14}\b",
0.1,
),
]

CONTEXT = [
"member",
"subscriber",
"insurance",
"policy",
]

def __init__(
self,
patterns: Optional[List[Pattern]] = None,
context: Optional[List[str]] = None,
supported_language: str = "en",
supported_entity: str = "US_HEALTH_INSURANCE_MEMBER_ID",
name: Optional[str] = None,
score_thresholds: Optional[Dict[str, float]] = None,
):
patterns = patterns if patterns else self.PATTERNS
context = context if context else self.CONTEXT
super().__init__(
supported_entity=supported_entity,
patterns=patterns,
context=context,
supported_language=supported_language,
name=name,
)
self.score_thresholds = (
score_thresholds
if score_thresholds is not None
else {supported_entity: 0.4}
)
Loading
Loading