You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
HuggingFaceNerRecognizer ships in default_recognizers.yaml with enabled: false and no model_name. Flipping that switch does not work: EntityRecognizer.__init__ calls load(), load() raised when model_name was unset, and the error therefore arrived during registry construction rather than at analyze time. The shipped entry advertised a toggle that always crashed.
The issue offers three acceptable fixes. I took the deferred-load one:
load() now returns early with an info-level log when model_name is unset, so the registry builds.
analyze() raises a ValueError naming what is missing and where to set it, including default_recognizers.yaml, so the failure surfaces at the point of use with an actionable message.
I deliberately did not give the shipped entry a default model_name. Committing the project to a specific HuggingFace model reference in shipped config is a product decision rather than a bug fix, and it would make the loader test download a model. Happy to switch to that if you would rather — it is a small change on top of this one.
Because the entry now loads, HuggingFaceNerRecognizer has been removed from the NOT_LOADABLE_FROM_SHIPPED_ENTRY exclusion in test_recognizers_loader_utils.py, which #2170 added with a guard test to keep the gap visible. That set is now empty, and the recognizer is covered by the normal load test like every other entry. I left the comment in place explaining why it is empty, so the next person to need it knows what it was for.
The existing test asserting the old ValueError from load() is updated to assert the new analyze-time error instead, and a test covers registry construction succeeding with the recognizer instantiated and unloaded.
presidio-analyzer/tests/test_huggingface_ner_recognizer.py and test_recognizers_loader_utils.py: 338 passed, 3 skipped. ruff check presidio_analyzer tests reports all checks passed.
Yours came first, so I am closing mine (#2228) rather than leaving two PRs on the same issue.
One thing from mine that might be worth folding in here. Leaving the recognizer inactive fixes the registry build, but a recognizer with no model then returns no entities from analyze(), and for a PII library an empty result reads as "this text is clean" rather than "this recognizer could not run". Raising there, with the same message load() logs, turns a silent gap into something the caller can see. Guarding on model_name rather than on a missing pipeline keeps lazy loading through load() behaving as it does today.
Feel free to take it or leave it, and thanks for getting to this one.
Disclosure: this comment and the PR I am closing were prepared with an AI coding assistant.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2222
HuggingFaceNerRecognizerships indefault_recognizers.yamlwithenabled: falseand nomodel_name. Flipping that switch does not work:EntityRecognizer.__init__callsload(),load()raised whenmodel_namewas unset, and the error therefore arrived during registry construction rather than at analyze time. The shipped entry advertised a toggle that always crashed.The issue offers three acceptable fixes. I took the deferred-load one:
load()now returns early with an info-level log whenmodel_nameis unset, so the registry builds.analyze()raises aValueErrornaming what is missing and where to set it, includingdefault_recognizers.yaml, so the failure surfaces at the point of use with an actionable message.I deliberately did not give the shipped entry a default
model_name. Committing the project to a specific HuggingFace model reference in shipped config is a product decision rather than a bug fix, and it would make the loader test download a model. Happy to switch to that if you would rather — it is a small change on top of this one.Because the entry now loads,
HuggingFaceNerRecognizerhas been removed from theNOT_LOADABLE_FROM_SHIPPED_ENTRYexclusion intest_recognizers_loader_utils.py, which #2170 added with a guard test to keep the gap visible. That set is now empty, and the recognizer is covered by the normal load test like every other entry. I left the comment in place explaining why it is empty, so the next person to need it knows what it was for.The existing test asserting the old
ValueErrorfromload()is updated to assert the new analyze-time error instead, and a test covers registry construction succeeding with the recognizer instantiated and unloaded.presidio-analyzer/tests/test_huggingface_ner_recognizer.pyandtest_recognizers_loader_utils.py: 338 passed, 3 skipped.ruff check presidio_analyzer testsreports all checks passed.