Convert Hindi number words to digits and normalize Devanagari text from ASR, OCR, or copy-paste. Handles the encoding variants, punctuation quirks, and number-word spellings that break search, equality checks, tokenization, and downstream extraction. Zero runtime dependencies — pure standard library.
I built this because Sakhi — an offline medical Hindi voice-to-form tool meant for ASHAs— kept reinventing the same regex passes against Whisper output: number words that never became digits, ५ and 5 treated as different characters, | where a danda । belonged, the same word encoded two different ways. This is that pass, extracted and generalized.
pip install hindi-normalizefrom hindi_normalize import normalize_transcript
normalize_transcript("आपका BP एक सौ दस बटा सत्तर है, वजन अट्ठावन kg")
# 'आपका BP 110/70 है, वजन 58 kg'normalize_transcript runs the whole pipeline. The individual transforms are also exported and usable on their own.
Hindi number words → digits, tolerant of ASR spelling variants (पाँच/पांच/पाच, सत्तर/सतर). Adjacent unrelated numbers are not summed — दो तीन दिन ("two-three days") stays 2 3 दिन, never 5.
from hindi_normalize import convert_numbers, parse_hindi_number
convert_numbers("एक सौ दस बटा सत्तर") # '110 बटा 70'
parse_hindi_number("नौ सौ निन्यानवे") # 999normalize_devanagari folds the character-level variants OCR/ASR/copy-paste introduce, following the operations the Indic NLP Library established as standard.
from hindi_normalize import normalize_devanagari
normalize_devanagari("वजन ५८ किलो | ठीक है") # 'वजन 58 किलो। ठीक है'Default (lossless, on):
- Invisibles — strip ZWJ/ZWNJ/ZWSP, BOM, soft hyphen, word joiner; no-break space → space
- NFC — unify nukta encodings (
क़as one codepoint vsक+ nukta) and matra order - Digits — Devanagari digits (
५) → ASCII (5) - Danda —
|→।,||→॥, collapse runs, drop space before a danda
Opt-in (lossy, off):
remove_nukta—क़→क,ज़→जnasals— fold nasal clusters to anusvara (हिन्दी→हिंदी)chandrabindu—ँ→ंvisarga— ASCII colon after a Devanagari letter → visarga (दु:ख→दुःख)
Unlike a naive unicodedata-category filter, nothing here strips the Unicode Mark category, so matras and the virama survive — the failure mode that silently deletes vowel signs from Indic text.
Map spoken/mis-recognized terms to canonical forms. Ships a maternal/child-health (ASHA home-visit) dictionary; pass your own for any domain.
from hindi_normalize import replace_terms, MEDICAL_TERMS
replace_terms("बीबी एक सौ दस बटा सत्तर", MEDICAL_TERMS) # 'BP एक सौ दस / सत्तर'
replace_terms("क ख", {"क": "K", "ख": "KH"}) # 'K KH'Collapse the back-to-back stutter recognisers emit.
from hindi_normalize import collapse_repetition
collapse_repetition("ठीकठीकठीकठीक है") # 'ठीक है'For runaway tail loops in long LLM generations (repeated sentences/paragraphs) and for general invisible/typographic cleanup of non-Devanagari text, compose with llmclean.
| Function | Purpose |
|---|---|
normalize_transcript(text, *, terms, numbers, devanagari, repetition, sentence_breaks) |
Full pipeline |
convert_numbers(text) |
Hindi number words → digits |
parse_hindi_number(text) |
Parse one number expression → int |
normalize_devanagari(text, *, ...) |
Character-level normalization |
strip_zero_width(text) |
Remove invisibles; NBSP → space |
devanagari_digits_to_ascii(text) / ascii_digits_to_devanagari(text) |
Digit conversion |
normalize_danda(text) |
Danda punctuation |
fold_nasals(text, *, chandrabindu) |
Nasal clusters → anusvara |
replace_terms(text, terms) |
Term dictionary replacement |
collapse_repetition(text, *, min_repeats) |
Collapse inline stutter |
Every function is guarded against non-string input and returns the input unchanged rather than raising.
MIT