Skip to content

feat(ocr): configurable OCR languages with Latin + Cyrillic support - #24

Closed
Venopacman wants to merge 2 commits into
ayushh0110:mainfrom
Venopacman:feat/ocr-multilingual
Closed

Venopacman wants to merge 2 commits into
ayushh0110:mainfrom
Venopacman:feat/ocr-multilingual

Conversation

@Venopacman

@Venopacman Venopacman commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Depends on #23 (same file, stacked on top of it). Until #23 merges, this PR also shows its commit. Review only the last commit, feat(ocr): ....

Problem

OCR is hardcoded to English. Accented Latin text loses its accents and Russian comes out as junk:

Real text Today
Die Größe der Datei überschreitet Die Graksa Datei uberschreitet
La reunión de mañana reunion manana
Спасибо, завтра посмотрю Cnachbo 3aBTpa

Fix

New setting OCR_LANGUAGES: comma-separated EasyOCR codes, default en. Nothing changes unless it's set.

EasyOCR loads one recognition model per Reader, and the Cyrillic model pairs only with English. So en,de,ru can't share one Reader. When a Cyrillic code is set:

  1. The main Reader finds the text boxes once and reads the Latin languages.
  2. A second Reader (detector=False) reads the same boxes with the Cyrillic model. Detection is the expensive part, so it doesn't run twice.
  3. For each box, the Cyrillic reading wins only if it has a letter that exists only in Cyrillic (б, д, и, л, я, ...) and is about as confident. Otherwise the Latin reading stays. This matters because the Cyrillic model reads MCP as MCР (Cyrillic Р) with high confidence.
  4. Words that mix both scripts are switched to whichever script most of their letters use.

An unsupported mix (for example ch_sim with de) logs a warning and falls back to English instead of turning OCR off.

Numbers

M5 Mac, CPU, canvas_size=800, 12 real screenshots:

OCR_LANGUAGES Sec/frame Peak RAM Chars
en 1.39 2.8 GB 17,553
en,es,de,fr,ru 2.37 3.2 GB 17,556

English output doesn't change. Each recognition model is only 15–22 MB. Detection sets the peak.

Synthetic screen with one sentence per language, en,es,de,fr,ru: accents come through (reunión, mañana, Größ, überschreitet, déploiemer, échou) and Russian reads correctly (Спасибо завтра посмотрю этот отчёт).

Known limits

  • The Cyrillic model sometimes puts a Latin V inside Russian words.
  • Short words (1–2 letters) are often missed at canvas 800. This is the same as on main with the memory PR.
  • Other scripts (Chinese, Japanese, Arabic) would need the same second-Reader pattern. Left out to keep this PR small.

Testing

  • New tests/test_ocr_multilingual.py: language split, lookalike cleanup, merge rule, fallback, and the two-Reader flow. It uses fake Readers, so CI downloads no extra models.
  • Full suite with all five PRs merged: 545 passed.

🤖 Generated with Claude Code

Venopacman and others added 2 commits October 6, 2026 10:06
…vas)

On an Apple Silicon Mac the screenmind process held ~8 GB: EasyOCR's
gpu=True picks MPS, and PyTorch's MPS allocator kept ~7.5 GB of unified
memory between frames.

Moving to CPU alone did not help (peak ~8.6 GB of RAM). The cost is the
CRAFT detector running over the whole image at canvas_size=2560. Memory
scales with that size, while recognition still reads crops from the
full-resolution image. Measured on 12 real 3024x1964 screenshots, CPU:

  canvas  peak RAM  text kept  sec/frame
  1920    7.7 GB    100%       7.8
  1280    6.0 GB    ~100%      2.5
  960     4.0 GB    96%        1.9
  800     3.1 GB    90%        1.7
  640     2.1 GB    71%        1.1

- Use the GPU only when CUDA is available (NVIDIA users keep GPU OCR).
- Pass canvas_size=800 to readtext (OCR_CANVAS_SIZE).

Process footprint went from ~8.1 GB to ~1.5 GB between frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OCR was hardcoded to English. Accented Latin text lost its accents
("Größe" -> "Graksa", "déploiement" -> "deploiemer") and Russian came out
as Latin junk ("Спасибо" -> "Cnachbo").

Add OCR_LANGUAGES (comma-separated EasyOCR codes, default "en", so nothing
changes unless it is set).

EasyOCR loads one recognition model per Reader, and the Cyrillic model
only pairs with English, so "en,de,ru" can't share a Reader. When a
Cyrillic code is set:
- the main Reader detects boxes once and recognizes the Latin languages,
- a second Reader (detector=False) recognizes the same boxes in Cyrillic,
- per box, the Cyrillic reading wins only if it has a Cyrillic-only letter
  (б, д, и, л, я, ...) and is about as confident. Otherwise Latin stays.
  This keeps English away from lookalikes: the Cyrillic model reads "MCP"
  as "MCР" (Cyrillic Р) with high confidence.
- mixed-script words are normalized to their majority script.

An unsupported mix (e.g. ch_sim with de) logs a warning and falls back to
English instead of disabling OCR.

Measured on an M5 Mac, CPU, canvas_size=800, 12 real screenshots:
  en               1.39 s/frame, 2.8 GB peak, 17553 chars
  en,es,de,fr,ru   2.37 s/frame, 3.2 GB peak, 17556 chars
Detection sets the peak; each recognition model is 15-22 MB.

Known limits: the Cyrillic model sometimes emits a Latin "V" inside
Russian words; short words (1-2 letters) are often missed at canvas 800.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Venopacman Venopacman closed this Oct 6, 2026
@Venopacman

Copy link
Copy Markdown
Contributor Author

Same here (as !23) - I don't think you'd need multilang support right now.
But let me know if you think that it would be a good thing to add - I would pack a proper PR then!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant