Explainer: Text Detection API

[alvinjiooo]

Summary

We have put together an explainer dedicated to the Text Detection API, focusing on the motivation, benefits for web developers, and the modern asynchronous API design.

The full explainer can be found here:
https://github.com/alvinjiooo/shape-detection-api/blob/text-detection-explainer/text_detection_explainer.md

Key Topics Covered

1. Motivation & Tradeoffs Today:

   • Comparison with client-side WebAssembly/JS OCR bundles (multi-megabyte download overhead, CPU/battery strain, lack of hardware acceleration).
   • Comparison with remote cloud vision services (network latency, recurring costs, failure offline, data privacy concerns).

2. Core API Surface (MVP):

   • Modern asynchronous initialization via static Promise<TextDetector> TextDetector.create() to ensure detector readiness before use and provide predictable error handling.
   • Line-level text detection via detect(ImageBitmapSource image) returning detected text, axis-aligned boundingBox, and oriented cornerPoints quadrilateral vertices.
   • Support across both Window and DedicatedWorker contexts to enable off-main-thread video and camera pipelines.

3. Future Work & Potential Extensions:

   • Language negotiation and availability checking (TextDetector.availability(options)).
   • Structural hierarchy (blocks, lines, words).

We welcome thoughts, feedback, and discussion from the community and implementers!

[alvinjiooo]

Hi @nondebug and @reillyeon,
PTAL and thanks!
Alvin

[tomayac]

Left a number of suggestions, but the general direction LGTM.

I wonder if there should be a way to opt in and out of features for different use cases:

• Pure OCR might not need the text position.
• Text blurring might not need the OCR'ed text.

[alvinjiooo]

Left a number of suggestions, but the general direction LGTM.

I wonder if there should be a way to opt in and out of features for different use cases:

• Pure OCR might not need the text position.
• Text blurring might not need the OCR'ed text.

Regarding opting in/out of boundingBox vs. rawValue, it seems only macOS (Vision.framework) and Tesseract support a bounding-box-only request mode (without running character recognition), whereas other platform OCR APIs (like Windows and Android) always return both together in a single pass.

I think maybe we can keep it in future work of this explainer can explore selective detection modes later when more OCR engines support it.

[tomayac]

LGTM, thanks for having taken all my comments into account :-)