Skip to content

fix(pdf): letters set apart with space glyphs read as one word - #1140

Merged
artiz merged 1 commit into
docling-project:masterfrom
artiz:claude/issue-1139-letter-spaced-space-glyphs
Nov 17, 2026
Merged

artiz merged 1 commit into
docling-project:masterfrom
artiz:claude/issue-1139-letter-spaced-space-glyphs

Conversation

@artiz

@artiz artiz commented Nov 17, 2026

Copy link
Copy Markdown
Member

Arbortext letter-spaces a heading as letter, space glyph, letter, and moves the pen after every space: back by a fifth of its advance inside a word, forward by one and a half between words. On the page that is tracking — EUR-Lex 02021R2178, Annex I, 1.1.1. KPI bezogen auf den Umsatz (Umsatz-KPI): letters 1.7 pt apart, words 5.1 pt — and pdftotext reads the words. With a space glyph at every letter, which docling-parse keeps as a character, the line cell read K P I b e z o g e n a u f …. The spaces are past the 30 % mark of drop_overprinted_spaces (#992), and the tracking rule of #921 never saw a chain: a space glyph always splits.

textparse::drop_letter_spacing_spaces takes those spaces out of the text layer. The rule is narrow, a deliberate divergence like #992's:

  • single glyphs and space glyphs alternate on one line (no run of words does that);
  • three or more consecutive spaces between letters each have part of their advance taken back (the next letter starts before 0.95 of it) and leave one gap in points, within 0.25 pt — and the letters are not one character repeated;
  • every space of the stretch that leaves that gap goes (the short words, the brackets and the hyphen), the others stay as the word spaces;
  • those, and the spaces between the stretch and its neighbours, must be at least 1.5 x the letter gap. A line whose letters are no closer than its words — spaces at their own width, a tightened line's von a b c d bis — has no letter spacing to tell from its word spacing and is left as drawn; so are C C C C C and 1 2 3 4 5.

The letter after a dropped space is marked (Glyph::spaced) and the word contraction keeps that gap inside the word (dp_lines::tracked_gaps). The detection of #921 alone would not do: it needs three letter gaps in one line run, and a justified heading — the document's 1.1.2, its words 26 pt apart — leaves KPI and auf alone on their line cells, which read KP I and a u f.

The document's twelve runs all read as pdftotext reads them, among them (CapEx)(CapEx-KPI): the typesetter set that word space no wider than the letter spacing, and it is gone on the page too. Nothing else in its 4,185 line cells changes. pdf_word_gap=legacy knows no tracking and still cuts at each letter gap, as for #921.

PDF snapshots 101/101 exact, groundtruth 9 / 21 strict, 511 diff lines: unchanged.

Tests: the reporter's construction through the parser — the heading, the justified heading, a word alone on its line, a fixed factor, legacy — and what must stay as drawn (spaces at or over their width, word spaces not plainly wider, a tightened line, one character repeated, digits). The first fails without either half of the change.

Closes #1139

Claude-Session: https://claude.ai/code/session_01QNSzBK759SLC5oULeR3YeB

Arbortext letter-spaces a heading as letter, space glyph, letter, and
moves the pen after every space: back by a fifth of its advance inside a
word, forward by one and a half between words. On the page that is
tracking — EUR-Lex 02021R2178, Annex I, `1.1.1. KPI bezogen auf den
Umsatz (Umsatz-KPI)`: letters 1.7 pt apart, words 5.1 pt — and pdftotext
reads the words. With a space glyph at every letter, which docling-parse
keeps as a character, the line cell read `K P I b e z o g e n a u f …`.
The spaces are past the 30 % mark of drop_overprinted_spaces (docling-project#992), and
the tracking rule of docling-project#921 never saw a chain: a space glyph always splits.

textparse::drop_letter_spacing_spaces takes those spaces out of the text
layer. The rule is narrow, a deliberate divergence like docling-project#992's:

- single glyphs and space glyphs alternate on one line (no run of words
  does that);
- three or more consecutive spaces between letters each have part of
  their advance taken back (the next letter starts before 0.95 of it)
  and leave one gap in points, within 0.25 pt — and the letters are not
  one character repeated;
- every space of the stretch that leaves that gap goes (the short words,
  the brackets and the hyphen), the others stay as the word spaces;
- those, and the spaces between the stretch and its neighbours, must be
  at least 1.5 x the letter gap. A line whose letters are no closer than
  its words — spaces at their own width, a tightened line's `von a b c d
  bis` — has no letter spacing to tell from its word spacing and is left
  as drawn; so are `C C C C C` and `1 2 3 4 5`.

The letter after a dropped space is marked (Glyph::spaced) and the word
contraction keeps that gap inside the word (dp_lines::tracked_gaps). The
detection of docling-project#921 alone would not do: it needs three letter gaps in one
line run, and a justified heading — the document's 1.1.2, its words
26 pt apart — leaves `KPI` and `auf` alone on their line cells, which
read `KP I` and `a u f`.

The document's twelve runs all read as pdftotext reads them, among them
`(CapEx)(CapEx-KPI)`: the typesetter set that word space no wider than
the letter spacing, and it is gone on the page too. Nothing else in its
4,185 line cells changes. pdf_word_gap=legacy knows no tracking and
still cuts at each letter gap, as for docling-project#921.

PDF snapshots 101/101 exact, groundtruth 9 / 21 strict, 511 diff lines:
unchanged.

Tests: the reporter's construction through the parser — the heading,
the justified heading, a word alone on its line, a fixed factor, legacy
— and what must stay as drawn (spaces at or over their width, word
spaces not plainly wider, a tightened line, one character repeated,
digits). The first fails without either half of the change.

Closes docling-project#1139

Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNSzBK759SLC5oULeR3YeB
@artiz
artiz merged commit d2a94df into docling-project:master Nov 17, 2026
13 checks passed
@artiz
artiz deleted the claude/issue-1139-letter-spaced-space-glyphs branch November 17, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PDF textparse reads an Arbortext letter-spaced heading as K P I b e z o g e n (EUR-Lex)

1 participant