Skip to content

fix(pdf): box glyphs by docling-parse's cap-height-scaled metrics - #1141

Merged
artiz merged 3 commits into
docling-project:masterfrom
artiz:claude/cambria-reading-order
Nov 17, 2026
Merged

artiz merged 3 commits into
docling-project:masterfrom
artiz:claude/cambria-reading-order

Conversation

@artiz

@artiz artiz commented Nov 17, 2026

Copy link
Copy Markdown
Member

Follow-up to #1129: the scrambled reading order on that issue's Cambria page.

Finding

It was our defect, not parity. Python docling 2.150 reads the page
(opendataloader-bench/pdfs/01030000000121.pdf) in the right order.

docling-parse does not box a glyph between the font's descent and ascent. It
scales both by CapHeight / Ascent (pdf_state<TEXT>:
compute_rect(font_descent*ratio, font_ascent*ratio, width), the ratio taken
when it lies in 0.05..=1), so the box tops out at the capitals. Our parser used
the descriptor's ascent and descent unscaled.

Face (from the issue's page) Descriptor Our box before docling-parse / ours now
Cambria regular Ascent 3117, Descent −2464, CapHeight 667 5.58 em 1.19 em
Cambria Bold Ascent 1047, Descent −299, CapHeight 667 1.35 em 0.86 em

With 56 pt boxes on lines 11.7 pt apart, no line cell was more than a fifth
inside its own one-line layout region (the rule needs more than half), so lines
were assigned to the wrong regions: steps 16 and 17 merged, step 17's second
line became a list item of its own, 19 came before 18, and the left table
column was printed again after the table.

This also corrects what I wrote in #1129 — that docling-parse reads Cambria's
boxes the same way. It does not.

The fix

textparse::font_ascent_descent returns the scaled pair where the descriptor
gives a plausible /CapHeight. A descriptor without one keeps its metrics.

On the issue's page:

  • all 44 line cells match docling-parse's boxes to 0.1 pt;
  • the Markdown is byte-identical to Python docling's (apart from the file's
    final newline);
  • the JSON has the same 15 text items and the same 2 × 2 table.

Effect on the corpus

The scaling applies to every font that declares a cap height, so line boxes
shift slightly everywhere (a 16.0 pt line cell on 2206.01062 becomes
docling-parse's 15.3 pt).

Measure Before After
Groundtruth, diff lines (21 files) 511 450
Groundtruth, strict 9 / 21 9 / 21
Live docling 2.150, diff lines (18 files) 475 414
Live docling 2.150, strict 6 / 18 6 / 18
Snapshots that moved — 17 of 101
Rust renderer vs the shim baselines: snapshots exact 75 / 101 77 / 101
Rust renderer vs the shim baselines: groundtruth 647 lines, 7 strict 597 lines, 7 strict

Both groundtruth gains are in two files: elsevier-00 287 → 236 and
table_misidentified_as_form 38 → 28. The other 19 are unmoved.

The 17 snapshots are our own baselines, so I scored the old baseline and the
new output against Python docling 2.150's own output for the same files (no
OCR): 1,023 → 653 diff lines.

Files
Closer to docling 11 (5 now byte-identical)
As far as before 4
A little further 2 — llncsdoc 38 → 42, the equations figure 4 → 6

One of the now-identical files looks like a loss but is docling's reading: a
figure title reads nput-Input Layer5, because its first glyph's box crosses
the page edge and docling-parse drops such a glyph.

tests/snapshots holds the 17 regenerated files; on them the run is 101/101.

Tests

Docs

Not covered

  • A standard-14 font with no descriptor still gets the 750 / −250 default
    where docling-parse reads the face's AFM metrics (and its AFM cap height).
    That approximation predates this PR and is unchanged.
  • The two snapshots that moved further from docling were not investigated.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QNSzBK759SLC5oULeR3YeB

artiz and others added 3 commits November 17, 2026 13:24
docling-parse does not box a glyph between the font's descent and
ascent: it scales both by CapHeight / Ascent (pdf_state<TEXT>,
`compute_rect(font_descent*ratio, font_ascent*ratio, width)`, the ratio
taken when it lies in 0.05..=1), so the box tops out at the capitals.
The port used the descriptor's ascent and descent unscaled.

For most text faces the difference is a few percent of a line. For a
face with inflated metrics it decides whether the page can be read:
Cambria's regular face declares /Ascent 3117 /Descent -2464 /CapHeight
667 — a 5.6 em box around every line unscaled, 1.19 em scaled. On
opendataloader-bench's 01030000000121.pdf every line cell was 56 pt
tall on lines 11.7 pt apart, no cell more than a fifth inside its own
one-line layout region, and the native pipeline assigned lines to the
wrong regions: steps 16 and 17 merged, step 17's second line a list
item of its own, 19 before 18, the left table column printed again
after the table. Python docling 2.150 (docling-parse 7.26) reads the
page in order.

font_ascent_descent now returns the scaled pair where the descriptor
gives a cap height. On that page the 44 line cells have docling-parse's
boxes to 0.1 pt, and the Markdown is docling's byte for byte.

The docling-project#1129 parser test keeps its inflated face by declaring no cap
height — the case its rules remain for.

Tests: textparse's the_char_box_is_scaled_by_cap_height_over_ascent.
The corpus baselines are measured in the next commit.

Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNSzBK759SLC5oULeR3YeB
… box

The char box of the previous commit moves the corpus toward docling:

- groundtruth 511 -> 450 diff lines, 9 / 21 strict as before:
  elsevier-00 287 -> 236, table_misidentified_as_form 38 -> 28, the
  other 19 files unmoved;
- against live docling 2.150 (docling-parse 7.25): 475 -> 414, 6 / 18
  strict as before, the same two files;
- 17 of the 101 snapshots move. Scored against docling 2.150's own
  output for those files (no OCR), they were 1,023 diff lines away and
  are 653: 11 closer — five byte-identical now, among them a figure
  title docling reads `nput-Input Layer5`, its first glyph's box
  crossing the page edge — 4 as far, 2 a little further (llncsdoc
  38 -> 42, the equations figure 4 -> 6).

tests/snapshots holds the 17 regenerated files; on them the run is
101/101 exact.

docs: PDF_CONFORMANCE.md gets the parity row for the char box and the
new headline, per-file and live numbers; MIGRATION.md the entry and its
three headline mentions. The docling-project#1129 entry and the comments it left in the
code said docling-parse reads Cambria's 5.6 em boxes as the port did —
it does not, it scales them — and now say what those rules remain for:
a face with inflated metrics and no cap height.

The renderer table in PDF_CONFORMANCE.md (shim against the Rust
renderer, dated 2026-11-11) is not re-measured here.

Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNSzBK759SLC5oULeR3YeB
…led box

The renderer table in PDF_CONFORMANCE.md compares the docling-parse shim
with the default Rust renderer against the shim-rendered baselines. With
the char box of the two commits before, re-measured on 2026-11-17:

- shim: 101 snapshots exact, 450 groundtruth diff lines, 9 strict
  (was 101 / 511 / 9);
- Rust renderer: 77 snapshots exact, 597 diff lines, 7 strict
  (was 75 / 647 / 7).

Per file the gap is where it was — 2203.01017v2 24 -> 66, redp5110
28 -> 53, right_to_left_03 exact -> 28, table_mislabeled_as_picture
24 -> 40, elsevier-00 236 -> 270, 2305.03393v1 exact -> 6 — with
table_misidentified_as_form now better under the Rust render, 28 -> 18.
MIGRATION.md's one-line mention follows.

Signed-off-by: Artem Kustikov <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QNSzBK759SLC5oULeR3YeB
@artiz
artiz force-pushed the claude/cambria-reading-order branch from 7874366 to cc0a1ea Compare November 17, 2026 12:25
@artiz
artiz merged commit 50326d2 into docling-project:master Nov 17, 2026
13 checks passed
@artiz
artiz deleted the claude/cambria-reading-order branch November 17, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant