Escape pipes in HTML/DOCX/XLSX table cells - #2465
Open
Aurora (Zhao0335) wants to merge 1 commit into
Open
Conversation
A literal | in an HTML table cell was emitted raw, so converted rows gained phantom columns. DOCX and XLSX/XLS are affected because both convert through the HTML path. Reuse the same escaping rule as the CSV converter (microsoft#2266 / microsoft#2464): double any run of backslashes immediately before the pipe, then escape the pipe. Fixes microsoft#2438 and microsoft#2436.
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A literal
|in an HTML table cell is data, not a column separator, but markdownify emitted it raw, so converted rows gained phantom columns.#2438 — HTML/DOCX tables. DOCX is affected because it converts via the HTML layer.
#2436 — XLSX/XLS tables. Both Excel converters build an HTML table with
DataFrame.to_htmland run it throughHtmlConverter, so they share the same bug.Repro (HTML, current main):
Repro (XLSX):
Fix
Override
convert_td/convert_thin_CustomMarkdownifyto apply the same cell escaping the CSV converter already uses (#2266 / #2464):(?<!\\)(\\*)\|).Because XLSX/XLS and DOCX both convert through this HTML layer, one change covers both Excel formats and DOCX without touching those converters.
Tests
test_html_converter.py: pipes in cells and headers, backslash+pipe, plain cells unchanged, newline collapse, colspan still expands.test_module_misc.py: XLSX pipe in cell and header.pytest packages/markitdown/tests/test_html_converter.py packages/markitdown/tests/test_module_misc.py -k "html or pipe or xlsx or csv or table"— 44 passed.Fixes #2438
Fixes #2436