Skip to content

Escape pipes in HTML/DOCX/XLSX table cells - #2465

Open
Aurora (Zhao0335) wants to merge 1 commit into
microsoft:mainfrom
Zhao0335:fix/escape-pipes-in-html-tables
Open

Escape pipes in HTML/DOCX/XLSX table cells#2465
Aurora (Zhao0335) wants to merge 1 commit into
microsoft:mainfrom
Zhao0335:fix/escape-pipes-in-html-tables

Conversation

@Zhao0335

Copy link
Copy Markdown

Problem

A literal | in an HTML table cell is data, not a column separator, but markdownify emitted it raw, so converted rows gained phantom columns.

#2438 — HTML/DOCX tables. DOCX is affected because it converts via the HTML layer.

#2436 — XLSX/XLS tables. Both Excel converters build an HTML table with DataFrame.to_html and run it through HtmlConverter, so they share the same bug.

Repro (HTML, current main):

import io
from markitdown import MarkItDown

html = (
    '<table><thead><tr><th>Name</th><th>Note</th></tr></thead>'
    '<tbody><tr><td>Alice</td><td>Has a | pipe</td></tr></tbody></table>'
)
print(MarkItDown().convert_stream(
    io.BytesIO(html.encode()), file_extension='.html').markdown)
# Before: | Alice | Has a | pipe |   (broken)
# After:  | Alice | Has a \| pipe |  (correct)

Repro (XLSX):

import io
import pandas as pd
from markitdown import MarkItDown

buf = io.BytesIO()
with pd.ExcelWriter(buf, engine="openpyxl") as w:
    pd.DataFrame({"a": ["x|y"], "b": ["2"]}).to_excel(w, index=False)
buf.seek(0)
print(MarkItDown().convert_stream(buf, file_extension=".xlsx").markdown)
# Before: | x|y | 2 |
# After:  | x\|y | 2 |

Fix

Override convert_td / convert_th in _CustomMarkdownify to apply the same cell escaping the CSV converter already uses (#2266 / #2464):

  • Double any run of backslashes immediately before a pipe, then escape the pipe ((?<!\\)(\\*)\|).
  • Collapse embedded newlines to spaces so the row stays on one line.

Because XLSX/XLS and DOCX both convert through this HTML layer, one change covers both Excel formats and DOCX without touching those converters.

Tests

  • test_html_converter.py: pipes in cells and headers, backslash+pipe, plain cells unchanged, newline collapse, colspan still expands.
  • test_module_misc.py: XLSX pipe in cell and header.

pytest packages/markitdown/tests/test_html_converter.py packages/markitdown/tests/test_module_misc.py -k "html or pipe or xlsx or csv or table" — 44 passed.

Fixes #2438
Fixes #2436

A literal | in an HTML table cell was emitted raw, so converted rows
gained phantom columns. DOCX and XLSX/XLS are affected because both
convert through the HTML path.

Reuse the same escaping rule as the CSV converter (microsoft#2266 / microsoft#2464):
double any run of backslashes immediately before the pipe, then escape
the pipe. Fixes microsoft#2438 and microsoft#2436.
@Zhao0335

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTML/DOCX: pipe characters in table cells break Markdown tables XLSX/XLS: pipe characters in cells break Markdown tables

1 participant