Improve efficiency of escaping pipes in CSVs

[afourney]

Problem

CSV pipe escaping introduced quadratic processing time for long runs of backslashes not immediately followed by a pipe.

The regex  r"(\\*)\|"  consumes the backslash run, backtracks looking for a pipe, then repeats from each subsequent position. A valid CSV containing a single 64 KiB backslash-only field took approximately 9 seconds to convert. The same slowdown occurs when a pipe appears later, separated from the backslashes by another character.

Fix

Add a negative lookbehind so matching cannot restart inside a backslash run:

_PIPE_ESCAPE_RE = re.compile(r"(?<!\\)(\\*)\|")

This makes processing linear rather than quadratic, reducing the reproduced 64 KiB case to milliseconds.

Output remains unchanged: pipes are escaped, backslashes immediately preceding pipes are doubled, unrelated backslashes are preserved, and newline handling is untouched. Regression coverage includes long backslash runs in both header and data cells, with no following pipe, an adjacent pipe, and a pipe separated by ordinary text.

[Copilot]

🟢 Approval recommended

The optimized regex preserves existing escaping behavior and the regression tests cover the reported performance cases.

Pull request overview

Optimizes CSV pipe escaping to avoid quadratic processing on long backslash runs while preserving output.

Changes:

• Adds a negative lookbehind to ensure linear regex processing.
• Adds regression coverage for long backslash runs in headers and cells.

File summaries

File	Description
packages/markitdown/src/markitdown/converters/_csv_converter.py	Optimizes the pipe-escaping regex.
packages/markitdown/tests/test_module_misc.py	Tests long backslash-run scenarios.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.