Skip to content

fix: treat closing fence longer than opener as code-block boundary in MarkdownHeaderSplitter (#12954) - #12993

Closed
Taranum01 wants to merge 1 commit into
deepset-ai:mainfrom
Taranum01:fix/12954-markdown-header-splitter-fences
Closed

Taranum01 wants to merge 1 commit into
deepset-ai:mainfrom
Taranum01:fix/12954-markdown-header-splitter-fences

Conversation

@Taranum01

Copy link
Copy Markdown

Summary

Fixes #12954.

MarkdownHeaderSplitter previously matched any ``` (or longer) sequence as a code-fence boundary using a single regex. That made a 4-backtick opener close on the first 3-backtick run, and any bare ``` outside an open fence was treated as a stray boundary that swallowed subsequent headings.

Per CommonMark, a closing fence must use the same character as the opener and be at least as long. This PR replaces the single regex with a per-line scan that enforces both rules.

Changes

  • haystack/components/preprocessors/markdown_header_splitter.py
    • Removed module-level _code_block_pattern (no longer used).
    • New per-line _is_closing_fence helper: same character as opener, length >= opener length, only whitespace follows.
    • _split_into_code_blocks walks the source line-by-line, opening/closing a block only when the line qualifies as a fence, and emits the original code_block lines verbatim.

Tests

  • test/components/preprocessors/test_markdown_header_splitter.py
    • test_opening_fence_does_not_close_on_short_run: opener of length 4 stays open until a >=4 run is seen.
    • test_stray_short_fence_outside_code_block_does_not_swallow_headings: a bare 3-backtick line with no opener does not consume the following heading.

@Taranum01
Taranum01 requested a review from a team as a code owner September 27, 2026 22:59
@Taranum01
Taranum01 requested review from julian-risch and removed request for a team September 27, 2026 22:59
@vercel

vercel Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Someone is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown
Contributor

Hi @Taranum01, thanks for your interest in contributing to Haystack! 🙏

⛔ First-time contributors can have at most 1 open pull request in this repository until it has been approved, so this PR was closed automatically. Your open pull request #12991 is unaffected. Once it has been approved by a maintainer, you are welcome to open more PRs. Feel free to reopen this one at that point.

See the contributing guidelines for details.

This is an automated message to help us keep the review queue healthy.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Taranum01 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MarkdownHeaderSplitter treats headings inside longer closing fences as headers

2 participants