Conversation
|
Someone is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @Taranum01, thanks for your interest in contributing to Haystack! 🙏 ⛔ First-time contributors can have at most 1 open pull request in this repository until it has been approved, so this PR was closed automatically. Your open pull request #12991 is unaffected. Once it has been approved by a maintainer, you are welcome to open more PRs. Feel free to reopen this one at that point. See the contributing guidelines for details. This is an automated message to help us keep the review queue healthy. |
|
Taranum01 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Summary
Fixes #12954.
MarkdownHeaderSplitterpreviously matched any```(or longer) sequence as a code-fence boundary using a single regex. That made a 4-backtick opener close on the first 3-backtick run, and any bare```outside an open fence was treated as a stray boundary that swallowed subsequent headings.Per CommonMark, a closing fence must use the same character as the opener and be at least as long. This PR replaces the single regex with a per-line scan that enforces both rules.
Changes
haystack/components/preprocessors/markdown_header_splitter.py_code_block_pattern(no longer used)._is_closing_fencehelper: same character as opener, length >= opener length, only whitespace follows._split_into_code_blockswalks the source line-by-line, opening/closing a block only when the line qualifies as a fence, and emits the originalcode_blocklines verbatim.Tests
test/components/preprocessors/test_markdown_header_splitter.pytest_opening_fence_does_not_close_on_short_run: opener of length 4 stays open until a >=4 run is seen.test_stray_short_fence_outside_code_block_does_not_swallow_headings: a bare 3-backtick line with no opener does not consume the following heading.