fix: sort SentenceWindowRetriever context before merging text - #12976
Open
LindseyZ1205 wants to merge 2 commits into
Open
LindseyZ1205 wants to merge 2 commits into
LindseyZ1205 wants to merge 2 commits into
Conversation
The context documents were merged into `context_windows` before being sorted by `split_id`. When chunks have no `split_idx_start` (for example chunks from MarkdownHeaderSplitter), `merge_documents_text` concatenates them in the order returned by the Document Store, so the context text could be scrambled while `context_documents` was correctly sorted. Sort first and merge the sorted list, in both run and run_async. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LindseyZ1205
requested review from
bogdankostic
and removed request for
a team
September 26, 2026 20:18
Contributor
|
@LindseyZ1205 is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
7 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
SentenceWindowRetrieverreturnscontext_windowsin Document Store order when chunks have nosplit_idx_start#12975Proposed Changes:
The shared
_assemble_contextpath merges context text before sorting documents bysplit_id. When chunks have nosplit_idx_start(for example chunks fromMarkdownHeaderSplitter),merge_documents_textconcatenates them in the Document Store's return order, whilecontext_documentsis sorted. Sort once before assembling both outputs. This preserves the upstream batched retrieval used by bothrunandrun_async.How did you test it?
hatch run fmt-checkandhatch run test:typespass for the changed source and two test files;git diff --checkpasses.Notes for the reviewer
The
run()docstring sayscontext_documentsare sorted by thesplit_idx_startmeta field, but the code sorts bysplit_id_meta_field. I left it unchanged to keep this PR focused, happy to update it here if you prefer.AI assistance: Claude Code assisted with the original bug investigation and patch. Codex assisted with adapting the fix to upstream batched retrieval, resolving the merge conflict, and running the checks above.
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:and added!in case the PR includes breaking changes.🤖 Generated with Claude Code