Skip to content

fix(CSVDocumentSplitter): use positional column index instead of column label - #12981

Draft
Tyagiquamar wants to merge 1 commit into
deepset-ai:mainfrom
Tyagiquamar:fix/csv-splitter-header-zero
Draft

Tyagiquamar wants to merge 1 commit into
deepset-ai:mainfrom
Tyagiquamar:fix/csv-splitter-header-zero

Conversation

@Tyagiquamar

Copy link
Copy Markdown

Problem

CSVDocumentSplitter accepts read_csv_kwargs, but setting header=0 makes a normal CSV crash during splitting. With header=0, pandas uses the first row as string column labels, and the splitter later executed int(split_df.columns[0]) for the col_idx_start metadata, raising on the string label. The sub-table sort also ordered by label instead of position.

Solution

Resolve the first column of each sub-table to its positional index in the original DataFrame with df.columns.get_loc(...) for both the sort key and the col_idx_start metadata. Behaviour with the default header=None is unchanged, where labels are already positional integers.

Testing

  • python -m pytest test/components/preprocessors/test_csv_document_splitter.py -q — 30 passed (Docker, python:3.12-bookworm), including a new regression test running the exact scenario from the issue: read_csv_kwargs={"header": 0} with a blank-row split, asserting col_idx_start == 0, correct row_idx_start values and split contents.

Fixes #12965

Note: this change was prepared with the help of an AI coding agent under human direction, per the AI-assistants contribution guidance.

…mn label

read_csv_kwargs allows overriding how pandas reads the CSV, e.g. setting
header=0 so the first row becomes the column labels. The splitter then
crashed at int(split_df.columns[0]) because the label is a string, and
sorted sub-tables by label instead of position.

Resolve the first column of each sub-table to its positional index in
the original DataFrame for both the sort and the col_idx_start metadata.

Fixes deepset-ai#12965

Signed-off-by: Tyagiquamar <Tyagiquamar@users.noreply.github.com>
@Tyagiquamar
Tyagiquamar requested a review from a team as a code owner September 27, 2026 09:52
@Tyagiquamar
Tyagiquamar requested review from julian-risch and removed request for a team September 27, 2026 09:52
@vercel

vercel Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

@Tyagiquamar is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @Tyagiquamar, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot
HaystackBot removed the request for review from julian-risch September 27, 2026 11:25
@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Sep 27, 2026
@HaystackBot
HaystackBot marked this pull request as draft September 27, 2026 11:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-pending PR is in draft until the contributor signs the CLA topic:tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CSVDocumentSplitter crashes when read_csv_kwargs sets header=0

3 participants