Skip to content

DocumentToImageContent still fails the whole batch with KeyError when a PDF page can't be converted #12911

Closed

Description

@shivsin25

Describe the bug
Since #12872, DocumentToImageContent returns None for an invalid document instead of failing the whole batch. This still doesn't work when a document points to a PDF page that can't be converted: the page is past the end of the PDF, or the PDF can't be read. _convert_pdf_to_images skips those pages and logs a warning, but _batch_convert_pdf_pages_to_images then looks them up with page_number_to_image[page_number] and raises a KeyError. The valid documents in the batch are lost too, including when it runs inside LLMDocumentContentExtractor.

Error message

KeyError: 99
  File "haystack/components/converters/image/image_utils.py", line 337, in _batch_convert_pdf_pages_to_images

Expected behavior
The document gets None in image_contents and the others are still converted. That's what the run() docstring describes and what test_run_none_images expects. That test only passes because it mocks _batch_convert_pdf_pages_to_images to return {}.

Additional context
In practice this happens when stored documents point to a PDF that was later replaced with a shorter version, when a PDF is corrupted or only partly downloaded, or when a splitter assigns a page number past the end (see #12909).

To Reproduce

from haystack import Document
from haystack.components.converters.image import DocumentToImageContent

pdf = "test/test_files/pdf/sample_pdf_1.pdf"
DocumentToImageContent().run(documents=[
    Document(content="", meta={"file_path": pdf, "page_number": 1}),
    Document(content="", meta={"file_path": pdf, "page_number": 99}),
])
# KeyError: 99

An unreadable PDF (for example a file containing just %PDF-1.4 not a real PDF) raises KeyError: 1 the same way.

I'd like to work on a fix.

System:

  • OS: Windows 11
  • Haystack version: main (3.3.0-rc0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions