feat(opensearch): Handle errors due to clause overflow caused by fuzziness

[bogdankostic]

Related Issues

• fixes https://github.com/deepset-ai/haystack-private/issues/200

Proposed Changes:

When a BM25 query with fuzziness="AUTO" exceeds OpenSearch's maxClauseCount (default 1024), the document store now catches the error and retries with fuzziness=0 (exact matching). This applies to both the sync (_bm25_retrieval) and async (_bm25_retrieval_async) methods.

The retry is intentionally skipped when:

• fuzziness is already 0 or "0" (retry would be redundant)
• A custom_query is provided

A warning is logged when the fallback is triggered.

How did you test it?

• Unit tests: Added 8 unit tests (4 sync + 4 async) covering:
  • the retry behavior
  • no-retry when fuzziness is already 0
  • no-retry with custom queries
  • re-raising of unrelated errors
• Integration tests: Added 2 integration tests (sync + async) that reproduce the too_many_clauses error by indexing similar 5-character words and querying with fuzziness="AUTO", verifying the fallback succeeds.

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[github-actions]

Coverage report (opensearch)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
integrations/opensearch/src/haystack_integrations/document_stores/opensearch
document_store.py
Project Total

This report was generated by python-coverage-comment-action

[davidsbatista]

In the request/discussion I see a RequestErrorinstead

Is this the correct exception to catch?

Unhandled exception
RequestError: RequestError(400, 'search_phase_execution_exception', 'too_many_clauses: maxClauseCount is set to 1024')

/home/haystackd/.local/lib/python3.12/site-packages/haystack/core/pipeline/pipeline.py, line 70, _run_component

/home/haystackd/.local/lib/python3.12/site-packages/haystack_integrations/components/retrievers/opensearch/bm25_retriever.py, line 269, run

/home/haystackd/.local/lib/python3.12/site-packages/haystack_integrations/components/retrievers/opensearch/bm25_retriever.py, line 266, run

[bogdankostic]

The issue was created when our integration still used version 2 of opensearch. My guess would be that the exception changed from version 2 to version 3. TransportError is what I get on my end with the latest OpenSearch version.

In any case, RequestError is a subclass of TransportError in opensearch-py (see the exception hierarchy). So catching TransportError also catches RequestError.

[davidsbatista]

perfect! thanks for the clarification 👍🏽

[davidsbatista]

looks good just doubtful about the correct exception to handle

[davidsbatista]

LGTM