diff --git a/docs-website/reference/integrations-api/opensearch.md b/docs-website/reference/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference/integrations-api/opensearch.md
+++ b/docs-website/reference/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.18/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.18/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.18/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.18/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.19/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.19/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.19/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.19/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.20/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.20/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.20/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.20/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.21/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.21/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.21/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.21/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.22/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.22/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.22/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.22/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.23/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.23/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.23/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.23/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.24/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.24/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.24/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.24/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.25/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.25/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.25/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.25/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.26/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.26/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.26/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.26/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.27/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.27/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.27/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.27/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.28/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.28/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.28/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.28/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.29/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.29/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.29/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.29/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.30/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.30/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.30/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.30/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-2.31/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-2.31/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-2.31/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-2.31/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-3.0/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-3.0/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-3.0/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-3.0/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-3.1/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-3.1/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-3.1/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-3.1/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters
diff --git a/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/opensearch.md
new file mode 100644
index 0000000000..ed25053261
--- /dev/null
+++ b/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/opensearch.md
@@ -0,0 +1,1975 @@
+---
+title: "OpenSearch"
+id: integrations-opensearch
+description: "OpenSearch integration for Haystack"
+slug: "/integrations-opensearch"
+---
+
+
+## haystack_integrations.components.retrievers.opensearch.bm25_retriever
+
+### OpenSearchBM25Retriever
+
+Fetches documents from OpenSearchDocumentStore using the keyword-based BM25 algorithm.
+
+BM25 computes a weighted word overlap between the query string and a document to determine its similarity.
+
+#### __init__
+
+```python
+__init__(
+ *,
+ document_store: OpenSearchDocumentStore,
+ filters: dict[str, Any] | None = None,
+ fuzziness: int | str = 0,
+ top_k: int = 10,
+ scale_score: bool = False,
+ all_terms_must_match: bool = False,
+ filter_policy: str | FilterPolicy = FilterPolicy.REPLACE,
+ custom_query: dict[str, Any] | None = None,
+ raise_on_failure: bool = True
+) -> None
+```
+
+Creates the OpenSearchBM25Retriever component.
+
+**Parameters:**
+
+- **document_store** (OpenSearchDocumentStore) – An instance of OpenSearchDocumentStore to use with the Retriever.
+- **filters** (dict\[str, Any\] | None) – Filters to narrow down the search for documents in the Document Store.
+- **fuzziness** (int | str) – Determines how approximate string matching is applied in full-text queries.
+ This parameter sets the number of character edits (insertions, deletions, or substitutions)
+ required to transform one word into another. For example, the "fuzziness" between the words
+ "wined" and "wind" is 1 because only one edit is needed to match them.
+
+Defaults to `0` (exact matching). Use `"AUTO"` for automatic adjustment based on term length.
+For detailed guidance, refer to the
+[OpenSearch fuzzy query documentation](https://opensearch.org/docs/latest/query-dsl/term/fuzzy/).
+
+- **top_k** (int) – Maximum number of documents to return.
+
+- **scale_score** (bool) – If `True`, scales the score of retrieved documents to a range between 0 and 1.
+ This is useful when comparing documents across different indexes.
+
+- **all_terms_must_match** (bool) – If `True`, all terms in the query string must be present in the
+ retrieved documents. This is useful when searching for short text where even one term
+ can make a difference.
+
+- **filter_policy** (str | FilterPolicy) – Policy to determine how filters are applied. Possible options:
+
+- `replace`: Runtime filters replace initialization filters. Use this policy to change the filtering scope
+ for specific queries.
+
+- `merge`: Runtime filters are merged with initialization filters.
+
+- **custom_query** (dict\[str, Any\] | None) – The query containing a mandatory `$query` and an optional `$filters` placeholder.
+
+ **An example custom_query:**
+
+ ```python
+ {
+ "query": {
+ "bool": {
+ "should": [{"multi_match": {
+ "query": "$query", // mandatory query placeholder
+ "type": "most_fields",
+ "fields": ["content", "title"]}}],
+ "filter": "$filters" // optional filter placeholder
+ }
+ }
+ }
+ ```
+
+An example `run()` method for this `custom_query`:
+
+```python
+retriever.run(
+ query="Why did the revenue increase?",
+ filters={
+ "operator": "AND",
+ "conditions": [
+ {"field": "meta.years", "operator": "==", "value": "2019"},
+ {"field": "meta.quarters", "operator": "in", "value": ["Q1", "Q2"]},
+ ],
+ },
+)
+```
+
+- **raise_on_failure** (bool) – Whether to raise an exception if the API call fails. Otherwise log a warning and return an empty list.
+
+**Raises:**
+
+- ValueError – If `document_store` is not an instance of OpenSearchDocumentStore.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serializes the component to a dictionary.
+
+**Returns:**
+
+- dict\[str, Any\] – Dictionary with serialized data.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> OpenSearchBM25Retriever
+```
+
+Deserializes the component from a dictionary.
+
+**Parameters:**
+
+- **data** (dict\[str, Any\]) – Dictionary to deserialize from.
+
+**Returns:**
+
+- OpenSearchBM25Retriever – Deserialized component.
+
+#### close
+
+```python
+close() -> None
+```
+
+Release the synchronous resources of the underlying Document Store.
+
+#### close_async
+
+```python
+close_async() -> None
+```
+
+Release the asynchronous resources of the underlying Document Store.
+
+#### run
+
+```python
+run(
+ query: str,
+ filters: dict[str, Any] | None = None,
+ all_terms_must_match: bool | None = None,
+ top_k: int | None = None,
+ fuzziness: int | str | None = None,
+ scale_score: bool | None = None,
+ custom_query: dict[str, Any] | None = None,
+ document_store: OpenSearchDocumentStore | None = None,
+) -> dict[str, list[Document]]
+```
+
+Retrieve documents using BM25 retrieval.
+
+**Parameters:**
+
+- **query** (str) – The query string.
+
+- **filters** (dict\[str, Any\] | None) – Filters applied to the retrieved documents. The way runtime filters are applied depends on
+ the `filter_policy` specified at Retriever's initialization.
+
+- **all_terms_must_match** (bool | None) – If `True`, all terms in the query string must be present in the
+ retrieved documents.
+
+- **top_k** (int | None) – Maximum number of documents to return.
+
+- **fuzziness** (int | str | None) – Fuzziness parameter for full-text queries to apply approximate string matching.
+ For more information, see [OpenSearch fuzzy query](https://opensearch.org/docs/latest/query-dsl/term/fuzzy/).
+
+- **scale_score** (bool | None) – If `True`, scales the score of retrieved documents to a range between 0 and 1.
+ This is useful when comparing documents across different indexes.
+
+- **custom_query** (dict\[str, Any\] | None) – A custom OpenSearch query. It must include a `$query` and may optionally
+ include a `$filters` placeholder.
+
+ **An example custom_query:**
+
+ ```python
+ {
+ "query": {
+ "bool": {
+ "should": [{"multi_match": {
+ "query": "$query", // mandatory query placeholder
+ "type": "most_fields",
+ "fields": ["content", "title"]}}],
+ "filter": "$filters" // optional filter placeholder
+ }
+ }
+ }
+ ```
+
+**For this custom_query, a sample `run()` could be:**
+
+```python
+retriever.run(
+ query="Why did the revenue increase?",
+ filters={
+ "operator": "AND",
+ "conditions": [
+ {"field": "meta.years", "operator": "==", "value": "2019"},
+ {"field": "meta.quarters", "operator": "in", "value": ["Q1", "Q2"]},
+ ],
+ },
+)
+```
+
+- **document_store** (OpenSearchDocumentStore | None) – Optionally, an instance of OpenSearchDocumentStore to use with the Retriever
+
+**Returns:**
+
+- dict\[str, list\[Document\]\] – A dictionary containing the retrieved documents with the following structure:
+- documents: List of retrieved Documents.
+
+#### run_async
+
+```python
+run_async(
+ query: str,
+ filters: dict[str, Any] | None = None,
+ all_terms_must_match: bool | None = None,
+ top_k: int | None = None,
+ fuzziness: int | str | None = None,
+ scale_score: bool | None = None,
+ custom_query: dict[str, Any] | None = None,
+ document_store: OpenSearchDocumentStore | None = None,
+) -> dict[str, list[Document]]
+```
+
+Asynchronously retrieve documents using BM25 retrieval.
+
+**Parameters:**
+
+- **query** (str) – The query string.
+- **filters** (dict\[str, Any\] | None) – Filters applied to the retrieved documents. The way runtime filters are applied depends on
+ the `filter_policy` specified at Retriever's initialization.
+- **all_terms_must_match** (bool | None) – If `True`, all terms in the query string must be present in the
+ retrieved documents.
+- **top_k** (int | None) – Maximum number of documents to return.
+- **fuzziness** (int | str | None) – Fuzziness parameter for full-text queries to apply approximate string matching.
+ For more information, see [OpenSearch fuzzy query](https://opensearch.org/docs/latest/query-dsl/term/fuzzy/).
+- **scale_score** (bool | None) – If `True`, scales the score of retrieved documents to a range between 0 and 1.
+ This is useful when comparing documents across different indexes.
+- **custom_query** (dict\[str, Any\] | None) – A custom OpenSearch query. It must include a `$query` and may optionally
+ include a `$filters` placeholder.
+- **document_store** (OpenSearchDocumentStore | None) – Optionally, an instance of OpenSearchDocumentStore to use with the Retriever
+
+**Returns:**
+
+- dict\[str, list\[Document\]\] – A dictionary containing the retrieved documents with the following structure:
+- documents: List of retrieved Documents.
+
+## haystack_integrations.components.retrievers.opensearch.embedding_retriever
+
+### OpenSearchEmbeddingRetriever
+
+Retrieves documents from the OpenSearchDocumentStore using a vector similarity metric.
+
+Must be connected to the OpenSearchDocumentStore to run.
+
+#### __init__
+
+```python
+__init__(
+ *,
+ document_store: OpenSearchDocumentStore,
+ filters: dict[str, Any] | None = None,
+ top_k: int = 10,
+ filter_policy: str | FilterPolicy = FilterPolicy.REPLACE,
+ custom_query: dict[str, Any] | None = None,
+ raise_on_failure: bool = True,
+ efficient_filtering: bool = False,
+ search_kwargs: dict[str, Any] | None = None
+) -> None
+```
+
+Create the OpenSearchEmbeddingRetriever component.
+
+**Parameters:**
+
+- **document_store** (OpenSearchDocumentStore) – An instance of OpenSearchDocumentStore to use with the Retriever.
+
+- **filters** (dict\[str, Any\] | None) – Filters applied when fetching documents from the Document Store.
+ Filters are applied during the approximate kNN search to ensure the Retriever returns
+ `top_k` matching documents.
+
+- **top_k** (int) – Maximum number of documents to return.
+
+- **filter_policy** (str | FilterPolicy) – Policy to determine how filters are applied. Possible options:
+
+- `merge`: Runtime filters are merged with initialization filters.
+
+- `replace`: Runtime filters replace initialization filters. Use this policy to change the filtering scope.
+
+- **custom_query** (dict\[str, Any\] | None) – The custom OpenSearch query containing a mandatory `$query_embedding` and
+ an optional `$filters` placeholder.
+
+ **An example custom_query:**
+
+ ```python
+ {
+ "query": {
+ "bool": {
+ "must": [
+ {
+ "knn": {
+ "embedding": {
+ "vector": "$query_embedding", // mandatory query placeholder
+ "k": 10000,
+ }
+ }
+ }
+ ],
+ "filter": "$filters" // optional filter placeholder
+ }
+ }
+ }
+ ```
+
+For this `custom_query`, an example `run()` could be:
+
+```python
+retriever.run(
+ query_embedding=embedding,
+ filters={
+ "operator": "AND",
+ "conditions": [
+ {"field": "meta.years", "operator": "==", "value": "2019"},
+ {"field": "meta.quarters", "operator": "in", "value": ["Q1", "Q2"]},
+ ],
+ },
+)
+```
+
+- **raise_on_failure** (bool) – If `True`, raises an exception if the API call fails.
+ If `False`, logs a warning and returns an empty list.
+- **efficient_filtering** (bool) – If `True`, the filter will be applied during the approximate kNN search.
+ This is only supported for knn engines "faiss" and "lucene" and does not work with the default "nmslib".
+- **search_kwargs** (dict\[str, Any\] | None) – Additional keyword arguments for finetuning the embedding search.
+ E.g., to specify `k` and `ef_search`
+
+```python
+{
+ "k": 20, # See https://docs.opensearch.org/latest/vector-search/vector-search-techniques/approximate-knn/#the-number-of-returned-results
+ "method_parameters": {
+ "ef_search": 512, # See https://docs.opensearch.org/latest/query-dsl/specialized/k-nn/index/#ef_search
+ }
+}
+```
+
+For a full list of available parameters, see the OpenSearch documentation:
+https://docs.opensearch.org/latest/query-dsl/specialized/k-nn/index/#request-body-fields
+
+**Raises:**
+
+- ValueError – If `document_store` is not an instance of OpenSearchDocumentStore.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serializes the component to a dictionary.
+
+**Returns:**
+
+- dict\[str, Any\] – Dictionary with serialized data.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> OpenSearchEmbeddingRetriever
+```
+
+Deserializes the component from a dictionary.
+
+**Parameters:**
+
+- **data** (dict\[str, Any\]) – Dictionary to deserialize from.
+
+**Returns:**
+
+- OpenSearchEmbeddingRetriever – Deserialized component.
+
+#### close
+
+```python
+close() -> None
+```
+
+Release the synchronous resources of the underlying Document Store.
+
+#### close_async
+
+```python
+close_async() -> None
+```
+
+Release the asynchronous resources of the underlying Document Store.
+
+#### run
+
+```python
+run(
+ query_embedding: list[float],
+ filters: dict[str, Any] | None = None,
+ top_k: int | None = None,
+ custom_query: dict[str, Any] | None = None,
+ efficient_filtering: bool | None = None,
+ document_store: OpenSearchDocumentStore | None = None,
+ search_kwargs: dict[str, Any] | None = None,
+) -> dict[str, list[Document]]
+```
+
+Retrieve documents using a vector similarity metric.
+
+**Parameters:**
+
+- **query_embedding** (list\[float\]) – Embedding of the query.
+
+- **filters** (dict\[str, Any\] | None) – Filters applied when fetching documents from the Document Store.
+ Filters are applied during the approximate kNN search to ensure the Retriever returns `top_k` matching
+ documents.
+ The way runtime filters are applied depends on the `filter_policy` selected when initializing the Retriever.
+
+- **top_k** (int | None) – Maximum number of documents to return.
+
+- **custom_query** (dict\[str, Any\] | None) – A custom OpenSearch query containing a mandatory `$query_embedding` and an
+ optional `$filters` placeholder.
+
+ **An example custom_query:**
+
+ ```python
+ {
+ "query": {
+ "bool": {
+ "must": [
+ {
+ "knn": {
+ "embedding": {
+ "vector": "$query_embedding", // mandatory query placeholder
+ "k": 10000,
+ }
+ }
+ }
+ ],
+ "filter": "$filters" // optional filter placeholder
+ }
+ }
+ }
+ ```
+
+For this `custom_query`, an example `run()` could be:
+
+```python
+retriever.run(
+ query_embedding=embedding,
+ filters={
+ "operator": "AND",
+ "conditions": [
+ {"field": "meta.years", "operator": "==", "value": "2019"},
+ {"field": "meta.quarters", "operator": "in", "value": ["Q1", "Q2"]},
+ ],
+ },
+)
+```
+
+- **efficient_filtering** (bool | None) – If `True`, the filter will be applied during the approximate kNN search.
+ This is only supported for knn engines "faiss" and "lucene" and does not work with the default "nmslib".
+- **document_store** (OpenSearchDocumentStore | None) – Optional instance of OpenSearchDocumentStore to use with the Retriever.
+- **search_kwargs** (dict\[str, Any\] | None) – Additional keyword arguments for finetuning the embedding search. If not provided,
+ defaults to the parameter set at initialization (if any).
+ E.g., to specify `k` and `ef_search`
+
+```python
+{
+ "k": 20, # See https://docs.opensearch.org/latest/vector-search/vector-search-techniques/approximate-knn/#the-number-of-returned-results
+ "method_parameters": {
+ "ef_search": 512, # See https://docs.opensearch.org/latest/query-dsl/specialized/k-nn/index/#ef_search
+ }
+}
+```
+
+For a full list of available parameters, see the OpenSearch documentation:
+https://docs.opensearch.org/latest/query-dsl/specialized/k-nn/index/#request-body-fields
+
+**Returns:**
+
+- dict\[str, list\[Document\]\] – Dictionary with key "documents" containing the retrieved Documents.
+- documents: List of Document similar to `query_embedding`.
+
+#### run_async
+
+```python
+run_async(
+ query_embedding: list[float],
+ filters: dict[str, Any] | None = None,
+ top_k: int | None = None,
+ custom_query: dict[str, Any] | None = None,
+ efficient_filtering: bool | None = None,
+ document_store: OpenSearchDocumentStore | None = None,
+ search_kwargs: dict[str, Any] | None = None,
+) -> dict[str, list[Document]]
+```
+
+Asynchronously retrieve documents using a vector similarity metric.
+
+**Parameters:**
+
+- **query_embedding** (list\[float\]) – Embedding of the query.
+
+- **filters** (dict\[str, Any\] | None) – Filters applied when fetching documents from the Document Store.
+ Filters are applied during the approximate kNN search to ensure the Retriever
+ returns `top_k` matching documents.
+ The way runtime filters are applied depends on the `filter_policy` selected when initializing the Retriever.
+
+- **top_k** (int | None) – Maximum number of documents to return.
+
+- **custom_query** (dict\[str, Any\] | None) – A custom OpenSearch query containing a mandatory `$query_embedding` and an
+ optional `$filters` placeholder.
+
+ **An example custom_query:**
+
+ ```python
+ {
+ "query": {
+ "bool": {
+ "must": [
+ {
+ "knn": {
+ "embedding": {
+ "vector": "$query_embedding", // mandatory query placeholder
+ "k": 10000,
+ }
+ }
+ }
+ ],
+ "filter": "$filters" // optional filter placeholder
+ }
+ }
+ }
+ ```
+
+For this `custom_query`, an example `run()` could be:
+
+```python
+retriever.run(
+ query_embedding=embedding,
+ filters={
+ "operator": "AND",
+ "conditions": [
+ {"field": "meta.years", "operator": "==", "value": "2019"},
+ {"field": "meta.quarters", "operator": "in", "value": ["Q1", "Q2"]},
+ ],
+ },
+)
+```
+
+- **efficient_filtering** (bool | None) – If `True`, the filter will be applied during the approximate kNN search.
+ This is only supported for knn engines "faiss" and "lucene" and does not work with the default "nmslib".
+- **document_store** (OpenSearchDocumentStore | None) – Optional instance of OpenSearchDocumentStore to use with the Retriever.
+- **search_kwargs** (dict\[str, Any\] | None) – Additional keyword arguments for finetuning the embedding search. If not provided,
+ defaults to the parameter set at initialization (if any).
+ E.g., to specify `k` and `ef_search`
+
+```python
+{
+ "k": 20, # See https://docs.opensearch.org/latest/vector-search/vector-search-techniques/approximate-knn/#the-number-of-returned-results
+ "method_parameters": {
+ "ef_search": 512, # See https://docs.opensearch.org/latest/query-dsl/specialized/k-nn/index/#ef_search
+ }
+}
+```
+
+For a full list of available parameters, see the OpenSearch documentation:
+https://docs.opensearch.org/latest/query-dsl/specialized/k-nn/index/#request-body-fields
+
+**Returns:**
+
+- dict\[str, list\[Document\]\] – Dictionary with key "documents" containing the retrieved Documents.
+- documents: List of Document similar to `query_embedding`.
+
+## haystack_integrations.components.retrievers.opensearch.metadata_retriever
+
+### OpenSearchMetadataRetriever
+
+Retrieves and ranks metadata from documents stored in an OpenSearchDocumentStore.
+
+It searches specified metadata fields for matches to a given query, ranks the results based on relevance using
+Jaccard similarity, and returns the top-k results containing only the specified metadata fields. Additionally, it
+adds a boost to the score of exact matches.
+
+The search is designed for metadata fields whose values are **text** (strings). It uses prefix, wildcard and fuzzy
+matching to find candidate documents; these query types operate only on text/keyword fields in OpenSearch.
+
+Metadata fields with **non-string types** (integers, floats, booleans, lists of non-strings) are indexed by
+OpenSearch as numeric, boolean, or array types. Those field types do not support prefix, wildcard, or full-text
+match queries, so documents are typically not found when you search only by such fields.
+
+**Mixed types** in the same metadata field (e.g. a list containing both strings and numbers) are not supported.
+
+Must be connected to the OpenSearchDocumentStore to run.
+
+Example:
+\`\`\`python
+from haystack import Document
+from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore
+from haystack_integrations.components.retrievers.opensearch import OpenSearchMetadataRetriever
+
+````
+# Create documents with metadata
+docs = [
+ Document(
+ content="Python programming guide",
+ meta={"category": "Python", "status": "active", "priority": 1, "author": "John Doe"}
+ ),
+ Document(
+ content="Java tutorial",
+ meta={"category": "Java", "status": "active", "priority": 2, "author": "Jane Smith"}
+ ),
+ Document(
+ content="Python advanced topics",
+ meta={"category": "Python", "status": "inactive", "priority": 3, "author": "John Doe"}
+ ),
+]
+document_store.write_documents(docs, refresh=True)
+
+# Create retriever specifying which metadata fields to search and return
+retriever = OpenSearchMetadataRetriever(
+ document_store=document_store,
+ metadata_fields=["category", "status", "priority"],
+ top_k=10,
+)
+
+# Search for metadata
+result = retriever.run(query="Python")
+
+# Result structure:
+# {
+# "metadata": [
+# {"category": "Python", "status": "active", "priority": 1},
+# {"category": "Python", "status": "inactive", "priority": 3},
+# ]
+# }
+#
+# Note: Only the specified metadata_fields are returned in the results.
+# Other metadata fields (like "author") and document content are excluded.
+```
+````
+
+#### __init__
+
+```python
+__init__(
+ *,
+ document_store: OpenSearchDocumentStore,
+ metadata_fields: list[str],
+ top_k: int = 20,
+ exact_match_weight: float = 0.6,
+ mode: Literal["strict", "fuzzy"] = "fuzzy",
+ fuzziness: int | Literal["AUTO"] = 2,
+ prefix_length: int = 0,
+ max_expansions: int = 200,
+ tie_breaker: float = 0.7,
+ jaccard_n: int = 3,
+ raise_on_failure: bool = True
+) -> None
+```
+
+Create the OpenSearchMetadataRetriever component.
+
+**Parameters:**
+
+- **document_store** (OpenSearchDocumentStore) – An instance of OpenSearchDocumentStore to use with the Retriever.
+- **metadata_fields** (list\[str\]) – List of metadata field names to search within each document's metadata.
+- **top_k** (int) – Maximum number of top results to return based on relevance. Default is 20.
+- **exact_match_weight** (float) – Weight to boost the score of exact matches in metadata fields.
+ Default is 0.6. It's used on both "strict" and "fuzzy" modes and applied after the search executes.
+- **mode** (Literal['strict', 'fuzzy']) – Search mode. "strict" uses prefix and wildcard matching,
+ "fuzzy" uses fuzzy matching with dis_max queries. Default is "fuzzy".
+ In both modes, results are scored using Jaccard similarity (n-gram based)
+ computed server-side via a Painless script; n is controlled by jaccard_n.
+- **fuzziness** (int | Literal['AUTO']) – Maximum allowed Damerau-Levenshtein distance (edit distance) for fuzzy matching.
+ Accepts an integer (e.g., 0, 1, 2) or "AUTO" which chooses based on term length.
+ Default is 2. Only applies when mode is "fuzzy".
+- **prefix_length** (int) – Number of leading characters that must match exactly before fuzzy matching applies.
+ Default is 0 (no prefix requirement). Only applies when mode is "fuzzy".
+- **max_expansions** (int) – Maximum number of term variations the fuzzy query can generate.
+ Default is 200. Only applies when mode is "fuzzy".
+- **tie_breaker** (float) – Weight (0..1) for other matching clauses in the dis_max query.
+ Boosts documents that match multiple clauses. Default is 0.7. Only applies when mode is "fuzzy".
+- **jaccard_n** (int) – N-gram size for Jaccard similarity scoring. Default 3; larger n favors longer token matches.
+- **raise_on_failure** (bool) – If `True`, raises an exception if the API call fails.
+ If `False`, logs a warning and returns an empty list.
+
+**Raises:**
+
+- ValueError – If `document_store` is not an instance of OpenSearchDocumentStore.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serializes the component to a dictionary.
+
+**Returns:**
+
+- dict\[str, Any\] – Dictionary with serialized data.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> OpenSearchMetadataRetriever
+```
+
+Deserializes the component from a dictionary.
+
+**Parameters:**
+
+- **data** (dict\[str, Any\]) – Dictionary to deserialize from.
+
+**Returns:**
+
+- OpenSearchMetadataRetriever – Deserialized component.
+
+#### close
+
+```python
+close() -> None
+```
+
+Release the synchronous resources of the underlying Document Store.
+
+#### close_async
+
+```python
+close_async() -> None
+```
+
+Release the asynchronous resources of the underlying Document Store.
+
+#### run
+
+```python
+run(
+ query: str,
+ *,
+ document_store: OpenSearchDocumentStore | None = None,
+ metadata_fields: list[str] | None = None,
+ top_k: int | None = None,
+ exact_match_weight: float | None = None,
+ mode: Literal["strict", "fuzzy"] | None = None,
+ fuzziness: int | Literal["AUTO"] | None = None,
+ prefix_length: int | None = None,
+ max_expansions: int | None = None,
+ tie_breaker: float | None = None,
+ jaccard_n: int | None = None,
+ filters: dict[str, Any] | None = None
+) -> dict[str, list[dict[str, Any]]]
+```
+
+Execute a search query against the metadata fields of documents stored in the Document Store.
+
+**Parameters:**
+
+- **query** (str) – The search query string, which can contain multiple comma-separated parts.
+ Each part will be searched across all specified fields.
+- **document_store** (OpenSearchDocumentStore | None) – The Document Store to run the query against.
+ If not provided, the one provided in `__init__` is used.
+- **metadata_fields** (list\[str\] | None) – List of metadata field names to search within.
+ If not provided, the fields provided in `__init__` are used.
+- **top_k** (int | None) – Maximum number of top results to return based on relevance.
+ The search retrieves up to 1000 hits from OpenSearch, then applies boosting and filters
+ the results to the top_k most relevant matches.
+ If not provided, the top_k provided in `__init__` is used.
+- **exact_match_weight** (float | None) – Weight to boost the score of exact matches in metadata fields.
+ If not provided, the exact_match_weight provided in `__init__` is used.
+- **mode** (Literal['strict', 'fuzzy'] | None) – Search mode. "strict" uses prefix and wildcard matching,
+ "fuzzy" uses fuzzy matching with dis_max queries.
+ In both modes, results are scored using Jaccard similarity (n-gram based) via a Painless script.
+ If not provided, the mode provided in `__init__` is used.
+- **fuzziness** (int | Literal['AUTO'] | None) – Maximum allowed Damerau-Levenshtein distance (edit distance) for fuzzy matching.
+ Accepts an integer (e.g., 0, 1, 2) or "AUTO" which chooses based on term length.
+ Only applies when mode is "fuzzy". If not provided, the fuzziness provided in `__init__` is used.
+- **prefix_length** (int | None) – Number of leading characters that must match exactly before fuzzy matching applies.
+ Only applies when mode is "fuzzy". If not provided, the prefix_length provided in `__init__` is used.
+- **max_expansions** (int | None) – Maximum number of term variations the fuzzy query can generate.
+ Only applies when mode is "fuzzy". If not provided, the max_expansions provided in `__init__` is used.
+- **tie_breaker** (float | None) – Weight (0..1) for other matching clauses; boosts docs matching multiple
+ clauses. Only applies when mode is "fuzzy". If not provided, the tie_breaker provided in `__init__` is used.
+- **jaccard_n** (int | None) – N-gram size for Jaccard similarity scoring. If not provided, the jaccard_n from `__init__`
+ is used.
+- **filters** (dict\[str, Any\] | None) – Additional filters to apply to the search query.
+
+**Returns:**
+
+- dict\[str, list\[dict\[str, Any\]\]\] – A dictionary containing the top-k retrieved metadata results.
+
+Example:
+\`\`\`python
+from haystack import Document
+
+````
+# First, add a document with matching metadata to the store
+store.write_documents([
+ Document(
+ content="Python programming guide",
+ meta={"category": "Python", "status": "active", "priority": 1}
+ )
+])
+
+retriever = OpenSearchMetadataRetriever(
+ document_store=store,
+ metadata_fields=["category", "status", "priority"]
+)
+result = retriever.run(query="Python, active")
+# Returns: {"metadata": [{"category": "Python", "status": "active", "priority": 1}]}
+```
+````
+
+#### run_async
+
+```python
+run_async(
+ query: str,
+ *,
+ document_store: OpenSearchDocumentStore | None = None,
+ metadata_fields: list[str] | None = None,
+ top_k: int | None = None,
+ exact_match_weight: float | None = None,
+ mode: Literal["strict", "fuzzy"] | None = None,
+ fuzziness: int | Literal["AUTO"] | None = None,
+ prefix_length: int | None = None,
+ max_expansions: int | None = None,
+ tie_breaker: float | None = None,
+ jaccard_n: int | None = None,
+ filters: dict[str, Any] | None = None
+) -> dict[str, list[dict[str, Any]]]
+```
+
+Asynchronously execute a search query against the metadata fields of documents stored in the Document Store.
+
+**Parameters:**
+
+- **query** (str) – The search query string, which can contain multiple comma-separated parts.
+ Each part will be searched across all specified fields.
+- **document_store** (OpenSearchDocumentStore | None) – The Document Store to run the query against.
+ If not provided, the one provided in `__init__` is used.
+- **metadata_fields** (list\[str\] | None) – List of metadata field names to search within.
+ If not provided, the fields provided in `__init__` are used.
+- **top_k** (int | None) – Maximum number of top results to return based on relevance.
+ The search retrieves up to 1000 hits from OpenSearch, then applies boosting and filters
+ the results to the top_k most relevant matches.
+ If not provided, the top_k provided in `__init__` is used.
+- **exact_match_weight** (float | None) – Weight to boost the score of exact matches in metadata fields.
+ If not provided, the exact_match_weight provided in `__init__` is used.
+- **mode** (Literal['strict', 'fuzzy'] | None) – Search mode. "strict" uses prefix and wildcard matching,
+ "fuzzy" uses fuzzy matching with dis_max queries.
+ In both modes, results are scored using Jaccard similarity (n-gram based) via a Painless script.
+ If not provided, the mode provided in `__init__` is used.
+- **fuzziness** (int | Literal['AUTO'] | None) – Maximum allowed Damerau-Levenshtein distance (edit distance) for fuzzy matching.
+ Accepts an integer (e.g., 0, 1, 2) or "AUTO" which chooses based on term length.
+ Only applies when mode is "fuzzy". If not provided, the fuzziness provided in `__init__` is used.
+- **prefix_length** (int | None) – Number of leading characters that must match exactly before fuzzy matching applies.
+ Only applies when mode is "fuzzy". If not provided, the prefix_length provided in `__init__` is used.
+- **max_expansions** (int | None) – Maximum number of term variations the fuzzy query can generate.
+ Only applies when mode is "fuzzy". If not provided, the max_expansions provided in `__init__` is used.
+- **tie_breaker** (float | None) – Weight (0..1) for other matching clauses; boosts docs matching multiple clauses.
+ Only applies when mode is "fuzzy". If not provided, the tie_breaker provided in `__init__` is used.
+- **jaccard_n** (int | None) – N-gram size for Jaccard similarity scoring. If not provided, the jaccard_n from `__init__`
+ is used.
+- **filters** (dict\[str, Any\] | None) – Additional filters to apply to the search query.
+
+**Returns:**
+
+- dict\[str, list\[dict\[str, Any\]\]\] – A dictionary containing the top-k retrieved metadata results.
+
+Example:
+\`\`\`python
+from haystack import Document
+
+````
+# First, add a document with matching metadata to the store
+await store.write_documents_async([
+ Document(
+ content="Python programming guide",
+ meta={"category": "Python", "status": "active", "priority": 1}
+ )
+])
+
+retriever = OpenSearchMetadataRetriever(
+ document_store=store,
+ metadata_fields=["category", "status", "priority"]
+)
+result = await retriever.run_async(query="Python, active")
+# Returns: {"metadata": [{"category": "Python", "status": "active", "priority": 1}]}
+```
+````
+
+## haystack_integrations.components.retrievers.opensearch.open_search_hybrid_retriever
+
+### OpenSearchHybridRetriever
+
+A hybrid retriever that combines embedding-based and keyword-based retrieval from OpenSearch.
+
+Example usage:
+
+Make sure you have "sentence-transformers>=3.0.0":
+
+```
+pip install haystack-ai datasets "sentence-transformers>=3.0.0"
+```
+
+And OpenSearch running. You can run OpenSearch with Docker:
+
+```
+docker run -d --name opensearch-nosec -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node"
+-e "DISABLE_SECURITY_PLUGIN=true" opensearchproject/opensearch:2.12.0
+```
+
+```python
+from haystack import Document
+# Requires: pip install sentence-transformers-haystack
+from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder
+from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentEmbedder
+from haystack_integrations.components.retrievers.opensearch import OpenSearchHybridRetriever
+from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore
+
+# Initialize the document store
+doc_store = OpenSearchDocumentStore(
+ hosts=[""],
+ index="document_store",
+ embedding_dim=384,
+)
+
+# Create some sample documents
+docs = [
+ Document(content="Machine learning is a subset of artificial intelligence."),
+ Document(content="Deep learning is a subset of machine learning."),
+ Document(content="Natural language processing is a field of AI."),
+ Document(content="Reinforcement learning is a type of machine learning."),
+ Document(content="Supervised learning is a type of machine learning."),
+]
+
+# Embed the documents and add them to the document store
+doc_embedder = SentenceTransformersDocumentEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
+docs = doc_embedder.run(docs)
+doc_store.write_documents(docs['documents'])
+
+# Initialize some haystack text embedder, in this case the SentenceTransformersTextEmbedder
+embedder = SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
+
+# Initialize the hybrid retriever
+retriever = OpenSearchHybridRetriever(
+ document_store=doc_store,
+ embedder=embedder,
+ top_k_bm25=3,
+ top_k_embedding=3,
+ join_mode="reciprocal_rank_fusion"
+)
+
+# Run the retriever
+results = retriever.run(query="What is reinforcement learning?", filters_bm25=None, filters_embedding=None)
+
+>> results['documents']
+{'documents': [Document(id=..., content: 'Reinforcement learning is a type of machine learning.', score: 1.0),
+ Document(id=..., content: 'Supervised learning is a type of machine learning.', score: 0.9760624679979518),
+ Document(id=..., content: 'Deep learning is a subset of machine learning.', score: 0.4919354838709677),
+ Document(id=..., content: 'Machine learning is a subset of artificial intelligence.', score: 0.4841269841269841)]}
+```
+
+#### __init__
+
+```python
+__init__(
+ document_store: OpenSearchDocumentStore,
+ *,
+ embedder: TextEmbedder,
+ filters_bm25: dict[str, Any] | None = None,
+ fuzziness: int | str = 0,
+ top_k_bm25: int = 10,
+ scale_score: bool = False,
+ all_terms_must_match: bool = False,
+ filter_policy_bm25: str | FilterPolicy = FilterPolicy.REPLACE,
+ custom_query_bm25: dict[str, Any] | None = None,
+ filters_embedding: dict[str, Any] | None = None,
+ top_k_embedding: int = 10,
+ filter_policy_embedding: str | FilterPolicy = FilterPolicy.REPLACE,
+ custom_query_embedding: dict[str, Any] | None = None,
+ search_kwargs_embedding: dict[str, Any] | None = None,
+ join_mode: str | JoinMode = JoinMode.RECIPROCAL_RANK_FUSION,
+ weights: list[float] | None = None,
+ top_k: int | None = None,
+ sort_by_score: bool = True,
+ **kwargs: Any
+) -> None
+```
+
+Initialize the OpenSearchHybridRetriever using both embedding-based and keyword-based retrieval methods.
+
+This is a super component to retrieve documents from OpenSearch using both retrieval methods.
+
+We don't explicitly define all the init parameters of the components in the constructor, for each
+of the components, since that would be around 20+ parameters. Instead, we define the most important ones
+and pass the rest as kwargs. This is to keep the constructor clean and easy to read.
+
+If you need to pass extra parameters to the components, you can do so by passing them as kwargs. It expects
+a dictionary with the component name as the key and the parameters as the value. The component name should be:
+
+```
+- "bm25_retriever" -> OpenSearchBM25Retriever
+- "embedding_retriever" -> OpenSearchEmbeddingRetriever
+```
+
+**Parameters:**
+
+- **document_store** (OpenSearchDocumentStore) – The OpenSearchDocumentStore to use for retrieval.
+- **embedder** (TextEmbedder) – A TextEmbedder to use for embedding the query.
+ See `haystack.components.embedders.types.protocol.TextEmbedder` for more information.
+- **filters_bm25** (dict\[str, Any\] | None) – Filters for the BM25 retriever.
+- **fuzziness** (int | str) – The fuzziness for the BM25 retriever.
+- **top_k_bm25** (int) – The number of results to return from the BM25 retriever.
+- **scale_score** (bool) – Whether to scale the score for the BM25 retriever.
+- **all_terms_must_match** (bool) – Whether all terms must match for the BM25 retriever.
+- **filter_policy_bm25** (str | FilterPolicy) – The filter policy for the BM25 retriever.
+- **custom_query_bm25** (dict\[str, Any\] | None) – A custom query for the BM25 retriever.
+- **filters_embedding** (dict\[str, Any\] | None) – Filters for the embedding retriever.
+- **top_k_embedding** (int) – The number of results to return from the embedding retriever.
+- **filter_policy_embedding** (str | FilterPolicy) – The filter policy for the embedding retriever.
+- **custom_query_embedding** (dict\[str, Any\] | None) – A custom query for the embedding retriever.
+- **search_kwargs_embedding** (dict\[str, Any\] | None) – Additional search kwargs for the embedding retriever.
+- **join_mode** (str | JoinMode) – The mode to use for joining the results from the BM25 and embedding retrievers.
+- **weights** (list\[float\] | None) – The weights for the joiner.
+- **top_k** (int | None) – The number of results to return from the joiner.
+- **sort_by_score** (bool) – Whether to sort the results by score.
+- \*\***kwargs** (Any) – Additional keyword arguments. Use the following keys to pass extra parameters to the retrievers:
+- "bm25_retriever" -> OpenSearchBM25Retriever
+- "embedding_retriever" -> OpenSearchEmbeddingRetriever
+
+#### warm_up
+
+```python
+warm_up() -> None
+```
+
+Warm up the underlying pipeline components.
+
+#### run
+
+```python
+run(
+ query: str,
+ filters_bm25: dict[str, Any] | None = None,
+ filters_embedding: dict[str, Any] | None = None,
+ top_k_bm25: int | None = None,
+ top_k_embedding: int | None = None,
+) -> dict[str, list[Document]]
+```
+
+Run the hybrid retrieval pipeline and return retrieved documents.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serialize OpenSearchHybridRetriever to a dictionary.
+
+**Returns:**
+
+- dict\[str, Any\] – Dictionary with serialized data.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> OpenSearchHybridRetriever
+```
+
+Deserialize an OpenSearchHybridRetriever from a dictionary.
+
+#### close
+
+```python
+close() -> None
+```
+
+Release the synchronous resources of the underlying Document Store.
+
+#### close_async
+
+```python
+close_async() -> None
+```
+
+Release the asynchronous resources of the underlying Document Store.
+
+## haystack_integrations.components.retrievers.opensearch.sql_retriever
+
+### OpenSearchSQLRetriever
+
+Executes raw OpenSearch SQL queries against an OpenSearchDocumentStore.
+
+This component allows you to execute SQL queries directly against the OpenSearch index,
+which is useful for fetching metadata, aggregations, and other structured data at runtime.
+
+Returns the raw JSON response from the OpenSearch SQL API.
+
+#### __init__
+
+```python
+__init__(
+ *,
+ document_store: OpenSearchDocumentStore,
+ raise_on_failure: bool = True,
+ fetch_size: int | None = None
+) -> None
+```
+
+Creates the OpenSearchSQLRetriever component.
+
+**Parameters:**
+
+- **document_store** (OpenSearchDocumentStore) – An instance of OpenSearchDocumentStore to use with the Retriever.
+- **raise_on_failure** (bool) – Whether to raise an exception if the API call fails. Otherwise, log a warning and return None.
+- **fetch_size** (int | None) – Optional number of results to fetch per page. If not provided, the default
+ fetch size set in OpenSearch is used.
+
+**Raises:**
+
+- ValueError – If `document_store` is not an instance of OpenSearchDocumentStore.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serializes the component to a dictionary.
+
+**Returns:**
+
+- dict\[str, Any\] – Dictionary with serialized data.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> OpenSearchSQLRetriever
+```
+
+Deserializes the component from a dictionary.
+
+**Parameters:**
+
+- **data** (dict\[str, Any\]) – Dictionary to deserialize from.
+
+**Returns:**
+
+- OpenSearchSQLRetriever – Deserialized component.
+
+#### close
+
+```python
+close() -> None
+```
+
+Release the synchronous resources of the underlying Document Store.
+
+#### close_async
+
+```python
+close_async() -> None
+```
+
+Release the asynchronous resources of the underlying Document Store.
+
+#### run
+
+```python
+run(
+ query: str,
+ document_store: OpenSearchDocumentStore | None = None,
+ fetch_size: int | None = None,
+) -> dict[str, dict[str, Any]]
+```
+
+Execute a raw OpenSearch SQL query against the index.
+
+**Parameters:**
+
+- **query** (str) – The OpenSearch SQL query to execute.
+- **document_store** (OpenSearchDocumentStore | None) – Optionally, an instance of OpenSearchDocumentStore to use with the Retriever.
+- **fetch_size** (int | None) – Optional number of results to fetch per page. If not provided, uses the value
+ specified during initialization, or the default fetch size set in OpenSearch.
+
+**Returns:**
+
+- dict\[str, dict\[str, Any\]\] – A dictionary containing the raw JSON response from OpenSearch SQL API:
+ - result: The raw JSON response from OpenSearch (dict) or None on error.
+
+Example:
+`python retriever = OpenSearchSQLRetriever(document_store=document_store) result = retriever.run( query="SELECT content, category FROM my_index WHERE category = 'A'" ) # result["result"] contains the raw OpenSearch JSON response # For regular queries: result["result"]["hits"]["hits"] contains documents # For aggregate queries: result["result"]["aggregations"] contains aggregations `
+
+#### run_async
+
+```python
+run_async(
+ query: str,
+ document_store: OpenSearchDocumentStore | None = None,
+ fetch_size: int | None = None,
+) -> dict[str, dict[str, Any]]
+```
+
+Asynchronously execute a raw OpenSearch SQL query against the index.
+
+**Parameters:**
+
+- **query** (str) – The OpenSearch SQL query to execute.
+- **document_store** (OpenSearchDocumentStore | None) – Optionally, an instance of OpenSearchDocumentStore to use with the Retriever.
+- **fetch_size** (int | None) – Optional number of results to fetch per page. If not provided, uses the value
+ specified during initialization, or the default fetch size set in OpenSearch.
+
+**Returns:**
+
+- dict\[str, dict\[str, Any\]\] – A dictionary containing the raw JSON response from OpenSearch SQL API:
+ - result: The raw JSON response from OpenSearch (dict) or None on error.
+
+Example:
+`python retriever = OpenSearchSQLRetriever(document_store=document_store) result = await retriever.run_async( query="SELECT content, category FROM my_index WHERE category = 'A'" ) # result["result"] contains the raw OpenSearch JSON response # For regular queries: result["result"]["hits"]["hits"] contains documents # For aggregate queries: result["result"]["aggregations"] contains aggregations `
+
+## haystack_integrations.document_stores.opensearch.document_store
+
+### OpenSearchDocumentStore
+
+An instance of an OpenSearch database you can use to store all types of data.
+
+This document store is a thin wrapper around the OpenSearch client.
+It allows you to store and retrieve documents from an OpenSearch index.
+
+Usage example:
+
+```python
+from haystack_integrations.document_stores.opensearch import (
+ OpenSearchDocumentStore,
+)
+from haystack import Document
+
+document_store = OpenSearchDocumentStore(hosts="localhost:9200")
+
+document_store.write_documents(
+ [
+ Document(content="My first document", id="1"),
+ Document(content="My second document", id="2"),
+ ]
+)
+
+print(document_store.count_documents())
+# 2
+
+print(document_store.filter_documents())
+# [Document(id='1', content='My first document', ...), Document(id='2', content='My second document', ...)]
+```
+
+#### __init__
+
+```python
+__init__(
+ *,
+ hosts: Hosts | None = None,
+ index: str = "default",
+ max_chunk_bytes: int = DEFAULT_MAX_CHUNK_BYTES,
+ embedding_dim: int = 768,
+ return_embedding: bool = False,
+ method: dict[str, Any] | None = None,
+ mappings: dict[str, Any] | None = None,
+ settings: dict[str, Any] | None = DEFAULT_SETTINGS,
+ create_index: bool = True,
+ http_auth: (
+ tuple[Secret, Secret]
+ | tuple[str, str]
+ | list[str]
+ | str
+ | AWSAuth
+ | None
+ ) = (
+ Secret.from_env_var("OPENSEARCH_USERNAME", strict=False),
+ Secret.from_env_var("OPENSEARCH_PASSWORD", strict=False),
+ ),
+ use_ssl: bool | None = None,
+ verify_certs: bool | None = None,
+ timeout: int | None = None,
+ nested_fields: list[str] | Literal["*"] | None = None,
+ **kwargs: Any
+) -> None
+```
+
+Creates a new OpenSearchDocumentStore instance.
+
+The `embeddings_dim`, `method`, `mappings`, and `settings` arguments are only used if the index does not
+exist and needs to be created. If the index already exists, its current configurations will be used.
+
+For more information on connection parameters, see the [official OpenSearch documentation](https://opensearch.org/docs/latest/clients/python-low-level/#connecting-to-opensearch)
+
+**Parameters:**
+
+- **hosts** (Hosts | None) – List of hosts running the OpenSearch client. Defaults to None
+- **index** (str) – Name of index in OpenSearch, if it doesn't exist it will be created. Defaults to "default"
+- **max_chunk_bytes** (int) – Maximum size of the requests in bytes. Defaults to 100MB
+- **embedding_dim** (int) – Dimension of the embeddings. Defaults to 768
+- **return_embedding** (bool) – Whether to return the embedding of the retrieved Documents. This parameter also applies to the
+ `filter_documents` and `filter_documents_async` methods.
+- **method** (dict\[str, Any\] | None) – The method definition of the underlying configuration of the approximate k-NN algorithm. Please
+ see the [official OpenSearch docs](https://opensearch.org/docs/latest/search-plugins/knn/knn-index/#method-definitions)
+ for more information. Defaults to None
+- **mappings** (dict\[str, Any\] | None) – The mapping of how the documents are stored and indexed. Please see the [official OpenSearch docs](https://opensearch.org/docs/latest/field-types/)
+ for more information. If None, it uses the embedding_dim and method arguments to create default mappings.
+ Defaults to None
+- **settings** (dict\[str, Any\] | None) – The settings of the index to be created. Please see the [official OpenSearch docs](https://opensearch.org/docs/latest/search-plugins/knn/knn-index/#index-settings)
+ for more information. Defaults to `{"index.knn": True}`.
+- **create_index** (bool) – Whether to create the index if it doesn't exist. Defaults to True
+- **http_auth** (tuple\[Secret, Secret\] | tuple\[str, str\] | list\[str\] | str | AWSAuth | None) – http_auth param passed to the underlying connection class.
+ For basic authentication with default connection class `Urllib3HttpConnection` this can be
+- a tuple of (username, password)
+- a list of [username, password]
+- a string of "username:password"
+ If not provided, will read values from OPENSEARCH_USERNAME and OPENSEARCH_PASSWORD environment variables.
+ For AWS authentication with `Urllib3HttpConnection` pass an instance of `AWSAuth`.
+ Defaults to None
+- **use_ssl** (bool | None) – Whether to use SSL. Defaults to None
+- **verify_certs** (bool | None) – Whether to verify certificates. Defaults to None
+- **timeout** (int | None) – Timeout in seconds. Defaults to None
+- **nested_fields** (list\[str\] | Literal['\*'] | None) – List of metadata field paths (without the `meta.` prefix) that should be mapped
+ as OpenSearch `nested` type, enabling multi-condition filtering on array-of-objects fields.
+ Pass `"*"` to auto-detect `list[dict]` fields and map them as nested from
+ the first `write_documents` batch.
+ When the index already exists, nested fields are discovered from the live mapping.
+ Defaults to None (no nested support).
+- \*\***kwargs** (Any) – Optional arguments that `OpenSearch` takes. For the full list of supported kwargs,
+ see the [official OpenSearch reference](https://opensearch-project.github.io/opensearch-py/api-ref/clients/opensearch_client.html)
+
+#### create_index
+
+```python
+create_index(
+ index: str | None = None,
+ mappings: dict[str, Any] | None = None,
+ settings: dict[str, Any] | None = None,
+) -> None
+```
+
+Creates an index in OpenSearch.
+
+Note that this method ignores the `create_index` argument from the constructor.
+
+**Parameters:**
+
+- **index** (str | None) – Name of the index to create. If None, the index name from the constructor is used.
+- **mappings** (dict\[str, Any\] | None) – The mapping of how the documents are stored and indexed. Please see the [official OpenSearch docs](https://opensearch.org/docs/latest/field-types/)
+ for more information. If None, the mappings from the constructor are used.
+- **settings** (dict\[str, Any\] | None) – The settings of the index to be created. Please see the [official OpenSearch docs](https://opensearch.org/docs/latest/search-plugins/knn/knn-index/#index-settings)
+ for more information. If None, the settings from the constructor are used.
+
+#### to_dict
+
+```python
+to_dict() -> dict[str, Any]
+```
+
+Serializes the component to a dictionary.
+
+**Returns:**
+
+- dict\[str, Any\] – Dictionary with serialized data.
+
+#### from_dict
+
+```python
+from_dict(data: dict[str, Any]) -> OpenSearchDocumentStore
+```
+
+Deserializes the component from a dictionary.
+
+**Parameters:**
+
+- **data** (dict\[str, Any\]) – Dictionary to deserialize from.
+
+**Returns:**
+
+- OpenSearchDocumentStore – Deserialized component.
+
+#### close
+
+```python
+close() -> None
+```
+
+Release the associated synchronous resources.
+
+#### close_async
+
+```python
+close_async() -> None
+```
+
+Release the associated asynchronous resources.
+
+#### count_documents
+
+```python
+count_documents() -> int
+```
+
+Returns how many documents are present in the document store.
+
+#### count_documents_async
+
+```python
+count_documents_async() -> int
+```
+
+Asynchronously returns the total number of documents in the document store.
+
+#### filter_documents
+
+```python
+filter_documents(filters: dict[str, Any] | None = None) -> list[Document]
+```
+
+Returns the documents that match the filters provided.
+
+For a detailed specification of the filters,
+refer to the [documentation](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\] | None) – The filters to apply to the document list.
+
+**Returns:**
+
+- list\[Document\] – A list of Documents that match the given filters.
+
+#### filter_documents_async
+
+```python
+filter_documents_async(filters: dict[str, Any] | None = None) -> list[Document]
+```
+
+Asynchronously returns the documents that match the filters provided.
+
+For a detailed specification of the filters,
+refer to the [documentation](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\] | None) – The filters to apply to the document list.
+
+**Returns:**
+
+- list\[Document\] – A list of Documents that match the given filters.
+
+#### write_documents
+
+```python
+write_documents(
+ documents: list[Document],
+ policy: DuplicatePolicy = DuplicatePolicy.NONE,
+ refresh: Literal["wait_for", True, False] = "wait_for",
+) -> int
+```
+
+Writes documents to the document store.
+
+**Parameters:**
+
+- **documents** (list\[Document\]) – A list of Documents to write to the document store.
+- **policy** (DuplicatePolicy) – The duplicate policy to use when writing documents.
+- **refresh** (Literal['wait_for', True, False]) – Controls when changes are made visible to search operations.
+- `True`: Force refresh immediately after the operation.
+- `False`: Do not refresh (better performance for bulk operations).
+- `"wait_for"`: Wait for the next refresh cycle (default, ensures read-your-writes consistency).
+ For more details, see the [OpenSearch refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/index-document/).
+
+**Returns:**
+
+- int – The number of documents written to the document store.
+
+**Raises:**
+
+- DuplicateDocumentError – If a document with the same id already exists in the document store
+ and the policy is set to `DuplicatePolicy.FAIL` (or not specified).
+
+#### write_documents_async
+
+```python
+write_documents_async(
+ documents: list[Document],
+ policy: DuplicatePolicy = DuplicatePolicy.NONE,
+ refresh: Literal["wait_for", True, False] = "wait_for",
+) -> int
+```
+
+Asynchronously writes documents to the document store.
+
+**Parameters:**
+
+- **documents** (list\[Document\]) – A list of Documents to write to the document store.
+- **policy** (DuplicatePolicy) – The duplicate policy to use when writing documents.
+- **refresh** (Literal['wait_for', True, False]) – Controls when changes are made visible to search operations.
+- `True`: Force refresh immediately after the operation.
+- `False`: Do not refresh (better performance for bulk operations).
+- `"wait_for"`: Wait for the next refresh cycle (default, ensures read-your-writes consistency).
+ For more details, see the [OpenSearch refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/index-document/).
+
+**Returns:**
+
+- int – The number of documents written to the document store.
+
+#### delete_documents
+
+```python
+delete_documents(
+ document_ids: list[str],
+ refresh: Literal["wait_for", True, False] = "wait_for",
+ routing: dict[str, str] | None = None,
+) -> None
+```
+
+Deletes documents that match the provided `document_ids` from the document store.
+
+**Parameters:**
+
+- **document_ids** (list\[str\]) – the document ids to delete
+- **refresh** (Literal['wait_for', True, False]) – Controls when changes are made visible to search operations.
+- `True`: Force refresh immediately after the operation.
+- `False`: Do not refresh (better performance for bulk operations).
+- `"wait_for"`: Wait for the next refresh cycle (default, ensures read-your-writes consistency).
+ For more details, see the [OpenSearch refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/index-document/).
+- **routing** (dict\[str, str\] | None) – A dictionary mapping document IDs to their routing values.
+ Routing values are used to determine the shard where documents are stored.
+ If provided, the routing value for each document will be used during deletion.
+
+#### delete_documents_async
+
+```python
+delete_documents_async(
+ document_ids: list[str],
+ refresh: Literal["wait_for", True, False] = "wait_for",
+ routing: dict[str, str] | None = None,
+) -> None
+```
+
+Asynchronously deletes documents that match the provided `document_ids` from the document store.
+
+**Parameters:**
+
+- **document_ids** (list\[str\]) – the document ids to delete
+- **refresh** (Literal['wait_for', True, False]) – Controls when changes are made visible to search operations.
+- `True`: Force refresh immediately after the operation.
+- `False`: Do not refresh (better performance for bulk operations).
+- `"wait_for"`: Wait for the next refresh cycle (default, ensures read-your-writes consistency).
+ For more details, see the [OpenSearch refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/index-document/).
+- **routing** (dict\[str, str\] | None) – A dictionary mapping document IDs to their routing values.
+ Routing values are used to determine the shard where documents are stored.
+ If provided, the routing value for each document will be used during deletion.
+
+#### delete_all_documents
+
+```python
+delete_all_documents(
+ recreate_index: bool = False, refresh: bool = True
+) -> None
+```
+
+Deletes all documents in the document store.
+
+**Parameters:**
+
+- **recreate_index** (bool) – If True, the index will be deleted and recreated with the original mappings and
+ settings. If False, all documents will be deleted using the `delete_by_query` API.
+ `recreate_index=True` is not supported when the configured index name is an alias; a
+ :class:`haystack.document_stores.errors.DocumentStoreError` is raised in that case.
+- **refresh** (bool) – If True, OpenSearch refreshes all shards involved in the delete by query after the request
+ completes. If False, no refresh is performed. For more details, see the
+ [OpenSearch delete_by_query refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/delete-by-query/).
+
+#### delete_all_documents_async
+
+```python
+delete_all_documents_async(
+ recreate_index: bool = False, refresh: bool = True
+) -> None
+```
+
+Asynchronously deletes all documents in the document store.
+
+**Parameters:**
+
+- **recreate_index** (bool) – If True, the index will be deleted and recreated with the original mappings and
+ settings. If False, all documents will be deleted using the `delete_by_query` API.
+ `recreate_index=True` is not supported when the configured index name is an alias; a
+ :class:`haystack.document_stores.errors.DocumentStoreError` is raised in that case.
+- **refresh** (bool) – If True, OpenSearch refreshes all shards involved in the delete by query after the request
+ completes. If False, no refresh is performed. For more details, see the
+ [OpenSearch delete_by_query refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/delete-by-query/).
+
+#### delete_by_filter
+
+```python
+delete_by_filter(filters: dict[str, Any], refresh: bool = False) -> int
+```
+
+Deletes all documents that match the provided filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to select documents for deletion.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+- **refresh** (bool) – If True, OpenSearch refreshes all shards involved in the delete by query after the request
+ completes so that subsequent reads (e.g. count_documents) see the update. If False, no refresh is
+ performed (better for bulk deletes). For more details, see the
+ [OpenSearch delete_by_query refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/delete-by-query/).
+
+**Returns:**
+
+- int – The number of documents deleted.
+
+#### delete_by_filter_async
+
+```python
+delete_by_filter_async(filters: dict[str, Any], refresh: bool = False) -> int
+```
+
+Asynchronously deletes all documents that match the provided filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to select documents for deletion.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+- **refresh** (bool) – If True, OpenSearch refreshes all shards involved in the delete by query after the request
+ completes so that subsequent reads see the update. If False, no refresh is performed. For more details,
+ see the [OpenSearch delete_by_query refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/delete-by-query/).
+
+**Returns:**
+
+- int – The number of documents deleted.
+
+#### update_by_filter
+
+```python
+update_by_filter(
+ filters: dict[str, Any], meta: dict[str, Any], refresh: bool = False
+) -> int
+```
+
+Updates the metadata of all documents that match the provided filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to select documents for updating.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+- **meta** (dict\[str, Any\]) – The metadata fields to update.
+- **refresh** (bool) – If True, OpenSearch refreshes all shards involved in the update by query after the request
+ completes. If False, no refresh is performed. For more details, see the
+ [OpenSearch update_by_query refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/update-by-query/).
+
+**Returns:**
+
+- int – The number of documents updated.
+
+#### update_by_filter_async
+
+```python
+update_by_filter_async(
+ filters: dict[str, Any], meta: dict[str, Any], refresh: bool = False
+) -> int
+```
+
+Asynchronously updates the metadata of all documents that match the provided filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to select documents for updating.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+- **meta** (dict\[str, Any\]) – The metadata fields to update.
+- **refresh** (bool) – If True, OpenSearch refreshes all shards involved in the update by query after the request
+ completes. If False, no refresh is performed. For more details, see the
+ [OpenSearch update_by_query refresh documentation](https://opensearch.org/docs/latest/api-reference/document-apis/update-by-query/).
+
+**Returns:**
+
+- int – The number of documents updated.
+
+#### count_documents_by_filter
+
+```python
+count_documents_by_filter(filters: dict[str, Any]) -> int
+```
+
+Returns the number of documents that match the provided filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to count documents.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+
+**Returns:**
+
+- int – The number of documents that match the filters.
+
+#### count_documents_by_filter_async
+
+```python
+count_documents_by_filter_async(filters: dict[str, Any]) -> int
+```
+
+Asynchronously returns the number of documents that match the provided filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to count documents.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+
+**Returns:**
+
+- int – The number of documents that match the filters.
+
+#### count_unique_metadata_by_filter
+
+```python
+count_unique_metadata_by_filter(
+ filters: dict[str, Any], metadata_fields: list[str]
+) -> dict[str, int]
+```
+
+Returns the number of unique values for each specified metadata field of the documents that match the filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to count documents.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+- **metadata_fields** (list\[str\]) – List of field names to calculate unique values for.
+ Field names can include or omit the "meta." prefix.
+
+**Returns:**
+
+- dict\[str, int\] – A dictionary mapping each metadata field name to the count of its unique values among the filtered
+ documents.
+
+**Raises:**
+
+- ValueError – If any of the requested fields don't exist in the index mapping.
+
+#### count_unique_metadata_by_filter_async
+
+```python
+count_unique_metadata_by_filter_async(
+ filters: dict[str, Any], metadata_fields: list[str]
+) -> dict[str, int]
+```
+
+Asynchronously returns the number of unique values for each specified metadata field matching the filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – The filters to apply to count documents.
+ For filter syntax, see [Haystack metadata filtering](https://docs.haystack.deepset.ai/docs/metadata-filtering)
+- **metadata_fields** (list\[str\]) – List of field names to calculate unique values for.
+ Field names can include or omit the "meta." prefix.
+
+**Returns:**
+
+- dict\[str, int\] – A dictionary mapping each metadata field name to the count of its unique values among the filtered
+ documents.
+
+**Raises:**
+
+- ValueError – If any of the requested fields don't exist in the index mapping.
+
+#### get_metadata_fields_info
+
+```python
+get_metadata_fields_info() -> dict[str, dict[str, str]]
+```
+
+Returns the information about the fields in the index.
+
+If we populated the index with documents like:
+
+```python
+ Document(content="Doc 1", meta={"category": "A", "status": "active", "priority": 1})
+ Document(content="Doc 2", meta={"category": "B", "status": "inactive"})
+```
+
+This method would return:
+
+```python
+ {
+ 'content': {'type': 'text'},
+ 'category': {'type': 'keyword'},
+ 'status': {'type': 'keyword'},
+ 'priority': {'type': 'long'},
+ }
+```
+
+**Returns:**
+
+- dict\[str, dict\[str, str\]\] – The information about the fields in the index.
+
+#### get_metadata_fields_info_async
+
+```python
+get_metadata_fields_info_async() -> dict[str, dict[str, str]]
+```
+
+Asynchronously returns the information about the fields in the index.
+
+If we populated the index with documents like:
+
+```python
+ Document(content="Doc 1", meta={"category": "A", "status": "active", "priority": 1})
+ Document(content="Doc 2", meta={"category": "B", "status": "inactive"})
+```
+
+This method would return:
+
+```python
+ {
+ 'content': {'type': 'text'},
+ 'category': {'type': 'keyword'},
+ 'status': {'type': 'keyword'},
+ 'priority': {'type': 'long'},
+ }
+```
+
+**Returns:**
+
+- dict\[str, dict\[str, str\]\] – The information about the fields in the index.
+
+#### get_metadata_field_min_max
+
+```python
+get_metadata_field_min_max(metadata_field: str) -> dict[str, int | None]
+```
+
+Returns the minimum and maximum values for the given metadata field.
+
+**Parameters:**
+
+- **metadata_field** (str) – The metadata field to get the minimum and maximum values for.
+
+**Returns:**
+
+- dict\[str, int | None\] – A dictionary with the keys "min" and "max", where each value is the minimum or maximum value of the
+ metadata field across all documents.
+
+#### get_metadata_field_min_max_async
+
+```python
+get_metadata_field_min_max_async(metadata_field: str) -> dict[str, int | None]
+```
+
+Asynchronously returns the minimum and maximum values for the given metadata field.
+
+**Parameters:**
+
+- **metadata_field** (str) – The metadata field to get the minimum and maximum values for.
+
+**Returns:**
+
+- dict\[str, int | None\] – A dictionary with the keys "min" and "max", where each value is the minimum or maximum value of the
+ metadata field across all documents.
+
+#### get_metadata_field_unique_values
+
+```python
+get_metadata_field_unique_values(
+ metadata_field: str,
+ search_term: str | None = None,
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
+```
+
+Returns unique values for a metadata field, optionally filtered by a search term.
+
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
+
+**Parameters:**
+
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
+
+**Returns:**
+
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
+
+#### get_metadata_field_unique_values_async
+
+```python
+get_metadata_field_unique_values_async(
+ metadata_field: str,
+ search_term: str | None = None,
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
+```
+
+Asynchronous counterpart of `get_metadata_field_unique_values`.
+
+**Parameters:**
+
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
+
+**Returns:**
+
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
+
+## haystack_integrations.document_stores.opensearch.filters
+
+### normalize_filters
+
+```python
+normalize_filters(
+ filters: dict[str, Any], nested_fields: set[str] | None = None
+) -> dict[str, Any]
+```
+
+Converts Haystack filters in OpenSearch compatible filters.
+
+**Parameters:**
+
+- **filters** (dict\[str, Any\]) – Haystack filter dictionary.
+- **nested_fields** (set\[str\] | None) – Set of metadata field paths that are mapped as `nested` type in OpenSearch.
+ When provided, conditions targeting sub-fields of these paths are wrapped in `nested` queries.
diff --git a/docs-website/reference_versioned_docs/version-3.2/integrations-api/opensearch.md b/docs-website/reference_versioned_docs/version-3.2/integrations-api/opensearch.md
index d0497f5b9e..ed25053261 100644
--- a/docs-website/reference_versioned_docs/version-3.2/integrations-api/opensearch.md
+++ b/docs-website/reference_versioned_docs/version-3.2/integrations-api/opensearch.md
@@ -1888,31 +1888,42 @@ Asynchronously returns the minimum and maximum values for the given metadata fie
get_metadata_field_unique_values(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
Returns unique values for a metadata field, optionally filtered by a search term.
-Uses composite aggregations for proper pagination beyond 10k results.
+Internally still backed by composite aggregations, which only support cursor-based iteration.
+Reaching offset `from_` therefore requires walking and discarding the first `from_` buckets -
+cost scales with `from_`, not `size`.
+
+**Note**: To keep this signature uniform across document stores, offset-based pagination is
+emulated on top of the cursor by re-fetching and discarding every bucket before `from_` on each
+call, requiring additional search round-trips proportional to `from_`.
+Performance may degrade for high-cardinality metadata fields and large from\_ values.
+
+**Note**: `total_count` is computed via an approximate cardinality aggregation; for fields with
+very high cardinality it may not be exact.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
#### get_metadata_field_unique_values_async
@@ -1920,31 +1931,30 @@ Uses composite aggregations for proper pagination beyond 10k results.
get_metadata_field_unique_values_async(
metadata_field: str,
search_term: str | None = None,
- size: int | None = 10000,
- after: dict[str, Any] | None = None,
-) -> tuple[list[str], dict[str, Any] | None]
+ from_: int = 0,
+ size: int = 10,
+ filters: dict[str, Any] | None = None,
+) -> tuple[list[Any], int]
```
-Asynchronously returns unique values for a metadata field, optionally filtered by a search term.
-
-Uses composite aggregations for proper pagination beyond 10k results.
+Asynchronous counterpart of `get_metadata_field_unique_values`.
**Parameters:**
-- **metadata_field** (str) – The metadata field to get unique values for.
-- **search_term** (str | None) – Optional term to filter the returned values by, matching as a case-insensitive substring
- of the metadata field's own value (not the document content). NOTE: The matching is done with a server-side
- script to accomplish the substring matching on the value of the field and this operation is quite expensive
- for a large corpus.
-- **size** (int | None) – The number of unique values to return per page. Defaults to 10000.
-- **after** (dict\[str, Any\] | None) – Optional pagination key from the previous response. Use None for the first page.
- For subsequent pages, pass the `after_key` from the previous response.
+- **metadata_field** (str) – The metadata field to get unique values for. Can include or omit the
+ "meta." prefix.
+- **search_term** (str | None) – Optional case-insensitive substring to filter the returned values by, matched
+ against the metadata field's own value (not the document content). NOTE: The matching is done
+ with a server-side script and is quite expensive for a large corpus.
+- **from\_** (int) – Offset to start returning values from. Defaults to 0.
+- **size** (int) – The number of unique values to return per page. Defaults to 10.
+- **filters** (dict\[str, Any\] | None) – Optional filters to restrict the documents considered.
**Returns:**
-- tuple\[list\[str\], dict\[str, Any\] | None\] – A tuple containing (list of unique values, after_key for pagination).
- The after_key is None when there are no more results. Use it in the `after` parameter
- for the next page.
+- tuple\[list\[Any\], int\] – A tuple of (list of unique values in their original type, total count of distinct values
+ for the field matching `search_term`). Note that filters also narrows down the number of documents
+ against which the search term is matched.
## haystack_integrations.document_stores.opensearch.filters