diff --git a/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/amazon_bedrock.md b/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/amazon_bedrock.md new file mode 100644 index 0000000000..5a7c4eb42e --- /dev/null +++ b/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/amazon_bedrock.md @@ -0,0 +1,1673 @@ +--- +title: "Amazon Bedrock" +id: integrations-amazon-bedrock +description: "Amazon Bedrock integration for Haystack" +slug: "/integrations-amazon-bedrock" +--- + + +## haystack_integrations.common.amazon_bedrock.errors + +### AmazonBedrockError + +Bases: Exception + +Any error generated by the Amazon Bedrock integration. + +This error wraps its source transparently in such a way that its attributes +can be accessed directly: for example, if the original error has a `message` attribute, +`AmazonBedrockError.message` will exist and have the expected content. + +### AWSConfigurationError + +Bases: AmazonBedrockError + +Exception raised when AWS is not configured correctly + +### AmazonBedrockConfigurationError + +Bases: AmazonBedrockError + +Exception raised when AmazonBedrock node is not configured correctly + +### AmazonBedrockInferenceError + +Bases: AmazonBedrockError + +Exception for issues that occur in the Bedrock inference node + +## haystack_integrations.common.amazon_bedrock.errors + +### AmazonBedrockError + +Bases: Exception + +Any error generated by the Amazon Bedrock integration. + +This error wraps its source transparently in such a way that its attributes +can be accessed directly: for example, if the original error has a `message` attribute, +`AmazonBedrockError.message` will exist and have the expected content. + +### AWSConfigurationError + +Bases: AmazonBedrockError + +Exception raised when AWS is not configured correctly + +### AmazonBedrockConfigurationError + +Bases: AmazonBedrockError + +Exception raised when AmazonBedrock node is not configured correctly + +### AmazonBedrockInferenceError + +Bases: AmazonBedrockError + +Exception for issues that occur in the Bedrock inference node + +## haystack_integrations.common.s3.errors + +### S3Error + +Bases: Exception + +Exception for issues that occur in the S3 based components + +### S3ConfigurationError + +Bases: S3Error + +Exception raised when AmazonS3 node is not configured correctly + +### S3StorageError + +Bases: S3Error + +This exception is raised when an error occurs while interacting with a S3Storage object. + +## haystack_integrations.common.s3.utils + +### S3Storage + +This class provides a storage class for downloading files from an AWS S3 bucket. + +#### __init__ + +```python +__init__( + s3_bucket: str, + session: Session, + s3_prefix: str | None = None, + endpoint_url: str | None = None, + config: Config | None = None, +) -> None +``` + +Initializes the S3Storage object with the provided parameters. + +**Parameters:** + +- **s3_bucket** (str) – The name of the S3 bucket to download files from. +- **session** (Session) – The session to use for the S3 client. +- **s3_prefix** (str | None) – The optional prefix of the files in the S3 bucket. + Can be used to specify folder or naming structure. + For example, if the file is in the folder "folder/subfolder/file.txt", + the s3_prefix should be "folder/subfolder/". If the file is in the root of the S3 bucket, + the s3_prefix should be None. +- **endpoint_url** (str | None) – The endpoint URL of the S3 bucket to download files from. +- **config** (Config | None) – The configuration to use for the S3 client. + +#### download + +```python +download(key: str, local_file_path: Path) -> None +``` + +Download a file from S3. + +**Parameters:** + +- **key** (str) – The key of the file to download. +- **local_file_path** (Path) – The folder path to download the file to. + It will be created if it does not exist. The file will be downloaded to + the folder with the same name as the key. + +**Raises:** + +- S3ConfigurationError – If the S3 session client cannot be created. +- S3StorageError – If the file does not exist in the S3 bucket + or the file cannot be downloaded. + +#### close + +```python +close() -> None +``` + +Close the S3 client owned by this storage instance. + +#### from_env + +```python +from_env( + *, + session: Session, + config: Config, + s3_bucket_name_env: str = "S3_DOWNLOADER_BUCKET" +) -> S3Storage +``` + +Create a S3Storage object from environment variables. + +The following environment variables are read: + +- `S3_DOWNLOADER_BUCKET` (or the value of `s3_bucket_name_env`): The name of the S3 bucket + to download files from. Required — raises `ValueError` if not set. +- `S3_DOWNLOADER_PREFIX`: Optional prefix to apply to all S3 keys (e.g. `"folder/subfolder/"`). +- `AWS_ENDPOINT_URL`: Optional custom endpoint URL, useful for S3-compatible services + such as MinIO or LocalStack. + +**Parameters:** + +- **session** (Session) – The boto3 `Session` to use when creating the S3 client. +- **config** (Config) – The botocore `Config` to apply to the S3 client. +- **s3_bucket_name_env** (str) – The name of the environment variable of the S3 bucket to download files from. + By default, the value is `"S3_DOWNLOADER_BUCKET"`. + +**Returns:** + +- S3Storage – A fully initialized `S3Storage` instance. + +**Raises:** + +- ValueError – If the environment variable specified by `s3_bucket_name_env` is not set + or is empty. + +## haystack_integrations.components.downloaders.s3.s3_downloader + +### S3Downloader + +A component for downloading files from AWS S3 Buckets to local filesystem. + +Supports filtering by file extensions. + +#### __init__ + +```python +__init__( + *, + aws_access_key_id: Secret | None = Secret.from_env_var( + "AWS_ACCESS_KEY_ID", strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + "AWS_SECRET_ACCESS_KEY", strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + "AWS_SESSION_TOKEN", strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + "AWS_DEFAULT_REGION", strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + "AWS_PROFILE", strict=False + ), + boto3_config: dict[str, Any] | None = None, + file_root_path: str | None = None, + file_extensions: list[str] | None = None, + file_name_meta_key: str = "file_name", + max_workers: int = 32, + max_cache_size: int = 100, + s3_key_generation_function: Callable[[Document], str] | None = None, + s3_bucket_name_env: str = "S3_DOWNLOADER_BUCKET" +) -> None +``` + +Initializes the `S3Downloader` with the provided parameters. + +Note that the AWS credentials are not required if the AWS environment is configured correctly. These are loaded +automatically from the environment or the AWS configuration file and do not need to be provided explicitly via +the constructor. If the AWS environment is not configured users need to provide the AWS credentials via the +constructor. Three required parameters are `aws_access_key_id`, `aws_secret_access_key`, +and `aws_region_name`. + +**Parameters:** + +- **aws_access_key_id** (Secret | None) – AWS access key ID. +- **aws_secret_access_key** (Secret | None) – AWS secret access key. +- **aws_session_token** (Secret | None) – AWS session token. +- **aws_region_name** (Secret | str | None) – AWS region name. +- **aws_profile_name** (Secret | None) – AWS profile name. +- **boto3_config** (dict\[str, Any\] | None) – Dictionary of configuration options for the underlying Boto3 client. + Can be used to tune [retry behavior](https://docs.aws.amazon.com/boto3/latest/guide/retries.html) + and other low-level settings like timeouts and connection management. +- **file_root_path** (str | None) – The path where the file will be downloaded. + Can be set through this parameter or the `FILE_ROOT_PATH` environment variable. + If none of them is set, a `ValueError` is raised. + Downloads are confined to this directory: a document whose file name resolves outside of it + (for example an absolute path or one containing `..`) is logged and skipped instead of written. +- **file_extensions** (list\[str\] | None) – The file extensions that are permitted to be downloaded. + By default, all file extensions are allowed. +- **max_workers** (int) – The maximum number of workers to use for concurrent downloads. +- **max_cache_size** (int) – The maximum number of files to cache. +- **file_name_meta_key** (str) – The name of the meta key that contains the file name to download. The file name + will also be used to create local file path for download, relative to `file_root_path`. + By default, the `Document.meta["file_name"]` is used. If you want to use a + different key in `Document.meta`, you can set it here. +- **s3_key_generation_function** (Callable\\[[Document\], str\] | None) – An optional function that generates the S3 key for the file to download. + If not provided, the default behavior is to use `Document.meta[file_name_meta_key]`. + The function must accept a `Document` object and return a string. + If the environment variable `S3_DOWNLOADER_PREFIX` is set, its value will be automatically + prefixed to the generated S3 key. +- **s3_bucket_name_env** (str) – The name of the environment variable of the S3 bucket to download files from. + By default, the value is `"S3_DOWNLOADER_BUCKET"`. + +**Raises:** + +- ValueError – If the `file_root_path` is not set through + the constructor or the `FILE_ROOT_PATH` environment variable. +- AWSConfigurationError – If the provided AWS credentials are invalid. + +#### warm_up + +```python +warm_up() -> None +``` + +Warm up the component by initializing the settings and storage. + +**Raises:** + +- ValueError – If the environment variable naming the S3 bucket (`s3_bucket_name_env`, by default + `S3_DOWNLOADER_BUCKET`) is not set. +- S3ConfigurationError – If the S3 client cannot be created. + +#### close + +```python +close() -> None +``` + +Close the owned S3 client. + +#### run + +```python +run(documents: list[Document]) -> dict[str, list[Document]] +``` + +Download files from AWS S3 Buckets to local filesystem. + +Return enriched `Document`s with the path of the downloaded file. + +**Parameters:** + +- **documents** (list\[Document\]) – Document containing the name of the file to download in the meta field. + +**Returns:** + +- dict\[str, list\[Document\]\] – A dictionary with: +- `documents`: The downloaded `Document`s; each has `meta['file_path']`. Documents whose file name + is missing, or resolves outside of `file_root_path`, are logged and skipped. + +**Raises:** + +- S3Error – If a download attempt fails or the file does not exist in the S3 bucket. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serialize the component to a dictionary. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> S3Downloader +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- S3Downloader – Deserialized component. + +## haystack_integrations.components.embedders.amazon_bedrock.document_embedder + +### AmazonBedrockDocumentEmbedder + +A component for computing Document embeddings using Amazon Bedrock. + +The embedding of each Document is stored in the `embedding` field of the Document. + +Usage example: + +```python +import os +from haystack.dataclasses import Document +from haystack_integrations.components.embedders.amazon_bedrock import ( + AmazonBedrockDocumentEmbedder, +) + +os.environ["AWS_ACCESS_KEY_ID"] = "..." +os.environ["AWS_SECRET_ACCESS_KEY_ID"] = "..." +os.environ["AWS_DEFAULT_REGION"] = "..." + +embedder = AmazonBedrockDocumentEmbedder( + model="cohere.embed-english-v3", + input_type="search_document", +) + +doc = Document(content="I love Paris in the winter.", meta={"name": "doc1"}) + +result = embedder.run([doc]) +print(result["documents"][0].embedding) + +# [0.002, 0.032, 0.504, ...] +``` + +#### __init__ + +```python +__init__( + model: str, + aws_access_key_id: Secret | None = Secret.from_env_var( + "AWS_ACCESS_KEY_ID", strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + "AWS_SECRET_ACCESS_KEY", strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + "AWS_SESSION_TOKEN", strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + "AWS_DEFAULT_REGION", strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + "AWS_PROFILE", strict=False + ), + batch_size: int = 32, + progress_bar: bool = True, + meta_fields_to_embed: list[str] | None = None, + embedding_separator: str = "\n", + boto3_config: dict[str, Any] | None = None, + **kwargs: Any +) -> None +``` + +Initializes the AmazonBedrockDocumentEmbedder with the provided parameters. + +The parameters are passed to the Amazon Bedrock client. + +Note that the AWS credentials are not required if the AWS environment is configured correctly. These are loaded +automatically from the environment or the AWS configuration file and do not need to be provided explicitly via +the constructor. If the AWS environment is not configured users need to provide the AWS credentials via the +constructor. Aside from model, three required parameters are `aws_access_key_id`, `aws_secret_access_key`, +and `aws_region_name`. + +**Parameters:** + +- **model** (str) – The embedding model to use. + Amazon Titan and Cohere embedding models are supported, for example: + "amazon.titan-embed-text-v1", "amazon.titan-embed-text-v2:0", "amazon.titan-embed-image-v1", + "cohere.embed-english-v3", "cohere.embed-multilingual-v3", "cohere.embed-v4:0". + To find all supported models, refer to the Amazon Bedrock + [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html) and + filter for "embedding", then select models from the Amazon Titan and Cohere series. +- **aws_access_key_id** (Secret | None) – AWS access key ID. +- **aws_secret_access_key** (Secret | None) – AWS secret access key. +- **aws_session_token** (Secret | None) – AWS session token. +- **aws_region_name** (Secret | str | None) – AWS region name. +- **aws_profile_name** (Secret | None) – AWS profile name. +- **batch_size** (int) – Number of Documents to encode at once. + Only Cohere models support batch inference. This parameter is ignored for Amazon Titan models. +- **progress_bar** (bool) – Whether to show a progress bar or not. Can be helpful to disable in production deployments + to keep the logs clean. +- **meta_fields_to_embed** (list\[str\] | None) – List of meta fields that should be embedded along with the Document text. +- **embedding_separator** (str) – Separator used to concatenate the meta fields to the Document text. +- **boto3_config** (dict\[str, Any\] | None) – Dictionary of configuration options for the underlying Boto3 client. + Can be used to tune [retry behavior](https://docs.aws.amazon.com/boto3/latest/guide/retries.html) + and other low-level settings like timeouts and connection management. +- **kwargs** (Any) – Additional parameters to pass for model inference. For example, `input_type` and `truncate` for + Cohere models, or `dimensions` and `normalize` for Amazon Titan Text Embeddings V2. + +**Raises:** + +- ValueError – If the model is not supported. +- AmazonBedrockConfigurationError – If the AWS environment is not configured correctly. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the Amazon Bedrock client. + +#### close + +```python +close() -> None +``` + +Close the Amazon Bedrock client. + +#### run + +```python +run(documents: list[Document]) -> dict[str, list[Document]] +``` + +Embed the provided `Document`s using the specified model. + +**Parameters:** + +- **documents** (list\[Document\]) – The `Document`s to embed. + +**Returns:** + +- dict\[str, list\[Document\]\] – A dictionary with the following keys: +- `documents`: The `Document`s with the `embedding` field populated. + +**Raises:** + +- AmazonBedrockInferenceError – If the inference fails. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> AmazonBedrockDocumentEmbedder +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- AmazonBedrockDocumentEmbedder – Deserialized component. + +## haystack_integrations.components.embedders.amazon_bedrock.document_image_embedder + +### AmazonBedrockDocumentImageEmbedder + +A component for computing Document embeddings based on images using Amazon Bedrock models. + +The embedding of each Document is stored in the `embedding` field of the Document. + +### Usage example + +```python +from haystack import Document +rom haystack_integrations.components.embedders.amazon_bedrock import AmazonBedrockDocumentImageEmbedder + +os.environ["AWS_ACCESS_KEY_ID"] = "..." +os.environ["AWS_SECRET_ACCESS_KEY_ID"] = "..." +os.environ["AWS_DEFAULT_REGION"] = "..." + +embedder = AmazonBedrockDocumentImageEmbedder(model="amazon.titan-embed-image-v1") + +documents = [ + Document(content="A photo of a cat", meta={"file_path": "cat.jpg"}), + Document(content="A photo of a dog", meta={"file_path": "dog.jpg"}), +] + +result = embedder.run(documents=documents) +documents_with_embeddings = result["documents"] +print(documents_with_embeddings) + +# [Document(id=..., +# content='A photo of a cat', +# meta={'file_path': 'cat.jpg', +# 'embedding_source': {'type': 'image', 'file_path_meta_field': 'file_path'}}, +# embedding=vector of size 512), +# ...] +``` + +#### __init__ + +```python +__init__( + *, + model: str, + aws_access_key_id: Secret | None = Secret.from_env_var( + "AWS_ACCESS_KEY_ID", strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + "AWS_SECRET_ACCESS_KEY", strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + "AWS_SESSION_TOKEN", strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + "AWS_DEFAULT_REGION", strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + "AWS_PROFILE", strict=False + ), + file_path_meta_field: str = "file_path", + root_path: str | None = None, + image_size: tuple[int, int] | None = None, + progress_bar: bool = True, + boto3_config: dict[str, Any] | None = None, + **kwargs: Any +) -> None +``` + +Creates a AmazonBedrockDocumentImageEmbedder component. + +**Parameters:** + +- **model** (str) – The embedding model to use. + Amazon Titan and Cohere multimodal embedding models are supported, for example: + "amazon.titan-embed-image-v1", "cohere.embed-english-v3", "cohere.embed-multilingual-v3", + "cohere.embed-v4:0". + To find all supported models, refer to the Amazon Bedrock + [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html) and + filter for "embedding", then select multimodal models from the Amazon Titan and Cohere series. +- **aws_access_key_id** (Secret | None) – AWS access key ID. +- **aws_secret_access_key** (Secret | None) – AWS secret access key. +- **aws_session_token** (Secret | None) – AWS session token. +- **aws_region_name** (Secret | str | None) – AWS region name. +- **aws_profile_name** (Secret | None) – AWS profile name. +- **file_path_meta_field** (str) – The metadata field in the Document that contains the file path to the image or PDF. +- **root_path** (str | None) – The root directory path where document files are located. If provided, file paths in + document metadata will be resolved relative to this path. If None, file paths are treated as absolute paths. +- **image_size** (tuple\[int, int\] | None) – If provided, resizes the image to fit within the specified dimensions (width, height) while + maintaining aspect ratio. This reduces file size, memory usage, and processing time, which is beneficial + when working with models that have resolution constraints or when transmitting images to remote services. +- **progress_bar** (bool) – If `True`, shows a progress bar when embedding documents. +- **boto3_config** (dict\[str, Any\] | None) – Dictionary of configuration options for the underlying Boto3 client. + Can be used to tune [retry behavior](https://docs.aws.amazon.com/boto3/latest/guide/retries.html) + and other low-level settings like timeouts and connection management. +- **kwargs** (Any) – Additional parameters to pass for model inference. + For example, `embeddingConfig` for Amazon Titan models and + `embedding_types` for Cohere models. + +**Raises:** + +- ValueError – If the model is not supported. +- AmazonBedrockConfigurationError – If the AWS environment is not configured correctly. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the Amazon Bedrock client. + +#### close + +```python +close() -> None +``` + +Close the Amazon Bedrock client. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> AmazonBedrockDocumentImageEmbedder +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- AmazonBedrockDocumentImageEmbedder – Deserialized component. + +#### run + +```python +run(documents: list[Document]) -> dict[str, list[Document]] +``` + +Embed a list of images. + +**Parameters:** + +- **documents** (list\[Document\]) – Documents to embed. + +**Returns:** + +- dict\[str, list\[Document\]\] – A dictionary with the following keys: +- `documents`: Documents with embeddings. + +## haystack_integrations.components.embedders.amazon_bedrock.text_embedder + +### AmazonBedrockTextEmbedder + +A component for embedding strings using Amazon Bedrock. + +Usage example: + +```python +import os +from haystack_integrations.components.embedders.amazon_bedrock import ( + AmazonBedrockTextEmbedder, +) + +os.environ["AWS_ACCESS_KEY_ID"] = "..." +os.environ["AWS_SECRET_ACCESS_KEY_ID"] = "..." +os.environ["AWS_DEFAULT_REGION"] = "..." + +embedder = AmazonBedrockTextEmbedder( + model="cohere.embed-english-v3", + input_type="search_query", +) + +print(text_embedder.run("I love Paris in the summer.")) + +# {'embedding': [0.002, 0.032, 0.504, ...]} +``` + +#### __init__ + +```python +__init__( + model: str, + aws_access_key_id: Secret | None = Secret.from_env_var( + "AWS_ACCESS_KEY_ID", strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + "AWS_SECRET_ACCESS_KEY", strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + "AWS_SESSION_TOKEN", strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + "AWS_DEFAULT_REGION", strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + "AWS_PROFILE", strict=False + ), + boto3_config: dict[str, Any] | None = None, + **kwargs: Any +) -> None +``` + +Initializes the AmazonBedrockTextEmbedder with the provided parameters. + +The parameters are passed to the Amazon Bedrock client. + +Note that the AWS credentials are not required if the AWS environment is configured correctly. These are loaded +automatically from the environment or the AWS configuration file and do not need to be provided explicitly via +the constructor. If the AWS environment is not configured users need to provide the AWS credentials via the +constructor. Aside from model, three required parameters are `aws_access_key_id`, `aws_secret_access_key`, +and `aws_region_name`. + +**Parameters:** + +- **model** (str) – The embedding model to use. + Amazon Titan and Cohere embedding models are supported, for example: + "amazon.titan-embed-text-v1", "amazon.titan-embed-text-v2:0", "amazon.titan-embed-image-v1", + "cohere.embed-english-v3", "cohere.embed-multilingual-v3", "cohere.embed-v4:0". + To find all supported models, refer to the Amazon Bedrock + [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html) and + filter for "embedding", then select models from the Amazon Titan and Cohere series. +- **aws_access_key_id** (Secret | None) – AWS access key ID. +- **aws_secret_access_key** (Secret | None) – AWS secret access key. +- **aws_session_token** (Secret | None) – AWS session token. +- **aws_region_name** (Secret | str | None) – AWS region name. +- **aws_profile_name** (Secret | None) – AWS profile name. +- **boto3_config** (dict\[str, Any\] | None) – Dictionary of configuration options for the underlying Boto3 client. + Can be used to tune [retry behavior](https://docs.aws.amazon.com/boto3/latest/guide/retries.html) + and other low-level settings like timeouts and connection management. +- **kwargs** (Any) – Additional parameters to pass for model inference. For example, `input_type` and `truncate` for + Cohere models, or `dimensions` and `normalize` for Amazon Titan Text Embeddings V2. + +**Raises:** + +- ValueError – If the model is not supported. +- AmazonBedrockConfigurationError – If the AWS environment is not configured correctly. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the Amazon Bedrock client. + +#### close + +```python +close() -> None +``` + +Close the Amazon Bedrock client. + +#### run + +```python +run(text: str) -> dict[str, list[float]] +``` + +Embeds the input text using the Amazon Bedrock model. + +**Parameters:** + +- **text** (str) – The input text to embed. + +**Returns:** + +- dict\[str, list\[float\]\] – A dictionary with the following keys: +- `embedding`: The embedding of the input text. + +**Raises:** + +- TypeError – If the input text is not a string. +- AmazonBedrockInferenceError – If the model inference fails. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> AmazonBedrockTextEmbedder +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- AmazonBedrockTextEmbedder – Deserialized component. + +## haystack_integrations.components.generators.amazon_bedrock.chat.chat_generator + +### AmazonBedrockChatGenerator + +Completes chats using LLMs hosted on Amazon Bedrock available via the Bedrock Converse API. + +For example, to use the Anthropic Claude 4.6 Sonnet model, initialize this component with the +'global.anthropic.claude-sonnet-4-6' model name. + +**Usage example** + +```python +from haystack_integrations.components.generators.amazon_bedrock import ( + AmazonBedrockChatGenerator, +) +from haystack.dataclasses import ChatMessage +from haystack.components.generators.utils import print_streaming_chunk + +messages = [ + ChatMessage.from_system( + "\nYou are a helpful, respectful and honest assistant, answer in German only" + ), + ChatMessage.from_user("What's Natural Language Processing?"), +] + + +client = AmazonBedrockChatGenerator( + model="global.anthropic.claude-sonnet-4-6", + streaming_callback=print_streaming_chunk, +) +client.run(messages, generation_kwargs={"max_tokens": 512}) +``` + +**Multimodal example** + +```python +from haystack.dataclasses import ChatMessage, ImageContent +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator + +generator = AmazonBedrockChatGenerator(model="global.anthropic.claude-sonnet-4-6") + +image_content = ImageContent.from_file_path(file_path="apple.jpg") + +message = ChatMessage.from_user(content_parts=["Describe the image using 10 words at most.", image_content]) + +response = generator.run(messages=[message])["replies"][0].text + +print(response) +> The image shows a red apple. +``` + +**Tool usage example** + +AmazonBedrockChatGenerator supports Haystack's unified tool architecture, allowing tools to be used +across different chat generators. The same tool definitions and usage patterns work consistently +whether using Amazon Bedrock, OpenAI, Ollama, or any other supported LLM providers. + +```python +from haystack.dataclasses import ChatMessage +from haystack.tools import Tool +from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator + +def weather(city: str): + return f'The weather in {city} is sunny and 32°C' + +# Define tool parameters +tool_parameters = { + "type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"] +} + +# Create weather tool +weather_tool = Tool( + name="weather", + description="useful to determine the weather in a given location", + parameters=tool_parameters, + function=weather +) + +# Initialize generator with tool +client = AmazonBedrockChatGenerator( + model="global.anthropic.claude-sonnet-4-6", + tools=[weather_tool] +) + +# Run initial query +messages = [ChatMessage.from_user("What's the weather like in Paris?")] +results = client.run(messages=messages) + +# Get tool call from response +tool_message = next(msg for msg in results["replies"] if msg.tool_call) +tool_call = tool_message.tool_call + +# Execute tool and send result back +weather_result = weather(**tool_call.arguments) +new_messages = [ + messages[0], + tool_message, + ChatMessage.from_tool(tool_result=weather_result, origin=tool_call) +] + +# Get final response +final_result = client.run(new_messages) +print(final_result["replies"][0].text) + +> Based on the information I've received, I can tell you that the weather in Paris is +> currently sunny with a temperature of 32°C (which is about 90°F). +``` + +**Prompt caching** + +This component supports prompt caching. You can use the `tools_cachepoint_config` parameter to configure the cache +point for tools. +To cache messages, you can use the `cachePoint` key in `ChatMessage.meta` attribute. + +```python +ChatMessage.from_user( + "Long message...", meta={"cachePoint": {"type": "default"}} +) +``` + +For more information, see the [Amazon Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html). + +**Authentication** + +AmazonBedrockChatGenerator uses AWS for authentication. You can use the AWS CLI to authenticate through your IAM. +For more information on setting up an IAM identity-based policy, see [Amazon Bedrock documentation] +(https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html). + +If the AWS environment is configured correctly, the AWS credentials are not required as they're loaded +automatically from the environment or the AWS configuration file. +If the AWS environment is not configured, set `aws_access_key_id`, `aws_secret_access_key`, +and `aws_region_name` as environment variables or pass them as +[Secret](https://docs.haystack.deepset.ai/docs/secret-management) arguments. Make sure the region you set +supports Amazon Bedrock. + +#### __init__ + +```python +__init__( + model: str, + aws_access_key_id: Secret | None = Secret.from_env_var( + ["AWS_ACCESS_KEY_ID"], strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + ["AWS_SECRET_ACCESS_KEY"], strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + ["AWS_SESSION_TOKEN"], strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + ["AWS_DEFAULT_REGION"], strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + ["AWS_PROFILE"], strict=False + ), + generation_kwargs: dict[str, Any] | None = None, + streaming_callback: StreamingCallbackT | None = None, + boto3_config: dict[str, Any] | None = None, + tools: ToolsType | None = None, + *, + guardrail_config: dict[str, str] | None = None, + tools_cachepoint_config: dict[str, str] | None = None, + system_cachepoint_config: dict[str, str] | None = None +) -> None +``` + +Initializes the `AmazonBedrockChatGenerator` with the provided parameters. + +The parameters are passed to the Amazon Bedrock client. + +Note that the AWS credentials are not required if the AWS environment is configured correctly. These are loaded +automatically from the environment or the AWS configuration file and do not need to be provided explicitly via +the constructor. If the AWS environment is not configured users need to provide the AWS credentials via the +constructor. Aside from model, three required parameters are `aws_access_key_id`, `aws_secret_access_key`, +and `aws_region_name`. + +**Parameters:** + +- **model** (str) – The model to use for text generation. The model must be available in Amazon Bedrock and must + be specified in the format outlined in the [Amazon Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html). + +- **aws_access_key_id** (Secret | None) – AWS access key ID. + +- **aws_secret_access_key** (Secret | None) – AWS secret access key. + +- **aws_session_token** (Secret | None) – AWS session token. + +- **aws_region_name** (Secret | str | None) – AWS region name. Make sure the region you set supports Amazon Bedrock. + +- **aws_profile_name** (Secret | None) – AWS profile name. + +- **generation_kwargs** (dict\[str, Any\] | None) – Optional dictionary of generation parameters. Some common parameters are: + +- `maxTokens`: Maximum number of tokens to generate. + +- `stopSequences`: List of stop sequences to stop generation. + +- `temperature`: Sampling temperature. + +- `topP`: Nucleus sampling parameter. + +- `response_format`: Request structured JSON output validated against a schema. Provide a dict with: + + - `schema` (required): a JSON Schema dict describing the expected output structure. + - `name` (optional): a name for the schema, defaults to `"response_schema"`. + - `description` (optional): a description of the schema. + + Example:: + + ``` + generation_kwargs = { + "response_format": { + "name": "person", + "schema": { + "type": "object", + "properties": { + "name": {"type": "string"}, + "age": {"type": "integer"}, + }, + "required": ["name", "age"], + "additionalProperties": False, + }, + } + } + ``` + + When set, the parsed JSON object is stored in `reply.meta["structured_output"]`. + You can find the model specific arguments in the AWS Bedrock API[documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html). + +- **streaming_callback** (StreamingCallbackT | None) – A callback function called when a new token is received from the stream. + By default, the model is not set up for streaming. To enable streaming, set this parameter to a callback + function that handles the streaming chunks. The callback function receives a + [StreamingChunk](https://docs.haystack.deepset.ai/docs/data-classes#streamingchunk) object and switches + the streaming mode on. + +- **boto3_config** (dict\[str, Any\] | None) – Dictionary of configuration options for the underlying Boto3 client. + Can be used to tune [retry behavior](https://docs.aws.amazon.com/boto3/latest/guide/retries.html) + and other low-level settings like timeouts and connection management. + +- **tools** (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. + Each tool should have a unique name. + +- **guardrail_config** (dict\[str, str\] | None) – Optional configuration for a guardrail that has been created in Amazon Bedrock. + This must be provided as a dictionary matching either + [GuardrailConfiguration](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GuardrailConfiguration.html). + or, in streaming mode (when `streaming_callback` is set), + [GuardrailStreamConfiguration](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_GuardrailStreamConfiguration.html). + If `trace` is set to `enabled`, the guardrail trace will be included under the `trace` key in the `meta` + attribute of the resulting `ChatMessage`. + Note: Enabling guardrails in streaming mode may introduce additional latency. + To manage this, you can adjust the `streamProcessingMode` parameter. + See the + [Guardrails Streaming documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-streaming.html) + for more information. + +- **tools_cachepoint_config** (dict\[str, str\] | None) – Optional configuration to use prompt caching for tools. + The dictionary must match the + [CachePointBlock schema](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CachePointBlock.html). + Example: `{"type": "default", "ttl": "5m"}` + +- **system_cachepoint_config** (dict\[str, str\] | None) – Optional configuration to use prompt caching for system messages. + The dictionary must match the + [CachePointBlock schema](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_CachePointBlock.html). + Example: `{"type": "default", "ttl": "5m"}` + +**Raises:** + +- ValueError – If the model name is empty or None. +- AmazonBedrockConfigurationError – If the AWS environment is not configured correctly or the model is + not supported. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the synchronous Amazon Bedrock client. + +#### close + +```python +close() -> None +``` + +Close the synchronous Amazon Bedrock client. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> AmazonBedrockChatGenerator +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary with serialized data. + +**Returns:** + +- AmazonBedrockChatGenerator – Instance of `AmazonBedrockChatGenerator`. + +#### run + +```python +run( + messages: list[ChatMessage] | str, + streaming_callback: StreamingCallbackT | None = None, + generation_kwargs: dict[str, Any] | None = None, + tools: ToolsType | None = None, +) -> dict[str, list[ChatMessage]] +``` + +Executes a synchronous inference call to the Amazon Bedrock model using the Converse API. + +Supports both standard and streaming responses depending on whether a streaming callback is provided. + +**Parameters:** + +- **messages** (list\[ChatMessage\] | str) – A list of `ChatMessage` objects forming the chat history. + If a string is provided, it is converted to a list containing a ChatMessage with user role. +- **streaming_callback** (StreamingCallbackT | None) – Optional callback for handling streaming outputs. +- **generation_kwargs** (dict\[str, Any\] | None) – Optional dictionary of generation parameters. These are merged per key with the + `generation_kwargs` passed at initialization: keys provided here take precedence, keys set only at + initialization are kept. Some common parameters are: +- `maxTokens`: Maximum number of tokens to generate. +- `stopSequences`: List of stop sequences to stop generation. +- `temperature`: Sampling temperature. +- `topP`: Nucleus sampling parameter. +- `response_format`: Request structured JSON output validated against a schema. +- **tools** (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. + Each tool should have a unique name. + +**Returns:** + +- dict\[str, list\[ChatMessage\]\] – A dictionary containing the model-generated replies under the `"replies"` key. + +**Raises:** + +- AmazonBedrockInferenceError – If the Bedrock inference API call fails. + +#### run_async + +```python +run_async( + messages: list[ChatMessage] | str, + streaming_callback: StreamingCallbackT | None = None, + generation_kwargs: dict[str, Any] | None = None, + tools: ToolsType | None = None, +) -> dict[str, list[ChatMessage]] +``` + +Executes an asynchronous inference call to the Amazon Bedrock model using the Converse API. + +Designed for use cases where non-blocking or concurrent execution is desired. + +**Parameters:** + +- **messages** (list\[ChatMessage\] | str) – A list of `ChatMessage` objects forming the chat history. + If a string is provided, it is converted to a list containing a ChatMessage with user role. +- **streaming_callback** (StreamingCallbackT | None) – Optional callback for handling streaming outputs. Async callbacks are preferred. +- **generation_kwargs** (dict\[str, Any\] | None) – Optional dictionary of generation parameters. These are merged per key with the + `generation_kwargs` passed at initialization: keys provided here take precedence, keys set only at + initialization are kept. Some common parameters are: +- `maxTokens`: Maximum number of tokens to generate. +- `stopSequences`: List of stop sequences to stop generation. +- `temperature`: Sampling temperature. +- `topP`: Nucleus sampling parameter. +- `response_format`: Request structured JSON output validated against a schema. +- **tools** (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. + Each tool should have a unique name. + +**Returns:** + +- dict\[str, list\[ChatMessage\]\] – A dictionary containing the model-generated replies under the `"replies"` key. + +**Raises:** + +- AmazonBedrockInferenceError – If the Bedrock inference API call fails. + +## haystack_integrations.components.rankers.amazon_bedrock.ranker + +### AmazonBedrockRanker + +Ranks Documents based on their similarity to the query using Amazon Bedrock's Cohere Rerank model. + +Documents are indexed from most to least semantically relevant to the query. + +Supported Amazon Bedrock models: + +- cohere.rerank-v3-5:0 +- amazon.rerank-v1:0 + +Usage example: + +```python +from haystack import Document +from haystack.utils import Secret +from haystack_integrations.components.rankers.amazon_bedrock import ( + AmazonBedrockRanker, +) + +ranker = AmazonBedrockRanker( + model="cohere.rerank-v3-5:0", + top_k=2, + aws_region_name=Secret.from_token("eu-central-1"), +) + +docs = [Document(content="Paris"), Document(content="Berlin")] +query = "What is the capital of germany?" +output = ranker.run(query=query, documents=docs) +docs = output["documents"] +``` + +AmazonBedrockRanker uses AWS for authentication. You can use the AWS CLI to authenticate through your IAM. +For more information on setting up an IAM identity-based policy, see [Amazon Bedrock documentation] +(https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html). + +If the AWS environment is configured correctly, the AWS credentials are not required as they're loaded +automatically from the environment or the AWS configuration file. +If the AWS environment is not configured, set `aws_access_key_id`, `aws_secret_access_key`, +and `aws_region_name` as environment variables or pass them as +[Secret](https://docs.haystack.deepset.ai/docs/secret-management) arguments. Make sure the region you set +supports Amazon Bedrock. + +#### __init__ + +```python +__init__( + model: str = "cohere.rerank-v3-5:0", + top_k: int = 10, + aws_access_key_id: Secret | None = Secret.from_env_var( + ["AWS_ACCESS_KEY_ID"], strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + ["AWS_SECRET_ACCESS_KEY"], strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + ["AWS_SESSION_TOKEN"], strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + ["AWS_DEFAULT_REGION"], strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + ["AWS_PROFILE"], strict=False + ), + max_chunks_per_doc: int | None = None, + meta_fields_to_embed: list[str] | None = None, + meta_data_separator: str = "\n", +) -> None +``` + +Creates an instance of the 'AmazonBedrockRanker'. + +**Parameters:** + +- **model** (str) – Amazon Bedrock model name for Cohere Rerank. Default is "cohere.rerank-v3-5:0". +- **top_k** (int) – The maximum number of documents to return. +- **aws_access_key_id** (Secret | None) – AWS access key ID. +- **aws_secret_access_key** (Secret | None) – AWS secret access key. +- **aws_session_token** (Secret | None) – AWS session token. +- **aws_region_name** (Secret | str | None) – AWS region name. +- **aws_profile_name** (Secret | None) – AWS profile name. +- **max_chunks_per_doc** (int | None) – If your document exceeds 512 tokens, this determines the maximum number of + chunks a document can be split into. If `None`, the default of 10 is used. + Note: This parameter is not currently used in the implementation but is included for future compatibility. +- **meta_fields_to_embed** (list\[str\] | None) – List of meta fields that should be concatenated + with the document content for reranking. +- **meta_data_separator** (str) – Separator used to concatenate the meta fields + to the Document content. + +**Raises:** + +- ValueError – If `model` is empty or if `top_k` is not > 0. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the Amazon Bedrock client. + +#### close + +```python +close() -> None +``` + +Close the Amazon Bedrock client. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> AmazonBedrockRanker +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – The dictionary to deserialize from. + +**Returns:** + +- AmazonBedrockRanker – The deserialized component. + +#### run + +```python +run( + query: str, documents: list[Document], top_k: int | None = None +) -> dict[str, list[Document]] +``` + +Use the Amazon Bedrock Reranker to re-rank the list of documents based on the query. + +**Parameters:** + +- **query** (str) – Query string. +- **documents** (list\[Document\]) – List of Documents. +- **top_k** (int | None) – The maximum number of Documents you want the Ranker to return. + +**Returns:** + +- dict\[str, list\[Document\]\] – A dictionary with the following keys: +- `documents`: List of Documents most similar to the given query in descending order of similarity. + +**Raises:** + +- ValueError – If `top_k` is not > 0. + +## haystack_integrations.components.retrievers.amazon_bedrock.knowledge_base_retriever + +### AmazonBedrockKnowledgeBaseRetriever + +Retrieves documents from an Amazon Bedrock Managed Knowledge Base. + +Uses AgenticRetrieveStream when available, falling back to the standard Retrieve API otherwise. + +Usage example: + +```python +from haystack.utils import Secret +from haystack_integrations.components.retrievers.amazon_bedrock import AmazonBedrockKnowledgeBaseRetriever + +retriever = AmazonBedrockKnowledgeBaseRetriever( + knowledge_base_id="ABCDEFGHIJ", + aws_region_name=Secret.from_token("eu-central-1"), +) + +result = retriever.run(query="What are the benefits of managed knowledge bases?") +for doc in result["documents"]: + print(doc.content) + print(doc.meta["source"]) + print(doc.score) +``` + +AmazonBedrockKnowledgeBaseRetriever uses AWS for authentication. You can use the AWS CLI to authenticate through +your IAM. For more information on setting up an IAM identity-based policy, see [Amazon Bedrock documentation] +(https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html). + +If the AWS environment is configured correctly, the AWS credentials are not required as they're loaded +automatically from the environment or the AWS configuration file. +If the AWS environment is not configured, set `aws_access_key_id`, `aws_secret_access_key`, +and `aws_region_name` as environment variables or pass them as +[Secret](https://docs.haystack.deepset.ai/docs/secret-management) arguments. + +#### __init__ + +```python +__init__( + knowledge_base_id: str | None = None, + aws_access_key_id: Secret | None = Secret.from_env_var( + "AWS_ACCESS_KEY_ID", strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + "AWS_SECRET_ACCESS_KEY", strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + "AWS_SESSION_TOKEN", strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + "AWS_DEFAULT_REGION", strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + "AWS_PROFILE", strict=False + ), + number_of_results: int = 5, + use_agentic_retrieval: bool | None = None, +) -> None +``` + +Create the AmazonBedrockKnowledgeBaseRetriever component. + +**Parameters:** + +- **knowledge_base_id** (str | None) – The ID of the Bedrock Knowledge Base. Falls back to the AWS_KNOWLEDGE_BASE_ID + environment variable. +- **aws_access_key_id** (Secret | None) – AWS access key ID. +- **aws_secret_access_key** (Secret | None) – AWS secret access key. +- **aws_session_token** (Secret | None) – AWS session token. +- **aws_region_name** (Secret | str | None) – AWS region name. +- **aws_profile_name** (Secret | None) – AWS profile name. +- **number_of_results** (int) – Maximum number of results to return. +- **use_agentic_retrieval** (bool | None) – If True, try AgenticRetrieveStream before plain Retrieve. + Defaults to the USE_AGENTIC_RETRIEVAL environment variable, or True. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the Amazon Bedrock client. + +#### close + +```python +close() -> None +``` + +Close the Amazon Bedrock client. + +#### run + +```python +run(query: str, top_k: int | None = None) -> dict[str, list[Document]] +``` + +Retrieve documents from the Bedrock Knowledge Base. + +**Parameters:** + +- **query** (str) – The search query. +- **top_k** (int | None) – Maximum number of results. Overrides number_of_results if provided. + +**Returns:** + +- dict\[str, list\[Document\]\] – A dictionary with a "documents" key containing the retrieved Documents. + +**Raises:** + +- AmazonBedrockInferenceError – If the retrieval call fails. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> AmazonBedrockKnowledgeBaseRetriever +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – The dictionary to deserialize from. + +**Returns:** + +- AmazonBedrockKnowledgeBaseRetriever – The deserialized component. + +## haystack_integrations.token_counters.amazon_bedrock.token_counter + +### AmazonBedrockTokenCounter + +Counts tokens with Amazon Bedrock's `CountTokens` API. + +Implements Haystack's `TokenCounter` protocol. Unlike local, tokenizer-based counters, this counter sends the +input to Bedrock's `CountTokens` operation, so the returned count reflects the model's exact tokenization, +including the formatting Bedrock applies to messages, system prompts, and tool schemas. + +The messages and tools are converted to the Bedrock `Converse` format (the same conversion the +`AmazonBedrockChatGenerator` uses), so the count matches what an equivalent `Converse` request would consume. + +Because it delegates to a server-side API, `count()` measures a complete, valid conversation rather than an +arbitrary set of messages: Bedrock validates the input the same way the `Converse` inference API does (it must +begin with a user message, and tool results must pair with the tool calls that produced them). This is the right +fit for sizing a whole request before sending it - its intended use - but it cannot size a stand-alone fragment +such as a single tool-result message. For fragment-level counting (for example inside a compactor that measures +individual messages), use a local counter such as `ApproximateTokenCounter`. + +## Usage Example: + +```python +from haystack.dataclasses import ChatMessage +from haystack_integrations.token_counters.amazon_bedrock import AmazonBedrockTokenCounter + +counter = AmazonBedrockTokenCounter(model="anthropic.claude-3-5-sonnet-20240620-v1:0") +messages = [ChatMessage.from_user("Hello, how are you?")] +token_count = counter.count(messages) +print(f"Token count: {token_count}") +``` + +#### __init__ + +```python +__init__( + model: str, + *, + aws_access_key_id: Secret | None = Secret.from_env_var( + ["AWS_ACCESS_KEY_ID"], strict=False + ), + aws_secret_access_key: Secret | None = Secret.from_env_var( + ["AWS_SECRET_ACCESS_KEY"], strict=False + ), + aws_session_token: Secret | None = Secret.from_env_var( + ["AWS_SESSION_TOKEN"], strict=False + ), + aws_region_name: Secret | str | None = Secret.from_env_var( + ["AWS_DEFAULT_REGION"], strict=False + ), + aws_profile_name: Secret | None = Secret.from_env_var( + ["AWS_PROFILE"], strict=False + ), + boto3_config: dict[str, Any] | None = None +) -> None +``` + +Initialize the counter. + +**Parameters:** + +- **model** (str) – The Bedrock model id (or ARN) whose tokenization should be used, for example + `"anthropic.claude-3-5-sonnet-20240620-v1:0"`. Token counts are model-specific. +- **aws_access_key_id** (Secret | None) – AWS access key ID. +- **aws_secret_access_key** (Secret | None) – AWS secret access key. +- **aws_session_token** (Secret | None) – AWS session token. +- **aws_region_name** (Secret | str | None) – AWS region name. Make sure the region you set supports Amazon Bedrock. +- **aws_profile_name** (Secret | None) – AWS profile name. +- **boto3_config** (dict\[str, Any\] | None) – Dictionary of configuration options for the underlying Boto3 client. + +**Raises:** + +- ValueError – If `model` is empty. + +#### warm_up + +```python +warm_up() -> None +``` + +Initialize the Amazon Bedrock client. + +**Raises:** + +- AmazonBedrockConfigurationError – If the AWS environment is not configured correctly. + +#### count + +```python +count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int +``` + +Return the number of input tokens Bedrock will use for the given messages and tools. + +`messages` must form a complete, valid conversation: Bedrock validates it the same way the `Converse` +inference API does (it must begin with a user message, and tool results must pair with their tool calls). +To size an arbitrary fragment such as a single message, use a local counter like `ApproximateTokenCounter`. + +**Parameters:** + +- **messages** (list\[ChatMessage\]) – The messages to measure. +- **tools** (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too. Pass them to + have them counted; leave as None to measure the messages alone. + +**Returns:** + +- int – The token count, or `0` when there is nothing to measure. + +**Raises:** + +- AmazonBedrockInferenceError – If the Bedrock `CountTokens` request fails. + +#### close + +```python +close() -> None +``` + +Close the Amazon Bedrock client and release its resources. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serialize the counter. + +**Returns:** + +- dict\[str, Any\] – A dictionary representation of the counter. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> AmazonBedrockTokenCounter +``` + +Deserialize the counter. + +**Parameters:** + +- **data** (dict\[str, Any\]) – A dictionary representation of the counter. + +**Returns:** + +- AmazonBedrockTokenCounter – The deserialized counter.