diff --git a/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/google_genai.md b/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/google_genai.md new file mode 100644 index 0000000000..9bde84f396 --- /dev/null +++ b/docs-website/reference_versioned_docs/version-3.2-unstable/integrations-api/google_genai.md @@ -0,0 +1,1159 @@ +--- +title: "Google GenAI" +id: integrations-google-genai +description: "Google GenAI integration for Haystack" +slug: "/integrations-google-genai" +--- + + +## haystack_integrations.components.embedders.google_genai.document_embedder + +### GoogleGenAIDocumentEmbedder + +Computes document embeddings using Google AI models. + +### Authentication examples + +**1. Gemini Developer API (API Key Authentication)** + +````python +from haystack_integrations.components.embedders.google_genai import GoogleGenAIDocumentEmbedder + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +document_embedder = GoogleGenAIDocumentEmbedder(model="gemini-embedding-001") + +**2. Vertex AI (Application Default Credentials)** +```python +from haystack_integrations.components.embedders.google_genai import GoogleGenAIDocumentEmbedder + +# Using Application Default Credentials (requires gcloud auth setup) +document_embedder = GoogleGenAIDocumentEmbedder( + api="vertex", + vertex_ai_project="my-project", + vertex_ai_location="us-central1", + model="gemini-embedding-001" +) +```` + +**3. Vertex AI (API Key Authentication)** + +```python +from haystack_integrations.components.embedders.google_genai import GoogleGenAIDocumentEmbedder + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +document_embedder = GoogleGenAIDocumentEmbedder( + api="vertex", + model="gemini-embedding-001" +) +``` + +### Usage example + +```python +from haystack import Document +from haystack_integrations.components.embedders.google_genai import GoogleGenAIDocumentEmbedder + +doc = Document(content="I love pizza!") + +document_embedder = GoogleGenAIDocumentEmbedder() + +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) + +# [0.017020374536514282, -0.023255806416273117, ...] +``` + +#### __init__ + +```python +__init__( + *, + api_key: Secret = Secret.from_env_var( + ["GOOGLE_API_KEY", "GEMINI_API_KEY"], strict=False + ), + api: Literal["gemini", "vertex"] = "gemini", + vertex_ai_project: str | None = None, + vertex_ai_location: str | None = None, + model: str = "gemini-embedding-001", + prefix: str = "", + suffix: str = "", + batch_size: int = 32, + progress_bar: bool = True, + meta_fields_to_embed: list[str] | None = None, + embedding_separator: str = "\n", + config: dict[str, Any] | None = None, + timeout: float | None = None, + max_retries: int | None = None +) -> None +``` + +Creates an GoogleGenAIDocumentEmbedder component. + +**Parameters:** + +- **api_key** (Secret) – Google API key, defaults to the `GOOGLE_API_KEY` and `GEMINI_API_KEY` environment variables. + Not needed if using Vertex AI with Application Default Credentials. + Go to https://aistudio.google.com/app/apikey for a Gemini API key. + Go to https://cloud.google.com/vertex-ai/generative-ai/docs/start/api-keys for a Vertex AI API key. +- **api** (Literal['gemini', 'vertex']) – Which API to use. Either "gemini" for the Gemini Developer API or "vertex" for Vertex AI. +- **vertex_ai_project** (str | None) – Google Cloud project ID for Vertex AI. Required when using Vertex AI with + Application Default Credentials. +- **vertex_ai_location** (str | None) – Google Cloud location for Vertex AI (e.g., "us-central1", "europe-west1"). + Required when using Vertex AI with Application Default Credentials. +- **model** (str) – The name of the model to use for calculating embeddings. + The default model is `gemini-embedding-001`. +- **prefix** (str) – A string to add at the beginning of each text. It can be used to specify a task type for + `gemini-embedding-2`. For available task types, see + [Gemini documentation](https://ai.google.dev/gemini-api/docs/embeddings#task-types). +- **suffix** (str) – A string to add at the end of each text. +- **batch_size** (int) – Number of documents to embed at once. +- **progress_bar** (bool) – If `True`, shows a progress bar when running. +- **meta_fields_to_embed** (list\[str\] | None) – List of metadata fields to embed along with the document text. +- **embedding_separator** (str) – Separator used to concatenate the metadata fields to the document text. +- **config** (dict\[str, Any\] | None) – A dictionary of keyword arguments to configure embedding content configuration. + See [Google API documentation](https://googleapis.github.io/python-genai/genai.html#genai.types.EmbedContentConfig) + for the available options. + Specifying task types in `config` does not take effect for `gemini-embedding-2`. + See [Gemini documentation](https://ai.google.dev/gemini-api/docs/embeddings#task-types) for more + information. +- **timeout** (float | None) – The timeout in seconds for the underlying Google GenAI client network requests. +- **max_retries** (int | None) – The maximum number of retries for the underlying Google GenAI client network requests. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the synchronous Google Gen AI client. + +#### warm_up_async + +```python +warm_up_async() -> None +``` + +Create the asynchronous Google Gen AI client. + +#### close + +```python +close() -> None +``` + +Close the synchronous Google Gen AI client. + +#### close_async + +```python +close_async() -> None +``` + +Close the asynchronous Google Gen AI client. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> GoogleGenAIDocumentEmbedder +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- GoogleGenAIDocumentEmbedder – Deserialized component. + +#### run + +```python +run(documents: list[Document]) -> dict[str, list[Document]] | dict[str, Any] +``` + +Embeds a list of documents. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- dict\[str, list\[Document\]\] | dict\[str, Any\] – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. +- `meta`: Information about the usage of the model. + +#### run_async + +```python +run_async( + documents: list[Document], +) -> dict[str, list[Document]] | dict[str, Any] +``` + +Embeds a list of documents asynchronously. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- dict\[str, list\[Document\]\] | dict\[str, Any\] – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. +- `meta`: Information about the usage of the model. + +## haystack_integrations.components.embedders.google_genai.multimodal_document_embedder + +### GoogleGenAIMultimodalDocumentEmbedder + +Computes non-textual document embeddings using Google AI models. + +It supports images, PDFs, video and audio files. They are mapped to vectors in a single vector space. + +To embed textual documents, use the GoogleGenAIDocumentEmbedder. +To embed a string, like a user query, use the GoogleGenAITextEmbedder. + +### Authentication examples + +**1. Gemini Developer API (API Key Authentication)** + +````python +from haystack_integrations.components.embedders.google_genai import GoogleGenAIMultimodalDocumentEmbedder + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +document_embedder = GoogleGenAIMultimodalDocumentEmbedder(model="gemini-embedding-2-preview") + +**2. Vertex AI (Application Default Credentials)** +```python +from haystack_integrations.components.embedders.google_genai import GoogleGenAIMultimodalDocumentEmbedder + +# Using Application Default Credentials (requires gcloud auth setup) +document_embedder = GoogleGenAIMultimodalDocumentEmbedder( + api="vertex", + vertex_ai_project="my-project", + vertex_ai_location="us-central1", + model="gemini-embedding-2-preview" +) +```` + +**3. Vertex AI (API Key Authentication)** + +```python +from haystack_integrations.components.embedders.google_genai import GoogleGenAIMultimodalDocumentEmbedder + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +document_embedder = GoogleGenAIMultimodalDocumentEmbedder( + api="vertex", + model="gemini-embedding-2-preview" +) +``` + +### Usage example + +```python +from haystack import Document +from haystack_integrations.components.embedders.google_genai import GoogleGenAIMultimodalDocumentEmbedder + +doc = Document(content=None, meta={"file_path": "path/to/image.jpg"}) + +document_embedder = GoogleGenAIMultimodalDocumentEmbedder() + +result = document_embedder.run([doc]) +print(result['documents'][0].embedding) + +# [0.017020374536514282, -0.023255806416273117, ...] +``` + +#### __init__ + +```python +__init__( + *, + api_key: Secret = Secret.from_env_var( + ["GOOGLE_API_KEY", "GEMINI_API_KEY"], strict=False + ), + api: Literal["gemini", "vertex"] = "gemini", + vertex_ai_project: str | None = None, + vertex_ai_location: str | None = None, + file_path_meta_field: str = "file_path", + root_path: str | None = None, + image_size: tuple[int, int] | None = None, + model: str = "gemini-embedding-2", + batch_size: int = 6, + progress_bar: bool = True, + config: dict[str, Any] | None = None, + timeout: float | None = None, + max_retries: int | None = None +) -> None +``` + +Creates an GoogleGenAIMultimodalDocumentEmbedder component. + +**Parameters:** + +- **api_key** (Secret) – Google API key, defaults to the `GOOGLE_API_KEY` and `GEMINI_API_KEY` environment variables. + Not needed if using Vertex AI with Application Default Credentials. + Go to https://aistudio.google.com/app/apikey for a Gemini API key. + Go to https://cloud.google.com/vertex-ai/generative-ai/docs/start/api-keys for a Vertex AI API key. +- **api** (Literal['gemini', 'vertex']) – Which API to use. Either "gemini" for the Gemini Developer API or "vertex" for Vertex AI. +- **vertex_ai_project** (str | None) – Google Cloud project ID for Vertex AI. Required when using Vertex AI with + Application Default Credentials. +- **vertex_ai_location** (str | None) – Google Cloud location for Vertex AI (e.g., "us-central1", "europe-west1"). + Required when using Vertex AI with Application Default Credentials. +- **file_path_meta_field** (str) – The metadata field in the Document that contains the file path to the file to embed. +- **root_path** (str | None) – The root directory path where document files are located. If provided, file paths in + document metadata will be resolved relative to this path and are guaranteed to stay within it. + If None, file paths are treated as absolute paths with no containment check. + If document metadata, in particular `file_path_meta_field`, may be influenced by untrusted input, + set `root_path` to a dedicated data directory so that path-traversal beyond it is rejected. +- **image_size** (tuple\[int, int\] | None) – Only used for images and PDF pages. If provided, resizes the image to fit within the specified dimensions + (width, height) while maintaining aspect ratio. This reduces file size, memory usage, and processing time, + which is beneficial when working with models that have resolution constraints or when transmitting images + to remote services. +- **model** (str) – The name of the model to use for calculating embeddings. +- **batch_size** (int) – Number of documents to embed at once. Maximum batch size varies depending on the input type. + See [Google AI documentation](https://ai.google.dev/gemini-api/docs/embeddings#supported-modalities) for + more information. +- **progress_bar** (bool) – If `True`, shows a progress bar when running. +- **config** (dict\[str, Any\] | None) – A dictionary of keyword arguments to configure embedding content configuration. + You can for example set the output dimensionality of the embedding: `{"output_dimensionality": 768}`. + See [Google API documentation](https://googleapis.github.io/python-genai/genai.html#genai.types.EmbedContentConfig) + for the available options. +- **timeout** (float | None) – The timeout in seconds for the underlying Google GenAI client network requests. +- **max_retries** (int | None) – The maximum number of retries for the underlying Google GenAI client network requests. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the synchronous Google Gen AI client. + +#### warm_up_async + +```python +warm_up_async() -> None +``` + +Create the asynchronous Google Gen AI client. + +#### close + +```python +close() -> None +``` + +Close the synchronous Google Gen AI client. + +#### close_async + +```python +close_async() -> None +``` + +Close the asynchronous Google Gen AI client. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> GoogleGenAIMultimodalDocumentEmbedder +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- GoogleGenAIMultimodalDocumentEmbedder – Deserialized component. + +#### run + +```python +run(documents: list[Document]) -> dict[str, list[Document]] | dict[str, Any] +``` + +Embeds a list of documents. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- dict\[str, list\[Document\]\] | dict\[str, Any\] – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. +- `meta`: Information about the usage of the model. + +**Raises:** + +- TypeError – If the input is not a list of `Documents`. +- ValueError – If a document is missing the file path metadata field, its file path escapes `root_path`, or its + MIME type is not supported. +- RuntimeError – If the conversion of some documents fails. + +#### run_async + +```python +run_async( + documents: list[Document], +) -> dict[str, list[Document]] | dict[str, Any] +``` + +Embeds a list of documents asynchronously. + +**Parameters:** + +- **documents** (list\[Document\]) – A list of documents to embed. + +**Returns:** + +- dict\[str, list\[Document\]\] | dict\[str, Any\] – A dictionary with the following keys: +- `documents`: A list of documents with embeddings. +- `meta`: Information about the usage of the model. + +**Raises:** + +- TypeError – If the input is not a list of `Documents`. +- ValueError – If a document is missing the file path metadata field, its file path escapes `root_path`, or its + MIME type is not supported. +- RuntimeError – If the conversion of some documents fails. + +## haystack_integrations.components.embedders.google_genai.text_embedder + +### GoogleGenAITextEmbedder + +Embeds strings using Google AI models. + +You can use it to embed user query and send it to an embedding Retriever. + +### Authentication examples + +**1. Gemini Developer API (API Key Authentication)** + +````python +from haystack_integrations.components.embedders.google_genai import GoogleGenAITextEmbedder + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +text_embedder = GoogleGenAITextEmbedder(model="gemini-embedding-001") + +**2. Vertex AI (Application Default Credentials)** +```python +from haystack_integrations.components.embedders.google_genai import GoogleGenAITextEmbedder + +# Using Application Default Credentials (requires gcloud auth setup) +text_embedder = GoogleGenAITextEmbedder( + api="vertex", + vertex_ai_project="my-project", + vertex_ai_location="us-central1", + model="gemini-embedding-001" +) +```` + +**3. Vertex AI (API Key Authentication)** + +```python +from haystack_integrations.components.embedders.google_genai import GoogleGenAITextEmbedder + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +text_embedder = GoogleGenAITextEmbedder( + api="vertex", + model="gemini-embedding-001" +) +``` + +### Usage example + +```python +from haystack_integrations.components.embedders.google_genai import GoogleGenAITextEmbedder + +text_to_embed = "I love pizza!" + +text_embedder = GoogleGenAITextEmbedder() + +print(text_embedder.run(text_to_embed)) + +# {'embedding': [0.017020374536514282, -0.023255806416273117, ...], +# 'meta': {'model': 'gemini-embedding-001-v2', +# 'usage': {'prompt_tokens': 4, 'total_tokens': 4}}} +``` + +#### __init__ + +```python +__init__( + *, + api_key: Secret = Secret.from_env_var( + ["GOOGLE_API_KEY", "GEMINI_API_KEY"], strict=False + ), + api: Literal["gemini", "vertex"] = "gemini", + vertex_ai_project: str | None = None, + vertex_ai_location: str | None = None, + model: str = "gemini-embedding-001", + prefix: str = "", + suffix: str = "", + config: dict[str, Any] | None = None, + timeout: float | None = None, + max_retries: int | None = None +) -> None +``` + +Creates an GoogleGenAITextEmbedder component. + +**Parameters:** + +- **api_key** (Secret) – Google API key, defaults to the `GOOGLE_API_KEY` and `GEMINI_API_KEY` environment variables. + Not needed if using Vertex AI with Application Default Credentials. + Go to https://aistudio.google.com/app/apikey for a Gemini API key. + Go to https://cloud.google.com/vertex-ai/generative-ai/docs/start/api-keys for a Vertex AI API key. +- **api** (Literal['gemini', 'vertex']) – Which API to use. Either "gemini" for the Gemini Developer API or "vertex" for Vertex AI. +- **vertex_ai_project** (str | None) – Google Cloud project ID for Vertex AI. Required when using Vertex AI with + Application Default Credentials. +- **vertex_ai_location** (str | None) – Google Cloud location for Vertex AI (e.g., "us-central1", "europe-west1"). + Required when using Vertex AI with Application Default Credentials. +- **model** (str) – The name of the model to use for calculating embeddings. + The default model is `gemini-embedding-001`. +- **prefix** (str) – A string to add at the beginning of each text. It can be used to specify a task type for + `gemini-embedding-2`. For available task types, see + [Gemini documentation](https://ai.google.dev/gemini-api/docs/embeddings#task-types). +- **suffix** (str) – A string to add at the end of each text to embed. +- **config** (dict\[str, Any\] | None) – A dictionary of keyword arguments to configure embedding content configuration. + See [Google API documentation](https://googleapis.github.io/python-genai/genai.html#genai.types.EmbedContentConfig) + for the available options. + Specifying task types in `config` does not take effect for `gemini-embedding-2`. + See [Gemini documentation](https://ai.google.dev/gemini-api/docs/embeddings#task-types) for more + information. +- **timeout** (float | None) – The timeout in seconds for the underlying Google GenAI client network requests. +- **max_retries** (int | None) – The maximum number of retries for the underlying Google GenAI client network requests. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the synchronous Google Gen AI client. + +#### warm_up_async + +```python +warm_up_async() -> None +``` + +Create the asynchronous Google Gen AI client. + +#### close + +```python +close() -> None +``` + +Close the synchronous Google Gen AI client. + +#### close_async + +```python +close_async() -> None +``` + +Close the asynchronous Google Gen AI client. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> GoogleGenAITextEmbedder +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- GoogleGenAITextEmbedder – Deserialized component. + +#### run + +```python +run(text: str) -> dict[str, list[float]] | dict[str, Any] +``` + +Embeds a single string. + +**Parameters:** + +- **text** (str) – Text to embed. + +**Returns:** + +- dict\[str, list\[float\]\] | dict\[str, Any\] – A dictionary with the following keys: +- `embedding`: The embedding of the input text. +- `meta`: Information about the usage of the model. + +#### run_async + +```python +run_async(text: str) -> dict[str, list[float]] | dict[str, Any] +``` + +Asynchronously embed a single string. + +This is the asynchronous version of the `run` method. It has the same parameters and return values +but can be used with `await` in async code. + +**Parameters:** + +- **text** (str) – Text to embed. + +**Returns:** + +- dict\[str, list\[float\]\] | dict\[str, Any\] – A dictionary with the following keys: +- `embedding`: The embedding of the input text. +- `meta`: Information about the usage of the model. + +## haystack_integrations.components.generators.google_genai.chat.chat_generator + +### GoogleGenAIChatGenerator + +A component for generating chat completions using Google's Gemini models via the Google Gen AI SDK. + +Supports models like gemini-3.8-flash and other Gemini variants. For Gemini 2.5 series models, +enables thinking features via `generation_kwargs={"thinking_budget": value}`. + +### Thinking Support (Gemini 2.5 and Gemini 3 Series) + +- **Reasoning transparency**: Models can show their reasoning process +- **Thought signatures**: Maintains thought context across multi-turn conversations with tools +- **Configurable thinking budgets**: Control token allocation for reasoning + +Configure thinking behavior: + +- `thinking_budget: -1`: Dynamic allocation (default) +- `thinking_budget: 0`: Disable thinking (Flash/Flash-Lite only) +- `thinking_budget: N`: Set explicit token budget + +### Multi-Turn Thinking with Thought Signatures + +Gemini uses **thought signatures** when tools are present - encrypted "save states" that maintain +context across turns. Include previous assistant responses in chat history for context preservation. + +### Authentication + +**Gemini Developer API**: Set `GOOGLE_API_KEY` or `GEMINI_API_KEY` environment variable +**Vertex AI**: Use `api="vertex"` with Application Default Credentials or API key + +### Authentication Examples + +**1. Gemini Developer API (API Key Authentication)** + +```python +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +chat_generator = GoogleGenAIChatGenerator(model="gemini-3.8-flash") +``` + +**2. Vertex AI (Application Default Credentials)** + +```python +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator + +# Using Application Default Credentials (requires gcloud auth setup) +chat_generator = GoogleGenAIChatGenerator( + api="vertex", + vertex_ai_project="my-project", + vertex_ai_location="us-central1", + model="gemini-3.8-flash", +) +``` + +**3. Vertex AI (API Key Authentication)** + +```python +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator + +# export the environment variable (GOOGLE_API_KEY or GEMINI_API_KEY) +chat_generator = GoogleGenAIChatGenerator( + api="vertex", + model="gemini-3.8-flash", +) +``` + +### Usage example + +```python +from haystack.dataclasses.chat_message import ChatMessage +from haystack.tools import Tool, Toolset +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator + +# Initialize the chat generator with thinking support +chat_generator = GoogleGenAIChatGenerator( + model="gemini-3.8-flash", + generation_kwargs={"thinking_budget": 1024} # Enable thinking with 1024 token budget +) + +# Generate a response +messages = [ChatMessage.from_user("Tell me about the future of AI")] +response = chat_generator.run(messages=messages) +print(response["replies"][0].text) + +# Access reasoning content if available +message = response["replies"][0] +if message.reasonings: + for reasoning in message.reasonings: + print("Reasoning:", reasoning.reasoning_text) + +# Tool usage example with thinking +def weather_function(city: str): + return f"The weather in {city} is sunny and 25°C" + +weather_tool = Tool( + name="weather", + description="Get weather information for a city", + parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}, + function=weather_function +) + +# Can use either List[Tool] or Toolset +chat_generator_with_tools = GoogleGenAIChatGenerator( + model="gemini-3.8-flash", + tools=[weather_tool], # or tools=Toolset([weather_tool]) + generation_kwargs={"thinking_budget": -1} # Dynamic thinking allocation +) + +messages = [ChatMessage.from_user("What's the weather in Paris?")] +response = chat_generator_with_tools.run(messages=messages) +``` + +### Usage example with structured output + +```python +from pydantic import BaseModel +from haystack.dataclasses.chat_message import ChatMessage +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator + +class City(BaseModel): + name: str + country: str + population: int + +chat_generator = GoogleGenAIChatGenerator( + model="gemini-3.8-flash", + generation_kwargs={"response_format": City} +) + +messages = [ChatMessage.from_user("Tell me about Paris")] +response = chat_generator.run(messages=messages) +print(response["replies"][0].text) # JSON output matching the City schema +``` + +### Usage example with FileContent embedded in a ChatMessage + +```python +from haystack.dataclasses import ChatMessage, FileContent +from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator + +file_content = FileContent.from_url("https://arxiv.org/pdf/2309.08632") +chat_message = ChatMessage.from_user(content_parts=[file_content, "Summarize this paper in 100 words."]) +chat_generator = GoogleGenAIChatGenerator() +response = chat_generator.run(messages=[chat_message]) +``` + +#### SUPPORTED_MODELS + +```python +SUPPORTED_MODELS: list[str] = [ + "gemini-3.8-flash", + "gemini-3.7-flash", + "gemini-3.6-flash", + "gemini-3.5-flash", + "gemini-3.5-flash-lite", + "gemini-3.1-pro-preview", + "gemini-3.1-flash-lite", + "gemini-3-flash-preview", + "gemini-2.5-pro", + "gemini-2.5-flash", + "gemini-2.5-flash-lite", +] + +``` + +A non-exhaustive list of chat models supported by this component. + +See https://ai.google.dev/gemini-api/docs/models for the full list of models and up-to-date model IDs. + +#### __init__ + +```python +__init__( + *, + api_key: Secret = Secret.from_env_var( + ["GOOGLE_API_KEY", "GEMINI_API_KEY"], strict=False + ), + api: Literal["gemini", "vertex"] = "gemini", + vertex_ai_project: str | None = None, + vertex_ai_location: str | None = None, + model: str = "gemini-3.8-flash", + generation_kwargs: dict[str, Any] | None = None, + safety_settings: list[dict[str, Any]] | None = None, + streaming_callback: StreamingCallbackT | None = None, + tools: ToolsType | None = None, + timeout: float | None = None, + max_retries: int | None = None +) -> None +``` + +Initialize a GoogleGenAIChatGenerator instance. + +**Parameters:** + +- **api_key** (Secret) – Google API key, defaults to the `GOOGLE_API_KEY` and `GEMINI_API_KEY` environment variables. + Not needed if using Vertex AI with Application Default Credentials. + Go to https://aistudio.google.com/app/apikey for a Gemini API key. + Go to https://cloud.google.com/vertex-ai/generative-ai/docs/start/api-keys for a Vertex AI API key. +- **api** (Literal['gemini', 'vertex']) – Which API to use. Either "gemini" for the Gemini Developer API or "vertex" for Vertex AI. +- **vertex_ai_project** (str | None) – Google Cloud project ID for Vertex AI. Required when using Vertex AI with + Application Default Credentials. +- **vertex_ai_location** (str | None) – Google Cloud location for Vertex AI (e.g., "us-central1", "europe-west1"). + Required when using Vertex AI with Application Default Credentials. +- **model** (str) – Name of the model to use (e.g., "gemini-3.8-flash") +- **generation_kwargs** (dict\[str, Any\] | None) – Configuration for generation (temperature, max_tokens, etc.). + For Gemini 2.5 series, supports `thinking_budget` to configure thinking behavior: +- `thinking_budget`: int, controls thinking token allocation + - `-1`: Dynamic (default for most models) + - `0`: Disable thinking (Flash/Flash-Lite only) + - Positive integer: Set explicit budget + For Gemini 3 series and newer, supports `thinking_level` to configure thinking depth: +- `thinking_level`: str, controls thinking (https://ai.google.dev/gemini-api/docs/thinking#levels-budgets) + - `minimal`: Matches the "no thinking" setting for most queries. The model may think very minimally for + complex coding tasks. Minimizes latency for chat or high throughput applications. + - `low`: Minimizes latency and cost. Best for simple instruction following, chat, or high-throughput + applications. + - `medium`: Balanced thinking for most tasks. + - `high`: (Default, dynamic): Maximizes reasoning depth. The model may take significantly longer to reach + a first token, but the output will be more carefully reasoned. +- **safety_settings** (list\[dict\[str, Any\]\] | None) – Safety settings for content filtering +- **streaming_callback** (StreamingCallbackT | None) – A callback function that is called when a new token is received from the stream. +- **tools** (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. + Each tool should have a unique name. +- **timeout** (float | None) – Timeout for Google GenAI client calls. If not set, it defaults to the default set by the Google GenAI + client. +- **max_retries** (int | None) – Maximum number of retries to attempt for failed requests. If not set, it defaults to the default set by + the Google GenAI client. + +#### warm_up + +```python +warm_up() -> None +``` + +Create the synchronous Google Gen AI client. + +#### warm_up_async + +```python +warm_up_async() -> None +``` + +Create the asynchronous Google Gen AI client. + +#### close + +```python +close() -> None +``` + +Close the synchronous Google Gen AI client. + +#### close_async + +```python +close_async() -> None +``` + +Close the asynchronous Google Gen AI client. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serializes the component to a dictionary. + +**Returns:** + +- dict\[str, Any\] – Dictionary with serialized data. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> GoogleGenAIChatGenerator +``` + +Deserializes the component from a dictionary. + +**Parameters:** + +- **data** (dict\[str, Any\]) – Dictionary to deserialize from. + +**Returns:** + +- GoogleGenAIChatGenerator – Deserialized component. + +#### run + +```python +run( + messages: list[ChatMessage] | str, + generation_kwargs: dict[str, Any] | None = None, + safety_settings: list[dict[str, Any]] | None = None, + streaming_callback: StreamingCallbackT | None = None, + tools: ToolsType | None = None, +) -> dict[str, Any] +``` + +Run the Google Gen AI chat generator on the given input data. + +**Parameters:** + +- **messages** (list\[ChatMessage\] | str) – A list of ChatMessage instances representing the input messages. + If a string is provided, it is converted to a list containing a ChatMessage with user role. +- **generation_kwargs** (dict\[str, Any\] | None) – Configuration for generation. These are merged per key with the + `generation_kwargs` passed during component initialization: keys provided here take precedence, + keys set only at initialization are kept. Supports `thinking_budget` for Gemini 2.5 series + thinking configuration. +- **safety_settings** (list\[dict\[str, Any\]\] | None) – Safety settings for content filtering. If provided, it will override the + default settings. +- **streaming_callback** (StreamingCallbackT | None) – A callback function that is called when a new token is + received from the stream. +- **tools** (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. + If provided, it will override the tools set during initialization. + +**Returns:** + +- dict\[str, Any\] – A dictionary with the following keys: +- `replies`: A list containing the generated ChatMessage responses. + +**Raises:** + +- RuntimeError – If there is an error in the Google Gen AI chat generation. +- ValueError – If a ChatMessage does not contain at least one of TextContent, ToolCall, or + ToolCallResult or if the role in ChatMessage is different from User, System, Assistant. + +#### run_async + +```python +run_async( + messages: list[ChatMessage] | str, + generation_kwargs: dict[str, Any] | None = None, + safety_settings: list[dict[str, Any]] | None = None, + streaming_callback: StreamingCallbackT | None = None, + tools: ToolsType | None = None, +) -> dict[str, Any] +``` + +Async version of the run method. Run the Google Gen AI chat generator on the given input data. + +**Parameters:** + +- **messages** (list\[ChatMessage\] | str) – A list of ChatMessage instances representing the input messages. + If a string is provided, it is converted to a list containing a ChatMessage with user role. +- **generation_kwargs** (dict\[str, Any\] | None) – Configuration for generation. These are merged per key with the + `generation_kwargs` passed during component initialization: keys provided here take precedence, + keys set only at initialization are kept. Supports `thinking_budget` for Gemini 2.5 series + thinking configuration. + See https://ai.google.dev/gemini-api/docs/thinking for possible values. +- **safety_settings** (list\[dict\[str, Any\]\] | None) – Safety settings for content filtering. If provided, it will override the + default settings. +- **streaming_callback** (StreamingCallbackT | None) – A callback function that is called when a new token is + received from the stream. +- **tools** (ToolsType | None) – A list of Tool and/or Toolset objects, or a single Toolset for which the model can prepare calls. + If provided, it will override the tools set during initialization. + +**Returns:** + +- dict\[str, Any\] – A dictionary with the following keys: +- `replies`: A list containing the generated ChatMessage responses. + +**Raises:** + +- RuntimeError – If there is an error in the async Google Gen AI chat generation. +- ValueError – If a ChatMessage does not contain at least one of TextContent, ToolCall, or + ToolCallResult or if the role in ChatMessage is different from User, System, Assistant. + +## haystack_integrations.token_counters.google_genai.token_counter + +### GoogleGenAITokenCounter + +Counts input tokens for Gemini models with Google's token counting API. + +Unlike local token counters, this counter sends the input to the `countTokens` endpoint of the Google Gen AI +SDK, so the returned count includes the model-specific formatting Gemini applies to messages. + +Inputs are assembled exactly as `GoogleGenAIChatGenerator` sends them: a leading system message becomes the +system instruction and the remaining messages become the request contents. + +### Backend support for system instructions and tools + +The Google Gen AI SDK only accepts a system instruction and tool schemas on `countTokens` when the client +targets Vertex AI. On the Gemini Developer API, a leading system message is therefore measured as a user turn, +which gives a close approximation rather than the exact count, and tools raise a `ValueError` instead of +silently returning a count that omits their schemas. Counting plain messages works on either backend. + +## Usage Example: + +```python +from haystack.dataclasses import ChatMessage +from haystack_integrations.token_counters.google_genai import GoogleGenAITokenCounter + +counter = GoogleGenAITokenCounter("gemini-3.8-flash") +messages = [ChatMessage.from_user("Hello, how are you?")] +token_count = counter.count(messages) +print(f"Token count: {token_count}") +``` + +#### __init__ + +```python +__init__( + model: str, + *, + api_key: Secret = Secret.from_env_var( + ["GOOGLE_API_KEY", "GEMINI_API_KEY"], strict=False + ), + api: Literal["gemini", "vertex"] = "gemini", + vertex_ai_project: str | None = None, + vertex_ai_location: str | None = None, + timeout: float | None = None, + max_retries: int | None = None +) -> None +``` + +Initialize the counter. + +**Parameters:** + +- **model** (str) – The model whose tokenization should be used. Token counts are model-specific, so count + against the same model you intend to generate with. +- **api_key** (Secret) – Google API key, defaults to the `GOOGLE_API_KEY` and `GEMINI_API_KEY` environment + variables. Not needed if using Vertex AI with Application Default Credentials. +- **api** (Literal['gemini', 'vertex']) – Which API to use. Either `gemini` for the Gemini Developer API or `vertex` for Vertex AI. +- **vertex_ai_project** (str | None) – Google Cloud project ID for Vertex AI. Required when using Vertex AI with + Application Default Credentials. +- **vertex_ai_location** (str | None) – Google Cloud location for Vertex AI (e.g., `us-central1`, `europe-west1`). + Required when using Vertex AI with Application Default Credentials. +- **timeout** (float | None) – Timeout for Google Gen AI client calls. If not set, it defaults to the default set by the + Google Gen AI client. +- **max_retries** (int | None) – Maximum number of retries to attempt for failed requests. If not set, it defaults to + the default set by the Google Gen AI client. + +#### warm_up + +```python +warm_up() -> None +``` + +Initialize the Google Gen AI client. + +#### count + +```python +count(messages: list[ChatMessage], tools: ToolsType | None = None) -> int +``` + +Return the number of input tokens Gemini will use for the given messages and tools. + +**Parameters:** + +- **messages** (list\[ChatMessage\]) – The messages to measure. A leading system message is measured as the system instruction on + Vertex AI and as a user turn on the Gemini Developer API, which cannot measure system instructions. +- **tools** (ToolsType | None) – Tools whose schemas are sent alongside the messages, and so consume tokens too. + +**Returns:** + +- int – The token count, or `0` when there is nothing to measure. + +**Raises:** + +- ValueError – If tools are passed while targeting the Gemini Developer API, which cannot measure them. + +#### close + +```python +close() -> None +``` + +Close the Google Gen AI client and its underlying HTTP resources. + +#### to_dict + +```python +to_dict() -> dict[str, Any] +``` + +Serialize the counter. + +**Returns:** + +- dict\[str, Any\] – A dictionary representation of the counter. + +#### from_dict + +```python +from_dict(data: dict[str, Any]) -> GoogleGenAITokenCounter +``` + +Deserialize the counter. + +**Parameters:** + +- **data** (dict\[str, Any\]) – The dictionary to deserialize from. + +**Returns:** + +- GoogleGenAITokenCounter – The deserialized counter.