diff --git a/skills/cloudflare/SKILL.md b/skills/cloudflare/SKILL.md index 2c7da21..dbf1802 100644 --- a/skills/cloudflare/SKILL.md +++ b/skills/cloudflare/SKILL.md @@ -1,6 +1,6 @@ --- name: cloudflare -description: Comprehensive Cloudflare platform skill covering Workers, Pages, storage (KV, D1, R2), AI (Workers AI, Vectorize, Agents SDK), feature flags (Flagship), networking (Tunnel, Spectrum), security (WAF, DDoS), and infrastructure-as-code (Terraform, Pulumi). Use for any Cloudflare development task. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. +description: Comprehensive Cloudflare platform skill covering Workers, Pages, storage (KV, D1, R2), AI (Workers AI, AI Search, Vectorize, Agents SDK, AI Gateway), feature flags (Flagship), networking (Tunnel, Spectrum), security (WAF, DDoS), and infrastructure-as-code (Terraform, Pulumi). Use for any Cloudflare development task. Biases towards retrieval from Cloudflare docs over pre-trained knowledge. references: - workers - pages @@ -65,7 +65,8 @@ Need storage? ├─ Object/file storage (S3-compatible) → r2/ ├─ Versioned file trees (repos, build outputs, checkpoints) → artifacts/ ├─ Message queue (async processing) → queues/ -├─ Vector embeddings (AI/semantic search) → vectorize/ +├─ Searchable documents / knowledge base (managed RAG) → ai-search/ +├─ Raw vector embeddings you compute yourself → vectorize/ ├─ Strongly-consistent per-entity state → durable-objects/ (DO storage) ├─ Secrets management → secrets-store/ ├─ Streaming ETL to R2 → pipelines/ @@ -78,11 +79,12 @@ Need storage? ``` Need AI? + ├─ Run inference (LLMs, embeddings, images) → workers-ai/ -├─ Vector database for RAG/search → vectorize/ ├─ Build stateful AI agents → agents-sdk/ -├─ Gateway for any AI provider (caching, routing) → ai-gateway/ -└─ AI-powered search widget → ai-search/ +├─ Search/RAG over your own content → ai-search/ +├─ Vector database for RAG/search → vectorize/ +└─ AI Gateway for any AI provider (caching, routing, multi-provider) → ai-gateway/ ``` ### "I need networking/connectivity" diff --git a/skills/cloudflare/references/agents-sdk/README.md b/skills/cloudflare/references/agents-sdk/README.md index f96f3ec..fd1ad65 100644 --- a/skills/cloudflare/references/agents-sdk/README.md +++ b/skills/cloudflare/references/agents-sdk/README.md @@ -86,4 +86,4 @@ export class MyAgent extends Agent { - durable-objects - Agent infrastructure - d1 - External database integration - workers-ai - AI model integration -- vectorize - Vector search for RAG patterns \ No newline at end of file +- ai-search - Managed search/RAG \ No newline at end of file diff --git a/skills/cloudflare/references/ai-search/README.md b/skills/cloudflare/references/ai-search/README.md index 52d766d..7305567 100644 --- a/skills/cloudflare/references/ai-search/README.md +++ b/skills/cloudflare/references/ai-search/README.md @@ -1,138 +1,135 @@ -# Cloudflare AI Search Reference +# Cloudflare AI Search -Expert guidance for implementing Cloudflare AI Search (formerly AutoRAG), Cloudflare's managed semantic search and RAG service. +Managed retrieval over your own content. Ingestion, chunking, embedding, storage, and ranking are handled for you; you get scored chunks back. -## Overview +**Formerly AutoRAG.** `env.AI.autorag()` still works but is frozen. See [gotchas.md](./gotchas.md#the-legacy-binding-trap). -**AI Search** is a managed RAG (Retrieval-Augmented Generation) pipeline that combines: -- Automatic semantic indexing of your content -- Vector similarity search -- Built-in LLM generation - -**Key value propositions:** -- **Zero vector management** - No manual embedding, indexing, or storage -- **Auto-indexing** - Content automatically re-indexed every 6 hours -- **Built-in generation** - Optional AI response generation from retrieved context -- **Multi-source** - Index from R2 buckets or website crawls - -**Data source options:** -- **R2 bucket** - Index files from Cloudflare R2 (supports MD, TXT, HTML, PDF, DOC, CSV, JSON) -- **Website** - Crawl and index website content (requires Cloudflare-hosted domain) - -**Indexing lifecycle:** -- Automatic 6-hour refresh cycle -- Manual "Force Sync" available (30s rate limit) -- Not designed for real-time updates +[The docs](https://developers.cloudflare.com/ai-search/) are authoritative over these files. ## Quick Start -**1. Create AI Search instance in dashboard:** -- Go to Cloudflare Dashboard → AI Search → Create -- Choose data source (R2 or website) -- Configure instance name and settings - -**2. Configure Worker:** - ```jsonc // wrangler.jsonc { - "ai": { - "binding": "AI" - } + "compatibility_date": "2026-03-27", // or later + "ai_search_namespaces": [{ "binding": "AI_SEARCH", "namespace": "default" }] } ``` -**3. Use in Worker:** - ```typescript +interface Env { AI_SEARCH: AiSearchNamespace } + export default { - async fetch(request, env) { - const answer = await env.AI.autorag("my-search-instance").aiSearch({ - query: "How do I configure caching?", - model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast" + async fetch(request, env: Env) { + const { chunks } = await env.AI_SEARCH.get("my-instance").search({ + messages: [{ role: "user", content: "How do I configure caching?" }], }); - - return Response.json({ answer: answer.response }); - } + return Response.json(chunks.map((c) => ({ source: c.item.key, text: c.text }))); + }, }; ``` -## When to Use AI Search - -### AI Search vs Vectorize - -| Factor | AI Search | Vectorize | -|--------|-----------|-----------| -| **Management** | Fully managed | Manual embedding + indexing | -| **Use when** | Want zero-ops RAG pipeline | Need custom embeddings/control | -| **Indexing** | Automatic (6hr cycle) | Manual via API | -| **Generation** | Built-in optional | Bring your own LLM | -| **Data sources** | R2 or website | Manual insert | -| **Best for** | Docs, support, enterprise search | Custom ML pipelines, real-time | - -### AI Search vs Direct Workers AI - -| Factor | AI Search | Workers AI (direct) | -|--------|-----------|---------------------| -| **Context** | Automatic retrieval | Manual context building | -| **Use when** | Need RAG (search + generate) | Simple generation tasks | -| **Indexing** | Built-in | Not applicable | -| **Best for** | Knowledge bases, docs | Simple chat, transformations | - -### search() vs aiSearch() - -| Method | Returns | Use When | -|--------|---------|----------| -| `search()` | Search results only | Building custom UI, need raw chunks | -| `aiSearch()` | AI response + results | Need ready-to-use answer (chatbot, Q&A) | - -### Real-time Updates Consideration - -**AI Search is NOT ideal if:** -- Need real-time content updates (<6 hours) -- Content changes multiple times per hour -- Strict freshness requirements - -**AI Search IS ideal if:** -- Content relatively stable (docs, policies, knowledge bases) -- 6-hour refresh acceptable -- Prefer zero-ops over real-time - -## Platform Limits - -| Limit | Value | -|-------|-------| -| Max instances per account | 10 | -| Max files per instance | 100,000 | -| Max file size | 4 MB | -| Index frequency | Every 6 hours | -| Force Sync rate limit | Once per 30 seconds | -| Filter nesting depth | 2 levels | -| Filters per compound | 10 | -| Score threshold range | 0.0 - 1.0 | - -## Reading Order - -Navigate these references based on your task: - -| Task | Read | Est. Time | -|------|------|-----------| -| **Understand AI Search** | README only | 5 min | -| **Implement basic search** | README → api.md | 10 min | -| **Configure data source** | README → configuration.md | 10 min | -| **Production patterns** | patterns.md | 15 min | -| **Debug issues** | gotchas.md | 10 min | -| **Full implementation** | README → api.md → patterns.md | 30 min | +**`compatibility_date` must be `2026-03-27` or later.** An older one is the usual cause of "the binding is undefined." Add `"remote": true` for `wrangler dev`; there is no local emulator. ## In This Reference -- **[api.md](api.md)** - API endpoints, methods, TypeScript interfaces -- **[configuration.md](configuration.md)** - Setup, data sources, wrangler config -- **[patterns.md](patterns.md)** - Common patterns, decision guidance, code examples -- **[gotchas.md](gotchas.md)** - Troubleshooting, code-level gotchas, limits +Read in this order. + +| File | When | Covers | +|---|---|---| +| [configuration.md](./configuration.md#decide-before-you-create) | Before creating an instance | Data sources, path filtering, indexing, models, metadata. The embedding model and data source are fixed for the life of an instance | +| [api.md](./api.md) | While implementing | Bindings, `search()`, filters, boosting, Items API, failure surfaces, REST and CLI | +| [patterns.md](./patterns.md) | Designing the integration | Agent tools, RAG you own, multitenancy, agent memory, retrieval tuning | +| [gotchas.md](./gotchas.md) | Results are empty or wrong | Migration traps, silent failures, limits, error codes | + +## Default to `search()` + +`search()` returns scored chunks. `chatCompletions()` runs a model over those chunks inside AI Search and returns prose. + +Use `chatCompletions()` only when the generated answer *is* the shipped artifact: an embedded FAQ box, a public endpoint. Everywhere else use `search()` and generate yourself. `chatCompletions()` exposes only `model:`, so you get no control over the prompt, no tool calling, no multi-turn state, and no chance to filter or dedupe chunks before generation. + +## Building an agent? Use the Agents SDK + +**The Agents SDK owns the loop; AI Search is one tool it calls.** An agent built on `chatCompletions()` has no memory, no tools, and no loop. + +```typescript +searchDocs = tool({ + description: "Search the product documentation.", + parameters: z.object({ query: z.string() }), + execute: async ({ query }) => { + const { chunks } = await this.env.AI_SEARCH.get("product-docs").search({ + messages: [{ role: "user", content: query }], + }); + return chunks.map((c) => ({ source: c.item.key, text: c.text })); + }, +}); +``` + +Full example in [patterns.md](./patterns.md#search-as-an-agent-tool). See [agents-sdk](../agents-sdk/). + +## Object Model + +``` +Account → Namespace → Instance → Items +``` + +The **instance** is the searchable unit. Names collide only within a namespace, so `docs` can exist in both `blog` and `support`. A `default` namespace always exists. + +**An instance is an ordinary API object, not provisioned infrastructure.** `create()` is a normal call and the per-account ceiling is high, so one instance per tenant or per agent is viable, and usually better than one shared index filtered by `tenantId`. See [patterns.md](./patterns.md#multitenancy). + +## Two Bindings + +| Binding | Access | Use when | +|---------|--------|----------| +| `ai_search_namespaces` | Every instance in a namespace | **Default.** Runtime instance choice, `create()` / `list()` / `delete()`, cross-instance search, per-tenant isolation | +| `ai_search` | One instance, fixed at deploy time | The Worker only ever reads one instance that already exists | + +The namespace handle is a strict superset; switching to it later means a config change, a code change, and a redeploy. See [api.md](./api.md#bindings). + +## Search Modes + +| Mode | Finds | To use it | +|------|-------|-----------| +| `vector` | Semantic: "how to ship my app" matches "deployment guide" | On by default | +| `keyword` | Exact terms via BM25: `ERR_CONNECTION_REFUSED` | Enable `index_method.keyword` | +| `hybrid` | Both, fused | Enable both index methods | + +New instances index **vector only**. Querying with `retrieval_type: "hybrid"` requires both index methods enabled at index time, which triggers a full reindex and lowers the file ceiling. See [gotchas.md](./gotchas.md#retrieval-can-only-use-an-index-you-built). + +## Data Sources + +An instance combines built-in storage with at most one external source. Pick by where the content already lives. + +| Content lives | Use | Freshness | +|---|---|---| +| Nowhere yet, or you own the write path | **Built-in storage** | Immediate, per file | +| An R2 bucket already | **R2** | Hours-scale sync | +| A website you own | **Crawl** | Hours-scale sync | + +Built-in storage indexes each file as it arrives through the [Items API](./api.md#items-api), so write-then-read works. That makes AI Search usable as agent memory: see [patterns.md](./patterns.md#per-agent-knowledge-isolation). + +Details in [configuration.md](./configuration.md#data-sources). + +## When to Use Something Else + +**Skip AI Search when** the corpus is small and structured enough that ordinary querying wins. + +**Use [vectorize](../vectorize/) instead when** you need your own chunking or embeddings, or vectors not derived from documents (image embeddings, user-preference vectors, recommendation features). + +| | AI Search | Vectorize | +|---|---|---| +| Chunking and embedding | Managed | Yours | +| Keyword/BM25 and hybrid fusion | Built in | Not available | +| Reranking, query rewriting, caching | Built in, toggleable | Yours to build | +| Ingest | Text, code, PDF, Office, images | Vectors only | + +Limits that shape a design are in [gotchas.md](./gotchas.md#limits). Do not pin numbers from memory. ## See Also -- [Cloudflare AI Search Docs](https://developers.cloudflare.com/ai-search/) -- [Workers AI Docs](https://developers.cloudflare.com/workers-ai/) -- [Vectorize Docs](https://developers.cloudflare.com/vectorize/) +- [AI Search docs](https://developers.cloudflare.com/ai-search/) - full option lists and current defaults +- [llms.txt](https://developers.cloudflare.com/ai-search/llms.txt) - index of every doc page, for retrieval +- [agents-sdk](../agents-sdk/) - the right home for agent loops +- [vectorize](../vectorize/) - the unmanaged alternative +- [workers-ai](../workers-ai/) - models for your own generation step +- [ai-gateway](../ai-gateway/) - routing and observability for model calls diff --git a/skills/cloudflare/references/ai-search/api.md b/skills/cloudflare/references/ai-search/api.md index b6220c4..d39d8be 100644 --- a/skills/cloudflare/references/ai-search/api.md +++ b/skills/cloudflare/references/ai-search/api.md @@ -1,87 +1,278 @@ # AI Search API Reference -## Workers Binding +## Complete Example + +Create, upload, and search through the namespace binding. ```typescript -const answer = await env.AI.autorag("instance-name").aiSearch(options); -const results = await env.AI.autorag("instance-name").search(options); -const instances = await env.AI.autorag("_").listInstances(); +// wrangler.jsonc: { "compatibility_date": "2026-03-27", +// "ai_search_namespaces": [{ "binding": "AI_SEARCH", "namespace": "default" }] } + +interface Env { AI_SEARCH: AiSearchNamespace } + +const INSTANCE = "product-docs"; + +export default { + async fetch(request: Request, env: Env) { + const url = new URL(request.url); + + // Synchronous and lazy: no network call, no validation. + const docs = env.AI_SEARCH.get(INSTANCE); + + // One-time setup. Omitting type/source gives built-in storage. + // create() throws if the instance already exists. + if (url.pathname === "/setup") { + await env.AI_SEARCH.create({ id: INSTANCE }).catch(() => {}); + return new Response("ready"); + } + + // Write. uploadAndPoll waits for indexing, so the next search sees it. + // Check the returned status: the wait is bounded, not guaranteed. + if (request.method === "POST") { + const key = url.searchParams.get("key")!; + const item = await docs.items.uploadAndPoll(key, await request.text()); + return Response.json({ key, status: item.status }); + } + + // Query. Using the search API to get back a list of relevant chunks. + const { chunks } = await docs.search({ + messages: [{ role: "user", content: url.searchParams.get("q")! }], + }); + + // Zero results is a normal outcome, not an error. + if (chunks.length === 0) return Response.json({ results: [] }); + + return Response.json({ + results: chunks.map((c) => ({ source: c.item.key, text: c.text, score: c.score })), + }); + }, +} satisfies ExportedHandler; ``` -## aiSearch() Options +## Bindings + +| Binding | Handle type | Gives you | +|---------|-------------|-----------| +| `ai_search` | `AiSearchInstance` | One instance, fixed at deploy time | +| `ai_search_namespaces` | `AiSearchNamespace` | Every instance in a namespace, chosen at runtime | ```typescript -interface AiSearchOptions { - query: string; // User query - model: string; // Workers AI model ID - system_prompt?: string; // LLM instructions - rewrite_query?: boolean; // Fix typos (default: false) - max_num_results?: number; // Max chunks (default: 10) - ranking_options?: { score_threshold?: number }; // 0.0-1.0 (default: 0.3) - reranking?: { enabled: boolean; model: string }; - stream?: boolean; // Stream response (default: false) - filters?: Filter; // Metadata filters - page?: string; // Pagination token +interface Env { + DOCS: AiSearchInstance; // ai_search + TENANTS: AiSearchNamespace; // ai_search_namespaces } + +const instance = env.TENANTS.get("tenant-abc"); ``` -## Response +Both handles expose `search()`, `chatCompletions()`, `info()`, `stats()`, `update()`, `items.*`, and `jobs.*`. The namespace handle adds `get()`, `list()`, `create()`, `delete()`, and cross-instance `search()`. + +Binding declaration, `remote: true`, and the `compatibility_date` minimum are in [configuration.md](configuration.md#worker-setup). **If a binding is `undefined` at runtime, check the compatibility date first.** + +## `search()` ```typescript -interface AiSearchResponse { - search_query: string; // Query used (rewritten if enabled) - response: string; // AI-generated answer - data: SearchResult[]; // Retrieved chunks - has_more: boolean; - next_page?: string; -} +const { search_query, chunks } = await instance.search({ + messages: [{ role: "user", content: "How do I configure caching?" }], + ai_search_options: { retrieval: { max_num_results: 8 } }, +}); +``` + +`messages` takes OpenAI-style `{ role, content }`. A bare `query: string` also works; pass one or the other, not both. + +**Building an agent? Give it `search()` as a tool** rather than retrieving on every turn. See [patterns.md](patterns.md#search-as-an-agent-tool). + +### Response -interface SearchResult { - id: string; - score: number; - content: string; - metadata: { filename: string; folder: string; timestamp: number }; +```typescript +{ + search_query, // the query actually searched, after any rewriting + chunks: [{ + id, text, score, + item: { key, timestamp, metadata }, // key = file path or source URL + scoring_details: { ... }, // per-signal scores and ranks + instance_id, // cross-instance search only + }], + errors, // cross-instance partial failures } ``` +`scoring_details` decomposes the score into vector, keyword, fusion, and reranking components. Log it before tuning anything. + +### Options that carry a decision + +Everything under `ai_search_options` has a working default. These are worth setting deliberately: + +| Option | Set it when | +|--------|-------------| +| `retrieval.max_num_results` | Feeding a model: cap the context you hand it | +| `retrieval.match_threshold` | You would rather return nothing than something wrong (raise), or a model filters downstream (lower) | +| `retrieval.filters` | Multitenancy, folder scoping, freshness. See [Filters](#filters) | +| `retrieval.retrieval_type` | Exact strings matter. Only reads an index built at index time. See [gotchas.md](gotchas.md#retrieval-can-only-use-an-index-you-built) | +| `retrieval.context_expansion` | Chunks come back truncated mid-thought | +| `retrieval.boost_by` | Fresh or high-priority content should outrank the rest. See [Boosting](#boosting) | +| `query_rewrite.enabled` | Input is raw human text. Leave off for agent-generated queries | +| `reranking.enabled` | Right documents, wrong order. Adds a model call | +| `cache.enabled` | Benchmarking, or reading right after a reindex. Caching is **on** by default | + +Full list and current defaults: [Workers binding search](https://developers.cloudflare.com/ai-search/api/search/workers-binding/). + +### Cross-instance search + +Namespace handle only. `instance_ids` is required; results are merged and re-ranked. + +```typescript +const { chunks, errors } = await env.TENANTS.search({ + messages: [{ role: "user", content: "refund policy" }], + ai_search_options: { instance_ids: ["product-docs", "changelog"] }, +}); +if (errors?.length) console.warn("partial results", errors); +``` + +Each chunk carries `instance_id` for attribution. Failures land in `errors` rather than throwing; see [Failure surfaces](#failure-surfaces). + +## `chatCompletions()` + +Retrieves and generates in one call, returning prose. Retrieval is unconditional and the prompt is not yours. Use it only when the generated answer *is* the shipped artifact; otherwise use [`search()`](#search). + +```typescript +const response = await instance.chatCompletions({ + messages: [{ role: "user", content: "What is Cloudflare?" }], + ai_search_options: { retrieval: { max_num_results: 5 } }, +}); +// response.choices[0].message.content, plus response.chunks in search() shape +``` + +OpenAI-shaped, extended with the retrieved `chunks`. `model` overrides the instance's generation model. + +**Streaming emits chunks first.** With `stream: true` the stream sends a single `event: chunks` frame, then `chat.completion.chunk` deltas, then `data: [DONE]`. That ordering lets a UI render sources before the answer. + ## Filters +Metadata keys with conditions, applied **before** retrieval. Both query methods take them. + +```typescript +ai_search_options: { + retrieval: { + filters: { + folder: "docs/getting-started/", // bare value = implicit $eq + timestamp: { $gte: 1735689600 }, + category: { $in: ["guides", "tutorials"] }, + }, + }, +} +``` + +Operators: `$eq` `$ne` `$in` `$nin` `$lt` `$lte` `$gt` `$gte`. **Multiple keys are ANDed implicitly.** There is no `and` / `or` key; use `$in` for what used to be an OR over one field. + +**"Starts with" needs a range query.** Bare equality matches only direct children: + +```typescript +// ❌ direct children of docs/ only, misses docs/guides/intro.md +filters: { folder: "docs/" } + +// ✅ whole subtree ('0' sorts after '/' in ASCII) +filters: { folder: { $gte: "docs/", $lt: "docs0" } } +``` + +Filterable fields are `filename`, `folder`, `timestamp`, plus custom fields declared on the instance *before* upload. See [configuration.md](configuration.md#custom-metadata). + +Reference: [Filtering](https://developers.cloudflare.com/ai-search/configuration/retrieval/filtering/). + +## Boosting + +Filters remove candidates before retrieval; `boost_by` reorders what retrieval already found. **It cannot promote a chunk the search step missed.** Widen retrieval first, then boost. + +```typescript +ai_search_options: { + retrieval: { + boost_by: [{ field: "timestamp", direction: "desc" }], // newest first + }, +} +``` + +| `direction` | Ranks higher | +|---|---| +| `desc` | Higher values: most recent, highest priority | +| `asc` | Lower values: oldest, lowest priority | +| `exists` | Documents that have the field | +| `not_exists` | Documents missing the field. How you suppress drafts | + +## Failure surfaces + +The binding throws. Narrow on `err.name`, keyed to the upstream HTTP status: + +| Status | `err.name` | Do | +|---|---|---| +| 404 | `AiSearchNotFoundError` | Fix the instance or namespace name. Do not retry | +| 5xx | `AiSearchInternalError` | Retry | +| other | `AiSearchError` | Read `message`. Do not retry blind | + +**Use `err.name`, not `instanceof`.** These are type declarations, not exported runtime classes, so there is no constructor to compare against. `message` carries the AI Search error string (`ai_search_not_found`, `namespace_not_found`), and numeric codes are in [API error codes](https://developers.cloudflare.com/ai-search/troubleshooting/api-error-codes/). The legacy `AutoRAGNotFoundError` family is not thrown here. + +## Items API + +Documents in an instance's built-in storage. Indexes immediately, no sync jobs. + ```typescript -// Comparison -{ column: "folder", operator: "gte", value: "docs/" } - -// Compound -{ operator: "and", filters: [ - { column: "folder", operator: "gte", value: "docs/" }, - { column: "timestamp", operator: "gte", value: 1704067200 } -]} +await instance.items.upload("faq.md", content); // string | ArrayBuffer | ReadableStream +// metadata fields must be declared on the instance first: configuration.md#custom-metadata +await instance.items.upload("guide.pdf", buf, { metadata: { category: "onboarding" } }); +const item = await instance.items.uploadAndPoll("handbook.txt", content); // blocks until indexed +await instance.items.list({ status, search, source }); +await instance.items.get(itemId).info(); // also .download(), and items.delete(itemId) ``` -**Operators:** `eq`, `ne`, `gt`, `gte`, `lt`, `lte` +**Status is a state machine**: `queued` → `running` → `completed` | `error`, plus `outdated` when the source changed since indexing. Only `completed` items are searchable. -**Built-in metadata:** `filename`, `folder`, `timestamp` (Unix seconds) +```typescript +// ❌ serializes the whole ingest, one round trip per file +for (const f of files) await instance.items.uploadAndPoll(f.key, f.body); + +// ✅ upload, then poll once +await Promise.all(files.map((f) => instance.items.upload(f.key, f.body))); +await instance.stats(); +``` -## Streaming +## Instances API + +Namespace handle only. ```typescript -const stream = await env.AI.autorag("docs").aiSearch({ query, model, stream: true }); -return new Response(stream, { headers: { "Content-Type": "text/event-stream" } }); +const instance = env.TENANTS.get("my-instance"); // sync, lazy, unvalidated +await env.TENANTS.list({ search }); +await env.TENANTS.create({ id: "knowledge-base" }); // omit type/source = built-in storage +await env.TENANTS.delete("old-docs"); // permanent, drops all indexed data ``` -## Error Types +`create()` requires only `id`, returns a usable handle, and **throws if the instance already exists** with no typed error to narrow on. Every ingestion, indexing, retrieval, and model setting is also a `create()` parameter, all settable later with `update()`. See [configuration.md](configuration.md). + +On both handles: `info()` (full config and status), `stats()` (indexing progress, error counts), `update()`. + +## Beyond the binding -| Error | Cause | -|-------|-------| -| `AutoRAGNotFoundError` | Instance doesn't exist | -| `AutoRAGUnauthorizedError` | Invalid/missing token | -| `AutoRAGValidationError` | Invalid parameters | +Inside your own Worker, use the binding: faster, and it authenticates itself. -## REST API +**REST** lives at `/accounts/{account_id}/ai-search/instances/{id}/...`, with `/ai-search/namespaces/{namespace}/...` for non-default namespaces, and takes the same body shape as the binding. **Wrangler** covers instances, namespaces, and jobs; every command takes `--json`. ```bash -curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/autorag/rags/{NAME}/ai-search \ - -H "Authorization: Bearer {TOKEN}" \ - -d '{"query": "...", "model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast"}' +npx wrangler ai-search --help +npx wrangler ai-search jobs create # sync an external source now ``` -Requires Service API token with "AI Search - Read" permission. +Both need an API token with **both** `AI Search:Edit` **and** `AI Search:Run`. Edit alone cannot query; Run alone cannot manage instances. + +**Public endpoint and MCP.** Enabled per instance ([configuration.md](configuration.md#public-endpoint-and-ui-snippets)). `/mcp` exposes a single `search` tool whose name and description you control: reach for it before writing a custom MCP server. + +## Old patterns + +`env.AI.autorag(name).search()` and `.aiSearch()` still work indefinitely but receive no new features: no namespaces, Items API, keyword or hybrid search, boosting, cross-instance search, or `scoring_details`. REST endpoints under `/autorag/rags/{name}/` are deprecated alongside them. + +Field-by-field migration mapping: [gotchas.md](gotchas.md#the-legacy-binding-trap). + +## See Also + +- [configuration.md](configuration.md) - creating and configuring the instance you are querying +- [patterns.md](patterns.md) - agent tools, RAG, multitenancy, tuning +- [gotchas.md](gotchas.md) - what to check when results are empty +- [AI Search API docs](https://developers.cloudflare.com/ai-search/api/) - current signatures and defaults diff --git a/skills/cloudflare/references/ai-search/configuration.md b/skills/cloudflare/references/ai-search/configuration.md index d1f34ad..9afe43e 100644 --- a/skills/cloudflare/references/ai-search/configuration.md +++ b/skills/cloudflare/references/ai-search/configuration.md @@ -1,88 +1,237 @@ # AI Search Configuration -## Worker Setup +Full option list, current defaults, ranges, and enums: the [configuration docs](https://developers.cloudflare.com/ai-search/configuration/), `instance.info()`, and `npx wrangler ai-search create --help`. + +## Worker setup ```jsonc // wrangler.jsonc { - "ai": { "binding": "AI" } + "compatibility_date": "2026-03-27", // or later + "ai_search_namespaces": [ + { "binding": "AI_SEARCH", "namespace": "default", "remote": true } + ] } ``` ```typescript -interface Env { - AI: Ai; -} +interface Env { AI_SEARCH: AiSearchNamespace } -const answer = await env.AI.autorag("my-instance").aiSearch({ - query: "How do I configure caching?", - model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast" +const { chunks } = await env.AI_SEARCH.get("my-instance").search({ + messages: [{ role: "user", content: "How do I configure caching?" }], }); ``` -## Data Sources +**Settings change from the Worker, not just the dashboard.** `update()` takes a partial of the same object `create()` takes, applies only the keys you pass, and returns the instance info that `info()` would return. + +```typescript +const instance = env.AI_SEARCH.get("my-instance"); + +const info = await instance.update({ + index_method: { vector: true, keyword: true }, + retrieval_options: { keyword_match_mode: "and" }, +}); +``` + +Omitted keys keep their current value, so read before you write only when you need the old value. What `update()` cannot fix is in [Decide before you create](#decide-before-you-create). + +- **`compatibility_date` must be at least `2026-03-27`.** An older one leaves the binding `undefined` at runtime, the most common setup failure. +- `remote: true` proxies `wrangler dev` to the deployed instance. There is no local emulator, so local dev hits real data and real query limits. +- Every account has a `default` namespace. Wrangler creates any other namespace you name on deploy. -### R2 Bucket +Use `ai_search` only when the Worker reads exactly one instance that already exists. It has no `get()`, `list()`, `create()`, `delete()`, or cross-instance search. See [api.md](api.md#bindings). -Dashboard: AI Search → Create Instance → Select R2 bucket +## Decide before you create -**Supported formats:** `.md`, `.txt`, `.html`, `.pdf`, `.doc`, `.docx`, `.csv`, `.json` +Surface these to the user rather than picking silently. -**Auto-indexed metadata:** `filename`, `folder`, `timestamp` +**Cannot be changed on an existing instance. Create a new one.** -### Website Crawler +| Decision | Why | +|----------|-----| +| **Embedding model** | Different dimensions, different vector space. Existing vectors are not converted, so there is nothing to migrate in place. | +| **Data source type** | An instance takes built-in storage plus at most one external source. Switching that source type in place is unsupported. | -Requirements: -- Domain on Cloudflare -- `sitemap.xml` at root -- Bot protection must allow `CloudflareAISearch` user agent +**Changing an indexing setting reindexes the whole instance.** -## Path Filtering (R2) +`index_method`, `indexing_options`, and `chunk` / `chunk_size` / `chunk_overlap` are applied to an item when it is indexed, so `update()` resyncs the existing corpus to apply them. +## Creating an instance + +**Default to built-in storage.** Only `id` is required. Omit `type` and `source` and the instance provisions its own R2 and Vectorize, and you write to it directly. + +```bash +npx wrangler ai-search create my-instance ``` -docs/**/*.md # All .md in docs/ recursively -**/*.draft.md # Exclude (use in exclude patterns) + +```typescript +// requires the namespace binding; create() returns the instance handle +const instance = await env.AI_SEARCH.create({ id: "my-instance" }); +await instance.items.upload("faq.md", content); ``` -## Indexing +Add `type` and `source` only to point at content that already lives elsewhere. See [Data sources](#data-sources). -- **Automatic:** Every 6 hours -- **Force Sync:** Dashboard button (30s rate limit between syncs) -- **Pause:** Settings → Pause Indexing (existing index remains searchable) +```bash +npx wrangler ai-search create my-instance --type r2 --source my-docs-bucket --hybrid-search +``` -## Service API Token +IDs are short, lowercase alphanumeric with `-` and `_`. Every ingestion, indexing, retrieval, and model setting is also a `create()` parameter; `create --help` lists current flags and the ambient types give the accepted shape. -Dashboard: AI Search → Instance → Use AI Search → API → Create Token +## Data sources -Permissions: -- **Read** - search operations -- **Edit** - instance management +An instance combines built-in storage with **one** external source. Pick by where the content already lives. -Store securely: -```bash -wrangler secret put AI_SEARCH_TOKEN +| Situation | Source | Freshness | +|-----------|--------|-----------| +| New corpus, or you own the write path | **Built-in storage** | Immediate, per file | +| Content already sitting in an R2 bucket | **R2** | Hours-scale sync | +| Content is a website you own | **Website crawl** | Hours-scale sync | + +**R2 is the only data source that needs a service API token**, with both `AI Search:Edit` and `AI Search:Run`. Pass its UUID as `token_id` when creating your first R2-backed instance. An expired token shows up as an instance that silently stops indexing. + +**Website crawl** requires a domain you own. Two non-obvious options: + +```typescript +source_params: { + include_items: ["**/docs/**"], // the leading ** matters, see Path filtering + web_crawler: { + parse_type: "sitemap", + parse_options: { use_browser_rendering: true }, // JS-rendered pages, else you index the shell + }, +} ``` -## Multi-Environment +Full `source_params`: [Data sources](https://developers.cloudflare.com/ai-search/configuration/data-source/). -```toml -# wrangler.toml -[env.production.vars] -AI_SEARCH_INSTANCE = "prod-docs" +## Path filtering + +`include_items` and `exclude_items` on `source_params` decide which paths enter the index at all. Distinct from query-time [`filters`](api.md#filters): path filtering controls what exists, `filters` controls what a query may see. Built-in storage needs neither. + +**Exclude is evaluated first and wins**, whether or not an include also matches. Then, if any include exists, an item must match at least one. Setting neither indexes everything. + +**Patterns are anchored to the whole path, not searched inside it.** The most common reason a filter indexes nothing: + +| Pattern | Matches | Skips | +|---|---|---| +| `docs/*` | `docs/file.pdf` | `docs/sub/file.pdf`, `site/docs/file.pdf` | +| `docs/**` | `docs/file.pdf`, `docs/sub/file.pdf` | `site/docs/file.pdf` | +| `**/docs/*` | `docs/file.pdf`, `site/docs/file.pdf` | `docs/sub/file.pdf` | + +```typescript +// ❌ misses subfolders and any nested docs/ directory +include_items: ["docs/*"] + +// ✅ catches every row above +include_items: ["**/docs/**"] +``` + +`*` stops at a `/`; `**` crosses them and matches zero segments as well as many. Those are the only two wildcards: no brace expansion, character classes, or negation, so `*.{md,pdf}` is two patterns. Patterns are **case-sensitive** (`/Blog/*` misses `/blog/post.html`) and compared **without normalization** (`/blog/` and `/blog` differ). + +**The matched string is shaped differently per source.** R2 patterns run against object paths with a leading slash (`/docs/guide.pdf`). Website patterns run against host-inclusive URLs without the scheme (`example.com/blog/post`), which is why a website pattern almost always needs a leading `**`. + +[Path filtering](https://developers.cloudflare.com/ai-search/configuration/indexing/path-filtering/). + +## Indexing and syncing -[env.staging.vars] -AI_SEARCH_INSTANCE = "staging-docs" +Built-in storage indexes immediately, per file, with no jobs. External sources sync on an hours-scale schedule. + +**Every sync of an external source is a job.** The job list is therefore the sync history, and where to look when a source is stale. + +```bash +npx wrangler ai-search jobs list # sync history +npx wrangler ai-search jobs logs # what one sync did +npx wrangler ai-search jobs create # sync now ``` +The same surface is on the binding, so a webhook handler triggers and inspects syncs without shelling out: + ```typescript -const answer = await env.AI.autorag(env.AI_SEARCH_INSTANCE).aiSearch({ query }); +await instance.jobs.create(); // sync now +await instance.jobs.list(); // sync history +await instance.jobs.get(jobId).logs(); // what one sync did +await instance.jobs.get(jobId).cancel(); // stop a running sync ``` -## Monitoring +**To keep an external source fresh, trigger a job on publish** from a Worker webhook or CI/CD, rather than shortening `sync_interval`. The endpoint is rate-limited, so this is a per-change trigger, not a polling loop. + +`stats()` is the monitoring surface: counts by state (`queued` / `running` / `completed` / `error` / `outdated`), last activity, embedding errors. + +## Search modes + +New instances index **vector only**. Enable keyword indexing to get BM25 or hybrid. ```typescript -const instances = await env.AI.autorag("_").listInstances(); -console.log(instances.find(i => i.name === "docs")); +await env.AI_SEARCH.create({ + id: "docs", + index_method: { vector: true, keyword: true }, // both = hybrid available + fusion_method: "rrf", // or "max" + indexing_options: { keyword_tokenizer: "porter" }, // or "trigram" + retrieval_options: { keyword_match_mode: "and" }, // or "or" +}); ``` -Dashboard shows: files indexed, status, last index time, storage usage. +| Setting | Choose | +|---------|--------| +| `keyword_tokenizer` | `trigram` for code, IDs, error strings, substring matching. `porter` (stemming) for prose | +| `fusion_method` | `rrf` to blend both signals. `max` when one should dominate | + +## Models + +Model calls route through AI Gateway, so Anthropic, OpenAI, Google, and others are selectable alongside `@cf/*` models. Set `ai_gateway_id` to route through your own gateway. + +Four slots: `embedding_model`, `ai_search_model` (generation, `chatCompletions()` only), `rewrite_model`, `reranking_model`. The last three are changeable with `update()`. **If you only use `search()`, `embedding_model` is the only one that matters**, and it is fixed for the life of the instance: see [Decide before you create](#decide-before-you-create). + +**Do not pin a model name from memory.** The supported list changes often and a wrong name fails at request time. Read [Supported models](https://developers.cloudflare.com/ai-search/configuration/models/supported-models/). + +## Metadata + +`filename`, `folder`, and `timestamp` are always available and always filterable. + +### Custom metadata + +**Declare fields on the instance before uploading items with them.** Undeclared keys are not indexed and cannot be filtered on, and this fails silently: the upload succeeds, the filter just never matches. The field-count cap is small, so declare only fields you will filter by. + +```typescript +await env.AI_SEARCH.create({ + id: "docs", + custom_metadata: [ + { field_name: "category", data_type: "text" }, // text | number | boolean | datetime + { field_name: "views", data_type: "number" }, + ], +}); + +await instance.items.upload("guide.pdf", buf, { metadata: { category: "onboarding" } }); +``` + +Adding a field later does not backfill already-indexed items; reupload or resync them. + +## Public endpoint and UI snippets + +Enable per instance in the dashboard (**Settings** → **Public Endpoint**) for an unauthenticated `https://.search.ai.cloudflare.com/` serving `/search`, `/chat/completions`, and `/mcp`, with rate limiting, CORS, and authorized hosts in the same panel. + +**`/mcp` exposes a single `search` tool whose name and description you control.** Reach for it before writing a custom MCP server. + +Drop-in web components (search bar, Cmd+K modal, chat bubble, full-page chat) are at [search.ai.cloudflare.com](https://search.ai.cloudflare.com/), with the current versioned script URL in the instance's dashboard panel. + +## Multi-environment + +Bindings are per-environment. Instance names only need to be unique within a namespace, so **make the environment be the namespace and keep instance names identical across them**. `env.AI_SEARCH.get("docs")` then resolves to whichever copy the deployed environment binds, and application code stays environment-agnostic. + +```toml +[[env.production.ai_search_namespaces]] +binding = "AI_SEARCH" +namespace = "production" + +[[env.staging.ai_search_namespaces]] +binding = "AI_SEARCH" +namespace = "staging" +``` + +Give staging its own instance rather than pointing `remote: true` at production. + +## See Also + +- [api.md](api.md) - querying the instance you just configured +- [gotchas.md](gotchas.md) - migration traps, limits, silent failures +- [AI Search configuration docs](https://developers.cloudflare.com/ai-search/configuration/) - full option list and current defaults diff --git a/skills/cloudflare/references/ai-search/gotchas.md b/skills/cloudflare/references/ai-search/gotchas.md index 04c987f..6a6e7ad 100644 --- a/skills/cloudflare/references/ai-search/gotchas.md +++ b/skills/cloudflare/references/ai-search/gotchas.md @@ -1,81 +1,177 @@ # AI Search Gotchas -## Type Safety +## The legacy binding trap + +Most AI Search code in circulation, including model pretraining, predates the current API. `env.AI.autorag(...)`, `aiSearch()`, a `data[]` response, or `{ type, key, value }` filters are all the legacy AutoRAG surface. It works indefinitely but receives no new features: no namespaces, Items API, hybrid search, boosting, cross-instance search, or `scoring_details`. + +| Legacy | Current | +|--------|---------| +| `"ai": { "binding": "AI" }` | `ai_search` or `ai_search_namespaces` binding | +| `env.AI.autorag("name")` | `env.MY_SEARCH` or `env.AI_SEARCH.get("name")` | +| `aiSearch()` | `chatCompletions()`, but prefer `search()` | +| `query: "..."` | `messages: [{ role, content }]` (`query` still accepted) | +| `data[]` | `chunks[]` | +| `data[].file_id` | `chunks[].id` | +| `data[].filename` | `chunks[].item.key` | +| `data[].content[].text` | `chunks[].text` | +| `data[].attributes.modified_date` | `chunks[].item.timestamp` | +| top-level `filters` | `ai_search_options.retrieval.filters` | +| `eq` / `gte` / `and` / `or` | `$eq` / `$gte`, implicit AND, no compound keys | +| `ranking_options.score_threshold` | `retrieval.match_threshold` per request, `score_threshold` on the instance | +| `hybrid_search_enabled` | `index_method: { vector, keyword }` (the old flag is deprecated) | +| `env.AI.autorag("_").listInstances()` | `env.AI_SEARCH.list()` | +| `/autorag/rags/{name}/ai-search` | `/ai-search/instances/{id}/search` | +| `AutoRAGNotFoundError` etc. | `AiSearchNotFoundError` / `AiSearchInternalError` / `AiSearchError`, narrowed on `err.name` | + +## Common Errors + +Symptoms that arrive with no error code. If you have a code, look it up rather than inferring it from the name: [API error codes](https://developers.cloudflare.com/ai-search/troubleshooting/api-error-codes/) for request-time and public endpoint failures, [indexing error codes](https://developers.cloudflare.com/ai-search/troubleshooting/indexing-error-codes/) for anything asynchronous. + +### "The binding is undefined" + +`compatibility_date` is below `2026-03-27`. That is the cause far more often than the binding declaration. You also need a recent `wrangler` and `@cloudflare/workers-types`; if types do not resolve, check whether `AiSearchInstance` exists in your installed version before assuming the config is wrong. + +### "My folder filter returns nothing" + +`{ folder: "docs/" }` is an equality check. It matches files directly in `docs/`, not `docs/guides/intro.md`. -**Timestamp precision:** Use seconds (10-digit), not milliseconds. ```typescript -const nowInSeconds = Math.floor(Date.now() / 1000); // Correct +// ❌ direct children only +filters: { folder: "docs/" } + +// ✅ whole subtree ('0' sorts after '/' in ASCII) +filters: { folder: { $gte: "docs/", $lt: "docs0" } } ``` -**Folder prefix matching:** Use `gte` for "starts with" on paths. +### "There is no `and` / `or` operator" + +Multiple keys are ANDed implicitly. There are no compound operators. Use `$in` for an OR over one field. + ```typescript -filters: { column: "folder", operator: "gte", value: "docs/api/" } // Matches nested +// ❌ legacy +{ operator: "or", filters: [ ... ] } + +// ✅ current +filters: { folder: { $in: ["docs/guides/", "docs/tutorials/"] } } +filters: { folder: "docs/api/", timestamp: { $gte: 1735689600 } } // AND +``` + +Operators are `$`-prefixed: `$eq` `$ne` `$in` `$nin` `$lt` `$lte` `$gt` `$gte`. Reference: [Filtering](https://developers.cloudflare.com/ai-search/configuration/retrieval/filtering/). + +### "My timestamp filter matches nothing" + +Your filter value and `item.timestamp` must be in the same unit. Read the unit off a real response before filtering. + +```typescript +console.log(chunks[0]?.item.timestamp); // 10 digits = seconds, 13 = milliseconds ``` -## Filter Limitations +### "My custom metadata filter never matches" -| Limit | Value | -|-------|-------| -| Max nesting depth | 2 levels | -| Filters per compound | 10 | -| `or` operator | Same column, `eq` only | +Metadata keys not declared on the instance are not indexed and cannot be filtered on. This fails silently: the upload succeeds, the filter just never matches. -**OR restriction example:** ```typescript -// ✅ Valid: same column, eq only -{ operator: "or", filters: [ - { column: "folder", operator: "eq", value: "docs/" }, - { column: "folder", operator: "eq", value: "guides/" } -]} +await instance.update({ + custom_metadata: [{ field_name: "category", data_type: "text" }], +}); +// only now is this filterable +await instance.items.upload("guide.pdf", buf, { metadata: { category: "onboarding" } }); ``` -## Indexing Issues +Adding a field later does not backfill already-indexed items; reupload or resync them. + +### "`get()` succeeded but `search()` throws" + +`env.AI_SEARCH.get("typo-name")` is synchronous and makes no network call, so you get a handle for an instance that may not exist. The error surfaces on the first `search()` / `info()` / `items.*` call. **Do not treat a successful `get()` as existence.** + +### "REST or Wrangler returns a permission error" -| Problem | Cause | Solution | -|---------|-------|----------| -| File not indexed | Unsupported format or >4MB | Check format (.md/.txt/.html/.pdf/.doc/.csv/.json) | -| Index out of sync | 6-hour index cycle | Wait or use "Force Sync" (30s rate limit) | -| Empty results | Index incomplete | Check dashboard for indexing status | +The token needs **both** `AI Search:Edit` **and** `AI Search:Run`. Edit alone cannot query; Run alone cannot manage instances. The old "AI Search - Read" permission name is gone. Workers bindings authenticate themselves. -## Auth Errors +## Retrieval can only use an index you built -| Error | Cause | Fix | -|-------|-------|-----| -| `AutoRAGUnauthorizedError` | Invalid/missing token | Create Service API token with AI Search permissions | -| `AutoRAGNotFoundError` | Wrong instance name | Verify exact name from dashboard | +Two settings, at two different times: -## Performance +- **Index time**, `index_method` on the instance: which indexes get built. +- **Retrieval time**, `retrieval_type` on the request: which of them a query reads. + +**To query with `retrieval_type: "hybrid"`, enable both `vector` and `keyword` at index time.** Same for `keyword` on its own. A query cannot read an index that was never built, and new instances build vector only. -**Slow responses (>3s):** ```typescript -// Add score threshold + limit results -ranking_options: { score_threshold: 0.5 }, -max_num_results: 10 +await instance.update({ index_method: { vector: true, keyword: true } }); ``` -**Empty results debug:** -1. Remove filters, test basic query -2. Lower `score_threshold` to 0.1 -3. Check index is populated +Changing `index_method` triggers a full reindex, and neither `keyword` nor `hybrid` returns a single keyword match until it finishes. Confirm it is live by reading `scoring_details.keyword_score` off a real query. ## Limits -| Resource | Limit | -|----------|-------| -| Instances per account | 10 | -| Files per instance | 100,000 | -| Max file size | 4 MB | -| Index frequency | 6 hours | +**Do not pin numeric limits from memory or from these files.** Read the [platform limits](https://developers.cloudflare.com/ai-search/platform/limits-pricing/). The ones that change a design: + +| Limit | What it decides | +|-------|-----------------| +| Files per instance | One big instance or many small ones. **Lower with hybrid search**, see below | +| Instances per account | Whether instance-per-tenant is viable at your scale | +| Max file size | Which source files need splitting before upload | +| Instances per cross-instance search | How wide a federated query can go | +| Custom metadata field count | Small, and fields must be declared before upload | +| Queries per month on Free | The limit you hit first while prototyping | + +### Hybrid search lowers the file ceiling + +**An instance holds fewer files with hybrid search than with vector alone** `update()` accepts the change without warning, and the shortfall appears as documents that stop getting indexed once the reindex passes the lower ceiling. Hybrid is a capacity decision, not only a relevance one. + +- A corpus sized against the vector-only ceiling may not fit once hybrid is on. Check `stats()` against the current limits first. +- If you need both hybrid and the full ceiling, split the corpus across instances and use [cross-instance search](api.md#cross-instance-search). + +## Silent failures -## Anti-Patterns +**Response caching is on by default**, matching on similarity rather than exact text. Reindexing does not immediately change answers, and retrieval benchmarks measure the cache, so tuning appears to do nothing. Turn it off while tuning, and check whether similarity hits can cross tenants before relying on it for per-tenant content. -**Use env vars for instance names:** ```typescript -const answer = await env.AI.autorag(env.AI_SEARCH_INSTANCE).aiSearch({...}); +ai_search_options: { cache: { enabled: false } } ``` -**Handle specific error types:** +**Indexing problems are invisible from a query.** A skipped or errored document is absent from results, and an oversized or unsupported file is skipped without an error. + ```typescript -if (error instanceof AutoRAGNotFoundError) { /* 404 */ } -if (error instanceof AutoRAGUnauthorizedError) { /* 401 */ } +await instance.stats(); // counts by state, embedding errors +await instance.items.list({ status: "error" }); // each carries an error code +await instance.items.list({ status: "skipped" }); // filtered out or unsupported ``` + +**Check whether the code is item-level or instance-level before debugging the document.** Item-level codes (`over_size`, `unsupported_type`, `blocked_by_robots_txt`) fail one item. Instance-level codes (`bucket_unauthorized`, `external_source_missing_api_token`, `hybrid_search_is_full`) **pause indexing for the whole instance**. Both tiers are listed in [indexing error codes](https://developers.cloudflare.com/ai-search/troubleshooting/indexing-error-codes/). + +**`retrieval.return_on_failure` defaults to true**, so a retrieval failure returns empty results instead of throwing. Zero chunks can mean the query failed, not that nothing matched. + +A crawl that indexes nothing is usually bot protection rather than your path patterns: allow the `Cloudflare-AI-Search` user agent, renamed from `Cloudflare-AutoRAG`. + +## Debugging empty results + +In order, each step ruling out a class of cause. + +1. `stats()`: is anything `completed`? If everything is `queued`, you are just early. +2. Drop `filters` entirely and rerun. If results appear, it is the filter; check the folder prefix trick above. +3. Lower the match threshold substantially. If results appear, it is tuning, not indexing. +4. Disable cache in case you are reading a stale empty result. +5. Inspect `scoring_details` on whatever does come back to see which retrieval path fired. +6. For cross-instance search, check `errors[]`: a failed instance returns partial results, not an exception. + +## Anti-patterns + +**Do not build agents on `chatCompletions()`.** No memory, no tools, no loop. Use the [Agents SDK](../agents-sdk/) with `search()` as a tool. See [patterns.md](patterns.md#search-as-an-agent-tool). + +**Do not skip the zero-chunk check.** If `chunks.length === 0`, return "I don't know" rather than prompting a model with empty context. + +**Do not apply the tenant filter at call sites.** Wrap it in one helper that requires `tenantId`, or use one instance per tenant. A single unfiltered `search()` is a cross-tenant leak. + +**Do not `uploadAndPoll()` in a bulk loop.** It serializes the whole ingest. Use `upload()` and poll `stats()` once. + +**Do not enable reranking and query rewriting by reflex.** Each is a model call on every query. Turn them on because `scoring_details` showed you a problem they solve. + +## See Also + +- [api.md](api.md) - filters, boosting, and failure surfaces in full +- [configuration.md](configuration.md) - the settings behind these traps +- [API error codes](https://developers.cloudflare.com/ai-search/troubleshooting/api-error-codes/) - request-time and public endpoint failures +- [Indexing error codes](https://developers.cloudflare.com/ai-search/troubleshooting/indexing-error-codes/) - asynchronous sync and crawl failures, item and instance level +- [AI Search docs](https://developers.cloudflare.com/ai-search/) - authoritative over this file diff --git a/skills/cloudflare/references/ai-search/patterns.md b/skills/cloudflare/references/ai-search/patterns.md index 4a70f08..926c3ff 100644 --- a/skills/cloudflare/references/ai-search/patterns.md +++ b/skills/cloudflare/references/ai-search/patterns.md @@ -1,85 +1,173 @@ # AI Search Patterns -## search() vs aiSearch() +## Search as an agent tool -| Use | Method | Returns | -|-----|--------|---------| -| Custom UI, analytics | `search()` | Raw chunks only (~100-300ms) | -| Chatbots, Q&A | `aiSearch()` | AI response + chunks (~500-2000ms) | +The [Agents SDK](../agents-sdk/) owns the loop, memory, and tool selection; AI Search is one tool it calls. -## rewrite_query +```typescript +import { Agent } from "agents"; +import { tool } from "ai"; +import { z } from "zod"; + +interface Env { AI_SEARCH: AiSearchNamespace } + +export class SupportAgent extends Agent { + searchDocs = tool({ + description: + "Search product documentation. Use for questions about features, " + + "configuration, limits, or error messages.", + parameters: z.object({ + query: z.string().describe("A specific, self-contained question"), + }), + execute: async ({ query }) => { + const { chunks } = await this.env.AI_SEARCH.get("product-docs").search({ + messages: [{ role: "user", content: query }], + ai_search_options: { retrieval: { max_num_results: 8 } }, + }); + return chunks.map((c) => ({ source: c.item.key, text: c.text })); + }, + }); +} +``` + +**The model picks tools from the description alone.** Say what is in the index and when to reach for it. + +**Return trimmed objects, not the raw response.** Every field becomes context the agent carries on every later turn. `key` and `text` are usually enough. + +**Turn off query rewriting for agent-issued queries.** The agent already wrote a clean, self-contained query; rewriting adds a model call and can drift from intent. Turn it on for raw human input. -| Setting | Use When | -|---------|----------| -| `true` | User input (typos, vague queries) | -| `false` | LLM-generated queries (already optimized) | +### Multiple indexes, one agent -## Multitenancy (Folder-Based) +Give the agent one tool per corpus rather than one tool with a corpus argument. The descriptions do the routing. ```typescript -const answer = await env.AI.autorag("saas-docs").aiSearch({ - query: "refund policy", - model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast", - filters: { - column: "folder", - operator: "gte", // "starts with" pattern - value: `tenants/${tenantId}/` - } -}); +searchDocs = tool({ description: "Public product documentation.", /* → get("docs") */ }); +searchTickets = tool({ description: "Past support tickets and resolutions.", /* → get("tickets") */ }); ``` -## Streaming +If corpora overlap, use [cross-instance search](#cross-instance-search) instead. + +## Multitenancy + +Prefer instance-per-tenant unless tenant count makes it impractical. + +### Instance per tenant (strong isolation) + +Isolation comes from the instance boundary, not a filter you have to get right at every call site. ```typescript -const stream = await env.AI.autorag("docs").aiSearch({ - query, model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast", stream: true -}); -return new Response(stream, { headers: { "Content-Type": "text/event-stream" } }); +async function searchForTenant(env: Env, tenantId: string, query: string) { + return env.TENANTS.get(`tenant-${tenantId}`) + .search({ messages: [{ role: "user", content: query }] }); +} + +// Provision on signup +await env.TENANTS.create({ id: `tenant-${tenantId}` }); +await env.TENANTS.get(`tenant-${tenantId}`).items.upload("welcome.md", welcomeDoc); ``` -## Score Threshold +Instances per account is a plan limit and the Paid ceiling is high; check the [platform limits](https://developers.cloudflare.com/ai-search/platform/limits-pricing/) before ruling this out on tenant count. -| Threshold | Use | -|-----------|-----| -| 0.3 (default) | Broad recall, exploratory | -| 0.5 | Balanced, production default | -| 0.7 | High precision, critical accuracy | +## Per-agent knowledge isolation + +Each agent gets its own instance, created at runtime. Built-in storage indexes immediately, so this is write-then-read memory. It replaces Durable Object storage plus your own embedding calls for agent recall. + +```typescript +export class ResearchAgent extends Agent { + private get memory() { + return this.env.AGENTS.get(`agent-${this.name}`); + } + + async onStart() { + await this.env.AGENTS.create({ id: `agent-${this.name}` }).catch(() => {}); // idempotent + } + + async remember(key: string, content: string) { + const item = await this.memory.items.uploadAndPoll(key, content); + return item.status; // the wait is bounded, not guaranteed + } + + async recall(query: string) { + const { chunks } = await this.memory.search({ + messages: [{ role: "user", content: query }], + }); + return chunks; + } +} +``` + +**Check the `status` `uploadAndPoll()` returns** rather than assuming `completed`, and degrade gracefully if it is still `running`. Use plain `upload()` when the next read does not depend on this write, and always for bulk. -## System Prompt Template +## Citations + +**Dedupe by `item.key`:** several chunks often come from one document. ```typescript -const systemPrompt = `You are a documentation assistant. -- Answer ONLY based on provided context -- If context doesn't contain answer, say "I don't have information" -- Include code examples from context`; +const sources = [...new Map( + chunks.map((c) => [c.item.key, { key: c.item.key, score: c.score }]) +).values()]; ``` -## Compound Filters +If you do stream `chatCompletions()`, the `event: chunks` frame arrives **before** the token deltas, so render the source list first. + +## Cross-instance search + +One query across several instances, merged and re-ranked. ```typescript -// OR: Multiple folders -filters: { - operator: "or", - filters: [ - { column: "folder", operator: "gte", value: "docs/api/" }, - { column: "folder", operator: "gte", value: "docs/auth/" } - ] -} +const { chunks, errors } = await env.AI_SEARCH.search({ + messages: [{ role: "user", content: query }], + ai_search_options: { + instance_ids: ["product-docs", "api-reference", "changelog"], + retrieval: { max_num_results: 15 }, + }, +}); -// AND: Folder + date -filters: { - operator: "and", - filters: [ - { column: "folder", operator: "gte", value: "docs/" }, - { column: "timestamp", operator: "gte", value: oneWeekAgoSeconds } - ] -} +if (errors?.length) console.warn("partial results", errors); ``` -## Reranking +**Always check `errors`.** Failures are returned rather than thrown, so a partial result set looks identical to a complete one. Instances per query is capped; see the [limits](https://developers.cloudflare.com/ai-search/platform/limits-pricing/). -Enable for high-stakes use cases (adds ~300ms latency): +## Tuning retrieval + +Reach for these in order, changing one at a time. Exhaust the levers that add nothing before adding a model call to every query. + +| Symptom | Lever | Adds | +|---------|-------|------| +| Misses exact strings (error codes, IDs, flags) | Enable keyword indexing, then `retrieval_type: "hybrid"` | A reindex; a lower file ceiling | +| Misses paraphrases | Lower the match threshold | Nothing | +| Chunks relevant but truncated mid-thought | `context_expansion: 1` to `3` | Tokens downstream | +| Right documents, wrong order | `reranking: { enabled: true }` | A model call per query | +| Stale content outranks fresh | [`boost_by`](api.md#boosting) with `direction: "desc"` | Nothing | +| Too much noise | Raise the threshold, lower `max_num_results` | Nothing | +| Vague human queries | Query rewriting | A model call per query | + +### Read `scoring_details` instead of guessing ```typescript -reranking: { enabled: true, model: "@cf/baai/bge-reranker-base" } +chunks.forEach((c) => + console.log(c.item.key, c.score, c.scoring_details.vector_score, + c.scoring_details.keyword_score, c.scoring_details.fusion_method)); ``` + +This tells you whether hybrid is active (`keyword_score` present and non-zero), whether the threshold is cutting good results, and whether reranking changed anything. **Disable caching while doing this** or you are reading a previous response. + +### Choosing a match threshold + +Read the current default off `info()` rather than assuming a number. + +| Direction | Use | +|-----------|-----| +| Below default | Broad recall; agent tools, where the model filters for you | +| Around default | General-purpose starting point | +| Above default | High precision; you would rather return nothing than something wrong | + +Agent tools want a lower threshold than direct-to-user results: an extra irrelevant chunk costs some context, a missing chunk costs the answer. + +## See Also + +- [api.md](api.md) - method signatures, filters, boosting, failure surfaces +- [configuration.md](configuration.md) - instance setup, data sources, keeping a source fresh +- [gotchas.md](gotchas.md) - what to check when results are empty +- [agents-sdk](../agents-sdk/) - the agent loop these patterns plug into +- [workers-ai](../workers-ai/) - models for the generation step you own