diff --git a/agent-source-of-truth/curl.mdx b/agent-source-of-truth/curl.mdx
index 6fded6451..85729e149 100644
--- a/agent-source-of-truth/curl.mdx
+++ b/agent-source-of-truth/curl.mdx
@@ -75,7 +75,7 @@ curl -X POST "https://api.firecrawl.dev/v2/search" \
Successful responses include `success`, `data`, optional `warning`, `id`, and `creditsUsed`.
-- `data.web`, `data.images`, `data.news`: result arrays; which keys appear depends on `sources` (by default only `data.web` is populated).
+- `data.web`, `data.news`, `data.images`, `data.developer`: result arrays; which keys appear depends on `sources` and `categories` (by default only `data.web` is populated).
- Web and news items include fields such as `title`, `url`, and (when `scrapeOptions` / formats request it) `markdown`, `html`, `rawHtml`, `links`, `screenshot`, `audio`, `video`, and `metadata`.
- Image items include fields such as `imageUrl`, `url`, and dimensions when available.
- `warning`: optional human-readable notice.
@@ -104,6 +104,17 @@ Successful responses include `success`, `data`, optional `warning`, `id`, and `c
- `{ "type": "github" }`
- `{ "type": "research" }`
- `{ "type": "pdf" }`
+ - `{ "type": "developer" }`
+
+- `includeDomains`
+ - Type: array of strings
+ - Use when: you want to restrict results to specific domains.
+ - Notes: mutually exclusive with `excludeDomains`.
+
+- `excludeDomains`
+ - Type: array of strings
+ - Use when: you want to exclude specific domains from results.
+ - Notes: mutually exclusive with `includeDomains`.
- `limit`
- Type: integer (minimum 1, maximum 100, default 5)
@@ -129,6 +140,10 @@ Successful responses include `success`, `data`, optional `warning`, `id`, and `c
- Type: integer (milliseconds, default 60000)
- Use when: you need a request timeout in milliseconds.
+- `highlights`
+ - Type: boolean (default true)
+ - Use when: you want query-relevant highlights in results.
+
- `enterprise`
- Type: array of strings (`"anon"` or `"zdr"` per item)
- Use when: you need enterprise search controls.
@@ -227,14 +242,19 @@ Successful responses include `success` and `data`. Common `data` fields (dependi
- `"summary"`: summary output
- `"changeTracking"`: change tracking output
- `"json"`: JSON extraction
+ - `"attributes"`: attribute extraction
- `"branding"`: branding profile output
+ - `"product"`: product data extraction
+ - `"menu"`: menu/navigation extraction
- `"audio"`: audio extraction
- `"video"`: video extraction
- - Format object fields:
- - `type`: one of the format strings above
- - `prompt`, `schema`: JSON extraction options for `type: "json"`
- - `modes`, `schema`, `prompt`, `tag`: change tracking options for `type: "changeTracking"`
- - `fullPage`, `quality`, `viewport`: screenshot options for `type: "screenshot"`
+ - Object-only format types:
+ - `{ "type": "json", "prompt": "...", "schema": {...} }`: at least one of `prompt` or `schema` required
+ - `{ "type": "question", "question": "..." }`: question-answer extraction
+ - `{ "type": "highlights", "query": "..." }`: relevant source-text extraction
+ - `{ "type": "screenshot", "fullPage": true, "quality": 80, "viewport": {...} }`: screenshot with options
+ - `{ "type": "changeTracking", "modes": ["git-diff"], "schema": {...}, "prompt": "...", "tag": "..." }`: `modes` required
+ - `{ "type": "attributes", "selectors": [{"selector": "a", "attribute": "href"}] }`: attribute extraction
- `headers`
- Type: object
@@ -316,6 +336,19 @@ Successful responses include `success` and `data`. Common `data` fields (dependi
- Type: boolean
- Use when: you want Firecrawl to cache the result.
+- `lockdown`
+ - Type: boolean
+ - Use when: you want to serve only previously cached results. Returns 404 if nothing is cached.
+
+- `redactPII`
+ - Type: boolean or object
+ - Use when: you want to redact personally identifiable information from output.
+ - Object fields: `mode` (`"accurate"`, `"aggressive"`, `"fast"`), `entities` (array of `"PERSON"`, `"EMAIL"`, `"PHONE"`, `"LOCATION"`, `"FINANCIAL"`, `"SECRET"`), `replaceStyle` (`"tag"`, `"mask"`, `"remove"`)
+
+- `auditMetadata`
+ - Type: object with `username` (string, required)
+ - Use when: you need user attribution for SIEM logging.
+
- `profile`
- Type: object with `name` and optional `saveChanges`
- Use when: you want a persistent browser profile shared across scrapes and interactions.
@@ -489,4 +522,6 @@ curl -X POST "https://api.firecrawl.dev/v2/support/docs-search" \
- `firecrawl-docs/api-reference/v2-openapi.json`
- `firecrawl/apps/js-sdk/firecrawl/src/v2/types.ts`
- `firecrawl/apps/python-sdk/firecrawl/v2/types.py`
-- `firecrawl/apps/rust-sdk/src/v2/scrape.rs`
+- `firecrawl/apps/rust-sdk/src/scrape.rs`
+- `firecrawl/apps/rust-sdk/src/search.rs`
+- `firecrawl/apps/rust-sdk/src/types.rs`
diff --git a/agent-source-of-truth/elixir.mdx b/agent-source-of-truth/elixir.mdx
index 703f6d6b2..b5f770257 100644
--- a/agent-source-of-truth/elixir.mdx
+++ b/agent-source-of-truth/elixir.mdx
@@ -10,7 +10,7 @@ Canonical Firecrawl Elixir source of truth for agents. Generated from SDK source
Add to `mix.exs`:
```elixir
-{:firecrawl, "~> 1.0.0"}
+{:firecrawl, "~> 1.9.1"}
```
## Authenticate
@@ -58,6 +58,10 @@ Use search to discover relevant pages from a query, then pick URLs to scrape or
country: "US",
ignore_invalid_urls: true,
timeout: 60000,
+ highlights: true,
+ include_domains: ["docs.firecrawl.dev"],
+ exclude_domains: ["old.firecrawl.dev"],
+ enterprise: ["zdr"],
scrape_options: [
formats: [
"markdown",
@@ -97,6 +101,14 @@ Use search to discover relevant pages from a query, then pick URLs to scrape or
- `"pdf"` or `:pdf`
- `%{type: "github" | "research" | "pdf"}`
+- `include_domains`
+ - Type: list of strings
+ - Use when: you want to restrict results to specific domains.
+
+- `exclude_domains`
+ - Type: list of strings
+ - Use when: you want to exclude results from specific domains.
+
- `limit`
- Type: integer
- Use when: you want to cap results.
@@ -121,6 +133,10 @@ Use search to discover relevant pages from a query, then pick URLs to scrape or
- Type: integer
- Use when: you need a request timeout in milliseconds.
+- `highlights`
+ - Type: boolean (default true)
+ - Use when: you want highlighted snippets in search results.
+
- `enterprise`
- Type: list of strings
- Use when: you need enterprise search controls.
@@ -160,8 +176,11 @@ Use scrape when you already have a URL and want structured content in one or mor
"markdown",
"links",
%{type: "json", prompt: "Extract plan names and prices."},
+ %{type: "question", question: "What is the enterprise plan price?"},
+ %{type: "highlights", query: "pricing tiers"},
%{type: "screenshot", fullPage: true, quality: 80, viewport: %{width: 1280, height: 720}},
- %{type: "changeTracking", modes: ["git-diff"], tag: "pricing"}
+ %{type: "changeTracking", modes: ["git-diff"], tag: "pricing"},
+ %{type: "attributes", selectors: [%{selector: "h1", attribute: "textContent"}]}
],
headers: %{"User-Agent" => "FirecrawlDocsBot/1.0"},
only_main_content: true,
@@ -179,6 +198,9 @@ Use scrape when you already have a URL and want structured content in one or mor
max_age: 86400000,
min_age: 1,
store_in_cache: true,
+ lockdown: false,
+ redact_pii: false,
+ audit_metadata: [username: "docs-bot"],
profile: [name: "docs-session", save_changes: true],
zero_data_retention: false
)
@@ -209,6 +231,9 @@ Use scrape when you already have a URL and want structured content in one or mor
- Format map fields:
- `type`: one of the format strings above
- `prompt`, `schema`: JSON extraction options for `type: "json"`
+ - `question`: question to answer for `type: "question"`
+ - `query`: search query for `type: "highlights"`
+ - `selectors`: list of selector maps for `type: "attributes"`
- `modes`, `schema`, `prompt`, `tag`: change tracking options for `type: "changeTracking"`
- `fullPage`, `quality`, `viewport`: screenshot options for `type: "screenshot"`
@@ -294,6 +319,18 @@ Use scrape when you already have a URL and want structured content in one or mor
- Type: boolean
- Use when: you want Firecrawl to cache the result.
+- `lockdown`
+ - Type: boolean
+ - Use when: you want to enable lockdown mode for the scrape.
+
+- `redact_pii`
+ - Type: boolean
+ - Use when: you want to redact personally identifiable information from the output.
+
+- `audit_metadata`
+ - Type: keyword list with `username:` (string, required)
+ - Use when: you need to attach audit metadata to the scrape request.
+
- `profile`
- Type: keyword list with `name:` and optional `save_changes:` (or `saveChanges:`)
- Use when: you want a persistent browser profile shared across scrapes and interactions.
@@ -373,6 +410,6 @@ Use interact when a page requires browser actions or code execution after a scra
## Source Of Truth
-- `firecrawl/apps/elixir-sdk/mix.exs`
+- `firecrawl/apps/elixir-sdk/mix.exs` (version 1.9.1)
- `firecrawl/apps/elixir-sdk/lib/firecrawl.ex`
- `firecrawl-docs/api-reference/v2-openapi.json`
diff --git a/agent-source-of-truth/java.mdx b/agent-source-of-truth/java.mdx
index 93da68332..a00f3ee1f 100644
--- a/agent-source-of-truth/java.mdx
+++ b/agent-source-of-truth/java.mdx
@@ -13,14 +13,14 @@ Maven:
com.firecrawl
firecrawl-java
- 1.2.0
+ 1.12.1
```
Gradle:
```gradle
-implementation("com.firecrawl:firecrawl-java:1.2.0")
+implementation("com.firecrawl:firecrawl-java:1.12.1")
```
## Authenticate
@@ -83,6 +83,8 @@ SearchOptions options = SearchOptions.builder()
.location("San Francisco,California,United States")
.ignoreInvalidURLs(true)
.timeout(60000)
+ .highlights(true)
+ .includeDomains(List.of("docs.firecrawl.dev"))
.scrapeOptions(
ScrapeOptions.builder()
.formats(List.of(
@@ -124,6 +126,14 @@ SearchData results = client.search("site:docs.firecrawl.dev crawl webhooks", opt
- `"pdf"`: PDF-focused results
- `{type: "github" | "research" | "pdf"}`: typed category map form
+- `options.includeDomains`
+ - Type: `List`
+ - Use when: you want to restrict results to specific domains.
+
+- `options.excludeDomains`
+ - Type: `List`
+ - Use when: you want to exclude specific domains from results.
+
- `options.limit`
- Type: Integer
- Use when: you want to cap results.
@@ -144,6 +154,10 @@ SearchData results = client.search("site:docs.firecrawl.dev crawl webhooks", opt
- Type: Integer
- Use when: you need a request timeout in milliseconds.
+- `options.highlights`
+ - Type: Boolean
+ - Use when: you want highlighted snippets in results. Defaults to true.
+
- `options.scrapeOptions`
- Type: `ScrapeOptions`
- Use when: you want to scrape each search result (see Scrape parameters for fields).
@@ -181,6 +195,7 @@ Document doc = client.scrape(
```java
import com.firecrawl.models.ScrapeOptions;
import com.firecrawl.models.JsonFormat;
+import com.firecrawl.models.AuditMetadata;
List