Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
186 changes: 186 additions & 0 deletions agent-quickstart/elixir.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
---
title: "Elixir Agent Quickstart"
description: "Canonical Firecrawl Elixir quickstart for external agents using search, scrape, and interact."
---

# Firecrawl Elixir Agent Quickstart

Canonical quickstart for agents integrating Firecrawl via the Elixir SDK. Generated from SDK source (`firecrawl` hex package) and the v2 OpenAPI spec. Function names and parameters match the auto-generated OpenAPI client.

## Install

Add to `mix.exs`:

```elixir
{:firecrawl, "~> 1.9"}
```

## Authenticate

```elixir
# config/runtime.exs or config.exs
config :firecrawl, api_key: System.get_env("FIRECRAWL_API_KEY")

# or pass api_key per call
{:ok, res} = Firecrawl.search_and_scrape(
[query: "site:docs.firecrawl.dev webhooks"],
api_key: "fc-your-api-key"
)
```

## When To Use What

- **`search`**: use when you start with a query and need discovery. Returns categorized results from web, news, and image sources.
- **`scrape`**: use when you already have a URL and want page content in one or more formats (markdown, HTML, JSON extraction, screenshots, etc.).
- **`interact`**: use when the page needs clicks, forms, or post-scrape browser actions. Operates on a browser session created by a prior scrape.

## Search

### Why use it

Discover relevant pages from a query, then pick URLs to scrape or interact with. Constrain results to a site with `site:` in the query (e.g. `site:docs.firecrawl.dev webhooks`).

### Preferred SDK method

`Firecrawl.search_and_scrape(params \\ [], opts \\ [])` → `{:ok, %Req.Response{}}` or `{:error, exception}`

Bang variant: `Firecrawl.search_and_scrape!(params, opts)` raises on error.

### Example

```elixir
{:ok, res} = Firecrawl.search_and_scrape(
query: "site:docs.firecrawl.dev webhook retries",
sources: [:web],
limit: 5,
scrape_options: [
formats: ["markdown"],
only_main_content: true
]
)
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `query` | string (required) | Search query. Use `site:example.com` to scope to a domain. |
| `sources` | list of atoms, strings, or maps | Which source types: `:web`, `:news`, `:images`, or `%{type: "web"}`. |
| `categories` | list of atoms, strings, or maps | Category filters: `:github`, `:research`, `:pdf`, or `%{type: "github"}`. |
| `include_domains` | list of strings | Restrict results to these domains. |
| `exclude_domains` | list of strings | Exclude results from these domains. |
| `limit` | integer | Max number of results. |
| `tbs` | string | Time-based filter (e.g. `qdr:d`, `qdr:w`). |
| `location` | string | Location string for geo-targeted results. |
| `country` | string | ISO 3166-1 alpha-2 country code (e.g. `"US"`). |
| `ignore_invalid_urls` | boolean | Drop URLs that cannot be scraped. |
| `timeout` | integer | Request timeout in milliseconds. |
| `highlights` | boolean | Generate query-relevant highlights. Defaults to `true`. |
| `enterprise` | list of strings | Enterprise options: `"zdr"`, `"anon"`. |
| `scrape_options` | keyword list | Scrape each search result (see Scrape parameters). |

## Scrape

### Why use it

Get structured content from a URL in one or more formats: markdown, HTML, JSON extraction, screenshots, and more.

### Preferred SDK method

`Firecrawl.scrape_and_extract_from_url(params \\ [], opts \\ [])` → `{:ok, %Req.Response{}}` or `{:error, exception}`

Bang variant: `Firecrawl.scrape_and_extract_from_url!(params, opts)` raises on error.

### Example

```elixir
{:ok, res} = Firecrawl.scrape_and_extract_from_url(
url: "https://example.com/pricing",
formats: [
"markdown",
%{type: "json", prompt: "Extract plan names and prices."}
],
only_main_content: true
)
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `url` | string (required) | URL to scrape. |
| `formats` | list of strings or maps | Output formats. Strings: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"changeTracking"`, `"json"`, `"branding"`, `"audio"`, `"video"`. Maps: `%{type: "json", prompt: ...}`, `%{type: "screenshot", fullPage: true}`, `%{type: "changeTracking", modes: ["git-diff"]}`. |
| `headers` | map | Custom HTTP headers. |
| `include_tags` | list of strings | Only include these HTML tags. |
| `exclude_tags` | list of strings | Exclude these HTML tags. |
| `only_main_content` | boolean | Strip nav, footer, and boilerplate. |
| `timeout` | integer | Timeout in milliseconds. Default 60000, range 1000–300000. |
| `wait_for` | integer | Wait for the page to render (milliseconds). |
| `mobile` | boolean | Use mobile viewport. |
| `parsers` | list of strings or maps | File parsing. E.g. `"pdf"` or `%{type: "pdf", mode: "auto", maxPages: 5}`. |
| `actions` | list of maps | Pre-scrape browser actions. Types: `wait`, `click`, `write`, `press`, `scroll`, `screenshot`, `scrape`, `executeJavascript`, `pdf`. |
| `location` | keyword list | Geo/language-aware scraping. Keys: `country:`, `languages:`. |
| `skip_tls_verification` | boolean | Skip TLS verification. |
| `remove_base64_images` | boolean | Drop base64 images from markdown output. |
| `block_ads` | boolean | Block ads and cookie popups. |
| `proxy` | atom | Proxy mode: `:basic`, `:enhanced`, `:auto`. |
| `max_age` | integer | Max age (ms) of cached content to reuse. |
| `min_age` | integer | Min age (ms) of cached content. |
| `store_in_cache` | boolean | Store result in Firecrawl cache. |
| `profile` | keyword list | Persistent browser profile. Keys: `name:`, `save_changes:`. |
| `zero_data_retention` | boolean | Enable zero data retention. |
| `lockdown` | boolean | Only serve cached results, no outbound request. |
| `redact_pii` | boolean | Redact personally identifiable information. |

## Interact

### Why use it

Control the browser session tied to a prior scrape job. Use for code execution in the browser runtime.

### Preferred SDK method

`Firecrawl.interact_with_scrape_browser_session(job_id, params \\ [], opts \\ [])` → `{:ok, %Req.Response{}}` or `{:error, exception}`

Bang variant: `Firecrawl.interact_with_scrape_browser_session!(job_id, params, opts)` raises on error.

### Example

```elixir
{:ok, res} = Firecrawl.interact_with_scrape_browser_session(
"<scrapeJobId>",
code: "console.log(await page.title());",
language: :node,
timeout: 60
)

# When done, stop the session:
{:ok, _} = Firecrawl.stop_interactive_scrape_browser_session("<scrapeJobId>")
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `job_id` | string (required, 1st arg) | Scrape job ID from the scrape response metadata. |
| `code` | string (required) | Code to execute in the browser session. |
| `language` | atom or string | Runtime: `:python`, `:node`, `:bash`. |
| `timeout` | integer | Execution timeout in seconds. |

### Stop session

`Firecrawl.stop_interactive_scrape_browser_session(job_id)` ends the browser session.

## Notes

- The Elixir client is **auto-generated from the OpenAPI spec**; function names are derived from operation IDs, not hand-written aliases.
- This SDK exposes **code-based interactions only**: there is no `prompt` parameter on `interact_with_scrape_browser_session`.
- Each function has a bang (`!`) variant that raises on error instead of returning `{:error, _}`.
- The SDK appends `"origin": "elixir-sdk@<version>"` to every request body for telemetry.
- Uses `snake_case` parameter keys in Elixir, converted to `camelCase` JSON keys on the wire.

## Source Of Truth

- `firecrawl/apps/elixir-sdk/mix.exs`
- `firecrawl/apps/elixir-sdk/lib/firecrawl.ex`
- `firecrawl-docs/api-reference/v2-openapi.json`
204 changes: 204 additions & 0 deletions agent-quickstart/java.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
---
title: "Java Agent Quickstart"
description: "Canonical Firecrawl Java quickstart for external agents using search, scrape, and interact."
---

# Firecrawl Java Agent Quickstart

Canonical quickstart for agents integrating Firecrawl via the Java SDK. Generated from SDK source (`firecrawl-java`) and the v2 OpenAPI spec. Method names and parameters match the SDK public API.

## Install

Maven:

```xml
<dependency>
<groupId>com.firecrawl</groupId>
<artifactId>firecrawl-java</artifactId>
<version>1.12.1</version>
</dependency>
```

Gradle:

```gradle
implementation("com.firecrawl:firecrawl-java:1.12.1")
```

## Authenticate

```java
import com.firecrawl.client.FirecrawlClient;

FirecrawlClient client = FirecrawlClient.builder()
.apiKey(System.getenv("FIRECRAWL_API_KEY"))
.build();
```

## When To Use What

- **`search`**: use when you start with a query and need discovery. Returns categorized results from web, news, and image sources.
- **`scrape`**: use when you already have a URL and want page content in one or more formats (markdown, HTML, JSON extraction, screenshots, etc.).
- **`interact`**: use when the page needs clicks, forms, or post-scrape browser actions. Operates on a browser session created by a prior scrape.

## Search

### Why use it

Discover relevant pages from a query, then pick URLs to scrape or interact with. Constrain results to a site with `site:` in the query (e.g. `site:docs.firecrawl.dev webhooks`).

### Preferred SDK method

- `client.search(query)` &rarr; `SearchData`
- `client.search(query, options)` &rarr; `SearchData`

### Example

```java
import com.firecrawl.models.SearchData;
import com.firecrawl.models.SearchOptions;
import com.firecrawl.models.ScrapeOptions;

SearchOptions options = SearchOptions.builder()
.sources(List.of("web"))
.limit(5)
.scrapeOptions(
ScrapeOptions.builder()
.formats(List.of("markdown"))
.onlyMainContent(true)
.build()
)
.build();

SearchData results = client.search("site:docs.firecrawl.dev webhook retries", options);
List<Map<String, Object>> web = results.getWeb();
```

Read result buckets with `getWeb()`, `getNews()`, and `getImages()`. Do not treat `SearchData` as a directly iterable list.

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `query` | String (required, 1st arg) | Search query. Use `site:example.com` to scope to a domain. |
| `options.sources` | `List<Object>` | Which source types: `"web"`, `"news"`, `"images"`, or `{type: ...}` maps. |
| `options.categories` | `List<Object>` | Category filters: `"github"`, `"research"`, `"pdf"`, or `{type: ...}` maps. |
| `options.includeDomains` | `List<String>` | Restrict results to these domains. Cannot combine with `excludeDomains`. |
| `options.excludeDomains` | `List<String>` | Exclude results from these domains. Cannot combine with `includeDomains`. |
| `options.limit` | Integer | Max number of results. |
| `options.tbs` | String | Time-based filter (e.g. `qdr:d`, `qdr:w`). |
| `options.location` | String | Location string for geo-targeted results. |
| `options.ignoreInvalidURLs` | Boolean | Drop URLs that cannot be scraped. |
| `options.timeout` | Integer | Request timeout in milliseconds. |
| `options.highlights` | Boolean | Generate query-relevant highlights for results. Defaults to `true`. |
| `options.scrapeOptions` | `ScrapeOptions` | Scrape each search result (see Scrape parameters). |

## Scrape

### Why use it

Get structured content from a URL in one or more formats: markdown, HTML, JSON extraction, screenshots, and more.

### Preferred SDK method

- `client.scrape(url)` &rarr; `Document`
- `client.scrape(url, options)` &rarr; `Document`

### Example

```java
import com.firecrawl.models.ScrapeOptions;
import com.firecrawl.models.JsonFormat;

ScrapeOptions options = ScrapeOptions.builder()
.formats(List.of(
"markdown",
JsonFormat.builder().prompt("Extract plan names and prices.").build()
))
.onlyMainContent(true)
.build();

Document doc = client.scrape("https://example.com/pricing", options);
System.out.println(doc.getMarkdown());
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `url` | String (required, 1st arg) | URL to scrape. |
| `options.formats` | `List<Object>` | Output formats. Strings: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"changeTracking"`, `"json"`, `"attributes"`, `"branding"`, `"audio"`, `"video"`. Objects: `JsonFormat.builder().prompt(...).schema(...).build()`, or maps for screenshot/changeTracking/attributes. |
| `options.headers` | `Map<String, String>` | Custom HTTP headers. |
| `options.includeTags` | `List<String>` | Only include these HTML tags. |
| `options.excludeTags` | `List<String>` | Exclude these HTML tags. |
| `options.onlyMainContent` | Boolean | Strip nav, footer, and boilerplate. |
| `options.timeout` | Integer | Timeout in milliseconds. |
| `options.waitFor` | Integer | Wait for the page to render (milliseconds). |
| `options.mobile` | Boolean | Use mobile viewport. |
| `options.parsers` | `List<Object>` | File parsing. E.g. `"pdf"` or `{type: "pdf", maxPages: 5}`. |
| `options.actions` | `List<Map<String, Object>>` | Pre-scrape browser actions. Types: `wait`, `click`, `write`, `press`, `scroll`, `screenshot`, `scrape`, `executeJavascript`, `pdf`. |
| `options.location` | `LocationConfig` | Geo/language-aware scraping. |
| `options.skipTlsVerification` | Boolean | Skip TLS verification. |
| `options.removeBase64Images` | Boolean | Drop base64 images from markdown. |
| `options.blockAds` | Boolean | Block ads and cookie popups. |
| `options.proxy` | String | Proxy mode: `"basic"`, `"stealth"`, `"enhanced"`, `"auto"`, or custom URL. |
| `options.maxAge` | Long | Max age (ms) of cached content to reuse. |
| `options.storeInCache` | Boolean | Store result in Firecrawl cache. |
| `options.lockdown` | Boolean | Only serve cached results, no outbound request. |
| `options.redactPII` | Boolean | Redact personally identifiable information. |
| `options.auditMetadata` | `AuditMetadata` | User attribution for SIEM logging. Has `username` field. |

## Interact

### Why use it

Control the browser session tied to a prior scrape job. Use for code execution in the browser runtime.

### Preferred SDK method

- `client.interact(jobId, code)` &mdash; defaults to `"node"` language
- `client.interact(jobId, code, language, timeout)` &mdash; `timeout` is seconds (1&ndash;300), null for API default

### Example

```java
import com.firecrawl.models.BrowserExecuteResponse;

BrowserExecuteResponse result = client.interact(
"<scrapeJobId>",
"console.log(await page.title());",
"node",
60
);

// When done, stop the session:
client.stopInteractiveBrowser("<scrapeJobId>");
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `jobId` | String (required) | Scrape job ID from the scrape response metadata. |
| `code` | String (required) | Code to execute in the browser session. |
| `language` | String | Runtime: `"python"`, `"node"`, `"bash"`. Defaults to `"node"`. |
| `timeout` | Integer | Execution timeout in seconds (1&ndash;300). Null uses API default. |

### Stop session

`client.stopInteractiveBrowser(jobId)` ends the browser session. Returns `BrowserDeleteResponse` with `isSuccess()`, `getSessionDurationMs()`, `getCreditsBilled()`, `getError()`.

## Notes

- The Java SDK exposes **code-based interactions only**: there is no `prompt` parameter on `interact` (unlike Node.js, Python, and Rust SDKs).
- Deprecated aliases: `scrapeExecute` &rarr; `interact`; `deleteScrapeBrowser` &rarr; `stopInteractiveBrowser`.
- Uses `camelCase` parameter names matching the JSON API contract.

## Source Of Truth

- `firecrawl/apps/java-sdk/build.gradle.kts`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/client/FirecrawlClient.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/SearchOptions.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/ScrapeOptions.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/Document.java`
- `firecrawl-docs/api-reference/v2-openapi.json`
Loading