Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 165 additions & 0 deletions agent-quickstart/elixir.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
---
title: "Elixir Agent Quickstart"
description: "Canonical Firecrawl Elixir quickstart for external agents using search, scrape, and interact."
---

Canonical Firecrawl Elixir quickstart for external agents. Aligned with `:firecrawl` hex package **v1.9.1** (`firecrawl/apps/elixir-sdk`) and the v2 OpenAPI spec. The Elixir client is OpenAPI-generated; function names and parameter keys come directly from the spec.

## Install

Add to `mix.exs`:

```elixir
{:firecrawl, "~> 1.9"}
```

## Authenticate

```elixir
# config/runtime.exs or config.exs
config :firecrawl, api_key: System.get_env("FIRECRAWL_API_KEY")

# Or pass api_key per call:
{:ok, res} = Firecrawl.search_and_scrape([query: "example"], api_key: "fc-your-api-key")
```

## When To Use What

- `search`: use when you start with a query and need discovery.
- `scrape`: use when you already have a URL and want page content.
- `interact`: use when the page needs clicks, forms, or post-scrape browser actions.

## Search

### Why use it

Use search to discover relevant pages from a query, then pick URLs to scrape or interact with. You can constrain results to a site with `site:`, for example `site:docs.firecrawl.dev crawl webhooks`.

### Preferred SDK method

`Firecrawl.search_and_scrape(params \\ [], opts \\ [])`

### Example

```elixir
{:ok, res} = Firecrawl.search_and_scrape(query: "site:docs.firecrawl.dev webhook retries")
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `query` | `string` (required) | The search query. Use `site:example.com` to limit results to a domain. |
| `sources` | `list` | Sources to search. Values: `"web"`, `"news"`, `"images"`, or atoms `:web`, `:news`, `:images`. |
| `categories` | `list` | Filter by category. Values: `"github"`, `"research"`, `"pdf"`, or atoms. |
| `include_domains` | `list(string)` | Restrict results to these domains. |
| `exclude_domains` | `list(string)` | Exclude these domains. |
| `limit` | `integer` | Cap results. |
| `tbs` | `string` | Time-based filter (e.g. `qdr:d`, `qdr:w`). |
| `location` | `string` | Location string for localized results. |
| `country` | `string` | ISO 3166-1 alpha-2 country code (e.g. `"US"`). |
| `ignore_invalid_urls` | `boolean` | Drop URLs that cannot be scraped. |
| `timeout` | `integer` | Request timeout in milliseconds. |
| `highlights` | `boolean` | Generate query-relevant highlights. Defaults to true. |
| `scrape_options` | `keyword list` | Scrape each search result (see Scrape parameters). |
| `enterprise` | `list(string)` | Enterprise options. Values: `"zdr"`, `"anon"`. |

## Scrape

### Why use it

Use scrape when you already have a URL and want structured content in one or more formats.

### Preferred SDK method

`Firecrawl.scrape_and_extract_from_url(params \\ [], opts \\ [])`

### Example

```elixir
{:ok, res} = Firecrawl.scrape_and_extract_from_url(
url: "https://docs.firecrawl.dev",
formats: ["markdown"]
)
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `url` | `string` (required) | The URL to scrape. |
| `formats` | `list` | Output formats. Strings: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"changeTracking"`, `"json"`, `"branding"`, `"audio"`, `"video"`. Maps: `%{type: "json", prompt: "..."}`, `%{type: "screenshot", fullPage: true}`, `%{type: "changeTracking", modes: ["git-diff"]}`. |
| `headers` | `map` | Custom request headers. |
| `include_tags` | `list(string)` | Include only specific HTML tags. |
| `exclude_tags` | `list(string)` | Exclude specific HTML tags. |
| `only_main_content` | `boolean` | Strip nav, footer, and other boilerplate. |
| `timeout` | `integer` | Timeout in milliseconds. Min: 1000, default: 60000, max: 300000. |
| `wait_for` | `integer` | Wait for the page to render (milliseconds). |
| `mobile` | `boolean` | Use a mobile viewport. |
| `parsers` | `list` | File parsing controls. Values: `"pdf"`, `%{type: "pdf", mode: "auto", maxPages: 5}`. |
| `actions` | `list(map)` | Pre-scrape browser actions. Types: `wait`, `screenshot`, `click`, `write`, `press`, `scroll`, `scrape`, `executeJavascript`, `pdf`. |
| `location` | `keyword list` | Geo or language-aware scraping. Keys: `country:`, `languages:`. |
| `skip_tls_verification` | `boolean` | Skip TLS verification. |
| `remove_base64_images` | `boolean` | Drop base64 images from markdown output. |
| `block_ads` | `boolean` | Block ads and cookie popups. |
| `proxy` | `atom` | Proxy mode. Values: `:basic`, `:enhanced`, `:auto`. |
| `max_age` | `integer` | Use cached data up to this age (milliseconds). |
| `min_age` | `integer` | Use cached data only if at least this old (milliseconds). |
| `store_in_cache` | `boolean` | Cache the result. |
| `lockdown` | `boolean` | Only serve cached results, never make an outbound request. |
| `redact_pii` | `boolean` | Redact personally identifiable information. |
| `profile` | `keyword list` | Persistent browser profile. Keys: `name:`, `save_changes:`. |
| `zero_data_retention` | `boolean` | Enable zero data retention for this scrape. |
| `audit_metadata` | `keyword list` | User attribution for SIEM logging. Key: `username:` (required). |

## Interact

### Why use it

Use interact when a page requires browser actions or code execution after a scrape starts.

### Preferred SDK method

`Firecrawl.interact_with_scrape_browser_session(job_id, params \\ [], opts \\ [])`

### Example

```elixir
{:ok, res} = Firecrawl.interact_with_scrape_browser_session(
"<scrapeJobId>",
code: "console.log(await page.title());",
language: :node,
timeout: 60
)
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `job_id` | `string` | Scrape job ID. First positional argument. |
| `code` | `string` (required) | Code to execute in the browser session. |
| `language` | `atom \| string` | Runtime. Values: `:python`, `:node`, `:bash`. |
| `timeout` | `integer` | Execution timeout in seconds. |
| `origin` | `string` | Optional origin label for telemetry. |

### Stop session

`Firecrawl.stop_interactive_scrape_browser_session(job_id, opts \\ [])` issues `DELETE /scrape/{jobId}/interact`.

```elixir
{:ok, res} = Firecrawl.stop_interactive_scrape_browser_session("<scrapeJobId>")
```

## Notes

- The Elixir SDK is OpenAPI-generated; function names come from the spec and are not renamed.
- Every public function has a bang (`!`) variant that raises on error instead of returning `{:error, _}`.
- This SDK exposes code-based interactions only (no `prompt` parameter on `interact_with_scrape_browser_session`).
- Per-request options (like `api_key:`, `base_url:`) are passed via the trailing `opts` keyword list.

## Source Of Truth

- `firecrawl/apps/elixir-sdk/mix.exs`
- `firecrawl/apps/elixir-sdk/lib/firecrawl.ex`
- `firecrawl-docs/api-reference/v2-openapi.json`
188 changes: 188 additions & 0 deletions agent-quickstart/java.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
---
title: "Java Agent Quickstart"
description: "Canonical Firecrawl Java quickstart for external agents using search, scrape, and interact."
---

Canonical Firecrawl Java quickstart for external agents. Aligned with `firecrawl-java` **v1.12.1** (`firecrawl/apps/java-sdk`) and the v2 OpenAPI spec.

## Install

Maven:

```xml
<dependency>
<groupId>com.firecrawl</groupId>
<artifactId>firecrawl-java</artifactId>
<version>1.12.1</version>
</dependency>
```

Gradle:

```gradle
implementation("com.firecrawl:firecrawl-java:1.12.1")
```

## Authenticate

```java
import com.firecrawl.client.FirecrawlClient;

FirecrawlClient client = FirecrawlClient.builder()
.apiKey(System.getenv("FIRECRAWL_API_KEY"))
.build();

// Or from environment/system property:
// FirecrawlClient client = FirecrawlClient.fromEnv();
```

## When To Use What

- `search`: use when you start with a query and need discovery.
- `scrape`: use when you already have a URL and want page content.
- `interact`: use when the page needs clicks, forms, or post-scrape browser actions.

## Search

### Why use it

Use search to discover relevant pages from a query, then pick URLs to scrape or interact with. You can constrain results to a site with `site:`, for example `site:docs.firecrawl.dev crawl webhooks`.

### Preferred SDK method

`client.search(query)` or `client.search(query, options)` → `SearchData`

### Example

```java
import com.firecrawl.models.SearchData;
import java.util.List;
import java.util.Map;

SearchData results = client.search("site:docs.firecrawl.dev webhook retries");
List<Map<String, Object>> web = results.getWeb();
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `query` | `String` | The search query. Use `site:example.com` to limit results to a domain. |
| `options.sources` | `List<Object>` | Sources to search. Values: `"web"`, `"news"`, `"images"`. |
| `options.categories` | `List<Object>` | Filter by category. Values: `"github"`, `"research"`, `"pdf"`. |
| `options.includeDomains` | `List<String>` | Restrict results to these domains. |
| `options.excludeDomains` | `List<String>` | Exclude these domains. |
| `options.limit` | `Integer` | Cap results. |
| `options.tbs` | `String` | Time-based filter (e.g. `qdr:d`, `qdr:w`). |
| `options.location` | `String` | Location string for localized results. |
| `options.ignoreInvalidURLs` | `Boolean` | Drop URLs that cannot be scraped. |
| `options.timeout` | `Integer` | Request timeout in milliseconds. |
| `options.highlights` | `Boolean` | Generate query-relevant highlights. Defaults to true. |
| `options.scrapeOptions` | `ScrapeOptions` | Scrape each search result (see Scrape parameters). |

**Return value:** `SearchData` with `getWeb()`, `getNews()`, `getImages()` (each `List<Map<String, Object>>`, may be null). Do not treat `SearchData` as a directly iterable list.

## Scrape

### Why use it

Use scrape when you already have a URL and want structured content in one or more formats.

### Preferred SDK method

`client.scrape(url)` or `client.scrape(url, options)` → `Document`

### Example

```java
import com.firecrawl.models.ScrapeOptions;
import com.firecrawl.models.Document;

Document doc = client.scrape(
"https://docs.firecrawl.dev",
ScrapeOptions.builder().formats(List.of("markdown")).build()
);
System.out.println(doc.getMarkdown());
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `url` | `String` | The URL to scrape. |
| `options.formats` | `List<Object>` | Output formats. Strings: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"changeTracking"`, `"json"`, `"attributes"`, `"branding"`, `"audio"`, `"video"`. Objects: `JsonFormat.builder().prompt("...").build()`, `QuestionFormat`, `HighlightsFormat`. |
| `options.headers` | `Map<String, String>` | Custom request headers. |
| `options.includeTags` | `List<String>` | Include only specific HTML tags. |
| `options.excludeTags` | `List<String>` | Exclude specific HTML tags. |
| `options.onlyMainContent` | `Boolean` | Strip nav, footer, and other boilerplate. |
| `options.timeout` | `Integer` | Timeout in milliseconds. |
| `options.waitFor` | `Integer` | Wait for the page to render (milliseconds). |
| `options.mobile` | `Boolean` | Use a mobile viewport. |
| `options.parsers` | `List<Object>` | File parsing controls. Values: `"pdf"`, `Map.of("type", "pdf", "maxPages", 10)`. |
| `options.actions` | `List<Map<String, Object>>` | Pre-scrape browser actions. Types: `wait`, `screenshot`, `click`, `write`, `press`, `scroll`, `scrape`, `executeJavascript`, `pdf`. |
| `options.location` | `LocationConfig` | Geo or language-aware scraping. Fields: `country`, `languages`. |
| `options.skipTlsVerification` | `Boolean` | Skip TLS verification. |
| `options.removeBase64Images` | `Boolean` | Drop base64 images from markdown output. |
| `options.blockAds` | `Boolean` | Block ads and cookie popups. |
| `options.proxy` | `String` | Proxy mode. Values: `"basic"`, `"stealth"`, `"enhanced"`, `"auto"`. |
| `options.maxAge` | `Long` | Use cached data up to this age (milliseconds). |
| `options.storeInCache` | `Boolean` | Cache the result. |
| `options.lockdown` | `Boolean` | Only serve cached results, never make an outbound request. |
| `options.redactPII` | `Boolean` | Redact personally identifiable information. |
| `options.auditMetadata` | `AuditMetadata` | User attribution for SIEM logging. Constructor arg: `username`. |

## Interact

### Why use it

Use interact when a page requires browser actions or code execution after a scrape starts.

### Preferred SDK method

- `client.interact(jobId, code)` — uses default language `node`
- `client.interact(jobId, code, language, timeout)` — `timeout` in seconds (1–300), null for API default (30s)

### Example

```java
import com.firecrawl.models.BrowserExecuteResponse;

BrowserExecuteResponse result = client.interact(
"<scrapeJobId>",
"console.log(await page.title());",
"node",
60
);
```

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `jobId` | `String` | Scrape job ID. |
| `code` | `String` | Code to run in the browser session. |
| `language` | `String` | Runtime. Values: `"python"`, `"node"`, `"bash"`. Default: `"node"`. |
| `timeout` | `Integer` | Execution timeout in seconds (1–300). Null for API default. |
| `origin` | `String` | Optional origin label for request attribution. |

### Stop session

`client.stopInteractiveBrowser(jobId)` → `BrowserDeleteResponse`

Ends the scrape-bound browser session. Response includes `isSuccess()`, `getSessionDurationMs()`, `getCreditsBilled()`.

## Notes

- The Java SDK exposes code-based interactions only: there is no `prompt` parameter on `interact` (unlike JS, Python, and Rust SDKs).
- Deprecated aliases: `scrapeExecute` → `interact`; `deleteScrapeBrowser` → `stopInteractiveBrowser`.
- Async variants are available: `scrapeAsync`, `searchAsync`, `interactAsync`, `stopInteractiveBrowserAsync` — all return `CompletableFuture`.
- Uses camelCase parameter names matching the Java convention.

## Source Of Truth

- `firecrawl/apps/java-sdk/build.gradle.kts`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/client/FirecrawlClient.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/ScrapeOptions.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/SearchOptions.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/Document.java`
- `firecrawl-docs/api-reference/v2-openapi.json`
Loading