Skip to content

docs: add agent quickstart files for all SDK languages #1217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
ericciarla wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

docs: add agent quickstart files for all SDK languages #1217

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 234 additions & 0 deletions agent-quickstart/elixir.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
---
title: "Elixir Agent Quickstart"
description: "Canonical Firecrawl Elixir quickstart for external agents using search, scrape, and interact."
---

# Firecrawl Elixir Agent Quickstart

This file is the canonical quickstart for external agents integrating Firecrawl with Elixir. Generated from SDK source (`firecrawl` hex package v1.9) and the Firecrawl OpenAPI spec.

The Elixir SDK is auto-generated from the Firecrawl OpenAPI spec. Function names match OpenAPI operation IDs.

## Install

Add to your `mix.exs`:

```elixir
defp deps do
[
{:firecrawl, "~> 1.4"}
]
end
```

## Authenticate

Configure the API key globally in your application config:

```elixir
# config/config.exs
config :firecrawl, api_key: "fc-YOUR_API_KEY"
```

Or pass it per-call:

```elixir
Firecrawl.scrape_and_extract_from_url([url: "https://example.com"], api_key: "fc-YOUR_API_KEY")
```

Additional per-call options:

| Option | Type | Default | Description |
|---|---|---|---|
| `api_key` | `string` | Application config | Override API key for this request. |
| `base_url` | `string` | `"https://api.firecrawl.dev/v2"` | Override base URL (self-hosted). |

A nil API key is allowed — scrape, search, and interact fall back to a keyless free tier (rate-limited per IP).

## When To Use What

- **`search_and_scrape`**: Start with a query, discover relevant pages. Returns search results with optional scraping of each result.
- **`scrape_and_extract_from_url`**: Start with a URL, get page content in markdown, HTML, or other formats.
- **`interact_with_scrape_browser_session`**: Execute code in a browser session from a prior scrape. Use for post-scrape browser automation.

## Search

### Why use it

Use `search_and_scrape` when you have a query and want to discover relevant web pages. Optionally scrape each result to get full page content.

### Preferred SDK function

```
Firecrawl.search_and_scrape(params, opts)
```

Bang variant: `Firecrawl.search_and_scrape!(params, opts)` raises on error.

### Example

```elixir
{:ok, response} = Firecrawl.search_and_scrape(
query: "firecrawl web scraping API",
limit: 5,
scrape_options: [formats: ["markdown"]]
)

for item <- response.body["data"]["web"] || [] do
IO.puts("#{item["url"]} #{String.slice(item["markdown"] || "", 0..200)}")
end
```

### Parameters

Parameters are passed as a keyword list. Validated at call time via NimbleOptions.

| Parameter | Type | Description |
|---|---|---|
| `query` | `string` | Search query (required). |
| `limit` | `integer` | Max results. |
| `sources` | `list(any)` | Sources to search. |
| `categories` | `list(any)` | Category filters. |
| `include_domains` | `list(string)` | Restrict results to these domains. |
| `exclude_domains` | `list(string)` | Exclude results from these domains. |
| `tbs` | `string` | Time-based search filter (e.g. `"qdr:d"`). |
| `location` | `string` | Location string for geo-targeted search. |
| `country` | `string` | Country code (ISO). |
| `timeout` | `integer` | Timeout in ms. |
| `highlights` | `boolean` | Generate query-relevant highlights. Server default: `true`. |
| `ignore_invalid_urls` | `boolean` | Skip invalid URLs. |
| `enterprise` | `list(string)` | Enterprise options: `["zdr"]` or `["anon"]`. |
| `scrape_options` | `keyword` | Options for scraping each result page (same params as scrape). |

**Return type:** `{:ok, %Req.Response{}}` or `{:error, exception}`. The response body is a decoded JSON map.

## Scrape

### Why use it

Use `scrape_and_extract_from_url` when you already have a URL and want page content as markdown, HTML, screenshots, structured JSON, or other formats.

### Preferred SDK function

```
Firecrawl.scrape_and_extract_from_url(params, opts)
```

Bang variant: `Firecrawl.scrape_and_extract_from_url!(params, opts)` raises on error.

### Example

```elixir
{:ok, response} = Firecrawl.scrape_and_extract_from_url(
url: "https://example.com",
formats: ["markdown", "links"],
only_main_content: true
)

data = response.body["data"]
IO.puts(data["markdown"])
IO.puts(data["metadata"]["title"])
```

### Parameters

Parameters are passed as a keyword list.

| Parameter | Type | Description |
|---|---|---|
| `url` | `string` | URL to scrape (required). |
| `formats` | `list(any)` | Output formats: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"json"`, `"changeTracking"`, `"branding"`, `"product"`, `"menu"`, `"audio"`, `"video"`, etc. |
| `headers` | `any` | Custom HTTP headers. |
| `include_tags` | `list(string)` | HTML tags to include. |
| `exclude_tags` | `list(string)` | HTML tags to exclude. |
| `only_main_content` | `boolean` | Only return main content. Server default: `true`. |
| `timeout` | `integer` | Timeout in ms. Server default: `60000`. |
| `wait_for` | `integer` | Wait time in ms before scraping. |
| `mobile` | `boolean` | Emulate mobile device. |
| `parsers` | `list(any)` | Parser configuration. |
| `actions` | `list(any)` | Browser actions before scraping. Each is a keyword list or map with `type` key. |
| `location` | `keyword` | Geolocation: `[country: "US", languages: ["en"]]`. |
| `skip_tls_verification` | `boolean` | Skip TLS cert verification. |
| `remove_base64_images` | `boolean` | Remove base64 images. |
| `block_ads` | `boolean` | Block ads. Server default: `true`. |
| `proxy` | `:basic \| :enhanced \| :auto` | Proxy mode. |
| `max_age` | `integer` | Use cached result if younger than this (ms). |
| `store_in_cache` | `boolean` | Cache the result. |
| `lockdown` | `boolean` | Serve from cache only. |
| `redact_pii` | `boolean` | Redact PII from output. |
| `profile` | `keyword` | Persistent browser profile. |
| `audit_metadata` | `keyword` | User attribution: `[username: "user"]`. |
| `zero_data_retention` | `boolean` | Enable zero data retention. |

**Return type:** `{:ok, %Req.Response{}}`. The response body `data` field is a map with `"markdown"`, `"html"`, `"metadata"`, `"links"`, etc.

## Interact

### Why use it

Use `interact_with_scrape_browser_session` to execute code in the browser session from a prior scrape. The scrape must have returned a job ID. Use it for post-scrape automation.

### Preferred SDK function

```
Firecrawl.interact_with_scrape_browser_session(job_id, params, opts)
```

Bang variant: `Firecrawl.interact_with_scrape_browser_session!(job_id, params, opts)` raises on error.

### Example

```elixir
# First scrape to get a job_id
{:ok, scrape_response} = Firecrawl.scrape_and_extract_from_url(
url: "https://example.com",
formats: ["markdown"]
)
job_id = scrape_response.body["data"]["metadata"]["jobId"]

# Interact with the browser session
{:ok, result} = Firecrawl.interact_with_scrape_browser_session(job_id,
code: "document.title",
language: :node
)

IO.puts(result.body["stdout"])
IO.puts(result.body["result"])
```

### Parameters

The first argument is `job_id` (string). Remaining parameters are passed as a keyword list.

| Parameter | Type | Description |
|---|---|---|
| `job_id` | `string` | Scrape job ID (required, first positional argument). |
| `code` | `string` | Code to execute (required). |
| `language` | `:python \| :node \| :bash` | Language for code execution. Default: `"node"` (server-side). |
| `timeout` | `integer` | Execution timeout in seconds. Server default: `30`. |

**Return type:** `{:ok, %Req.Response{}}`. Body contains `"success"`, `"stdout"`, `"result"`, `"stderr"`, `"exitCode"`, `"killed"`, `"error"`.

### Stop a session

```elixir
Firecrawl.stop_interactive_scrape_browser_session(job_id)
```

## Notes

- **Auto-generated from OpenAPI**: Function names match OpenAPI operation IDs and are verbose (e.g. `scrape_and_extract_from_url` instead of `scrape`).
- **snake_case in, camelCase out**: Pass snake_case keywords (e.g. `only_main_content: true`). They are automatically converted to camelCase JSON keys.
- **NimbleOptions validation**: Parameters are validated at call time. Passing an unrecognized key or wrong type returns `{:error, %NimbleOptions.ValidationError{}}` immediately.
- **Bang variants**: Every function `foo/n` has a `foo!/n` that raises `Firecrawl.Error` instead of returning `{:error, _}`.
- **No client struct**: Unlike other SDKs, there is no connection object to create upfront. A fresh `Req` client is built per call.
- **Response bodies are raw JSON maps**: No typed response structs — access data via map keys like `response.body["data"]["markdown"]`.
- **No deprecated aliases**: The generated client has one function per operation.
- **Keyless free tier**: Works without an API key (rate-limited per IP).
- **Batch scraping**: `Firecrawl.scrape_and_extract_from_urls/2` is available for batch operations.

## Source Of Truth

- `/firecrawl/apps/elixir-sdk/lib/firecrawl.ex`
- `/firecrawl/apps/elixir-sdk/mix.exs`
- `/firecrawl-docs/api-reference/v2-openapi.json`
Loading