Skip to content

docs(agent-quickstart): add per-language quickstarts for search, scrape, and interact #1236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
ericciarla wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

docs(agent-quickstart): add per-language quickstarts for search, scrape, and interact #1236

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
201 changes: 201 additions & 0 deletions agent-quickstart/elixir.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
---
title: "Elixir Agent Quickstart"
description: "Canonical Firecrawl Elixir quickstart for external agents using search, scrape, and interact."
---

# Firecrawl Elixir Agent Quickstart

Canonical quickstart for external agents. Generated from SDK source (`:firecrawl` hex package) and the v2 OpenAPI spec. Function names and parameter keys are auto-generated from the OpenAPI spec.

## Install

Add to `mix.exs`:

```elixir
{:firecrawl, "~> 1.9"}
```

## Authenticate

```elixir
# config/runtime.exs or config.exs
config :firecrawl, api_key: System.get_env("FIRECRAWL_API_KEY")

# Or pass api_key per call:
{:ok, res} = Firecrawl.scrape_and_extract_from_url(
[url: "https://example.com", formats: ["markdown"]],
api_key: "fc-your-api-key"
)
```

All functions accept `:api_key` and `:base_url` (default `"https://api.firecrawl.dev/v2"`) in the trailing `opts` keyword list.

## When To Use What

- **`search`**: start with a query and need discovery. Returns ranked results, optionally with scraped content.
- **`scrape`**: already have a URL and want page content in one or more formats.
- **`interact`**: the page needs clicks, forms, or post-scrape browser actions. Requires a scrape job ID from a prior scrape.

## Search

### Why use it

Discover relevant pages from a query, then pick URLs to scrape or interact with. Use `site:example.com` in the query to limit results to a domain.

### Preferred SDK method

`Firecrawl.search_and_scrape(params \\ [], opts \\ [])`

### Example

```elixir
{:ok, res} = Firecrawl.search_and_scrape(
query: "site:docs.firecrawl.dev webhook retries",
sources: [:web, :news],
limit: 10,
scrape_options: [
formats: ["markdown"],
only_main_content: true
]
)

web_results = res.body["data"]["web"]
```

### Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| `query` | `:string` (required) | Search query. Use `site:example.com` to scope to a domain. |
| `sources` | `{:list, :any}` | Sources: `:web`, `:news`, `:images` (atoms or strings). |
| `categories` | `{:list, :any}` | Filters: `:github`, `:research`, `:pdf` (atoms or strings). |
| `include_domains` | `{:list, :string}` | Domains to include. |
| `exclude_domains` | `{:list, :string}` | Domains to exclude. |
| `limit` | `:integer` | Maximum results. |
| `tbs` | `:string` | Time-based filter (e.g. `"qdr:d"` for past day). |
| `location` | `:string` | Geographic location for localized results. |
| `country` | `:string` | ISO 3166-1 alpha-2 country code (e.g. `"US"`). |
| `ignore_invalid_urls` | `:boolean` | Drop URLs that cannot be scraped. |
| `timeout` | `:integer` | Timeout in milliseconds. |
| `highlights` | `:boolean` | Query-relevant highlights. Default: `true`. |
| `enterprise` | `{:list, :string}` | Enterprise options: `"zdr"`, `"anon"`. |
| `scrape_options` | `:keyword_list` | Scrape each result (see Scrape parameters). |

## Scrape

### Why use it

Get structured content from a URL in one or more formats — markdown, HTML, JSON extraction, screenshots, and more.

### Preferred SDK method

`Firecrawl.scrape_and_extract_from_url(params \\ [], opts \\ [])`

### Example

```elixir
{:ok, res} = Firecrawl.scrape_and_extract_from_url(
url: "https://example.com/pricing",
formats: [
"markdown",
"links",
%{type: "json", prompt: "Extract plan names and prices."}
],
only_main_content: true,
wait_for: 1000
)

markdown = res.body["data"]["markdown"]
json_data = res.body["data"]["json"]
```

### Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| `url` | `:string` (required) | Target URL to scrape. |
| `formats` | `{:list, :any}` | Output formats: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"json"`, `"changeTracking"`, `"attributes"`, `"branding"`, `"audio"`, `"video"`. Also format maps with `type` key. |
| `headers` | `:any` | Custom HTTP headers as a map. |
| `include_tags` | `{:list, :string}` | HTML tags to include. |
| `exclude_tags` | `{:list, :string}` | HTML tags to exclude. |
| `only_main_content` | `:boolean` | Strip nav, footer, boilerplate. |
| `timeout` | `:integer` | Timeout in milliseconds. Min: 1000, max: 300000. |
| `wait_for` | `:integer` | Wait time in ms before scraping. |
| `mobile` | `:boolean` | Emulate a mobile device. |
| `parsers` | `{:list, :any}` | Parser config: `"pdf"` or `%{type: "pdf", mode: "auto", maxPages: 5}`. |
| `actions` | `{:list, :any}` | Browser actions (see action types below). |
| `location` | `:keyword_list` | `[country: "US", languages: ["en-US"]]`. |
| `skip_tls_verification` | `:boolean` | Skip TLS verification. |
| `remove_base64_images` | `:boolean` | Drop base64 images from markdown. |
| `block_ads` | `:boolean` | Block ads and cookie popups. |
| `proxy` | `{:in, [:basic, :enhanced, :auto]}` | Proxy mode. |
| `max_age` | `:integer` | Max cache age in milliseconds. |
| `min_age` | `:integer` | Min cache age in milliseconds. |
| `store_in_cache` | `:boolean` | Store result in cache. |
| `lockdown` | `:boolean` | Only serve cached results. |
| `redact_pii` | `:boolean` | Redact personally identifiable information. |
| `profile` | `:keyword_list` | `[name: "my-session", save_changes: true]`. |
| `zero_data_retention` | `:boolean` | Enable zero data retention. |
| `audit_metadata` | `:keyword_list` | `[username: "agent-x"]`. |

### Action types

Pass as list of maps: `%{type: "wait", milliseconds: 750}`, `%{type: "click", selector: "#accept"}`, `%{type: "write", text: "hello"}`, `%{type: "press", key: "Enter"}`, `%{type: "scroll", direction: "down"}`, `%{type: "scrape"}`, `%{type: "executeJavascript", script: "..."}`, `%{type: "screenshot"}`, `%{type: "pdf"}`.

## Interact

### Why use it

Execute code in the browser session tied to a scrape job. The Elixir SDK exposes code-based interactions only (no `prompt` parameter).

### Preferred SDK method

`Firecrawl.interact_with_scrape_browser_session(job_id, params \\ [], opts \\ [])`

### Example

```elixir
{:ok, scrape_res} = Firecrawl.scrape_and_extract_from_url(
url: "https://example.com",
formats: ["markdown"]
)

job_id = scrape_res.body["data"]["metadata"]["scrapeId"]

{:ok, res} = Firecrawl.interact_with_scrape_browser_session(
job_id,
code: "console.log(await page.title());",
language: :node,
timeout: 60
)
```

### Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| `job_id` | `String.t()` (first arg) | Scrape job ID from response metadata. |
| `code` | `:string` (required) | Code to execute in the browser session. |
| `language` | `{:in, [:python, :node, :bash]}` | Runtime for code execution. |
| `timeout` | `:integer` | Execution timeout in seconds. |

### Stop session

`Firecrawl.stop_interactive_scrape_browser_session(job_id)` → `DELETE /scrape/{jobId}/interact`. Ends the browser session.

## Notes

- The Elixir SDK is auto-generated from the OpenAPI spec. Function names are verbose (e.g. `scrape_and_extract_from_url`) and come directly from operation IDs.
- Parameters use snake_case atom keys — they are automatically converted to camelCase JSON.
- Nested keyword lists are recursively camelCased (e.g. `scrape_options: [only_main_content: true]` → `{"scrapeOptions": {"onlyMainContent": true}}`).
- Atom values (except `true`/`false`/`nil`) are converted to strings (e.g. `proxy: :basic` → `"proxy": "basic"`).
- Every function has a bang (`!`) variant that raises on error instead of returning `{:error, _}`.
- Responses are raw `Req.Response` structs — access data via `res.body["data"]`.
- Parameters are validated client-side with NimbleOptions before sending.
- The Elixir SDK does not support `prompt` on interact — only code-based interactions.

## Source Of Truth

- `firecrawl/apps/elixir-sdk/mix.exs`
- `firecrawl/apps/elixir-sdk/lib/firecrawl.ex`
- `firecrawl-docs/api-reference/v2-openapi.json`
Loading