From f53a53a9a003ec309ff2e490e66bb1b1e9755862 Mon Sep 17 00:00:00 2001 From: Chenxin Yan Date: Mon, 17 Aug 2026 16:14:48 -0400 Subject: [PATCH 1/3] docs(skills): correct behavior claims --- skills/firecrawl-agent/SKILL.md | 39 ++++++++++++++++++++++-------- skills/firecrawl-download/SKILL.md | 10 +++++--- skills/firecrawl-interact/SKILL.md | 17 +++++++------ skills/firecrawl-parse/SKILL.md | 18 +++++++------- skills/firecrawl-scrape/SKILL.md | 26 ++++++++++---------- skills/firecrawl-search/SKILL.md | 4 +-- 6 files changed, 69 insertions(+), 45 deletions(-) diff --git a/skills/firecrawl-agent/SKILL.md b/skills/firecrawl-agent/SKILL.md index 18fd81bff5..48b38aa57a 100644 --- a/skills/firecrawl-agent/SKILL.md +++ b/skills/firecrawl-agent/SKILL.md @@ -30,18 +30,37 @@ firecrawl agent "extract products" --schema '{"type":"object","properties":{"nam firecrawl agent "get feature list" --urls "" --wait -o .firecrawl/features.json ``` +## Job IDs + +Without `--wait`, the command returns a job ID. A UUID positional argument is auto-detected as a status check: + +```bash +# Check once (equivalent to adding --status) +firecrawl agent "" + +# Wait on an existing job, polling every 10 seconds for up to 5 minutes +firecrawl agent "" --wait --poll-interval 10 --timeout 300 + +# Cancel an active job +firecrawl agent "" --cancel +``` + ## Options -| Option | Description | -| ---------------------- | ----------------------------------------- | -| `--urls ` | Starting URLs for the agent | -| `--model ` | Model to use: spark-1-mini or spark-1-pro | -| `--schema ` | JSON schema for structured output | -| `--schema-file ` | Path to JSON schema file | -| `--max-credits ` | Credit limit for this agent run | -| `--wait` | Wait for agent to complete | -| `--pretty` | Pretty print JSON output | -| `-o, --output ` | Output file path | +| Option | Description | +| --------------------------- | --------------------------------------------------- | +| `--urls ` | Starting URLs for the agent | +| `--model ` | Model to use: spark-1-mini or spark-1-pro | +| `--schema ` | JSON schema for structured output | +| `--schema-file ` | Path to JSON schema file | +| `--max-credits ` | Credit limit for this agent run | +| `--status` | Check a job ID's status | +| `--cancel` | Cancel an active job ID | +| `--wait` | Wait for agent to complete | +| `--poll-interval ` | Polling interval while waiting (default: 5 seconds) | +| `--timeout ` | Stop waiting after this duration (default: none) | +| `--pretty` | Pretty print JSON output | +| `-o, --output ` | Output file path | ## Tips diff --git a/skills/firecrawl-download/SKILL.md b/skills/firecrawl-download/SKILL.md index d2beeb7c66..82a026d949 100644 --- a/skills/firecrawl-download/SKILL.md +++ b/skills/firecrawl-download/SKILL.md @@ -11,7 +11,7 @@ allowed-tools: > **Experimental.** Convenience command that combines `map` + `scrape` to save an entire site as local files. -Maps the site first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. All scrape options work with download. Always pass `-y` to skip the confirmation prompt. +Maps the site origin first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. Use `--include-paths` to scope a non-root URL to one section. Always pass `-y` to skip the confirmation prompt. ## When to use @@ -33,10 +33,10 @@ firecrawl download https://docs.example.com --format markdown,links --screenshot # Creates per page: index.md + links.txt + screenshot.png # Filter to specific sections -firecrawl download https://docs.example.com --include-paths "/features,/sdks" +firecrawl download https://docs.example.com --include-paths "/features,/sdks" -y # Skip translations -firecrawl download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR" +firecrawl download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR" -y # Full combo firecrawl download https://docs.example.com \ @@ -58,7 +58,9 @@ firecrawl download https://docs.example.com \ | `--allow-subdomains` | Include subdomain pages | | `-y` | Skip confirmation prompt (always use in automated flows) | -## Scrape options (all work with download) +## Supported scrape options + +Only the options listed below are supported: `-f `, `-H`, `-S`, `--screenshot`, `--full-page-screenshot`, `--only-main-content`, `--include-tags`, `--exclude-tags`, `--wait-for`, `--max-age`, `--country`, `--languages` diff --git a/skills/firecrawl-interact/SKILL.md b/skills/firecrawl-interact/SKILL.md index 7468dd2083..446493ca68 100644 --- a/skills/firecrawl-interact/SKILL.md +++ b/skills/firecrawl-interact/SKILL.md @@ -25,14 +25,17 @@ Interact with scraped pages in a live browser session. Scrape a page first, then # 1. Scrape a page (scrape ID is saved automatically) firecrawl scrape "" -# 2. Interact with the page using natural language -firecrawl interact --prompt "Click the login button" -firecrawl interact --prompt "Fill in the email field with test@example.com" -firecrawl interact --prompt "Extract the pricing table" +# 2. Interact with the page using a positional prompt +firecrawl interact "Click the login button" +firecrawl interact "Fill in the email field with test@example.com" +firecrawl interact "Extract the pricing table" + +# A UUID first argument is auto-detected as the scrape ID +firecrawl interact "" "Extract the pricing table" # 3. Or use code for precise control -firecrawl interact --code "agent-browser click @e5" --language bash -firecrawl interact --code "agent-browser snapshot -i" --language bash +firecrawl interact --code "click @e5" --bash +firecrawl interact --code "snapshot -i" --bash # 4. Stop the session when done firecrawl interact stop @@ -72,7 +75,7 @@ firecrawl scrape "https://app.example.com" --profile my-app --no-save-changes ## Tips - Always scrape first — `interact` requires a scrape ID from a previous `firecrawl scrape` call -- The scrape ID is saved automatically, so you don't need `--scrape-id` for subsequent interact calls +- The scrape ID is saved automatically, so you don't need `--scrape-id` for subsequent interact calls. Saved sessions may expire after about 10 minutes; re-scrape if the CLI warns that the session is stale - Use `firecrawl interact stop` to free resources when done - For parallel work, scrape multiple pages and interact with each using `--scrape-id` diff --git a/skills/firecrawl-parse/SKILL.md b/skills/firecrawl-parse/SKILL.md index f350cb2a98..cfae90cef7 100644 --- a/skills/firecrawl-parse/SKILL.md +++ b/skills/firecrawl-parse/SKILL.md @@ -9,7 +9,7 @@ allowed-tools: # firecrawl parse -Turn a local document into clean markdown on disk. Supports **PDF, DOCX, DOC, ODT, RTF, XLSX, XLS, HTML/HTM/XHTML**. +Turn a local document into clean markdown on disk. Supports **PDF, DOCX, DOC, ODT, RTF, XLSX, XLS, HTML/HTM**. ## When to use @@ -39,14 +39,14 @@ Then `head`, `grep`, `rg` etc., or incrementally read the file - don't load the ## Options -| Option | Description | -| ---------------------- | --------------------------------------- | -| `-S, --summary` | AI-generated summary | -| `-Q, --query ` | Ask a question about the parsed content | -| `-o, --output ` | Output file path — **always use this** | -| `-f, --format ` | `markdown` (default), `html`, `summary` | -| `--timeout ` | Timeout for the parse job | -| `--timing` | Show request duration | +| Option | Description | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | +| `-S, --summary` | AI-generated summary | +| `-Q, --query ` | Ask a question about the parsed content | +| `-o, --output ` | Output file path — **always use this** | +| `-f, --format ` | Comma-separated: `markdown`, `html`, `rawHtml`, `links`, `images`, `summary`, `json`, `attributes`. Multiple formats output JSON | +| `--timeout ` | Timeout for the parse job | +| `--timing` | Show request duration | ## Tips diff --git a/skills/firecrawl-scrape/SKILL.md b/skills/firecrawl-scrape/SKILL.md index 6c6612756c..7866ba86a9 100644 --- a/skills/firecrawl-scrape/SKILL.md +++ b/skills/firecrawl-scrape/SKILL.md @@ -29,7 +29,7 @@ firecrawl scrape "" --only-main-content -o .firecrawl/page.md # Wait for JS to render, then scrape firecrawl scrape "" --wait-for 3000 -o .firecrawl/page.md -# Multiple URLs (each saved to .firecrawl/) +# Multiple URLs (markdown only; each saved to .firecrawl/; -o is ignored) firecrawl scrape https://example.com https://example.com/blog https://example.com/docs # Get markdown and links together @@ -41,23 +41,23 @@ firecrawl scrape "https://example.com/pricing" --query "What is the enterprise p ## Options -| Option | Description | -| ------------------------ | ---------------------------------------------------------------- | -| `-f, --format ` | Output formats: markdown, html, rawHtml, links, screenshot, json | -| `-Q, --query ` | Ask a question about the page content (5 credits) | -| `-H` | Include HTTP headers in output | -| `--only-main-content` | Strip nav, footer, sidebar — main content only | -| `--wait-for ` | Wait for JS rendering before scraping | -| `--include-tags ` | Only include these HTML tags | -| `--exclude-tags ` | Exclude these HTML tags | -| `--redact-pii` | Redact personally identifiable information from output | -| `-o, --output ` | Output file path | +| Option | Description | +| ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | +| `-f, --format ` | Output formats: markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding | +| `-Q, --query ` | Ask a question about the page content (5 credits) | +| `-H` | Include HTTP headers in output | +| `--only-main-content` | Strip nav, footer, sidebar — main content only | +| `--wait-for ` | Wait for JS rendering before scraping | +| `--include-tags ` | Only include these HTML tags | +| `--exclude-tags ` | Exclude these HTML tags | +| `--redact-pii` | Redact personally identifiable information from output | +| `-o, --output ` | Output file path | ## Tips - **Prefer plain scrape over `--query`.** Scrape to a file, then use `grep`, `head`, or read the markdown directly — you can search and reason over the full content yourself. Use `--query` only when you want a single targeted answer without saving the page (costs 5 extra credits). - **Try scrape before interact.** Scrape handles static pages and JS-rendered SPAs. Only escalate to `interact` when you need interaction (clicks, form fills, pagination). -- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. +- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. This mode only saves usable markdown; other formats are JSON-stringified into `.md` files, and `-o` is ignored. - Single format outputs raw content. Multiple formats (e.g., `--format markdown,links`) output JSON. - Always quote URLs — shell interprets `?` and `&` as special characters. - Naming convention: `.firecrawl/{site}-{path}.md` diff --git a/skills/firecrawl-search/SKILL.md b/skills/firecrawl-search/SKILL.md index 00c110ed24..770aa1b9b3 100644 --- a/skills/firecrawl-search/SKILL.md +++ b/skills/firecrawl-search/SKILL.md @@ -135,10 +135,10 @@ Search costs 2 credits. After you've actually used the results (or decided they - **Idempotent:** re-submitting for the same search id returns success but no extra refund. - **`--silent &`** is the right pattern — exit code 0 even on failure, so a rejected/expired call never crashes your pipeline. -Read the search response's `id`: +Verify the search returned results before reading its `id`. Zero-result searches do not write the output file, so never send feedback from a missing or stale file. ```bash -SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json) +SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json) ``` Then send feedback. Pick the rating that matches what actually happened: From 43872bef4566518dd186d7b627252e8468fbf4e8 Mon Sep 17 00:00:00 2001 From: Chenxin Yan Date: Mon, 17 Aug 2026 16:24:25 -0400 Subject: [PATCH 2/3] docs(skills): clarify multi-URL scrape drops non-markdown formats --- skills/firecrawl-scrape/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/firecrawl-scrape/SKILL.md b/skills/firecrawl-scrape/SKILL.md index 7866ba86a9..305ff5fe5b 100644 --- a/skills/firecrawl-scrape/SKILL.md +++ b/skills/firecrawl-scrape/SKILL.md @@ -57,7 +57,7 @@ firecrawl scrape "https://example.com/pricing" --query "What is the enterprise p - **Prefer plain scrape over `--query`.** Scrape to a file, then use `grep`, `head`, or read the markdown directly — you can search and reason over the full content yourself. Use `--query` only when you want a single targeted answer without saving the page (costs 5 extra credits). - **Try scrape before interact.** Scrape handles static pages and JS-rendered SPAs. Only escalate to `interact` when you need interaction (clicks, form fills, pagination). -- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. This mode only saves usable markdown; other formats are JSON-stringified into `.md` files, and `-o` is ignored. +- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. This mode only saves markdown; if markdown isn't requested, the whole response is JSON-stringified into the `.md` file (other formats are otherwise dropped), and `-o` is ignored. - Single format outputs raw content. Multiple formats (e.g., `--format markdown,links`) output JSON. - Always quote URLs — shell interprets `?` and `&` as special characters. - Naming convention: `.firecrawl/{site}-{path}.md` From 30fe1a2e4863f6474f4619a12e1cd6179dc61194 Mon Sep 17 00:00:00 2001 From: Chenxin Yan Date: Mon, 17 Aug 2026 18:27:14 -0400 Subject: [PATCH 3/3] docs(skills): fix remaining wrong claims found in review Adversarial review against src/ surfaced wrong claims that survived the first audit pass, in the same files this PR touches: - scrape: -H is --html (raw HTML shortcut), not HTTP headers - interact: --language does not exist; the CLI uses --node/--python/--bash - download: the command is only registered as 'firecrawl x download' (experimental), so all examples used a nonexistent invocation; the 'only these options' list omitted --lockdown; the wizard example is TTY-only and blocks agents without -y - search: spell out guard-failure behavior and split the missing-file vs stale-file hazards - scrape/agent: reword confusing multi-URL parenthetical and dedupe the --wait tip - cli: mirror the guarded jq search-id recipe so it doesn't drift from the corrected search skill --- skills/firecrawl-agent/SKILL.md | 2 +- skills/firecrawl-cli/SKILL.md | 7 +++---- skills/firecrawl-download/SKILL.md | 18 +++++++++--------- skills/firecrawl-interact/SKILL.md | 16 ++++++++-------- skills/firecrawl-scrape/SKILL.md | 4 ++-- skills/firecrawl-search/SKILL.md | 4 ++-- 6 files changed, 25 insertions(+), 26 deletions(-) diff --git a/skills/firecrawl-agent/SKILL.md b/skills/firecrawl-agent/SKILL.md index 48b38aa57a..ff5c029bc9 100644 --- a/skills/firecrawl-agent/SKILL.md +++ b/skills/firecrawl-agent/SKILL.md @@ -64,7 +64,7 @@ firecrawl agent "" --cancel ## Tips -- Always use `--wait` to get results inline. Without it, returns a job ID. +- Use `--wait` for inline results; without it you get a job ID (see [Job IDs](#job-ids)). - Use `--schema` for predictable, structured output — otherwise the agent returns freeform data. - Agent runs consume more credits than simple scrapes. Use `--max-credits` to cap spending. - For simple single-page extraction, prefer `scrape` — it's faster and cheaper. diff --git a/skills/firecrawl-cli/SKILL.md b/skills/firecrawl-cli/SKILL.md index d6ef72e504..5049da84ac 100644 --- a/skills/firecrawl-cli/SKILL.md +++ b/skills/firecrawl-cli/SKILL.md @@ -194,9 +194,7 @@ The `check` response then carries a per-field diff (paths like `plans[0].price`) }, "snapshot": { "json": { - "plans": [ - /* current full extraction */ - ] + "plans": [/* current full extraction */] } } } @@ -269,7 +267,8 @@ jq -r '.data.web[] | "\(.title): \(.url)"' .firecrawl/search.json Search costs 2 credits per call. **After you finish using a search result**, send structured feedback in the background. The first feedback per search id refunds 1 credit and feeds search-quality improvements. ```bash -SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json) +# Guard: zero-result searches write no output file, so verify before reading the id +SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json) firecrawl search-feedback "$SEARCH_ID" \ --rating good \ diff --git a/skills/firecrawl-download/SKILL.md b/skills/firecrawl-download/SKILL.md index 82a026d949..70a9cb9ac0 100644 --- a/skills/firecrawl-download/SKILL.md +++ b/skills/firecrawl-download/SKILL.md @@ -7,7 +7,7 @@ allowed-tools: - Bash(npx firecrawl *) --- -# firecrawl download +# firecrawl download (invoked as `firecrawl x download`) > **Experimental.** Convenience command that combines `map` + `scrape` to save an entire site as local files. @@ -22,24 +22,24 @@ Maps the site origin first to discover pages, then scrapes each one into nested ## Quick start ```bash -# Interactive wizard (picks format, screenshots, paths for you) -firecrawl download https://docs.example.com +# Interactive wizard (humans at a TTY only — agents must pass -y or the command blocks on a prompt) +firecrawl x download https://docs.example.com # With screenshots -firecrawl download https://docs.example.com --screenshot --limit 20 -y +firecrawl x download https://docs.example.com --screenshot --limit 20 -y # Multiple formats (each saved as its own file per page) -firecrawl download https://docs.example.com --format markdown,links --screenshot --limit 20 -y +firecrawl x download https://docs.example.com --format markdown,links --screenshot --limit 20 -y # Creates per page: index.md + links.txt + screenshot.png # Filter to specific sections -firecrawl download https://docs.example.com --include-paths "/features,/sdks" -y +firecrawl x download https://docs.example.com --include-paths "/features,/sdks" -y # Skip translations -firecrawl download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR" -y +firecrawl x download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR" -y # Full combo -firecrawl download https://docs.example.com \ +firecrawl x download https://docs.example.com \ --include-paths "/features,/sdks" \ --exclude-paths "/zh,/ja" \ --only-main-content \ @@ -62,7 +62,7 @@ firecrawl download https://docs.example.com \ Only the options listed below are supported: -`-f `, `-H`, `-S`, `--screenshot`, `--full-page-screenshot`, `--only-main-content`, `--include-tags`, `--exclude-tags`, `--wait-for`, `--max-age`, `--country`, `--languages` +`-f `, `-H`, `-S`, `--lockdown`, `--screenshot`, `--full-page-screenshot`, `--only-main-content`, `--include-tags`, `--exclude-tags`, `--wait-for`, `--max-age`, `--country`, `--languages` ## See also diff --git a/skills/firecrawl-interact/SKILL.md b/skills/firecrawl-interact/SKILL.md index 446493ca68..82822891dd 100644 --- a/skills/firecrawl-interact/SKILL.md +++ b/skills/firecrawl-interact/SKILL.md @@ -43,14 +43,14 @@ firecrawl interact stop ## Options -| Option | Description | -| --------------------- | ------------------------------------------------- | -| `--prompt ` | Natural language instruction (use this OR --code) | -| `--code ` | Code to execute in the browser session | -| `--language ` | Language for code: bash, python, node | -| `--timeout ` | Execution timeout (default: 30, max: 300) | -| `--scrape-id ` | Target a specific scrape (default: last scrape) | -| `-o, --output ` | Output file path | +| Option | Description | +| -------------------------------- | ------------------------------------------------- | +| `--prompt ` | Natural language instruction (use this OR --code) | +| `--code ` | Code to execute in the browser session | +| `--node` / `--python` / `--bash` | Language for `--code` (default: node) | +| `--timeout ` | Execution timeout (default: 30, max: 300) | +| `--scrape-id ` | Target a specific scrape (default: last scrape) | +| `-o, --output ` | Output file path | ## Profiles diff --git a/skills/firecrawl-scrape/SKILL.md b/skills/firecrawl-scrape/SKILL.md index 305ff5fe5b..5d78a39ea2 100644 --- a/skills/firecrawl-scrape/SKILL.md +++ b/skills/firecrawl-scrape/SKILL.md @@ -45,7 +45,7 @@ firecrawl scrape "https://example.com/pricing" --query "What is the enterprise p | ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | | `-f, --format ` | Output formats: markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding | | `-Q, --query ` | Ask a question about the page content (5 credits) | -| `-H` | Include HTTP headers in output | +| `-H` | Output raw HTML (shortcut for `--format html`) | | `--only-main-content` | Strip nav, footer, sidebar — main content only | | `--wait-for ` | Wait for JS rendering before scraping | | `--include-tags ` | Only include these HTML tags | @@ -57,7 +57,7 @@ firecrawl scrape "https://example.com/pricing" --query "What is the enterprise p - **Prefer plain scrape over `--query`.** Scrape to a file, then use `grep`, `head`, or read the markdown directly — you can search and reason over the full content yourself. Use `--query` only when you want a single targeted answer without saving the page (costs 5 extra credits). - **Try scrape before interact.** Scrape handles static pages and JS-rendered SPAs. Only escalate to `interact` when you need interaction (clicks, form fills, pagination). -- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. This mode only saves markdown; if markdown isn't requested, the whole response is JSON-stringified into the `.md` file (other formats are otherwise dropped), and `-o` is ignored. +- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. This mode saves markdown only and ignores `-o`; other requested formats are dropped. If markdown wasn't requested, the whole JSON response is written into the `.md` file. - Single format outputs raw content. Multiple formats (e.g., `--format markdown,links`) output JSON. - Always quote URLs — shell interprets `?` and `&` as special characters. - Naming convention: `.firecrawl/{site}-{path}.md` diff --git a/skills/firecrawl-search/SKILL.md b/skills/firecrawl-search/SKILL.md index 770aa1b9b3..7c40364dea 100644 --- a/skills/firecrawl-search/SKILL.md +++ b/skills/firecrawl-search/SKILL.md @@ -135,13 +135,13 @@ Search costs 2 credits. After you've actually used the results (or decided they - **Idempotent:** re-submitting for the same search id returns success but no extra refund. - **`--silent &`** is the right pattern — exit code 0 even on failure, so a rejected/expired call never crashes your pipeline. -Verify the search returned results before reading its `id`. Zero-result searches do not write the output file, so never send feedback from a missing or stale file. +Verify the search returned results before reading its `id`. Zero-result searches write no output file, so the file may be missing — or left over from an earlier search. Only send feedback when this guard succeeds: ```bash SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json) ``` -Then send feedback. Pick the rating that matches what actually happened: +If the guard fails (non-zero exit: missing file or zero results), skip feedback. Otherwise pick the rating that matches what actually happened: ```bash # Results were useful, with notes on what was still missing