diff --git a/skills/firecrawl-agent/SKILL.md b/skills/firecrawl-agent/SKILL.md index 18fd81bff5..ff5c029bc9 100644 --- a/skills/firecrawl-agent/SKILL.md +++ b/skills/firecrawl-agent/SKILL.md @@ -30,22 +30,41 @@ firecrawl agent "extract products" --schema '{"type":"object","properties":{"nam firecrawl agent "get feature list" --urls "" --wait -o .firecrawl/features.json ``` +## Job IDs + +Without `--wait`, the command returns a job ID. A UUID positional argument is auto-detected as a status check: + +```bash +# Check once (equivalent to adding --status) +firecrawl agent "" + +# Wait on an existing job, polling every 10 seconds for up to 5 minutes +firecrawl agent "" --wait --poll-interval 10 --timeout 300 + +# Cancel an active job +firecrawl agent "" --cancel +``` + ## Options -| Option | Description | -| ---------------------- | ----------------------------------------- | -| `--urls ` | Starting URLs for the agent | -| `--model ` | Model to use: spark-1-mini or spark-1-pro | -| `--schema ` | JSON schema for structured output | -| `--schema-file ` | Path to JSON schema file | -| `--max-credits ` | Credit limit for this agent run | -| `--wait` | Wait for agent to complete | -| `--pretty` | Pretty print JSON output | -| `-o, --output ` | Output file path | +| Option | Description | +| --------------------------- | --------------------------------------------------- | +| `--urls ` | Starting URLs for the agent | +| `--model ` | Model to use: spark-1-mini or spark-1-pro | +| `--schema ` | JSON schema for structured output | +| `--schema-file ` | Path to JSON schema file | +| `--max-credits ` | Credit limit for this agent run | +| `--status` | Check a job ID's status | +| `--cancel` | Cancel an active job ID | +| `--wait` | Wait for agent to complete | +| `--poll-interval ` | Polling interval while waiting (default: 5 seconds) | +| `--timeout ` | Stop waiting after this duration (default: none) | +| `--pretty` | Pretty print JSON output | +| `-o, --output ` | Output file path | ## Tips -- Always use `--wait` to get results inline. Without it, returns a job ID. +- Use `--wait` for inline results; without it you get a job ID (see [Job IDs](#job-ids)). - Use `--schema` for predictable, structured output — otherwise the agent returns freeform data. - Agent runs consume more credits than simple scrapes. Use `--max-credits` to cap spending. - For simple single-page extraction, prefer `scrape` — it's faster and cheaper. diff --git a/skills/firecrawl-cli/SKILL.md b/skills/firecrawl-cli/SKILL.md index d6ef72e504..5049da84ac 100644 --- a/skills/firecrawl-cli/SKILL.md +++ b/skills/firecrawl-cli/SKILL.md @@ -194,9 +194,7 @@ The `check` response then carries a per-field diff (paths like `plans[0].price`) }, "snapshot": { "json": { - "plans": [ - /* current full extraction */ - ] + "plans": [/* current full extraction */] } } } @@ -269,7 +267,8 @@ jq -r '.data.web[] | "\(.title): \(.url)"' .firecrawl/search.json Search costs 2 credits per call. **After you finish using a search result**, send structured feedback in the background. The first feedback per search id refunds 1 credit and feeds search-quality improvements. ```bash -SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json) +# Guard: zero-result searches write no output file, so verify before reading the id +SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json) firecrawl search-feedback "$SEARCH_ID" \ --rating good \ diff --git a/skills/firecrawl-download/SKILL.md b/skills/firecrawl-download/SKILL.md index d2beeb7c66..70a9cb9ac0 100644 --- a/skills/firecrawl-download/SKILL.md +++ b/skills/firecrawl-download/SKILL.md @@ -7,11 +7,11 @@ allowed-tools: - Bash(npx firecrawl *) --- -# firecrawl download +# firecrawl download (invoked as `firecrawl x download`) > **Experimental.** Convenience command that combines `map` + `scrape` to save an entire site as local files. -Maps the site first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. All scrape options work with download. Always pass `-y` to skip the confirmation prompt. +Maps the site origin first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. Use `--include-paths` to scope a non-root URL to one section. Always pass `-y` to skip the confirmation prompt. ## When to use @@ -22,24 +22,24 @@ Maps the site first to discover pages, then scrapes each one into nested directo ## Quick start ```bash -# Interactive wizard (picks format, screenshots, paths for you) -firecrawl download https://docs.example.com +# Interactive wizard (humans at a TTY only — agents must pass -y or the command blocks on a prompt) +firecrawl x download https://docs.example.com # With screenshots -firecrawl download https://docs.example.com --screenshot --limit 20 -y +firecrawl x download https://docs.example.com --screenshot --limit 20 -y # Multiple formats (each saved as its own file per page) -firecrawl download https://docs.example.com --format markdown,links --screenshot --limit 20 -y +firecrawl x download https://docs.example.com --format markdown,links --screenshot --limit 20 -y # Creates per page: index.md + links.txt + screenshot.png # Filter to specific sections -firecrawl download https://docs.example.com --include-paths "/features,/sdks" +firecrawl x download https://docs.example.com --include-paths "/features,/sdks" -y # Skip translations -firecrawl download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR" +firecrawl x download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR" -y # Full combo -firecrawl download https://docs.example.com \ +firecrawl x download https://docs.example.com \ --include-paths "/features,/sdks" \ --exclude-paths "/zh,/ja" \ --only-main-content \ @@ -58,9 +58,11 @@ firecrawl download https://docs.example.com \ | `--allow-subdomains` | Include subdomain pages | | `-y` | Skip confirmation prompt (always use in automated flows) | -## Scrape options (all work with download) +## Supported scrape options -`-f `, `-H`, `-S`, `--screenshot`, `--full-page-screenshot`, `--only-main-content`, `--include-tags`, `--exclude-tags`, `--wait-for`, `--max-age`, `--country`, `--languages` +Only the options listed below are supported: + +`-f `, `-H`, `-S`, `--lockdown`, `--screenshot`, `--full-page-screenshot`, `--only-main-content`, `--include-tags`, `--exclude-tags`, `--wait-for`, `--max-age`, `--country`, `--languages` ## See also diff --git a/skills/firecrawl-interact/SKILL.md b/skills/firecrawl-interact/SKILL.md index 7468dd2083..82822891dd 100644 --- a/skills/firecrawl-interact/SKILL.md +++ b/skills/firecrawl-interact/SKILL.md @@ -25,14 +25,17 @@ Interact with scraped pages in a live browser session. Scrape a page first, then # 1. Scrape a page (scrape ID is saved automatically) firecrawl scrape "" -# 2. Interact with the page using natural language -firecrawl interact --prompt "Click the login button" -firecrawl interact --prompt "Fill in the email field with test@example.com" -firecrawl interact --prompt "Extract the pricing table" +# 2. Interact with the page using a positional prompt +firecrawl interact "Click the login button" +firecrawl interact "Fill in the email field with test@example.com" +firecrawl interact "Extract the pricing table" + +# A UUID first argument is auto-detected as the scrape ID +firecrawl interact "" "Extract the pricing table" # 3. Or use code for precise control -firecrawl interact --code "agent-browser click @e5" --language bash -firecrawl interact --code "agent-browser snapshot -i" --language bash +firecrawl interact --code "click @e5" --bash +firecrawl interact --code "snapshot -i" --bash # 4. Stop the session when done firecrawl interact stop @@ -40,14 +43,14 @@ firecrawl interact stop ## Options -| Option | Description | -| --------------------- | ------------------------------------------------- | -| `--prompt ` | Natural language instruction (use this OR --code) | -| `--code ` | Code to execute in the browser session | -| `--language ` | Language for code: bash, python, node | -| `--timeout ` | Execution timeout (default: 30, max: 300) | -| `--scrape-id ` | Target a specific scrape (default: last scrape) | -| `-o, --output ` | Output file path | +| Option | Description | +| -------------------------------- | ------------------------------------------------- | +| `--prompt ` | Natural language instruction (use this OR --code) | +| `--code ` | Code to execute in the browser session | +| `--node` / `--python` / `--bash` | Language for `--code` (default: node) | +| `--timeout ` | Execution timeout (default: 30, max: 300) | +| `--scrape-id ` | Target a specific scrape (default: last scrape) | +| `-o, --output ` | Output file path | ## Profiles @@ -72,7 +75,7 @@ firecrawl scrape "https://app.example.com" --profile my-app --no-save-changes ## Tips - Always scrape first — `interact` requires a scrape ID from a previous `firecrawl scrape` call -- The scrape ID is saved automatically, so you don't need `--scrape-id` for subsequent interact calls +- The scrape ID is saved automatically, so you don't need `--scrape-id` for subsequent interact calls. Saved sessions may expire after about 10 minutes; re-scrape if the CLI warns that the session is stale - Use `firecrawl interact stop` to free resources when done - For parallel work, scrape multiple pages and interact with each using `--scrape-id` diff --git a/skills/firecrawl-parse/SKILL.md b/skills/firecrawl-parse/SKILL.md index f350cb2a98..cfae90cef7 100644 --- a/skills/firecrawl-parse/SKILL.md +++ b/skills/firecrawl-parse/SKILL.md @@ -9,7 +9,7 @@ allowed-tools: # firecrawl parse -Turn a local document into clean markdown on disk. Supports **PDF, DOCX, DOC, ODT, RTF, XLSX, XLS, HTML/HTM/XHTML**. +Turn a local document into clean markdown on disk. Supports **PDF, DOCX, DOC, ODT, RTF, XLSX, XLS, HTML/HTM**. ## When to use @@ -39,14 +39,14 @@ Then `head`, `grep`, `rg` etc., or incrementally read the file - don't load the ## Options -| Option | Description | -| ---------------------- | --------------------------------------- | -| `-S, --summary` | AI-generated summary | -| `-Q, --query ` | Ask a question about the parsed content | -| `-o, --output ` | Output file path — **always use this** | -| `-f, --format ` | `markdown` (default), `html`, `summary` | -| `--timeout ` | Timeout for the parse job | -| `--timing` | Show request duration | +| Option | Description | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | +| `-S, --summary` | AI-generated summary | +| `-Q, --query ` | Ask a question about the parsed content | +| `-o, --output ` | Output file path — **always use this** | +| `-f, --format ` | Comma-separated: `markdown`, `html`, `rawHtml`, `links`, `images`, `summary`, `json`, `attributes`. Multiple formats output JSON | +| `--timeout ` | Timeout for the parse job | +| `--timing` | Show request duration | ## Tips diff --git a/skills/firecrawl-scrape/SKILL.md b/skills/firecrawl-scrape/SKILL.md index 6c6612756c..5d78a39ea2 100644 --- a/skills/firecrawl-scrape/SKILL.md +++ b/skills/firecrawl-scrape/SKILL.md @@ -29,7 +29,7 @@ firecrawl scrape "" --only-main-content -o .firecrawl/page.md # Wait for JS to render, then scrape firecrawl scrape "" --wait-for 3000 -o .firecrawl/page.md -# Multiple URLs (each saved to .firecrawl/) +# Multiple URLs (markdown only; each saved to .firecrawl/; -o is ignored) firecrawl scrape https://example.com https://example.com/blog https://example.com/docs # Get markdown and links together @@ -41,23 +41,23 @@ firecrawl scrape "https://example.com/pricing" --query "What is the enterprise p ## Options -| Option | Description | -| ------------------------ | ---------------------------------------------------------------- | -| `-f, --format ` | Output formats: markdown, html, rawHtml, links, screenshot, json | -| `-Q, --query ` | Ask a question about the page content (5 credits) | -| `-H` | Include HTTP headers in output | -| `--only-main-content` | Strip nav, footer, sidebar — main content only | -| `--wait-for ` | Wait for JS rendering before scraping | -| `--include-tags ` | Only include these HTML tags | -| `--exclude-tags ` | Exclude these HTML tags | -| `--redact-pii` | Redact personally identifiable information from output | -| `-o, --output ` | Output file path | +| Option | Description | +| ------------------------ | ----------------------------------------------------------------------------------------------------------------------- | +| `-f, --format ` | Output formats: markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding | +| `-Q, --query ` | Ask a question about the page content (5 credits) | +| `-H` | Output raw HTML (shortcut for `--format html`) | +| `--only-main-content` | Strip nav, footer, sidebar — main content only | +| `--wait-for ` | Wait for JS rendering before scraping | +| `--include-tags ` | Only include these HTML tags | +| `--exclude-tags ` | Exclude these HTML tags | +| `--redact-pii` | Redact personally identifiable information from output | +| `-o, --output ` | Output file path | ## Tips - **Prefer plain scrape over `--query`.** Scrape to a file, then use `grep`, `head`, or read the markdown directly — you can search and reason over the full content yourself. Use `--query` only when you want a single targeted answer without saving the page (costs 5 extra credits). - **Try scrape before interact.** Scrape handles static pages and JS-rendered SPAs. Only escalate to `interact` when you need interaction (clicks, form fills, pagination). -- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. +- Multiple URLs are scraped concurrently — check `firecrawl --status` for your concurrency limit. This mode saves markdown only and ignores `-o`; other requested formats are dropped. If markdown wasn't requested, the whole JSON response is written into the `.md` file. - Single format outputs raw content. Multiple formats (e.g., `--format markdown,links`) output JSON. - Always quote URLs — shell interprets `?` and `&` as special characters. - Naming convention: `.firecrawl/{site}-{path}.md` diff --git a/skills/firecrawl-search/SKILL.md b/skills/firecrawl-search/SKILL.md index 00c110ed24..7c40364dea 100644 --- a/skills/firecrawl-search/SKILL.md +++ b/skills/firecrawl-search/SKILL.md @@ -135,13 +135,13 @@ Search costs 2 credits. After you've actually used the results (or decided they - **Idempotent:** re-submitting for the same search id returns success but no extra refund. - **`--silent &`** is the right pattern — exit code 0 even on failure, so a rejected/expired call never crashes your pipeline. -Read the search response's `id`: +Verify the search returned results before reading its `id`. Zero-result searches write no output file, so the file may be missing — or left over from an earlier search. Only send feedback when this guard succeeds: ```bash -SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json) +SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json) ``` -Then send feedback. Pick the rating that matches what actually happened: +If the guard fails (non-zero exit: missing file or zero results), skip feedback. Otherwise pick the rating that matches what actually happened: ```bash # Results were useful, with notes on what was still missing