From 1a1e722b8357cb1bfe6987686c6bf3bac7e6ee40 Mon Sep 17 00:00:00 2001 From: Chenxin Yan Date: Mon, 17 Aug 2026 16:14:21 -0400 Subject: [PATCH] docs(skills): clarify auth and API-side behavior --- skills/firecrawl-cli/SKILL.md | 4 +++- skills/firecrawl-crawl/SKILL.md | 8 +++++--- skills/firecrawl-download/SKILL.md | 2 ++ skills/firecrawl-interact/SKILL.md | 2 +- skills/firecrawl-map/SKILL.md | 4 +++- skills/firecrawl-monitor/SKILL.md | 9 ++++++--- skills/firecrawl-parse/SKILL.md | 2 +- skills/firecrawl-scrape/SKILL.md | 2 +- skills/firecrawl-search/SKILL.md | 2 +- 9 files changed, 23 insertions(+), 12 deletions(-) diff --git a/skills/firecrawl-cli/SKILL.md b/skills/firecrawl-cli/SKILL.md index d6ef72e504..95076a7e5b 100644 --- a/skills/firecrawl-cli/SKILL.md +++ b/skills/firecrawl-cli/SKILL.md @@ -135,7 +135,7 @@ firecrawl monitor update --state paused firecrawl monitor delete ``` -Schedules accept cron (`--cron "*/5 * * * *"`) or natural language (`--schedule "every 5 minutes"`). Minimum interval is 5 minutes. Targets are `--page ` for one page, `--scrape-urls a,b,c` for multiple scrape URLs, or `--crawl-url ` for a whole-site crawl each check. Use `--goal` for flag-based monitor creation, or include `"goal": "..."` in JSON payloads. Note: `--state` (not `--status`) sets active/paused; `--page-status` (not `--status`) filters page results on `check` — avoids collision with the global `--status` flag. Monitoring is not available for zero-data-retention teams. +Schedules accept cron (`--cron "*/5 * * * *"`) or natural language (`--schedule "every 5 minutes"`). Minimum interval is 5 minutes. Targets are `--page ` for one page, `--scrape-urls a,b,c` for multiple scrape URLs, `--crawl-url ` for a whole-site crawl each check, or `--queries ` plus required `--goal` for web-search monitors; tune web search with `--search-window` and `--max-results`. Use `--goal` for flag-based monitor creation, or include `"goal": "..."` in JSON payloads. Note: `--state` (not `--status`) sets active/paused; `--page-status` (not `--status`) filters page results on `check` — avoids collision with the global `--status` flag. Monitoring is not available for zero-data-retention teams. **JSON-mode change tracking:** By default monitors diff each page's markdown and you get a unified text diff back. When you care about **specific structured fields** (price, headline, in-stock flag, items in a list) instead of the whole page, add a `changeTracking` format with `modes: ["json"]` and a JSON schema to the target's `scrapeOptions.formats`. The flag-based form doesn't cover this — pass a JSON body via file or stdin: @@ -317,6 +317,8 @@ For interact, scrape multiple pages and interact with each independently using t ## Credit Usage +Requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login. + ```bash firecrawl credit-usage firecrawl credit-usage --json --pretty -o .firecrawl/credits.json diff --git a/skills/firecrawl-crawl/SKILL.md b/skills/firecrawl-crawl/SKILL.md index ca6e6b5aaf..46d8013b08 100644 --- a/skills/firecrawl-crawl/SKILL.md +++ b/skills/firecrawl-crawl/SKILL.md @@ -11,11 +11,13 @@ allowed-tools: Bulk extract content from a website. Crawls pages following links up to a depth/limit. +**Prerequisite:** `crawl` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login. + ## When to use - You need content from many pages on a site (e.g., all `/docs/`) - You want to extract an entire site section -- Step 4 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → **crawl** → interact +- Step 4 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → **crawl** → monitor → interact ## Quick start @@ -47,9 +49,9 @@ firecrawl crawl ## Tips -- Always use `--wait` when you need the results immediately. Without it, crawl returns a job ID for async polling. +- Always use `--wait` when you need the results immediately. It has no default timeout; use `--timeout ` to bound polling. Without `--wait`, crawl returns a job ID for async polling. - Use `--include-paths` to scope the crawl — don't crawl an entire site when you only need one section. -- Crawl consumes credits per page. Check `firecrawl credit-usage` before large crawls. +- Crawl consumes credits per page. Check `firecrawl credit-usage` before large crawls (`credit-usage` requires authentication). ## See also diff --git a/skills/firecrawl-download/SKILL.md b/skills/firecrawl-download/SKILL.md index d2beeb7c66..cd7966df31 100644 --- a/skills/firecrawl-download/SKILL.md +++ b/skills/firecrawl-download/SKILL.md @@ -11,6 +11,8 @@ allowed-tools: > **Experimental.** Convenience command that combines `map` + `scrape` to save an entire site as local files. +**Prerequisite:** `download` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login. + Maps the site first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. All scrape options work with download. Always pass `-y` to skip the confirmation prompt. ## When to use diff --git a/skills/firecrawl-interact/SKILL.md b/skills/firecrawl-interact/SKILL.md index 7468dd2083..631b091f46 100644 --- a/skills/firecrawl-interact/SKILL.md +++ b/skills/firecrawl-interact/SKILL.md @@ -16,7 +16,7 @@ Interact with scraped pages in a live browser session. Scrape a page first, then - Content requires interaction: clicks, form fills, pagination, login - `scrape` failed because content is behind JavaScript interaction - You need to navigate a multi-step flow -- Last resort in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → crawl → **interact** +- Last resort in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → crawl → monitor → **interact** - **Never use interact for web searches** — use `search` instead ## Quick start diff --git a/skills/firecrawl-map/SKILL.md b/skills/firecrawl-map/SKILL.md index 77eaacf591..2d9f54caa1 100644 --- a/skills/firecrawl-map/SKILL.md +++ b/skills/firecrawl-map/SKILL.md @@ -11,11 +11,13 @@ allowed-tools: Discover URLs on a site. Use `--search` to find a specific page within a large site. +**Prerequisite:** `map` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login. + ## When to use - You need to find a specific subpage on a large site - You want a list of all URLs on a site before scraping or crawling -- Step 3 in the [workflow escalation pattern](firecrawl-cli): search → scrape → **map** → crawl → interact +- Step 3 in the [workflow escalation pattern](firecrawl-cli): search → scrape → **map** + scrape → crawl → monitor → interact ## Quick start diff --git a/skills/firecrawl-monitor/SKILL.md b/skills/firecrawl-monitor/SKILL.md index dc23d1d551..a2ee2a4990 100644 --- a/skills/firecrawl-monitor/SKILL.md +++ b/skills/firecrawl-monitor/SKILL.md @@ -31,7 +31,7 @@ The first three watch URLs you already have. **Web search** is the odd one out - **Monitoring the web** for _new_ results rather than changes to a known page — new launches, funding rounds, papers, news, releases, or brand mentions surfaced by search across the whole web (a **web monitor**: `--queries` + `--goal`) - "Alert me when...", "notify me when...", "email me if...", "send a webhook when...", "ping me if X changes", "track this page", "monitor the web for...", "watch for new..." - Anywhere the user would otherwise wire up cron + a scraper + a diff library + SMTP themselves -- Step 5 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → crawl → **monitor** → interact +- Step 5 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → crawl → **monitor** → interact **Bias toward `monitor`** whenever the request implies notifications or recurrence. A single page read once = `scrape`. A single page where the user wants to be told when it changes = `monitor --page --goal "..." --email|--webhook-url ...`. @@ -114,10 +114,13 @@ Subcommands: `create | list | get | update | delete | run | checks | check`. | `--retention-days ` | Snapshot retention window | | `--state ` | `active` or `paused` (update only — use `--state`, not `--status`) | | `--page-status ` | Filter `check` results: `same`, `new`, `changed`, `removed`, `error` | +| `--limit ` | Max results (`list`, `checks`) or page results (`check`) | +| `--offset ` | Result offset (`list`, `checks`) | +| `--skip ` | Page-result offset (`check`) | | `-o, --output ` | Output file path | | `--pretty` | Pretty-print JSON output | -Minimum schedule interval is **15 minutes**. Monitoring is **not available for zero-data-retention teams**. +Minimum schedule interval is **5 minutes**. Monitoring is **not available for zero-data-retention teams**. ## Web monitors (monitor the web) @@ -148,7 +151,7 @@ For a web monitor, **queries control recall** (what the search retrieves) and ** - One query per **distinct** subject. Several facets of one subject = one query; only split for genuinely separate entities (e.g. "OpenAI, Anthropic, and Google"). - No `site:` operators in queries — use `--include-domains` / `--exclude-domains`. -**What good looks like:** a healthy web monitor mostly returns `new: 0` and alerts only on genuinely new, on-goal results. If most results come back `ignored`, the queries pull noise the goal rejects — tighten the queries. If a topic returns nothing for long stretches, the queries are too narrow or `--search-window` too tight — broaden them. If the user dismisses alerts, the goal is too broad — add an intent-specific `Ignore ...`. The aim is high precision with enough recall: every alert worth acting on, nothing real missed. +**What good looks like:** a healthy web monitor mostly returns `new: 0` and alerts only on genuinely new, on-goal results. If many retrieved results are off-goal, the queries pull noise the goal rejects — tighten the queries. If a topic returns nothing for long stretches, the queries are too narrow or `--search-window` too tight — broaden them. If the user dismisses alerts, the goal is too broad — add an intent-specific `Ignore ...`. The aim is high precision with enough recall: every alert worth acting on, nothing real missed. ## Writing a good `--goal` diff --git a/skills/firecrawl-parse/SKILL.md b/skills/firecrawl-parse/SKILL.md index f350cb2a98..a64eab7d95 100644 --- a/skills/firecrawl-parse/SKILL.md +++ b/skills/firecrawl-parse/SKILL.md @@ -54,7 +54,7 @@ Then `head`, `grep`, `rg` etc., or incrementally read the file - don't load the - Max upload size: **50 MB** per file. - Credits: ~1 per PDF page; HTML is 1 flat. - Check `.firecrawl/` before re-parsing the same file. -- To check your credit balance (recommended for batch processing and similar workflows), use the `firecrawl credit-usage` command. +- To check your credit balance (recommended for batch processing and similar workflows), use `firecrawl credit-usage` (requires authentication). ## See also diff --git a/skills/firecrawl-scrape/SKILL.md b/skills/firecrawl-scrape/SKILL.md index 6c6612756c..0540200da3 100644 --- a/skills/firecrawl-scrape/SKILL.md +++ b/skills/firecrawl-scrape/SKILL.md @@ -15,7 +15,7 @@ Scrape one or more URLs. Returns clean, LLM-optimized markdown. Multiple URLs ar - You have a specific URL and want its content - The page is static or JS-rendered (SPA) -- Step 2 in the [workflow escalation pattern](firecrawl-cli): search → **scrape** → map → crawl → interact +- Step 2 in the [workflow escalation pattern](firecrawl-cli): search → **scrape** → map + scrape → crawl → monitor → interact ## Quick start diff --git a/skills/firecrawl-search/SKILL.md b/skills/firecrawl-search/SKILL.md index 00c110ed24..6bcb542787 100644 --- a/skills/firecrawl-search/SKILL.md +++ b/skills/firecrawl-search/SKILL.md @@ -16,7 +16,7 @@ Web search with optional content scraping. Returns search results as JSON, optio - You don't have a specific URL yet - You need to find pages, answer questions, or discover sources - You need research papers — see [Paper search](#paper-search), which routes to `firecrawl research`, not to `search --categories research` -- First step in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → crawl → interact +- First step in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → crawl → monitor → interact ## Quick start