Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion skills/firecrawl-cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ firecrawl monitor update <monitorId> --state paused
firecrawl monitor delete <monitorId>
```

Schedules accept cron (`--cron "*/5 * * * *"`) or natural language (`--schedule "every 5 minutes"`). Minimum interval is 5 minutes. Targets are `--page <url>` for one page, `--scrape-urls a,b,c` for multiple scrape URLs, or `--crawl-url <url>` for a whole-site crawl each check. Use `--goal` for flag-based monitor creation, or include `"goal": "..."` in JSON payloads. Note: `--state` (not `--status`) sets active/paused; `--page-status` (not `--status`) filters page results on `check` — avoids collision with the global `--status` flag. Monitoring is not available for zero-data-retention teams.
Schedules accept cron (`--cron "*/5 * * * *"`) or natural language (`--schedule "every 5 minutes"`). Minimum interval is 5 minutes. Targets are `--page <url>` for one page, `--scrape-urls a,b,c` for multiple scrape URLs, `--crawl-url <url>` for a whole-site crawl each check, or `--queries <q,...>` plus required `--goal` for web-search monitors; tune web search with `--search-window` and `--max-results`. Use `--goal` for flag-based monitor creation, or include `"goal": "..."` in JSON payloads. Note: `--state` (not `--status`) sets active/paused; `--page-status` (not `--status`) filters page results on `check` — avoids collision with the global `--status` flag. Monitoring is not available for zero-data-retention teams.

**JSON-mode change tracking:** By default monitors diff each page's markdown and you get a unified text diff back. When you care about **specific structured fields** (price, headline, in-stock flag, items in a list) instead of the whole page, add a `changeTracking` format with `modes: ["json"]` and a JSON schema to the target's `scrapeOptions.formats`. The flag-based form doesn't cover this — pass a JSON body via file or stdin:

Expand Down Expand Up @@ -317,6 +317,8 @@ For interact, scrape multiple pages and interact with each independently using t

## Credit Usage

Requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

```bash
firecrawl credit-usage
firecrawl credit-usage --json --pretty -o .firecrawl/credits.json
Expand Down
8 changes: 5 additions & 3 deletions skills/firecrawl-crawl/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,13 @@ allowed-tools:

Bulk extract content from a website. Crawls pages following links up to a depth/limit.

**Prerequisite:** `crawl` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

## When to use

- You need content from many pages on a site (e.g., all `/docs/`)
- You want to extract an entire site section
- Step 4 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → **crawl** → interact
- Step 4 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → **crawl** → monitor → interact

## Quick start

Expand Down Expand Up @@ -47,9 +49,9 @@ firecrawl crawl <job-id>

## Tips

- Always use `--wait` when you need the results immediately. Without it, crawl returns a job ID for async polling.
- Always use `--wait` when you need the results immediately. It has no default timeout; use `--timeout <seconds>` to bound polling. Without `--wait`, crawl returns a job ID for async polling.
- Use `--include-paths` to scope the crawl — don't crawl an entire site when you only need one section.
- Crawl consumes credits per page. Check `firecrawl credit-usage` before large crawls.
- Crawl consumes credits per page. Check `firecrawl credit-usage` before large crawls (`credit-usage` requires authentication).

## See also

Expand Down
2 changes: 2 additions & 0 deletions skills/firecrawl-download/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ allowed-tools:

> **Experimental.** Convenience command that combines `map` + `scrape` to save an entire site as local files.

**Prerequisite:** `download` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

Maps the site first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. All scrape options work with download. Always pass `-y` to skip the confirmation prompt.

## When to use
Expand Down
2 changes: 1 addition & 1 deletion skills/firecrawl-interact/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Interact with scraped pages in a live browser session. Scrape a page first, then
- Content requires interaction: clicks, form fills, pagination, login
- `scrape` failed because content is behind JavaScript interaction
- You need to navigate a multi-step flow
- Last resort in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → crawl → **interact**
- Last resort in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → crawl → monitor → **interact**
- **Never use interact for web searches** — use `search` instead

## Quick start
Expand Down
4 changes: 3 additions & 1 deletion skills/firecrawl-map/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,13 @@ allowed-tools:

Discover URLs on a site. Use `--search` to find a specific page within a large site.

**Prerequisite:** `map` requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

## When to use

- You need to find a specific subpage on a large site
- You want a list of all URLs on a site before scraping or crawling
- Step 3 in the [workflow escalation pattern](firecrawl-cli): search → scrape → **map** → crawl → interact
- Step 3 in the [workflow escalation pattern](firecrawl-cli): search → scrape → **map** + scrape → crawl → monitor → interact

## Quick start

Expand Down
9 changes: 6 additions & 3 deletions skills/firecrawl-monitor/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The first three watch URLs you already have. **Web search** is the odd one out
- **Monitoring the web** for _new_ results rather than changes to a known page — new launches, funding rounds, papers, news, releases, or brand mentions surfaced by search across the whole web (a **web monitor**: `--queries` + `--goal`)
- "Alert me when...", "notify me when...", "email me if...", "send a webhook when...", "ping me if X changes", "track this page", "monitor the web for...", "watch for new..."
- Anywhere the user would otherwise wire up cron + a scraper + a diff library + SMTP themselves
- Step 5 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → crawl → **monitor** → interact
- Step 5 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → crawl → **monitor** → interact

**Bias toward `monitor`** whenever the request implies notifications or recurrence. A single page read once = `scrape`. A single page where the user wants to be told when it changes = `monitor --page <url> --goal "..." --email|--webhook-url ...`.

Expand Down Expand Up @@ -114,10 +114,13 @@ Subcommands: `create | list | get | update | delete | run | checks | check`.
| `--retention-days <n>` | Snapshot retention window |
| `--state <state>` | `active` or `paused` (update only — use `--state`, not `--status`) |
| `--page-status <state>` | Filter `check` results: `same`, `new`, `changed`, `removed`, `error` |
| `--limit <n>` | Max results (`list`, `checks`) or page results (`check`) |
| `--offset <n>` | Result offset (`list`, `checks`) |
| `--skip <n>` | Page-result offset (`check`) |
| `-o, --output <path>` | Output file path |
| `--pretty` | Pretty-print JSON output |

Minimum schedule interval is **15 minutes**. Monitoring is **not available for zero-data-retention teams**.
Minimum schedule interval is **5 minutes**. Monitoring is **not available for zero-data-retention teams**.

## Web monitors (monitor the web)

Expand Down Expand Up @@ -148,7 +151,7 @@ For a web monitor, **queries control recall** (what the search retrieves) and **
- One query per **distinct** subject. Several facets of one subject = one query; only split for genuinely separate entities (e.g. "OpenAI, Anthropic, and Google").
- No `site:` operators in queries — use `--include-domains` / `--exclude-domains`.

**What good looks like:** a healthy web monitor mostly returns `new: 0` and alerts only on genuinely new, on-goal results. If most results come back `ignored`, the queries pull noise the goal rejects — tighten the queries. If a topic returns nothing for long stretches, the queries are too narrow or `--search-window` too tight — broaden them. If the user dismisses alerts, the goal is too broad — add an intent-specific `Ignore ...`. The aim is high precision with enough recall: every alert worth acting on, nothing real missed.
**What good looks like:** a healthy web monitor mostly returns `new: 0` and alerts only on genuinely new, on-goal results. If many retrieved results are off-goal, the queries pull noise the goal rejects — tighten the queries. If a topic returns nothing for long stretches, the queries are too narrow or `--search-window` too tight — broaden them. If the user dismisses alerts, the goal is too broad — add an intent-specific `Ignore ...`. The aim is high precision with enough recall: every alert worth acting on, nothing real missed.

## Writing a good `--goal`

Expand Down
2 changes: 1 addition & 1 deletion skills/firecrawl-parse/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Then `head`, `grep`, `rg` etc., or incrementally read the file - don't load the
- Max upload size: **50 MB** per file.
- Credits: ~1 per PDF page; HTML is 1 flat.
- Check `.firecrawl/` before re-parsing the same file.
- To check your credit balance (recommended for batch processing and similar workflows), use the `firecrawl credit-usage` command.
- To check your credit balance (recommended for batch processing and similar workflows), use `firecrawl credit-usage` (requires authentication).

## See also

Expand Down
2 changes: 1 addition & 1 deletion skills/firecrawl-scrape/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Scrape one or more URLs. Returns clean, LLM-optimized markdown. Multiple URLs ar

- You have a specific URL and want its content
- The page is static or JS-rendered (SPA)
- Step 2 in the [workflow escalation pattern](firecrawl-cli): search → **scrape** → map → crawl → interact
- Step 2 in the [workflow escalation pattern](firecrawl-cli): search → **scrape** → map + scrape → crawl → monitor → interact

## Quick start

Expand Down
2 changes: 1 addition & 1 deletion skills/firecrawl-search/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Web search with optional content scraping. Returns search results as JSON, optio
- You don't have a specific URL yet
- You need to find pages, answer questions, or discover sources
- You need research papers — see [Paper search](#paper-search), which routes to `firecrawl research`, not to `search --categories research`
- First step in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → crawl → interact
- First step in the [workflow escalation pattern](firecrawl-cli): search → scrape → map + scrape → crawl → monitor → interact

## Quick start

Expand Down
Loading