Skip to content

docs(skills): fix behavior claims that don't match the CLI - #191

Open
chenxin-yan wants to merge 3 commits into
firecrawl:mainfrom
chenxin-yan:skills/fix-behavior-claims
Open

docs(skills): fix behavior claims that don't match the CLI#191
chenxin-yan wants to merge 3 commits into
firecrawl:mainfrom
chenxin-yan:skills/fix-behavior-claims

Conversation

@chenxin-yan

Copy link
Copy Markdown

Docs-only. A skill-vs-src/ consistency audit found SKILL.md claims that don't match what the CLI actually does. Since skills are agent-consumed, each wrong claim turns into a failing command or a silent wrong result. Every fix below was verified against src/.

Wrong flags / commands

  • scrape: -H documented as "include HTTP headers" — it's actually --html (raw HTML shortcut)
  • interact: table documented a nonexistent --language <lang>; the real flags are --node/--python/--bash. Examples also used --prompt where the CLI takes a positional prompt
  • download: examples invoked firecrawl download, which doesn't exist — the command is only registered as firecrawl x download

Wrong or incomplete behavior claims

  • download: removed the false "All scrape options work with download" (only a subset exists); added the missing --lockdown to that list; noted mapping starts from the site origin; added -y to examples that otherwise block on a confirm prompt (agents have no TTY)
  • parse: removed unsupported XHTML; corrected -f/--format (comma-separated, 8 values, multi-format → JSON)
  • scrape: completed the --format list (11 values); documented that multi-URL mode saves markdown only and ignores -o
  • search: zero-result searches write no output file — added a guarded jq -er recipe before the feedback flow (mirrored in the cli skill so the two don't drift)
  • interact: documented UUID auto-detection and the ~10-minute scrape-session staleness warning
  • agent: documented the job-ID flow (--status, --cancel, --poll-interval default 5s, --timeout default none)

Each claim was independently reviewed against src/ (file/line evidence in the commit messages).

Adversarial review against src/ surfaced wrong claims that survived the
first audit pass, in the same files this PR touches:

- scrape: -H is --html (raw HTML shortcut), not HTTP headers
- interact: --language does not exist; the CLI uses --node/--python/--bash
- download: the command is only registered as 'firecrawl x download'
  (experimental), so all examples used a nonexistent invocation; the
  'only these options' list omitted --lockdown; the wizard example is
  TTY-only and blocks agents without -y
- search: spell out guard-failure behavior and split the missing-file
  vs stale-file hazards
- scrape/agent: reword confusing multi-URL parenthetical and dedupe the
  --wait tip
- cli: mirror the guarded jq search-id recipe so it doesn't drift from
  the corrected search skill

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 7 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="skills/firecrawl-search/SKILL.md">

<violation number="1" location="skills/firecrawl-search/SKILL.md:141">
P3: The guard only detects a missing/empty result file; it cannot detect a stale file left over from an earlier search. The same fixed path (`search-react-hooks.json`) is reused, so a zero-result search that writes nothing leaves the previous successful file, whose `data` arrays are non-empty and pass `any(.data[]; length > 0)`. The agent would then send feedback with the old id, contradicting the guard's stated protection ("may be left over from an earlier search. Only send feedback when this guard succeeds"). Consider using a per-query path (the doc's own naming convention is `search-{query}.json`) or noting explicitly that the guard does not detect staleness.</violation>
</file>

<file name="skills/firecrawl-parse/SKILL.md">

<violation number="1" location="skills/firecrawl-parse/SKILL.md:47">
P3: The `-f/--format` list is incomplete: it omits `screenshot`, `changeTracking`, and `branding`. Parse's `--format` is parsed by the shared `parseFormats` in `src/utils/options.ts`, which accepts all 11 `VALID_FORMATS` (markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding) and rejects nothing at parse time. The doc reads as an exhaustive list, so an agent will believe those three values are invalid and the accuracy goal of this PR is not fully met. Either add the missing values or state that only these 8 are meaningful for parse.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic


```bash
SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json)
SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json)

@cubic-dev-ai cubic-dev-ai Bot Aug 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The guard only detects a missing/empty result file; it cannot detect a stale file left over from an earlier search. The same fixed path (search-react-hooks.json) is reused, so a zero-result search that writes nothing leaves the previous successful file, whose data arrays are non-empty and pass any(.data[]; length > 0). The agent would then send feedback with the old id, contradicting the guard's stated protection ("may be left over from an earlier search. Only send feedback when this guard succeeds"). Consider using a per-query path (the doc's own naming convention is search-{query}.json) or noting explicitly that the guard does not detect staleness.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At skills/firecrawl-search/SKILL.md, line 141:

<comment>The guard only detects a missing/empty result file; it cannot detect a stale file left over from an earlier search. The same fixed path (`search-react-hooks.json`) is reused, so a zero-result search that writes nothing leaves the previous successful file, whose `data` arrays are non-empty and pass `any(.data[]; length > 0)`. The agent would then send feedback with the old id, contradicting the guard's stated protection ("may be left over from an earlier search. Only send feedback when this guard succeeds"). Consider using a per-query path (the doc's own naming convention is `search-{query}.json`) or noting explicitly that the guard does not detect staleness.</comment>

<file context>
@@ -135,13 +135,13 @@ Search costs 2 credits. After you've actually used the results (or decided they
 
 ```bash
-SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json)
+SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json)

</file context>


</details>

<a href="https://www.cubic.dev/action/fix/violation/f9db8d38-4b32-4628-9050-5e0141f165ae" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://cubic.dev/buttons/fix-with-cubic-dark.svg">
    <source media="(prefers-color-scheme: light)" srcset="https://cubic.dev/buttons/fix-with-cubic-light.svg">
    <img alt="Fix with cubic" src="https://cubic.dev/buttons/fix-with-cubic-dark.svg">
  </picture>
</a>

| `-S, --summary` | AI-generated summary |
| `-Q, --query <prompt>` | Ask a question about the parsed content |
| `-o, --output <path>` | Output file path — **always use this** |
| `-f, --format <formats>` | Comma-separated: `markdown`, `html`, `rawHtml`, `links`, `images`, `summary`, `json`, `attributes`. Multiple formats output JSON |

@cubic-dev-ai cubic-dev-ai Bot Aug 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The -f/--format list is incomplete: it omits screenshot, changeTracking, and branding. Parse's --format is parsed by the shared parseFormats in src/utils/options.ts, which accepts all 11 VALID_FORMATS (markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding) and rejects nothing at parse time. The doc reads as an exhaustive list, so an agent will believe those three values are invalid and the accuracy goal of this PR is not fully met. Either add the missing values or state that only these 8 are meaningful for parse.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At skills/firecrawl-parse/SKILL.md, line 47:

<comment>The `-f/--format` list is incomplete: it omits `screenshot`, `changeTracking`, and `branding`. Parse's `--format` is parsed by the shared `parseFormats` in `src/utils/options.ts`, which accepts all 11 `VALID_FORMATS` (markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding) and rejects nothing at parse time. The doc reads as an exhaustive list, so an agent will believe those three values are invalid and the accuracy goal of this PR is not fully met. Either add the missing values or state that only these 8 are meaningful for parse.</comment>

<file context>
@@ -39,14 +39,14 @@ Then `head`, `grep`, `rg` etc., or incrementally read the file - don't load the
+| `-S, --summary`          | AI-generated summary                                                                                                             |
+| `-Q, --query <prompt>`   | Ask a question about the parsed content                                                                                          |
+| `-o, --output <path>`    | Output file path — **always use this**                                                                                           |
+| `-f, --format <formats>` | Comma-separated: `markdown`, `html`, `rawHtml`, `links`, `images`, `summary`, `json`, `attributes`. Multiple formats output JSON |
+| `--timeout <ms>`         | Timeout for the parse job                                                                                                        |
+| `--timing`               | Show request duration                                                                                                            |
</file context>
Fix with cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant