docs(skills): fix behavior claims that don't match the CLI - #191
docs(skills): fix behavior claims that don't match the CLI#191chenxin-yan wants to merge 3 commits into
Conversation
Adversarial review against src/ surfaced wrong claims that survived the first audit pass, in the same files this PR touches: - scrape: -H is --html (raw HTML shortcut), not HTTP headers - interact: --language does not exist; the CLI uses --node/--python/--bash - download: the command is only registered as 'firecrawl x download' (experimental), so all examples used a nonexistent invocation; the 'only these options' list omitted --lockdown; the wizard example is TTY-only and blocks agents without -y - search: spell out guard-failure behavior and split the missing-file vs stale-file hazards - scrape/agent: reword confusing multi-URL parenthetical and dedupe the --wait tip - cli: mirror the guarded jq search-id recipe so it doesn't drift from the corrected search skill
There was a problem hiding this comment.
2 issues found across 7 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="skills/firecrawl-search/SKILL.md">
<violation number="1" location="skills/firecrawl-search/SKILL.md:141">
P3: The guard only detects a missing/empty result file; it cannot detect a stale file left over from an earlier search. The same fixed path (`search-react-hooks.json`) is reused, so a zero-result search that writes nothing leaves the previous successful file, whose `data` arrays are non-empty and pass `any(.data[]; length > 0)`. The agent would then send feedback with the old id, contradicting the guard's stated protection ("may be left over from an earlier search. Only send feedback when this guard succeeds"). Consider using a per-query path (the doc's own naming convention is `search-{query}.json`) or noting explicitly that the guard does not detect staleness.</violation>
</file>
<file name="skills/firecrawl-parse/SKILL.md">
<violation number="1" location="skills/firecrawl-parse/SKILL.md:47">
P3: The `-f/--format` list is incomplete: it omits `screenshot`, `changeTracking`, and `branding`. Parse's `--format` is parsed by the shared `parseFormats` in `src/utils/options.ts`, which accepts all 11 `VALID_FORMATS` (markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding) and rejects nothing at parse time. The doc reads as an exhaustive list, so an agent will believe those three values are invalid and the accuracy goal of this PR is not fully met. Either add the missing values or state that only these 8 are meaningful for parse.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
|
|
||
| ```bash | ||
| SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json) | ||
| SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json) |
There was a problem hiding this comment.
P3: The guard only detects a missing/empty result file; it cannot detect a stale file left over from an earlier search. The same fixed path (search-react-hooks.json) is reused, so a zero-result search that writes nothing leaves the previous successful file, whose data arrays are non-empty and pass any(.data[]; length > 0). The agent would then send feedback with the old id, contradicting the guard's stated protection ("may be left over from an earlier search. Only send feedback when this guard succeeds"). Consider using a per-query path (the doc's own naming convention is search-{query}.json) or noting explicitly that the guard does not detect staleness.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At skills/firecrawl-search/SKILL.md, line 141:
<comment>The guard only detects a missing/empty result file; it cannot detect a stale file left over from an earlier search. The same fixed path (`search-react-hooks.json`) is reused, so a zero-result search that writes nothing leaves the previous successful file, whose `data` arrays are non-empty and pass `any(.data[]; length > 0)`. The agent would then send feedback with the old id, contradicting the guard's stated protection ("may be left over from an earlier search. Only send feedback when this guard succeeds"). Consider using a per-query path (the doc's own naming convention is `search-{query}.json`) or noting explicitly that the guard does not detect staleness.</comment>
<file context>
@@ -135,13 +135,13 @@ Search costs 2 credits. After you've actually used the results (or decided they
```bash
-SEARCH_ID=$(jq -r '.id' .firecrawl/search-react-hooks.json)
+SEARCH_ID=$(jq -er 'select(any(.data[]; length > 0)) | .id' .firecrawl/search-react-hooks.json)
</file context>
</details>
<a href="https://www.cubic.dev/action/fix/violation/f9db8d38-4b32-4628-9050-5e0141f165ae" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://cubic.dev/buttons/fix-with-cubic-dark.svg">
<source media="(prefers-color-scheme: light)" srcset="https://cubic.dev/buttons/fix-with-cubic-light.svg">
<img alt="Fix with cubic" src="https://cubic.dev/buttons/fix-with-cubic-dark.svg">
</picture>
</a>
| | `-S, --summary` | AI-generated summary | | ||
| | `-Q, --query <prompt>` | Ask a question about the parsed content | | ||
| | `-o, --output <path>` | Output file path — **always use this** | | ||
| | `-f, --format <formats>` | Comma-separated: `markdown`, `html`, `rawHtml`, `links`, `images`, `summary`, `json`, `attributes`. Multiple formats output JSON | |
There was a problem hiding this comment.
P3: The -f/--format list is incomplete: it omits screenshot, changeTracking, and branding. Parse's --format is parsed by the shared parseFormats in src/utils/options.ts, which accepts all 11 VALID_FORMATS (markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding) and rejects nothing at parse time. The doc reads as an exhaustive list, so an agent will believe those three values are invalid and the accuracy goal of this PR is not fully met. Either add the missing values or state that only these 8 are meaningful for parse.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At skills/firecrawl-parse/SKILL.md, line 47:
<comment>The `-f/--format` list is incomplete: it omits `screenshot`, `changeTracking`, and `branding`. Parse's `--format` is parsed by the shared `parseFormats` in `src/utils/options.ts`, which accepts all 11 `VALID_FORMATS` (markdown, html, rawHtml, links, images, screenshot, summary, changeTracking, json, attributes, branding) and rejects nothing at parse time. The doc reads as an exhaustive list, so an agent will believe those three values are invalid and the accuracy goal of this PR is not fully met. Either add the missing values or state that only these 8 are meaningful for parse.</comment>
<file context>
@@ -39,14 +39,14 @@ Then `head`, `grep`, `rg` etc., or incrementally read the file - don't load the
+| `-S, --summary` | AI-generated summary |
+| `-Q, --query <prompt>` | Ask a question about the parsed content |
+| `-o, --output <path>` | Output file path — **always use this** |
+| `-f, --format <formats>` | Comma-separated: `markdown`, `html`, `rawHtml`, `links`, `images`, `summary`, `json`, `attributes`. Multiple formats output JSON |
+| `--timeout <ms>` | Timeout for the parse job |
+| `--timing` | Show request duration |
</file context>
Docs-only. A skill-vs-
src/consistency audit found SKILL.md claims that don't match what the CLI actually does. Since skills are agent-consumed, each wrong claim turns into a failing command or a silent wrong result. Every fix below was verified againstsrc/.Wrong flags / commands
-Hdocumented as "include HTTP headers" — it's actually--html(raw HTML shortcut)--language <lang>; the real flags are--node/--python/--bash. Examples also used--promptwhere the CLI takes a positional promptfirecrawl download, which doesn't exist — the command is only registered asfirecrawl x downloadWrong or incomplete behavior claims
--lockdownto that list; noted mapping starts from the site origin; added-yto examples that otherwise block on a confirm prompt (agents have no TTY)-f/--format(comma-separated, 8 values, multi-format → JSON)--formatlist (11 values); documented that multi-URL mode saves markdown only and ignores-ojq -errecipe before the feedback flow (mirrored in the cli skill so the two don't drift)--status,--cancel,--poll-intervaldefault 5s,--timeoutdefault none)Each claim was independently reviewed against
src/(file/line evidence in the commit messages).