Skip to content

fix(tools): tool descriptions must describe, not direct the model - #116

Open
zenrows-dev-pool[bot] wants to merge 3 commits into
mainfrom
claude/mcp-tool-descriptions-review-77rtgt
Open

zenrows-dev-pool[bot] wants to merge 3 commits into
mainfrom
claude/mcp-tool-descriptions-review-77rtgt

Conversation

@zenrows-dev-pool

@zenrows-dev-pool zenrows-dev-pool Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Rewrites the server instructions text and the main tool descriptions. Tool names, parameters and behaviour are unchanged except where noted below.

Why

Directory reviewers read these strings. Anthropic's policy asks each tool to say what it does and when it fits, but its submission form rules out text that instructs the model ("call X, then Y", "poll every 30 seconds", "do not ..."). OpenAI's guidelines ask for completeness and visible side effects. The text being replaced reads as a sequence of orders, and leaves out costs the caller is charged.

Every description now gives what the tool does and returns, what it costs, its side effects, what each error means, and how it relates to the other tools as a fact about data ("the job_id is what batch_wait takes") rather than as an instruction to call them.

Scope

src/server.ts instructions, scrape (+ js_render, premium_proxy, wait_for, response_type), extract (+ fallback_autoparse), prepare_domain, prepared_domain_status, batch_create, batch_wait, batch_results, batch_content, crawl_create, crawl_wait, crawl_results, browser_navigate, browser_close. The rest of the tools follow separately.

account_usage and batch_status are unchanged: both already read as descriptions rather than instructions.

Costs and side effects that were missing

  • browser_navigate carried no cost at all. It now states 5 credits per minute, 25,000 credits per GB, and the two idle timeouts (3 minutes without a call, 15 minutes after opening).
  • extract now carries the rate card, that Extract adds no charge of its own, and that neither a REQS007 answer nor a processed page with nothing on it is billed.
  • prepare_domain's monthly slot is labelled a side effect.
  • crawl_create keeps CRAWL_NOT_ENABLED, CRAWL_TOO_MANY_CRAWLS, the limit of 3 active jobs shared with Batch, and retry_after.
  • crawl_create documents the webhook parameter, which posts signed events to an external URL.

Two things worth a second look

scrape no longer contradicts its own parameter. The tool description drops PDF, because a PDF is decoded as text here (src/server.ts returns an image only for an image content type, everything else goes through TextDecoder), so the result is not a usable PDF. response_type still accepts pdf, and now says what it actually produces. Removing the value from the enum would be the cleaner fix, but it changes the input contract for anyone sending it today, so it is left as its own decision rather than folded into a description change.

crawl_create no longer tells the model to check its own patterns against the start page. That sentence is an instruction, which is what this branch removes everywhere else, and its test assertion went with it. crawl_get still carries the same sentence and is not in this batch, so the behaviour is still documented somewhere while the remaining tools are done.

Verification

npm test             383/383 pass
npm run typecheck    clean
npm run lint         clean
npm run check:server-json  OK @ 2.19.0

Two test files move with the wording they pin: tests/descriptions.test.ts (the prepare_domain guidance, now stated as a fact) and tests/crawl-tool.test.ts (the dropped crawl_create instruction). Both failed against the new text before they were updated.

The descriptions were also dumped over a real in-memory MCP client (tools/list plus the initialize instructions) and read end to end, so what reviewers see is what was checked, not just the source strings.

Typed fix rather than docs so release-please cuts a release; the hosted server is the point of the change, and a docs commit would not bump it.

Directory reviewers read these strings. Anthropic's policy asks each tool to
say what it does and when it fits, but its submission form rules out text that
instructs the model ("call X, then Y", "poll every 30 seconds", "do not ...");
OpenAI's guidelines ask for completeness and visible side effects. The current
text reads as a sequence of orders, and leaves out costs a caller is charged.

Every description now states what the tool does and returns, what it costs,
what its side effects are, what each error means, and how it relates to the
other tools as a fact about data ("the job_id is what batch_wait takes")
rather than as an instruction to call them.

Two gaps this closes rather than rewrites: browser_navigate carried no cost at
all, and now gives the per-minute and per-GB rates and the two idle timeouts;
prepare_domain's monthly slot is now labelled a side effect.

Covers the server instructions and the main tools. The step-by-step sequences
removed here already exist in the error responses and prepared_domain_status's
"next" field.
@tedi-z
tedi-z requested review from AnderRV and imoital November 11, 2026 15:23
@tedi-z tedi-z self-assigned this Nov 11, 2026
Review found that the rewrite lost facts its own goal keeps: what a call
costs, what an error means, and how a tool's output feeds the next one.

- extract had no cost statement left. It now gives the rate card, and says
  that Extract adds no charge of its own and that neither REQS007 nor a
  processed page with nothing on it is billed.
- crawl_create regained CRAWL_NOT_ENABLED and CRAWL_TOO_MANY_CRAWLS, the
  limit of 3 active jobs shared with Batch, and retry_after.
- crawl_results says again that content_url is what crawl_content takes.
- crawl_wait says again that a progressToken gets progress notifications,
  which batch_wait already documented.

Three wording decisions from the same review:

- crawl_create no longer tells the model to check its own patterns against
  the start page. That was an instruction, which is what this branch
  removes everywhere else; crawl_get still carries it, for the next batch.
- prepare_domain states the example-URL guidance as a fact rather than a
  recommendation, and its test moves with it.
- scrape no longer contradicts its own parameter: response_type says what
  'pdf' actually produces here, a text decode rather than a PDF file.
  Removing the enum value instead would change the input contract, so that
  stays a separate decision.

Typed fix so a release reaches the hosted server; docs alone would not.
@zenrows-dev-pool zenrows-dev-pool Bot changed the title docs(tools): tool descriptions must describe, not direct the model fix(tools): tool descriptions must describe, not direct the model Nov 11, 2026

@imoital imoital left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is relevant, but I hope it helps.

Comment thread src/tools/extract.ts Outdated
- domain_not_enabled: mode "auto" is not available on this domain; mode "css" (css_extractor) works there.

Pages are fetched in automatic mode by default (the mode_auto parameter): JavaScript rendering and premium proxies are added only when the site needs them, and only the configuration that succeeded is billed. Setting js_render or premium_proxy, or mode_auto to false, turns automatic mode off and bills every request at that fixed configuration.
Extract adds no charge of its own: a request costs the underlying fetch, 1 credit basic, 5 with js_render, 10 with premium_proxy, 25 with both. Neither a REQS007 answer nor a page Extract processed and found nothing on ("noDataExtracted") is billed.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this right? The description this replaces said mode auto is "currently free, billing may apply later", and https://docs.zenrows.com/mcp/overview#pricing still says the same. The Vercel AI SDK page says 1 credit.

Comment thread src/tools/browser.ts
Comment on lines +53 to +54
a session closes after 3 minutes without a browser_* call, or 15 minutes after it opened,
and is billed until it closes.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do these two timeouts come from? The 5 credits per minute and 25,000 per GB match the docs, but I can't find "3 minutes without a browser_* call" or "15 minutes after it opened" in this repo. The docs only describe a session TTL of 3 minutes by default, configurable from 1 to 15: https://docs.zenrows.com/browser-sessions/features/session-ttl

Comment thread src/tools/crawl.ts
For finding the pages behind one start page, such as a listing, a category or a blog index.
Runs in the background: returns crawl_id at once, which the other crawl_* tools take.
With webhook, signed events are posted to an https URL when the crawl starts and when it ends; the account needs a webhook signing key, else CRAWL_INVALID_REQUEST (webhook_signing_key_required).
Each page the crawl opens is billed in credits, and the price per page depends on the site; max_pages caps how many pages it opens (default 100) and max_items caps how many URLs it keeps (default 100). The start page is never a result, but its fetch is billed: a crawl whose include_patterns / exclude_patterns match nothing still costs that one scrape. A crawl whose pages held nothing new to follow ends completed with no results and no error.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs give a number here: 1 to 25 credits per page (https://docs.zenrows.com/crawl/endpoints#pricing). Suggest replacing "the price per page depends on the site" with "1 to 25 credits per page, depending on the site". The max_pages parameter at line 198 still says "each billed as one scrape", so it would be worth aligning that too.

Comment thread src/tools/crawl.ts
Runs in the background: returns crawl_id at once, which the other crawl_* tools take.
With webhook, signed events are posted to an https URL when the crawl starts and when it ends; the account needs a webhook signing key, else CRAWL_INVALID_REQUEST (webhook_signing_key_required).
Each page the crawl opens is billed in credits, and the price per page depends on the site; max_pages caps how many pages it opens (default 100) and max_items caps how many URLs it keeps (default 100). The start page is never a result, but its fetch is billed: a crawl whose include_patterns / exclude_patterns match nothing still costs that one scrape. A crawl whose pages held nothing new to follow ends completed with no results and no error.
CRAWL_NOT_ENABLED: Crawl is not enabled for this account, and a retry gets the same answer. CRAWL_TOO_MANY_CRAWLS: the account is at its limit of 3 active jobs by default, shared with Batch, and retry_after says how many seconds until a slot frees.`,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "at its limit of 3 active jobs by default" reads oddly. Maybe "has reached its limit of active jobs (3 by default), shared with Batch".

Comment thread src/tools/batch.ts
Returns job_id, status and stats at once; the job runs asynchronously. Then: batch_wait until it ends, batch_results to list rows (paged), batch_content to read one task's page. A run that ends failed (out of credits, inactive subscription, key cap) answers ok:false with error.code and server_code; batch_rerun recovers it. Submits of 10,000+ tasks keep writing tasks for a while (ingesting:true).
Errors: BATCH_INVALID_REQUEST (400, server_code and invalid_tasks say what), BATCH_INSUFFICIENT_CREDITS (402), BATCH_KEY_CAP_REACHED (402, this key's cap), BATCH_QUOTA_EXCEEDED (429, job limit), AUTH_INVALID (401; server_code subscription_inactive means fix Billing), BATCH_ACCESS_DENIED (403, do not retry), BACKEND_UNAVAILABLE.
Returns job_id, status and stats at once; the job runs asynchronously. The job_id is what batch_status, batch_wait, batch_results and batch_content take: batch_wait reports when the run ends, batch_results lists its rows (paged) and batch_content returns one task's page. A run that ends failed (out of credits, inactive subscription, key cap) answers ok:false with error.code and server_code; batch_rerun recovers it. Submits of 10,000+ tasks keep writing tasks for a while (ingesting:true).
Errors: BATCH_INVALID_REQUEST (400, server_code and invalid_tasks say what), BATCH_INSUFFICIENT_CREDITS (402), BATCH_KEY_CAP_REACHED (402, this key's cap), BATCH_QUOTA_EXCEEDED (429, job limit), AUTH_INVALID (401; server_code subscription_inactive means the subscription is inactive), BATCH_ACCESS_DENIED (403; a retry gets the same answer), BACKEND_UNAVAILABLE.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "subscription_inactive means the subscription is inactive" repeats itself, and the old "fix Billing" told the reader where the problem is. Maybe "means the subscription needs attention in Billing", which is a fact rather than an instruction.

// Extract maps a page type better from several pages of that type, so agents are asked for
// more than one, without the schema refusing a single URL.
test("prepare_domain recommends 3 or more example URLs of the same page type and still accepts one", async () => {
test("prepare_domain says 3 or more example URLs of the same page type give the best results, and still accepts one", async () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing pins the rule this PR introduces, so directive wording can come back unnoticed. Suggest a test next to the FLAGGED one: a list of the rewritten tools and a regex for directive phrasing ("do not", "poll every", "then call"), asserting no match. The regex needs care: "another batch_wait call" is a noun use and should pass. The list can grow as the remaining tools are done.

The description claimed Extract "adds no charge of its own". It has one:
the gateway charges product.Extract separately from the fetch, one unit per
successful parse, and resolves the rate per subscription. Every plan's rate
is unset today, which is why the claim looked true, and why it would have
gone stale the day a rate is set without anyone editing this text.

Says what the pricing page says instead: free for now, billing may apply
later, and the fetch is what a call costs today.

Also names the two unbilled answers by status, 403 REQS007 and 422
no_data_extracted, and says neither the fetch nor Extract is charged for
them, which the previous wording left open.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants