Repository navigation
fix(tools): tool descriptions must describe, not direct the model - #116
zenrows-dev-pool[bot] wants to merge 3 commits into
Conversation
Directory reviewers read these strings. Anthropic's policy asks each tool to
say what it does and when it fits, but its submission form rules out text that
instructs the model ("call X, then Y", "poll every 30 seconds", "do not ...");
OpenAI's guidelines ask for completeness and visible side effects. The current
text reads as a sequence of orders, and leaves out costs a caller is charged.
Every description now states what the tool does and returns, what it costs,
what its side effects are, what each error means, and how it relates to the
other tools as a fact about data ("the job_id is what batch_wait takes")
rather than as an instruction to call them.
Two gaps this closes rather than rewrites: browser_navigate carried no cost at
all, and now gives the per-minute and per-GB rates and the two idle timeouts;
prepare_domain's monthly slot is now labelled a side effect.
Covers the server instructions and the main tools. The step-by-step sequences
removed here already exist in the error responses and prepared_domain_status's
"next" field.
Review found that the rewrite lost facts its own goal keeps: what a call costs, what an error means, and how a tool's output feeds the next one. - extract had no cost statement left. It now gives the rate card, and says that Extract adds no charge of its own and that neither REQS007 nor a processed page with nothing on it is billed. - crawl_create regained CRAWL_NOT_ENABLED and CRAWL_TOO_MANY_CRAWLS, the limit of 3 active jobs shared with Batch, and retry_after. - crawl_results says again that content_url is what crawl_content takes. - crawl_wait says again that a progressToken gets progress notifications, which batch_wait already documented. Three wording decisions from the same review: - crawl_create no longer tells the model to check its own patterns against the start page. That was an instruction, which is what this branch removes everywhere else; crawl_get still carries it, for the next batch. - prepare_domain states the example-URL guidance as a fact rather than a recommendation, and its test moves with it. - scrape no longer contradicts its own parameter: response_type says what 'pdf' actually produces here, a text decode rather than a PDF file. Removing the enum value instead would change the input contract, so that stays a separate decision. Typed fix so a release reaches the hosted server; docs alone would not.
imoital
left a comment
There was a problem hiding this comment.
I'm not sure if this is relevant, but I hope it helps.
| - domain_not_enabled: mode "auto" is not available on this domain; mode "css" (css_extractor) works there. | ||
|
|
||
| Pages are fetched in automatic mode by default (the mode_auto parameter): JavaScript rendering and premium proxies are added only when the site needs them, and only the configuration that succeeded is billed. Setting js_render or premium_proxy, or mode_auto to false, turns automatic mode off and bills every request at that fixed configuration. | ||
| Extract adds no charge of its own: a request costs the underlying fetch, 1 credit basic, 5 with js_render, 10 with premium_proxy, 25 with both. Neither a REQS007 answer nor a page Extract processed and found nothing on ("noDataExtracted") is billed. |
There was a problem hiding this comment.
Is this right? The description this replaces said mode auto is "currently free, billing may apply later", and https://docs.zenrows.com/mcp/overview#pricing still says the same. The Vercel AI SDK page says 1 credit.
| a session closes after 3 minutes without a browser_* call, or 15 minutes after it opened, | ||
| and is billed until it closes. |
There was a problem hiding this comment.
Where do these two timeouts come from? The 5 credits per minute and 25,000 per GB match the docs, but I can't find "3 minutes without a browser_* call" or "15 minutes after it opened" in this repo. The docs only describe a session TTL of 3 minutes by default, configurable from 1 to 15: https://docs.zenrows.com/browser-sessions/features/session-ttl
| For finding the pages behind one start page, such as a listing, a category or a blog index. | ||
| Runs in the background: returns crawl_id at once, which the other crawl_* tools take. | ||
| With webhook, signed events are posted to an https URL when the crawl starts and when it ends; the account needs a webhook signing key, else CRAWL_INVALID_REQUEST (webhook_signing_key_required). | ||
| Each page the crawl opens is billed in credits, and the price per page depends on the site; max_pages caps how many pages it opens (default 100) and max_items caps how many URLs it keeps (default 100). The start page is never a result, but its fetch is billed: a crawl whose include_patterns / exclude_patterns match nothing still costs that one scrape. A crawl whose pages held nothing new to follow ends completed with no results and no error. |
There was a problem hiding this comment.
The docs give a number here: 1 to 25 credits per page (https://docs.zenrows.com/crawl/endpoints#pricing). Suggest replacing "the price per page depends on the site" with "1 to 25 credits per page, depending on the site". The max_pages parameter at line 198 still says "each billed as one scrape", so it would be worth aligning that too.
| Runs in the background: returns crawl_id at once, which the other crawl_* tools take. | ||
| With webhook, signed events are posted to an https URL when the crawl starts and when it ends; the account needs a webhook signing key, else CRAWL_INVALID_REQUEST (webhook_signing_key_required). | ||
| Each page the crawl opens is billed in credits, and the price per page depends on the site; max_pages caps how many pages it opens (default 100) and max_items caps how many URLs it keeps (default 100). The start page is never a result, but its fetch is billed: a crawl whose include_patterns / exclude_patterns match nothing still costs that one scrape. A crawl whose pages held nothing new to follow ends completed with no results and no error. | ||
| CRAWL_NOT_ENABLED: Crawl is not enabled for this account, and a retry gets the same answer. CRAWL_TOO_MANY_CRAWLS: the account is at its limit of 3 active jobs by default, shared with Batch, and retry_after says how many seconds until a slot frees.`, |
There was a problem hiding this comment.
Nit: "at its limit of 3 active jobs by default" reads oddly. Maybe "has reached its limit of active jobs (3 by default), shared with Batch".
| Returns job_id, status and stats at once; the job runs asynchronously. Then: batch_wait until it ends, batch_results to list rows (paged), batch_content to read one task's page. A run that ends failed (out of credits, inactive subscription, key cap) answers ok:false with error.code and server_code; batch_rerun recovers it. Submits of 10,000+ tasks keep writing tasks for a while (ingesting:true). | ||
| Errors: BATCH_INVALID_REQUEST (400, server_code and invalid_tasks say what), BATCH_INSUFFICIENT_CREDITS (402), BATCH_KEY_CAP_REACHED (402, this key's cap), BATCH_QUOTA_EXCEEDED (429, job limit), AUTH_INVALID (401; server_code subscription_inactive means fix Billing), BATCH_ACCESS_DENIED (403, do not retry), BACKEND_UNAVAILABLE. | ||
| Returns job_id, status and stats at once; the job runs asynchronously. The job_id is what batch_status, batch_wait, batch_results and batch_content take: batch_wait reports when the run ends, batch_results lists its rows (paged) and batch_content returns one task's page. A run that ends failed (out of credits, inactive subscription, key cap) answers ok:false with error.code and server_code; batch_rerun recovers it. Submits of 10,000+ tasks keep writing tasks for a while (ingesting:true). | ||
| Errors: BATCH_INVALID_REQUEST (400, server_code and invalid_tasks say what), BATCH_INSUFFICIENT_CREDITS (402), BATCH_KEY_CAP_REACHED (402, this key's cap), BATCH_QUOTA_EXCEEDED (429, job limit), AUTH_INVALID (401; server_code subscription_inactive means the subscription is inactive), BATCH_ACCESS_DENIED (403; a retry gets the same answer), BACKEND_UNAVAILABLE. |
There was a problem hiding this comment.
Nit: "subscription_inactive means the subscription is inactive" repeats itself, and the old "fix Billing" told the reader where the problem is. Maybe "means the subscription needs attention in Billing", which is a fact rather than an instruction.
| // Extract maps a page type better from several pages of that type, so agents are asked for | ||
| // more than one, without the schema refusing a single URL. | ||
| test("prepare_domain recommends 3 or more example URLs of the same page type and still accepts one", async () => { | ||
| test("prepare_domain says 3 or more example URLs of the same page type give the best results, and still accepts one", async () => { |
There was a problem hiding this comment.
Nothing pins the rule this PR introduces, so directive wording can come back unnoticed. Suggest a test next to the FLAGGED one: a list of the rewritten tools and a regex for directive phrasing ("do not", "poll every", "then call"), asserting no match. The regex needs care: "another batch_wait call" is a noun use and should pass. The list can grow as the remaining tools are done.
The description claimed Extract "adds no charge of its own". It has one: the gateway charges product.Extract separately from the fetch, one unit per successful parse, and resolves the rate per subscription. Every plan's rate is unset today, which is why the claim looked true, and why it would have gone stale the day a rate is set without anyone editing this text. Says what the pricing page says instead: free for now, billing may apply later, and the fetch is what a call costs today. Also names the two unbilled answers by status, 403 REQS007 and 422 no_data_extracted, and says neither the fetch nor Extract is charged for them, which the previous wording left open.
Rewrites the server instructions text and the main tool descriptions. Tool names, parameters and behaviour are unchanged except where noted below.
Why
Directory reviewers read these strings. Anthropic's policy asks each tool to say what it does and when it fits, but its submission form rules out text that instructs the model ("call X, then Y", "poll every 30 seconds", "do not ..."). OpenAI's guidelines ask for completeness and visible side effects. The text being replaced reads as a sequence of orders, and leaves out costs the caller is charged.
Every description now gives what the tool does and returns, what it costs, its side effects, what each error means, and how it relates to the other tools as a fact about data ("the job_id is what batch_wait takes") rather than as an instruction to call them.
Scope
src/server.tsinstructions,scrape(+js_render,premium_proxy,wait_for,response_type),extract(+fallback_autoparse),prepare_domain,prepared_domain_status,batch_create,batch_wait,batch_results,batch_content,crawl_create,crawl_wait,crawl_results,browser_navigate,browser_close. The rest of the tools follow separately.account_usageandbatch_statusare unchanged: both already read as descriptions rather than instructions.Costs and side effects that were missing
browser_navigatecarried no cost at all. It now states 5 credits per minute, 25,000 credits per GB, and the two idle timeouts (3 minutes without a call, 15 minutes after opening).extractnow carries the rate card, that Extract adds no charge of its own, and that neither aREQS007answer nor a processed page with nothing on it is billed.prepare_domain's monthly slot is labelled a side effect.crawl_createkeepsCRAWL_NOT_ENABLED,CRAWL_TOO_MANY_CRAWLS, the limit of 3 active jobs shared with Batch, andretry_after.crawl_createdocuments thewebhookparameter, which posts signed events to an external URL.Two things worth a second look
scrapeno longer contradicts its own parameter. The tool description drops PDF, because a PDF is decoded as text here (src/server.tsreturns an image only for an image content type, everything else goes throughTextDecoder), so the result is not a usable PDF.response_typestill acceptspdf, and now says what it actually produces. Removing the value from the enum would be the cleaner fix, but it changes the input contract for anyone sending it today, so it is left as its own decision rather than folded into a description change.crawl_createno longer tells the model to check its own patterns against the start page. That sentence is an instruction, which is what this branch removes everywhere else, and its test assertion went with it.crawl_getstill carries the same sentence and is not in this batch, so the behaviour is still documented somewhere while the remaining tools are done.Verification
Two test files move with the wording they pin:
tests/descriptions.test.ts(theprepare_domainguidance, now stated as a fact) andtests/crawl-tool.test.ts(the droppedcrawl_createinstruction). Both failed against the new text before they were updated.The descriptions were also dumped over a real in-memory MCP client (
tools/listplus theinitializeinstructions) and read end to end, so what reviewers see is what was checked, not just the source strings.Typed
fixrather thandocsso release-please cuts a release; the hosted server is the point of the change, and adocscommit would not bump it.