From 3f1921b675c2b02c082ff848a805a7da8c3125f0 Mon Sep 17 00:00:00 2001 From: MrScraper <261125219+ai-mrscraper@users.noreply.github.com> Date: Mon, 2 Nov 2026 16:58:09 +0700 Subject: [PATCH] Update the web search skill for MCP server 0.3.0 MrScraper's MCP server 0.3.0 moves webSearch to the new web search endpoint: results are a ranked list, region sets the country, and the new num_results asks for 10 to 100 results. The raw, format, language, page, and render_js options are gone, so the skill no longer mentions them or the search results URL example. --- CHANGELOG.md | 10 +++++ package-lock.json | 4 +- package.json | 2 +- skills/mrscraper-serp/SKILL.md | 77 +++++++++++----------------------- skills/mrscraper/SKILL.md | 4 +- 5 files changed, 39 insertions(+), 58 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c00d8a9..eca6c24 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,16 @@ All notable changes to the MrScraper OpenCode plugin are documented here. The project follows [Semantic Versioning](https://semver.org/). +## 0.4.0 - 2026-11-02 + +- Updated the `mrscraper-serp` skill for MrScraper MCP server 0.3.0, which + moved `webSearch` to MrScraper's new web search endpoint. Results are a + ranked list (rank, title, url, description, source); `region` sets the + results' country and the new `num_results` asks for 10 to 100 results. The + `raw`, `format`, `language`, `page`, and `render_js` options are gone, and + so is the search results URL example. Skills are identical to the MrScraper + Claude plugin 0.3.0. + ## 0.3.2 - 2026-10-29 - Reworded search as web search in the package diff --git a/package-lock.json b/package-lock.json index efa8647..fa4aeca 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@mrscraper/opencode", - "version": "0.3.2", + "version": "0.4.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@mrscraper/opencode", - "version": "0.3.2", + "version": "0.4.0", "license": "MIT", "devDependencies": { "@opencode-ai/plugin": "1.19.0", diff --git a/package.json b/package.json index e7c6297..8908b19 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@mrscraper/opencode", - "version": "0.3.2", + "version": "0.4.0", "description": "MrScraper for OpenCode: the hosted MrScraper MCP server plus fetch-first skills for web scraping, structured data extraction, and web search.", "type": "module", "main": "index.ts", diff --git a/skills/mrscraper-serp/SKILL.md b/skills/mrscraper-serp/SKILL.md index dd9dd49..41ada1c 100644 --- a/skills/mrscraper-serp/SKILL.md +++ b/skills/mrscraper-serp/SKILL.md @@ -1,7 +1,7 @@ --- name: mrscraper-serp description: | - Discover public pages through web search with MrScraper's webSearch tool using a search query or a search results URL. Use when no target URL is known or the user asks to inspect web search results. After discovery, retrieve selected pages with mrscraper-fetch. + Discover public pages through web search with MrScraper's webSearch tool. Use when no target URL is known or the user asks to inspect web search results. After discovery, retrieve selected pages with mrscraper-fetch. --- # Discover Pages with MrScraper MCP Web Search @@ -13,86 +13,58 @@ status, or broader routing. ## Step 1 — Build the Query -Include the important entity, attribute, location, or date in url, and set -raw=false to receive parsed results: +Pass the query in url. Include the important entity, attribute, location, or +date: { - "url": "web scraping best practices", - "raw": false + "url": "web scraping best practices" } -The input can also be a complete search results URL: +The input can also be a search results URL; its q parameter becomes the query. - { - "url": "https://www.google.com/search?q=web+scraping&gl=us&hl=en&start=20", - "raw": false - } - -For a search results URL, the tool reads q, gl, hl, and start. Explicit region, -language, and page inputs take precedence. - -## Step 2 — Set Locale and Pagination +## Step 2 — Set the Country and Result Count -Set both country and language when results must match a locale: +Set region to a two-letter country code when results must come from one +country. It defaults to us: { "url": "running shoes", - "raw": false, - "region": "id", - "language": "id", - "page": 2 - } - -Start with the first page unless the user requests broader coverage. Inspect -the current page before requesting another. - -## Step 3 — Choose the Output - -raw defaults to true, which returns result-page HTML as the Python SDK does. -Set raw=false for parsed JSON; organic results are in data.data.organic, next -to sections such as people_also_ask and pagination. - -Request result-page HTML when the page itself is needed: - - { - "url": "iphone 17" + "region": "id" } -Use JavaScript rendering for dynamic result features such as AI Overview: +The search returns 10 results by default. Ask for more, up to 100, only when +the user needs broader coverage: { - "url": "what is web scraping", - "raw": false, - "render_js": true + "url": "running shoes", + "region": "id", + "num_results": 30 } ### Parameters | MCP input | Default | Request mapping | Use | | --- | --- | --- | --- | -| url | required | Body query | Search query or complete search results URL. | -| raw | true | Body format=html | Return result-page HTML; set false to use format. | -| format | json | Body format | json or html when raw=false. | -| region | URL value or omitted | Body region | Result country, such as us or id. | -| language | URL value or omitted | Body language | Result language, such as en or id. | -| page | URL value or omitted | Body page | Positive, one-based results page. | -| render_js | false | Body renderJs | Render JavaScript-dependent result features. | +| url | required | Body query | Search query, or a search results URL whose q value is used. | +| region | us | Body country | Two-letter result country, such as us or id. | +| num_results | 10 | Body numResults | Number of results, from 10 to 100; fewer may come back. | | timeout | 600 | MCP upstream deadline | Maximum wait in seconds; not sent in the search body. | Authentication is handled by the MCP client's OAuth 2.1 connection and is never a tool input. -## Step 4 — Inspect the Results +## Step 3 — Inspect the Results The complete response envelope is available in MCP structuredContent and as formatted JSON text. Prefer structuredContent. Check isError, error, and -status_code before using data. +status_code before using data. The results are a ranked list in data.data; +each result has rank, title, url, description, and source. When the user requests a search artifact, save the successful data or full envelope with local file-writing capability; webSearch has no output-path input. -## Step 5 — Continue with Relevant URLs +## Step 4 — Continue with Relevant URLs Select only URLs relevant to the user's goal, then load [mrscraper-fetch](../mrscraper-fetch/SKILL.md) to retrieve and inspect each @@ -102,12 +74,11 @@ Run independent follow-up URLs in parallel when the environment supports safe parallel execution. Keep the number of pages proportional to the requested coverage. If the user only requested URL discovery, stop after the search. -## Step 6 — Handle Weak or Missing Results +## Step 5 — Handle Weak or Missing Results -- Set both region and language when the locale is wrong. +- Set region when the results come from the wrong country. - Rewrite noisy queries around the important entity, attribute, and location. -- Request another result page only after checking the current page. -- Retry once with render_js=true when a dynamic result feature is required. +- Raise num_results only after checking the first results. - Increase timeout when the MCP request deadline is too short. - Use [mrscraper](../mrscraper/SKILL.md) when the connection or authentication fails. diff --git a/skills/mrscraper/SKILL.md b/skills/mrscraper/SKILL.md index 37c08cb..0c60b51 100644 --- a/skills/mrscraper/SKILL.md +++ b/skills/mrscraper/SKILL.md @@ -158,8 +158,8 @@ account usage, not the progress of a request. replaced by scraperAPI, webSearch, and status. - Page incomplete or missing dynamic content: load mrscraper-fetch and revise page-loading inputs deliberately. -- Search results weak: load mrscraper-serp and refine the query, locale, or - page. +- Search results weak: load mrscraper-serp and refine the query, country, or + number of results. ## Limits