Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,19 @@
All notable changes to the MrScraper OpenCode plugin are documented here. The
project follows [Semantic Versioning](https://semver.org/).

## 0.2.0 - 2026-10-12

- Added OpenCode 2 support. OpenCode 2 does not run OpenCode 1 plugins; the
package now default-exports one object with an OpenCode 2 `setup()`, which
registers the `mrscraper` MCP server and the four skills through MCP and
skill transforms, and an OpenCode 1 `server()`, which keeps the existing
config hook.
- OpenCode 1 now needs 1.18.29 or later (the first release that loads the
object form). On older releases, pin `@mrscraper/opencode@0.1.0`.
- The `MrScraperPlugin` named export was removed; use the default export.
- Documented OpenCode 2 install, sign-in, API-key configuration
(`mcp.servers`), and update commands.

## 0.1.0 - 2026-10-05

- Added the initial `@mrscraper/opencode` plugin: registers the hosted
Expand Down
26 changes: 17 additions & 9 deletions PUBLISHING.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,22 @@ Use this checklist for every public release of `@mrscraper/opencode`.
The package must contain `CHANGELOG.md`, `LICENSE`, `README.md`,
`index.ts`, `package.json`, and the four `skills/*/SKILL.md` files.

- [ ] In a clean OpenCode profile, add the packed tarball
(`"plugin": ["@mrscraper/opencode@file:/absolute/path/mrscraper-opencode-<version>.tgz"]`),
then confirm:
- [ ] Test the packed package in clean profiles (separate `HOME` and `XDG_*`
directories) of OpenCode 2 and of the oldest supported OpenCode 1
(1.18.29):
- OpenCode 2: extract the tarball and add the extracted `package/`
directory's absolute path to `plugins`; OpenCode 1: add
`"plugin": ["@mrscraper/opencode@file:/absolute/path/mrscraper-opencode-<version>.tgz"]`;
- `opencode mcp list` shows `mrscraper` as `needs authentication`;
- `opencode mcp auth mrscraper` completes in the browser and
`opencode mcp list` then shows `connected`;
- `opencode debug skill` lists `mrscraper`, `mrscraper-fetch`,
`mrscraper-scrape`, and `mrscraper-serp`;
- the skills list (`opencode api skill.list` in OpenCode 2,
`opencode debug skill` in OpenCode 1) includes `mrscraper`,
`mrscraper-fetch`, `mrscraper-scrape`, and `mrscraper-serp`;
- with the README's API-key configuration, `opencode mcp list` shows
`connected`, which also proves that a user-defined server is kept;
- the prompt `Fetch https://www.scrapethissite.com/pages/simple/ and summarize the page.`
calls `mrscraper_fetch` and summarizes the page.
calls the `fetch` tool and summarizes the page.

## Publish

Expand All @@ -53,14 +59,16 @@ gh release create v<version> --title "v<version>" --notes "See CHANGELOG.md for
```

Then confirm `npm view @mrscraper/opencode version` returns the new version
and `opencode plugin @mrscraper/opencode --global` installs it in a clean
profile.
and that `opencode plugin add @mrscraper/opencode` (OpenCode 2) and
`opencode plugin @mrscraper/opencode --global` (OpenCode 1) install it in clean
profiles.

## Listings

- [OpenCode ecosystem](https://opencode.ai/docs/ecosystem#plugins): one row in
the Plugins table of `packages/web/src/content/docs/ecosystem.mdx`, through a
pull request to `anomalyco/opencode` that follows its PR template.
pull request to `anomalyco/opencode` against `dev` (OpenCode 2 since
2026-10-10; V1 pull requests are closed) that follows its PR template.
- [awesome-opencode](https://github.com/awesome-opencode/awesome-opencode):
one YAML file in `data/plugins/`.
- [opencode.cafe](https://opencode.cafe/submit): community catalog form.
134 changes: 93 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,51 @@ saved scraper reruns, stored results, and account usage. The plugin also
includes four focused skills that help OpenCode choose the right workflow and
preserve raw page content.

The same package works in OpenCode 2 and OpenCode 1.

## What this plugin adds

- An MCP server named `mrscraper` that points at the hosted Streamable HTTP
endpoint `https://mcp.mrscraper.com/mcp`.
- OAuth 2.1 sign-in handled by OpenCode; no credential is stored in this
package.
- The `mrscraper`, `mrscraper-fetch`, `mrscraper-scrape`, and `mrscraper-serp`
skills, loaded on demand through OpenCode's `skill` tool.
skills.
- A fetch-first workflow that keeps raw page responses available for analysis,
verification, and follow-up transformations.

If your configuration already defines `mcp.mrscraper`, the plugin keeps your
entry and only adds the skills.
If your configuration already defines a `mrscraper` MCP server, the plugin
keeps your entry and only adds the skills. It also leaves any skill with the
same name that you already have in place.

## Requirements

- [OpenCode](https://opencode.ai/docs/) (verified with 1.18.34).
- OpenCode 2 (verified with 2.0.26), or OpenCode 1.18.29 or later (verified
with 1.18.29 and 1.19.0). On older OpenCode 1 releases, pin
`@mrscraper/opencode@0.1.0`.
- A [MrScraper](https://app.mrscraper.com) account.
- Permission to access and process the target content.

## Install

### OpenCode 2

```bash
opencode plugin add @mrscraper/opencode
```

This adds the package to `plugins` in your global configuration. To enable it
for one project instead, add it to that project's `opencode.jsonc`:

```jsonc
{
"$schema": "https://opencode.ai/config.json",
"plugins": ["@mrscraper/opencode"]
}
```

### OpenCode 1

```bash
opencode plugin @mrscraper/opencode --global
```
Expand All @@ -42,25 +65,18 @@ Or add the package to the `plugin` list in `~/.config/opencode/opencode.json`
}
```

OpenCode installs npm plugins when it starts and caches them under
`~/.cache/opencode/packages/`.
OpenCode installs npm plugins when it starts and caches them.

## Sign in

```bash
opencode mcp auth mrscraper
```

OpenCode opens the MrScraper consent page and waits up to five minutes for the
browser to return to `http://127.0.0.1:19876/mcp/oauth/callback`. It requests
the `scrape:read`, `scrape:write`, and `account:read` scopes advertised by the
server, plus `offline_access` for token refresh, and stores the tokens in
`~/.local/share/opencode/mcp-auth.json`. Never paste OAuth tokens or API keys
into chat.

On a remote machine, open the printed URL in a local browser after forwarding
the callback port, for example `ssh -L 19876:127.0.0.1:19876 <host>`, or use
an API key instead.
In OpenCode 2 you can also run `/mcps` in a session, select `mrscraper`, and
sign in there. OpenCode opens the MrScraper consent page in your browser,
registers its own OAuth client, and stores the tokens outside your project
configuration. Never paste OAuth tokens or API keys into chat.

Check the connection:

Expand All @@ -71,11 +87,37 @@ opencode mcp list
`mrscraper` shows `needs authentication` before sign-in and `connected`
afterwards.

On a machine without a local browser, use an API key instead (next section).
OpenCode 1 waits for the browser on `http://127.0.0.1:19876/mcp/oauth/callback`,
so you can also forward that port, for example
`ssh -L 19876:127.0.0.1:19876 <host>`.

## Use an API key instead of OAuth

Create a key at [app.mrscraper.com/api-tokens](https://app.mrscraper.com/api-tokens),
export it as `MRSCRAPER_API_KEY` in the environment that starts OpenCode, and
define the server yourself:
define the server yourself.

OpenCode 2 (`opencode.jsonc`):

```jsonc
{
"$schema": "https://opencode.ai/config.json",
"plugins": ["@mrscraper/opencode"],
"mcp": {
"servers": {
"mrscraper": {
"type": "remote",
"url": "https://mcp.mrscraper.com/mcp",
"oauth": false,
"headers": { "Authorization": "Bearer {env:MRSCRAPER_API_KEY}" }
}
}
}
}
```

OpenCode 1 (`opencode.json`):

```json
{
Expand All @@ -95,30 +137,29 @@ define the server yourself:
```

OpenCode substitutes `{env:MRSCRAPER_API_KEY}` when it loads the configuration,
so the key stays out of the file. An API key carries your full account
authority; OAuth scopes do not apply to it.
so the key stays out of the file. In OpenCode 2 the substitution happens in the
background service, so the variable must be set where the service starts; after
setting it, run `opencode service restart` from that shell. An API key carries
your full account authority; OAuth scopes do not apply to it.

## Try it

```text
Fetch https://www.scrapethissite.com/pages/simple/ and summarize the page.
```

OpenCode prefixes MCP tools with the server name:
The server provides these tools. OpenCode 1 shows them as `mrscraper_<tool>`;
OpenCode 2 reaches MCP tools through Code Mode by default.

| Tool | Purpose |
| --- | --- |
| `mrscraper_fetch` | Retrieve and preserve a known page's raw response. |
| `mrscraper_scrape` | Run managed structured extraction or bounded site mapping. |
| `mrscraper_serp` | Discover public pages through Google. |
| `mrscraper_rerun` | Reuse a saved AI or manual scraper configuration. |
| `mrscraper_results` | Browse and filter stored result records. |
| `mrscraper_result` | Retrieve one stored result or poll an asynchronous run. |
| `mrscraper_status` | Inspect subscription usage and request outcomes. |

To keep the tools out of every session, add `"tools": { "mrscraper_*": false }`
to your configuration and enable them per agent; see
[MCP servers](https://opencode.ai/docs/mcp-servers/#manage).
| `fetch` | Retrieve and preserve a known page's raw response. |
| `scrape` | Run managed structured extraction or bounded site mapping. |
| `serp` | Discover public pages through Google. |
| `rerun` | Reuse a saved AI or manual scraper configuration. |
| `results` | Browse and filter stored result records. |
| `result` | Retrieve one stored result or poll an asynchronous run. |
| `status` | Inspect subscription usage and request outcomes. |

## Fetch-first routing

Expand Down Expand Up @@ -150,17 +191,25 @@ Review MrScraper's [MCP documentation](https://docs.mrscraper.com/docs/getting-s

## Update or remove

An unpinned entry stays on the version OpenCode first cached. To control
updates, pin a version and change it when you upgrade:
OpenCode 2 checks unpinned npm plugins for updates without installing them.
Update or remove the plugin with:

```bash
opencode plugin update @mrscraper/opencode
opencode plugin remove @mrscraper/opencode
```

In OpenCode 1, an unpinned entry stays on the version OpenCode first cached.
To control updates, pin a version and change it when you upgrade:

```json
{
"plugin": ["@mrscraper/opencode@0.1.0"]
"plugin": ["@mrscraper/opencode@0.2.0"]
}
```

To remove MrScraper, delete the entry from `plugin`, then remove the stored
OAuth tokens:
To remove it from OpenCode 1, delete the entry from `plugin`. In either
version, remove the stored OAuth tokens afterwards:

```bash
opencode mcp logout mrscraper
Expand All @@ -179,7 +228,8 @@ account.
## Development

```text
index.ts registers the MCP server and the skills directory
index.ts default export with an OpenCode 2 `setup()` and an OpenCode 1
`server()`; both register the MCP server and the skills
skills/ MCP-oriented skills, copied verbatim from
mrscraper-com/mrscraper-claude-plugin (plugins/mrscraper/skills)
```
Expand All @@ -190,10 +240,12 @@ npm run check
npm pack --dry-run
```

Test a local checkout by adding its absolute path to `plugin` in an OpenCode
configuration, then run `opencode mcp list` and `opencode debug skill`. Record
user-visible changes in [CHANGELOG.md](CHANGELOG.md) and follow
[PUBLISHING.md](PUBLISHING.md) for releases.
Test a local checkout in both majors before a release. In OpenCode 2, add the
checkout's absolute path to `plugins`, then run `opencode mcp list` and
`opencode api skill.list`. In OpenCode 1, add it to `plugin`, then run
`opencode mcp list` and `opencode debug skill`. Record user-visible changes in
[CHANGELOG.md](CHANGELOG.md) and follow [PUBLISHING.md](PUBLISHING.md) for
releases.

## License

Expand Down
78 changes: 73 additions & 5 deletions index.ts
Original file line number Diff line number Diff line change
@@ -1,19 +1,23 @@
import type { Plugin } from "@opencode-ai/plugin"
import type { Plugin as OpenCode1Plugin } from "@opencode-ai/plugin"
import type { Plugin as OpenCode2, Skill } from "@opencode/plugin"
import { readdir, readFile } from "node:fs/promises"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"

const SERVER_NAME = "mrscraper"
const MCP_URL = "https://mcp.mrscraper.com/mcp"
const SKILLS_DIR = join(dirname(fileURLToPath(import.meta.url)), "skills")

// OpenCode reads `skills.paths` (see https://opencode.ai/config.json), but the
// OpenCode 1 reads `skills.paths` (see https://opencode.ai/config.json), but the
// plugin SDK's Config type does not declare it yet.
type SkillsConfig = { skills?: { paths?: string[]; urls?: string[] } }

export const MrScraperPlugin: Plugin = async () => ({
// OpenCode 1: add the server and the skills directory through the config hook.
const server: OpenCode1Plugin = async () => ({
async config(input) {
input.mcp ??= {}
// Keep a user-defined entry, such as the API-key fallback or `enabled: false`.
input.mcp.mrscraper ??= { type: "remote", url: MCP_URL }
input.mcp[SERVER_NAME] ??= { type: "remote", url: MCP_URL }

const config = input as typeof input & SkillsConfig
config.skills ??= {}
Expand All @@ -22,4 +26,68 @@ export const MrScraperPlugin: Plugin = async () => ({
},
})

export default MrScraperPlugin
// OpenCode 2: register the same server and skills through domain transforms.
const definition = {
id: "mrscraper",
async setup(ctx) {
const skills = await readSkills()
await ctx.mcp.transform((editor) => {
// Keep a user-defined `mcp.servers.mrscraper`, such as the API-key fallback.
if (!editor.get(SERVER_NAME)) editor.set(SERVER_NAME, { type: "remote", url: MCP_URL })
})
await ctx.skill.transform((editor) => {
for (const skill of skills) if (!editor.get(skill.id)) editor.add(skill)
})
},
} satisfies OpenCode2.Plugin

// Mirrors OpenCode 2's own SKILL.md loading: the folder name is the ID, the
// frontmatter supplies `name` and `description`, and the body is the content.
async function readSkills(): Promise<Skill.Info[]> {
const skills: Skill.Info[] = []
const folders = await readdir(SKILLS_DIR, { withFileTypes: true }).catch(() => [])
for (const folder of folders) {
if (!folder.isDirectory()) continue
const path = join(SKILLS_DIR, folder.name, "SKILL.md")
const text = await readFile(path, "utf8").catch(() => undefined)
const markdown = text === undefined ? undefined : parseMarkdown(text)
if (!markdown) continue
const { name, description } = markdown.frontmatter
skills.push({
id: folder.name as Skill.Info["id"],
name: (name || folder.name) as Skill.Info["name"],
...(description ? { description } : {}),
path: path as Skill.Info["path"],
content: markdown.content,
})
}
return skills
}

// Reads the flat frontmatter these skills use: plain or quoted scalars and
// `|` or `>` block scalars. Nested keys are ignored.
function parseMarkdown(text: string) {
const match = /^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n|$)([\s\S]*)$/.exec(text)
if (!match) return undefined
const frontmatter: Record<string, string> = {}
const lines = match[1].split(/\r?\n/)
for (let i = 0; i < lines.length; i++) {
const field = /^([A-Za-z0-9_-]+):\s*(.*)$/.exec(lines[i])
if (!field) continue
const [, key, value] = field
if (!/^[|>][-+]?$/.test(value)) {
frontmatter[key] = value.replace(/^(["'])(.*)\1$/, "$2").trim()
continue
}
const block: string[] = []
while (i + 1 < lines.length && (lines[i + 1].trim() === "" || /^\s/.test(lines[i + 1]))) block.push(lines[++i])
const indent = Math.min(...block.filter((line) => line.trim()).map((line) => line.length - line.trimStart().length))
const literal = block.map((line) => line.slice(indent)).join("\n")
frontmatter[key] = (value.startsWith(">") ? literal.replace(/([^\n])\n(?=[^\n])/g, "$1 ") : literal).trim()
}
return { frontmatter, content: match[2] }
}

// One default export serves both majors: OpenCode 1 (1.18.29 or later) calls
// `server()`, and OpenCode 2 reads `id` and `setup()` and ignores `server()`.
export default { ...definition, server }
Loading
Loading