Web Fetch API

The browser runs on your cloud server, not your laptop.

A lightweight service that deploys a headless Chromium browser to your cloud server. Your local machine needs nothing — no Playwright, no Chrome, no dependencies. Just a curl command.

curl http://your-server:6689/fetch?url=https://example.com

Bypasses Cloudflare, WAF, JS-rendered pages. Returns clean text. REST API + MCP for AI agents.

---

Why

Running Playwright locally works, but:

• Requires 400MB+ browser install on every machine
• Dies when you close your laptop
• Can't be shared across devices or team members

Fix: deploy the browser once to a $5/mo cloud server. Call it from anything — laptop, phone, Raspberry Pi, CI pipeline, AI agent.

Works for any website, any region.

• A Chinese server with direct connection → CSDN, B站, Zhihu, Baidu
• A Chinese server with a proxy → Google, GitHub, Wikipedia
• An AWS server in US/EU → the open web
• A Chinese server logged into Gitee → your private repos

You pick the server location. The browser lives there. You call it from anywhere. No geo-blocks, no local setup, no hassle.

Looking for something practical to run on your cloud server? This is it. A cheap VPS no longer sits idle — it becomes your remote browser, your AI agent's web tool, and part of your personal infrastructure. Pair it with other lightweight services (a hot topics API, a personal fetch proxy) and you've got a useful server for a few dollars a month.

---

Features

• Browser in the cloud — your machine needs zero setup. Just curl.
• MCP tool interface — plug into Claude, Cursor, any MCP host
• Cookie persistence — log in once, reuse session across requests
• Clean content — strips ads, nav, scripts; returns article text
• Screenshot — full-page screenshots as base64
• Self-hosted — your server, your data, no third-party SaaS

---

Quick Start

Local

pip install -r requirements.txt
playwright install chromium
python server.py

Server runs at http://localhost:6689.

Docker

docker build -t web-fetch-api .
docker run -d -p 6689:6689 --restart unless-stopped web-fetch-api

---

AI Agent Integration (MCP)

This server exposes an MCP (Model Context Protocol) endpoint at /mcp. Any MCP-compatible client can connect directly.

Claude Code

Configure in your MCP settings:

{
  "mcpServers": {
    "web-fetch": {
      "url": "http://your-server:6689/mcp"
    }
  }
}

The tool fetch_page(url, timeout) becomes available for fetching web content.

Other MCP clients (Cursor, custom agents)

Same pattern — point to http://your-server:6689/mcp and use the fetch_page tool.

---

REST API

Fetch a page

GET /fetch?url=<url>&timeout=30

{
  "url": "https://example.com",
  "title": "Example Domain",
  "content": "...clean text..."
}

Screenshot

GET /screenshot?url=<url>&timeout=15

Returns base64 PNG.

Health

GET /health

Cookie management

Method	Endpoint	Description
GET	/cookies	List saved cookie domains
POST	/cookies	Add cookies (JSON body)
DELETE	/cookies	Clear all cookies

---

Deploy to Cloud

1. Get any Linux VPS (1 GB RAM minimum)
2. Install Python and Chromium:

pip install -r requirements.txt
playwright install chromium

3. Run:

nohup python3 server.py > fetch.log 2>&1 &

4. Verify:

curl http://your-server-ip:6689/health

---

Configuration

Variable	Default	Description
FETCH_PROXY	—	HTTP proxy for the browser
COOKIE_FILE	cookies.json	Cookie persistence file
CHROME_PATH	—	Path to system Chrome (e.g. /usr/bin/google-chrome-stable). Leave unset to use Playwright's bundled browser

---

Project Structure

├── server.py          # FastAPI + MCP server
├── client.py          # CLI client
├── requirements.txt   # Python dependencies
├── Dockerfile         # Container image
├── deploy.sh          # Deployment template
└── README.md

---

Architecture

Client ──HTTP──▶ Server (:6689) ──Playwright──▶ Headless Chromium
                    │
                    ├── /fetch  (REST)
                    ├── /mcp    (MCP for AI agents)
                    └── /screenshot (REST)

No browser on your machine. No local dependencies.

---

License

MIT