Minifetch API

Minifetch is a hosted toolkit for web developers and AI agents. Search, scrape, extract & SEO audit web pages. Pay per fetch, no subscription.

• ✅ Always pay-per-fetch at competitive prices.
• ✅ Sign up for an account & get free starter credits. 🎉🎉
• ⛔ No charge for blocked pages (403 errors).

---

👉 Full API docs | LLMs.txt and SKILL.md for agents | Questions? Join our Discord

---

Prerequisites

• Node.js v18+ & NPM
• A valid API key from our dashboard
• Or an Ethereum or Solana private key for making USDC payments on Base or Solana networks.

Payments. Two ways to pay:

1. Credit card + API key. Get started free - visit our dashboard to Sign Up. Create a Minifetch account and it will be auto-loaded with credits worth up to 25 free technical SEO page audits. Top up with your credit card later.
2. USDC on Base or Solana. Just load your wallet with USDC, you're ready. No "gas token" (ETH or SOL) required. You don't need a Minifetch account either, just load your wallet and go!

Install

Add to your project:

npm install minifetch-api --save`

Quick Start

import Minifetch from "minifetch-api";

// First, initialize the client with your payment choice-
// pick option 1 or 2:

// 1. API Key Payments:
const client = new Minifetch({
  apiKey: process.env.MINIFETCH_API_KEY
});

// 2. x402 Payments:
//   - Network options: "base" or "solana"
//   - Private key from wallet that has a small amt of USDC
const client = new Minifetch({
  network: "base",
  privateKey: process.env.BASE_PRIVATE_KEY,
});

// Now you're ready to call the API methods:
try {
  const url = "example.com";
  const response = await client.checkAndRunSeoPageAudit(url);
  // 200 "ok" responses:
  console.log(response);
} catch (err) {
  // No charge for errors or 403 blocked urls!
  console.log(err);
}

API Methods

After the Quick Start, you have the following methods to use.

Wrap these methods in a try/catch just like in the Quick Start example above. Code examples can be also found in the Github repository /example- directories.

All API methods default to POST unless you set options to { method: 'GET' } and pass in as the second argument.

The checkAndExtract methods check the target URL's robots.txt file to ensure its not blocked and tell us your preferred crawl delay (defaults to 1 second between requests to your domain). So fetching 10 URLs takes at least 10 seconds to complete by default. This is by design, so Minifetch never hammers your server or slows it down for your real users. If you own the site and want to allow Minifetch access or to set custom rules for it, read How To Unblock Minifetch.

await client.searchByKeyword("green tea");
// Price: $0.002
// Keyword web search by Ceramic.ai that returns ranked results as
// structured JSON: a title, URL, and text snippet per result.

await client.checkAndExtractUrlMetadata(url, options);
// Price: $0.002
// Extracts rich structured metadata from your URL: redirects, favicons,
// title, description, canonical, SEO-related meta tags, Open Graph and
// Twitter card tags, JSON-LD and more. Returns everything by default;
// narrow the response with the `fields` query param (pass atomic field
// names, ex: ?fields=title,description or the named groups: network,meta,
// og,twitter and/or meta:<name> for a specific page meta tag)
// Options:
// { fields: ['canonical', 'og:url'] } - narrows to custom selected fields
// { omitEmpty: true } - defaults to false
// { includeResponseBody: true } - defaults to false

await client.checkAndExtractUrlLinks(url);
// Price: $0.002
// Extracts all links from your URL categorized by type (internal/
// external/ anchor). Detects image links, `rel` attributes (nofollow,
// sponsored, ugc, etc), `title` and `target`, plus image detection.
// Summary stats include the most-linked-to internal pages (with
// anchor text variants used for each) and top external domains by
// link count.

await client.checkAndExtractUrlContent(url, options);
// Price: $0.002
// For site owners auditing AI readability: returns the clean markdown
// an LLM extracts from your page after nav, ads, & scripts are stripped.
// See what survives for AEO and AI search; respects robots.txt.
// Options: { includeMediaUrls: true } - defaults to false.

await client.checkAndExtractUrlPreview(url);
// Price: $0.002
// Extracts all fields used for a page's share previews: the lightweight
// cards that represent the page on social platforms, chat apps, and AI.

await client.checkAndRunSeoPageAudit(url);
// Price: $0.01
// Runs a full technical SEO audit on your URL. Combines data from
// the other API endpoints and runs checks that each return a PASS/
// WARN/ FAIL result with no black-box scoring. Just deterministic,
// composable signal you can act on or pipe into an agent.
// Audit rules are documented in the skill file:
// https://minifetch.com/skills/seo-page-audit/SKILL.md

For max control, you can also use the following methods directly:

await client.preflightCheck(url, options);
// Free: check if `minfetch` user agent can access target URL via robots.txt
// Options: { "fresh": true } - bypass 24hr robots.txt cache, defaults to false

// Paid methods:
await client.extractUrlMetadata(url, options); // same options as above
await client.extractUrlLinks(url);
await client.extractUrlContent(url, options); // same options as above
await client.extractUrlPreview(url);
await client.runSeoPageAudit(url);

---

Error Types

When you wrap the functions above in a try/catch, here are some of the errors you may encounter. You are never charged for target URLs that are blocked or error.

• "InvalidURLError: Invalid url ${url}"
  • The URL is malformed in some way, correct it and try again.
• "RobotsBlockedError: URL is blocked by robots.txt"
  • Minifetch is explicitly blocked by the website's robots.txt, cannot be fetched. If this is your site, read our tutorial How To Unblock Minifetch
• "Network Error: 402 Payment Required"
  • Check your wallet -- likely you ran out of USDC to pay!
• "Network Error: 429 Too Many Requests"
  • Back off and retry, max 5-10 requests per second.
• "Network Error: 502 Bad Gateway"
  • URLs that pass their robots.txt check but are blocked anyway via 403 or other tactics may error like this. No charge.
• "503 Service Temporarily Unavailable"
  • Likely encountering upstream timeout errors on the target URL.

---

How We Fetch Web Pages

Minifetch extracts publicly available metadata and content from pages accessible without authentication or javascript execution.

Every response carries a proxy block: the minifetch user agent we sent and whether robots.txt was obeyed on the fetch. Proof of how the fetch happened, not just a promise, for regulated use-cases where provenance matters.

Future add-ons:

• Residential proxies for hard-to-reach pages
• Javascript execution
• Access to authenticated or logged-in content

---

USDC Best Practices

• Uses the x402 Protocol. No "gas token" (ETH or SOL) required, just load your wallet with USDC on Base or Solana network.
• LLMs & Agents should never have direct access to your private key. They will expose it!
• Only keep a small amount of USDC in the wallet whose private key you use.
• Keep that wallet/ account separate from the rest of your onchain funds.
• Pass your private key into the Minifetch API Client with an environment variable.

---

License

MIT / Copyright (c) 2026 Lauren Garcia This package is an API client for Minifetch.com. The client code is open source, but use of the Minifetch API is subject to the Minifetch.com Terms of Service.