Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

Browserbase Scraper Skill

An OpenClaw skill for scraping Cloudflare-protected websites using Stagehand + Browserbase cloud browsers.

Why This Skill?

Traditional scraping methods fail on protected sites:

  • curl / fetch → Blocked by Cloudflare
  • Playwright headless → Detected as bot
  • Playwright stealth → Still blocked on many sites

This skill uses Browserbase cloud browsers which appear as real users, bypassing bot protection automatically.

Installation

# Install dependencies
npm install @browserbasehq/stagehand zod

# Set up API keys
export BROWSERBASE_API_KEY=your_key
export BROWSERBASE_PROJECT_ID=your_project_id  
export GOOGLE_GENERATIVE_AI_API_KEY=your_gemini_key

Get Browserbase credentials at: https://browserbase.com

Usage

Tell OpenClaw to scrape a protected site:

Scrape all product listings from https://protected-site.com/products
Extract title, price, and URL for each item.

The skill will automatically use Stagehand + Browserbase to bypass protection.

Example Script

See scripts/example_scraper.js for a complete working example that:

  1. Initializes Browserbase cloud browser
  2. Navigates to search results
  3. Handles Cloudflare challenges
  4. Extracts structured data with AI
  5. Saves results to JSON

Cron Integration

Schedule regular scraping jobs:

openclaw cron add \
  --name "Daily Scrape" \
  --cron "0 6 * * *" \
  --session isolated \
  --message "Run the browserbase scraper on https://example.com"

Limitations

  • Browserbase free tier: 1 concurrent session
  • Extraction quality depends on page structure
  • Some sites may still block after repeated access

Credits

Developed while building PTS Delta tracking for Porsche auction data.

License

MIT

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages