rustdoll

Stealth-first Chromium automation for Rust, straight over the Chrome DevTools Protocol — no WebDriver, no chromedriver.

rustdoll is a Rust port of the Python pydoll library. It drives a real Chrome, Chromium, or Edge browser by speaking CDP directly, with humanized input and anti-bot handling built in.

Status: early and under active construction. The port is being built phase by phase and the public API is still unstable — expect breaking changes. Porting conventions and the roadmap live in RUST_PORT_PLAN.md.

Features

• Direct CDP control of Chrome/Chromium or Edge — no WebDriver and no external driver binary. The browser is auto-discovered; each launch gets an isolated temp profile and a clean async shutdown.
• Navigation & scripting — go_to, read title / current_url / page_source, and run arbitrary JS with execute_script.
• Element location — builder finders (find().id(..).class_name(..).text(..).first()), CSS and XPath (query, query_all), Shadow DOM, and iframe traversal (same-origin and cross-origin OOPIFs).
• Human-like interactions — mouse, keyboard, and scrolling with statistical humanization; the RNG is seedable for deterministic tests.
• CDP events — subscribe to page / network / runtime / DOM / fetch events with async callbacks, plus a network-log buffer.
• Request handling — intercept and continue / fail / fulfill requests, make in-page HTTP calls via fetch-over-CDP, and record HAR.
• Capture — screenshots (to file or bytes), print-to-PDF, and single-file page bundles.
• Cookies, dialogs, and User-Agent override.
• Structured extraction — #[derive(Extract)] maps CSS selectors to typed structs at compile time.
• Cloudflare Turnstile — bypass_cloudflare / enable_auto_solve_cloudflare.

Requirements

• Rust 1.80+ (workspace MSRV).
• A Chromium-family browser installed: Google Chrome, Chromium, or Microsoft Edge. rustdoll finds it on common install paths and your PATH; you can also point at a specific binary with ChromiumOptions::set_binary_location.

Installation

rustdoll is not published to crates.io yet — depend on it via git:

[dependencies]
rustdoll = { git = "https://github.com/hozantaher/rustdoll" }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Quick start

use rustdoll::{Chrome, ChromiumOptions};

#[tokio::main]
async fn main() -> rustdoll::Result<()> {
    // Launch a real browser. `ci_headless()` is a headless + no-sandbox preset;
    // use `ChromiumOptions::default()` (headed) or the builder for custom flags.
    let chrome = Chrome::with_options(ChromiumOptions::ci_headless())?;
    let tab = chrome.start().await?; // the first tab

    tab.go_to("https://example.com").await?;
    println!("title: {}", tab.title().await?);

    // Locate an element (waiting up to 5s) and interact with it.
    if let Some(link) = tab.find().tag_name("a").timeout(5).first().await? {
        println!("first link: {}", link.text().await?);
        link.click().await?;
    }

    // No async Drop: shut down explicitly. This also removes the temp profile.
    chrome.stop().await?;
    Ok(())
}

Custom launch flags use the mutable builder:

let mut options = ChromiumOptions::default();
options
    .set_headless(true)
    .add_argument("--window-size=1920,1080");
let chrome = Chrome::with_options(options)?;

Structured extraction

Annotate a struct with selectors and let the Extract derive pull a whole page into typed data. The field type controls cardinality: a scalar is required, Option<T> is optional, and Vec<T> collects all matches. Non-String scalars are parsed via FromStr; add attribute = "..." to read an attribute instead of the element's text.

use rustdoll::Extract;

#[derive(Extract, Debug)]
struct Product {
    #[extract(selector = "#name")]
    name: String,
    #[extract(selector = "#price")]
    price: f64,                                  // parsed from text via FromStr
    #[extract(selector = "#promo")]
    promo: Option<String>,                       // None if the selector misses
    #[extract(selector = ".tag")]
    tags: Vec<String>,                           // every match
    #[extract(selector = "#link", attribute = "href")]
    link: String,                                // reads the attribute, not text
}

// after `tab.go_to(...)`:
let product: Product = tab.extract().await?;

More capabilities

Events (async callbacks over CDP):

tab.enable_network_events().await?;
tab.on("Network.responseReceived", |event| async move {
    println!("{}", event["params"]["response"]["url"]);
}, /* temporary = */ false);

Screenshots, PDF, cookies, and solving a Cloudflare challenge:

tab.take_screenshot("shot.png").await?;
tab.print_to_pdf("page.pdf").await?;
let cookies = tab.get_cookies().await?;
let solved: bool = tab.bypass_cloudflare(30.0).await?;

Workspace layout

crates/
  rustdoll/         # main library: browser lifecycle, connection, elements,
                    # interactions, requests, extraction
  rustdoll-cdp/     # CDP wire types + hand-written command builders
  rustdoll-derive/  # the #[derive(Extract)] proc-macro

Development

cargo build

# Most integration tests launch a REAL headless Chrome and FAIL (not skip)
# without one, so a Chromium-family browser must be installed:
cargo test

# Browser-free subset (no Chrome needed):
cargo test -p rustdoll --lib --test connection

Before a change is considered done, all three of these should pass — clippy warnings are treated as errors:

cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test

See CLAUDE.md for the porting conventions (pydoll parity, verbatim error strings, the hand-written CDP layer, and more).

Relationship to pydoll

rustdoll follows pydoll's design closely — error messages are kept verbatim and each module notes the pydoll file it ports — while adapting to Rust idioms: async fns instead of async properties, builder finders instead of keyword arguments, and #[derive(Extract)] instead of pydantic models. Humanization aims for statistical, not byte-exact, parity with the original.

License

MIT. rustdoll is a Rust port of pydoll by AutoscrapeLabs; see LICENSE.