🔍 Web Analyzer

Enterprise Domain Security & Intelligence Platform

High-performance Rust toolkit for web reconnaissance, security assessment, and technology fingerprinting

---

🚀 Quick Start

cargo add web-analyzer

Or add to your Cargo.toml:

[dependencies]
web-analyzer = "0.1"
tokio = { version = "1", features = ["full"] }

use web_analyzer::domain_info::get_domain_info;

#[tokio::main]
async fn main() {
    let info = get_domain_info("example.com", None).await.unwrap();
    println!("IP: {:?}, Score: {}/100", info.ipv4, info.security_score);
}

Selective features: cargo add web-analyzer --no-default-features --features domain-info,security-analysis

---

✨ Features

• 15 modular analysis modules organized in 3 security pillars
• Feature-gated compilation — include only what you need
• Fully async — built on Tokio for concurrent analysis
• Zero Python dependencies — pure Rust with system tool integration
• Real-time UI streaming — ScanProgress MPSC capabilities for high-latency modules
• Comprehensive output — all results serialize to JSON via Serde
• WordPress deep analysis — version, theme, plugins, user enumeration, XMLRPC
• 36-service subdomain takeover DB with exploitation difficulty ratings
• Parallel bulk domain validation with atomic counters
• Mobile Graceful Degradation — *_mobile.rs pure-rust polyfills for seamless Android/iOS compilation bridging DNS (hickory) and HTTP/TLS (reqwest).

---

📦 Module Overview

🔎 Intelligence Gathering

Module	Lines	Description	Docs
domain_info	524	WHOIS (raw TCP), SSL certificates, DNS, port scanning, security score	📖
domain_dns	77	A, AAAA, MX, NS, SOA, TXT, CNAME records via dig	📖
seo_analysis	711	13 analysis categories, schema markup, 13 tracking tools, scoring	📖
web_technologies	785	10 servers, 8 backend, 7 frontend, 12 JS libs, 8 CSS, 11 CMS, 9 e-commerce, 6 CDN, 8 analytics, 8 WAF, WordPress analysis	📖
domain_validator	437	DNS + HTTP + SSL validation, parallel bulk processing, 34 skip patterns	📖

🕵️ Reconnaissance

Module	Lines	Description	Docs
subdomain_discovery	194	Subfinder integration, multi-part TLDs, automated HTTP live-probing	📖
contact_spy	400	BFS crawl, email/phone/social extraction (15 platforms), validation	📖
advanced_content_scanner	754	24 secret patterns, 13 JS vulnerability checks, SSRF, sensitive files	📖

🛡️ Security Assessment

Module	Lines	Description	Docs
security_analysis	679	WAF detection (7 providers), SSL grading A+ to F, CORS, cookies, composite score	📖
subdomain_takeover	379	36-service vulnerability DB, 5 detection cases, exploitation difficulty, mitigation	📖
cloudflare_bypass	274	Origin IP discovery via history lookup, TCP verification, private IP filter	📖
nmap_zero_day	249	Nmap integration, NVD CVE lookup, Exploit-DB, CVSS severity	📖
api_security_scanner	1046	9 test suites: SQLi, XSS, SSRF, path traversal, CORS, auth, rate limiting, header injection	📖
geo_analysis	189	llms.txt, WebMCP HTML features, AI crawler directives	📖

📚 Full documentation index: docs/readme.md

---

🚀 Quick Start

Installation

git clone https://github.com/keyvanarasteh/web-analyzer.git
cd web-analyzer
cargo build --all-features

Usage

use web_analyzer::domain_info::get_domain_info;
use web_analyzer::security_analysis::analyze_security;
use web_analyzer::web_technologies::detect_web_technologies;
use web_analyzer::subdomain_takeover::check_subdomain_takeover;

#[tokio::main]
async fn main() {
    let domain = "example.com";

    // Domain intelligence
    let info = get_domain_info(domain, None).await.unwrap();
    println!("IP: {:?}", info.dns_info.a_records);

    // Security posture
    let security = analyze_security(domain, None).await.unwrap();
    println!("Grade: {} ({}/100)", security.grade, security.security_score);

    // Technology fingerprinting
    let tech = detect_web_technologies(domain).await.unwrap();
    println!("Server: {} | CMS: {:?}", tech.web_server, tech.cms);

    // Subdomain takeover check
    let subs = vec!["blog.example.com".into(), "shop.example.com".into()];
    let takeover = check_subdomain_takeover(domain, &subs, None).await.unwrap();
    println!("Vulnerable: {}", takeover.statistics.vulnerable_count);
}

---

⚙️ Feature Flags

Include only what you need:

[dependencies]
web-analyzer = { version = "0.1.8", features = ["domain-info", "security-analysis"] }

All feature flags

# Intelligence Gathering
domain-info = []
domain-dns = []
seo-analysis = []
web-technologies = []
domain-validator = []

# Reconnaissance
subdomain-discovery = []
contact-spy = []
advanced-content-scanner = []

# Security Assessment
security-analysis = []
subdomain-takeover = []
cloudflare-bypass = []
nmap-zero-day = []
api-security-scanner = []
geo-analysis = []

# Mobile Graceful Degradation Variants
domain-info-mobile = ["hickory-resolver", "x509-parser", "rustls"]
domain-dns-mobile = ["hickory-resolver"]
domain-validator-mobile = ["hickory-resolver", "x509-parser", "rustls"]
security-analysis-mobile = ["x509-parser", "rustls"]
subdomain-takeover-mobile = ["hickory-resolver"]

---

🔧 External Dependencies

Tool	Required By	Install
dig	domain_dns, domain_info, domain_validator, subdomain_takeover	sudo apt install dnsutils
nmap	nmap_zero_day	sudo apt install nmap
subfinder	subdomain_discovery	go install github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
openssl	security_analysis, domain_validator	sudo apt install openssl
whois	domain_info	sudo apt install whois

---

🏗️ Build

# All modules
cargo build --all-features

# Specific modules only
cargo build --features "domain-info,security-analysis,web-technologies"

# Release build
cargo build --all-features --release

# Run tests
cargo test --all-features

---

📁 Project Structure

web-analyzer/
├── Cargo.toml                        # Dependencies & feature flags
├── README.md                         # This file
├── src/
│   ├── lib.rs                        # Module registry (feature-gated)
│   ├── payloads.rs                   # Compile-time embedded payloads
│   │
│   │── domain_info.rs                # WHOIS, SSL, DNS, ports
│   │── domain_dns.rs                 # DNS record queries
│   │── seo_analysis.rs               # SEO (13 categories)
│   │── web_technologies.rs           # Tech fingerprinting (16 categories)
│   │── domain_validator.rs           # Bulk domain validation
│   │
│   │── subdomain_discovery.rs        # Subfinder integration
│   │── contact_spy.rs                # Contact info extraction
│   │── advanced_content_scanner.rs   # Secret & vuln scanning
│   │
│   │── security_analysis.rs          # Security posture assessment
│   │── subdomain_takeover.rs         # Takeover vulnerability detection
│   │── cloudflare_bypass.rs          # Origin IP discovery
│   │── nmap_zero_day.rs              # CVE & exploit detection
│   │── api_security_scanner.rs       # API vulnerability testing
│   │── geo_analysis.rs              # AI/LLM readiness
│   │
│   └── Mobile Polyfills/             # `*_mobile.rs` files for Android/iOS fallbacks
│
├── docs/                             # Module documentation (14 files)
│   ├── readme.md                     # Documentation index
│   └── [module_name].md
│
├── payloads/                         # Static payload files
└── tests/                            # Integration tests

🤝 OpenSource World Help Requests (Ideas & Features)

We are actively looking for contributors to help expand the engine into an enterprise-grade OSINT and exploitation framework. If you are a Rust developer or security researcher, please consider contributing to the following planned features:

Feature Name	Description	Request Document
Shodan & Censys Integration	Passive reconnaissance via external IoT search engines	Read Request
Cloud Bucket Enumerator	Brute-force exposed S3, Azure, and GCP storage buckets	Read Request
GitHub Secrets Dumper	Automated dorking for exposed organization credentials	Read Request
Archival Data Miner	Extract forgotten endpoints using Wayback Machine & OTX	Read Request
Dark Web Leak Monitor	Cross-reference domain emails against known credential breaches	Read Request
CT Log Subdomain Monitor	Real-time infrastructure discovery via Certificate Transparency logs	Read Request
JS Source Map Extractor	Decompile .js.map files to extract raw source code and secrets	Read Request
Web Cache Vulnerability Scanner	Detect Web Cache Deception & Poisoning misconfigurations	Read Request
WAF Evasion & Smuggling Tester	Automate HTTP Request Smuggling and path obfuscation vectors	Read Request
BGP Route Leak Monitor	Analyze ASN announcements for hijacking and routing anomalies	Read Request
GraphQL Introspection Fuzzer	Dump full API schemas and fuzz variables for IDOR mutations	Read Request
WebSocket Vulnerability Analyzer	Hook into WSS streams to test for Cross-Site WebSocket Hijacking	Read Request
Supply Chain Dependency Auditor	Hunt for Dependency Confusion vulns via leaked package.json	Read Request
Mobile API Endpoint Extractor	Parse raw APK DEX strings to uncover hidden staging API URLs	Read Request
Active Directory Exposure Scanner	Extract internal NetBIOS/Domain names via NTLM HTTP Challenges	Read Request

---

📊 Stats

Metric	Value
Total modules	15
Total Rust lines	6,725
Feature flags	14
Documentation files	14
Vulnerable services DB	36
WAF providers detected	8
Secret patterns	24
CMS platforms	11
Web servers	10

---

👤 Author

Keyvan Arasteh @keyvanarasteh

---

_{Built with 🦀 Rust — İstinye University}