RouteFluxMap - Tor Network Visualization

A modern, real-time visualization of the Tor network showing relay bandwidth and data flow patterns.

🚀 Features

• Interactive Map: Explore Tor relays worldwide with WebGL-powered visualization
• Particle Flow Animation: Watch simulated traffic flow between relays
• Historical Data: Navigate through historical snapshots of the network
• Country Statistics: Click on countries to see connection statistics and outliers
• Mobile Friendly: Responsive design works on all devices
• Zero Maintenance: Static site with automated data updates

🛠 Tech Stack

Layer	Technology
Framework	Astro
Interactive UI	React
Visualization	Deck.gl
Maps	MapLibre GL
Styling	Tailwind CSS
Hosting	Cloudflare Pages
Data Storage	Cloudflare R2
Data Pipeline	Local Cron Job

📦 Quick Start

Prerequisites

• Node.js 20+
• npm (or pnpm/yarn)

Development

# Clone the repository
git clone https://github.com/1aeo/routefluxmap.git
cd routefluxmap

# Install dependencies
npm install

# Start development server
npm run dev

Visit http://localhost:4321 to see the app.

Build

# Build for production
npm run build

# Preview production build
npm run preview

🔧 Configuration

All configuration is in deploy/config.env:

cp deploy/config.env.template deploy/config.env
nano deploy/config.env

Key settings:

# Storage priority (first = primary for frontend, second = fallback)
STORAGE_ORDER=do,r2

# Site URLs
PUBLIC_SITE_URL=https://your-site-url
PUBLIC_METRICS_URL=https://your-metrics-url

# Storage credentials
DO_SPACES_BUCKET=your-bucket
DO_SPACES_REGION=nyc3
R2_BUCKET=your-bucket
R2_CUSTOM_DOMAIN=data.yourdomain.com

The frontend automatically tries the primary storage first, then falls back to the secondary if primary fails.

📊 Data Pipeline

The data is fetched hourly from the Tor Onionoo API via a local cron job:

1. Fetch: Download relay data from Onionoo
2. Geolocate: Look up IP coordinates using MaxMind GeoLite2
3. Aggregate: Group relays by location
4. Upload: Store processed JSON in Cloudflare R2

Manual Data Fetch

# Fetch relay data + country data + geolocate (all in one)
npm run fetch-data

# Upload to R2 and/or DO Spaces
./deploy/scripts/update.sh

Historical Data Processing

The script can fetch historical data from Tor Collector archives:

# Fetch a specific day (mm/dd/yy)
npx tsx scripts/fetch-all-data.ts 12/07/25

# Fetch an entire month (mm/yy)
npx tsx scripts/fetch-all-data.ts 11/25

# Fetch an entire year (yy)
npx tsx scripts/fetch-all-data.ts 25

# With custom parallelism and thread settings
npx tsx scripts/fetch-all-data.ts 11/25 --parallel=4 --threads=2

Resource Requirements

Historical data processing downloads and parses large archive files (~400MB compressed). Resource usage depends on the --threads setting:

Setting	RAM Usage	Speed	Notes
--threads=1	~4GB	Slowest	Minimum memory systems
--threads=4	~6GB	Fast	Default, recommended
--threads=0	~9GB	Fast	All CPU cores (same speed as -T4)

Typical processing times:

• 1 day: ~3 seconds
• 1 month: ~70 seconds (first run), ~23 seconds (cached)
• 1 year: ~15 minutes (first run)

Caching: The script caches bandwidth data per month (~500KB JSON). Subsequent runs for the same month are ~3x faster.

Low-memory systems: Use --threads=1 --parallel=1 for systems with <8GB RAM.

🚀 Deployment

Static Site (Cloudflare Pages)

Deploy the frontend using the local Wrangler-based deploy (allium-deploy style):

# One-time setup (on deploy server)
./deploy/scripts/pages-setup.sh

# Edit credentials
nano deploy/config.env

# Deploy
./deploy/scripts/pages-deploy.sh

# Or via npm
npm run deploy:pages

See deploy/README.md for detailed setup.

Setup Guides

See docs/setup/ for detailed guides:

• Cloudflare Pages Deploy - Local Wrangler deploy
• MaxMind GeoIP
• Cron Setup

🏗 Project Structure

routefluxmap/
├── src/
│   ├── components/      # React components
│   │   ├── map/         # Map visualization
│   │   └── ui/          # UI controls
│   ├── lib/             # Utilities and config
│   ├── layouts/         # Astro layouts
│   ├── pages/           # Routes
│   └── styles/          # Global CSS
├── public/              # Static assets
├── scripts/             # Data fetch scripts
├── deploy/              # All deployment (data + static site)
│   ├── scripts/         # Data upload + Pages deploy scripts
│   ├── pages/           # CF Pages _headers, _redirects
│   └── config.env       # All credentials (gitignored)
├── docs/                # Documentation
│   ├── setup/           # Setup guides
│   └── features/        # Feature specs
└── tests/               # Unit tests

🗺 Architecture

┌─────────────────────────────────────────────────────────┐
│  Cloudflare Pages (Static Site)                         │
│  └── Astro + React + Deck.gl + MapLibre                │
└────────────────────────┬────────────────────────────────┘
                         │ fetch JSON
                         ▼
┌─────────────────────────────────────────────────────────┐
│  Cloudflare R2 (Data Storage)                           │
│  ├── index.json       # Date index                      │
│  ├── current/*.json   # Daily relay snapshots           │
│  └── geo/*.json       # Country boundaries              │
└────────────────────────┬────────────────────────────────┘
                         ▲ hourly upload
                         │
┌─────────────────────────────────────────────────────────┐
│  Local Cron Job (Data Pipeline)                         │
│  └── Fetch Onionoo → GeoIP → Aggregate → Upload        │
└─────────────────────────────────────────────────────────┘

📜 License

This project is licensed under the Apache License 2.0.

🙏 Credits

• Originally created by Uncharted Software (2015)
• Modernized by the RouteFluxMap community (2025)
• Data from The Tor Project
• GeoIP data from MaxMind
• Map tiles from CartoDB

🔗 Links

• Live Demo
• Original TorFlow
• Tor Project
• Onionoo API