🚀 SEO Indexer

High-performance, multi-engine SEO indexing CLI tool.

Automatically submit your sitemap URLs to Google, Bing, and Yandex — get your pages crawled and indexed faster.

---

✨ Features

Feature	Description
Google Indexing API	Submits URLs directly via the official Indexing API v3
Bing & Yandex Pings	Notifies search engines of sitemap updates via ping endpoints
IndexNow Protocol	Built-in support for the IndexNow bulk submission standard
Recursive Sitemap Parsing	Crawls sitemap index files and nested sitemaps automatically
Duplicate Prevention	SQLite-backed history database prevents re-submitting indexed URLs
Concurrency Control	Configurable parallel request limit to respect API rate limits
Progress Tracking	Real-time progress bar with per-URL status updates
Summary Reports	Aggregated statistics after each indexing run

📋 Prerequisites

• Node.js v18.0.0 or higher
• A Google Cloud service account with the Indexing API enabled
• Your site verified in Google Search Console

⚡ Quick Start

# Clone the repository
git clone https://github.com/kilicdev/seo-indexer.git
cd seo-indexer

# Install dependencies
npm install

# Add your Google service account credentials
cp /path/to/your/service_account.json ./service_account.json

# Run the indexer
npm start

🔧 Google Indexing API Setup

Step-by-step guide

1. Go to Google Cloud Console.
2. Create a new project (e.g., seo-indexer).
3. Navigate to API & Services → Library and enable the Web Search Indexing API.
4. Go to IAM & Admin → Service Accounts and create a new service account.
5. Under the Keys tab, generate a new key in JSON format.
6. Save the downloaded file as service_account.json in the project root.
7. In Google Search Console, add the service account's client_email as an Owner of your property.

Note: The service_account.json file contains sensitive credentials. It is excluded from version control via .gitignore. Never commit this file to a public repository.

🖥️ Usage

npm start

The interactive CLI will prompt you for:

? Enter your sitemap URL: https://example.com/sitemap.xml
? Concurrent request limit: 5

What Happens

1. Crawl — Recursively parses your sitemap (supports sitemap index files)
2. Filter — Checks the local database and skips previously indexed URLs
3. Index — Submits new URLs to Google via the Indexing API
4. Ping — Notifies Bing and Yandex of your sitemap update
5. Report — Displays aggregated results

Example Output

╔══════════════════════════════════════════════╗
║           SEO INDEXER by kilicdev            ║
║        Multi-Engine Indexing Tool v1         ║
╚══════════════════════════════════════════════╝

✔ Found 142 URLs.
✔ 38 new URLs to index. (104 skipped)

=== GOOGLE INDEXING API ===
Progress |████████████████████| 100% || 38/38 URLs

✔ Google Indexing complete.

=== BING & YANDEX SITEMAP PING ===
✔ Bing and Yandex pings sent successfully.

=== SUMMARY REPORT ===
┌─────────┬──────────┬─────────┬───────┐
│ (index) │  engine  │ status  │ count │
├─────────┼──────────┼─────────┼───────┤
│    0    │ 'google' │ 'success' │  38  │
└─────────┴──────────┴─────────┴───────┘

✔ All operations completed. Database updated.

📁 Project Structure

seo-indexer/
├── src/
│   ├── index.js              # CLI entry point & orchestration
│   ├── core/
│   │   ├── crawler.js        # Recursive sitemap parser
│   │   └── db.js             # SQLite persistence layer
│   ├── engines/
│   │   ├── google.js         # Google Indexing API v3 engine
│   │   ├── indexnow.js       # IndexNow protocol engine
│   │   └── pings.js          # Legacy ping engine (Bing/Yandex)
│   └── utils/
│       └── logger.js         # Colored console logging
├── package.json
├── LICENSE
├── CONTRIBUTING.md
└── .gitignore

🔌 Supported Engines

Engine	Method	Status
Google	Indexing API v3	✅ Active
Bing	Sitemap Ping	✅ Active
Yandex	Sitemap Ping	✅ Active
IndexNow	IndexNow Protocol	🔧 Available (requires API key)

📊 How It Works

┌─────────────┐     ┌──────────────┐     ┌───────────────┐
│   Sitemap    │────▶│   Crawler    │────▶│   URL Filter  │
│   (XML)      │     │  (Recursive) │     │  (SQLite DB)  │
└─────────────┘     └──────────────┘     └──────┬────────┘
                                                 │
                              ┌──────────────────┼──────────────────┐
                              ▼                  ▼                  ▼
                     ┌────────────┐     ┌────────────┐     ┌────────────┐
                     │   Google   │     │    Bing    │     │   Yandex   │
                     │ Indexing   │     │   Ping     │     │   Ping     │
                     │   API v3   │     │            │     │            │
                     └────────────┘     └────────────┘     └────────────┘

🤝 Contributing

Contributions are welcome! Please read the Contributing Guide for details on the process for submitting pull requests.

📜 License

This project is licensed under the MIT License — see the LICENSE file for details.

💬 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions