Distriby

A modern, distributed CDN solution for file storage and delivery.
Optimized for speed, security, and scalability, Distriby seamlessly handles global file distribution with built-in auto-optimization and multi-node synchronization.

---

Table of Contents

1. Features
2. Getting Started
   • Requirements
   • Installation
3. API Endpoints
4. How It Works
5. Architecture Diagram
6. Contributing
7. License
8. Documentation

---

Features

• 🚀 Global File Distribution: Serve files from the node closest to the user, reducing latency with DNS Anycast (Cloudflare or internal management).
• 🛡️ Security at Its Core:
  • Protection against DDoS and unauthorized file access.
  • Configurable support for Cloudflare with dynamic activation.
• ⚡ File Auto-Optimization:
  • Compression for assets (Brotli, Gzip).
  • Image conversion to WebP.
  • Optional video/audio transcoding with FFmpeg.
• 🔄 Multi-Node Synchronization:
  • Sub-nodes automatically replicate optimized files from the primary node using gRPC.
  • Configurable node health monitoring and synchronization.
• 📡 REST API: Upload, delete, retrieve, and monitor files seamlessly.
• 📈 Monitoring: View the status of all nodes, including disk usage, uptime, and traffic.
• 🌐 DNS Anycast: Seamless routing to the closest node, with or without Cloudflare.

---

Getting Started

Requirements

1. Go (v1.23+)
2. FFmpeg (for video/audio transcoding)
3. cwebp (for image optimization)
4. gzip (for text compression)
5. Cloudflare account (optional, for DNS Anycast)

---

Installation

Step 1: Clone the repository

git clone https://github.com/YuketsuSh/distriby.git
cd distriby

Step 2: Build the project

Make sure Go is installed, then build the project:

go build -o distriby

Step 3: Configure environment

Create a .env file based on the provided .env.example:

cp .env.example .env

Set the following variables:

NODE_ENV=development
PORT=8080
GRPC_PORT=9090
USE_CLOUDFLARE=false
AUTH_SECRET=my_secure_token
DEBUG_MODE=true

NODE_TYPE=primary      # "primary" for main node or "secondary" for sub-node
SYNC_ENABLED=true      # Enable/disable synchronization for subnodes

Step 4: Start the server

./distriby

---

API Endpoints

Details about the API, including usage and examples, have been moved to the DOCUMENTATION.md.

---

How It Works

1. Primary Node:

   • Centralized control for managing files.
   • Handles file uploads, deletions, and retrievals.
   • Optimizes files automatically before synchronization.
   • Monitors and synchronizes active secondary nodes.

2. Secondary Nodes:

   • Receive optimized files via gRPC from the primary node.
   • Serve files directly to clients based on their proximity.

3. File Optimization:

   • Images are converted to WebP using cwebp
   • Text file are compressed using gzip
   • Videos are transcoded using FFmpeg (H.264/AAC)
   • Audio files are transcoded to MP3 format

4. DNS Anycast:

   • If Cloudflare is enabled, traffic is routed to the nearest node.
   • If Cloudflare is disabled, Distriby uses internal routing based on IP geolocation.

5. Monitoring:

   • The primary node tracks the health and status of all sub-nodes.

6. Configuration Persistence:

   • The primary node maintains node configurations in a nodes_config.json file.
   • This file is created automatically if it doesn't exist.

---

Architecture Diagram

+---------------------+
|     Client/User     |
+---------------------+
           |
           v
+---------------------+
|   Cloudflare DNS    |
+---------------------+
           |
           v
+-------------------------------+
| Distriby Primary Node         |
| - API for file management     |
| - File optimization (Brotli)  |
| - Compression (WebP)          |
+-------------------------------+
           |
           v
+-------------------------------+
| Distriby Sub-Nodes (Region X) |
| - File syncing with Rclone    |
| - Local file serving          |
+-------------------------------+

---

Contributing

Although Distriby is a private project for Velyorix, internal contributions are welcome. If you are part of the team and want to improve Distriby:

Steps to Contribute

1. Fork the repository.
2. Create a new branch (git checkout -b feature-name).
3. Commit your changes (git commit -m "Add new feature").
4. Push to your branch (git push origin feature-name).
5. Open a Pull Request.

---

License

Distriby is a private project for Velyorix. Redistribution is restricted unless authorized.

---

Documentation

Full documentation for Distriby, including API details and advanced configurations, can be found in the DOCUMENTATION.md.

---

Quick Links

• Documentation
• Issue Tracker
• Contributing Guidelines