Concurrent Log Processing System

A Rust application simulating a concurrent backend log processing pipeline. Built as a learning exercise for concurrent programming concepts.

What it does

Simulates 5 independent services generating log entries simultaneously and feeding them into a shared in-memory database for analysis.

Pipeline

[5 Producer Threads] --> mpsc channel --> [Main Thread] --> RwLock<HashMap> --> [Rayon Report]

1. 5 producer threads each generate 20 random log entries (Info, Warning, Error) and send them through a channel
2. Main thread receives all entries and writes them into a shared database organized by service ID
3. Rayon processes the database in parallel to generate the final report

Concurrency concepts used

Concept	Usage
std::thread	Producer threads
mpsc::channel	Communication from producers to main thread
Arc<RwLock<T>>	Shared database — multiple parallel reads, exclusive writes
Arc<AtomicUsize>	Lock-free counter for total entries processed
rayon	Parallel iteration over the database for the report

Output example

Service 0: 20 entries, 7 errors
Service 1: 20 entries, 6 errors
Service 2: 20 entries, 8 errors
Service 3: 20 entries, 5 errors
Service 4: 20 entries, 7 errors
Services with most errors (8): [2]
Total processed: 100

How to run

Requirements: Rust installed — rustup.rs

git clone <your-repo-url>
cd big_exercise
cargo run

Dependencies

• rayon — data parallelism
• rand — random log level generation