Problem: Current single-threaded log processing is slow for large files and lacks detailed per-stage progress visibility.
Solution: Enhance backend with parallel chunk processing (4 workers). Add pipeline visualization showing individual stages: File Loading → Parsing → Hash Calculation → Anomaly Detection → Report Generation. Display real-time per-stage progress percentages, estimated completion time, and worker utilization. Add cancel button for long-running jobs.
Files Modified: main.py, logic.py, js/pipelineProgress.js, js/workerStatus.js, styles/progress.css
Expected Outcome: 3x faster processing for large log files with transparent progress tracking.
@sutanjoyb Please assign this under NSOC 2026 (Level 3)
Problem: Current single-threaded log processing is slow for large files and lacks detailed per-stage progress visibility.
Solution: Enhance backend with parallel chunk processing (4 workers). Add pipeline visualization showing individual stages: File Loading → Parsing → Hash Calculation → Anomaly Detection → Report Generation. Display real-time per-stage progress percentages, estimated completion time, and worker utilization. Add cancel button for long-running jobs.
Files Modified:
main.py,logic.py,js/pipelineProgress.js,js/workerStatus.js,styles/progress.cssExpected Outcome: 3x faster processing for large log files with transparent progress tracking.
@sutanjoyb Please assign this under NSOC 2026 (Level 3)