Read/Write Monitor Dashboard

This project provides a simple monitoring system to track and visualize the number of read and write operations performed by different applications. It includes collectors for gathering events, a storage layer for persistence, and a dashboard for visualization.

---

Core Features

• File Collector: Monitors file read/write operations.
• Database Collector: Wraps around DB queries and logs operations.
• API Collector: Provides an endpoint to log external read/write operations.
• Persistent Storage: SQLite database for event storage.
• ORM Integration: SQLAlchemy ORM for managing models.
• Interactive Dashboard: Built with Streamlit for interactive visualization.
• Data Analytics: Provides charts for operation counts, application-wise statistics, and trends over time.

---

Installation & Setup

1. Clone the repository.
2. Install dependencies.
3. Initialize the database.
4. Run the application.
5. Start the main service, which boots up both the data collectors and the visualization dashboard.

---

Usage Guide

By default, the services will spin up on the following local ports:

• API Ingestion Endpoint: http://127.0.0.1:6000.
• Streamlit Dashboard: http://localhost:8501.

Integrate the FileCollector or DBCollector directly into your Python scripts to start tracking local data, or send HTTP requests to the APICollector to log events from external microservices. Navigate to the Streamlit URL in your browser to view the real-time analytics.

---

Technology Stack

Domain	Technology	Purpose
Language	Python	Core logic and scripting.
API Framework	Flask	Ingestion endpoints for external logs.
Frontend UI	Streamlit	Data visualization and dashboarding.
Database	SQLite & SQLAlchemy	Persistent event storage and ORM.
File Monitoring	Watchdog	Real-time file system event tracking.

---

Project Structure

readwrite-monitor-dashboard/
│── main.py              # Entry point: starts collectors and dashboard
│── requirements.txt     # Python dependencies
│── config.py            # Configuration (DB path, ports, etc.)
│
├── collector/
│   │── __init__.py
│   │── base_collector.py
│   │── file_collector.py
│   │── db_collector.py
│   │── api_collector.py
│
├── storage/
│   │── __init__.py
│   │── models.py        # SQLAlchemy models
│   │── db.py            # DB initialization and session handling
│
└── dashboard/
    │── __init__.py
    │── app.py           # Streamlit dashboard
    │── charts.py        # Chart utilities

---

---

Roadmap for Future Improvements

• Integrate cloud storage collectors for AWS S3, Google Cloud Storage, and Azure Blob.
• Implement authentication and role-based access control (RBAC) for the dashboard.
• Add functionality to export data reports in CSV, Excel, and PDF formats.
• Upgrade to WebSocket support for real-time, low-latency data streaming.

---

Author: Shreeja Karajagi, E. Saaruvathani, and Nachikaet Anoop.

Contributions, issues, and feature requests are welcome! Feel free to open an issue or submit a pull request.