An end-to-end data engineering and supervisory analytics platform simulating RBI-grade consumer protection and market intelligence operations.
FinCompass is a full-stack data engineering and analytics platform that simulates the Reserve Bank of India's (RBI) consumer protection and market intelligence operations end-to-end. It ingests synthetic grievance records, cleans and structures them into a relational database, applies statistical hypothesis testing and time-series forecasting, trains an NLP classifier on raw complaint text, builds a vector search index for semantic Q&A, and serves everything through an interactive 6-page Streamlit supervisory dashboard — complete with automated PDF/Word/Excel brief generation.
This project was purpose-built to demonstrate the core competencies required for RBI Young Professional (YP) roles in Data Analytics and Policy Research: data pipeline engineering, statistical inference, forecasting, applied ML/NLP, and translating quantitative findings into policy-ready supervisory briefs.
┌───────────────────────────┐
│ 15,000 Complaints │
│ (Synthetic Raw Data) │
└──────────────┬──────────────┘
│ ETL Pipeline (pandas & NumPy)
▼
┌───────────────────────────┐
│ clean_validate.py │
│ (Z-score outliers, DQ) │
└──────────────┬──────────────┘
│ Ingestion (SQLAlchemy)
▼
┌───────────────────────────┐
│ fincompass.db │
│ (SQLite3 Relational DB) │
└──────┬─────────────┬────────┘
│ │
ML & NLP ▼ ▼ Analytics & Time-Series
┌───────────────────────┐ ┌───────────────────────┐
│ train_classifier.py │ │ stats_analysis.py │
│ (TF-IDF + LogReg) │ │ (Welch's t-test, OLS) │
└────────────┬──────────┘ └────────────┬──────────┘
▼ ▼
┌───────────────────────┐ ┌───────────────────────┐
│ complaint_classifier │ │ forecasting.py │
│ (Saved .pkl) │ │ (SARIMA Forecast JSON) │
└────────────┬──────────┘ └────────────┬──────────┘
│ │
└─────────────┬──────────────┘
▼
┌───────────────────────────┐
│ streamlit_app │◀──┐ RAG Assistant
│ (6-Page App) │ │
└──────────────┬──────────────┘ │
│ │
▼ ┌──────────┴──────────┐
┌───────────────────────┐ │ chatbot.py │
│ Word / Excel MIS │ │ (LangChain + Chroma) │
│ Generators │ └──────────┬──────────┘
└───────────────────────┘ │
▼
┌─────────────────────┐
│ Gemini Flash API │
└─────────────────────┘
| Layer | Technologies | Purpose |
|---|---|---|
| 🐍 Core Language | Python 3.10+ | Primary language for pipeline, analytics, and modeling |
| 🗄️ Database | SQLite3, SQLAlchemy | Relational storage with window functions, JOINs |
| ⚙️ Pipeline | Pandas, NumPy | Structured ETL validation, cleaning, log generation |
| 📊 Statistics | SciPy, StatsModels | Welch's t-test, OLS regression, Mann-Kendall trend test |
| 📈 Time-Series | statsmodels SARIMAX | 6-month seasonal complaint volume forecasting |
| 🤖 ML / NLP | scikit-learn | TF-IDF pipeline + Multinomial Logistic Regression |
| 🧠 GenAI / RAG | LangChain, ChromaDB, Gemini Flash | Semantic vector search & Q&A over supervisory briefs |
| 📝 Automation | python-docx, openpyxl, APScheduler | Automated Word briefs, conditional-formatted Excel MIS |
| 🎨 Visualization | Plotly, Streamlit | Interactive multi-page dashboard |
Synthetic data is modeled to reflect realistic system loads under the Integrated Ombudsman Scheme, rather than uniform random distributions:
- Market share weighting — SBI accounts for ~20% of complaints, matching its domestic footprint, followed by public sector peers.
- Macroeconomic trends — Digital Banking Fraud is modeled with a 25% YoY compound growth rate, simulating post-COVID digital transaction growth.
- Supervisory discrepancy — Public sector banks are assigned a gamma distribution for
resolution_days, yielding a ~43-day mean (~40% slower than private counterparts).
- Outliers in
resolution_daysflagged via a standard Z-score threshold (Z > 3). - Inconsistencies repaired — e.g.,
resolution_daysset to null for pending complaints, and resolved entries missing day counts filled with the bank type's median. - Full execution steps and metrics exported to
etl/etl_log.txt.
- Welch's t-test — independent two-sample test with unequal variances, comparing public vs. private bank resolution days.
- OLS Regression — predicts resolution days from bank type, channel, and complaint category to isolate systemic friction points.
- Mann-Kendall Test — non-parametric test for a monotonic upward trend in Digital Banking Fraud.
- Python 3.10+
# 1. Clone the repository
git clone https://github.com/yourusername/FinCompass.git
cd FinCompass
# 2. Install dependencies
pip install -r requirements.txt
# 3. Run the end-to-end orchestrator
# (data generation → validation → DB load → ML training → stats → vector store → MIS generation)
python run_all.pyCreate a .env file at the project root:
GEMINI_API_KEY=your_google_studio_api_key_here
If no key is provided, the chatbot gracefully falls back to a local semantic synthesis mode.
streamlit run app/streamlit_app.py| Finding | Statistical Evidence |
|---|---|
| 🏛️ Systemic backlogs in public sector banks | Welch's t-test: p < 0.001 (null hypothesis rejected) |
| 📱 Digital Banking Fraud rising sharply | Mann-Kendall Tau: +0.84, p < 0.0001 |
| 🏢 Branch-filed complaints add friction | OLS regression: +14.2 days vs. online filings |
I. Institutional Action Plan The persistent gap in grievance redressal velocity between public and private sector banks points to a structural bottleneck in consumer protection. The analysis recommends the Department of Supervision (DoS) mandate specialized workflow automation for public sector banks whose average resolution time exceeds a 45-day threshold, alongside periodic spot audits of internal bank ombudsman frameworks.
II. Mitigation of Digital Transaction Risks Given the statistically validated acceleration in Digital Banking Fraud, the analysis suggests a regulatory update to the Master Direction on Digital Banking Security Controls — including mandatory real-time transaction cooling periods for high-risk accounts and stricter security audits for third-party UPI integrations, with particular focus on Small Finance Banks showing sudden QoQ complaint spikes.
| # | Page | Description |
|---|---|---|
| 1 | Executive Dashboard | High-level KPIs and complaint volume overview |
| 2 | Complaint Deep Dive | Real-time NLP classifier on raw complaint text |
| 3 | DoS Supervisory Monitoring | Bank-level resolution velocity tracking |
| 4 | Heatmap & Geographic Analysis | Regional complaint density visualization |
| 5 | Time-Series Forecasting & Stats | SARIMA forecasts and hypothesis test results |
| 6 | AI Policy Assistant | RAG-powered chat over monthly supervisory briefs |
📸 Replace this table with actual screenshots/GIFs per page once available — a 2-column image grid works well here.
- Swap synthetic data generator for anonymized real-world grievance datasets
- Add role-based access control for the supervisory dashboard
- Expand forecasting to per-bank and per-region granularity
- CI/CD pipeline for automated model retraining
- Deploy live demo (Streamlit Cloud / Docker)
This project uses entirely synthetic data and is built as an independent portfolio/demonstration project. It is not affiliated with, endorsed by, or representative of official RBI systems, data, or policy positions.
Contributions, issues, and feature requests are welcome! Feel free to check the issues page.
This project is licensed under the MIT License — see the LICENSE file for details.
Built with 🛡️ for data-driven supervisory intelligence.