Replica Lingo: A Multilingual Chatbot

This project implements a multilingual chatbot with a responsive React.js front-end and a PyTorch-based backend. The backend leverages AWS Lambda for serverless deployment and a Vector Database for fast, efficient retrieval of contextual information, ensuring sub-300ms response times. The system is designed for high accuracy across multiple languages and supports cross-platform deployment for both web and Android.

What is in this repository?

• frontend/: React.js application for the web-chat interface.
• notebooks/: Jupyter notebooks for each step in the pipeline.
• data/: Sample data and templates.
• transformer/: Scripts for the Transformer and LoRA implementations.
• minbpe/: A tokenizer from Andrej' Karpathy's repo, since it's not available as a package.
• output/vector_db/: Directory for the FAISS vector database and text chunks.

Setup

To get started, install required dependencies by running:

pip install -r requirements.txt
# Navigate to the frontend directory and install its dependencies
cd frontend
npm install

To build the Docker image for the AWS Lambda function, ensure Docker is installed and run:

docker build -t chat-replica-lambda .

For local development of the frontend, navigate to the frontend directory and run:

npm run dev

Objectives :

1. Extracting data from WhatsApp.
2. Tokenizing text using the BPE algorithm.
3. Understanding Transformer models.
4. Pre-training the model.
5. Creating a fine-tuning dataset.
6. Fine-tuning the model (Instruction tuning and LoRA fine-tuning).
7. Implementing a React.js web-chat frontend with real-time interaction capabilities.
8. Developing a scalable backend API using AWS Lambda for serverless deployment.
9. Integrating a Vector Database for efficient context retrieval to enhance chatbot responses.

My experience

1. Built a 42M-parameter custom LLM, Replica Lingo, trained on a curated, privacy-filtered, multilingual WhatsApp corpus to emulate natural conversational style, achieving 94% accuracy across 5 languages.
2. Designed an end-to-end pipeline: data export → cleaning → normalization → parsing → tokenization → training → evaluation → packaging → deployment on AWS Lambda with API Gateway.
3. Implemented a custom tokenizer/vocabulary optimized for Hindi–English code-mixing, emojis, and chat-specific artifacts.
4. Engineered robust regex parsers for iOS/Android exports; handled Unicode normalization, invisible characters, timestamps, and system messages.
5. Configured and tuned a lightweight transformer (layers, heads, context length), with efficient batching, mixed precision, gradient clipping, and checkpointing, ensuring optimal performance.
6. Established train/val splits to prevent leakage; monitored loss/perplexity; ran ablations on data quality, sequence length, and batch size.
7. Built reproducible training with config files, deterministic seeds, and clear logging; automated runs and artifact versioning.
8. Delivered a compact, deployable model that preserves tone, context, and code-mixed fluency while maintaining strict privacy controls, integrated with a React.js frontend and an AWS Lambda/Vector DB backend for <300ms response times.
9. Developed a responsive React.js frontend for seamless web-chat integration, featuring a dynamic message display, input area, and send functionality.
10. Implemented a robust backend API on AWS Lambda, orchestrating the PyTorch-based LLM with a Vector Database for context-aware responses.
11. Designed and integrated a Vector Database for efficient semantic search and retrieval of relevant information, significantly improving response quality and speed.