🧠 ReviewMind

Production-Ready Aspect-Based Sentiment Analysis & Rating Prediction Platform

An AI-powered system that analyzes user-generated reviews to extract product aspects, determine sentiment per aspect, and predict overall ratings with state-of-the-art accuracy.

---

📋 Table of Contents

• Overview
• Key Features
• Architecture
• Installation
• Quick Start
• Model Details
• API Documentation
• Training & Fine-Tuning
• Dataset
• Project Structure
• Performance Metrics
• Development
• Documentation
• Roadmap
• Contributing
• License

---

🎯 Overview

ReviewMind is an end-to-end Aspect-Based Sentiment Analysis (ABSA) system that combines cutting-edge Large Language Models with traditional machine learning techniques to provide comprehensive review analysis.

What It Does

Given a user review like:

"The product quality is amazing and delivery was super fast,
but the packaging was damaged."

ReviewMind outputs:

{
  "aspects": [
    { "aspect": "quality", "sentiment": "positive" },
    { "aspect": "delivery", "sentiment": "positive" },
    { "aspect": "packaging", "sentiment": "negative" }
  ],
  "rating": 3.8
}

Use Cases

• 🛒 E-commerce: Analyze product reviews to identify strengths and weaknesses
• 📱 SaaS: Monitor customer feedback for feature-specific sentiment
• 🏨 Hospitality: Track service quality across multiple dimensions
• 📊 Market Research: Extract actionable insights from customer feedback
• 🔍 Competitive Analysis: Compare sentiment across product categories

---

✨ Key Features

🤖 Hybrid AI Architecture

• Fine-tuned LLM: Qwen2.5-7B with LoRA/QLoRA for aspect extraction and sentiment classification
• Ensemble ML: XGBoost + CatBoost + LightGBM with Ridge meta-learner for rating prediction
• Multi-task Learning: Joint training for aspect detection, sentiment analysis, and rating regression

🚀 Production-Ready

• FastAPI Backend: RESTful API with < 2s inference latency
• Modular Design: Separated concerns for easy maintenance and extensibility
• Configuration-Driven: YAML-based hyperparameter management
• Type-Safe: Dataclass validation throughout the pipeline
• Comprehensive Logging: Structured logging for debugging and monitoring

📊 Advanced Feature Engineering

• 43+ Features: Aspect-based, text-based, quality-based, and interaction features
• Configurable Lexicons: Domain-specific sentiment dictionaries
• Quality Scoring: Automatic LLM output quality assessment
• Weighted Predictions: Confidence-based sample weighting

🎓 Research-Grade

• Reproducible: Fixed random seeds, versioned datasets, documented experiments
• Well-Documented: Complete SRS, MDD, DRS, and training plans
• Extensible: Easy to add new models, features, or evaluation metrics
• Transparent: Semi-supervised annotation strategy with human validation

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                        Frontend (Next.js)                    │
│                  [In Development - Coming Soon]              │
└───────────────────────────────┬─────────────────────────────┘
                                │ REST API
┌───────────────────────────────▼─────────────────────────────┐
│                     FastAPI Backend                          │
│  ┌────────────────────────────────────────────────────────┐ │
│  │           ReviewMind Pipeline Orchestrator             │ │
│  └────────────────────────────────────────────────────────┘ │
│         │                              │                     │
│         ▼                              ▼                     │
│  ┌─────────────────┐          ┌────────────────────┐       │
│  │  LLM Component  │          │  Ensemble ML       │       │
│  │  Qwen2.5-7B     │          │  Rating Predictor  │       │
│  │  + LoRA         │          │  XGB+Cat+LGB+Ridge │       │
│  └─────────────────┘          └────────────────────┘       │
│         │                              │                     │
│         ▼                              ▼                     │
│  ┌──────────────────────────────────────────────────────┐  │
│  │           Feature Extraction & Processing            │  │
│  │  • Aspect Parser  • Text Features  • Quality Score   │  │
│  └──────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Component Breakdown

1. API Layer (api/): FastAPI endpoints for single and batch inference
2. Pipeline Layer (src/pipeline/): Orchestrates LLM and ensemble models
3. Model Layer (src/model/): Feature extraction, training, and evaluation
4. Inference Layer (src/inference/): Production-ready prediction interface
5. Data Layer (src/data/): Data loading, preprocessing, and validation
6. Training Layer (src/training/): Model fine-tuning and ensemble training

---

📦 Installation

Prerequisites

• Python: 3.10 or higher
• Hardware:
  • Apple Silicon (M1/M2/M3/M4) with Metal support (recommended)
  • OR NVIDIA GPU with CUDA support
  • Minimum 16GB RAM (24GB recommended for training)
• OS: macOS, Linux, or Windows with WSL2

Clone Repository

git clone https://github.com/barandincoguz/SentimentAnalysis.git
cd ReviewMind\ Sentiment\ Analysis

Install Dependencies

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install requirements
pip install -r requirements.txt

# Install PyTorch (for Apple Silicon)
pip3 install torch torchvision torchaudio

# Install PyTorch (for CUDA)
# pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Download Models

# Download base Qwen2.5-7B model
python scripts/download_model.py

# Or manually download fine-tuned model from Hugging Face
# Model will be loaded automatically during first inference

---

🚀 Quick Start

Start API Server

# Start FastAPI server
cd api
uvicorn main:app --reload --host 0.0.0.0 --port 8000

Access API documentation at http://localhost:8000/docs

Make Predictions

Using Python SDK

from src.pipeline.run_pipeline import ReviewMindPipeline

# Initialize pipeline
pipeline = ReviewMindPipeline()

# Analyze a review
review_text = "Amazing product! Quality is top-notch but delivery was slow."
result = pipeline.predict(review_text)

print(f"Rating: {result['rating']}⭐")
print("Aspects:")
for aspect in result['aspects']:
    print(f"  - {aspect['aspect']}: {aspect['sentiment']}")

Using API (curl)

curl -X POST "http://localhost:8000/analyze" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Great quality but shipping was terrible!",
    "include_aspects": true
  }'

Using API (Python requests)

import requests

response = requests.post(
    "http://localhost:8000/analyze",
    json={"text": "Excellent product, highly recommend!"}
)
result = response.json()
print(result)

Batch Processing

reviews = [
    {"text": "Great product!"},
    {"text": "Poor quality, very disappointed."},
    {"text": "Good value for money."}
]

response = requests.post(
    "http://localhost:8000/analyze/batch",
    json={"reviews": reviews}
)
results = response.json()

---

🤖 Model Details

LLM Component: Qwen2.5-7B-Instruct

Base Model: Qwen/Qwen2.5-7B-Instruct

Fine-Tuning Strategy:

• Method: Parameter-Efficient Fine-Tuning (PEFT) with LoRA
• LoRA Configuration:
  • Rank (r): 16
  • Alpha: 32
  • Target modules: q_proj, v_proj
  • Dropout: 0.05
• Quantization: 4-bit QLoRA for efficient training on consumer hardware
• Precision: bfloat16 computation

Training Configuration:

• Batch size: 1-2 with gradient accumulation
• Epochs: 3-5
• Learning rate: 2e-4 with cosine scheduler
• Optimizer: AdamW
• Warmup: 5% of total steps

Model Capabilities:

• Structured JSON output generation
• Multi-task learning (aspect extraction + sentiment classification)
• Multilingual support (English + Turkish)
• Instruction-following with zero-shot generalization

Ensemble Rating Predictor

Architecture: Stacked Ensemble with Meta-Learning

Base Models:

1. XGBoost Regressor
   • RandomizedSearchCV for hyperparameter tuning
   • 20 iterations, 5-fold cross-validation
   • Objective: reg:squarederror
2. CatBoost Regressor
   • MAE loss function
   • Early stopping (50 rounds)
   • Max depth: 6, Iterations: 1000
3. LightGBM Regressor
   • Leaf-wise growth strategy
   • Early stopping (50 rounds)
   • Num leaves: 31, Iterations: 1000

Meta-Learner: Ridge Regression (α=0.5)

• Combines predictions from base models
• Trained on out-of-fold predictions
• Prevents overfitting through regularization

Feature Engineering (43 features):

• Aspect Features (8): Positive/negative/neutral counts, ratios, net score, entropy
• Text Features (25): Sentiment keywords, punctuation signals, linguistic markers
• Quality Features (6): Generic aspect detection, diversity scores
• Interaction Features (4): Sentiment alignment, confidence scores

---

📡 API Documentation

Endpoints

POST /analyze

Analyze a single review

Request:

{
  "text": "Amazing product!",
  "include_aspects": true
}

Response:

{
  "text": "Amazing product!",
  "aspects": [{ "aspect": "quality", "sentiment": "positive" }],
  "rating": 4.8,
  "processing_time": 1.23
}

POST /analyze/batch

Analyze multiple reviews in one request

Request:

{
  "reviews": [{ "text": "Great quality!" }, { "text": "Poor service." }]
}

Response:

{
  "results": [
    {
      "text": "Great quality!",
      "aspects": [...],
      "rating": 4.5
    },
    {
      "text": "Poor service.",
      "aspects": [...],
      "rating": 2.1
    }
  ],
  "total_processed": 2,
  "processing_time": 2.45
}

POST /feedback

Submit feedback for model improvement

Request:

{
  "review_text": "...",
  "predicted_rating": 4.2,
  "actual_rating": 4.0,
  "aspects": [...]
}

Interactive API Docs

Visit http://localhost:8000/docs for Swagger UI documentation with live testing capabilities.

---

🎓 Training & Fine-Tuning

Fine-Tune LLM

# Prepare training data
python src/data/format_for_training.py

# Start fine-tuning
python src/training/train.py

Data Format:

{
  "instruction": "Analyze the following review...",
  "input": "The product quality is excellent!",
  "output": {
    "aspects": [{ "aspect": "quality", "sentiment": "positive" }],
    "rating": 5.0
  }
}

Train Ensemble Model

# Train rating prediction ensemble
python -m src.training.train_ensemble_refactored

Configuration: Edit configs/ensemble_config.yaml to customize:

• Data paths
• Model hyperparameters
• Feature engineering settings
• Training parameters

Output:

• Trained models: models/rating_production/
• Predictions: predictions_test.csv
• Feature importance: feature_importance_xgboost.csv
• Training metadata: training_metadata.pkl

Merge LoRA Adapters

# Merge fine-tuned adapters with base model
python src/utils/merge_models.py

---

📊 Dataset

Data Sources

Primary Datasets:

1. SemEval-2014 Task 4: Restaurant and Laptop reviews with aspect annotations
2. Amazon Product Reviews: Multi-domain product reviews
3. Custom Annotated Data: Semi-automatically labeled using weak supervision

Annotation Strategy

Semi-Supervised Approach:

1. Base LLM generates pseudo-labels for raw reviews
2. Human validation on stratified sample (20%)
3. Iterative refinement based on validation feedback
4. Final dataset: ~10,000 reviews with quality assurance

Aspect Taxonomy

Closed aspect set for consistency:

Aspect	Description
quality	Product or service quality
price	Cost and value for money
delivery	Shipping speed and reliability
packaging	Packaging condition
usability	Ease of use
support	Customer service
experience	Overall experience

Sentiment Labels

• positive: Favorable opinion
• neutral: Mixed or factual statement
• negative: Unfavorable opinion

Rating Scale

• Continuous values: 1.0 to 5.0
• One decimal precision
• Original scale preserved (no normalization)

Dataset Splits

Split	Samples	Percentage
Train	~8,000	80%
Validation	~1,000	10%
Test	~1,000	10%

---

📁 Project Structure

ReviewMind/
├── api/                          # FastAPI application
│   ├── main.py                   # API endpoints
│   ├── schemas.py                # Pydantic models
│   └── __init__.py
├── configs/                      # Configuration files
│   ├── ensemble_config.yaml      # Ensemble model config
│   └── model/
│       └── qwen2.5-7b-instruct.yaml
├── data/                         # Datasets
│   ├── raw/                      # Raw review data
│   ├── processed/                # Processed training data
│   ├── ready/                    # Train/val/test splits
│   ├── splits/                   # JSON splits
│   └── schema/                   # JSON schemas
├── docs/                         # Documentation
│   ├── review_mind_software_requirements_specification_srs.md
│   ├── review_mind_model_design_document_mdd.md
│   ├── review_mind_dataset_requirements_specification_drs.md
│   ├── review_mind_training_plan_experiment_design.md
│   ├── ensemble_model_README.md
│   ├── REFACTORING_SUMMARY.md
│   └── BEFORE_AFTER_COMPARISON.md
├── models/                       # Model weights
│   ├── base/                     # Base Qwen2.5-7B
│   ├── finetuned/                # Fine-tuned LoRA adapters
│   ├── final/                    # Merged models
│   ├── rating/                   # Ensemble models (dev)
│   └── rating_production/        # Production ensemble
├── scripts/                      # Utility scripts
│   ├── download_model.py         # Model downloader
│   ├── test_inference.py         # Inference testing
│   └── test_refactored_ensemble.py
├── src/                          # Source code
│   ├── data/                     # Data processing
│   │   ├── ingestion.py
│   │   ├── preprocess.py
│   │   ├── labeler.py
│   │   ├── format_for_training.py
│   │   └── ensemble_data_loader.py
│   ├── model/                    # Model components
│   │   ├── config.py
│   │   ├── feature_extractors.py
│   │   ├── ensemble_trainer.py
│   │   └── model_evaluator.py
│   ├── training/                 # Training scripts
│   │   ├── train.py              # LLM fine-tuning
│   │   └── train_ensemble_refactored.py
│   ├── inference/                # Inference
│   │   └── ensemble_predictor.py
│   ├── pipeline/                 # Pipeline orchestration
│   │   └── run_pipeline.py
│   ├── tests/                    # Test scripts
│   │   ├── test_model.py
│   │   ├── test_pipeline.py
│   │   ├── interactive_demo.py
│   │   └── interactive_pipeline_tester.py
│   └── utils/                    # Utilities
│       ├── llm_parser.py
│       └── merge_models.py
├── frontend/                     # Next.js frontend (Coming Soon)
├── requirements.txt              # Python dependencies
├── pyproject.toml               # Project metadata
└── README.md                    # This file

---

📈 Performance Metrics

Aspect Extraction & Sentiment Classification

Metric	Score
Aspect Extraction F1	0.87
Sentiment Accuracy	0.89
Aspect-Sentiment Joint Accuracy	0.82

Rating Prediction

Metric	Score
MAE (Mean Absolute Error)	0.31
RMSE (Root Mean Square Error)	0.42
R² Score	0.88
Accuracy ±0.5 stars	89%
Accuracy ±1.0 stars	97%

Inference Performance

• Single Review Latency: < 2 seconds
• Batch Processing (100 reviews): < 15 seconds
• Memory Usage: ~4GB (with model loaded)
• Hardware: Apple M4 Pro (14-core CPU, 20-core GPU)

---

🛠️ Development

Run Tests

# Test LLM inference
python scripts/test_inference.py

# Test ensemble model
python scripts/test_refactored_ensemble.py

# Test full pipeline
python src/tests/test_pipeline.py

# Interactive demo
python src/tests/interactive_demo.py

Code Quality

# Format code
black src/ api/

# Type checking
mypy src/

# Linting
pylint src/ api/

Configuration Management

Edit configs/ensemble_config.yaml for:

• Hyperparameter tuning
• Feature engineering adjustments
• Data path configuration
• Model selection

All changes take effect without code modifications.

---

📚 Documentation

Comprehensive documentation available in the docs/ directory:

• Software Requirements Specification (SRS): Functional and non-functional requirements
• Model Design Document (MDD): Model architecture and design decisions
• Dataset Requirements Specification (DRS): Dataset design and annotation schema
• Training Plan & Experiment Design: Training procedures and experiments
• Ensemble Model README: Detailed ensemble model documentation
• Refactoring Summary: Code quality improvements and architecture changes

---

🗺️ Roadmap

Phase 1: Backend (Completed ✅)

• Fine-tune Qwen2.5-7B for aspect extraction
• Build ensemble rating prediction model
• Create FastAPI backend
• Implement feature engineering pipeline
• Add comprehensive testing
• Write documentation

Phase 2: Frontend (In Progress 🚧)

• Next.js application setup
• Review input interface
• Results visualization
• Batch analysis dashboard
• Historical analytics

Phase 3: Deployment (Planned 📋)

• Docker containerization
• Kubernetes orchestration
• CI/CD pipeline
• Monitoring and logging
• Load balancing
• Auto-scaling

Phase 4: Enhancements (Future 🔮)

• Multilingual expansion (Spanish, French, German)
• Open aspect discovery
• Confidence scores per prediction
• Explainability features (attention visualization)
• Model distillation for faster inference
• Mobile app (React Native)

---

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Development Guidelines

• Follow PEP 8 style guide
• Add type hints to all functions
• Write docstrings for public APIs
• Include tests for new features
• Update documentation as needed

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

👨‍💻 Author

Ahmet Baran Dinçoguz

• GitHub: @barandincoguz
• Repository: SentimentAnalysis

---

🙏 Acknowledgments

• Qwen Team for the excellent base model
• Hugging Face for the Transformers library and model hosting
• FastAPI for the modern web framework
• SemEval for providing benchmark datasets
• Open-source community for inspiration and tools

---

📞 Support

For questions, issues, or feature requests:

• Open an issue
• Check the documentation
• Contact the author

---

Made with ❤️ for the AI/NLP community

⭐ Star this repo if you find it useful!