Facebook Ads Intelligence System

Author: Anushka Sagar

A production-ready multi-agent system that analyzes Facebook Ads performance like a human Marketing Analyst—automatically diagnosing issues and recommending improvements.

---

What Makes This System Special

This isn't just another data analysis script. It's a fully orchestrated AI system that demonstrates:

• Agentic AI Architecture - Five specialized agents working together, each with a clear purpose
• Production-Grade Reliability - Retry logic, schema validation, drift detection, fallback strategies
• Explainable AI - Every decision is logged, scored, and traceable
• Enterprise Testing - Automated tests, CI/CD pipeline, deterministic test mode
• Real Business Value - Generates actionable insights and creative recommendations

---

Quick Start

# Setup
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Run analysis
python src/run.py "Analyze ROAS drop in last 7 days"

What you get:

• insights.json - Validated hypotheses with confidence scores
• creatives.json - Creative recommendations for underperforming ads
• report.md - Executive summary with actionable recommendations

---

How It Works: The 5-Agent Pipeline

User Query
    ↓
 Planner Agent → Creates analysis strategy
    ↓
 Data Agent → Validates & summarizes data (detects drift)
    ↓
 Insight Agent → Generates hypotheses (LLM-powered)
    ↓
 Eval Agent → Fact-checks with real metrics
    ↓
 Creative Agent → Recommends improvements (LLM-powered)
    ↓
 Final Reports

Why This Architecture Works

Strategic AI Usage:

• 3 LLM Agents (Planner, Insight, Creative) - For reasoning and creativity
• 2 Rule-Based Agents (Data, Eval) - For reliability and validation

Result: Best of both worlds—creative intelligence meets deterministic accuracy.

---

Key Technical Achievements

1. Robust JSON Parsing

• Handles malformed LLM outputs gracefully
• Multiple format support (arrays, objects, nested structures)
• Auto-repair for partial JSON
• Never crashes on bad formatting

2. Production-Ready Error Handling

# Every agent has:
- Retry logic with exponential backoff
- Fallback strategies when LLM fails
- Comprehensive logging per agent
- Graceful degradation

3. Test-Driven Development

pytest src/tests/ -v  # Runs in seconds, no LLM calls

• Deterministic test mode (PYTEST_ACTIVE=1)
• CI/CD with GitHub Actions
• Separate test/production logging

4. Data Quality Controls

• Schema validation - Catches CSV structure changes
• Drift detection - Warns about distribution shifts
• Metric instrumentation - Tracks execution time, confidence scores

---

Project Structure

.
├── config/                     # System configuration
├── data/                       # Input CSV dataset
├── prompts/                    # LLM prompts for each agent
├── reports/                    # Generated outputs
├── logs/
│   ├── metrics/                # Performance tracking
│   ├── run/                    # Production logs
│   └── tests/                  # Test logs
└── src/
    ├── agents/                 # The 5 specialized agents
    ├── orchestrator/           # Pipeline controller
    ├── tests/                  # Automated test suite
    └── utils/                  # Helpers, retry, validation

---

Why This Demonstrates Engineering Excellence

Problem Solving

• Solves real marketing problems (ROAS drops, low CTR)
• Generates actionable recommendations, not just insights
• Mimics how human analysts actually work

Architecture

• Clean separation of concerns (each agent is independent)
• Config-driven (easy to modify without code changes)
• Modular design (agents can be swapped or extended)

Reliability

• Multiple layers of error handling
• Fallback strategies at every step
• Comprehensive logging for debugging
• Test coverage with CI/CD

Observability

• Per-agent execution metrics
• Trace IDs for debugging
• Confidence scoring throughout pipeline
• CSV metrics logging for analysis

🔬 Best Practices

• Type hints and validation
• Structured logging (Loguru)
• Linting (Ruff)
• GitHub Actions CI/CD
• Production-ready code quality

---

Example Output

Insight Generated:

{
  "id": "H1",
  "driver": "Audience fatigue from high frequency",
  "confidence": 0.78,
  "evidence": {
    "roas_before": 2.3,
    "roas_after": 1.8,
    "ctr_drop": -25%
  }
}

Creative Recommendation:

{
  "campaign_id": "123",
  "current_ctr": 0.8,
  "recommendations": [
    "Test benefit-focused headlines",
    "Add social proof elements",
    "Refresh visual creative"
  ]
}

---

Testing & CI/CD

Automated Testing

make test  # Runs full test suite
make lint  # Code quality checks

GitHub Actions Pipeline

• Automatic linting on commit
• Full test suite execution
• Fail-fast if quality drops
• Python 3.12 environment matching production

---

Tech Stack

• Python 3.12 - Modern Python features
• LangChain - Agent orchestration
• OpenAI/Ollama - LLM integration
• Pandas - Data processing
• Loguru - Structured logging
• PyTest - Testing framework
• Ruff - Fast Python linter
• GitHub Actions - CI/CD

---

Quick Commands

make install      # Install dependencies
make run          # Run analysis pipeline
make test         # Run tests
make lint         # Check code quality

---

What This Project Demonstrates

Multi-Agent AI Systems - Coordinated agents with specialized roles
Production Engineering - Error handling, logging, monitoring ,Exception handling, Drift dectection, validation LLM Integration - Prompt engineering, JSON parsing, retry logic
Software Engineering - Testing, CI/CD, code quality
Problem Solving - Real business value, actionable outputs
System Design - Modular, maintainable, scalable architecture

Built with attention to detail and engineering best practices. Ready for review. 🚀