VitalCard Rewards Engine

A high-performance, extensible rewards calculation engine for VitalCard that processes credit card transactions and calculates point earnings based on configurable business rules.

📖 Read the documentation →

📌 About

Rewards look like a simple multiplication problem — amount × rate = points — right up until you put them behind a real payment network. Settlement files land in S3 in bursts of hundreds of thousands. Webhooks arrive out of order. Lambdas time out mid-batch and get retried. And unlike a cache miss or a dropped analytics event, a mistake here is financial: credit the same transaction twice and you have handed out points that a customer will spend and an accountant will eventually come asking about.

This project is a reference implementation of that problem, built serverless-first on AWS Lambda, SQS, and DynamoDB. The rule engine itself is deliberately boring — declarative JSON rules, four calculation types, dot-notation field paths — because the interesting engineering is everywhere else: in making sure each transaction is counted exactly once, at volume, while failures are happening.

What's actually interesting here

Exactly-once processing, not "at-least-once and hope." Every event gets a deterministic idempotency key derived from its own contents (event_id, timestamp, amount, user), checked before any work is done. Batch progress is checkpointed every 100 events, so a Lambda that dies 800 events into a 1,000-event batch resumes rather than restarts — and resuming can't double-credit the first 800. See Exactly-Once Processing.

Explicit SQS acknowledgment. The obvious Lambda + SQS integration acknowledges messages implicitly when the handler returns — which quietly deletes messages the handler never actually finished. This engine tracks receipt handles per message and acknowledges only the ones that provably succeeded, so partial batch failures retry the failures instead of losing them. The failure mode and the fix are written up in Message Acknowledgment Strategy.

Batch-first data access. Naive per-transaction processing costs one Lambda invocation and one DynamoDB round trip each. Batching metadata and rule loads collapses 1,000 transactions into a single invocation and roughly 10 database operations — about a 100× reduction in DynamoDB calls, and the difference between a rewards pipeline you can afford to run on Black Friday and one you can't. Numbers in Batch Processing Performance.

Failure modes enumerated up front. Lambda timeouts, memory exhaustion, cold starts, DynamoDB throttling, partial batch writes, network partitions — each documented with its blast radius, how you detect it in CloudWatch, and what the system does about it. Failure Modes Analysis.

Status

Alpha — a reference implementation, not a deployed production system. The core rule engine, batch processors, and resilience machinery are implemented and unit-tested; test coverage is still partial and the infrastructure-as-code layer is in progress. Read it as a worked example of how to build correctness-critical event processing on serverless primitives, not as something to point at your card network on Monday.

🚀 Features

Extensible Rule Engine: Support complex business rules with multiple conditions and calculation types
High Performance: Process transactions in <500ms, handle 10K+ TPS
Scalable Architecture: Auto-scale based on load, support batch and real-time processing
Complete Audit Trail: Full transaction and calculation history
Metadata Enrichment: Support for ML and external API data enrichment
Multiple Calculation Types: Multiplier, bonus points, percentage, and tiered calculations
Batch Processing: Efficient batch loading of metadata and rules
Error Handling: Comprehensive error handling with Dead Letter Queues

🏗️ Architecture

The rewards engine follows a serverless, event-driven architecture:

┌─────────────────────────────────────────────────────────────────┐
│                        Event Sources                            │
├─────────────────┬───────────────────┬───────────────────────────┤
│ S3 Settlement   │ Real-time APIs    │ ML Enrichment Pipeline   │
│ Files           │ (Webhooks)        │ (Async Processing)        │
└─────────────────┴───────────────────┴───────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Event Ingestion Layer                       │
├─────────────────┬───────────────────┬───────────────────────────┤
│ S3 Trigger      │ API Gateway       │ EventBridge Scheduler     │
│ → Lambda        │ → Lambda          │ → Lambda                  │
└─────────────────┴───────────────────┴───────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Message Queue                             │
│                    SQS + DLQ                                   │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                   Core Processing Engine                       │
├─────────────────┬───────────────────┬───────────────────────────┤
│ Event Processor │ Rule Engine       │ Enrichment Processor      │
│ (Lambda)        │ (Lambda)          │ (Lambda)                  │
└─────────────────┴───────────────────┴───────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Data Storage                              │
├─────────────────┬───────────────────┬───────────────────────────┤
│ Events Table    │ Metadata Table    │ Rewards Ledger           │
│ (DynamoDB)      │ (DynamoDB)        │ (DynamoDB)                │
└─────────────────┴───────────────────┴───────────────────────────┘

📊 Data Model

DynamoDB Tables

1. Events Table

Stores all incoming events (transactions, enrollments, milestones)

Partition Key: EVENT#{event_type}#{user_id}
Sort Key: {timestamp_iso}#{event_id}

2. Metadata Table

Unified table for all metadata types with efficient batch access

Partition Key: {entity_type}#{entity_id}
Sort Key: META#{metadata_type}

Entity Types: USER, MCC, MERCHANT, RULE, PROGRAM
Metadata Types: BASE, LIMITS, PREFERENCES, LOCATION, ENRICHMENT

3. Rules Table

Stores business rules with versioning

Partition Key: RULE#{rule_id}
Sort Key: VERSION#{version}

4. Rewards Ledger Table

Stores all reward calculations and maintains audit trail

Partition Key: USER#{user_id}
Sort Key: {timestamp_iso}#{event_id}#{rule_id}

🎯 Rule Engine

Supported Operators

Comparison: equals, not_equals, greater_than, less_than, etc.
Set Operations: in, not_in
String Operations: contains, regex_match
Date Operations: date_after, date_before
Logical Operations: and, or, not

Field Path Resolution

Support nested field access with dot notation:

event.amount - Direct event field
event.metadata.location - Nested metadata
metadata.user_metadata.BASE.tier - User tier
computed.is_weekend - Computed fields

Calculation Types

1. Multiplier Calculation

{
  "type": "multiplier",
  "base_rate": 1.0,
  "multiplier": 5.0,
  "max_monthly_earn": 50000,
  "max_per_transaction": 5000
}

2. Bonus Points

{
  "type": "bonus_points",
  "bonus_amount": 500,
  "once_per_month": true
}

3. Percentage Calculation

{
  "type": "percentage",
  "percentage": 2.5,
  "max_monthly_earn": 10000
}

4. Tiered Calculation

{
  "type": "tiered",
  "tiers": [
    {"min_amount": 0, "max_amount": 100, "rate": 2.0},
    {"min_amount": 100, "max_amount": 500, "rate": 3.0},
    {"min_amount": 500, "rate": 5.0}
  ]
}

🚀 Getting Started

Prerequisites

Python 3.11+
AWS CLI configured
Docker (for local development)

Installation

Clone the repository

git clone <repository-url>
cd rewards-engine

Install dependencies
```
pip install -r requirements.txt
```

Set up environment variables

cp .env.example .env
# Edit .env with your configuration

Run tests
```
pytest tests/unit/ -v
```

Local Development

Start local DynamoDB

docker run -p 8000:8000 amazon/dynamodb-local

Seed sample data
```
python scripts/seed_data.py
```
Run the application
```
python -m src.handlers.event_processor
```

📝 Usage Examples

Processing a Transaction Event

from src.core.models import Event, EventType
from src.core.rule_engine import RuleEngine
from datetime import datetime
from decimal import Decimal

# Create event
event = Event(
    event_id="txn_001",
    event_type=EventType.TRANSACTION,
    user_id="user_12345",
    timestamp=datetime(2025, 1, 15, 10, 30, 0),
    amount=Decimal("45.80"),
    mcc="5912",
    merchant_name="CVS Pharmacy"
)

# Load metadata and rules
metadata = await metadata_loader.load_metadata_for_event(event)
rules = await rule_loader.load_applicable_rules(event)

# Evaluate rules
rule_engine = RuleEngine()
results = await rule_engine.evaluate_rules(event, rules, metadata)

# Process results
for result in results:
    print(f"Rule: {result.rule_id}")
    print(f"Points earned: {result.points_earned}")
    print(f"Multiplier used: {result.multiplier_used}")

Creating a Business Rule

from src.core.models import Rule
from datetime import datetime

rule = Rule(
    rule_id="medical_spend_health_plus",
    version="1.0",
    name="Health+ Medical Spending Bonus",
    description="5x points for medical spending with Health+ subscription",
    conditions={
        "and": [
            {"field": "event.mcc", "operator": "in", "values": ["5912", "8011", "8021", "8062"]},
            {"field": "metadata.user_metadata.BASE.health_plus_status", "operator": "equals", "value": "active"},
            {"field": "event.event_type", "operator": "equals", "value": "transaction"}
        ]
    },
    earn_calculation={
        "type": "multiplier",
        "base_rate": 1.0,
        "multiplier": 5.0,
        "max_per_transaction": 5000
    },
    metadata_dependencies=["user_metadata", "mcc_metadata"],
    valid_from=datetime(2025, 1, 1),
    priority=100,
    active=True
)

🧪 Testing

Running Tests

# Unit tests
pytest tests/unit/ -v

# Integration tests
pytest tests/integration/ -v

# Performance tests
pytest tests/performance/ -v

# All tests with coverage
pytest --cov=src --cov-report=html

Test Structure

Unit Tests: Test individual components in isolation
Integration Tests: Test component interactions
Performance Tests: Test system performance under load
End-to-End Tests: Test complete workflows

🚀 Deployment

AWS CDK Deployment

Install CDK
```
npm install -g aws-cdk
```
Deploy to staging
```
cdk deploy RewardsEngineStack-staging
```
Deploy to production
```
cdk deploy RewardsEngineStack-prod
```

Environment Configuration

The system supports multiple environments with different configurations:

Development: Pay-per-request DynamoDB, minimal resources
Staging: Provisioned DynamoDB, moderate resources
Production: On-demand DynamoDB, full resources with monitoring

📊 Monitoring & Observability

CloudWatch Metrics

Transaction processing latency
Rule evaluation performance
Error rates and types
Queue depth and processing rates

CloudWatch Logs

Structured logging with correlation IDs
Error tracking and debugging
Performance monitoring

X-Ray Tracing

Distributed tracing across Lambda functions
Performance bottleneck identification
Dependency mapping

🔧 Configuration

Environment Variables

# AWS Configuration
AWS_REGION=us-east-1
AWS_ACCOUNT_ID=123456789012

# DynamoDB Tables
EVENTS_TABLE_NAME=events
METADATA_TABLE_NAME=metadata
RULES_TABLE_NAME=rules
REWARDS_LEDGER_TABLE_NAME=rewards_ledger

# SQS Queues
EVENT_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/...
DLQ_URL=https://sqs.us-east-1.amazonaws.com/...

# Lambda Configuration
LAMBDA_MEMORY=1024
LAMBDA_TIMEOUT=60
LAMBDA_RESERVED_CONCURRENCY=100

# Processing Configuration
MAX_BATCH_SIZE=10
PROCESSING_TIMEOUT_SECONDS=30
METADATA_CACHE_TTL_SECONDS=300

# Logging
LOG_LEVEL=INFO
LOG_FORMAT=json

# Environment
ENVIRONMENT=prod
SERVICE_NAME=rewards-engine

🤝 Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Development Guidelines

Follow PEP 8 style guidelines
Write comprehensive tests for new features
Update documentation for API changes
Use type hints throughout the codebase
Follow the existing error handling patterns

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

For support and questions:

Create an issue in the GitHub repository
Contact the development team
Check the documentation in the docs/ directory

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
docs		docs
scripts		scripts
src		src
tests		tests
.coverage		.coverage
.gitignore		.gitignore
.python-version		.python-version
README.md		README.md
coverage.xml		coverage.xml
main.py		main.py
mkdocs.yml		mkdocs.yml
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt
uv.lock		uv.lock

Latest commit

History

Folders and files

Repository files navigation