AI-Powered Calorie Tracker with Local Food Recognition
Cal AI is a full-stack calorie tracking web application that uses a locally-running AI model to recognize food from images. Simply snap a photo of your meal, and the app identifies the food, estimates nutritional values, and logs it to your daily tracker — all without sending your data to any external AI service.
- 🤖 Local AI Model — Food recognition runs on your machine via a ViT (Vision Transformer) model trained on Food-101. No cloud API keys needed for scanning.
- 📸 Snap & Scan — Take a photo or upload an image to instantly classify food and get nutritional breakdown.
- ⚖️ Adjustable Portions — Slide to set portion size (25g–800g) and watch macro values recalculate live.
- 📊 Dashboard & Progress — Track daily calories, macros, weight trends, and streaks with smooth animated charts.
- 🔐 Supabase Auth — Secure email-based authentication with Row Level Security.
- 📱 Mobile-First Design — Optimized for phones with bottom navigation, swipeable calendars, and touch-friendly interactions.
- Camera capture or image upload for food recognition
- Text search — type a food name (e.g., "pizza", "sushi") for quick lookup against 101 food categories
- AI confidence score displayed for each classification
- Portion size slider (25g–800g) with live macro recalculation
- Editable results — adjust food name, ingredients, and individual macro values before logging
- Daily macro rings — visual progress for Calories, Protein, Carbs, and Fat
- Swipeable weekly calendar — tap any day to view that day's meals
- Monthly heatmap view — see goal adherence at a glance (green = met goal, red = exceeded)
- Meal cards — view, edit, or delete logged meals with inline editing
- Weight tracking chart — interactive line chart with 7D/30D/60D/90D timeframes
- Streak counter — consecutive days of meal logging
- Weight logging modal — quick weight entry with unit preference (lbs/kg)
- Avatar upload — profile photo with Supabase Storage
- Personal details — name, weight, goal weight, unit preference
- Daily goals — customizable targets for calories, protein, carbs, and fat
| Layer | Technology |
|---|---|
| Frontend | Next.js 16, React 19, TypeScript |
| Styling | Tailwind CSS 4 |
| Animations | Framer Motion |
| Charts | Recharts |
| Icons | Lucide React |
| Auth & Database | Supabase (PostgreSQL + Auth + Storage) |
| AI Model | PyTorch + HuggingFace Transformers (nateraw/food — ViT on Food-101) |
| Model Server | Python Flask |
| Nutritional Data | Bundled JSON database (101 food categories, per-100g macros) |
calorie_tracker_webapp/
├── src/
│ ├── app/
│ │ ├── page.tsx # Landing/splash page
│ │ ├── layout.tsx # Root layout
│ │ ├── globals.css # Global styles
│ │ ├── auth/ # Authentication page
│ │ ├── actions/
│ │ │ └── scan.ts # Server actions (calls model API)
│ │ └── (app)/ # Authenticated app routes
│ │ ├── layout.tsx # App layout (header + nav)
│ │ ├── home/page.tsx # Dashboard
│ │ ├── scan/page.tsx # Food scanner
│ │ ├── progress/page.tsx # Weight & progress charts
│ │ └── profile/page.tsx # User profile & settings
│ ├── components/
│ │ ├── ui/ # Reusable UI components
│ │ ├── layout/ # Navigation components
│ │ ├── auth/ # Auth components
│ │ └── shared/ # Shared components
│ └── lib/
│ ├── supabase.ts # Supabase client
│ └── utils.ts # Utility functions
├── food-recognition-model/
│ ├── server.py # Flask API server
│ ├── requirements.txt # Python dependencies
│ ├── config.json # Model configuration
│ ├── nutritional_database.json # 101 food categories (per-100g macros)
│ └── classification_model.h5 # Model reference
├── supabase/
│ └── schema.sql # Database schema + RLS policies
├── start.sh # One-command startup script
├── .env.example # Environment template
└── package.json # Node dependencies
- Node.js ≥ 18
- Python ≥ 3.10
- Supabase account (free tier works)
git clone https://github.com/your-username/calorie_tracker_webapp.git
cd calorie_tracker_webapp
npm install- Create a new project at supabase.com
- Run the SQL from
supabase/schema.sqlin the Supabase SQL Editor - Copy your project URL and anon key
cp .env.example .envEdit .env with your Supabase credentials:
NEXT_PUBLIC_SUPABASE_URL="https://your-project.supabase.co"
NEXT_PUBLIC_SUPABASE_ANON_KEY="your-anon-key"
FOOD_MODEL_URL=http://localhost:5001cd food-recognition-model
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txtNote: The first run downloads the ViT model (~350 MB). After that, it loads from cache instantly.
Option A — Single command (recommended):
./start.shThis starts both servers concurrently and handles cleanup on Ctrl+C.
Option B — Manual (two terminals):
# Terminal 1 — AI Model Server
cd food-recognition-model
source venv/bin/activate
python server.py
# Terminal 2 — Next.js
npm run devNavigate to http://localhost:3000 in your browser.
The Python model server runs at http://localhost:5001:
| Endpoint | Method | Description |
|---|---|---|
/api/health |
GET | Health check — returns model status and food count |
/api/classify |
POST | Image classification — accepts multipart/form-data with image field |
/api/classify-text |
POST | Text search — accepts JSON {"text": "pizza"} |
{
"food_name": "Pizza",
"calories": 500,
"protein": 24,
"carbs": 60,
"fat": 20,
"portion_grams": 200,
"confidence": 94.2,
"per_100g": {
"calories": 250,
"protein": 12.0,
"carbs": 30.0,
"fat": 10.0
},
"ingredients": [
{ "name": "Pizza", "calories": 500, "protein": 24, "carbs": 60, "fat": 20 }
],
"all_predictions": [
{ "label": "Pizza", "confidence": 94.2 },
{ "label": "Bruschetta", "confidence": 2.1 }
]
}The app uses three main tables in Supabase:
profiles meals weight_logs
├── id (UUID) ├── id (UUID) ├── id (UUID)
├── display_name ├── user_id (FK) ├── user_id (FK)
├── weight ├── food_name ├── weight
├── goal_weight ├── calories └── logged_date
├── daily_*_goal ├── protein
├── avatar_url ├── carbs
├── fat
├── ingredients
├── image_url
└── created_at
All tables use Row Level Security (RLS) — users can only access their own data.
The model recognizes 101 food categories from the Food-101 dataset, including:
View all 101 categories
| Apple Pie | Baby Back Ribs | Baklava | Beef Carpaccio |
| Beef Tartare | Beet Salad | Beignets | Bibimbap |
| Bread Pudding | Breakfast Burrito | Bruschetta | Caesar Salad |
| Cannoli | Caprese Salad | Cappuccino | Carrot Cake |
| Ceviche | Cheesecake | Cheese Plate | Chicken Curry |
| Chicken Quesadilla | Chicken Wings | Chocolate Cake | Chocolate Mousse |
| Churros | Clam Chowder | Club Sandwich | Crab Cakes |
| Crème Brûlée | Croque Madame | Cupcakes | Deviled Eggs |
| Donuts | Dumplings | Eggs Benedict | Escargots |
| Falafel | Filet Mignon | Fish And Chips | Foie Gras |
| French Fries | French Onion Soup | French Toast | Fried Calamari |
| Fried Rice | Frozen Yogurt | Garlic Bread | Gnocchi |
| Greek Salad | Grilled Cheese Sandwich | Grilled Salmon | Guacamole |
| Gyoza | Hamburger | Hot And Sour Soup | Hot Dog |
| Huevos Rancheros | Hummus | Ice Cream | Lasagna |
| Lobster Bisque | Lobster Roll Sandwich | Macaroni And Cheese | Macarons |
| Miso Soup | Mussels | Nachos | Omelette |
| Onion Rings | Oysters | Pad Thai | Paella |
| Pancakes | Panna Cotta | Peking Duck | Pho |
| Pizza | Pork Chop | Poutine | Prime Rib |
| Pulled Pork Sandwich | Ramen | Ravioli | Red Velvet Cake |
| Risotto | Samosa | Sashimi | Scallops |
| Seaweed Salad | Shrimp And Grits | Spaghetti Bolognese | Spaghetti Carbonara |
| Spring Rolls | Steak | Strawberry Shortcake | Sushi |
| Tacos | Takoyaki | Tiramisu | Tuna Tartare |
| Waffles |
MIT
Built with ❤️ by Anvesh Mishra