Computer Vision · Retail Intelligence · Real-Time Shelf Compliance
Retail chains rely on planograms — precise layout plans defining where each product should sit, in what order, and with how many facings — to maximize sales and guarantee contractual shelf space for brands. In practice, these plans are rarely fully respected: on average, 25% of shelves are non-compliant at any given time, and stores spend 8–12 hours per week on manual audits to catch it. The cost of getting it wrong is real — non-compliant shelves generate up to 15–20% less revenue, while chains sustaining high compliance see gains of up to +15% on controlled categories (ECR Europe, Nielsen; P&G, Unilever). VisShop Planogram AI replaces these slow, error-prone manual checks with an automated computer vision pipeline that verifies shelf compliance in minutes, cell by cell, without any image ever leaving the store.
The system takes a single shelf photo and turns it into a full compliance report through five stages:
Raw photo → Perspective correction → Grid mapping → Detection → Recognition → Comparison → Report + Dashboard
- Calibration — a one-time, per-camera setup marks the shelf's corners and row/column boundaries, defining the grid used for every future photo from that angle.
- Perspective correction — the raw photo is flattened into a clean top-down view of the shelf.
- Detection — a fine-tuned YOLOv11s model locates every product on the shelf and assigns it a grid position.
- Recognition — each detected product is identified by comparing its visual embedding (SigLIP) against a reference database via FAISS nearest-neighbor search, refined with color/shape/texture matching and OCR.
- Comparison & reporting — detected products are checked cell-by-cell against the reference planogram, flagging each position as correct, missing, misplaced, or unidentified, with results visualized on an annotated shelf image and a live dashboard.
| Category | Tools | Why |
|---|---|---|
| Object Detection | YOLOv11s (Ultralytics) | Fast, accurate, real-time-capable detection well suited to dense shelf scenes, with a lightweight footprint for edge deployment. |
| Product Recognition | SigLIP, FAISS | Enables identifying products via similarity search instead of training a classifier per SKU — critical for a catalog with hundreds of products. |
| Text Matching | EasyOCR | Disambiguates visually similar products by reading packaging text. |
| Image Processing | OpenCV, scikit-image, Albumentations | Handles perspective correction, grid mapping, and synthetic data augmentation for new product references. |
- Fully on-device — every step, from detection to comparison, runs locally with no image upload to the cloud.
- No retraining for new products — new SKUs are added with a handful of reference photos, not a full model retrain.
- Robust to duplicate/ambiguous catalog entries — an alias-resolution system keeps recognition accurate even when the source catalog has inconsistencies.
- Hardware-flexible — automatically takes advantage of GPU acceleration (e.g. NVIDIA Jetson) when available, with no code changes.
- Reusable calibration — each shelf is calibrated once and reused across all future checks from the same camera angle.
Validated on real shelf photos covering 28 distinct products, the system reached 72% average recognition accuracy. Current accuracy is mainly bottlenecked by reference database quality and camera framing consistency — both addressed through ongoing improvements to the product database and calibration workflow.
Developed as part of an AI / Computer Vision internship at VisShop AI.