🔍 ML-Procurement

EU Public Procurement Anomaly Detection

An intelligent monitoring pipeline that detects suspicious patterns in EU public procurement data using unsupervised machine learning.

---

📋 About the Project

ML-Procurement is an intelligent EU public procurement monitoring system built to improve transparency and surface anomalies in European public tenders. It automatically retrieves procurement notices from the EU TED (Tenders Electronic Daily) API, preprocesses the data, and applies unsupervised machine learning algorithms to identify unusual or potentially suspicious patterns.

Public procurement accounts for roughly 14% of EU GDP — making it a critical domain for anti-fraud analysis, policy transparency, and accountability.

---

🚨 What It Detects

The system flags anomalies across several dimensions:

• Contract value outliers — awards that are statistically unusual compared to similar contracts in the same category, region, or period
• Bidder behaviour — single-bidder awards, repeated winner patterns, unusual supplier concentration
• Procurement trends — deviations from historical baselines in volume, category distribution, or geography

---

🧠 Machine Learning Pipeline

EU TED API  ──►  Data Collection  ──►  Preprocessing & Feature Engineering
                                               │
                              ┌────────────────┼────────────────┐
                              ▼                ▼                ▼
                      Isolation Forest      DBSCAN          K-Means
                      (global outliers)  (density gaps)  (centroid distance)
                              └────────────────┼────────────────┘
                                               ▼
                                    Ensemble Anomaly Score
                                               │
                                               ▼
                              Visualisation + Report Output

Algorithms

Algorithm	Role
Isolation Forest	Detects global outliers — contracts that are statistically isolated from the majority
DBSCAN	Density-based clustering; contracts in sparse regions are flagged as anomalous
K-Means	Groups similar contracts; high distance from any centroid signals an outlier

Scores from all three models are combined into an ensemble anomaly ranking for more robust and less model-dependent results.

---

🛠️ Technical Stack

Component	Technology
Language	Python 3
ML	scikit-learn (Isolation Forest, DBSCAN, K-Means)
Data	pandas, NumPy
API	requests (EU TED REST API)
Visualisation	matplotlib, seaborn
Storage	SQLite / CSV

---

📁 Project Structure

ML-Procurement/
├── components/          # Modular pipeline components (ingestion, preprocessing, models)
├── data/                # Raw and processed procurement data
├── models/              # Saved model artefacts
├── clusters/            # Cluster output files
├── output/              # Anomaly reports
├── visualization/       # Plot generation
├── tests/               # Test suite
├── ted_ml_pipeline.py   # Main pipeline entrypoint
└── clustering_pipeline.py

---

🚀 Getting Started

git clone https://github.com/aIex-personal/ML-Procurement.git
cd ML-Procurement

pip install -r requirements.txt

# Run the full pipeline
python ted_ml_pipeline.py

---

📊 Output

• Anomaly report — ranked list of flagged procurement notices with scores and explanations
• Cluster plots — visualisations showing where anomalies sit relative to the broader dataset
• Pipeline log — full execution log at pipeline.log

---

🌐 Data Source

All procurement data is sourced from the EU TED API — the official platform for European public procurement notices, published under EU open data policies.

https://ted.europa.eu/api/