Market Rent Intelligence Pipeline

Read-only ETL pipeline and analytics dashboard for the Irish residential property market. Ingests, normalises, and cross-references three official government datasets — RTB (Residential Tenancy Register), PPR (Property Price Register), and CSO (Central Statistics Office) — then exposes a Streamlit dashboard and a FastAPI query layer.

---

Architecture

Data Sources        Ingest              Clean              Marts          Serve
RTB ─────────────► ingest_rtb.py ────► clean_rtb.py ────►
PPR ─────────────► ingest_ppr.py ────► clean_ppr.py ────► build_marts ──► Streamlit
CSO ─────────────► ingest_cso.py ────► clean_cso.py ────►               FastAPI

Stack

Layer	Tools
Ingestion / transformation	Python · pandas · pyarrow
Analytics	DuckDB
Dashboard	Streamlit
API	FastAPI

---

Setup

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Update config/sources.json with current download URLs from the RTB, PPR, and CSO portals.

---

Run the Pipeline

# 1 — Ingest
python src/ingest/ingest_rtb.py
python src/ingest/ingest_ppr.py
python src/ingest/ingest_cso.py

# 2 — Clean
python src/clean/clean_rtb.py
python src/clean/clean_ppr.py
python src/clean/clean_cso.py

# 3 — Build marts
python src/marts/build_marts.py

Each stage is idempotent — re-running is safe.

---

Serve

Dashboard

streamlit run src/app/app.py

API

uvicorn src.api.main:app --reload
# GET /prices?area=Dublin

---

Data Sources

Source	Description
RTB	Residential Tenancy Register — registered tenancy and rent data
PPR	Property Price Register — residential sale prices
CSO	Central Statistics Office — housing market statistics

All data is sourced read-only from official public datasets. No writes to any government system.