An end-to-end Azure data engineering pipeline implementing a metadata-driven Medallion Architecture to transform raw GitHub CSV data into a structured Star Schema. The project leverages ADF, Databricks (PySpark), and Synapse Analytics to build a scalable, secure, and production-ready data warehouse.
python json csv sql azure pyspark data-engineering warehouse databricks azure-data-factory azure-sql dynamic-pipelines azure-synapse-analytics linked-services databricks-clusters medallion-architecture serverless-sql-pool
-
Updated
Apr 1, 2026 - Jupyter Notebook