From 5d5e7f9bb63de37e8d187bc814ec3dfa8a5eccd8 Mon Sep 17 00:00:00 2001 From: Tobias Wietelmann Date: Mon, 27 Jul 2026 20:58:30 +0200 Subject: [PATCH 1/6] Add GitHub Pages configuration --- docs/_config.yml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 docs/_config.yml diff --git a/docs/_config.yml b/docs/_config.yml new file mode 100644 index 0000000..b06ac29 --- /dev/null +++ b/docs/_config.yml @@ -0,0 +1,15 @@ +title: Flask Country Data API +description: Reproducible World Bank ingestion, SQLite persistence, SQL data-quality checks and a documented Flask read API. +theme: jekyll-theme-cayman +show_downloads: false +repository: DataTideHH/flask-country-data-api +url: https://datatidehh.github.io +baseurl: /flask-country-data-api +lang: en + +plugins: + - jekyll-relative-links + +relative_links: + enabled: true + collections: true From 0fcb43c6b5b3fc1ba8b5f472ace71cecda49dcf5 Mon Sep 17 00:00:00 2001 From: Tobias Wietelmann Date: Mon, 27 Jul 2026 20:59:03 +0200 Subject: [PATCH 2/6] Add compact project landing page --- docs/index.md | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 docs/index.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..f23ac16 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,75 @@ +--- +layout: default +title: Flask Country Data API +description: Reproducible World Bank ingestion, SQLite persistence, SQL data-quality checks and a documented Flask read API. +--- + +# Flask Country Data API + +A compact data-integration project that turns external World Bank country metadata and population observations into a controlled, explainable SQLite-backed API. + +The project is designed as evidence for Data/BI, process-analysis and Python/API work rather than as a generic backend showcase. + +## Controlled data workflow + +```text +World Bank API or versioned fixtures +→ source validation +→ normalization +→ transactional SQLite persistence +→ SQL data-quality checks +→ versioned Flask read API +``` + +Normal API requests read the last successfully committed local database state. External retrieval happens only through an explicit refresh command, so source availability is separated from data delivery. + +## Portfolio evidence + +| Area | What the repository demonstrates | +|---|---| +| Data integration | Explicit source retrieval, timeout handling and deterministic fixtures | +| Data modelling | Country master data, annual population observations and ingestion history | +| Data quality | Ten named SQL checks with error and warning severity | +| Process observability | Successful and failed refresh runs with timestamps and row counts | +| API design | Versioned endpoints, bounded parameters and stable JSON errors | +| Documentation | Architecture, ERD, data dictionary, provenance and OpenAPI 3.1 | +| Reproducibility | Python 3.12 CI on Ubuntu and Windows without live network calls | + +## API surface + +| Endpoint | Purpose | +|---|---| +| `GET /health` | Database availability and basic counts | +| `GET /api/v1/countries` | Filtered country list | +| `GET /api/v1/countries/` | Country details and latest population | +| `GET /api/v1/countries//population` | Population history for a selected period | +| `GET /api/v1/summary` | Dataset-level metrics | +| `GET /api/v1/data-quality` | SQL data-quality report | +| `GET /api/v1/ingestion-runs` | Recent refresh execution history | + +The machine-readable contract is maintained in [`openapi/openapi.yaml`](https://github.com/DataTideHH/flask-country-data-api/blob/main/openapi/openapi.yaml). + +## Documentation + +- [Architecture and component responsibilities](architecture.md) +- [Relational data model](data-model.md) +- [Data dictionary](data-dictionary.md) +- [Data provenance](data-provenance.md) +- [Data-quality rules and status logic](data-quality-notes.md) +- [SQLite schema](https://github.com/DataTideHH/flask-country-data-api/blob/main/sql/schema.sql) +- [Version-controlled SQL quality checks](https://github.com/DataTideHH/flask-country-data-api/blob/main/sql/data_quality_queries.sql) + +## Scope boundary + +This is intentionally a small, understandable project. It does not add an unrelated frontend, authentication system, container platform or cloud deployment. The value lies in the controlled path from external data to validated persistence, observable processing and a documented read contract. + +## Connected portfolio projects + +- [Music Production Data Lab](https://datatidehh.github.io/music-production-data-lab/) — relational modelling, SQL reporting views, data quality and a documented Power BI semantic model +- [Network Operations Data Lab](https://datatidehh.github.io/network-operations-data-lab/) — Python, SQL and BI-oriented reporting over sanitized infrastructure records +- [Spring Boot Process API Basics](https://datatidehh.github.io/spring-boot-process-api-basics/) — supporting Java REST API progression +- [DataTideHH portfolio overview](https://datatidehh.github.io/DataTideHH/) + +## Repository + +Source code, tests and complete setup instructions: [github.com/DataTideHH/flask-country-data-api](https://github.com/DataTideHH/flask-country-data-api) From 431546526467292d5f4d81d5a2713fe90a23787f Mon Sep 17 00:00:00 2001 From: Tobias Wietelmann Date: Mon, 27 Jul 2026 20:59:46 +0200 Subject: [PATCH 3/6] Apply unified DataTideHH Pages design --- docs/assets/css/style.scss | 291 +++++++++++++++++++++++++++++++++++++ 1 file changed, 291 insertions(+) create mode 100644 docs/assets/css/style.scss diff --git a/docs/assets/css/style.scss b/docs/assets/css/style.scss new file mode 100644 index 0000000..a7a1ee1 --- /dev/null +++ b/docs/assets/css/style.scss @@ -0,0 +1,291 @@ +--- +--- + +@import "{{ site.theme }}"; + +:root { + --hanse-blue: #0a2f4f; + --hanse-blue-deep: #07243c; + --tide-teal: #0b92b6; + --tide-teal-light: #43c4df; + --paper: #fafaf7; + --surface: #ffffff; + --ink: #0b1820; + --ink-muted: #4a5662; + --line: rgba(11, 24, 32, 0.12); + --code-bg: #eef4f5; + --shadow: 0 18px 50px rgba(10, 47, 79, 0.08); + + --brand-sans: "Manrope", -apple-system, BlinkMacSystemFont, "Segoe UI", "Helvetica Neue", Arial, sans-serif; + --brand-display: "Montserrat", -apple-system, BlinkMacSystemFont, "Segoe UI", "Helvetica Neue", Arial, sans-serif; + --brand-mono: ui-monospace, "SFMono-Regular", "SF Mono", "Cascadia Mono", Menlo, Consolas, monospace; +} + +html { + background: var(--paper); +} + +body { + background: var(--paper); + color: var(--ink); + font-family: var(--brand-sans); + font-size: 17px; + line-height: 1.65; + -webkit-font-smoothing: antialiased; + text-rendering: optimizeLegibility; +} + +.page-header { + position: relative; + overflow: hidden; + color: #fff; + background-color: var(--hanse-blue); + background-image: + radial-gradient(circle at 12% 18%, rgba(67, 196, 223, 0.22), transparent 34%), + radial-gradient(circle at 88% 88%, rgba(255, 255, 255, 0.10), transparent 33%), + linear-gradient(135deg, var(--hanse-blue-deep) 0%, var(--hanse-blue) 52%, var(--tide-teal) 100%); + border-bottom: 4px solid var(--tide-teal); +} + +.page-header::after { + content: ""; + position: absolute; + left: -5%; + right: -5%; + bottom: -62px; + height: 112px; + opacity: 0.34; + pointer-events: none; + background: + radial-gradient(75% 90% at 20% 0%, rgba(255, 255, 255, 0.22) 0 49%, transparent 50%), + radial-gradient(75% 90% at 72% 10%, rgba(67, 196, 223, 0.24) 0 49%, transparent 50%); +} + +.project-name, +.main-content h1, +.main-content h2, +.main-content h3, +.main-content h4 { + font-family: var(--brand-display); +} + +.project-name { + position: relative; + z-index: 1; + font-weight: 800; + letter-spacing: -0.025em; +} + +.project-tagline { + position: relative; + z-index: 1; + max-width: 54rem; + margin-right: auto; + margin-left: auto; + color: rgba(255, 255, 255, 0.84); + font-weight: 400; +} + +.page-header .btn { + position: relative; + z-index: 1; + border-color: rgba(255, 255, 255, 0.34); + background: rgba(255, 255, 255, 0.08); + color: #fff; + backdrop-filter: blur(6px); +} + +.page-header .btn:hover, +.page-header .btn:focus { + border-color: rgba(255, 255, 255, 0.68); + background: rgba(255, 255, 255, 0.16); +} + +.main-content { + color: var(--ink); +} + +.main-content h1, +.main-content h2, +.main-content h3, +.main-content h4 { + color: var(--hanse-blue); + font-weight: 700; + letter-spacing: -0.015em; +} + +.main-content h2 { + margin-top: 2.4rem; + padding-bottom: 0.45rem; + border-bottom: 1px solid var(--line); +} + +.main-content a { + color: var(--tide-teal); + font-weight: 600; + text-decoration-thickness: 0.08em; + text-underline-offset: 0.18em; +} + +.main-content a:hover, +.main-content a:focus { + color: var(--hanse-blue); +} + +a:focus-visible, +.btn:focus-visible { + outline: 3px solid var(--tide-teal-light); + outline-offset: 3px; +} + +.main-content code { + color: var(--hanse-blue); + background-color: var(--code-bg); + font-family: var(--brand-mono); + border: 1px solid var(--line); + border-radius: 0.3rem; +} + +.main-content pre { + background: var(--surface); + border: 1px solid var(--line); + border-left: 4px solid var(--tide-teal); + border-radius: 0.55rem; + box-shadow: var(--shadow); +} + +.main-content pre code { + color: var(--ink); + background: transparent; + border: 0; +} + +.main-content blockquote { + color: var(--ink-muted); + border-left: 0.35rem solid var(--tide-teal); + background: rgba(11, 146, 182, 0.055); + padding: 0.8rem 1.1rem; + border-radius: 0 0.45rem 0.45rem 0; +} + +.main-content table { + display: table; + width: 100%; + background: var(--surface); + border-collapse: collapse; + box-shadow: 0 8px 28px rgba(10, 47, 79, 0.045); +} + +.main-content table th { + color: #fff; + background: var(--hanse-blue); + font-family: var(--brand-display); + font-weight: 700; +} + +.main-content table th, +.main-content table td { + border-color: var(--line); +} + +.main-content table tr:nth-child(even) td { + background: rgba(11, 146, 182, 0.035); +} + +.main-content hr { + height: 1px; + background: var(--line); +} + +.site-footer { + color: var(--ink-muted); + border-top-color: var(--line); +} + +.site-footer-owner, +.site-footer-credits { + color: var(--ink-muted); +} + +::selection { + color: var(--ink); + background: rgba(67, 196, 223, 0.34); +} + +@media screen and (min-width: 64em) { + .page-header { + padding: 4.8rem 6rem 5.2rem; + } + + .main-content { + max-width: 68rem; + padding-top: 3rem; + } +} + +@media screen and (max-width: 42em) { + body { + font-size: 16px; + } + + .page-header { + padding-bottom: 3.2rem; + } + + .main-content table { + display: block; + overflow-x: auto; + } +} + +@media (prefers-color-scheme: dark) { + :root { + --hanse-blue: #6fb4dc; + --hanse-blue-deep: #07131d; + --tide-teal: #2dc3e8; + --tide-teal-light: #7bdcf0; + --paper: #0a1620; + --surface: #102330; + --ink: #eef2f5; + --ink-muted: #9aa7b4; + --line: rgba(238, 242, 245, 0.14); + --code-bg: #132a38; + --shadow: 0 18px 50px rgba(0, 0, 0, 0.20); + } + + .page-header { + background-image: + radial-gradient(circle at 12% 18%, rgba(45, 195, 232, 0.19), transparent 34%), + radial-gradient(circle at 88% 88%, rgba(255, 255, 255, 0.07), transparent 33%), + linear-gradient(135deg, #06111a 0%, #0a2f4f 58%, #08748f 100%); + } + + .main-content h1, + .main-content h2, + .main-content h3, + .main-content h4 { + color: var(--hanse-blue); + } + + .main-content table th { + color: #fff; + background: #0a2f4f; + } + + .main-content table tr:nth-child(even) td { + background: rgba(45, 195, 232, 0.04); + } + + .main-content blockquote { + background: rgba(45, 195, 232, 0.065); + } +} + +@media (prefers-reduced-motion: reduce) { + *, + *::before, + *::after { + scroll-behavior: auto !important; + transition: none !important; + } +} From 6abc8246a3893c4b3e6f7665fa4be4c011bd7ea3 Mon Sep 17 00:00:00 2001 From: Tobias Wietelmann Date: Mon, 27 Jul 2026 21:00:21 +0200 Subject: [PATCH 4/6] Validate the OpenAPI contract in tests --- requirements-dev.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/requirements-dev.txt b/requirements-dev.txt index 14e8787..0af8052 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -1,2 +1,3 @@ -r requirements.txt PyYAML>=6.0,<7 +openapi-spec-validator>=0.7,<1 From 8da57a9078efe053ae3f33ece9a1b4902b03ddb3 Mon Sep 17 00:00:00 2001 From: Tobias Wietelmann Date: Mon, 27 Jul 2026 21:00:46 +0200 Subject: [PATCH 5/6] Validate OpenAPI semantics and route coverage --- tests/test_openapi.py | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/tests/test_openapi.py b/tests/test_openapi.py index 615c7f8..5462a81 100644 --- a/tests/test_openapi.py +++ b/tests/test_openapi.py @@ -5,6 +5,7 @@ from pathlib import Path import yaml +from openapi_spec_validator import validate from country_api import create_app @@ -13,24 +14,27 @@ class OpenApiContractTest(unittest.TestCase): - def test_openapi_document_is_valid_yaml_with_expected_paths(self): + def test_openapi_document_is_valid_and_covers_runtime_routes(self): document = yaml.safe_load(OPENAPI_PATH.read_text(encoding="utf-8")) + validate(document) self.assertEqual(document["openapi"], "3.1.0") self.assertEqual(document["info"]["title"], "Flask Country Data API") - expected_paths = { - "/", - "/health", - "/api/v1/countries", - "/api/v1/countries/{code}", - "/api/v1/countries/{code}/population", - "/api/v1/summary", - "/api/v1/data-quality", - "/api/v1/ingestion-runs", - "/openapi/openapi.yaml", - } - self.assertEqual(set(document["paths"]), expected_paths) + with tempfile.TemporaryDirectory() as temporary_directory: + app = create_app( + { + "TESTING": True, + "DATABASE_PATH": str(Path(temporary_directory) / "country.sqlite"), + } + ) + runtime_paths = { + rule.rule.replace("", "{code}") + for rule in app.url_map.iter_rules() + if rule.endpoint != "static" + } + + self.assertEqual(set(document["paths"]), runtime_paths) operation_ids = [ operation["operationId"] From a06a61b7f4cd06d9f53628138efc9c0701db385e Mon Sep 17 00:00:00 2001 From: Tobias Wietelmann Date: Mon, 27 Jul 2026 21:01:31 +0200 Subject: [PATCH 6/6] Link the project site and strengthen contract evidence --- README.md | 33 ++++++++++++--------------------- 1 file changed, 12 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 282dc35..ae4d6e7 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ [![CI](https://github.com/DataTideHH/flask-country-data-api/actions/workflows/ci.yml/badge.svg)](https://github.com/DataTideHH/flask-country-data-api/actions/workflows/ci.yml) +[Project site](https://datatidehh.github.io/flask-country-data-api/) · [OpenAPI contract](openapi/openapi.yaml) · [Architecture](docs/architecture.md) + **Reproducible World Bank ingestion workflow with source validation, constrained SQLite persistence, versioned Flask endpoints, SQL data-quality checks, OpenAPI documentation and cross-platform automated tests.** ## Portfolio purpose @@ -40,8 +42,8 @@ Normal HTTP requests never call the World Bank directly. Data acquisition is an | Process observability | `ingestion_runs` records start, completion, status, row counts and failures | | SQL / Data quality | Ten named checks executed from `sql/data_quality_queries.sql` | | API design | Versioned endpoints, bounded parameters and consistent JSON contracts | -| Documentation | OpenAPI 3.1, architecture diagram, ERD, data dictionary and provenance mapping | -| Automated validation | Unit, persistence, route, CLI, quality and contract tests on Ubuntu and Windows | +| Documentation | OpenAPI 3.1, architecture diagram, ERD, data dictionary, provenance mapping and a compact project site | +| Automated validation | Unit, persistence, route, CLI, quality and validated OpenAPI contract tests on Ubuntu and Windows | ## Architecture @@ -100,7 +102,7 @@ GET /api/v1/data-quality GET /api/v1/ingestion-runs?limit=5 ``` -The complete contract is versioned in [`openapi/openapi.yaml`](openapi/openapi.yaml). +The complete contract is versioned in [`openapi/openapi.yaml`](openapi/openapi.yaml). CI validates both OpenAPI semantics and coverage of the implemented Flask routes. ## Summary response @@ -224,7 +226,7 @@ python -m flask --app country_api:create_app refresh-data \ --to-year 2024 ``` -A failed live refresh is recorded in `ingestion_runs` and does not replace previously committed data. +A failed live refresh is recorded in `ingestion_runs`. Data replacement itself is transactional, so validation or persistence failures do not partially replace the selected country and year range. ## Tests and CI @@ -245,8 +247,9 @@ GitHub Actions validates Python 3.12 on Ubuntu and Windows. The workflow: 1. compiles application and test modules 2. runs unit, persistence, route, CLI, data-quality and OpenAPI tests -3. builds a deterministic SQLite database from fixtures -4. executes the persisted SQL quality report and requires `passed` +3. validates the OpenAPI 3.1 document and its coverage of runtime routes +4. builds a deterministic SQLite database from fixtures +5. executes the persisted SQL quality report and requires `passed` Automated tests never perform live HTTP requests. @@ -256,18 +259,11 @@ Automated tests never perform live HTTP requests. flask-country-data-api/ ├── .github/workflows/ci.yml ├── country_api/ -│ ├── __init__.py -│ ├── cli.py -│ ├── database.py -│ ├── errors.py -│ ├── routes.py -│ ├── service.py -│ ├── validation.py -│ └── world_bank.py ├── data/fixtures/ -│ ├── world-bank-countries.json -│ └── world-bank-population.json ├── docs/ +│ ├── _config.yml +│ ├── index.md +│ ├── assets/css/style.scss │ ├── architecture.md │ ├── data-dictionary.md │ ├── data-model.md @@ -278,11 +274,6 @@ flask-country-data-api/ │ ├── data_quality_queries.sql │ └── schema.sql ├── tests/ -│ ├── test_openapi.py -│ ├── test_reporting.py -│ ├── test_routes_and_cli.py -│ ├── test_service_and_database.py -│ └── test_validation.py ├── main.py ├── requirements-dev.txt ├── requirements.txt