diff --git a/README.md b/README.md
index 223f272..6b13be0 100644
--- a/README.md
+++ b/README.md
@@ -3,108 +3,273 @@
[](https://github.com/aleattene/learning-retention-analytics/actions/workflows/test.yml)
[](https://github.com/aleattene/learning-retention-analytics/actions/workflows/code_quality.yml)
[](https://codecov.io/gh/aleattene/learning-retention-analytics)
+[](https://github.com/aleattene/learning-retention-analytics/commits/main)
[](https://www.python.org/downloads/)
+[](https://duckdb.org/)
+[](https://jupyter.org/)
+[](https://pandas.pydata.org/)
+[](https://matplotlib.org/)
[](LICENSE)
[](https://analyse.kmi.open.ac.uk/open_dataset)
+> 32,593 enrollments and seven online courses: where dropout concentrates and which signals announce it
+> within the first 28 days.
+
---
+
+
## Overview
-A **product analytics case study** that analyzes student retention and
-drop-out in online education using the
+A **product analytics case study** that analyzes student retention and drop-out
+in online education, using the
[Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset):
-32,593 enrollments by 28,785 distinct students, 7 courses, complete
-behavioral clickstream.
+- **32,593** enrollments
+- **28,785** distinct students
+- **7** courses
+- complete **behavioral** clickstream
-The project follows a **SQL-driven analytical pipeline**: DuckDB as a
-local-first analytical database, descriptive and inferential statistics,
-and a Looker Studio dashboard.
+The project follows a **SQL-driven** analytical pipeline:
+- **DuckDB** as local analytical database
+- descriptive and inferential **statistics**
-### Why this matters
-Online education platforms face 40-60% drop-out rates. Understanding
-**where**, **when**, and **why** students disengage is the foundation
-for any retention strategy, whether in EdTech, SaaS subscriptions,
-or fitness app engagement.
-> **Dashboard**: Looker Studio *(coming soon)*
+### Why this matters
+
+Online education platforms record average **drop-out rates** of 40-60%.
+Understanding **where**, **when** and **why** students disengage is the foundation of any **retention
+strategy**, whether in EdTech, SaaS subscriptions or fitness apps.
---
+
+
## Business Questions
-| # | Question | Analytical approach |
-|---|----------|---------------------|
-| BQ1 | Where and when do students drop out? | Cohort analysis, cumulative dropout curves, cliff detection |
-| BQ2 | Which early behavioral signals predict drop-out? | Engagement segmentation (first 28 days), t-test, effect size |
-| BQ3 | Does demographics or behavior predict outcome more strongly? | Chi-square, Cramer's V, comparative analysis |
-| BQ4 | How do course characteristics affect retention? | Cross-course comparison, correlation with retention rates |
-| BQ5 | Top 3 actionable interventions for a platform operator? | Segment sizing, impact estimation, cost-benefit framing |
+**Five questions** drive the entire analysis, **from diagnosis to action**:
+
+| # | Question | Analytical approach |
+|---|-----------------------------------------------------------------------|--------------------------------------------------------------------------------|
+| **BQ1** | Where and when do students drop out? | Cohort analysis, cumulative dropout curves and cliff detection. |
+| **BQ2** | Which early behavioral signals predict drop-out? | Engagement segmentation (first 28 days), t-tests and effect sizes. |
+| **BQ3** | Does demographics or behavior predict the outcome more strongly? | Chi-square, Cramer's V and comparative analysis. |
+| **BQ4** | How do course characteristics affect retention? | Cross-course comparison and correlation with retention rates. |
+| **BQ5** | Top 3 actionable interventions for a platform operator? | Segment sizing, impact estimation and cost-benefit analysis. |
+
+---
+
+
+
+## Key Findings
+
+The full analysis is available in the [Executive Report](reports/REPORT.md).
+In a nutshell:
+
+- **BQ1**: roughly **1 enrollment in 3** ends in explicit withdrawal. Dropout concentrates around assessment
+deadlines and grade releases.
+
+
+- **BQ2**: all **8 early behavioral signals** (first 28 days) are significantly associated with dropout.
+Engagement volume (engagement decile, active days, total clicks) dominates the effect size ranking.
+
+
+- **BQ3**: **behavior is a far stronger predictor** than demographics. Within every education level,
+high engagement beats low engagement.
+
+
+- **BQ4**: **completion rates** range from 37% to 71% across the 7 modules. Suggestive patterns with assessment
+density, but n = 7 prevents inferential conclusions.
+
+
+- **BQ5**: **three behavior-based interventions** (ghost activation, assessment checkpoint, week-3
+re-engagement) cover the majority of at-risk students.
+
+---
+
+
+
+### The story in three charts: the problem, the insight and the action
+
+
+*Where the problem lives (BQ1): withdrawal accumulates steadily, with course-specific cliffs around
+assessment deadlines.*
+
+
+
+
+*The core insight (BQ3): early behavioral signals carry far larger effect sizes than any demographic attribute.*
+
+
+
+
+*The action (BQ5): candidate interventions positioned by estimated impact and implementation cost.*
+
+---
+
+
+
+## Dashboard
+
+An interactive Looker Studio dashboard (retention rates by course, dropout curves, at-risk segments) is
+under construction: release is planned for the last quarter of 2026 (with Milestone 03 of the
+[roadmap](#project-status)).
+
+---
+
+
+
+## Method and Limits
+
+The analysis is entirely **observational**: it measures associations, not causal relationships, and uses no
+machine learning models (a deliberate scope choice: analytics, not data science).
+The scope does not rule out a future data science extension, which could build on the same pipeline and the
+same data.
+
+- **Statistical rigor**: every comparison reports p-values and effect sizes (Cohen's d, Cramer's V), with
+multiple comparison correction (Bonferroni\* and Benjamini-Hochberg\*\*) and 95% confidence intervals.
+
+
+- **Effect size before p-value**: with ~32K observations even trivial differences reach statistical
+significance; predictors are ranked by effect size.
+
+
+- **Declared limitation**: there are only 7 courses (n = 7), so course-level comparisons (BQ4) remain
+descriptive, with no inferential conclusions.
+
+\* **Bonferroni**: when many statistical tests run together, the probability that at least one comes out
+"significant" by pure chance grows. The Bonferroni correction compensates by making the significance
+threshold stricter: it divides it by the number of tests performed. More:
+[Bonferroni correction](https://en.wikipedia.org/wiki/Bonferroni_correction) (Wikipedia).
+
+\*\* **Benjamini-Hochberg**: an alternative, less drastic correction: instead of protecting every single test,
+it controls the expected share of false positives among the results declared significant. In this
+project all findings stay significant under both corrections. More:
+[False discovery rate](https://en.wikipedia.org/wiki/False_discovery_rate) (Wikipedia).
+
+The complete guide to reading the numbers (scales and worked examples) lives in the Methodology section
+of the [Executive Report](reports/REPORT.md).
---
+
+
## Methodological Transferability
-Every analytical pattern in this project is portable to other domains:
+Every analytical **pattern** in this project is **portable** to other domains:
+
+| Pattern | EdTech (this project) | SaaS Retention | Subscription Churn | Fitness App |
+|---------|--------------------------|----------------|-------------------|------------------------------------------------------|
+| Cohort analysis | Dropout by enrollment cohort | Trial-to-paid conversion by signup month | Renewal rate by subscription tier | 30-day retention by onboarding flow |
+| Funnel analysis | Enrollment - first click - assessment - completion | Signup - activation - habit - upgrade | Subscribe - use - renew | Download - first workout - weekly habit |
+| Engagement segmentation | Click intensity in first 28 days | Feature adoption in first 14 days | Usage frequency before the renewal window | Session frequency in the first month |
+| Survival-style analysis | Cumulative withdrawal curves | Time-to-churn Kaplan-Meier | Subscription survival by plan type | Days-to-lapse by activity type |
+
+---
+
+
+
+## Analysis Notebooks
+
+The full analysis lives in **7 notebooks**: **2 exploratory**, one for each of the **5 business questions**.
-| Pattern | EdTech (this project) | SaaS Retention | Subscription Churn | Fitness App |
-|---------|----------------------|----------------|---------------------|-------------|
-| Cohort analysis | Enrollment cohort dropout | Trial-to-paid conversion by signup month | Renewal rate by subscription tier | 30-day retention by onboarding flow |
-| Funnel analysis | Registration -> first click -> assessment -> completion | Signup -> activation -> habit -> upgrade | Subscribe -> engage -> renew | Download -> first workout -> weekly habit |
-| Engagement segmentation | Click intensity in first 28 days | Feature adoption in first 14 days | Usage frequency before renewal window | Session frequency in first month |
-| Survival-style dropout | Cumulative withdrawal curves | Time-to-churn Kaplan-Meier | Subscription survival by plan type | Days-to-lapse by activity type |
+| # | Notebook | Focus |
+|--------|----------|------------------------------------------------------------------------|
+| **01** | [EDA: Student Base](notebooks/01_eda_student_base.ipynb) | Population profile, outcomes and data quality baseline. |
+| **02** | [EDA: Engagement Patterns](notebooks/02_eda_engagement_patterns.ipynb) | Clickstream behavior, engagement typologies and ghost students. |
+| **03** | [BQ1: Dropout Timing](notebooks/03_bq1_dropout_timing.ipynb) | Cumulative dropout curves and cliff detection. |
+| **04** | [BQ2: Early Signals](notebooks/04_bq2_early_signals.ipynb) | First-28-day behavioral signals and effect size ranking. |
+| **05** | [BQ3: Demographics vs Behavior](notebooks/05_bq3_demographics_vs_behavior.ipynb) | Comparative predictive strength of the two variable families. |
+| **06** | [BQ4: Course Comparison](notebooks/06_bq4_course_comparison.ipynb) | Course design characteristics and retention. |
+| **07** | [BQ5: Recommendations Synthesis](notebooks/07_bq5_recommendations_synthesis.ipynb) | Segment sizing, priority matrix and top 3 interventions. |
---
+
+
## Tech Stack
-| Layer | Technology | Rationale |
-|-------|------------|-----------|
-| Analytical DB | **DuckDB** (local-first) | Zero-cost, SQL-first, BigQuery migration path |
-| SQL dialect | **ANSI SQL** only | No DuckDB-specific syntax - cloud-portable |
-| Language | **Python 3.13+** | Pipeline orchestration, statistics, visualization |
-| Statistics | **SciPy + statsmodels** | t-test, chi-square, confidence intervals, effect sizes |
-| Visualization | **Matplotlib + Seaborn** | Publication-quality charts |
-| Dashboard | **Looker Studio** | Free, shareable, Google Sheets as data source |
-| CI/CD | **GitHub Actions** | Automated testing + linting |
-| Code quality | **black + ruff + pre-commit** | Consistent formatting and linting |
+| Layer | Technology | Rationale |
+|---------|------------------------------|---------------------------------------------------------------|
+| Analytical DB | **DuckDB** (local-first) | Zero cost, SQL-first and a BigQuery migration path. |
+| SQL dialect | **ANSI SQL** only | No DuckDB-specific syntax and cloud portability. |
+| Language | **Python 3.13+** | Pipeline orchestration, statistics and visualization. |
+| Statistics | **SciPy and statsmodels** | t-tests, chi-square, confidence intervals and effect sizes. |
+| Visualization | **Matplotlib and Seaborn** | Publication-quality charts. |
+| Dashboard | **Looker Studio** | Free, shareable and Google Sheets as data source. |
+| CI/CD | **GitHub Actions** | Automated testing and linting. |
+| Code quality | **black, ruff and pre-commit** | Consistent formatting and linting. |
---
+
+
+## Dataset
+
+The [Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset) contains **32,593 course
+enrollments** by **28,785 distinct students**, across **7 modules** (22 presentations) at The Open University (UK).
+
+| Table | Description | Key columns |
+|---------|-----------------------------------------------------|----------------|
+| studentInfo | Demographics with final outcome | id_student, final_result |
+| studentRegistration | Enrollment and/or unenrollment dates | date_registration, date_unregistration |
+| studentVle | Clickstream (daily clicks per resource) | id_site, date, sum_click |
+| studentAssessment | Assessment scores | id_assessment, score |
+| assessments | Assessment metadata | assessment_type, date, weight |
+| vle | VLE (Virtual Learning Environment) resource metadata | activity_type |
+| courses | Course metadata | module_presentation_length |
+
+**Target variable**: the enrollment's final outcome (`final_result`). In the original data it takes four
+values: Pass, Distinction (pass with merit), Fail and Withdrawn (withdrew from the course).
+For the analysis the four outcomes are grouped into two classes (outcome binarization):
+
+- **Completed**: Pass or Distinction, the course was finished successfully;
+- **Not completed**: Fail or Withdrawn, the course was not carried through, whether by failing or by dropping out.
+
+> **Citation**: Kuzilek, J., Hlosta, M., & Zdrahal, Z. (2017).
+> Open University Learning Analytics dataset.
+> *Scientific Data*, 4, 170171.
+> Licensed under [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/).
+
+---
+
+
+
## Project Structure
```
project_root/
-├── run_pipeline.py # Entrypoint: orchestrates ETL
+├── run_pipeline.py # Entrypoint: orchestrates the ETL
├── src/
-│ ├── config.py # Paths, constants, env vars
+│ ├── config.py # Paths, constants, environment variables
│ ├── db/connection.py # DB abstraction (DuckDB now, BigQuery later)
│ ├── pipeline/
-│ │ ├── step_01_ingest.py # CSV OULAD → raw DuckDB tables
-│ │ ├── step_02_transform.py # Raw tables → analytical views
-│ │ ├── step_03_export.py # Views → CSV + optional Sheets push
-│ │ └── step_04_stats.py # BQ2/BQ3 statistical tests → CSV
+│ │ ├── step_01_ingest.py # OULAD CSVs: raw DuckDB tables
+│ │ ├── step_02_transform.py # Raw tables: analytical views
+│ │ ├── step_03_export.py # Views: CSVs with optional Sheets push
+│ │ └── step_04_stats.py # BQ2/BQ3 statistical tests: CSVs
│ ├── stats/tests.py # Statistical test wrappers
│ ├── sheets/push.py # Google Sheets integration
│ └── utils/ # Logging, runtime utilities
├── sql/
-│ ├── schema.sql # DDL for 7 raw OULAD tables
+│ ├── schema.sql # DDL for the 7 raw OULAD tables
│ ├── views/ # 5 analytical views
│ └── queries/ # 5 business question queries
-├── notebooks/ # 7 analysis notebooks (EDA + BQ1-BQ5)
+├── notebooks/ # 7 analysis notebooks (EDA and BQ1-BQ5)
├── reports/
│ ├── REPORT.md # Executive report (IT mirror in reports/it/)
│ └── figures/ # Charts exported by the notebooks
├── data_sample/ # Synthetic data (~200 students) for CI
├── it/ # Italian mirror of this README
-├── tests/ # pytest suite (unit + integration + stress)
-└── .github/workflows/ # test.yml + code_quality.yml
+├── tests/ # pytest suite (unit, integration and stress)
+└── .github/workflows/ # test.yml and code_quality.yml
```
---
-## Quick Start
+
+
+## Getting Started
### Prerequisites
@@ -118,20 +283,27 @@ project_root/
git clone https://github.com/aleattene/learning-retention-analytics.git
cd learning-retention-analytics
-# Create and activate virtual environment
+# Create and activate the virtual environment
python -m venv .venv
-source .venv/bin/activate
-# Install dependencies from pinned lockfiles
+# macOS and Linux
+source .venv/bin/activate
+
+# Windows:
+.venv\Scripts\activate
+
+# Install dependencies from the pinned lockfiles
pip install pip-tools
pip-sync requirements-dev.txt
-# Install pre-commit hooks
+# Install the pre-commit hooks
pre-commit install
+
+# To update dependencies, edit the `.in` files and re-compile with the following command
+pip-compile requirements.in && pip-compile requirements-dev.in && pip-compile requirements-test.in
```
-> **Maintainer note**: to update dependencies, edit the `.in` files and
-> re-compile: `pip-compile requirements.in && pip-compile requirements-dev.in && pip-compile requirements-test.in`
+
### Download the OULAD dataset
@@ -139,11 +311,12 @@ pre-commit install
python scripts/download_oulad.py
```
-This downloads the full OULAD dataset (~450 MB) into `data/raw/`.
+This command downloads the full OULAD dataset (~450 MB) into `data/raw/`.
-> **Note**: the download script is not yet published *(coming soon)*. In the
-> meantime, the dataset can be downloaded manually from the official OULAD
-> page linked in the [Dataset](#dataset) section.
+> **Note**: the download script is not yet published *(coming in September 2026)*.
+>
+> In the meantime, the dataset can be downloaded manually from the official OULAD page linked in the
+> [Dataset](#dataset) section.
### Run the pipeline
@@ -167,95 +340,37 @@ pytest tests/test_smoke.py -v
---
-## Dataset
-
-The [Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset)
-contains 32,593 course enrollments by 28,785 distinct students across
-7 modules (22 presentations) at The Open University (UK).
-
-| Table | Description | Key columns |
-|-------|-------------|-------------|
-| studentInfo | Demographics + final outcome | id_student, final_result |
-| studentRegistration | Enrollment/unenrollment dates | date_registration, date_unregistration |
-| studentVle | Clickstream (daily clicks per resource) | id_site, date, sum_click |
-| studentAssessment | Assessment scores | id_assessment, score |
-| assessments | Assessment metadata | assessment_type, date, weight |
-| vle | Virtual Learning Environment (VLE) resource metadata | activity_type |
-| courses | Course metadata | module_presentation_length |
+
-**Target variable**: `final_result` ∈ {Pass, Distinction, Fail, Withdrawn},
-binarized as Completed (Pass + Distinction) vs Not completed (Fail + Withdrawn).
-
-> **Citation**: Kuzilek, J., Hlosta, M., & Zdrahal, Z. (2017).
-> Open University Learning Analytics dataset.
-> *Scientific Data*, 4, 170171.
-> Licensed under [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/).
-
----
-
-## Analysis Notebooks
+## Documentation
-The full analysis lives in 7 notebooks: two exploratory, one per
-business question.
+The full extended documentation is in preparation: it will be published soon, with Milestone 02
+of the [roadmap](#project-status).
-| # | Notebook | Focus |
-|---|----------|-------|
-| 01 | [EDA: Student Base](notebooks/01_eda_student_base.ipynb) | Population profile, outcomes, data quality baseline |
-| 02 | [EDA: Engagement Patterns](notebooks/02_eda_engagement_patterns.ipynb) | Clickstream behavior, engagement typologies, ghost students |
-| 03 | [BQ1: Dropout Timing](notebooks/03_bq1_dropout_timing.ipynb) | Cumulative dropout curves, cliff detection |
-| 04 | [BQ2: Early Signals](notebooks/04_bq2_early_signals.ipynb) | First-28-day behavioral signals, effect size ranking |
-| 05 | [BQ3: Demographics vs Behavior](notebooks/05_bq3_demographics_vs_behavior.ipynb) | Comparative predictive strength of the two feature families |
-| 06 | [BQ4: Course Comparison](notebooks/06_bq4_course_comparison.ipynb) | Course design features vs retention |
-| 07 | [BQ5: Recommendations Synthesis](notebooks/07_bq5_recommendations_synthesis.ipynb) | Segment sizing, priority matrix, top 3 interventions |
+| Document | Content |
+|-----------|----------------------------------------------------------------------|
+| [Executive Report](reports/REPORT.md) | Full BQ1–BQ5 analysis with figures and numbers. |
+| [Methodology](docs/METHODOLOGY.md) | Statistical approach, design choices and trade-offs. *(coming soon)* |
+| [Transferability](docs/TRANSFERABILITY.md) | Pattern portability to SaaS, subscriptions and fitness. *(coming soon)* |
+| [Cloud Migration](docs/MIGRATION.md) | DuckDB to BigQuery path, gaps and checklist. *(coming soon)* |
+| [ADR](docs/ADR.md) | Architectural decisions. *(coming soon)* |
+| [Testing](docs/TESTING.md) | Test architecture, strategy and decisions. *(coming soon)* |
---
-## Key Findings
+
-The full analysis is available in the [Executive Report](reports/REPORT.md).
-In summary:
-
-- **BQ1**: roughly 1 in 3 enrollments ends in explicit withdrawal; dropout
- clusters around assessment deadlines and grade releases
-- **BQ2**: all 8 early behavioral signals (first 28 days) are significantly
- associated with dropout; engagement volume (engagement decile, active days,
- total clicks) dominates the effect size ranking
-- **BQ3**: behavior predicts outcome far more strongly than demographics;
- within every education level, high engagement beats low engagement
-- **BQ4**: completion rates range from 37% to 71% across the 7 modules;
- suggestive patterns with assessment density, but n = 7 prevents inferential
- conclusions
-- **BQ5**: three behavior-based interventions (ghost activation, assessment
- checkpoint, week-3 re-engagement) cover the majority of at-risk students
-
-The story in three charts: the problem, the insight, the action.
+## Project Status
-
-*Where the problem lives (BQ1): withdrawal accumulates steadily, with
-course-specific cliffs around assessment deadlines.*
-
-
-*The core insight (BQ3): early behavioral signals carry far larger effect
-sizes than any demographic attribute.*
-
-
-*The action (BQ5): candidate interventions positioned by estimated impact
-and implementation cost.*
+- [x] **Milestone 01**: end-to-end retention analysis (ETL pipeline, 7 analysis notebooks, inferential
+statistics with dedicated export, EN/IT README and executive report, test suite with CI, v1.0 release)
+- [ ] **Milestone 02**: publication of the extended documentation (methodology, transferability, cloud
+migration, architectural decisions, testing) and of the download and data generation scripts
+- [ ] **Milestone 03**: interactive Looker Studio dashboard connected to the pipeline via Google Sheets
---
-## Documentation
-
-| Document | Content |
-|----------|---------|
-| [Executive Report](reports/REPORT.md) | Full BQ1–BQ5 analysis with figures and numbers |
-| [Methodology](docs/METHODOLOGY.md) | Statistical approach, design choices, trade-offs *(coming soon)* |
-| [Transferability](docs/TRANSFERABILITY.md) | Pattern portability to SaaS, subscriptions, fitness *(coming soon)* |
-| [Cloud Migration](docs/MIGRATION.md) | DuckDB to BigQuery path, gaps and checklist *(coming soon)* |
-| [ADR](docs/ADR.md) | Architectural decisions with rationale *(coming soon)* |
-| [Testing](docs/TESTING.md) | Test architecture, strategy, and decisions *(coming soon)* |
-
----
+
## Author
@@ -263,9 +378,10 @@ and implementation cost.*
---
-## License
+
+
+## Licenses
-This project is licensed under the [MIT License](LICENSE).
+This **project** is licensed under the [MIT License](LICENSE).
-The OULAD dataset is licensed under
-[CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) - see citation above.
+The **OULAD** dataset is licensed under [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/).
diff --git a/it/README.md b/it/README.md
index 0fe2dfe..b8b1b77 100644
--- a/it/README.md
+++ b/it/README.md
@@ -3,75 +3,239 @@
[](https://github.com/aleattene/learning-retention-analytics/actions/workflows/test.yml)
[](https://github.com/aleattene/learning-retention-analytics/actions/workflows/code_quality.yml)
[](https://codecov.io/gh/aleattene/learning-retention-analytics)
+[](https://github.com/aleattene/learning-retention-analytics/commits/main)
[](https://www.python.org/downloads/)
+[](https://duckdb.org/)
+[](https://jupyter.org/)
+[](https://pandas.pydata.org/)
+[](https://matplotlib.org/)
[](../LICENSE)
[](https://analyse.kmi.open.ac.uk/open_dataset)
+> 32.593 iscrizioni e sette corsi online: dove si concentra l'abbandono e quali segnali lo annunciano
+> già nei primi 28 giorni.
+
---
+
+
## Panoramica
-Un **case study di product analytics** che analizza la retention e
-l'abbandono degli studenti nella formazione online, utilizzando
+Un **caso studio di product analytics** che analizza la retention e l'abbandono degli studenti
+nella formazione online, utilizzando
l'[Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset):
-32.593 iscrizioni di 28.785 studenti distinti, 7 corsi, clickstream
-comportamentale completo.
+- **32.593** iscrizioni
+- **28.785** studenti distinti
+- **7** corsi
+- clickstream **comportamentale** completo
-Il progetto segue una **pipeline analitica SQL-driven**: DuckDB come
-database analitico locale, statistica descrittiva e inferenziale, e una
-dashboard in Looker Studio.
+Il progetto segue una pipeline analitica **SQL-driven**:
+- **DuckDB** come database analitico locale
+- **statistica** descrittiva e inferenziale
-### Perché è rilevante
-Le piattaforme di formazione online registrano tassi di abbandono del
-40-60%. Comprendere **dove**, **quando** e **perché** gli studenti si
-disimpegnano è la base di qualsiasi strategia di retention, che si
-tratti di EdTech, abbonamenti SaaS o app per il fitness.
-> **Dashboard**: Looker Studio *(in arrivo)*
+### Perché è rilevante
+
+Le piattaforme di formazione online registrano mediamente **tassi di abbandono** del 40-60%.
+Comprendere **dove**, **quando** e **perché** gli studenti si disimpegnano è la base di qualsiasi **strategia di
+retention**, che si tratti di EdTech, abbonamenti a SaaS o app per il fitness.
---
+
+
## Domande di Business
-| # | Domanda | Approccio analitico |
-|---|---------|---------------------|
-| BQ1 | Dove e quando gli studenti abbandonano? | Analisi per coorte, curve cumulative di abbandono, rilevamento punti critici |
-| BQ2 | Quali segnali comportamentali precoci predicono l'abbandono? | Segmentazione dell'engagement (primi 28 giorni), t-test, effect size |
-| BQ3 | Le variabili demografiche o comportamentali predicono meglio l'esito? | Chi-quadrato, V di Cramer, analisi comparativa |
-| BQ4 | Come le caratteristiche dei corsi influenzano la retention? | Confronto tra corsi, correlazione con i tassi di retention |
-| BQ5 | Top 3 interventi azionabili per un operatore di piattaforma? | Dimensionamento dei segmenti, stima dell'impatto, analisi costi-benefici |
+**Cinque domande** guidano l'intera analisi, **dalla diagnosi all'azione**:
+
+| # | Domanda | Approccio analitico |
+|---|-----------------------------------------------------------------------|--------------------------------------------------------------------------------|
+| **BQ1** | Dove e quando gli studenti abbandonano? | Analisi per coorte, curve cumulative di abbandono e rilevamento punti critici. |
+| **BQ2** | Quali segnali comportamentali precoci predicono l'abbandono? | Segmentazione dell'engagement (primi 28 giorni), t-test ed effect size. |
+| **BQ3** | Le variabili demografiche o comportamentali predicono meglio l'esito? | Chi-quadrato, V di Cramér e analisi comparativa. |
+| **BQ4** | Come le caratteristiche dei corsi influenzano la retention? | Confronto tra corsi e correlazione con i tassi di retention. |
+| **BQ5** | Migliori 3 interventi concreti per un operatore di piattaforma? | Dimensionamento dei segmenti, stima dell'impatto e analisi costi-benefici. |
+
+---
+
+
+
+## Risultati Principali
+
+L'analisi completa è disponibile nel [Report Esecutivo](../reports/it/REPORT.md).
+In pillole:
+
+- **BQ1**: circa **1 iscrizione su 3** termina con il ritiro esplicito. Il dropout si concentra intorno a scadenze
+di valutazione e rilascio voti.
+
+
+- **BQ2**: tutti gli **8 segnali comportamentali precoci** (primi 28 giorni) sono significativamente associati
+all'abbandono. Il volume di engagement (decile di engagement, giorni attivi, click totali) domina il ranking per
+effect size.
+
+
+- **BQ3**: **il comportamento è un predittore** molto più forte della demografia. In ogni livello di istruzione,
+l'engagement alto batte l'engagement basso.
+
+
+- **BQ4**: i **tassi di completamento** variano dal 37% al 71% tra i 7 moduli. Pattern suggestivi con la densità di
+valutazioni, ma n = 7 non consente conclusioni inferenziali.
+
+
+- **BQ5**: **tre interventi behavior-based** (attivazione ghost, checkpoint valutazioni, re-engagement terza
+settimana) coprono la maggioranza degli studenti a rischio.
+
+---
+
+
+
+### La storia in tre grafici: il problema, l'insight e l'azione
+
+
+*Dove vive il problema (BQ1): il ritiro si accumula costantemente, con punti critici specifici per corso intorno alle
+scadenze di valutazione.*
+
+
+
+
+*L'insight centrale (BQ3): i segnali comportamentali precoci mostrano effect size molto più ampi di qualunque attributo demografico.*
+
+
+
+
+*L'azione (BQ5): gli interventi candidati posizionati per impatto stimato e costo di implementazione.*
+
+---
+
+
+
+## Dashboard
+
+Una dashboard interattiva in Looker Studio (tassi di retention per corso, curve di abbandono, segmenti a rischio) è in
+fase di realizzazione: è previsto il rilascio per l'ultimo trimestre del 2026 (con la Milestone 03 della
+[roadmap](#stato-del-progetto)).
+
+---
+
+
+
+## Metodo e Limiti
+
+L'analisi è interamente **osservazionale**: misura associazioni, non relazioni causali e non utilizza modelli di
+machine learning (scelta deliberata di perimetro: analytics e non data science).
+Il perimetro non esclude comunque un'integrazione futura della data science, che potrebbe innestarsi sulla stessa
+pipeline e sugli stessi dati.
+
+- **Rigore statistico**: ogni confronto riporta p-value ed effect size (d di Cohen, V di Cramér), con correzione per
+confronti multipli (Bonferroni\* e Benjamini-Hochberg\*\*) e intervalli di confidenza al 95%.
+
+
+- **Effect size prima del p-value**: con ~32K osservazioni anche differenze banali raggiungono la significatività
+statistica; il criterio di ranking dei predittori è la dimensione dell'effetto.
+
+
+- **Limite dichiarato**: i corsi sono solo 7 (n = 7), quindi i confronti a livello di corso (BQ4) restano descrittivi,
+senza conclusioni inferenziali.
+
+\* **Bonferroni**: quando si eseguono molti test statistici insieme, cresce la probabilità che almeno uno
+risulti "significativo" per puro caso. La correzione di Bonferroni compensa rendendo la soglia di
+significatività più severa: la divide per il numero di test eseguiti. Approfondimento:
+[Bonferroni correction](https://en.wikipedia.org/wiki/Bonferroni_correction) (Wikipedia, in inglese).
+
+\*\* **Benjamini-Hochberg**: correzione alternativa e meno drastica: invece di proteggere ogni singolo test,
+tiene sotto controllo la quota attesa di falsi positivi tra i risultati dichiarati significativi. In questo
+progetto tutti i risultati restano significativi con entrambe le correzioni. Approfondimento:
+[False discovery rate](https://en.wikipedia.org/wiki/False_discovery_rate) (Wikipedia, in inglese).
+
+La guida completa alla lettura dei numeri (scale ed esempi svolti) si trova nella sezione Metodologia
+del [Report Esecutivo](../reports/it/REPORT.md).
---
+
+
## Trasferibilità Metodologica
-Ogni pattern analitico di questo progetto è portabile ad altri domini:
+Ogni **pattern** analitico di questo progetto è **portabile** ad altri domini:
+
+| Pattern | EdTech (questo progetto) | Retention SaaS | Churn Abbonamenti | App Fitness |
+|---------|--------------------------|----------------|-------------------|------------------------------------------------------|
+| Analisi per coorte | Abbandono per coorte di iscrizione | Conversione trial-to-paid per mese di registrazione | Tasso di rinnovo per fascia di abbonamento | Retention a 30 giorni per flusso di onboarding |
+| Analisi funnel | Iscrizione - primo click - valutazione - completamento | Registrazione - attivazione - abitudine - upgrade | Sottoscrizione - utilizzo - rinnovo | Download - primo allenamento - abitudine settimanale |
+| Segmentazione engagement | Intensità click nei primi 28 giorni | Adozione funzionalità nei primi 14 giorni | Frequenza d'uso prima della finestra di rinnovo | Frequenza sessioni nel primo mese |
+| Analisi di sopravvivenza | Curve cumulative di ritiro | Time-to-churn Kaplan-Meier | Sopravvivenza abbonamento per tipo di piano | Giorni al disimpegno per tipo di attività |
+
+---
+
+
+
+## Notebook di Analisi
+
+L'analisi completa vive in **7 notebook**: **2 esplorativi**, uno per ognuna delle **5 domande di business**.
-| Pattern | EdTech (questo progetto) | Retention SaaS | Churn Abbonamenti | App Fitness |
-|---------|--------------------------|----------------|-------------------|-------------|
-| Analisi per coorte | Abbandono per coorte di iscrizione | Conversione trial-to-paid per mese di registrazione | Tasso di rinnovo per fascia di abbonamento | Retention a 30 giorni per flusso di onboarding |
-| Analisi funnel | Iscrizione -> primo click -> valutazione -> completamento | Registrazione -> attivazione -> abitudine -> upgrade | Sottoscrizione -> utilizzo -> rinnovo | Download -> primo allenamento -> abitudine settimanale |
-| Segmentazione engagement | Intensità click nei primi 28 giorni | Adozione funzionalità nei primi 14 giorni | Frequenza d'uso prima della finestra di rinnovo | Frequenza sessioni nel primo mese |
-| Analisi di sopravvivenza | Curve cumulative di ritiro | Time-to-churn Kaplan-Meier | Sopravvivenza abbonamento per tipo di piano | Giorni-al-disimpegno per tipo di attività |
+| # | Notebook | Focus |
+|--------|----------|------------------------------------------------------------------------|
+| **01** | [EDA: Student Base](../notebooks/01_eda_student_base.ipynb) | Profilo della popolazione, esiti e baseline di qualità dei dati. |
+| **02** | [EDA: Engagement Patterns](../notebooks/02_eda_engagement_patterns.ipynb) | Comportamento clickstream, tipologie di engagement e studenti ghost. |
+| **03** | [BQ1: Dropout Timing](../notebooks/03_bq1_dropout_timing.ipynb) | Curve cumulative di abbandono e rilevamento punti critici. |
+| **04** | [BQ2: Early Signals](../notebooks/04_bq2_early_signals.ipynb) | Segnali comportamentali nei primi 28 giorni e ranking per effect size. |
+| **05** | [BQ3: Demographics vs Behavior](../notebooks/05_bq3_demographics_vs_behavior.ipynb) | Forza predittiva comparata delle due famiglie di variabili. |
+| **06** | [BQ4: Course Comparison](../notebooks/06_bq4_course_comparison.ipynb) | Caratteristiche di design dei corsi e retention. |
+| **07** | [BQ5: Recommendations Synthesis](../notebooks/07_bq5_recommendations_synthesis.ipynb) | Dimensionamento dei segmenti, matrice di priorità e top 3 interventi. |
---
+
+
## Stack Tecnologico
-| Livello | Tecnologia | Motivazione |
-|---------|------------|-------------|
-| DB analitico | **DuckDB** (local-first) | Costo zero, SQL-first, percorso di migrazione a BigQuery |
-| Dialetto SQL | Solo **ANSI SQL** | Nessuna sintassi DuckDB-specifica, portabile su cloud |
-| Linguaggio | **Python 3.13+** | Orchestrazione pipeline, statistica, visualizzazione |
-| Statistica | **SciPy + statsmodels** | t-test, chi-quadrato, intervalli di confidenza, effect size |
-| Visualizzazione | **Matplotlib + Seaborn** | Grafici di qualità pubblicabile |
-| Dashboard | **Looker Studio** | Gratuito, condivisibile, Google Sheets come data source |
-| CI/CD | **GitHub Actions** | Testing e linting automatizzati |
-| Qualità codice | **black + ruff + pre-commit** | Formattazione e linting consistenti |
+| Livello | Tecnologia | Motivazione |
+|---------|------------------------------|---------------------------------------------------------------|
+| DB analitico | **DuckDB** (local-first) | Costo zero, SQL-first e percorso di migrazione a BigQuery. |
+| Dialetto SQL | Solo **ANSI SQL** | Nessuna sintassi DuckDB-specifica e portabilità su cloud. |
+| Linguaggio | **Python 3.13+** | Orchestrazione pipeline, statistica e visualizzazione. |
+| Statistica | **SciPy e statsmodels** | t-test, chi-quadrato, intervalli di confidenza ed effect size. |
+| Visualizzazione | **Matplotlib e Seaborn** | Grafici di qualità pubblicabile. |
+| Dashboard | **Looker Studio** | Gratuito, condivisibile e Google Sheets come data source. |
+| CI/CD | **GitHub Actions** | Testing e linting automatizzati. |
+| Qualità codice | **black, ruff e pre-commit** | Formattazione e linting consistenti. |
+
+---
+
+
+
+## Dataset
+
+L'[Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset) contiene **32.593 iscrizioni**
+ai **corsi** di **28.785 studenti** distinti, distribuite su **7 moduli** (22 presentazioni) presso la Open University (UK).
+
+| Tabella | Descrizione | Colonne chiave |
+|---------|-----------------------------------------------------|----------------|
+| studentInfo | Dati demografici con esito finale | id_student, final_result |
+| studentRegistration | Date di iscrizione e/o cancellazione | date_registration, date_unregistration |
+| studentVle | Clickstream (click giornalieri per risorsa) | id_site, date, sum_click |
+| studentAssessment | Punteggi delle valutazioni | id_assessment, score |
+| assessments | Metadati delle valutazioni | assessment_type, date, weight |
+| vle | Metadati risorse VLE (Virtual Learning Environment) | activity_type |
+| courses | Metadati dei corsi | module_presentation_length |
+
+**Variabile target**: l'esito finale dell'iscrizione (`final_result`). Nei dati originali può assumere quattro
+valori: Pass (promosso), Distinction (promosso con merito), Fail (bocciato) e Withdrawn (ritirato dal corso).
+Per l'analisi i quattro esiti sono raggruppati in due classi (binarizzazione dell'outcome):
+
+- **Completato**: Pass o Distinction, il corso è stato concluso con successo;
+- **Non completato**: Fail o Withdrawn, il corso non è stato portato a termine, per bocciatura o per abbandono.
+
+> **Citazione**: Kuzilek, J., Hlosta, M., & Zdrahal, Z. (2017).
+> Open University Learning Analytics dataset.
+> *Scientific Data*, 4, 170171.
+> Distribuito con licenza [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/).
---
+
+
## Struttura del Progetto
```
@@ -81,10 +245,10 @@ project_root/
│ ├── config.py # Path, costanti, variabili d'ambiente
│ ├── db/connection.py # Astrazione DB (DuckDB ora, BigQuery in futuro)
│ ├── pipeline/
-│ │ ├── step_01_ingest.py # CSV OULAD → tabelle raw DuckDB
-│ │ ├── step_02_transform.py # Tabelle raw → viste analitiche
-│ │ ├── step_03_export.py # Viste → CSV + push opzionale su Sheets
-│ │ └── step_04_stats.py # Test statistici BQ2/BQ3 → CSV
+│ │ ├── step_01_ingest.py # CSV OULAD: tabelle raw DuckDB
+│ │ ├── step_02_transform.py # Tabelle raw: viste analitiche
+│ │ ├── step_03_export.py # Viste: CSV con push opzionale su Sheets
+│ │ └── step_04_stats.py # Test statistici BQ2/BQ3: CSV
│ ├── stats/tests.py # Wrapper per test statistici
│ ├── sheets/push.py # Integrazione Google Sheets
│ └── utils/ # Logging, utilità runtime
@@ -92,19 +256,21 @@ project_root/
│ ├── schema.sql # DDL per le 7 tabelle raw OULAD
│ ├── views/ # 5 viste analitiche
│ └── queries/ # 5 query per le domande di business
-├── notebooks/ # 7 notebook di analisi (EDA + BQ1-BQ5)
+├── notebooks/ # 7 notebook di analisi (EDA e BQ1-BQ5)
├── reports/
│ ├── REPORT.md # Report esecutivo (specchio IT in reports/it/)
│ └── figures/ # Grafici esportati dai notebook
├── data_sample/ # Dati sintetici (~200 studenti) per CI
├── it/ # Specchio italiano di questo README
-├── tests/ # Suite pytest (unit + integration + stress)
-└── .github/workflows/ # test.yml + code_quality.yml
+├── tests/ # Suite pytest (unit, integration e stress)
+└── .github/workflows/ # test.yml e code_quality.yml
```
---
-## Avvio Rapido
+
+
+## Avvio Progetto
### Prerequisiti
@@ -120,18 +286,25 @@ cd learning-retention-analytics
# Crea e attiva l'ambiente virtuale
python -m venv .venv
-source .venv/bin/activate
-# Installa le dipendenze dai lockfile pinnati
+# macOS e Linux
+source .venv/bin/activate
+
+# Windows:
+.venv\Scripts\activate
+
+# Installa le dipendenze dai lockfile
pip install pip-tools
pip-sync requirements-dev.txt
# Installa i pre-commit hook
pre-commit install
+
+# Per aggiornare le dipendenze, modificare i file `.in` e ricompilare con il seguente comando
+pip-compile requirements.in && pip-compile requirements-dev.in && pip-compile requirements-test.in
```
-> **Nota maintainer**: per aggiornare le dipendenze, modificare i file `.in`
-> e ricompilare: `pip-compile requirements.in && pip-compile requirements-dev.in && pip-compile requirements-test.in`
+
### Download del dataset OULAD
@@ -139,11 +312,12 @@ pre-commit install
python scripts/download_oulad.py
```
-Questo scarica il dataset OULAD completo (~450 MB) in `data/raw/`.
+Con questo comando viene effettuato il download del dataset OULAD completo (~450 MB) in `data/raw/`.
-> **Nota**: lo script di download non è ancora pubblicato *(in arrivo)*.
-> Nel frattempo il dataset può essere scaricato manualmente dalla pagina
-> ufficiale OULAD linkata nella sezione [Dataset](#dataset).
+> **Attenzione**: lo script di download non è ancora pubblicato *(in arrivo nel mese di settembre 2026)*.
+>
+> Nel frattempo il dataset può essere scaricato manualmente dalla pagina ufficiale OULAD linkata nella sezione
+> [Dataset](#dataset).
### Esecuzione della pipeline
@@ -167,96 +341,37 @@ pytest tests/test_smoke.py -v
---
-## Dataset
+
-L'[Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset)
-contiene 32.593 iscrizioni ai corsi di 28.785 studenti distinti, distribuite
-su 7 moduli (22 presentazioni) presso la Open University (UK).
-
-| Tabella | Descrizione | Colonne chiave |
-|---------|-------------|----------------|
-| studentInfo | Dati demografici + esito finale | id_student, final_result |
-| studentRegistration | Date di iscrizione/cancellazione | date_registration, date_unregistration |
-| studentVle | Clickstream (click giornalieri per risorsa) | id_site, date, sum_click |
-| studentAssessment | Punteggi delle valutazioni | id_assessment, score |
-| assessments | Metadati delle valutazioni | assessment_type, date, weight |
-| vle | Metadati risorse VLE (Virtual Learning Environment) | activity_type |
-| courses | Metadati dei corsi | module_presentation_length |
+## Documentazione Completa
-**Variabile target**: `final_result` ∈ {Pass, Distinction, Fail, Withdrawn},
-binarizzata come Completato (Pass + Distinction) vs Non completato (Fail + Withdrawn).
+La documentazione estesa e completa è in preparazione: sarà pubblicata a breve, con la Milestone 02
+della [roadmap](#stato-del-progetto).
-> **Citazione**: Kuzilek, J., Hlosta, M., & Zdrahal, Z. (2017).
-> Open University Learning Analytics dataset.
-> *Scientific Data*, 4, 170171.
-> Distribuito con licenza [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/).
+| Documento | Contenuto |
+|-----------|----------------------------------------------------------------------|
+| [Report Esecutivo](../reports/it/REPORT.md) | Analisi completa BQ1–BQ5 con figure e numeri. |
+| [Metodologia](../docs/it/METHODOLOGY.md) | Approccio statistico, scelte progettuali e trade-off. *(in arrivo)* |
+| [Trasferibilità](../docs/it/TRANSFERABILITY.md) | Portabilità dei pattern a SaaS, abbonamenti e fitness. *(in arrivo)* |
+| [Migrazione Cloud](../docs/it/MIGRATION.md) | Percorso da DuckDB a BigQuery, gap e checklist. *(in arrivo)* |
+| [ADR](../docs/it/ADR.md) | Decisioni architetturali. *(in arrivo)* |
+| [Testing](../docs/it/TESTING.md) | Architettura di test, strategia e decisioni. *(in arrivo)* |
---
-## Notebook di Analisi
-
-L'analisi completa vive in 7 notebook: due esplorativi, uno per ogni
-domanda di business.
-
-| # | Notebook | Focus |
-|---|----------|-------|
-| 01 | [EDA: Student Base](../notebooks/01_eda_student_base.ipynb) | Profilo della popolazione, esiti, baseline di qualità dei dati |
-| 02 | [EDA: Engagement Patterns](../notebooks/02_eda_engagement_patterns.ipynb) | Comportamento clickstream, tipologie di engagement, studenti ghost |
-| 03 | [BQ1: Dropout Timing](../notebooks/03_bq1_dropout_timing.ipynb) | Curve cumulative di abbandono, rilevamento punti critici |
-| 04 | [BQ2: Early Signals](../notebooks/04_bq2_early_signals.ipynb) | Segnali comportamentali nei primi 28 giorni, ranking per effect size |
-| 05 | [BQ3: Demographics vs Behavior](../notebooks/05_bq3_demographics_vs_behavior.ipynb) | Forza predittiva comparata delle due famiglie di variabili |
-| 06 | [BQ4: Course Comparison](../notebooks/06_bq4_course_comparison.ipynb) | Caratteristiche di design dei corsi e retention |
-| 07 | [BQ5: Recommendations Synthesis](../notebooks/07_bq5_recommendations_synthesis.ipynb) | Dimensionamento dei segmenti, matrice di priorità, top 3 interventi |
-
----
+
-## Risultati Principali
+## Stato del Progetto
-L'analisi completa è disponibile nel [Report Esecutivo](../reports/it/REPORT.md).
-In sintesi:
-
-- **BQ1**: circa 1 iscrizione su 3 termina con il ritiro esplicito; il dropout
- si concentra intorno a scadenze di valutazione e rilascio voti
-- **BQ2**: tutti gli 8 segnali comportamentali precoci (primi 28 giorni) sono
- significativamente associati all'abbandono; il volume di engagement (decile
- di engagement, giorni attivi, click totali) domina il ranking per effect size
-- **BQ3**: il comportamento è un predittore molto più forte della demografia;
- in ogni livello di istruzione, l'engagement alto batte l'engagement basso
-- **BQ4**: i tassi di completamento variano dal 37% al 71% tra i 7 moduli;
- pattern suggestivi con la densità di valutazioni, ma n = 7 non consente
- conclusioni inferenziali
-- **BQ5**: tre interventi behavior-based (attivazione ghost, checkpoint
- valutazioni, re-engagement settimana 3) coprono la maggioranza degli
- studenti a rischio
-
-La storia in tre grafici: il problema, l'insight, l'azione.
-
-
-*Dove vive il problema (BQ1): il ritiro si accumula costantemente, con
-punti critici specifici per corso intorno alle scadenze di valutazione.*
-
-
-*L'insight centrale (BQ3): i segnali comportamentali precoci mostrano
-effect size molto più ampi di qualunque attributo demografico.*
-
-
-*L'azione (BQ5): gli interventi candidati posizionati per impatto stimato
-e costo di implementazione.*
+- [x] **Milestone 01**: analisi end-to-end della retention (pipeline ETL, 7 notebook di analisi, statistica
+inferenziale con export dedicato, README e report esecutivo EN/IT, suite di test con CI, release v1.0)
+- [ ] **Milestone 02**: pubblicazione della documentazione estesa (metodologia, trasferibilità, migrazione cloud,
+decisioni architetturali, testing) e degli script di download e generazione dati
+- [ ] **Milestone 03**: dashboard interattiva in Looker Studio collegata alla pipeline via Google Sheets
---
-## Documentazione
-
-| Documento | Contenuto |
-|-----------|-----------|
-| [Report Esecutivo](../reports/it/REPORT.md) | Analisi completa BQ1–BQ5 con figure e numeri |
-| [Metodologia](../docs/it/METHODOLOGY.md) | Approccio statistico, scelte progettuali, trade-off *(in arrivo)* |
-| [Trasferibilità](../docs/it/TRANSFERABILITY.md) | Portabilità dei pattern a SaaS, abbonamenti, fitness *(in arrivo)* |
-| [Migrazione Cloud](../docs/it/MIGRATION.md) | Percorso da DuckDB a BigQuery, gap e checklist *(in arrivo)* |
-| [ADR](../docs/it/ADR.md) | Decisioni architetturali con razionale *(in arrivo)* |
-| [Testing](../docs/it/TESTING.md) | Architettura di test, strategia e decisioni *(in arrivo)* |
-
----
+
## Autore
@@ -264,9 +379,10 @@ e costo di implementazione.*
---
-## Licenza
+
+
+## Licenze
-Questo progetto è distribuito con [Licenza MIT](../LICENSE).
+Questo **progetto** è distribuito con [Licenza MIT](../LICENSE).
-Il dataset OULAD è distribuito con licenza
-[CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) - vedi citazione sopra.
+Il dataset **OULAD** è distribuito con licenza [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/).
diff --git a/notebooks/03_bq1_dropout_timing.ipynb b/notebooks/03_bq1_dropout_timing.ipynb
index 372629f..71a2315 100644
--- a/notebooks/03_bq1_dropout_timing.ipynb
+++ b/notebooks/03_bq1_dropout_timing.ipynb
@@ -77,7 +77,8 @@
"- Notebooks live in `notebooks/` but project modules are in `src/` at the project root. We add the project root to `sys.path` so that `from src.config import ...` works.\n",
"- All database queries go through `src.db.connection.execute_query()`, the project's DB abstraction layer (ADR-003).\n",
"- BQ1's primary SQL query lives in `sql/queries/q_bq1_dropout_curves.sql` and is loaded at runtime from disk. Additional analysis queries in this notebook may be defined inline, but they still execute through the same DB abstraction layer (ADR-003).\n",
- "- Figures are saved to `reports/figures/` at 150 DPI. Since `nbstripout` removes notebook outputs before commit, the saved PNGs are the persistent visual record."
+ "- Figures are saved to `reports/figures/` at 150 DPI. Since `nbstripout` removes notebook outputs before commit, the saved PNGs are the persistent visual record.\n",
+ "- **Figure localization:** figures embedded by the README and the executive report are rendered twice with identical filenames: English to `reports/figures/` (linked by the EN documents) and Italian to `reports/figures/it/` (linked by the IT documents). Notebook-only figures stay English, like the notebooks themselves (ADR-005)."
]
},
{
@@ -201,9 +202,70 @@
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": null,
"id": "5",
"metadata": {},
+ "outputs": [],
+ "source": [
+ "# --- Figure localization (EN canonical + IT) ---\n",
+ "# README and executive report ship in two languages. Figures EMBEDDED by\n",
+ "# those documents are rendered twice with the same filename: English to\n",
+ "# reports/figures/ (linked by the EN docs), Italian to reports/figures/it/\n",
+ "# (linked by the IT docs). Notebook-only figures keep the plain save_fig()\n",
+ "# path and stay English, like the notebooks themselves (ADR-005).\n",
+ "# Only chart furniture (titles, axis labels, legends) is localized;\n",
+ "# data-derived values (module codes, days, counts) keep their source form.\n",
+ "LANGS = ('en', 'it')\n",
+ "FIG_DIRS = {'en': FIGURES_DIR, 'it': FIGURES_DIR / 'it'}\n",
+ "for _dir in FIG_DIRS.values():\n",
+ " _dir.mkdir(parents=True, exist_ok=True)\n",
+ "\n",
+ "\n",
+ "def save_fig_lang(fig, name: str, lang: str) -> None:\n",
+ " \"\"\"Save a localized figure; display only the EN (canonical) render.\"\"\"\n",
+ " path = FIG_DIRS[lang] / f'{name}.png'\n",
+ " fig.savefig(path, dpi=FIG_DPI, bbox_inches='tight', facecolor='white')\n",
+ " print(f' Saved: {path}')\n",
+ " if lang == 'en':\n",
+ " plt.show()\n",
+ " else:\n",
+ " plt.close(fig)\n",
+ "\n",
+ "\n",
+ "# EN entries reuse the shared label constants (single source of truth);\n",
+ "# IT entries translate the same chart furniture.\n",
+ "L10N = {\n",
+ " 'en': {\n",
+ " 'dropout_day': LABEL_DROPOUT_DAY,\n",
+ " 'cumulative_dropout': LABEL_CUMULATIVE_DROPOUT,\n",
+ " 'overlaid_title': 'Cumulative Dropout Curves: All Course-Presentations',\n",
+ " 'module': 'Module',\n",
+ " 'day_word': 'day',\n",
+ " 'n_dropouts': 'Number of dropouts',\n",
+ " 'cliffs_title': 'Top Dropout Cliff Events (days exceeding 95th percentile)',\n",
+ " 'n_precourse': 'Number of pre-course withdrawals',\n",
+ " 'precourse_title': 'Pre-Course Withdrawals by Module (withdrew before day 0)',\n",
+ " },\n",
+ " 'it': {\n",
+ " 'dropout_day': \"Giorno di abbandono (rispetto all'inizio del corso)\",\n",
+ " 'cumulative_dropout': 'Tasso cumulativo di abbandono (%)',\n",
+ " 'overlaid_title': 'Curve cumulative di abbandono: tutte le presentazioni dei corsi',\n",
+ " 'module': 'Modulo',\n",
+ " 'day_word': 'giorno',\n",
+ " 'n_dropouts': 'Numero di abbandoni',\n",
+ " 'cliffs_title': 'Principali eventi cliff di abbandono (giorni oltre il 95° percentile)',\n",
+ " 'n_precourse': 'Numero di ritiri pre-corso',\n",
+ " 'precourse_title': 'Ritiri pre-corso per modulo (ritiro prima del giorno 0)',\n",
+ " },\n",
+ "}\n",
+ "print(f'Figure localization ready: {\", \".join(LANGS)}')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6",
+ "metadata": {},
"source": [
"## 2. Dropout Overview: Scale and Rate\n",
"\n",
@@ -215,7 +277,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "6",
+ "id": "7",
"metadata": {},
"outputs": [],
"source": [
@@ -289,7 +351,7 @@
},
{
"cell_type": "markdown",
- "id": "7",
+ "id": "8",
"metadata": {},
"source": [
"> **Key finding:** Roughly one-third of all enrollments end in explicit withdrawal. The withdrawal rate varies across modules: some courses lose a significantly larger share of students than others.\n",
@@ -299,7 +361,7 @@
},
{
"cell_type": "markdown",
- "id": "8",
+ "id": "9",
"metadata": {},
"source": [
"## 3. Cumulative Dropout Curves\n",
@@ -314,7 +376,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "9",
+ "id": "10",
"metadata": {},
"outputs": [],
"source": [
@@ -330,35 +392,37 @@
"# --- Overlaid cumulative dropout curves ---\n",
"# One line per course-presentation, colored by module. This shows the\n",
"# full landscape: how different cohorts decay over time.\n",
- "fig, ax = plt.subplots(figsize=(12, 7))\n",
+ "# Rendered per language: embedded by both READMEs and both reports.\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=(12, 7))\n",
"\n",
- "for (module, pres), group in df_curves.groupby(['code_module', 'code_presentation']):\n",
- " ax.step(\n",
- " group['dropout_day'], group['cumulative_dropout_rate_pct'],\n",
- " where='post', color=PALETTE_COURSE[module], alpha=0.7,\n",
- " linewidth=1.5, label=None,\n",
- " )\n",
+ " for (module, pres), group in df_curves.groupby(['code_module', 'code_presentation']):\n",
+ " ax.step(\n",
+ " group['dropout_day'], group['cumulative_dropout_rate_pct'],\n",
+ " where='post', color=PALETTE_COURSE[module], alpha=0.7,\n",
+ " linewidth=1.5, label=None,\n",
+ " )\n",
"\n",
- "# Add one legend entry per module (not per presentation)\n",
- "for module in _MODULE_ORDER:\n",
- " if module in df_curves['code_module'].values:\n",
- " ax.plot([], [], color=PALETTE_COURSE[module], linewidth=2, label=module)\n",
- "\n",
- "ax.set_xlabel(LABEL_DROPOUT_DAY)\n",
- "ax.set_ylabel(LABEL_CUMULATIVE_DROPOUT)\n",
- "ax.set_title('Cumulative Dropout Curves: All Course-Presentations')\n",
- "ax.legend(title='Module', bbox_to_anchor=(1.02, 1), loc='upper left')\n",
- "ax.set_ylim(0, None)\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '03_dropout_curves_overlaid')\n",
- "plt.show()"
+ " # Add one legend entry per module (not per presentation)\n",
+ " for module in _MODULE_ORDER:\n",
+ " if module in df_curves['code_module'].values:\n",
+ " ax.plot([], [], color=PALETTE_COURSE[module], linewidth=2, label=module)\n",
+ "\n",
+ " ax.set_xlabel(L['dropout_day'])\n",
+ " ax.set_ylabel(L['cumulative_dropout'])\n",
+ " ax.set_title(L['overlaid_title'])\n",
+ " ax.legend(title=L['module'], bbox_to_anchor=(1.02, 1), loc='upper left')\n",
+ " ax.set_ylim(0, None)\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '03_dropout_curves_overlaid', lang)"
]
},
{
"cell_type": "code",
"execution_count": null,
- "id": "10",
+ "id": "11",
"metadata": {},
"outputs": [],
"source": [
@@ -404,7 +468,7 @@
},
{
"cell_type": "markdown",
- "id": "11",
+ "id": "12",
"metadata": {},
"source": [
"> **Interpretation:**\n",
@@ -417,7 +481,7 @@
},
{
"cell_type": "markdown",
- "id": "12",
+ "id": "13",
"metadata": {},
"source": [
"## 4. Identifying Dropout Cliffs\n",
@@ -430,7 +494,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "13",
+ "id": "14",
"metadata": {},
"outputs": [],
"source": [
@@ -470,34 +534,36 @@
"print(df_cliff_summary.head(15).to_string(index=False))\n",
"\n",
"# --- Visualization: top cliff events across all modules ---\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
"df_cliff_top = df_cliff_summary.head(20)\n",
- "fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
- "colors = [PALETTE_COURSE.get(m, '#999999') for m in df_cliff_top['code_module']]\n",
- "ax.barh(\n",
- " range(len(df_cliff_top)),\n",
- " df_cliff_top['total_dropouts'],\n",
- " color=colors, edgecolor='white',\n",
- ")\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
+ " colors = [PALETTE_COURSE.get(m, '#999999') for m in df_cliff_top['code_module']]\n",
+ " ax.barh(\n",
+ " range(len(df_cliff_top)),\n",
+ " df_cliff_top['total_dropouts'],\n",
+ " color=colors, edgecolor='white',\n",
+ " )\n",
"\n",
- "# Label each bar with module + day for identification\n",
- "labels = [\n",
- " f\"{row['code_module']} - day {int(row['dropout_day'])}\"\n",
- " for _, row in df_cliff_top.iterrows()\n",
- "]\n",
- "ax.set_yticks(range(len(df_cliff_top)))\n",
- "ax.set_yticklabels(labels)\n",
- "ax.set_xlabel('Number of dropouts')\n",
- "ax.set_title('Top Dropout Cliff Events (days exceeding 95th percentile)')\n",
- "ax.invert_yaxis()\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '03_dropout_cliffs')\n",
- "plt.show()"
+ " # Label each bar with module and day for identification\n",
+ " labels = [\n",
+ " f\"{row['code_module']} - {L['day_word']} {int(row['dropout_day'])}\"\n",
+ " for _, row in df_cliff_top.iterrows()\n",
+ " ]\n",
+ " ax.set_yticks(range(len(df_cliff_top)))\n",
+ " ax.set_yticklabels(labels)\n",
+ " ax.set_xlabel(L['n_dropouts'])\n",
+ " ax.set_title(L['cliffs_title'])\n",
+ " ax.invert_yaxis()\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '03_dropout_cliffs', lang)"
]
},
{
"cell_type": "markdown",
- "id": "14",
+ "id": "15",
"metadata": {},
"source": [
"> **Interpretation:**\n",
@@ -510,7 +576,7 @@
},
{
"cell_type": "markdown",
- "id": "15",
+ "id": "16",
"metadata": {},
"source": [
"## 5. Normalized Timeline: Percentage Through Course\n",
@@ -527,7 +593,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "16",
+ "id": "17",
"metadata": {},
"outputs": [],
"source": [
@@ -587,7 +653,7 @@
},
{
"cell_type": "markdown",
- "id": "17",
+ "id": "18",
"metadata": {},
"source": [
"> **Interpretation:**\n",
@@ -600,7 +666,7 @@
},
{
"cell_type": "markdown",
- "id": "18",
+ "id": "19",
"metadata": {},
"source": [
"## 6. Pre-Course Withdrawals\n",
@@ -613,7 +679,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "19",
+ "id": "20",
"metadata": {},
"outputs": [],
"source": [
@@ -646,31 +712,33 @@
"print(df_precourse.to_string(index=False))\n",
"\n",
"# --- Bar chart: pre-course withdrawals by module ---\n",
- "fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
- "colors = [PALETTE_COURSE.get(m, '#999999') for m in df_precourse['code_module']]\n",
- "bars = ax.barh(\n",
- " df_precourse['code_module'], df_precourse['n_precourse'],\n",
- " color=colors, edgecolor='white',\n",
- ")\n",
- "for bar, (_, row) in zip(bars, df_precourse.iterrows()):\n",
- " ax.text(\n",
- " bar.get_width() + ax.get_xlim()[1] * 0.01,\n",
- " bar.get_y() + bar.get_height() / 2,\n",
- " f\"{int(row['n_precourse']):,} ({row['pct_of_precourse']:.1f}%)\",\n",
- " va='center', fontsize=9, color='#333333',\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
+ " colors = [PALETTE_COURSE.get(m, '#999999') for m in df_precourse['code_module']]\n",
+ " bars = ax.barh(\n",
+ " df_precourse['code_module'], df_precourse['n_precourse'],\n",
+ " color=colors, edgecolor='white',\n",
" )\n",
- "ax.set_xlabel('Number of pre-course withdrawals')\n",
- "ax.set_title('Pre-Course Withdrawals by Module (withdrew before day 0)')\n",
- "ax.invert_yaxis()\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '03_precourse_withdrawals')\n",
- "plt.show()"
+ " for bar, (_, row) in zip(bars, df_precourse.iterrows()):\n",
+ " ax.text(\n",
+ " bar.get_width() + ax.get_xlim()[1] * 0.01,\n",
+ " bar.get_y() + bar.get_height() / 2,\n",
+ " f\"{int(row['n_precourse']):,} ({row['pct_of_precourse']:.1f}%)\",\n",
+ " va='center', fontsize=9, color='#333333',\n",
+ " )\n",
+ " ax.set_xlabel(L['n_precourse'])\n",
+ " ax.set_title(L['precourse_title'])\n",
+ " ax.invert_yaxis()\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '03_precourse_withdrawals', lang)"
]
},
{
"cell_type": "markdown",
- "id": "20",
+ "id": "21",
"metadata": {},
"source": [
"> **Key finding:** A measurable share of all withdrawals happen before the course even begins. These students registered but never engaged with any content.\n",
@@ -683,7 +751,7 @@
},
{
"cell_type": "markdown",
- "id": "21",
+ "id": "22",
"metadata": {},
"source": [
"## 7. Dropout Timing by Demographics\n",
@@ -696,7 +764,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "22",
+ "id": "23",
"metadata": {},
"outputs": [],
"source": [
@@ -797,7 +865,7 @@
},
{
"cell_type": "markdown",
- "id": "23",
+ "id": "24",
"metadata": {},
"source": [
"> **Interpretation:**\n",
@@ -810,7 +878,7 @@
},
{
"cell_type": "markdown",
- "id": "24",
+ "id": "25",
"metadata": {},
"source": [
"## 8. Course Design and Dropout Patterns\n",
@@ -823,7 +891,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "25",
+ "id": "26",
"metadata": {},
"outputs": [],
"source": [
@@ -905,7 +973,7 @@
},
{
"cell_type": "markdown",
- "id": "26",
+ "id": "27",
"metadata": {},
"source": [
"> **Interpretation:**\n",
@@ -918,7 +986,7 @@
},
{
"cell_type": "markdown",
- "id": "27",
+ "id": "28",
"metadata": {},
"source": [
"## 9. Key Takeaways and Next Steps\n",
@@ -955,7 +1023,7 @@
},
{
"cell_type": "markdown",
- "id": "28",
+ "id": "29",
"metadata": {},
"source": [
"> **From timing to signals:** This notebook established *when* students leave: the temporal landscape of dropout. But timing alone does not enable prevention. The next step is to identify *what early behaviors predict departure*.\n",
@@ -971,7 +1039,15 @@
"name": "python3"
},
"language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
"name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
"version": "3.13.0"
}
},
diff --git a/notebooks/04_bq2_early_signals.ipynb b/notebooks/04_bq2_early_signals.ipynb
index 094d933..bc1b18a 100644
--- a/notebooks/04_bq2_early_signals.ipynb
+++ b/notebooks/04_bq2_early_signals.ipynb
@@ -82,7 +82,8 @@
"- All database queries go through `src.db.connection.execute_query()`, the project's DB abstraction layer (ADR-003).\n",
"- BQ2's primary SQL query lives in `sql/queries/q_bq2_early_signals.sql` and is loaded at runtime from disk.\n",
"- Statistical tests use `src.stats.tests`, project wrappers around scipy that standardize output format (test statistic, p-value, effect size, confidence interval).\n",
- "- Figures are saved to `reports/figures/` at 150 DPI."
+ "- Figures are saved to `reports/figures/` at 150 DPI.\n",
+ "- **Figure localization:** figures embedded by the README and the executive report are rendered twice with identical filenames: English to `reports/figures/` (linked by the EN documents) and Italian to `reports/figures/it/` (linked by the IT documents). Notebook-only figures stay English, like the notebooks themselves (ADR-005)."
]
},
{
@@ -275,9 +276,89 @@
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": null,
"id": "7",
"metadata": {},
+ "outputs": [],
+ "source": [
+ "# --- Figure localization (EN canonical + IT) ---\n",
+ "# README and executive report ship in two languages. Figures EMBEDDED by\n",
+ "# those documents are rendered twice with the same filename: English to\n",
+ "# reports/figures/ (linked by the EN docs), Italian to reports/figures/it/\n",
+ "# (linked by the IT docs). Notebook-only figures keep the plain save_fig()\n",
+ "# path and stay English, like the notebooks themselves (ADR-005).\n",
+ "# Only chart furniture (titles, axis labels, legends) is localized;\n",
+ "# data-derived values keep their source form.\n",
+ "LANGS = ('en', 'it')\n",
+ "FIG_DIRS = {'en': FIGURES_DIR, 'it': FIGURES_DIR / 'it'}\n",
+ "for _dir in FIG_DIRS.values():\n",
+ " _dir.mkdir(parents=True, exist_ok=True)\n",
+ "\n",
+ "\n",
+ "def save_fig_lang(fig, name: str, lang: str) -> None:\n",
+ " \"\"\"Save a localized figure; display only the EN (canonical) render.\"\"\"\n",
+ " path = FIG_DIRS[lang] / f'{name}.png'\n",
+ " fig.savefig(path, dpi=FIG_DPI, bbox_inches='tight', facecolor='white')\n",
+ " print(f' Saved: {path}')\n",
+ " if lang == 'en':\n",
+ " plt.show()\n",
+ " else:\n",
+ " plt.close(fig)\n",
+ "\n",
+ "\n",
+ "# Italian display names for the 8 signals (same keys as SIGNAL_LABELS)\n",
+ "SIGNAL_LABELS_IT = {\n",
+ " 'active_days_first_28': 'Giorni attivi (primi 28)',\n",
+ " 'total_clicks_first_28': 'Click totali (primi 28)',\n",
+ " 'avg_clicks_per_active_day': 'Click medi per giorno attivo',\n",
+ " 'last_active_day_in_window': 'Ultimo giorno attivo nella finestra',\n",
+ " 'engagement_decile_in_course': 'Decile di engagement',\n",
+ " 'first_score': 'Punteggio prima valutazione',\n",
+ " 'first_submit_day': 'Giorno prima consegna',\n",
+ " 'date_registration': 'Giorno di registrazione',\n",
+ "}\n",
+ "\n",
+ "# EN entries reuse the shared label constants (single source of truth);\n",
+ "# IT entries translate the same chart furniture.\n",
+ "L10N = {\n",
+ " 'en': {\n",
+ " 'signal_labels': SIGNAL_LABELS,\n",
+ " 'effect_size': LABEL_EFFECT_SIZE,\n",
+ " 'completion_rate': LABEL_COMPLETION_RATE,\n",
+ " 'forest_title': 'BQ2: Early Signal Ranking by Effect Size\\n'\n",
+ " '(green = significant after BH correction, gray = not significant)',\n",
+ " 'small': 'Small',\n",
+ " 'medium': 'Medium',\n",
+ " 'large': 'Large',\n",
+ " 'dose_word': 'Dose-Response',\n",
+ " 'dose_suptitle': 'Dose-Response: Completion Rate by Signal Quartile (top 3 signals)',\n",
+ " 'ghost_segments': ['Ghost\\n(zero activity)', 'Active\\n(>= 1 active day)'],\n",
+ " 'ghost_title': 'Completion Rate: Ghost vs. Active Students\\n(with 95% Bootstrap CI)',\n",
+ " },\n",
+ " 'it': {\n",
+ " 'signal_labels': SIGNAL_LABELS_IT,\n",
+ " 'effect_size': 'd di Cohen',\n",
+ " 'completion_rate': 'Tasso di completamento (%)',\n",
+ " 'forest_title': 'BQ2: ranking dei segnali precoci per effect size\\n'\n",
+ " '(verde = significativo dopo correzione BH, grigio = non significativo)',\n",
+ " 'small': 'Piccolo',\n",
+ " 'medium': 'Medio',\n",
+ " 'large': 'Grande',\n",
+ " 'dose_word': 'Dose-risposta',\n",
+ " 'dose_suptitle': 'Dose-risposta: tasso di completamento per quartile del segnale (top 3 segnali)',\n",
+ " 'ghost_segments': ['Ghost\\n(zero attività)', 'Attivi\\n(>= 1 giorno attivo)'],\n",
+ " 'ghost_title': 'Tasso di completamento: studenti ghost vs attivi\\n'\n",
+ " '(con intervalli di confidenza bootstrap al 95%)',\n",
+ " },\n",
+ "}\n",
+ "print(f'Figure localization ready: {\", \".join(LANGS)}')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8",
+ "metadata": {},
"source": [
"## 3. Descriptive Comparison\n",
"\n",
@@ -289,7 +370,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "8",
+ "id": "9",
"metadata": {},
"outputs": [],
"source": [
@@ -325,7 +406,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "9",
+ "id": "10",
"metadata": {},
"outputs": [],
"source": [
@@ -367,7 +448,7 @@
},
{
"cell_type": "markdown",
- "id": "10",
+ "id": "11",
"metadata": {},
"source": [
"> **Visual impression:** All six engagement signals show visible separation between completers and non-completers. The distributions overlap substantially (this is expected with behavioral data), but the central tendencies (means, medians) differ consistently.\n",
@@ -377,7 +458,7 @@
},
{
"cell_type": "markdown",
- "id": "11",
+ "id": "12",
"metadata": {},
"source": [
"## 4. Statistical Testing: T-Tests\n",
@@ -397,7 +478,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "12",
+ "id": "13",
"metadata": {},
"outputs": [],
"source": [
@@ -442,7 +523,7 @@
},
{
"cell_type": "markdown",
- "id": "13",
+ "id": "14",
"metadata": {},
"source": [
"## 5. Multiple Comparison Correction\n",
@@ -458,7 +539,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "14",
+ "id": "15",
"metadata": {},
"outputs": [],
"source": [
@@ -489,7 +570,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "15",
+ "id": "16",
"metadata": {},
"outputs": [],
"source": [
@@ -525,7 +606,7 @@
},
{
"cell_type": "markdown",
- "id": "16",
+ "id": "17",
"metadata": {},
"source": [
"> **Interpretation:** With a large dataset (~32K enrollments), all 8 signals remain significant even after the conservative Bonferroni correction (8/8 under both Bonferroni and BH). This confirms that the differences are real, but statistical significance alone does not tell us which signals are *practically meaningful*. That is what the effect size ranking (next section) addresses."
@@ -533,7 +614,7 @@
},
{
"cell_type": "markdown",
- "id": "17",
+ "id": "18",
"metadata": {},
"source": [
"## 6. Effect Size Ranking: Forest Plot\n",
@@ -552,7 +633,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "18",
+ "id": "19",
"metadata": {},
"outputs": [],
"source": [
@@ -560,51 +641,56 @@
"# Sort by absolute effect size for consistent visual ranking.\n",
"df_forest = df_results.sort_values('abs_cohens_d', ascending=True).reset_index(drop=True)\n",
"\n",
- "fig, ax = plt.subplots(figsize=(10, 7))\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=(10, 7))\n",
"\n",
- "y_pos = np.arange(len(df_forest))\n",
+ " y_pos = np.arange(len(df_forest))\n",
"\n",
- "# Color based on significance after BH correction\n",
- "colors = ['#55A868' if sig else '#999999' for sig in df_forest['sig_bh']]\n",
+ " # Color based on significance after BH correction\n",
+ " colors = ['#55A868' if sig else '#999999' for sig in df_forest['sig_bh']]\n",
"\n",
- "# Plot effect sizes as points only.\n",
- "# Do not draw CI whiskers here: the available test CI is for the mean\n",
- "# difference, not for Cohen's d, and arbitrary whiskers would be misleading.\n",
- "ax.scatter(df_forest['cohens_d'], y_pos, color=colors, s=80, zorder=3, edgecolor='white')\n",
+ " # Plot effect sizes as points only.\n",
+ " # Do not draw CI whiskers here: the available test CI is for the mean\n",
+ " # difference, not for Cohen's d, and arbitrary whiskers would be misleading.\n",
+ " ax.scatter(df_forest['cohens_d'], y_pos, color=colors, s=80, zorder=3, edgecolor='white')\n",
"\n",
- "# Reference lines for effect size interpretation\n",
- "for d_ref, label in [(0.2, 'Small'), (0.5, 'Medium'), (0.8, 'Large')]:\n",
- " ax.axvline(x=d_ref, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
- " ax.axvline(x=-d_ref, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
- " ax.text(d_ref, len(df_forest) - 0.3, label, ha='center', fontsize=8, color='gray')\n",
+ " # Reference lines for effect size interpretation.\n",
+ " # Threshold labels are anchored to the axes top via a blended transform\n",
+ " # (x in data coordinates, y in axes fraction): a fixed data-coordinate y\n",
+ " # would land beyond the axis limit and collide with the two-line title.\n",
+ " for d_ref, ref_label in [(0.2, L['small']), (0.5, L['medium']), (0.8, L['large'])]:\n",
+ " ax.axvline(x=d_ref, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
+ " ax.axvline(x=-d_ref, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
+ " ax.text(d_ref, 0.99, ref_label, transform=ax.get_xaxis_transform(),\n",
+ " ha='center', va='top', fontsize=8, color='gray')\n",
"\n",
- "# Zero line: no effect\n",
- "ax.axvline(x=0, color='black', linewidth=0.8)\n",
+ " # Zero line: no effect\n",
+ " ax.axvline(x=0, color='black', linewidth=0.8)\n",
"\n",
- "# Annotate each point with the exact d value\n",
- "for i, (_, row) in enumerate(df_forest.iterrows()):\n",
- " ax.text(\n",
- " row['cohens_d'] + 0.03 if row['cohens_d'] >= 0 else row['cohens_d'] - 0.03,\n",
- " i + 0.15,\n",
- " f\"d = {row['cohens_d']:.3f}\",\n",
- " ha='left' if row['cohens_d'] >= 0 else 'right',\n",
- " fontsize=8, color='#333333',\n",
- " )\n",
+ " # Annotate each point with the exact d value\n",
+ " for i, (_, row) in enumerate(df_forest.iterrows()):\n",
+ " ax.text(\n",
+ " row['cohens_d'] + 0.03 if row['cohens_d'] >= 0 else row['cohens_d'] - 0.03,\n",
+ " i + 0.15,\n",
+ " f\"d = {row['cohens_d']:.3f}\",\n",
+ " ha='left' if row['cohens_d'] >= 0 else 'right',\n",
+ " fontsize=8, color='#333333',\n",
+ " )\n",
"\n",
- "ax.set_yticks(y_pos)\n",
- "ax.set_yticklabels(df_forest['signal_label'])\n",
- "ax.set_xlabel(LABEL_EFFECT_SIZE)\n",
- "ax.set_title('BQ2: Early Signal Ranking by Effect Size\\n'\n",
- " '(green = significant after BH correction, gray = not significant)')\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '04_forest_plot_effect_sizes')\n",
- "plt.show()"
+ " ax.set_yticks(y_pos)\n",
+ " ax.set_yticklabels([L['signal_labels'][s] for s in df_forest['signal']])\n",
+ " ax.set_xlabel(L['effect_size'])\n",
+ " ax.set_title(L['forest_title'])\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '04_forest_plot_effect_sizes', lang)"
]
},
{
"cell_type": "markdown",
- "id": "19",
+ "id": "20",
"metadata": {},
"source": [
"> **Key finding:** The forest plot reveals a clear ranking of early behavioral signals. Engagement-volume metrics lead: within-course engagement decile (d = 0.97), active days (d = 0.90), and total clicks (d = 0.63); the remaining signals (last active day, click intensity, first score and timing) fall between |d| = 0.11 and |d| = 0.55 and provide complementary information.\n",
@@ -614,7 +700,7 @@
},
{
"cell_type": "markdown",
- "id": "20",
+ "id": "21",
"metadata": {},
"source": [
"## 7. Signal Deep-Dives\n",
@@ -627,7 +713,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "21",
+ "id": "22",
"metadata": {},
"outputs": [],
"source": [
@@ -635,62 +721,63 @@
"# Select the 3 signals with the largest absolute Cohen's d.\n",
"top_signals = df_results.head(3)['signal'].tolist()\n",
"\n",
- "fig, axes = plt.subplots(1, 3, figsize=(18, 5))\n",
- "\n",
- "for i, col in enumerate(top_signals):\n",
- " ax = axes[i]\n",
- " label = SIGNAL_LABELS[col]\n",
- "\n",
- " # Create quartile bins for this signal\n",
- " # Use pd.qcut; for signals with many identical values (like decile),\n",
- " # duplicates='drop' prevents errors.\n",
- " valid = df_signals[[col, 'completed']].dropna().copy()\n",
- " try:\n",
- " valid['bin'] = pd.qcut(valid[col], q=4, duplicates='drop')\n",
- " except ValueError:\n",
- " # If qcut fails (too few unique values), use raw values\n",
- " valid['bin'] = valid[col]\n",
- "\n",
- " dose_response = (\n",
- " valid.groupby('bin', observed=True)\n",
- " .agg(n=('completed', 'count'), rate=('completed', 'mean'))\n",
- " .reset_index()\n",
- " )\n",
- " dose_response['rate_pct'] = (dose_response['rate'] * 100).round(1)\n",
- "\n",
- " # Bar chart with completion rate per quartile\n",
- " x_labels = [str(b) for b in dose_response['bin']]\n",
- " x_pos = range(len(dose_response))\n",
- " # Gradient from red (low completion) to green (high completion)\n",
- " n_bins = len(dose_response)\n",
- " gradient = [plt.cm.RdYlGn(j / max(n_bins - 1, 1)) for j in range(n_bins)]\n",
- "\n",
- " bars = ax.bar(x_pos, dose_response['rate_pct'], color=gradient, edgecolor='white')\n",
- " for bar, (_, row) in zip(bars, dose_response.iterrows()):\n",
- " ax.text(\n",
- " bar.get_x() + bar.get_width() / 2, bar.get_height() + 1,\n",
- " f\"{row['rate_pct']:.1f}%\\n(n={int(row['n']):,})\",\n",
- " ha='center', fontsize=8, color='#333333',\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, axes = plt.subplots(1, 3, figsize=(18, 5))\n",
+ "\n",
+ " for i, col in enumerate(top_signals):\n",
+ " ax = axes[i]\n",
+ " label = L['signal_labels'][col]\n",
+ "\n",
+ " # Create quartile bins for this signal\n",
+ " # Use pd.qcut; for signals with many identical values (like decile),\n",
+ " # duplicates='drop' prevents errors.\n",
+ " valid = df_signals[[col, 'completed']].dropna().copy()\n",
+ " try:\n",
+ " valid['bin'] = pd.qcut(valid[col], q=4, duplicates='drop')\n",
+ " except ValueError:\n",
+ " # If qcut fails (too few unique values), use raw values\n",
+ " valid['bin'] = valid[col]\n",
+ "\n",
+ " dose_response = (\n",
+ " valid.groupby('bin', observed=True)\n",
+ " .agg(n=('completed', 'count'), rate=('completed', 'mean'))\n",
+ " .reset_index()\n",
" )\n",
- "\n",
- " ax.set_xticks(x_pos)\n",
- " ax.set_xticklabels(x_labels, rotation=30, ha='right', fontsize=8)\n",
- " ax.set_xlabel(label)\n",
- " ax.set_ylabel(LABEL_COMPLETION_RATE)\n",
- " ax.set_title(f'Dose-Response: {label}')\n",
- " ax.set_ylim(0, 100)\n",
- " sns.despine(ax=ax)\n",
- "\n",
- "fig.suptitle('Dose-Response: Completion Rate by Signal Quartile (top 3 signals)',\n",
- " fontsize=14, y=1.02)\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '04_top_signal_dose_response')\n",
- "plt.show()"
+ " dose_response['rate_pct'] = (dose_response['rate'] * 100).round(1)\n",
+ "\n",
+ " # Bar chart with completion rate per quartile\n",
+ " x_labels = [str(b) for b in dose_response['bin']]\n",
+ " x_pos = range(len(dose_response))\n",
+ " # Gradient from red (low completion) to green (high completion)\n",
+ " n_bins = len(dose_response)\n",
+ " gradient = [plt.cm.RdYlGn(j / max(n_bins - 1, 1)) for j in range(n_bins)]\n",
+ "\n",
+ " bars = ax.bar(x_pos, dose_response['rate_pct'], color=gradient, edgecolor='white')\n",
+ " for bar, (_, row) in zip(bars, dose_response.iterrows()):\n",
+ " ax.text(\n",
+ " bar.get_x() + bar.get_width() / 2, bar.get_height() + 1,\n",
+ " f\"{row['rate_pct']:.1f}%\\n(n={int(row['n']):,})\",\n",
+ " ha='center', fontsize=8, color='#333333',\n",
+ " )\n",
+ "\n",
+ " ax.set_xticks(x_pos)\n",
+ " ax.set_xticklabels(x_labels, rotation=30, ha='right', fontsize=8)\n",
+ " ax.set_xlabel(label)\n",
+ " ax.set_ylabel(L['completion_rate'])\n",
+ " ax.set_title(f\"{L['dose_word']}: {label}\")\n",
+ " ax.set_ylim(0, 100)\n",
+ " sns.despine(ax=ax)\n",
+ "\n",
+ " fig.suptitle(L['dose_suptitle'], fontsize=14, y=1.02)\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '04_top_signal_dose_response', lang)"
]
},
{
"cell_type": "markdown",
- "id": "22",
+ "id": "23",
"metadata": {},
"source": [
"> **Interpretation:** The dose-response plots confirm that the relationship between top signals and completion is **monotonic** (or near-monotonic): higher engagement consistently predicts higher completion rates. This is important: it means the signal is useful across its entire range, not just at extremes.\n",
@@ -700,7 +787,7 @@
},
{
"cell_type": "markdown",
- "id": "23",
+ "id": "24",
"metadata": {},
"source": [
"## 8. Assessment-Based Signals\n",
@@ -713,7 +800,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "24",
+ "id": "25",
"metadata": {},
"outputs": [],
"source": [
@@ -770,7 +857,7 @@
},
{
"cell_type": "markdown",
- "id": "25",
+ "id": "26",
"metadata": {},
"source": [
"> **Interpretation:**\n",
@@ -783,7 +870,7 @@
},
{
"cell_type": "markdown",
- "id": "26",
+ "id": "27",
"metadata": {},
"source": [
"## 9. Ghost Student Signal\n",
@@ -796,7 +883,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "27",
+ "id": "28",
"metadata": {},
"outputs": [],
"source": [
@@ -823,54 +910,55 @@
"print(f'\\n Gap: {active_rate - ghost_rate:.1f} percentage points')\n",
"\n",
"# --- Bar chart with bootstrap CI whiskers ---\n",
- "fig, ax = plt.subplots(figsize=(8, 5))\n",
- "\n",
- "segments = ['Ghost\\n(zero activity)', 'Active\\n(>= 1 active day)']\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
"rates = [ghost_rate, active_rate]\n",
"ci_lower = [ghost_ci[0] * 100, active_ci[0] * 100]\n",
"ci_upper = [ghost_ci[1] * 100, active_ci[1] * 100]\n",
"colors_bar = ['#C44E52', '#55A868']\n",
"\n",
- "bars = ax.bar(segments, rates, color=colors_bar, edgecolor='white', width=0.5)\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=(8, 5))\n",
"\n",
- "# Add CI whiskers\n",
- "for i, bar in enumerate(bars):\n",
- " ax.plot(\n",
- " [bar.get_x() + bar.get_width() / 2] * 2,\n",
- " [ci_lower[i], ci_upper[i]],\n",
- " color='black', linewidth=2,\n",
- " )\n",
- " # Caps\n",
- " cap_width = 0.08\n",
- " for y_cap in [ci_lower[i], ci_upper[i]]:\n",
+ " bars = ax.bar(L['ghost_segments'], rates, color=colors_bar, edgecolor='white', width=0.5)\n",
+ "\n",
+ " # Add CI whiskers\n",
+ " for i, bar in enumerate(bars):\n",
" ax.plot(\n",
- " [bar.get_x() + bar.get_width() / 2 - cap_width,\n",
- " bar.get_x() + bar.get_width() / 2 + cap_width],\n",
- " [y_cap, y_cap],\n",
+ " [bar.get_x() + bar.get_width() / 2] * 2,\n",
+ " [ci_lower[i], ci_upper[i]],\n",
" color='black', linewidth=2,\n",
" )\n",
+ " # Caps\n",
+ " cap_width = 0.08\n",
+ " for y_cap in [ci_lower[i], ci_upper[i]]:\n",
+ " ax.plot(\n",
+ " [bar.get_x() + bar.get_width() / 2 - cap_width,\n",
+ " bar.get_x() + bar.get_width() / 2 + cap_width],\n",
+ " [y_cap, y_cap],\n",
+ " color='black', linewidth=2,\n",
+ " )\n",
+ "\n",
+ " # Annotate with rate + CI\n",
+ " for i, bar in enumerate(bars):\n",
+ " ax.text(\n",
+ " bar.get_x() + bar.get_width() / 2,\n",
+ " ci_upper[i] + 2,\n",
+ " f'{rates[i]:.1f}%\\n[{ci_lower[i]:.1f}–{ci_upper[i]:.1f}%]',\n",
+ " ha='center', fontsize=10, color='#333333',\n",
+ " )\n",
"\n",
- "# Annotate with rate + CI\n",
- "for i, bar in enumerate(bars):\n",
- " ax.text(\n",
- " bar.get_x() + bar.get_width() / 2,\n",
- " ci_upper[i] + 2,\n",
- " f'{rates[i]:.1f}%\\n[{ci_lower[i]:.1f}–{ci_upper[i]:.1f}%]',\n",
- " ha='center', fontsize=10, color='#333333',\n",
- " )\n",
- "\n",
- "ax.set_ylabel(LABEL_COMPLETION_RATE)\n",
- "ax.set_title('Completion Rate: Ghost vs. Active Students\\n(with 95% Bootstrap CI)')\n",
- "ax.set_ylim(0, max(ci_upper) + 15)\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '04_ghost_vs_active_completion')\n",
- "plt.show()"
+ " ax.set_ylabel(L['completion_rate'])\n",
+ " ax.set_title(L['ghost_title'])\n",
+ " ax.set_ylim(0, max(ci_upper) + 15)\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '04_ghost_vs_active_completion', lang)"
]
},
{
"cell_type": "markdown",
- "id": "28",
+ "id": "29",
"metadata": {},
"source": [
"> **Key finding:** The gap between ghost and active students is enormous and the 95% bootstrap confidence intervals do not overlap: 4.8% completion for ghosts (CI 4.2-5.4%) versus 54.3% for active students (CI 53.7-54.9%). Ghost students, those with zero VLE activity in the first 28 days, have a near-zero completion rate.\n",
@@ -884,7 +972,7 @@
},
{
"cell_type": "markdown",
- "id": "29",
+ "id": "30",
"metadata": {},
"source": [
"## 10. Key Takeaways and Next Steps\n",
@@ -922,7 +1010,7 @@
},
{
"cell_type": "markdown",
- "id": "30",
+ "id": "31",
"metadata": {},
"source": [
"> **From signals to comparison:** This notebook identified *which* early behaviors predict dropout and ranked them by effect size. The next question is: do these behavioral signals outperform demographic factors? Or does a student's background matter more than what they actually do on the platform?\n",
@@ -938,7 +1026,15 @@
"name": "python3"
},
"language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
"name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
"version": "3.13.0"
}
},
diff --git a/notebooks/05_bq3_demographics_vs_behavior.ipynb b/notebooks/05_bq3_demographics_vs_behavior.ipynb
index 4ecc11c..0081414 100644
--- a/notebooks/05_bq3_demographics_vs_behavior.ipynb
+++ b/notebooks/05_bq3_demographics_vs_behavior.ipynb
@@ -79,7 +79,8 @@
"- All database queries go through `src.db.connection.execute_query()`, the project's DB abstraction layer (ADR-003).\n",
"- BQ3's primary SQL query lives in `sql/queries/q_bq3_demographics_vs_behavior.sql` and is loaded at runtime from disk.\n",
"- Statistical tests use `src.stats.tests`, project wrappers around scipy. This notebook introduces `chi_square_test()` (Cramér's V) alongside the `independent_t_test()` (Cohen's d) used in NB04.\n",
- "- Figures are saved to `reports/figures/` at 150 DPI."
+ "- Figures are saved to `reports/figures/` at 150 DPI.\n",
+ "- **Figure localization:** figures embedded by the README and the executive report are rendered twice with identical filenames: English to `reports/figures/` (linked by the EN documents) and Italian to `reports/figures/it/` (linked by the IT documents). Notebook-only figures stay English, like the notebooks themselves (ADR-005)."
]
},
{
@@ -297,9 +298,102 @@
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": null,
"id": "7",
"metadata": {},
+ "outputs": [],
+ "source": [
+ "# --- Figure localization (EN canonical + IT) ---\n",
+ "# README and executive report ship in two languages. Figures EMBEDDED by\n",
+ "# those documents are rendered twice with the same filename: English to\n",
+ "# reports/figures/ (linked by the EN docs), Italian to reports/figures/it/\n",
+ "# (linked by the IT docs). Notebook-only figures keep the plain save_fig()\n",
+ "# path and stay English, like the notebooks themselves (ADR-005).\n",
+ "# Only chart furniture (titles, axis labels, legends) is localized;\n",
+ "# data-derived values (education categories, counts) keep their source form.\n",
+ "LANGS = ('en', 'it')\n",
+ "FIG_DIRS = {'en': FIGURES_DIR, 'it': FIGURES_DIR / 'it'}\n",
+ "for _dir in FIG_DIRS.values():\n",
+ " _dir.mkdir(parents=True, exist_ok=True)\n",
+ "\n",
+ "\n",
+ "def save_fig_lang(fig, name: str, lang: str) -> None:\n",
+ " \"\"\"Save a localized figure; display only the EN (canonical) render.\"\"\"\n",
+ " path = FIG_DIRS[lang] / f'{name}.png'\n",
+ " fig.savefig(path, dpi=FIG_DPI, bbox_inches='tight', facecolor='white')\n",
+ " print(f' Saved: {path}')\n",
+ " if lang == 'en':\n",
+ " plt.show()\n",
+ " else:\n",
+ " plt.close(fig)\n",
+ "\n",
+ "\n",
+ "# Italian display names for all tested features (same keys as ALL_LABELS)\n",
+ "ALL_FEATURE_LABELS_IT = {\n",
+ " 'gender': 'Genere',\n",
+ " 'age_band': 'Fascia di età',\n",
+ " 'highest_education': 'Livello di istruzione',\n",
+ " 'imd_band': 'Fascia IMD',\n",
+ " 'disability': 'Disabilità',\n",
+ " 'region': 'Regione',\n",
+ " 'num_of_prev_attempts': 'Tentativi precedenti',\n",
+ " 'studied_credits': 'Crediti studiati',\n",
+ " 'active_days_first_28': 'Giorni attivi (primi 28)',\n",
+ " 'total_clicks_first_28': 'Click totali (primi 28)',\n",
+ " 'avg_clicks_per_active_day': 'Click medi per giorno attivo',\n",
+ " 'engagement_decile_in_course': 'Decile di engagement (solo attivi)',\n",
+ " 'submitted_first_assessment': 'Prima valutazione consegnata',\n",
+ " 'first_score': 'Primo punteggio (solo submitter)',\n",
+ "}\n",
+ "\n",
+ "# EN entries reuse the shared label constants (single source of truth);\n",
+ "# IT entries translate the same chart furniture.\n",
+ "L10N = {\n",
+ " 'en': {\n",
+ " 'all_labels': ALL_LABELS,\n",
+ " 'effect_size': LABEL_EFFECT_SIZE,\n",
+ " 'cramers_v': LABEL_CRAMERS_V,\n",
+ " 'completion_rate': LABEL_COMPLETION_RATE,\n",
+ " 'small': 'Small',\n",
+ " 'medium': 'Medium',\n",
+ " 'cont_title': \"Continuous Features: |Cohen's d|\\n(blue = demographic, green = behavioral)\",\n",
+ " 'cat_title': \"Categorical Demographics: Cramér's V\\n(blue = significant)\",\n",
+ " 'suptitle': 'BQ3: Demographics vs Behavior - Effect Size Comparison\\n'\n",
+ " '(all-enrollment features only; conditional features reported below)',\n",
+ " 'edu_xlabel': 'Highest Education Level',\n",
+ " 'edu_title': 'Completion Rate: Education Level × Engagement\\n'\n",
+ " '(split at median = {median:.0f} active days in first 28 days)',\n",
+ " 'engagement': 'Engagement',\n",
+ " 'high_eng': 'High engagement',\n",
+ " 'low_eng': 'Low engagement',\n",
+ " },\n",
+ " 'it': {\n",
+ " 'all_labels': ALL_FEATURE_LABELS_IT,\n",
+ " 'effect_size': 'd di Cohen',\n",
+ " 'cramers_v': 'V di Cramér',\n",
+ " 'completion_rate': 'Tasso di completamento (%)',\n",
+ " 'small': 'Piccolo',\n",
+ " 'medium': 'Medio',\n",
+ " 'cont_title': 'Variabili continue: |d di Cohen|\\n(blu = demografiche, verde = comportamentali)',\n",
+ " 'cat_title': 'Demografiche categoriche: V di Cramér\\n(blu = significative)',\n",
+ " 'suptitle': 'BQ3: demografia vs comportamento - confronto degli effect size\\n'\n",
+ " '(solo variabili su tutte le iscrizioni; variabili condizionali riportate a parte)',\n",
+ " 'edu_xlabel': 'Livello di istruzione più alto',\n",
+ " 'edu_title': 'Tasso di completamento: livello di istruzione × engagement\\n'\n",
+ " '(divisione alla mediana = {median:.0f} giorni attivi nei primi 28)',\n",
+ " 'engagement': 'Engagement',\n",
+ " 'high_eng': 'Engagement alto',\n",
+ " 'low_eng': 'Engagement basso',\n",
+ " },\n",
+ "}\n",
+ "print(f'Figure localization ready: {\", \".join(LANGS)}')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8",
+ "metadata": {},
"source": [
"## 3. Part A: Demographic Associations\n",
"\n",
@@ -315,7 +409,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "8",
+ "id": "9",
"metadata": {},
"outputs": [],
"source": [
@@ -396,7 +490,7 @@
},
{
"cell_type": "markdown",
- "id": "9",
+ "id": "10",
"metadata": {},
"source": [
"> **Interpretation:** Most demographic features show statistically significant associations with completion, but significance alone is not the point. With ~32K enrollments, even trivial differences reach significance. The critical question is **effect size**: are these associations *meaningful*?\n",
@@ -407,7 +501,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "10",
+ "id": "11",
"metadata": {},
"outputs": [],
"source": [
@@ -471,7 +565,7 @@
},
{
"cell_type": "markdown",
- "id": "11",
+ "id": "12",
"metadata": {},
"source": [
"> **Visual impression:** While completion rates vary across demographic categories (especially for education level and IMD band), the differences are modest. No single demographic group has a near-zero or near-100% completion rate. Demographics shift the probability slightly, but they do not determine the outcome.\n",
@@ -482,7 +576,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "12",
+ "id": "13",
"metadata": {},
"outputs": [],
"source": [
@@ -513,12 +607,15 @@
"# Reference lines for both metrics:\n",
"# - Cramér's V small threshold = 0.1\n",
"# - Cohen's d small threshold = 0.2\n",
+ "# Threshold labels are anchored to the axes top via a blended transform\n",
+ "# (x in data coordinates, y in axes fraction): a fixed data-coordinate y\n",
+ "# would land beyond the axis limit and collide with the title.\n",
"ax.axvline(x=0.1, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
- "ax.text(0.1, len(df_demo_effects) - 0.3, 'Small (V)',\n",
- " ha='center', fontsize=8, color='gray')\n",
+ "ax.text(0.1, 0.99, 'Small (V)', transform=ax.get_xaxis_transform(),\n",
+ " ha='center', va='top', fontsize=8, color='gray')\n",
"ax.axvline(x=0.2, color='gray', linestyle='--', linewidth=0.8, alpha=0.6)\n",
- "ax.text(0.2, len(df_demo_effects) - 0.3, 'Small (d)',\n",
- " ha='center', fontsize=8, color='gray')\n",
+ "ax.text(0.2, 0.99, 'Small (d)', transform=ax.get_xaxis_transform(),\n",
+ " ha='center', va='top', fontsize=8, color='gray')\n",
"\n",
"ax.set_yticks(y_pos)\n",
"ax.set_yticklabels(df_demo_effects['label'])\n",
@@ -533,7 +630,7 @@
},
{
"cell_type": "markdown",
- "id": "13",
+ "id": "14",
"metadata": {},
"source": [
"> **Part A Summary:** All 8 demographic features show statistically significant associations with completion after multiple comparison correction. However, the effect sizes remain **small to modest**: Cramér's V peaks at about 0.15 (highest education 0.150, IMD band 0.134, all others below 0.09) and |Cohen's d| for the numeric demographics stays below 0.3 (studied credits 0.28, previous attempts 0.21). Demographics tell us *who is slightly more likely* to complete, but they lack the discriminative power to identify at-risk students with confidence."
@@ -541,7 +638,7 @@
},
{
"cell_type": "markdown",
- "id": "14",
+ "id": "15",
"metadata": {},
"source": [
"## 4. Part B: Behavioral Associations\n",
@@ -556,7 +653,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "15",
+ "id": "16",
"metadata": {},
"outputs": [],
"source": [
@@ -611,7 +708,7 @@
},
{
"cell_type": "markdown",
- "id": "16",
+ "id": "17",
"metadata": {},
"source": [
"> **Interpretation:** All 6 behavioral features show statistically significant associations with completion. More importantly, the **effect sizes are substantially larger** than the demographic ones. Every behavioral feature reaches at least a medium effect size (|d| between 0.52 and 0.90), compared to the small-to-modest demographic effects (|d| up to 0.28, V up to 0.15).\n",
@@ -628,7 +725,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "17",
+ "id": "18",
"metadata": {},
"outputs": [],
"source": [
@@ -650,11 +747,14 @@
" va='center', fontsize=9, color='#333333',\n",
" )\n",
"\n",
- "# Reference lines for Cohen's d thresholds\n",
+ "# Reference lines for Cohen's d thresholds.\n",
+ "# Threshold labels are anchored to the axes top via a blended transform\n",
+ "# (x in data coordinates, y in axes fraction): a fixed data-coordinate y\n",
+ "# would land beyond the axis limit and collide with the title.\n",
"for d_ref, ref_label in [(0.2, 'Small'), (0.5, 'Medium'), (0.8, 'Large')]:\n",
" ax.axvline(x=d_ref, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
- " ax.text(d_ref, len(df_behav_plot) - 0.3, ref_label,\n",
- " ha='center', fontsize=8, color='gray')\n",
+ " ax.text(d_ref, 0.99, ref_label, transform=ax.get_xaxis_transform(),\n",
+ " ha='center', va='top', fontsize=8, color='gray')\n",
"\n",
"ax.set_yticks(y_pos)\n",
"ax.set_yticklabels(df_behav_plot['label'])\n",
@@ -669,7 +769,7 @@
},
{
"cell_type": "markdown",
- "id": "18",
+ "id": "19",
"metadata": {},
"source": [
"> **Part B Summary:** Behavioral features show medium-to-large effect sizes (|d| ≈ 0.5–0.9), with the strongest signals coming from engagement volume metrics and first assessment submission. These effects are 2 to 4 times larger than the demographic effects in Part A (2.7× on average across comparable metrics)."
@@ -677,7 +777,7 @@
},
{
"cell_type": "markdown",
- "id": "19",
+ "id": "20",
"metadata": {},
"source": [
"## 5. Part C: The Verdict on Demographics vs Behavior\n",
@@ -694,7 +794,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "20",
+ "id": "21",
"metadata": {},
"outputs": [],
"source": [
@@ -705,85 +805,86 @@
"# Conditional features (engagement_decile, first_score) are excluded from the\n",
"# left panel because their effect sizes are measured on different populations.\n",
"# They are reported separately in the quantitative summary below.\n",
- "fig, (ax1, ax2) = plt.subplots(\n",
- " 1, 2, figsize=(16, 7),\n",
- " gridspec_kw={'width_ratios': [3, 2]},\n",
- ")\n",
- "\n",
- "# --- Left panel: Cohen's d for continuous features ---\n",
- "# Combine numeric demographics and behavioral features on the same scale.\n",
- "# Color by feature type only to avoid mixing significance flags derived from\n",
- "# different multiple-testing correction families in one shared visual encoding.\n",
- "# Exclude conditional features (different population base).\n",
"df_behav_comparable = df_behav[~df_behav['feature'].isin(CONDITIONAL_FEATURES)]\n",
"\n",
+ "# Keep the 'feature' key alongside the label so each language can render\n",
+ "# its own display names from the same sorted frame.\n",
"df_d_compare = pd.concat([\n",
- " df_demo_ttest[['label', 'effect_size']].assign(\n",
+ " df_demo_ttest[['feature', 'effect_size']].assign(\n",
" feature_type='Demographic'\n",
" ),\n",
- " df_behav_comparable[['label', 'abs_cohens_d']].rename(\n",
+ " df_behav_comparable[['feature', 'abs_cohens_d']].rename(\n",
" columns={'abs_cohens_d': 'effect_size'}\n",
" ).assign(feature_type='Behavioral'),\n",
"]).sort_values('effect_size', ascending=True).reset_index(drop=True)\n",
"\n",
- "y_pos_left = np.arange(len(df_d_compare))\n",
- "palette_type = {'Demographic': '#4C72B0', 'Behavioral': '#55A868'}\n",
- "colors_left = [palette_type[t] for t in df_d_compare['feature_type']]\n",
- "\n",
- "ax1.barh(y_pos_left, df_d_compare['effect_size'], color=colors_left, edgecolor='white')\n",
+ "df_chi_plot = df_demo_chi.sort_values('effect_size', ascending=True).reset_index(drop=True)\n",
"\n",
- "for i, (_, row) in enumerate(df_d_compare.iterrows()):\n",
- " ax1.text(\n",
- " row['effect_size'] + 0.01, i,\n",
- " f\"|d| = {row['effect_size']:.3f}\",\n",
- " va='center', fontsize=9, color='#333333',\n",
+ "# Rendered per language: embedded by both READMEs and both reports.\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, (ax1, ax2) = plt.subplots(\n",
+ " 1, 2, figsize=(16, 7),\n",
+ " gridspec_kw={'width_ratios': [3, 2]},\n",
" )\n",
"\n",
- "# Reference lines for Cohen's d\n",
- "for d_ref, ref_label in [(0.2, 'Small'), (0.5, 'Medium')]:\n",
- " ax1.axvline(x=d_ref, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
- " ax1.text(d_ref, len(df_d_compare) - 0.3, ref_label,\n",
- " ha='center', fontsize=8, color='gray')\n",
+ " # --- Left panel: Cohen's d for continuous features ---\n",
+ " # Color by feature type only to avoid mixing significance flags derived from\n",
+ " # different multiple-testing correction families in one shared visual encoding.\n",
+ " y_pos_left = np.arange(len(df_d_compare))\n",
+ " palette_type = {'Demographic': '#4C72B0', 'Behavioral': '#55A868'}\n",
+ " colors_left = [palette_type[t] for t in df_d_compare['feature_type']]\n",
"\n",
- "ax1.set_yticks(y_pos_left)\n",
- "ax1.set_yticklabels(df_d_compare['label'])\n",
- "ax1.set_xlabel(f'|{LABEL_EFFECT_SIZE}|')\n",
- "ax1.set_title(f'Continuous Features: |{LABEL_EFFECT_SIZE}|\\n'\n",
- " '(blue = demographic, green = behavioral)')\n",
- "sns.despine(ax=ax1)\n",
+ " ax1.barh(y_pos_left, df_d_compare['effect_size'], color=colors_left, edgecolor='white')\n",
"\n",
- "# --- Right panel: Cramér's V for categorical demographics ---\n",
- "df_chi_plot = df_demo_chi.sort_values('effect_size', ascending=True).reset_index(drop=True)\n",
- "y_pos_right = np.arange(len(df_chi_plot))\n",
- "colors_right = ['#4C72B0' if sig else '#CCCCCC' for sig in df_chi_plot['significant']]\n",
- "\n",
- "ax2.barh(y_pos_right, df_chi_plot['effect_size'], color=colors_right, edgecolor='white')\n",
+ " for i, (_, row) in enumerate(df_d_compare.iterrows()):\n",
+ " ax1.text(\n",
+ " row['effect_size'] + 0.01, i,\n",
+ " f\"|d| = {row['effect_size']:.3f}\",\n",
+ " va='center', fontsize=9, color='#333333',\n",
+ " )\n",
"\n",
- "for i, (_, row) in enumerate(df_chi_plot.iterrows()):\n",
- " ax2.text(\n",
- " row['effect_size'] + 0.002, i,\n",
- " f\"V = {row['effect_size']:.4f}\",\n",
- " va='center', fontsize=9, color='#333333',\n",
- " )\n",
+ " # Reference lines for Cohen's d.\n",
+ " # Threshold labels are anchored to the axes top via a blended transform\n",
+ " # (x in data coordinates, y in axes fraction): a fixed data-coordinate y\n",
+ " # would land beyond the axis limit and collide with the panel title.\n",
+ " for d_ref, ref_label in [(0.2, L['small']), (0.5, L['medium'])]:\n",
+ " ax1.axvline(x=d_ref, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
+ " ax1.text(d_ref, 0.99, ref_label, transform=ax1.get_xaxis_transform(),\n",
+ " ha='center', va='top', fontsize=8, color='gray')\n",
+ "\n",
+ " ax1.set_yticks(y_pos_left)\n",
+ " ax1.set_yticklabels([L['all_labels'][f] for f in df_d_compare['feature']])\n",
+ " ax1.set_xlabel(f\"|{L['effect_size']}|\")\n",
+ " ax1.set_title(L['cont_title'])\n",
+ " sns.despine(ax=ax1)\n",
+ "\n",
+ " # --- Right panel: Cramér's V for categorical demographics ---\n",
+ " y_pos_right = np.arange(len(df_chi_plot))\n",
+ " colors_right = ['#4C72B0' if sig else '#CCCCCC' for sig in df_chi_plot['significant']]\n",
+ "\n",
+ " ax2.barh(y_pos_right, df_chi_plot['effect_size'], color=colors_right, edgecolor='white')\n",
+ "\n",
+ " for i, (_, row) in enumerate(df_chi_plot.iterrows()):\n",
+ " ax2.text(\n",
+ " row['effect_size'] + 0.002, i,\n",
+ " f\"V = {row['effect_size']:.4f}\",\n",
+ " va='center', fontsize=9, color='#333333',\n",
+ " )\n",
"\n",
- "ax2.axvline(x=0.1, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
- "ax2.text(0.1, len(df_chi_plot) - 0.3, 'Small',\n",
- " ha='center', fontsize=8, color='gray')\n",
+ " ax2.axvline(x=0.1, color='gray', linestyle=':', linewidth=0.8, alpha=0.6)\n",
+ " ax2.text(0.1, 0.99, L['small'], transform=ax2.get_xaxis_transform(),\n",
+ " ha='center', va='top', fontsize=8, color='gray')\n",
"\n",
- "ax2.set_yticks(y_pos_right)\n",
- "ax2.set_yticklabels(df_chi_plot['label'])\n",
- "ax2.set_xlabel(LABEL_CRAMERS_V)\n",
- "ax2.set_title(f'Categorical Demographics: {LABEL_CRAMERS_V}\\n(blue = significant)')\n",
- "sns.despine(ax=ax2)\n",
+ " ax2.set_yticks(y_pos_right)\n",
+ " ax2.set_yticklabels([L['all_labels'][f] for f in df_chi_plot['feature']])\n",
+ " ax2.set_xlabel(L['cramers_v'])\n",
+ " ax2.set_title(L['cat_title'])\n",
+ " sns.despine(ax=ax2)\n",
"\n",
- "fig.suptitle(\n",
- " 'BQ3: Demographics vs Behavior - Effect Size Comparison\\n'\n",
- " '(all-enrollment features only; conditional features reported below)',\n",
- " fontsize=14, y=1.02,\n",
- ")\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '05_demographics_vs_behavior_comparison')\n",
- "plt.show()\n",
+ " fig.suptitle(L['suptitle'], fontsize=14, y=1.02)\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '05_demographics_vs_behavior_comparison', lang)\n",
"\n",
"# --- Quantitative summary ---\n",
"# Compare d-vs-d only on comparable samples (all-enrollment metrics).\n",
@@ -817,7 +918,7 @@
},
{
"cell_type": "markdown",
- "id": "21",
+ "id": "22",
"metadata": {},
"source": [
"> **The Verdict: Behavior wins, decisively.**\n",
@@ -834,7 +935,7 @@
},
{
"cell_type": "markdown",
- "id": "22",
+ "id": "23",
"metadata": {},
"source": [
"## 6. Deep Dive: Education Level × Engagement\n",
@@ -847,7 +948,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "23",
+ "id": "24",
"metadata": {},
"outputs": [],
"source": [
@@ -878,37 +979,40 @@
" .index.tolist()\n",
")\n",
"\n",
- "fig, ax = plt.subplots(figsize=(12, 6))\n",
- "sns.barplot(\n",
- " data=interaction, x='highest_education', y='rate_pct',\n",
- " hue='engagement',\n",
- " palette={LABEL_HIGH_ENG: '#55A868', LABEL_LOW_ENG: '#C44E52'},\n",
- " order=edu_order, ax=ax, edgecolor='white',\n",
- ")\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "# Education level tick labels are data values and keep their source form.\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=(12, 6))\n",
+ " sns.barplot(\n",
+ " data=interaction, x='highest_education', y='rate_pct',\n",
+ " hue='engagement',\n",
+ " hue_order=[LABEL_HIGH_ENG, LABEL_LOW_ENG],\n",
+ " palette={LABEL_HIGH_ENG: '#55A868', LABEL_LOW_ENG: '#C44E52'},\n",
+ " order=edu_order, ax=ax, edgecolor='white',\n",
+ " )\n",
"\n",
- "# Annotate bars with completion rate\n",
- "for container in ax.containers:\n",
- " for bar in container:\n",
- " height = bar.get_height()\n",
- " if height > 0:\n",
- " ax.text(\n",
- " bar.get_x() + bar.get_width() / 2, height + 1,\n",
- " f'{height:.0f}%', ha='center', fontsize=8, color='#333333',\n",
- " )\n",
- "\n",
- "ax.set_xlabel('Highest Education Level')\n",
- "ax.set_ylabel(LABEL_COMPLETION_RATE)\n",
- "ax.set_title(\n",
- " 'Completion Rate: Education Level × Engagement\\n'\n",
- " f'(split at median = {median_active:.0f} active days in first 28 days)'\n",
- ")\n",
- "ax.set_ylim(0, 100)\n",
- "ax.legend(title='Engagement')\n",
- "plt.xticks(rotation=30, ha='right')\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '05_education_engagement_interaction')\n",
- "plt.show()\n",
+ " # Annotate bars with completion rate\n",
+ " for container in ax.containers:\n",
+ " for bar in container:\n",
+ " height = bar.get_height()\n",
+ " if height > 0:\n",
+ " ax.text(\n",
+ " bar.get_x() + bar.get_width() / 2, height + 1,\n",
+ " f'{height:.0f}%', ha='center', fontsize=8, color='#333333',\n",
+ " )\n",
+ "\n",
+ " ax.set_xlabel(L['edu_xlabel'])\n",
+ " ax.set_ylabel(L['completion_rate'])\n",
+ " ax.set_title(L['edu_title'].format(median=median_active))\n",
+ " ax.set_ylim(0, 100)\n",
+ " # Display-only legend localization: handles follow hue_order\n",
+ " handles, _ = ax.get_legend_handles_labels()\n",
+ " ax.legend(handles, [L['high_eng'], L['low_eng']], title=L['engagement'])\n",
+ " plt.xticks(rotation=30, ha='right')\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '05_education_engagement_interaction', lang)\n",
"\n",
"# --- Quantify the within-education engagement gaps ---\n",
"print('\\n=== Engagement Gap Within Each Education Level ===\\n')\n",
@@ -925,7 +1029,7 @@
},
{
"cell_type": "markdown",
- "id": "24",
+ "id": "25",
"metadata": {},
"source": [
"> **Key finding:** Within every education level, high-engagement students dramatically outperform low-engagement students. The within-education-level gap (engagement effect) is consistently **larger** than the between-level gap (education effect at the same engagement level).\n",
@@ -935,7 +1039,7 @@
},
{
"cell_type": "markdown",
- "id": "25",
+ "id": "26",
"metadata": {},
"source": [
"## 7. Ethical Framing\n",
@@ -956,7 +1060,7 @@
},
{
"cell_type": "markdown",
- "id": "26",
+ "id": "27",
"metadata": {},
"source": [
"## 8. Key Takeaways and Next Steps\n",
@@ -989,7 +1093,7 @@
},
{
"cell_type": "markdown",
- "id": "27",
+ "id": "28",
"metadata": {},
"source": [
"> **From predictors to design:** This notebook established that *what students do* matters more than *who they are*. The next question shifts focus from student-level features to **course-level design**: do some courses retain students better than others, and what design characteristics correlate with higher retention?\n",
@@ -1005,7 +1109,15 @@
"name": "python3"
},
"language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
"name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
"version": "3.13.0"
}
},
diff --git a/notebooks/06_bq4_course_comparison.ipynb b/notebooks/06_bq4_course_comparison.ipynb
index bf81465..23f5ef8 100644
--- a/notebooks/06_bq4_course_comparison.ipynb
+++ b/notebooks/06_bq4_course_comparison.ipynb
@@ -77,7 +77,8 @@
"- All database queries go through `src.db.connection.execute_query()`, the project's DB abstraction layer (ADR-003).\n",
"- BQ4's primary SQL query lives in `sql/queries/q_bq4_course_comparison.sql` and is loaded at runtime from disk. The query aggregates course-level metrics from `v_course_profile`, `v_engagement_daily`, and `v_engagement_early`.\n",
"- No statistical test functions are imported: with only 7 data points, all analysis is descriptive.\n",
- "- Figures are saved to `reports/figures/` at 150 DPI."
+ "- Figures are saved to `reports/figures/` at 150 DPI.\n",
+ "- **Figure localization:** figures embedded by the README and the executive report are rendered twice with identical filenames: English to `reports/figures/` (linked by the EN documents) and Italian to `reports/figures/it/` (linked by the IT documents). Notebook-only figures stay English, like the notebooks themselves (ADR-005)."
]
},
{
@@ -263,9 +264,70 @@
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": null,
"id": "7",
"metadata": {},
+ "outputs": [],
+ "source": [
+ "# --- Figure localization (EN canonical + IT) ---\n",
+ "# README and executive report ship in two languages. Figures EMBEDDED by\n",
+ "# those documents are rendered twice with the same filename: English to\n",
+ "# reports/figures/ (linked by the EN docs), Italian to reports/figures/it/\n",
+ "# (linked by the IT docs). Notebook-only figures keep the plain save_fig()\n",
+ "# path and stay English, like the notebooks themselves (ADR-005).\n",
+ "# Only chart furniture (titles, axis labels, legends) is localized;\n",
+ "# data-derived values (module codes, rates, counts) keep their source form.\n",
+ "LANGS = ('en', 'it')\n",
+ "FIG_DIRS = {'en': FIGURES_DIR, 'it': FIGURES_DIR / 'it'}\n",
+ "for _dir in FIG_DIRS.values():\n",
+ " _dir.mkdir(parents=True, exist_ok=True)\n",
+ "\n",
+ "\n",
+ "def save_fig_lang(fig, name: str, lang: str) -> None:\n",
+ " \"\"\"Save a localized figure; display only the EN (canonical) render.\"\"\"\n",
+ " path = FIG_DIRS[lang] / f'{name}.png'\n",
+ " fig.savefig(path, dpi=FIG_DPI, bbox_inches='tight', facecolor='white')\n",
+ " print(f' Saved: {path}')\n",
+ " if lang == 'en':\n",
+ " plt.show()\n",
+ " else:\n",
+ " plt.close(fig)\n",
+ "\n",
+ "\n",
+ "# EN entries reuse the shared label constants (single source of truth);\n",
+ "# IT entries translate the same chart furniture.\n",
+ "L10N = {\n",
+ " 'en': {\n",
+ " 'completion_rate': LABEL_COMPLETION_RATE,\n",
+ " 'ranking_title': 'Course Completion Rate Ranking\\n(averaged across presentations)',\n",
+ " 'overall_label': 'Overall rate: {rate:.1f}%',\n",
+ " 'length_label': DESIGN_LABELS['avg_course_length_days'],\n",
+ " 'density_label': DESIGN_LABELS['avg_assessment_density'],\n",
+ " 'length_title': 'Course Length vs Completion Rate',\n",
+ " 'density_title': 'Assessment Density vs Completion Rate',\n",
+ " 'design_suptitle': 'Course Design Features vs Completion Rate\\n'\n",
+ " '(bubble size = total enrollment)',\n",
+ " },\n",
+ " 'it': {\n",
+ " 'completion_rate': 'Tasso di completamento (%)',\n",
+ " 'ranking_title': 'Ranking dei corsi per tasso di completamento\\n(media sulle presentazioni)',\n",
+ " 'overall_label': 'Tasso complessivo: {rate:.1f}%',\n",
+ " 'length_label': 'Durata del corso (giorni)',\n",
+ " 'density_label': 'Densità di valutazioni (per 30 giorni)',\n",
+ " 'length_title': 'Durata del corso vs tasso di completamento',\n",
+ " 'density_title': 'Densità di valutazioni vs tasso di completamento',\n",
+ " 'design_suptitle': 'Caratteristiche di design del corso vs tasso di completamento\\n'\n",
+ " '(dimensione bolla = iscrizioni totali)',\n",
+ " },\n",
+ "}\n",
+ "print(f'Figure localization ready: {\", \".join(LANGS)}')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8",
+ "metadata": {},
"source": [
"> **First look:** The 7 OULAD modules span a wide range of completion rates and course designs. Some modules are shorter with fewer assessments; others are longer with more resources and higher assessment density. The question is whether these design differences relate to retention performance.\n",
">\n",
@@ -274,7 +336,7 @@
},
{
"cell_type": "markdown",
- "id": "8",
+ "id": "9",
"metadata": {},
"source": [
"## 3. Course Ranking by Completion\n",
@@ -287,7 +349,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "9",
+ "id": "10",
"metadata": {},
"outputs": [],
"source": [
@@ -298,43 +360,46 @@
" 'avg_completion_rate_pct', ascending=True\n",
").reset_index(drop=True)\n",
"\n",
- "fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
- "\n",
- "y_pos = np.arange(len(df_ranked))\n",
- "colors = [PALETTE_COURSE[m] for m in df_ranked['code_module']]\n",
- "\n",
- "ax.barh(y_pos, df_ranked['avg_completion_rate_pct'], color=colors, edgecolor='white')\n",
- "\n",
- "# Annotate each bar with completion rate and enrollment volume\n",
- "for i, (_, row) in enumerate(df_ranked.iterrows()):\n",
- " rate = row['avg_completion_rate_pct']\n",
- " enrolled = int(row['total_enrolled'])\n",
- " ax.text(\n",
- " rate + 0.8, i,\n",
- " f'{rate:.1f}% (n={enrolled:,})',\n",
- " va='center', fontsize=9, color='#333333',\n",
- " )\n",
- "\n",
"# True overall completion rate from raw counts across all presentations\n",
"# (total_completed / total_enrolled), not a simple/unweighted average of module-level averages\n",
"overall_rate = (\n",
" df_bq4['total_completed'].sum() / df_bq4['total_enrolled'].sum() * 100\n",
")\n",
- "ax.axvline(\n",
- " x=overall_rate, color='gray', linestyle='--', linewidth=1,\n",
- " label=f'Overall rate: {overall_rate:.1f}%',\n",
- ")\n",
"\n",
- "ax.set_yticks(y_pos)\n",
- "ax.set_yticklabels(df_ranked['code_module'])\n",
- "ax.set_xlabel(LABEL_COMPLETION_RATE)\n",
- "ax.set_title('Course Completion Rate Ranking\\n(averaged across presentations)')\n",
- "ax.legend(loc='lower right')\n",
- "ax.set_xlim(0, 100)\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '06_course_completion_ranking')\n",
- "plt.show()\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
+ "\n",
+ " y_pos = np.arange(len(df_ranked))\n",
+ " colors = [PALETTE_COURSE[m] for m in df_ranked['code_module']]\n",
+ "\n",
+ " ax.barh(y_pos, df_ranked['avg_completion_rate_pct'], color=colors, edgecolor='white')\n",
+ "\n",
+ " # Annotate each bar with completion rate and enrollment volume\n",
+ " for i, (_, row) in enumerate(df_ranked.iterrows()):\n",
+ " rate = row['avg_completion_rate_pct']\n",
+ " enrolled = int(row['total_enrolled'])\n",
+ " ax.text(\n",
+ " rate + 0.8, i,\n",
+ " f'{rate:.1f}% (n={enrolled:,})',\n",
+ " va='center', fontsize=9, color='#333333',\n",
+ " )\n",
+ "\n",
+ " ax.axvline(\n",
+ " x=overall_rate, color='gray', linestyle='--', linewidth=1,\n",
+ " label=L['overall_label'].format(rate=overall_rate),\n",
+ " )\n",
+ "\n",
+ " ax.set_yticks(y_pos)\n",
+ " ax.set_yticklabels(df_ranked['code_module'])\n",
+ " ax.set_xlabel(L['completion_rate'])\n",
+ " ax.set_title(L['ranking_title'])\n",
+ " ax.legend(loc='lower right')\n",
+ " ax.set_xlim(0, 100)\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '06_course_completion_ranking', lang)\n",
"\n",
"# --- Quantify the spread ---\n",
"best = df_ranked.iloc[-1]\n",
@@ -351,7 +416,7 @@
},
{
"cell_type": "markdown",
- "id": "10",
+ "id": "11",
"metadata": {},
"source": [
"> **Key finding:** The completion rate gap between the best- and worst-performing modules is substantial. This variation is not random: it persists across multiple presentations of the same module, suggesting that something about the course itself (design, subject difficulty, student selection) drives the difference.\n",
@@ -361,7 +426,7 @@
},
{
"cell_type": "markdown",
- "id": "11",
+ "id": "12",
"metadata": {},
"source": [
"## 4. Course Design Features vs Completion\n",
@@ -379,68 +444,66 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "12",
+ "id": "13",
"metadata": {},
"outputs": [],
"source": [
"# --- Course design features vs completion rate ---\n",
"# 1x2 scatter panel: course length and assessment density vs completion.\n",
"# Bubble size proportional to enrollment volume for context.\n",
- "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=FIG_SIZE_WIDE)\n",
- "\n",
"# Scale enrollment to bubble sizes (100–600 pixel range)\n",
"enrolled = df_bq4['total_enrolled'].values\n",
"size_scale = 100 + 500 * (enrolled - enrolled.min()) / max(enrolled.max() - enrolled.min(), 1)\n",
"\n",
- "# --- Left: course length vs completion ---\n",
- "for i, (_, row) in enumerate(df_bq4.iterrows()):\n",
- " ax1.scatter(\n",
- " row['avg_course_length_days'], row['avg_completion_rate_pct'],\n",
- " color=PALETTE_COURSE[row['code_module']],\n",
- " s=size_scale[i], edgecolor='white', linewidth=1.5, zorder=3,\n",
- " )\n",
- " ax1.annotate(\n",
- " row['code_module'],\n",
- " (row['avg_course_length_days'], row['avg_completion_rate_pct']),\n",
- " fontsize=9, fontweight='bold', ha='center', va='bottom',\n",
- " xytext=(0, 8), textcoords='offset points',\n",
- " )\n",
- "ax1.set_xlabel(DESIGN_LABELS['avg_course_length_days'])\n",
- "ax1.set_ylabel(LABEL_COMPLETION_RATE)\n",
- "ax1.set_title('Course Length vs Completion Rate')\n",
- "sns.despine(ax=ax1)\n",
- "\n",
- "# --- Right: assessment density vs completion ---\n",
- "for i, (_, row) in enumerate(df_bq4.iterrows()):\n",
- " ax2.scatter(\n",
- " row['avg_assessment_density'], row['avg_completion_rate_pct'],\n",
- " color=PALETTE_COURSE[row['code_module']],\n",
- " s=size_scale[i], edgecolor='white', linewidth=1.5, zorder=3,\n",
- " )\n",
- " ax2.annotate(\n",
- " row['code_module'],\n",
- " (row['avg_assessment_density'], row['avg_completion_rate_pct']),\n",
- " fontsize=9, fontweight='bold', ha='center', va='bottom',\n",
- " xytext=(0, 8), textcoords='offset points',\n",
- " )\n",
- "ax2.set_xlabel(DESIGN_LABELS['avg_assessment_density'])\n",
- "ax2.set_ylabel(LABEL_COMPLETION_RATE)\n",
- "ax2.set_title('Assessment Density vs Completion Rate')\n",
- "sns.despine(ax=ax2)\n",
- "\n",
- "fig.suptitle(\n",
- " 'Course Design Features vs Completion Rate\\n'\n",
- " '(bubble size = total enrollment)',\n",
- " fontsize=14, y=1.02,\n",
- ")\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '06_course_design_vs_completion')\n",
- "plt.show()"
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, (ax1, ax2) = plt.subplots(1, 2, figsize=FIG_SIZE_WIDE)\n",
+ "\n",
+ " # --- Left: course length vs completion ---\n",
+ " for i, (_, row) in enumerate(df_bq4.iterrows()):\n",
+ " ax1.scatter(\n",
+ " row['avg_course_length_days'], row['avg_completion_rate_pct'],\n",
+ " color=PALETTE_COURSE[row['code_module']],\n",
+ " s=size_scale[i], edgecolor='white', linewidth=1.5, zorder=3,\n",
+ " )\n",
+ " ax1.annotate(\n",
+ " row['code_module'],\n",
+ " (row['avg_course_length_days'], row['avg_completion_rate_pct']),\n",
+ " fontsize=9, fontweight='bold', ha='center', va='bottom',\n",
+ " xytext=(0, 8), textcoords='offset points',\n",
+ " )\n",
+ " ax1.set_xlabel(L['length_label'])\n",
+ " ax1.set_ylabel(L['completion_rate'])\n",
+ " ax1.set_title(L['length_title'])\n",
+ " sns.despine(ax=ax1)\n",
+ "\n",
+ " # --- Right: assessment density vs completion ---\n",
+ " for i, (_, row) in enumerate(df_bq4.iterrows()):\n",
+ " ax2.scatter(\n",
+ " row['avg_assessment_density'], row['avg_completion_rate_pct'],\n",
+ " color=PALETTE_COURSE[row['code_module']],\n",
+ " s=size_scale[i], edgecolor='white', linewidth=1.5, zorder=3,\n",
+ " )\n",
+ " ax2.annotate(\n",
+ " row['code_module'],\n",
+ " (row['avg_assessment_density'], row['avg_completion_rate_pct']),\n",
+ " fontsize=9, fontweight='bold', ha='center', va='bottom',\n",
+ " xytext=(0, 8), textcoords='offset points',\n",
+ " )\n",
+ " ax2.set_xlabel(L['density_label'])\n",
+ " ax2.set_ylabel(L['completion_rate'])\n",
+ " ax2.set_title(L['density_title'])\n",
+ " sns.despine(ax=ax2)\n",
+ "\n",
+ " fig.suptitle(L['design_suptitle'], fontsize=14, y=1.02)\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '06_course_design_vs_completion', lang)"
]
},
{
"cell_type": "markdown",
- "id": "13",
+ "id": "14",
"metadata": {},
"source": [
"> **Visual patterns:**\n",
@@ -453,7 +516,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "14",
+ "id": "15",
"metadata": {},
"outputs": [],
"source": [
@@ -489,7 +552,7 @@
},
{
"cell_type": "markdown",
- "id": "15",
+ "id": "16",
"metadata": {},
"source": [
"> **Correlation summary:** The Spearman rank correlations provide a compact summary of which features *move in the same direction* as completion rate. Features marked as notable (|ρ| ≥ 0.79) would pass the significance threshold for n=7, but even these should be treated as hypotheses.\n",
@@ -499,7 +562,7 @@
},
{
"cell_type": "markdown",
- "id": "16",
+ "id": "17",
"metadata": {},
"source": [
"## 5. Engagement Intensity by Course\n",
@@ -515,7 +578,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "17",
+ "id": "18",
"metadata": {},
"outputs": [],
"source": [
@@ -573,7 +636,7 @@
},
{
"cell_type": "markdown",
- "id": "18",
+ "id": "19",
"metadata": {},
"source": [
"> **Engagement variation:** Engagement intensity varies across courses, but the relationship with completion is not straightforward. A course might have high per-session engagement (many clicks per visit) but low completion if students eventually disengage.\n",
@@ -587,7 +650,7 @@
},
{
"cell_type": "markdown",
- "id": "19",
+ "id": "20",
"metadata": {},
"source": [
"## 6. Course Design Heatmap\n",
@@ -604,7 +667,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "20",
+ "id": "21",
"metadata": {},
"outputs": [],
"source": [
@@ -681,7 +744,7 @@
},
{
"cell_type": "markdown",
- "id": "21",
+ "id": "22",
"metadata": {},
"source": [
"> **Reading the heatmap:** Look for vertical color patterns (features that consistently differ between high- and low-retention courses) and horizontal patterns (modules with consistently above- or below-average characteristics).\n",
@@ -696,7 +759,7 @@
},
{
"cell_type": "markdown",
- "id": "22",
+ "id": "23",
"metadata": {},
"source": [
"## 7. Hypotheses and Limitations\n",
@@ -728,7 +791,7 @@
},
{
"cell_type": "markdown",
- "id": "23",
+ "id": "24",
"metadata": {},
"source": [
"## 8. Key Takeaways and Next Steps\n",
@@ -760,7 +823,7 @@
},
{
"cell_type": "markdown",
- "id": "24",
+ "id": "25",
"metadata": {},
"source": [
"> **From course profiles to action:** This notebook characterized *how courses differ* in design, engagement, and retention. Combined with the student-level insights from BQ1–BQ3, we now have the full picture needed to answer the final question: what can a platform operator actually do?\n",
@@ -783,6 +846,8 @@
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
"version": "3.13.0"
}
},
diff --git a/notebooks/07_bq5_recommendations_synthesis.ipynb b/notebooks/07_bq5_recommendations_synthesis.ipynb
index 6af5f04..ff054de 100644
--- a/notebooks/07_bq5_recommendations_synthesis.ipynb
+++ b/notebooks/07_bq5_recommendations_synthesis.ipynb
@@ -37,7 +37,7 @@
"**What came before:**\n",
"- **NB03** (BQ1): where and when students drop out: dropout curves, cliff detection\n",
"- **NB04** (BQ2): early behavioral signals that predict dropout: effect sizes, dose-response\n",
- "- **NB05** (BQ3): demographics vs behavior: behavior predicts outcome 2–5× more strongly\n",
+ "- **NB05** (BQ3): demographics vs behavior: behavior predicts outcome 2–4× more strongly\n",
"- **NB06** (BQ4): course design vs retention: descriptive course profiles, exploratory correlations\n",
"\n",
"> **Methodological transferability:** This synthesis pattern (segment sizing → intervention design → impact estimation → prioritization) is the standard \"churn intervention playbook\" in SaaS product analytics. The three segments (ghost users, feature non-adopters, early disengagers) map directly to subscription churn, fitness app retention, and freemium conversion contexts."
@@ -84,7 +84,8 @@
"- BQ5's primary SQL query lives in `sql/queries/q_bq5_segment_sizing.sql` and is loaded at runtime from disk. It sizes three intervention segments from `v_student_enriched` and `v_engagement_early`.\n",
"- Additional inline SQL queries compute impact estimates for each recommendation. These are specific to this notebook's synthesis narrative and not reusable as standalone queries (consistent with the inline query pattern in NB03).\n",
"- No statistical test imports: this is a synthesis notebook, all evidence comes from NB03–NB06.\n",
- "- Figures are saved to `reports/figures/` at 150 DPI."
+ "- Figures are saved to `reports/figures/` at 150 DPI.\n",
+ "- **Figure localization:** figures embedded by the README and the executive report are rendered twice with identical filenames: English to `reports/figures/` (linked by the EN documents) and Italian to `reports/figures/it/` (linked by the IT documents). Notebook-only figures stay English, like the notebooks themselves (ADR-005)."
]
},
{
@@ -138,23 +139,6 @@
"# --- Visualization defaults ---\n",
"sns.set_theme(style='whitegrid', font_scale=1.1)\n",
"\n",
- "# Segment palette: one color per intervention segment for consistent\n",
- "# identification across all figures. Colors chosen for semantic clarity:\n",
- "# red = most critical (ghost), orange = medium (assessment), blue = re-engagement.\n",
- "PALETTE_SEGMENT = {\n",
- " 'Ghost students': '#C44E52',\n",
- " 'Assessment non-submitters': '#DD8452',\n",
- " 'Early disengagers': '#4C72B0',\n",
- "}\n",
- "SEGMENT_ORDER = list(PALETTE_SEGMENT.keys())\n",
- "\n",
- "# Shared axis labels and section headers: defined as constants to avoid\n",
- "# duplicated string literals flagged by static analysis\n",
- "LABEL_COMPLETION_RATE = 'Completion rate (%)'\n",
- "LABEL_NON_COMPLETION_RATE = 'Non-completion rate (%)'\n",
- "LABEL_NUM_STUDENTS = 'Number of students'\n",
- "HEADER_SCENARIO = '=== Scenario Analysis ==='\n",
- "\n",
"FIG_DPI = 150\n",
"FIG_SIZE = (10, 6)\n",
"FIG_SIZE_WIDE = (16, 5)\n",
@@ -195,9 +179,172 @@
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": null,
"id": "5",
"metadata": {},
+ "outputs": [],
+ "source": [
+ "# --- Display labels (single source of truth) ---\n",
+ "# Segment, overlap-category and recommendation names are used as dictionary\n",
+ "# KEYS shared across cells: the palette, the L10N maps and the analysis\n",
+ "# DataFrames must agree on the exact spelling, otherwise a lookup fails at\n",
+ "# runtime (or a bar silently falls back to gray). Defining each label once\n",
+ "# makes that cross-cell contract explicit and typo-proof.\n",
+ "\n",
+ "# Segment palette: one color per intervention segment for consistent\n",
+ "# identification across all figures. Colors chosen for semantic clarity:\n",
+ "# red = most critical (ghost), orange = medium (assessment), blue = re-engagement.\n",
+ "SEG_GHOST = 'Ghost students'\n",
+ "SEG_NON_SUBMITTER = 'Assessment non-submitters'\n",
+ "SEG_DISENGAGER = 'Early disengagers'\n",
+ "PALETTE_SEGMENT = {\n",
+ " SEG_GHOST: '#C44E52',\n",
+ " SEG_NON_SUBMITTER: '#DD8452',\n",
+ " SEG_DISENGAGER: '#4C72B0',\n",
+ "}\n",
+ "SEGMENT_ORDER = list(PALETTE_SEGMENT.keys())\n",
+ "\n",
+ "# Overlap categories (Section 3): exclusive membership first, then shared.\n",
+ "# The tuple order is load-bearing: the overlap DataFrame builds its counts\n",
+ "# in this exact order.\n",
+ "CAT_GHOST_ONLY = 'Ghost only'\n",
+ "CAT_NON_SUBMITTER_ONLY = 'Non-submitter only'\n",
+ "CAT_DISENGAGER_ONLY = 'Disengager only'\n",
+ "CAT_GHOST_NON_SUBMITTER = 'Ghost + Non-submitter'\n",
+ "CAT_NON_SUBMITTER_DISENGAGER = 'Non-submitter + Disengager'\n",
+ "CAT_GHOST_DISENGAGER = 'Ghost + Disengager'\n",
+ "CAT_ALL_THREE = 'All three'\n",
+ "OVERLAP_CATEGORIES = (\n",
+ " CAT_GHOST_ONLY,\n",
+ " CAT_NON_SUBMITTER_ONLY,\n",
+ " CAT_DISENGAGER_ONLY,\n",
+ " CAT_GHOST_NON_SUBMITTER,\n",
+ " CAT_NON_SUBMITTER_DISENGAGER,\n",
+ " CAT_GHOST_DISENGAGER,\n",
+ " CAT_ALL_THREE,\n",
+ ")\n",
+ "\n",
+ "# Recommendation names (Sections 4-8): aligned index-by-index with\n",
+ "# SEGMENT_ORDER (one intervention per segment), so the priority matrix\n",
+ "# can zip them row-wise.\n",
+ "REC_ACTIVATION = 'Ghost Student Activation'\n",
+ "REC_CHECKPOINT = 'First Assessment Checkpoint'\n",
+ "REC_REENGAGEMENT = 'Week 3 Re-engagement'\n",
+ "RECOMMENDATIONS = (REC_ACTIVATION, REC_CHECKPOINT, REC_REENGAGEMENT)\n",
+ "\n",
+ "# Shared axis labels and section headers: defined as constants to avoid\n",
+ "# duplicated string literals flagged by static analysis\n",
+ "LABEL_NON_COMPLETION_RATE = 'Non-completion rate (%)'\n",
+ "LABEL_NUM_STUDENTS = 'Number of students'\n",
+ "HEADER_SCENARIO = '=== Scenario Analysis ==='\n",
+ "\n",
+ "print(f'Display labels ready: {len(SEGMENT_ORDER)} segments, '\n",
+ " f'{len(OVERLAP_CATEGORIES)} overlap categories, '\n",
+ " f'{len(RECOMMENDATIONS)} recommendations')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# --- Figure localization (EN canonical + IT) ---\n",
+ "# README and executive report ship in two languages. Figures EMBEDDED by\n",
+ "# those documents are rendered twice with the same filename: English to\n",
+ "# reports/figures/ (linked by the EN docs), Italian to reports/figures/it/\n",
+ "# (linked by the IT docs). Notebook-only figures keep the plain save_fig()\n",
+ "# path and stay English, like the notebooks themselves (ADR-005).\n",
+ "# Only chart furniture (titles, axis labels, legends) is localized;\n",
+ "# data-derived values (counts, rates) keep their source form. Segment and\n",
+ "# recommendation names are display labels, so they get a per-language map.\n",
+ "LANGS = ('en', 'it')\n",
+ "FIG_DIRS = {'en': FIGURES_DIR, 'it': FIGURES_DIR / 'it'}\n",
+ "for _dir in FIG_DIRS.values():\n",
+ " _dir.mkdir(parents=True, exist_ok=True)\n",
+ "\n",
+ "\n",
+ "def save_fig_lang(fig, name: str, lang: str) -> None:\n",
+ " \"\"\"Save a localized figure; display only the EN (canonical) render.\"\"\"\n",
+ " path = FIG_DIRS[lang] / f'{name}.png'\n",
+ " fig.savefig(path, dpi=FIG_DPI, bbox_inches='tight', facecolor='white')\n",
+ " print(f' Saved: {path}')\n",
+ " if lang == 'en':\n",
+ " plt.show()\n",
+ " else:\n",
+ " plt.close(fig)\n",
+ "\n",
+ "\n",
+ "# Both language maps are keyed by the display-label constants above, so a\n",
+ "# renamed segment/category/recommendation cannot drift out of sync with its\n",
+ "# translation. EN display entries are identity maps built by comprehension\n",
+ "# (single source of truth, no literal duplication); IT entries translate\n",
+ "# the same chart furniture. Italian segment names match the executive\n",
+ "# report terminology.\n",
+ "L10N = {\n",
+ " 'en': {\n",
+ " 'segment_display': {s: s for s in SEGMENT_ORDER},\n",
+ " 'num_students': LABEL_NUM_STUDENTS,\n",
+ " 'non_completion_rate': LABEL_NON_COMPLETION_RATE,\n",
+ " 'size_title': 'Segment Size',\n",
+ " 'rate_title': 'Non-completion Rate by Segment',\n",
+ " 'overall_label': 'Overall: {rate:.1f}%',\n",
+ " 'sizing_suptitle': 'BQ5 Segment Sizing: Who Should We Target?\\n'\n",
+ " '(total enrollments: {total:,})',\n",
+ " 'category_display': {c: c for c in OVERLAP_CATEGORIES},\n",
+ " 'overlap_title': 'Segment Overlap: Exclusive and Shared Membership\\n'\n",
+ " '(gray bars = students in multiple segments)',\n",
+ " 'rec_display': {r: r for r in RECOMMENDATIONS},\n",
+ " 'cost_ticks': ['Low', 'Medium', 'Medium-High'],\n",
+ " 'cost_xlabel': 'Implementation Cost',\n",
+ " 'impact_ylabel': 'Estimated Additional Completions\\n(middle scenario)',\n",
+ " 'matrix_title': 'Priority Matrix: Impact vs Cost\\n(bubble size = segment size)',\n",
+ " },\n",
+ " 'it': {\n",
+ " 'segment_display': {\n",
+ " SEG_GHOST: 'Studenti ghost',\n",
+ " SEG_NON_SUBMITTER: 'Non-submitter',\n",
+ " SEG_DISENGAGER: 'Early disengager',\n",
+ " },\n",
+ " 'num_students': 'Numero di studenti',\n",
+ " 'non_completion_rate': 'Tasso di non completamento (%)',\n",
+ " 'size_title': 'Dimensione del segmento',\n",
+ " 'rate_title': 'Tasso di non completamento per segmento',\n",
+ " 'overall_label': 'Complessivo: {rate:.1f}%',\n",
+ " 'sizing_suptitle': 'BQ5 dimensionamento dei segmenti: chi dovremmo raggiungere?\\n'\n",
+ " '(iscrizioni totali: {total:,})',\n",
+ " 'category_display': {\n",
+ " CAT_GHOST_ONLY: 'Solo ghost',\n",
+ " CAT_NON_SUBMITTER_ONLY: 'Solo non-submitter',\n",
+ " CAT_DISENGAGER_ONLY: 'Solo disengager',\n",
+ " CAT_GHOST_NON_SUBMITTER: 'Ghost e non-submitter',\n",
+ " CAT_NON_SUBMITTER_DISENGAGER: 'Non-submitter e disengager',\n",
+ " CAT_GHOST_DISENGAGER: 'Ghost e disengager',\n",
+ " CAT_ALL_THREE: 'Tutti e tre',\n",
+ " },\n",
+ " 'overlap_title': 'Sovrapposizione dei segmenti: appartenenza esclusiva e condivisa\\n'\n",
+ " '(barre grigie = studenti in più segmenti)',\n",
+ " 'rec_display': {\n",
+ " REC_ACTIVATION: 'Attivazione Ghost',\n",
+ " REC_CHECKPOINT: 'Checkpoint Valutazioni',\n",
+ " REC_REENGAGEMENT: 'Re-engagement Settimana 3',\n",
+ " },\n",
+ " 'cost_ticks': ['Basso', 'Medio', 'Medio-alto'],\n",
+ " 'cost_xlabel': 'Costo di implementazione',\n",
+ " 'impact_ylabel': 'Completamenti aggiuntivi stimati\\n(scenario intermedio)',\n",
+ " 'matrix_title': 'Matrice di priorità: impatto vs costo\\n'\n",
+ " '(dimensione bolla = dimensione del segmento)',\n",
+ " },\n",
+ "}\n",
+ "print(f'Figure localization ready: {\", \".join(LANGS)}')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7",
+ "metadata": {},
"source": [
"## 2. Segment Sizing: The Three Target Populations\n",
"\n",
@@ -209,13 +356,13 @@
"| **Assessment non-submitters** | No assessment submitted in first 28 days | Missing the first deadline is a powerful binary signal. BQ2 identified `submitted_first_assessment` as a key predictor. |\n",
"| **Early disengagers** | Had VLE activity in days 0–14 but zero activity in days 15–28 | Started but lost momentum. BQ1 (NB03) showed mid-course dropout cliffs, often at assessment points. |\n",
"\n",
- "**Design principle:** All three segments are defined by *behavior*, not demographics. This is consistent with BQ3's finding (NB05) that behavioral signals predict outcome 2–5× more strongly than demographic features. Interventions targeting behavior are both more effective and more ethical."
+ "**Design principle:** All three segments are defined by *behavior*, not demographics. This is consistent with BQ3's finding (NB05) that behavioral signals predict outcome 2–4× more strongly than demographic features. Interventions targeting behavior are both more effective and more ethical."
]
},
{
"cell_type": "code",
"execution_count": null,
- "id": "6",
+ "id": "8",
"metadata": {},
"outputs": [],
"source": [
@@ -268,66 +415,68 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "7",
+ "id": "9",
"metadata": {},
"outputs": [],
"source": [
"# --- Figure: Segment sizing overview ---\n",
"# 1x2 panel: segment sizes (left) and non-completion rates (right).\n",
"# Segments ordered by severity (SEGMENT_ORDER) for consistent reading.\n",
- "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=FIG_SIZE_WIDE)\n",
- "\n",
- "y_pos = np.arange(len(segments))\n",
- "colors = [PALETTE_SEGMENT[s] for s in segments['segment']]\n",
- "\n",
- "# --- Left: segment size ---\n",
- "ax1.barh(y_pos, segments['n'], color=colors, edgecolor='white')\n",
- "for i, (_, row) in enumerate(segments.iterrows()):\n",
- " ax1.text(\n",
- " row['n'] + total_students * 0.01, i,\n",
- " f\"{row['n']:,} ({row['pct_of_total']:.1f}%)\",\n",
- " va='center', fontsize=10, color='#333333',\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, (ax1, ax2) = plt.subplots(1, 2, figsize=FIG_SIZE_WIDE)\n",
+ "\n",
+ " y_pos = np.arange(len(segments))\n",
+ " colors = [PALETTE_SEGMENT[s] for s in segments['segment']]\n",
+ " segment_labels = [L['segment_display'][s] for s in segments['segment']]\n",
+ "\n",
+ " # --- Left: segment size ---\n",
+ " ax1.barh(y_pos, segments['n'], color=colors, edgecolor='white')\n",
+ " for i, (_, row) in enumerate(segments.iterrows()):\n",
+ " ax1.text(\n",
+ " row['n'] + total_students * 0.01, i,\n",
+ " f\"{row['n']:,} ({row['pct_of_total']:.1f}%)\",\n",
+ " va='center', fontsize=10, color='#333333',\n",
+ " )\n",
+ " ax1.set_yticks(y_pos)\n",
+ " ax1.set_yticklabels(segment_labels)\n",
+ " ax1.set_xlabel(L['num_students'])\n",
+ " ax1.set_title(L['size_title'])\n",
+ " sns.despine(ax=ax1)\n",
+ "\n",
+ " # --- Right: non-completion rate ---\n",
+ " ax2.barh(y_pos, segments['non_completion_rate'], color=colors, edgecolor='white')\n",
+ " for i, (_, row) in enumerate(segments.iterrows()):\n",
+ " ax2.text(\n",
+ " row['non_completion_rate'] + 1, i,\n",
+ " f\"{row['non_completion_rate']:.1f}%\",\n",
+ " va='center', fontsize=10, color='#333333',\n",
+ " )\n",
+ " # Overall baseline reference line\n",
+ " ax2.axvline(\n",
+ " x=overall_non_completion, color='gray', linestyle='--', linewidth=1,\n",
+ " label=L['overall_label'].format(rate=overall_non_completion),\n",
" )\n",
- "ax1.set_yticks(y_pos)\n",
- "ax1.set_yticklabels(segments['segment'])\n",
- "ax1.set_xlabel(LABEL_NUM_STUDENTS)\n",
- "ax1.set_title('Segment Size')\n",
- "sns.despine(ax=ax1)\n",
- "\n",
- "# --- Right: non-completion rate ---\n",
- "ax2.barh(y_pos, segments['non_completion_rate'], color=colors, edgecolor='white')\n",
- "for i, (_, row) in enumerate(segments.iterrows()):\n",
- " ax2.text(\n",
- " row['non_completion_rate'] + 1, i,\n",
- " f\"{row['non_completion_rate']:.1f}%\",\n",
- " va='center', fontsize=10, color='#333333',\n",
+ " ax2.set_yticks(y_pos)\n",
+ " ax2.set_yticklabels(segment_labels)\n",
+ " ax2.set_xlabel(L['non_completion_rate'])\n",
+ " ax2.set_title(L['rate_title'])\n",
+ " ax2.set_xlim(0, 105)\n",
+ " ax2.legend(loc='lower right')\n",
+ " sns.despine(ax=ax2)\n",
+ "\n",
+ " fig.suptitle(\n",
+ " L['sizing_suptitle'].format(total=total_students),\n",
+ " fontsize=14, y=1.02,\n",
" )\n",
- "# Overall baseline reference line\n",
- "ax2.axvline(\n",
- " x=overall_non_completion, color='gray', linestyle='--', linewidth=1,\n",
- " label=f'Overall: {overall_non_completion:.1f}%',\n",
- ")\n",
- "ax2.set_yticks(y_pos)\n",
- "ax2.set_yticklabels(segments['segment'])\n",
- "ax2.set_xlabel(LABEL_NON_COMPLETION_RATE)\n",
- "ax2.set_title('Non-completion Rate by Segment')\n",
- "ax2.set_xlim(0, 105)\n",
- "ax2.legend(loc='lower right')\n",
- "sns.despine(ax=ax2)\n",
- "\n",
- "fig.suptitle(\n",
- " 'BQ5 Segment Sizing: Who Should We Target?\\n'\n",
- " f'(total enrollments: {total_students:,})',\n",
- " fontsize=14, y=1.02,\n",
- ")\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '07_segment_sizing_overview')\n",
- "plt.show()"
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '07_segment_sizing_overview', lang)"
]
},
{
"cell_type": "markdown",
- "id": "8",
+ "id": "10",
"metadata": {},
"source": [
"> **Segment profile:** All three segments show non-completion rates substantially above the overall baseline. The gap between each segment's rate and the platform average quantifies the \"excess non-completion\": the portion potentially addressable through targeted intervention.\n",
@@ -337,7 +486,7 @@
},
{
"cell_type": "markdown",
- "id": "9",
+ "id": "11",
"metadata": {},
"source": [
"## 3. Segment Overlap\n",
@@ -352,7 +501,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "10",
+ "id": "12",
"metadata": {},
"outputs": [],
"source": [
@@ -463,23 +612,17 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "11",
+ "id": "13",
"metadata": {},
"outputs": [],
"source": [
"# --- Figure: Segment overlap ---\n",
"# Horizontal bar showing exclusive and overlapping membership counts.\n",
"# This helps understand how much the interventions would target the same students.\n",
+ "# The 'n' list is aligned position-by-position with OVERLAP_CATEGORIES\n",
+ "# (exclusive first, then shared): reordering one side breaks the pairing.\n",
"overlap_data = pd.DataFrame({\n",
- " 'category': [\n",
- " 'Ghost only',\n",
- " 'Non-submitter only',\n",
- " 'Disengager only',\n",
- " 'Ghost + Non-submitter',\n",
- " 'Non-submitter + Disengager',\n",
- " 'Ghost + Disengager',\n",
- " 'All three',\n",
- " ],\n",
+ " 'category': list(OVERLAP_CATEGORIES),\n",
" 'n': [\n",
" int(row['ghost_only']),\n",
" int(row['nonsub_only']),\n",
@@ -495,43 +638,44 @@
"# Filter out zero-count categories for cleaner visualization\n",
"overlap_data = overlap_data[overlap_data['n'] > 0].sort_values('n', ascending=True)\n",
"\n",
- "# Color mapping: exclusive categories get the segment color,\n",
- "# overlap categories get a neutral gray\n",
- "overlap_colors = []\n",
- "for cat in overlap_data['category']:\n",
- " if cat == 'Ghost only':\n",
- " overlap_colors.append(PALETTE_SEGMENT['Ghost students'])\n",
- " elif cat == 'Non-submitter only':\n",
- " overlap_colors.append(PALETTE_SEGMENT['Assessment non-submitters'])\n",
- " elif cat == 'Disengager only':\n",
- " overlap_colors.append(PALETTE_SEGMENT['Early disengagers'])\n",
- " else:\n",
- " overlap_colors.append('#999999')\n",
- "\n",
- "fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
- "y_pos = np.arange(len(overlap_data))\n",
- "\n",
- "ax.barh(y_pos, overlap_data['n'].values, color=overlap_colors, edgecolor='white')\n",
- "for i, (_, cat_row) in enumerate(overlap_data.iterrows()):\n",
- " n_val = cat_row['n']\n",
- " pct = 100.0 * n_val / total_students\n",
- " ax.text(\n",
- " n_val + total_students * 0.005, i,\n",
- " f'{n_val:,} ({pct:.1f}%)',\n",
- " va='center', fontsize=9, color='#333333',\n",
- " )\n",
- "\n",
- "ax.set_yticks(y_pos)\n",
- "ax.set_yticklabels(overlap_data['category'].values)\n",
- "ax.set_xlabel(LABEL_NUM_STUDENTS)\n",
- "ax.set_title(\n",
- " 'Segment Overlap: Exclusive and Shared Membership\\n'\n",
- " '(gray bars = students in multiple segments)'\n",
- ")\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '07_segment_overlap')\n",
- "plt.show()\n",
+ "# Color mapping: exclusive categories inherit their segment color; shared\n",
+ "# categories fall back to neutral gray (they belong to more than one segment,\n",
+ "# so no single segment color would be honest).\n",
+ "EXCLUSIVE_CATEGORY_SEGMENT = {\n",
+ " CAT_GHOST_ONLY: SEG_GHOST,\n",
+ " CAT_NON_SUBMITTER_ONLY: SEG_NON_SUBMITTER,\n",
+ " CAT_DISENGAGER_ONLY: SEG_DISENGAGER,\n",
+ "}\n",
+ "overlap_colors = [\n",
+ " PALETTE_SEGMENT[EXCLUSIVE_CATEGORY_SEGMENT[cat]]\n",
+ " if cat in EXCLUSIVE_CATEGORY_SEGMENT\n",
+ " else '#999999'\n",
+ " for cat in overlap_data['category']\n",
+ "]\n",
+ "\n",
+ "# Rendered per language: embedded by the executive report (EN and IT).\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
+ " y_pos = np.arange(len(overlap_data))\n",
+ "\n",
+ " ax.barh(y_pos, overlap_data['n'].values, color=overlap_colors, edgecolor='white')\n",
+ " for i, (_, cat_row) in enumerate(overlap_data.iterrows()):\n",
+ " n_val = cat_row['n']\n",
+ " pct = 100.0 * n_val / total_students\n",
+ " ax.text(\n",
+ " n_val + total_students * 0.005, i,\n",
+ " f'{n_val:,} ({pct:.1f}%)',\n",
+ " va='center', fontsize=9, color='#333333',\n",
+ " )\n",
+ "\n",
+ " ax.set_yticks(y_pos)\n",
+ " ax.set_yticklabels([L['category_display'][c] for c in overlap_data['category']])\n",
+ " ax.set_xlabel(L['num_students'])\n",
+ " ax.set_title(L['overlap_title'])\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '07_segment_overlap', lang)\n",
"\n",
"# --- Key overlap metric for downstream impact estimation ---\n",
"# Percentage of ghost students who are also non-submitters\n",
@@ -547,7 +691,7 @@
},
{
"cell_type": "markdown",
- "id": "12",
+ "id": "14",
"metadata": {},
"source": [
"> **Overlap insight:** Ghost students and assessment non-submitters overlap heavily: a student who never accesses the VLE cannot submit an assessment. This means **Recommendations 1 and 2 largely target the same population** from different angles. Early disengagers, by definition, had *some* initial activity, so they overlap less with ghost students. This makes Recommendation 3 an independent intervention targeting a different failure mode.\n",
@@ -557,7 +701,7 @@
},
{
"cell_type": "markdown",
- "id": "13",
+ "id": "15",
"metadata": {},
"source": [
"## 4. Recommendation 1: Ghost Student Activation\n",
@@ -569,7 +713,7 @@
"### Evidence from BQ1–BQ4\n",
"\n",
"- **BQ2 (NB04):** Early engagement (active days, total clicks in first 28 days) has the largest effect size among all behavioral predictors. Students with zero early activity have near-zero completion rates.\n",
- "- **BQ3 (NB05):** Behavior predicts outcome 2–5× more strongly than demographics. This means we should target *what students do* (or fail to do), not *who they are*.\n",
+ "- **BQ3 (NB05):** Behavior predicts outcome 2–4× more strongly than demographics. This means we should target *what students do* (or fail to do), not *who they are*.\n",
"- **BQ1 (NB03):** A significant fraction of withdrawals happen in the first two weeks, before most students have even established a study routine.\n",
"\n",
"### Proposed Intervention\n",
@@ -587,7 +731,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "14",
+ "id": "16",
"metadata": {},
"outputs": [],
"source": [
@@ -652,7 +796,7 @@
},
{
"cell_type": "markdown",
- "id": "15",
+ "id": "17",
"metadata": {},
"source": [
"> **Interpretation:** The completion rate gap between ghost and active students is substantial. Even a modest activation rate (20%) would produce meaningful additional completions because the segment is large and the gap is wide.\n",
@@ -662,7 +806,7 @@
},
{
"cell_type": "markdown",
- "id": "16",
+ "id": "18",
"metadata": {},
"source": [
"## 5. Recommendation 2: First Assessment Checkpoint\n",
@@ -692,7 +836,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "17",
+ "id": "19",
"metadata": {},
"outputs": [],
"source": [
@@ -753,7 +897,7 @@
},
{
"cell_type": "markdown",
- "id": "18",
+ "id": "20",
"metadata": {},
"source": [
"> **Interpretation:** The gap between submitters and non-submitters is stark. Assessment submission is both a *signal* (it reveals commitment) and a *mechanism* (it creates accountability). This dual nature makes it an ideal intervention point.\n",
@@ -763,7 +907,7 @@
},
{
"cell_type": "markdown",
- "id": "19",
+ "id": "21",
"metadata": {},
"source": [
"## 6. Recommendation 3: Week 3 Re-engagement Campaign\n",
@@ -793,7 +937,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "20",
+ "id": "22",
"metadata": {},
"outputs": [],
"source": [
@@ -873,7 +1017,7 @@
},
{
"cell_type": "markdown",
- "id": "21",
+ "id": "23",
"metadata": {},
"source": [
"> **Interpretation:** Early disengagers have already demonstrated willingness to engage: they are not ghost students. This means a re-engagement nudge has a plausible mechanism: reminding someone who *was* active to come back. The impact estimate uses a conservative target (halfway between disengaged and sustained rates) because re-engagement after a gap is harder than sustained momentum.\n",
@@ -883,7 +1027,7 @@
},
{
"cell_type": "markdown",
- "id": "22",
+ "id": "24",
"metadata": {},
"source": [
"## 7. Priority Matrix\n",
@@ -899,17 +1043,15 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "23",
+ "id": "25",
"metadata": {},
"outputs": [],
"source": [
"# --- Priority matrix data ---\n",
+ "# RECOMMENDATIONS and SEGMENT_ORDER are aligned index-by-index (one\n",
+ "# intervention per segment), so building the rows positionally is safe.\n",
"priority = pd.DataFrame({\n",
- " 'recommendation': [\n",
- " 'Ghost Student Activation',\n",
- " 'First Assessment Checkpoint',\n",
- " 'Week 3 Re-engagement',\n",
- " ],\n",
+ " 'recommendation': list(RECOMMENDATIONS),\n",
" 'segment': SEGMENT_ORDER,\n",
" 'segment_size': [\n",
" segments.loc[0, 'n'],\n",
@@ -951,15 +1093,13 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "24",
+ "id": "26",
"metadata": {},
"outputs": [],
"source": [
"# --- Figure: Priority matrix ---\n",
"# Bubble chart: x = cost (categorical), y = estimated impact,\n",
"# bubble size proportional to segment size, colored by segment.\n",
- "fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
- "\n",
"# Scale bubble sizes for visual clarity (200-800 pixel range)\n",
"sizes = priority['segment_size'].values.astype(float)\n",
"size_min, size_max = sizes.min(), sizes.max()\n",
@@ -971,54 +1111,55 @@
"\n",
"colors = [PALETTE_SEGMENT[s] for s in priority['segment']]\n",
"\n",
- "# Plot scatter points first so axes scale to the actual data\n",
- "for i, (_, p_row) in enumerate(priority.iterrows()):\n",
- " ax.scatter(\n",
- " p_row['cost_score'], p_row['est_additional_completions'],\n",
- " s=bubble_sizes[i], color=colors[i],\n",
- " edgecolor='white', linewidth=2, zorder=3, alpha=0.85,\n",
- " )\n",
- " # Label each bubble with recommendation name\n",
- " ax.annotate(\n",
- " p_row['recommendation'],\n",
- " (p_row['cost_score'], p_row['est_additional_completions']),\n",
- " fontsize=9, fontweight='bold', ha='center', va='bottom',\n",
- " xytext=(0, 12), textcoords='offset points',\n",
- " )\n",
- " # Annotate with impact number inside/below bubble\n",
- " ax.annotate(\n",
- " f\"~{p_row['est_additional_completions']:,}\",\n",
- " (p_row['cost_score'], p_row['est_additional_completions']),\n",
- " fontsize=8, ha='center', va='top',\n",
- " xytext=(0, -10), textcoords='offset points', color='#555555',\n",
+ "# Rendered per language: embedded by both READMEs and both reports.\n",
+ "for lang in LANGS:\n",
+ " L = L10N[lang]\n",
+ " fig, ax = plt.subplots(figsize=FIG_SIZE)\n",
+ "\n",
+ " # Plot scatter points first so axes scale to the actual data\n",
+ " for i, (_, p_row) in enumerate(priority.iterrows()):\n",
+ " ax.scatter(\n",
+ " p_row['cost_score'], p_row['est_additional_completions'],\n",
+ " s=bubble_sizes[i], color=colors[i],\n",
+ " edgecolor='white', linewidth=2, zorder=3, alpha=0.85,\n",
+ " )\n",
+ " # Label each bubble with recommendation name\n",
+ " ax.annotate(\n",
+ " L['rec_display'][p_row['recommendation']],\n",
+ " (p_row['cost_score'], p_row['est_additional_completions']),\n",
+ " fontsize=9, fontweight='bold', ha='center', va='bottom',\n",
+ " xytext=(0, 12), textcoords='offset points',\n",
+ " )\n",
+ " # Annotate with impact number inside/below bubble\n",
+ " ax.annotate(\n",
+ " f\"~{p_row['est_additional_completions']:,}\",\n",
+ " (p_row['cost_score'], p_row['est_additional_completions']),\n",
+ " fontsize=8, ha='center', va='top',\n",
+ " xytext=(0, -10), textcoords='offset points', color='#555555',\n",
+ " )\n",
+ "\n",
+ " # Add subtle quadrant shading AFTER plotting so ylim reflects actual data\n",
+ " ax.axhspan(\n",
+ " ymin=0, ymax=ax.get_ylim()[1],\n",
+ " xmin=0, xmax=0.33, alpha=0.04, color='green',\n",
" )\n",
"\n",
- "# Add subtle quadrant shading AFTER plotting so ylim reflects actual data\n",
- "ax.axhspan(\n",
- " ymin=0, ymax=ax.get_ylim()[1],\n",
- " xmin=0, xmax=0.33, alpha=0.04, color='green',\n",
- ")\n",
- "\n",
- "ax.set_xticks([1, 2, 3])\n",
- "ax.set_xticklabels(['Low', 'Medium', 'Medium-High'])\n",
- "ax.set_xlabel('Implementation Cost')\n",
- "ax.set_ylabel('Estimated Additional Completions\\n(middle scenario)')\n",
- "ax.set_title(\n",
- " 'Priority Matrix: Impact vs Cost\\n'\n",
- " '(bubble size = segment size)'\n",
- ")\n",
- "ax.set_xlim(0.4, 3.6)\n",
- "# Ensure y-axis starts at 0 for honest visual comparison\n",
- "ax.set_ylim(bottom=0)\n",
- "sns.despine()\n",
- "fig.tight_layout()\n",
- "save_fig(fig, '07_priority_matrix')\n",
- "plt.show()"
+ " ax.set_xticks([1, 2, 3])\n",
+ " ax.set_xticklabels(L['cost_ticks'])\n",
+ " ax.set_xlabel(L['cost_xlabel'])\n",
+ " ax.set_ylabel(L['impact_ylabel'])\n",
+ " ax.set_title(L['matrix_title'])\n",
+ " ax.set_xlim(0.4, 3.6)\n",
+ " # Ensure y-axis starts at 0 for honest visual comparison\n",
+ " ax.set_ylim(bottom=0)\n",
+ " sns.despine()\n",
+ " fig.tight_layout()\n",
+ " save_fig_lang(fig, '07_priority_matrix', lang)"
]
},
{
"cell_type": "markdown",
- "id": "25",
+ "id": "27",
"metadata": {},
"source": [
"> **Reading the matrix:** The ideal intervention sits in the top-left corner: high impact, low cost. Ghost Student Activation is the clear \"quick win\": it targets the largest segment with the highest non-completion rate, and requires only email automation. The First Assessment Checkpoint offers solid impact at moderate cost. The Week 3 Re-engagement Campaign has the highest implementation complexity but targets a distinct population (not overlapping with ghosts), making it a valuable addition.\n",
@@ -1028,7 +1169,7 @@
},
{
"cell_type": "markdown",
- "id": "26",
+ "id": "28",
"metadata": {},
"source": [
"## 8. Implementation Roadmap\n",
@@ -1078,7 +1219,7 @@
},
{
"cell_type": "markdown",
- "id": "27",
+ "id": "29",
"metadata": {},
"source": [
"## 9. Limitations and Caveats\n",
@@ -1106,7 +1247,7 @@
},
{
"cell_type": "markdown",
- "id": "28",
+ "id": "30",
"metadata": {},
"source": [
"## 10. Key Takeaways\n",
@@ -1125,7 +1266,7 @@
"|----|-----|-------------|\n",
"| 03 | BQ1 | Dropout is not uniform: it concentrates in cliffs at specific course milestones. Pre-course withdrawal is a significant fraction. |\n",
"| 04 | BQ2 | Early behavioral signals (active days, clicks, assessment submission) predict dropout with large effect sizes. The first 28 days are the critical window. |\n",
- "| 05 | BQ3 | Behavior predicts outcome 2–5× more strongly than demographics. Interventions should target what students *do*, not who they *are*. |\n",
+ "| 05 | BQ3 | Behavior predicts outcome 2–4× more strongly than demographics. Interventions should target what students *do*, not who they *are*. |\n",
"| 06 | BQ4 | Completion rates vary substantially across courses. Course design features (assessment density, resource diversity) show suggestive associations with retention. |\n",
"| 07 | BQ5 | Three actionable interventions (ghost activation, assessment checkpoint, re-engagement campaign) target the largest at-risk segments with data-estimated impact. |\n",
"\n",
@@ -1140,7 +1281,7 @@
},
{
"cell_type": "markdown",
- "id": "29",
+ "id": "31",
"metadata": {},
"source": [
"> **From analysis to action:** This project started with a dataset and five questions. Seven notebooks later, we have a complete picture: when students leave, what predicts it, what matters more (behavior), how courses differ, and, most importantly, what a platform operator can do about it. The three interventions proposed here are not speculative: they are sized by real data, supported by statistical evidence, and ranked by feasibility.\n",
@@ -1156,7 +1297,15 @@
"name": "python3"
},
"language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
"name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
"version": "3.13.0"
}
},
diff --git a/reports/REPORT.md b/reports/REPORT.md
index 05fee89..0af6cc8 100644
--- a/reports/REPORT.md
+++ b/reports/REPORT.md
@@ -2,8 +2,8 @@
> **Data-driven analysis of student retention and drop-out in online education**
-> **Data**: Open University Learning Analytics Dataset (OULAD), 32,593 enrollments,
-> 7 courses. Historical dataset (2013–2014 cohorts), stable: no updates expected.
+> **Data**: [Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset),
+> 32,593 enrollments across 7 courses. Historical dataset (2013–2014 cohorts), stable: no updates expected.
> **Author**: [Alessandro Attene](https://www.linkedin.com/in/aleattene)
@@ -11,18 +11,18 @@
> **Last revised**: September 2026
-> **Audience**: Head of Product | Observational analysis (associations, not causal
-> claims), no machine learning models
-
---
+
+
## Executive Summary
Roughly one in three enrollments ends in explicit withdrawal, and dropout is not
-random: it clusters around course milestones. Behavioral signals from the first
-28 days predict the outcome far more strongly than any demographic variable, and
-they make it possible to identify at-risk students early with interventions that
-require no demographic profiling.
+random: it clusters around course milestones (assessment deadlines and grade releases).
+
+Behavioral signals from the first 28 days predict the outcome far more strongly than any
+demographic variable, and they make it possible to identify at-risk students early with
+interventions that require no demographic profiling.
### The Five Key Numbers
@@ -37,19 +37,24 @@ require no demographic profiling.
\* Both effect size measures (Cohen's d and Cramer's V) are explained, with scale
and a worked example, in the [Methodology](#methodology) section.
-**Recommended actions** (details in BQ5): ghost-student activation by day 3, a
-checkpoint before the first assessment deadline, and week-3 re-engagement.
+**Recommended actions** (details in BQ5):
+- ghost-student activation by day 3
+- a checkpoint before the first assessment deadline
+- week-3 re-engagement
---
+
+
## Methodology
This report synthesizes findings from a SQL-driven analytical pipeline applied to the
-OULAD dataset: 32,593 student-course enrollments across 7 modules, with complete
-behavioral clickstream from the university's Virtual Learning Environment (VLE),
-assessment records, and demographic profiles.
+OULAD dataset: 32,593 student-course enrollments across 7 modules, with:
+- complete behavioral clickstream from the university's Virtual Learning Environment (VLE)
+- assessment records
+- demographic profiles.
-**Outcome definition:** Each enrollment is classified into one of two classes:
+**Outcome definition.** Each enrollment is classified into one of two classes:
- **Completed**: final result Pass or Distinction
- **Not completed**: final result Fail or Withdrawn
@@ -61,12 +66,17 @@ retention analysis.
| Method | Used for | Reported metrics |
|--------|----------|------------------|
-| Welch's t-test | Continuous signals vs. outcome | t-statistic, p-value, Cohen's d |
+| Welch's t-test\* | Continuous signals vs. outcome | t-statistic, p-value, Cohen's d |
| Chi-square test | Categorical demographics vs. outcome | chi-square, p-value, Cramer's V |
-| Bonferroni + Benjamini-Hochberg | Multiple comparison correction | Adjusted p-values |
-| Bootstrap CI\* | Extreme-rate groups (e.g., ghost students) | 95% confidence intervals |
+| [Bonferroni](https://en.wikipedia.org/wiki/Bonferroni_correction) and [Benjamini-Hochberg](https://en.wikipedia.org/wiki/False_discovery_rate) | Multiple comparison correction | Adjusted p-values |
+| Bootstrap CI\*\* | Extreme-rate groups (e.g., ghost students) | 95% confidence intervals |
+
+\* The t-test compares the means of two groups and asks whether the observed difference could be explained by
+pure chance. Welch's variant does not assume the two groups share the same variability: it is the safer choice
+when the compared groups differ substantially in size, as they do here (completers vs. non-completers).
+More: [Welch's t-test](https://en.wikipedia.org/wiki/Welch%27s_t-test) (Wikipedia).
-\* CI = Confidence Interval: the range of values within which, with 95% confidence,
+\*\* CI = Confidence Interval: the range of values within which, with 95% confidence,
the true value lies. Here it is estimated via bootstrap, that is by resampling the
observed data many times.
@@ -88,26 +98,31 @@ units of typical variability (standard deviations). Reference scale:
- d ≈ 0.5: medium effect
- d ≥ 0.8: large effect
-*Worked example with the project's real data:* in the first 28 days, students who
-eventually complete the course are active on average 12.8 days; those who do not,
-6.5. The difference (6.33 days), divided by the pooled standard deviation of the
-two groups (about 7.05 days), gives d = 6.33 / 7.05 ≈ 0.90: a large effect.
+**Worked example with the project's real data.**
+*In the first 28 days, students who will eventually complete the course are active on
+average 12.8 days; those who will not, 6.5. The difference (6.33 days), divided by the
+pooled standard deviation of the two groups (about 7.05 days), gives
+d = 6.33 / 7.05 ≈ 0.90: a large effect.*
**Cramer's V** (for categorical variables): measures the strength of association
-between two variables on a scale from 0 (no association) to 1 (perfect
-association). With a binary outcome, values around 0.1 indicate a weak
-association, around 0.3 medium, 0.5 and above strong.
+between two variables on a scale from 0 to 1:
+- **0**: **no** association
+- **1**: **perfect** association
+
+With a binary outcome, values around 0.1 indicate a **weak** association, around 0.3 **medium**, 0.5 and above **strong**.
-*Worked example with the project's real data:* for education level the chi-square
-test yields 737.2 on 32,593 enrollments; with a binary outcome the formula reduces
-to V = square root of (737.2 / 32,593) ≈ 0.15: a weak association.
+**Worked example with the project's real data.**
+*For education level the chi-square test yields 737.2 on 32,593 enrollments.
+With a binary outcome the formula reduces to V = square root of (737.2 / 32,593) ≈ 0.15: a **weak** association.*
---
+
+
## BQ1: Where and When Do Students Drop Out?
-> **Key finding:** Approximately one in three enrollments ends in explicit withdrawal.
-> Dropout is not random: it clusters around course milestones, and its temporal
+> **Key finding:** approximately **one in three enrollments** ends in **explicit withdrawal**.
+> Dropout is not random: it clusters around specific course milestones, and its temporal
> profile differs across modules.
Across the 7 OULAD modules, withdrawal rates range from **11.8%** (module GGG) to
@@ -115,57 +130,83 @@ Across the 7 OULAD modules, withdrawal rates range from **11.8%** (module GGG) t
of all enrollments: a substantial share of the student population that never reaches
completion.
-Cumulative dropout curves reveal **distinct temporal profiles** per course. Some modules
-experience steep early attrition (an onboarding failure pattern), while others show more
-gradual mid-course decline. Within the same module, different presentations (cohorts)
-follow broadly similar trajectories, suggesting that course design, not random cohort
-variation, drives the dropout shape.
+Cumulative dropout curves reveal **distinct temporal profiles** per course.
+Some modules lose many students within the very first weeks (the typical signature of a
+failing onboarding), while others show a more gradual mid-course decline.
+Within the same module, different presentations (cohorts) follow broadly similar
+trajectories, suggesting that course design, not random cohort variation, drives the
+dropout shape.
+
+A note for reading the chart: the horizontal axis starts at negative values because
+enrollment opens well ahead of the actual course start (day 0). A withdrawal in the
+negative range of the axis therefore means the student unenrolled before the course even
+began: this phenomenon is analyzed a little further down in this section.
*Cumulative dropout curves show distinct temporal profiles per course. Each line
represents one course-presentation, colored by module.*
-**Cliff events**, days with disproportionately large numbers of withdrawals (above the
-95th percentile for that course), align with assessment deadlines and grade releases.
-These are actionable: interventions can be timed to precede known cliff dates.
+The second pattern concerns **cliff events\***: days when withdrawals do not grow
+gradually but spike all at once, as if students fell off a step together (hence the
+name, from cliff). These spikes do not land on random days: they coincide with
+assessment deadlines and grade releases.
+
+\* Cliff event: a day with a disproportionately high number of withdrawals compared to
+the rest of the course, above the 95th percentile (that is, with more withdrawals than
+95% of the other days of that course).
-*Cliff events detected via p95 threshold. The largest single-day dropout spikes
-correspond to course milestones.*
+*Cliff events detected via p95 threshold. As can easily be seen, the largest single-day
+dropout spikes correspond to course milestones: assessment deadlines and grade releases.*
+
+For whoever runs the platform this regularity is good news: if the critical days are
+predictable, action can be taken in advance. A reminder, or an offer of help, sent a few
+days before a deadline lands exactly when the risk of dropping out peaks.
More than a quarter of explicit withdrawals (26.6%, 2,678 of 10,072) occur **before
the course even starts** (dropout day < 0). These pre-course withdrawals represent
-pure registration churn: students who enrolled but never experienced any content. This is an activation problem, not an
-academic one.
+pure registration churn: students who enrolled but never experienced any content.
+This is an **activation problem**, not an academic one.
-*Pre-course withdrawals by module. These students need onboarding nudges, not
-academic support.*
+*Pre-course withdrawals by module. These students do not need academic support: they
+need a gentle welcome nudge that walks them to their first login.*
-Knowing *when* students leave raises the next question: can we see it coming?
+Knowing **when** students leave raises the next question: **can we see it coming**?
---
+
+
## BQ2: Which Early Signals Predict Dropout?
-> **Key finding:** All 8 early engagement metrics tested are significantly associated
+> **Key finding:** all 8 early engagement metrics tested are significantly associated
> with dropout after multiple comparison correction (8/8 after both Bonferroni and
> Benjamini-Hochberg). The strongest predictors are engagement-volume metrics:
> within-course engagement decile, active days, and total clicks in the first 28 days.
-
-Using only the first 28 days of enrollment data, we tested 8 behavioral signals for
-their association with eventual completion. Effect size (Cohen's d), not p-value, is
-the primary ranking criterion, because with ~32K observations significance is easy to
-achieve.
-
-The **forest plot** below ranks all signals by absolute effect size. Engagement-volume
-metrics dominate the ranking: within-course engagement decile (d = 0.97), active days
-(d = 0.90), and total clicks (d = 0.63), followed by last active day, first assessment
-score, and average click intensity (d between 0.52 and 0.55; assessment-based signals
-are computed on the submitter subpopulation only).
+> In plain terms: how much, and how often, a student uses the platform in the first
+> four weeks already says a great deal about how the course will end.
+
+Using only the first 28 days of enrollment data, we tested the association between
+**8 behavioral signals** and eventual course completion.
+Effect size (Cohen's d), not p-value, is the **primary ranking criterion**, because with
+~32K observations significance is easy to achieve.
+
+The **forest plot** below ranks all signals by **absolute effect size**.
+Engagement-volume metrics dominate the ranking:
+- within-course engagement decile (d = 0.97), that is the student's position in their
+course's engagement ranking, split into ten bands
+- active days (d = 0.90)
+- total clicks (d = 0.63)
+
+Next, with medium effects (d between 0.52 and 0.55), come last active day, first
+assessment score and average click intensity; first submission day and registration day
+close the ranking.
+Assessment-based signals are computed on the subpopulation of students who submitted at
+least one assessment (the submitters).
@@ -173,12 +214,15 @@ are computed on the submitter subpopulation only).
Benjamini-Hochberg correction. Vertical reference lines mark small, medium, and
large effect thresholds.*
-The most dramatic contrast is between **ghost students** (those with zero VLE activity
-in the first 28 days) and active students. Ghost students have a near-zero completion
-rate, while active students complete at a rate close to the platform average. The 95%
-bootstrap confidence intervals do not overlap. (Note: BQ5 broadens this definition to
-include near-zero activity, that is at most 1 active day and fewer than 10 clicks, to
-capture the full at-risk segment for intervention targeting.)
+The starkest contrast is between **ghost students** (those with zero VLE activity
+in the first 28 days) and active students:
+- ghost students have a near-zero completion rate
+- active students complete at a rate close to the platform average.
+
+The 95% bootstrap confidence intervals do not overlap.
+(Note: BQ5 broadens this definition to include near-zero activity, that is at most 1
+active day and fewer than 10 clicks, to capture the full at-risk segment when selecting
+the recipients of the interventions.)
@@ -187,39 +231,60 @@ rates. Error bars show 95% bootstrap confidence intervals.*
The dose-response relationship is **monotonic**: more engagement consistently predicts
higher completion, with no threshold or diminishing returns. This means the signal is
-useful across its entire range, not just at extremes.
+useful across its entire range of values, not just at extremes.
*Completion rate by signal quartile for the top 3 predictors. The relationship is
graded, not binary.*
-Two additional insights strengthen the signal portfolio. **Assessment submission** is a
-powerful binary predictor: students who submitted at least one assessment in the first
-28 days complete at substantially higher rates than non-submitters. And **consistency
-beats intensity**: regular daily logins predict completion more strongly than high
-click-per-session bursts.
+Two additional insights strengthen the signal portfolio:
+
+- **assessment submission** is a powerful binary predictor: students who submitted at
+least one assessment in the first 28 days complete at substantially higher rates than
+those who submitted nothing
+
-These behavioral signals are strong. But are they merely proxies for demographics?
+- **consistency beats intensity**: regular daily logins predict completion more
+strongly than a few concentrated sessions with very many clicks
+
+It is true that these behavioral signals are strong. But are they merely a reflection of demographics?
---
+
+
## BQ3: What Matters More, Demographics or Behavior?
-> **Key finding:** Behavior dominates. Behavioral effect sizes are multiple times larger
-> than demographic effect sizes. Within every education level, high engagement
-> dramatically outperforms low engagement.
+> **Key finding:** **behavior dominates**. Behavioral effect sizes are multiple times
+> larger than demographic effect sizes. Within every education level, high engagement
+> clearly outperforms low engagement.
+
+Against the final course outcome (completed or not completed), we tested:
+- **6 categorical demographic variables**:
+ - gender
+ - age band
+ - education level
+ - Index of Multiple Deprivation (IMD) band
+ - disability
+ - region
+
+
+- **2 numeric demographic variables**:
+ - previous attempts
+ - studied credits
+
+The final result is that **all 8** are **statistically significant** after
+**Benjamini-Hochberg** correction ([more on Wikipedia](https://en.wikipedia.org/wiki/False_discovery_rate)),
+but their **effect sizes** are **uniformly weak**.
+The strongest demographic predictor (highest education level) reaches a Cramer's V of approximately **0.15**.
+The IMD band follows at **0.13**, and all other demographic variables stay below **0.09**.
-We tested 6 categorical demographic features (gender, age band, education level,
-Index of Multiple Deprivation (IMD) band, disability, region) and 2 numeric demographic features (previous attempts, studied
-credits) against completion outcome. All 8 are statistically significant after
-Benjamini-Hochberg correction, but their effect sizes are uniformly weak. The strongest
-demographic predictor (highest education) reaches a Cramer's V of approximately **0.15**;
-the IMD band follows at **0.13**, and all other demographic variables fall below **0.09**.
+By contrast, **behavioral variables** (active days, total clicks, assessment
+submission, click intensity) show **effect sizes** several times **larger**.
-By contrast, behavioral features (active days, total clicks, assessment submission,
-click intensity) show effect sizes several times larger. The gap is stark: behavioral
-signals predict outcome far more strongly than any demographic variable.
+The gap is stark: **behavioral signals predict outcome far more strongly than any
+demographic variable**.
@@ -227,7 +292,7 @@ signals predict outcome far more strongly than any demographic variable.
substantial: behavioral signals are consistently stronger.*
The critical test: does engagement merely reflect demographics? The interaction plot
-below shows that within **every education level**, high-engagement students dramatically
+below shows that within **every education level**, high-engagement students clearly
outperform low-engagement students. A student with lower formal education but high
engagement has a better chance of completing than a highly educated student who does not
engage with the platform.
@@ -235,61 +300,79 @@ engage with the platform.
*Within every education level, the engagement gap dwarfs the education gap.
-Behavior is the swing factor, not background.*
+**Behavior** is the **swing factor**, not background.*
-This finding has an **ethical dimension**: behavioral signals are both statistically
-stronger *and* actionable. Demographics cannot be changed; behavior can be influenced
-through platform design. Targeting behavior avoids the fairness concerns inherent in
+This finding also has an **ethical dimension**: behavioral signals are not only the
+statistically stronger ones, they are also the ones the platform can actually act on.
+A student's demographics cannot be changed; their behavior can, through platform design.
+Focusing interventions on behavior also avoids the fairness concerns inherent in
demographic profiling.
-Does course design itself influence engagement levels?
+In light of this, the next question comes naturally: *is it course design itself that
+influences engagement levels?*
---
+
+
## BQ4: How Do Course Characteristics Affect Retention?
-> **Key finding:** Completion rates vary substantially across the 7 modules: from
-> **37.4%** (CCC) to **70.9%** (AAA), a **33.5 percentage point** gap. Suggestive
-> patterns emerge around assessment density and course length, but with only 7 data
-> points no inferential conclusions are possible.
+> **Key finding:** completion rates vary substantially across the 7 modules, going from
+> **37.4%** (CCC) to **70.9%** (AAA), a **33.5 percentage point** gap.
+> Suggestive patterns emerge around assessment density and course length, but with only
+> 7 courses (7 data points) no inferential conclusions are possible.
-The ranking chart below shows the full spread. Module AAA retains nearly three-quarters
-of its students; module CCC loses almost two-thirds.
+The chart below shows the full ranking. Module AAA retains nearly three-quarters
+of its students, while module CCC loses almost two-thirds.
*Completion rates vary from 37.4% to 70.9% across the 7 OULAD modules.*
Exploratory scatter plots reveal suggestive patterns between course design features
-(assessment density, course length) and completion rates. However, with n = 7, any
-correlation is descriptive, not inferential: Spearman rank correlation requires
-`|rho| > 0.79` for significance at this sample size.
+(assessment density, course length) and completion rates. However, as noted, with n = 7
+any correlation is descriptive, not inferential: Spearman's correlation\* requires
+|rho| > 0.79 for significance with a sample this small.
+
+\* Spearman's correlation measures how much two quantities move together by looking at
+the order of the values (their rankings) instead of the exact values: rho = 1 when one
+always grows as the other grows, rho = 0 when there is no relationship, rho = -1 when
+the relationship is perfectly inverse. With only 7 courses, only a near-perfect
+relationship (|rho| > 0.79) can be distinguished from pure chance. More:
+[Spearman's rank correlation coefficient](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient)
+(Wikipedia).
*Assessment density and course length show suggestive associations with completion.
Each point is one module (averaged across its presentations).*
-**Critical caveats:** These patterns are confounded by at least three factors: (1) subject
-difficulty, since some modules teach inherently harder material; (2) student
-self-selection, since more motivated students may choose certain courses; (3)
-institutional investment, since resource allocation varies across departments. Course design is a lever worth studying,
-but it requires more data (more courses, or experimental variation) to draw conclusions.
+**Critical caveats.** These patterns are entangled with at least three factors the data cannot separate:
+- subject difficulty, since some modules teach inherently harder material
+- student self-selection, since more motivated students may choose certain courses
+- institutional investment, since resource allocation varies across departments
+
+Course design is therefore a lever worth studying, but it requires more data (more
+courses, or experimental variation) to draw conclusions that genuinely support decisions.
-Drawing from all four analyses above, we now propose three concrete interventions.
+Drawing on all four analyses above, **we now propose three concrete interventions**.
---
+
+
## BQ5: Top 3 Recommended Interventions
-> **Key finding:** Three behavior-based interventions, ordered by impact-to-cost ratio,
-> together address the majority of at-risk students. Because segments overlap
-> significantly, a sequenced rollout avoids redundant outreach.
+> **Key finding:** three behavior-based interventions, ordered by impact-to-cost ratio,
+> together address the majority of at-risk students.
+> Because the segments overlap significantly, a sequenced rollout of the interventions
+> avoids contacting the same students repeatedly with redundant messages.
### Target Segments
-The BQ5 query sizes three student segments defined by observable, actionable criteria,
-not demographics. All definitions use first-28-day behavioral data.
+The BQ5 query sizes three student segments defined by criteria that are observable and
+that the platform can act on directly, not demographic ones.
+All definitions use first-28-day behavioral data.
| Segment | Definition | Size | Non-completion rate |
|---------|-----------|------|---------------------|
@@ -297,41 +380,54 @@ not demographics. All definitions use first-28-day behavioral data.
| **Assessment non-submitters** | No assessment submitted in first 28 days | **11,494** (35.3%) | **71.8%** |
| **Early disengagers** | Activity in days 0–14, zero in days 15–28 | **2,213** (6.8%) | **77.8%** |
-All three segments show non-completion rates far above the platform baseline (~53%).
+A note on the metric: the table reports the **non-completion** rate (fails and
+withdrawals together), not the withdrawal rate used in BQ1. For intervention design both
+negative outcomes matter: a student who reaches the end of the course and fails is still
+a student the platform did not manage to carry across the finish line.
+
+All three segments show non-completion rates far above the platform baseline (~53%).
Ghost students complete at just 7.7%, against a platform average of 47.2%.
+
+
+*Size of the three target segments and their non-completion rates, compared with the
+platform-wide value.*
+
### The Three Interventions
-| | Ghost Activation | Assessment Checkpoint | Week 3 Re-engagement |
-|---|---|---|---|
-| **Priority** | 1: Quick win | 2: Build next | 3: Invest when ready |
-| **Trigger** | Zero VLE\* activity by day 3 | 3 days before first deadline, not submitted | 3+ consecutive inactive days after initial activity |
-| **Action** | Email sequence: day-3 welcome + day-7 follow-up with first-step link | Reminder with assessment preview and time estimate | "We miss you" email with progress summary and peer comparison |
-| **Cost** | **Low** (email automation only) | **Medium** (deadline-aware triggers + course calendar) | **Medium-High** (real-time activity tracking + personalization) |
-| **Evidence** | BQ2: early engagement is strongest predictor; BQ3: behavior > demographics | BQ2: assessment submission is a key binary signal; BQ1: cliffs at deadlines | BQ1: mid-course dropout cliffs at weeks 3–4; BQ2: last-active-day predictor |
-| **Impact estimate** | Largest: widest gap between segment and platform rate | Medium: substantial submitter vs non-submitter gap | Medium: targets distinct failure mode from ghosts |
-
-\* VLE = Virtual Learning Environment, the online platform where the course
-content lives.
-
-**Impact estimation approach:** For each intervention, we model conservative conversion
-scenarios (10–25% of targeted students change behavior). Converted ghost students are
-assumed to achieve the platform-average completion rate, not the active-student rate.
-Re-engaged students are assumed to reach a rate halfway between disengaged and sustained.
+| | Ghost Activation | Assessment Checkpoint | Week 3 Re-engagement |
+|---|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|
+| **Priority** | 1: Quick win | 2: Build next | 3: Invest when ready |
+| **Trigger** | Zero VLE\* activity by day 3 | 3 days before first deadline, not submitted | 3+ consecutive inactive days after initial activity |
+| **Action** | Email sequence: day-3 welcome and day-7 follow-up with first-step link | Reminder with assessment preview and time estimate | "We miss you" email with progress summary and peer comparison |
+| **Cost** | **Low** (email automation only) | **Medium** (deadline-aware triggers and course calendar) | **Medium-High** (real-time activity tracking and personalization) |
+| **Evidence** | BQ2: early engagement is strongest predictor; BQ3: behavior > demographics | BQ2: submission is a key binary signal; BQ1: cliffs at deadlines | BQ1: mid-course dropout cliffs at weeks 3-4; BQ2: last-active-day predictor |
+| **Impact estimate** | Largest: widest gap between segment and platform rate | Medium: substantial submitter vs non-submitter gap | Medium: intercepts a different dropout mode than ghosts |
+
+\* VLE = Virtual Learning Environment, the online platform where the course content lives.
+
+**Impact estimation approach:** for each intervention we model conservative conversion
+scenarios, assuming that only 10–25% of the students reached change their behavior.
+We further assume that converted ghost students can reach the platform-average
+completion rate (not the active-student rate) and that re-engaged students settle at a
+rate halfway between disengaged and sustained.
These are deliberately conservative assumptions.
### Segment Overlap
Ghost students and assessment non-submitters **overlap heavily**: a student with zero
-VLE access cannot submit an assessment. This means interventions 1 and 2 largely target
-the same population from different angles; their impact should not be summed naively.
-Early disengagers, by definition, had initial activity: they overlap less with ghosts,
-making intervention 3 an independent lever targeting a different failure mode.
+VLE access cannot submit an assessment. This means interventions 1 and 2 largely reach
+the same population from different angles. Their impact should therefore not be summed
+naively.
+Early disengagers, by definition, did have initial activity: they overlap less with
+ghosts, which makes the third intervention ([Week 3 Re-engagement](#the-three-interventions))
+an independent lever that intercepts a different dropout mode.
-*Impact-vs-cost priority matrix. Ghost Activation is the clear quick win:
-largest segment, highest excess non-completion, lowest cost.*
+*Impact-to-cost priority matrix. Ghost Activation is the clear quick win (the largest
+result for the smallest effort): largest segment, highest excess non-completion and
+lowest cost.*
@@ -340,40 +436,53 @@ The ghost–non-submitter overlap is substantial.*
---
+
+
## Limitations and Caveats
- **Observational data only.** All effect sizes and completion rate differences are
- associations, not causal relationships. Engaged students may be inherently more
- motivated: engagement could be a proxy, not a cause.
-- **Historical data.** OULAD covers 2013–2014 cohorts at the UK Open University.
- Student behavior and online learning platforms have evolved significantly since then.
+associations, not causal relationships. Engaged students may be inherently more
+motivated: engagement could be a proxy (an indicator reflecting something else, such as
+motivation), not a cause.
+
+
+- **Historical data.** OULAD covers 2013–2014 cohorts at the UK Open University. Student
+behavior and online learning platforms have changed significantly since then.
+
+
- **BQ4 limited by n = 7.** With only 7 modules, no inferential statistics are possible
- for course-level analysis. Design feature patterns are hypotheses, not conclusions.
+for course-level analysis. Design feature patterns are hypotheses, not conclusions.
+
+
- **Impact estimates are assumptions.** Conversion rates (10–25%) are plausible
- projections based on industry benchmarks, not measured outcomes. No A/B testing data
- exists in the dataset.
+projections based on industry benchmarks, not measured outcomes. No A/B testing data
+exists in the dataset.
+
+
- **No cost data.** Implementation cost estimates (Low / Medium / Medium-High) are
- qualitative. Actual engineering effort depends on existing platform infrastructure.
-- **Ethical note.** All interventions target behavior, not demographics. Automated
- outreach should include opt-out mechanisms to respect student autonomy.
+qualitative. Actual engineering effort depends on existing platform infrastructure.
+
+
+- **Ethical note.** All interventions act on behavior, not demographics. Automated
+communications to students should always include an opt-out mechanism, to respect
+student autonomy.
---
-## Chart Provenance
+
+
+## Appendix: Provenance of Charts and Numbers
All figures in this report are generated by the 7 analysis notebooks in
[`notebooks/`](../notebooks/): the numeric prefix of each image file matches the
notebook that produces it (for example `03_dropout_curves_overlaid.png` comes from
-`03_bq1_dropout_timing.ipynb`). The notebooks read the CSV files exported by the
-pipeline into `data/analysis/` (local folder, not versioned) and save the charts
-into `reports/figures/`.
-
-To regenerate the figures from a repository clone:
-
-1. run the pipeline: `python -m run_pipeline`
-2. run the notebooks in order (01 through 07): each notebook re-exports its own figures
-
-Full setup instructions are in the [README](../README.md), Quick Start section.
-The numbers quoted in the text are verified against the same pipeline-exported
-CSVs, in particular the statistical exports (`stats_*.csv`) for effect sizes and
-confidence intervals.
+`03_bq1_dropout_timing.ipynb`).
+The notebooks read the CSVs exported by the pipeline and save each figure in two
+languages under the same file name: English in `reports/figures/` (used by the EN
+documents), Italian in `reports/figures/it/` (used by the Italian report and README).
+
+The numbers quoted in the text are verified against the same pipeline-exported CSVs, in
+particular the statistical exports (`stats_*.csv`) for effect sizes and confidence
+intervals.
+Instructions to regenerate figures and data from a repository clone are in the
+[README](../README.md), Getting Started section.
diff --git a/reports/figures/03_course_design_vs_dropout.png b/reports/figures/03_course_design_vs_dropout.png
index 9b7ad5d..eb6629d 100644
Binary files a/reports/figures/03_course_design_vs_dropout.png and b/reports/figures/03_course_design_vs_dropout.png differ
diff --git a/reports/figures/03_dropout_by_demographics.png b/reports/figures/03_dropout_by_demographics.png
index bad2ca9..d8966bf 100644
Binary files a/reports/figures/03_dropout_by_demographics.png and b/reports/figures/03_dropout_by_demographics.png differ
diff --git a/reports/figures/04_assessment_signal.png b/reports/figures/04_assessment_signal.png
index 9cc3b14..939b130 100644
Binary files a/reports/figures/04_assessment_signal.png and b/reports/figures/04_assessment_signal.png differ
diff --git a/reports/figures/04_correction_comparison.png b/reports/figures/04_correction_comparison.png
index 45e71e2..fe05796 100644
Binary files a/reports/figures/04_correction_comparison.png and b/reports/figures/04_correction_comparison.png differ
diff --git a/reports/figures/04_forest_plot_effect_sizes.png b/reports/figures/04_forest_plot_effect_sizes.png
index cc8b3e6..47bf14e 100644
Binary files a/reports/figures/04_forest_plot_effect_sizes.png and b/reports/figures/04_forest_plot_effect_sizes.png differ
diff --git a/reports/figures/04_ghost_vs_active_completion.png b/reports/figures/04_ghost_vs_active_completion.png
index c186ea3..e0f124d 100644
Binary files a/reports/figures/04_ghost_vs_active_completion.png and b/reports/figures/04_ghost_vs_active_completion.png differ
diff --git a/reports/figures/04_signals_violins.png b/reports/figures/04_signals_violins.png
index c5c56af..f0d0ad8 100644
Binary files a/reports/figures/04_signals_violins.png and b/reports/figures/04_signals_violins.png differ
diff --git a/reports/figures/04_top_signal_dose_response.png b/reports/figures/04_top_signal_dose_response.png
index 32328c2..0b882ea 100644
Binary files a/reports/figures/04_top_signal_dose_response.png and b/reports/figures/04_top_signal_dose_response.png differ
diff --git a/reports/figures/05_behavior_effect_sizes.png b/reports/figures/05_behavior_effect_sizes.png
index 897526e..41b9b28 100644
Binary files a/reports/figures/05_behavior_effect_sizes.png and b/reports/figures/05_behavior_effect_sizes.png differ
diff --git a/reports/figures/05_demographic_effect_sizes.png b/reports/figures/05_demographic_effect_sizes.png
index 40dd337..84b0161 100644
Binary files a/reports/figures/05_demographic_effect_sizes.png and b/reports/figures/05_demographic_effect_sizes.png differ
diff --git a/reports/figures/05_demographics_vs_behavior_comparison.png b/reports/figures/05_demographics_vs_behavior_comparison.png
index 18872ad..d36406f 100644
Binary files a/reports/figures/05_demographics_vs_behavior_comparison.png and b/reports/figures/05_demographics_vs_behavior_comparison.png differ
diff --git a/reports/figures/it/03_dropout_cliffs.png b/reports/figures/it/03_dropout_cliffs.png
new file mode 100644
index 0000000..d20d385
Binary files /dev/null and b/reports/figures/it/03_dropout_cliffs.png differ
diff --git a/reports/figures/it/03_dropout_curves_overlaid.png b/reports/figures/it/03_dropout_curves_overlaid.png
new file mode 100644
index 0000000..7bba01c
Binary files /dev/null and b/reports/figures/it/03_dropout_curves_overlaid.png differ
diff --git a/reports/figures/it/03_precourse_withdrawals.png b/reports/figures/it/03_precourse_withdrawals.png
new file mode 100644
index 0000000..a58efcb
Binary files /dev/null and b/reports/figures/it/03_precourse_withdrawals.png differ
diff --git a/reports/figures/it/04_forest_plot_effect_sizes.png b/reports/figures/it/04_forest_plot_effect_sizes.png
new file mode 100644
index 0000000..9cf1edf
Binary files /dev/null and b/reports/figures/it/04_forest_plot_effect_sizes.png differ
diff --git a/reports/figures/it/04_ghost_vs_active_completion.png b/reports/figures/it/04_ghost_vs_active_completion.png
new file mode 100644
index 0000000..c93b54b
Binary files /dev/null and b/reports/figures/it/04_ghost_vs_active_completion.png differ
diff --git a/reports/figures/it/04_top_signal_dose_response.png b/reports/figures/it/04_top_signal_dose_response.png
new file mode 100644
index 0000000..f9e9992
Binary files /dev/null and b/reports/figures/it/04_top_signal_dose_response.png differ
diff --git a/reports/figures/it/05_demographics_vs_behavior_comparison.png b/reports/figures/it/05_demographics_vs_behavior_comparison.png
new file mode 100644
index 0000000..396ce2b
Binary files /dev/null and b/reports/figures/it/05_demographics_vs_behavior_comparison.png differ
diff --git a/reports/figures/it/05_education_engagement_interaction.png b/reports/figures/it/05_education_engagement_interaction.png
new file mode 100644
index 0000000..055e863
Binary files /dev/null and b/reports/figures/it/05_education_engagement_interaction.png differ
diff --git a/reports/figures/it/06_course_completion_ranking.png b/reports/figures/it/06_course_completion_ranking.png
new file mode 100644
index 0000000..bfec2d7
Binary files /dev/null and b/reports/figures/it/06_course_completion_ranking.png differ
diff --git a/reports/figures/it/06_course_design_vs_completion.png b/reports/figures/it/06_course_design_vs_completion.png
new file mode 100644
index 0000000..08dde5a
Binary files /dev/null and b/reports/figures/it/06_course_design_vs_completion.png differ
diff --git a/reports/figures/it/07_priority_matrix.png b/reports/figures/it/07_priority_matrix.png
new file mode 100644
index 0000000..41f1989
Binary files /dev/null and b/reports/figures/it/07_priority_matrix.png differ
diff --git a/reports/figures/it/07_segment_overlap.png b/reports/figures/it/07_segment_overlap.png
new file mode 100644
index 0000000..8e8a2a5
Binary files /dev/null and b/reports/figures/it/07_segment_overlap.png differ
diff --git a/reports/figures/it/07_segment_sizing_overview.png b/reports/figures/it/07_segment_sizing_overview.png
new file mode 100644
index 0000000..d0bd4e0
Binary files /dev/null and b/reports/figures/it/07_segment_sizing_overview.png differ
diff --git a/reports/it/REPORT.md b/reports/it/REPORT.md
index 556be68..c6187a7 100644
--- a/reports/it/REPORT.md
+++ b/reports/it/REPORT.md
@@ -1,9 +1,9 @@
# Analisi della Retention Studentesca: Report Esecutivo 
-> **Analisi data-driven della retention e dell'abbandono degli studenti nella formazione online**
+> Analisi **data-driven** della **retention** e dell'abbandono degli studenti nella **formazione online**
-> **Dati**: Open University Learning Analytics Dataset (OULAD), 32.593 iscrizioni,
-> 7 corsi. Dataset storico (coorti 2013–2014), stabile: nessun aggiornamento previsto.
+> **Dati**: [Open University Learning Analytics Dataset (OULAD)](https://analyse.kmi.open.ac.uk/open_dataset),
+> 32.593 iscrizioni su 7 corsi. Dataset storico (coorti 2013–2014), stabile: nessun aggiornamento previsto.
> **Autore**: [Alessandro Attene](https://www.linkedin.com/in/aleattene)
@@ -11,18 +11,18 @@
> **Ultima revisione**: settembre 2026
-> **Destinatario**: Head of Product | Analisi osservazionale (associazioni, non
-> relazioni causali), nessun modello di machine learning
-
---
+
+
## Sintesi esecutiva
-Circa un'iscrizione su tre termina con il ritiro esplicito, e l'abbandono non è
-casuale: si concentra intorno alle tappe del corso. I segnali comportamentali dei
-primi 28 giorni predicono l'esito molto più fortemente di qualsiasi variabile
-demografica, e permettono di individuare presto gli studenti a rischio con
-interventi che non richiedono alcuna profilazione demografica.
+Circa un'iscrizione su tre termina con il ritiro esplicito e l'abbandono non risulta essere casuale: si concentra
+infatti intorno alle tappe del corso (scadenze delle valutazioni e rilascio dei voti).
+
+I segnali comportamentali dei primi 28 giorni predicono l'esito molto più fortemente di qualsiasi variabile
+demografica e permettono inoltre di individuare presto gli studenti a rischio, con interventi che non richiedono
+alcuna profilazione demografica.
### I cinque numeri chiave
@@ -37,273 +37,336 @@ interventi che non richiedono alcuna profilazione demografica.
\* Le due misure di effect size (d di Cohen e V di Cramér) sono spiegate, con scala
ed esempio svolto, nella sezione [Metodologia](#metodologia).
-**Azioni raccomandate** (dettagli in BQ5): attivazione degli studenti ghost entro il
-giorno 3, checkpoint prima della prima scadenza di valutazione, re-engagement alla
-settimana 3.
+**Azioni raccomandate** (dettagli in BQ5):
+- attivazione degli studenti ghost entro il giorno 3
+- checkpoint prima della prima scadenza di valutazione
+- re-engagement alla settimana 3
---
+
+
## Metodologia
-Questo report sintetizza i risultati di una pipeline analitica SQL-driven applicata al
-dataset OULAD: 32.593 iscrizioni studente-corso distribuite su 7 moduli, con clickstream
-comportamentale completo dal Virtual Learning Environment (VLE) dell'università, record
-delle valutazioni e profili demografici.
+Questo report sintetizza i risultati di una pipeline analitica SQL-driven applicata al dataset OULAD: 32.593
+iscrizioni studente-corso distribuite su 7 moduli, con:
+- clickstream comportamentale completo dal Virtual Learning Environment (VLE) dell'università
+- record delle valutazioni
+- profili demografici.
-**Definizione dell'outcome:** Ogni iscrizione è classificata in una di due classi:
+**Definizione dell'outcome.** Ogni iscrizione è classificata in una delle due seguenti classi:
- **Completato**: esito finale Pass o Distinction
- **Non completato**: esito finale Fail o Withdrawn
-Questa suddivisione binaria è coerente con la letteratura OULAD e consente
-un'analisi di retention pulita.
+Questa suddivisione binaria è coerente con la letteratura OULAD e consente un'analisi di retention pulita.
**Toolkit statistico:**
-| Metodo | Utilizzato per | Metriche riportate |
-|--------|----------------|---------------------|
-| Welch's t-test | Segnali continui vs. outcome | t-statistic, p-value, d di Cohen |
-| Test chi-quadrato | Variabili demografiche categoriche vs. outcome | chi-quadrato, p-value, V di Cramér |
-| Bonferroni + Benjamini-Hochberg | Correzione per confronti multipli | p-value corretti |
-| Bootstrap CI\* | Gruppi con tassi estremi (es. studenti ghost) | Intervalli di confidenza al 95% |
+| Metodo | Utilizzato per | Metriche riportate |
+|---------------------------------|----------------|---------------------|
+| t-test di Welch\* | Segnali continui vs outcome | t-statistic, p-value, d di Cohen |
+| Test chi-quadrato | Variabili demografiche categoriche vs outcome | chi-quadrato, p-value, V di Cramér |
+| [Bonferroni](https://en.wikipedia.org/wiki/Bonferroni_correction) e [Benjamini-Hochberg](https://en.wikipedia.org/wiki/False_discovery_rate) | Correzione per confronti multipli | p-value corretti |
+| Bootstrap CI\*\* | Gruppi con tassi estremi (es. studenti ghost) | Intervalli di confidenza al 95% |
+
+\* Il t-test confronta le medie di due gruppi e valuta se la differenza osservata sia spiegabile dal puro caso.
+La variante di Welch non assume che i due gruppi abbiano la stessa variabilità: è la scelta più prudente quando i
+gruppi confrontati hanno dimensioni molto diverse, come qui (chi completa e chi non completa). Approfondimento:
+[Welch's t-test](https://en.wikipedia.org/wiki/Welch%27s_t-test) (Wikipedia, in inglese).
-\* CI = Confidence Interval (intervallo di confidenza): la forbice di valori entro
-cui, con il 95% di confidenza, cade il valore vero. Qui è stimata via bootstrap,
-cioè ricampionando molte volte i dati osservati.
+\*\* CI = Confidence Interval (intervallo di confidenza): la forbice di valori entro cui, con il 95% di confidenza,
+cade il valore vero. Qui è stimata via bootstrap, cioè ricampionando molte volte i dati osservati.
-Tutti i test utilizzano una soglia di significatività alfa = 0,05. L'effect size, non il
-p-value, è il criterio primario per classificare i predittori, perché con ~32K osservazioni
-anche differenze banali raggiungono la significatività statistica. Non vengono utilizzati
-modelli di machine learning. Tutti i risultati sono associazioni osservazionali.
+Tutti i test utilizzano una soglia di significatività alfa = 0,05. L'effect size, non il p-value, è il criterio
+primario per classificare i predittori, perché con ~32K osservazioni anche differenze banali raggiungono la
+significatività statistica. Non vengono utilizzati modelli di machine learning.
+Tutti i risultati sono associazioni osservazionali.
### Come leggere i numeri
-**p-value**: indica quanto sarebbe improbabile osservare una differenza almeno così
-grande se, nella realtà, non ci fosse alcuna differenza. Sotto la soglia alfa = 0,05
-la differenza si dice statisticamente significativa.
+**Engagement**: termine inglese che indica la partecipazione attiva dello studente alla piattaforma (accessi,
+giorni di attività, click sui materiali). Non ha un equivalente italiano altrettanto compatto ("coinvolgimento
+attivo" è la resa più vicina) ed è quindi mantenuto come termine tecnico in tutto il report.
+
+**p-value**: indica quanto sarebbe improbabile osservare una differenza almeno così grande se, nella realtà, non ci
+fosse alcuna differenza. Sotto la soglia alfa = 0,05 la differenza si dice statisticamente significativa.
-**d di Cohen** (per variabili numeriche): misura la distanza tra due gruppi in unità
-di variabilità tipica (deviazioni standard). Scala di riferimento:
+**d di Cohen** (per variabili numeriche): misura la distanza tra due gruppi in unità di variabilità tipica (deviazioni
+standard). Scala di riferimento:
- d ≈ 0,2: effetto piccolo
- d ≈ 0,5: effetto medio
- d ≥ 0,8: effetto grande
-*Esempio svolto con i dati reali del progetto:* nei primi 28 giorni, chi poi completa
-il corso è attivo in media 12,8 giorni; chi non completa, 6,5. La differenza (6,33
-giorni), rapportata alla deviazione standard aggregata dei due gruppi (circa 7,05
-giorni), dà d = 6,33 / 7,05 ≈ 0,90: un effetto grande.
+**Esempio con i dati reali del progetto.**
+*Nei primi 28 giorni, chi poi completerà il corso è attivo in media 12,8 giorni. Chi non lo completerà, 6,5.
+La differenza (6,33 giorni), rapportata alla deviazione standard aggregata dei due gruppi (circa 7,05 giorni),
+restituisce d = 6,33 / 7,05 ≈ 0,90: un effetto grande.*
-**V di Cramér** (per variabili categoriche): misura la forza dell'associazione tra
-due variabili su una scala che va da 0 (nessuna associazione) a 1 (associazione
-perfetta). Con un outcome binario, valori intorno a 0,1 indicano un'associazione
-debole, intorno a 0,3 media, da 0,5 in su forte.
+**V di Cramér** (per variabili categoriche): misura la forza dell'associazione tra due variabili su una scala che va
+da 0 a 1:
+- **0**: **nessuna** associazione
+- **1**: associazione **perfetta**
-*Esempio svolto con i dati reali del progetto:* per il livello di istruzione il test
-chi-quadrato vale 737,2 su 32.593 iscrizioni; con outcome binario la formula si
-riduce a V = radice quadrata di (737,2 / 32.593) ≈ 0,15: un'associazione debole.
+Con un outcome binario, valori intorno a 0,1 indicano un'associazione **debole**, intorno a 0,3 **media**, da 0,5 in su **forte**.
+
+**Esempio con i dati reali del progetto.**
+*Per il livello di istruzione il test **chi-quadrato** vale 737,2 su 32.593 iscrizioni.
+Con outcome binario la formula si riduce a V = radice quadrata di (737,2 / 32.593) ≈ 0,15: un'associazione **debole**.*
---
+
+
## BQ1: Dove e quando gli studenti abbandonano?
-> **Risultato chiave:** Circa un'iscrizione su tre termina con il ritiro esplicito.
-> L'abbandono non è casuale: si concentra intorno a tappe del corso, e il suo
-> profilo temporale varia tra i moduli.
+> **Risultato chiave:** circa **un'iscrizione su tre** termina con il **ritiro esplicito**.
+> L'abbandono non è casuale: si concentra intorno a specifiche tappe del corso, e il suo profilo temporale varia
+> tra i moduli.
+
+Tra i 7 moduli OULAD, i tassi di ritiro vanno dall'**11,8%** (modulo GGG) al **44,2%** (modulo CCC).
+Il tasso di ritiro complessivo ponderato è circa il **31%** di tutte le iscrizioni: una quota significativa della
+popolazione studentesca che non raggiunge mai il completamento.
+
+Le curve cumulative di abbandono rivelano **profili temporali distinti** per corso.
+Alcuni moduli perdono molti studenti già nelle prime settimane (il segnale tipico di un onboarding che fallisce),
+mentre altri presentano un declino più graduale a metà corso.
+All'interno dello stesso modulo, presentazioni (coorti) diverse seguono traiettorie sostanzialmente simili,
+suggerendo che è il design del corso, non la variazione casuale della coorte, a determinare la forma dell'abbandono.
+
+Una nota per leggere il grafico: l'asse orizzontale parte da valori negativi perché l'iscrizione ai corsi apre con
+largo anticipo rispetto all'inizio delle lezioni (il giorno 0). Un ritiro nella zona negativa dell'asse significa
+quindi che lo studente si è cancellato prima ancora che il corso cominciasse: il fenomeno è analizzato poco più
+avanti in questa sezione.
-Tra i 7 moduli OULAD, i tassi di ritiro vanno dall'**11,8%** (modulo GGG) al
-**44,2%** (modulo CCC). Il tasso di ritiro complessivo ponderato è circa il **31%**
-di tutte le iscrizioni: una quota significativa della popolazione studentesca che non
-raggiunge mai il completamento.
+
-Le curve cumulative di abbandono rivelano **profili temporali distinti** per corso. Alcuni
-moduli mostrano un'attrizione precoce ripida (pattern di fallimento dell'onboarding),
-mentre altri presentano un declino più graduale a metà corso. All'interno dello stesso
-modulo, presentazioni (coorti) diverse seguono traiettorie sostanzialmente simili,
-suggerendo che è il design del corso, non la variazione casuale della coorte, a
-determinare la forma dell'abbandono.
+*Le curve cumulative di abbandono mostrano profili temporali distinti per corso.
+Ogni linea rappresenta una presentazione del corso, colorata per modulo.*
-
+Il secondo pattern riguarda i **cliff event\***: giorni in cui i ritiri non crescono in modo graduale, ma si
+impennano all'improvviso, come se gli studenti cadessero tutti insieme da un gradino (in inglese cliff, precipizio).
+Questi picchi non arrivano in giorni qualsiasi: coincidono con le scadenze delle valutazioni e con il rilascio
+dei voti.
-*Le curve cumulative di abbandono mostrano profili temporali distinti per corso. Ogni linea
-rappresenta una presentazione del corso, colorata per modulo.*
+\* Cliff event: giorno con un numero di ritiri sproporzionatamente alto rispetto al resto del corso, sopra il
+95° percentile (cioè con più ritiri del 95% degli altri giorni di quel corso).
-I **cliff event**, giorni con un numero sproporzionatamente alto di ritiri (sopra il
-95° percentile per quel corso), coincidono con le scadenze delle valutazioni e il rilascio
-dei voti. Questi sono azionabili: gli interventi possono essere programmati prima delle
-date dei cliff event.
+
-
+*Cliff event rilevati tramite soglia p95. Come si può facilmente osservare, i picchi di abbandono più grandi in un
+singolo giorno corrispondono a tappe del corso: scadenze delle valutazioni e rilascio dei voti.*
-*Cliff event rilevati tramite soglia p95. I picchi di abbandono più grandi in un singolo
-giorno corrispondono a tappe del corso.*
+Per chi gestisce la piattaforma questa regolarità è una buona notizia: se i giorni critici sono prevedibili, si può
+agire in anticipo. Un promemoria, o un'offerta di aiuto, inviati pochi giorni prima di una scadenza arrivano proprio
+nel momento in cui il rischio di abbandono è massimo.
-Oltre un quarto dei ritiri espliciti (26,6%, 2.678 su 10.072) avviene **prima ancora
-dell'inizio del corso** (giorno di dropout < 0). Questi ritiri pre-corso rappresentano
-puro churn di registrazione: studenti che si sono iscritti ma non hanno mai fruito
-di alcun contenuto. Si tratta di un problema
-di attivazione, non accademico.
+Oltre un quarto dei ritiri espliciti (26,6%, ovvero 2.678 su 10.072) avviene **prima ancora dell'inizio del corso**
+(giorno di dropout < 0). Questi ritiri pre-corso rappresentano puro churn di registrazione: studenti che si sono
+iscritti ma non hanno mai fruito di alcun contenuto.
+Si tratta di un **problema di attivazione**, non accademico.
-
+
-*Ritiri pre-corso per modulo. Questi studenti necessitano di nudge di onboarding, non di
-supporto accademico.*
+*Ritiri pre-corso per modulo. Questi studenti non hanno bisogno di supporto accademico: serve piuttosto una spinta
+gentile di benvenuto (in gergo, un nudge) che li accompagni fino al primo accesso.*
-Sapere *quando* gli studenti se ne vanno solleva la domanda successiva: possiamo prevederlo?
+Sapere **quando** gli studenti se ne vanno solleva la domanda successiva: **possiamo prevederlo**?
---
+
+
## BQ2: Quali segnali precoci predicono l'abbandono?
-> **Risultato chiave:** Tutte le 8 metriche di engagement precoce testate sono
+> **Risultato chiave:** tutte le 8 metriche di engagement precoce testate sono
> significativamente associate all'abbandono dopo correzione per confronti multipli
> (8/8 dopo Bonferroni e Benjamini-Hochberg). I predittori più forti sono le metriche
> di volume dell'engagement: decile di engagement intra-corso, giorni attivi e click
-> totali nei primi 28 giorni.
+> totali nei primi 28 giorni. In parole semplici: quanto, e quanto spesso, uno studente
+> usa la piattaforma nelle prime quattro settimane dice già molto su come finirà il corso.
-Utilizzando solo i dati dei primi 28 giorni di iscrizione, abbiamo testato 8 segnali
-comportamentali per la loro associazione con il completamento finale. L'effect size
-(d di Cohen), non il p-value, è il criterio primario di classificazione, perché con
+Utilizzando solo i dati dei primi 28 giorni di iscrizione, abbiamo testato l'associazione tra **8 segnali
+comportamentali** e il completamento finale del corso.
+L'effect size (d di Cohen), non il p-value, è il **criterio primario** di **classificazione**, dato che con
~32K osservazioni la significatività è facile da raggiungere.
-Il **forest plot** sottostante classifica tutti i segnali per effect size assoluto.
-Le metriche di volume dell'engagement dominano la classifica: decile di engagement
-intra-corso (d = 0,97), giorni attivi (d = 0,90) e click totali (d = 0,63), seguiti
-dall'ultimo giorno attivo, dal punteggio della prima valutazione e dall'intensità media
-dei click (d tra 0,52 e 0,55; i segnali basati sulle valutazioni sono calcolati sulla
-sola sottopopolazione dei submitter).
+Il **forest plot** sottostante classifica tutti i segnali per **effect size assoluto**.
+Le metriche di volume dell'engagement dominano la classifica:
+- decile di engagement intra-corso (d = 0,97), cioè la posizione dello studente nella classifica di engagement
+del proprio corso, divisa in dieci fasce
+- giorni attivi (d = 0,90)
+- click totali (d = 0,63)
+
+Seguono, con effetti medi (d tra 0,52 e 0,55), l'ultimo giorno attivo, il punteggio della prima valutazione e
+l'intensità media dei click; chiudono la classifica il giorno della prima consegna e il giorno di registrazione.
+I segnali basati sulle valutazioni sono calcolati sulla sola sottopopolazione di chi ha consegnato almeno una
+valutazione (i submitter).
-
+
*Tutti gli 8 segnali classificati per d di Cohen. I punti verdi indicano significatività
dopo correzione Benjamini-Hochberg. Le linee di riferimento verticali segnano le soglie
di effect size piccolo, medio e grande.*
-Il contrasto più drammatico è tra gli **studenti ghost** (quelli con zero attività VLE
-nei primi 28 giorni) e gli studenti attivi. Gli studenti ghost hanno un tasso di
-completamento prossimo allo zero, mentre gli studenti attivi completano a un tasso vicino
-alla media della piattaforma. Gli intervalli di confidenza bootstrap al 95% non si
-sovrappongono. (Nota: BQ5 amplia questa definizione per includere attività quasi nulla,
-cioè al massimo 1 giorno attivo e meno di 10 click, per catturare l'intero segmento a
-rischio ai fini del targeting degli interventi.)
+Il contrasto più netto è tra gli **studenti ghost** (quelli con zero attività VLE
+nei primi 28 giorni) e gli studenti attivi:
+- gli studenti ghost hanno un tasso di completamento prossimo allo zero
+- gli studenti attivi completano a un tasso vicino alla media della piattaforma.
+
+Gli intervalli di confidenza bootstrap al 95% non si sovrappongono.
+(Nota: BQ5 amplia questa definizione per includere l'attività quasi nulla, cioè al massimo 1 giorno attivo e meno
+di 10 click, per catturare l'intero segmento a rischio quando si individuano i destinatari degli interventi.)
-
+
*Gli studenti ghost (zero attività VLE nei primi 28 giorni) hanno tassi di completamento
prossimi allo zero. Le barre di errore mostrano intervalli di confidenza bootstrap al 95%.*
La relazione dose-risposta è **monotonica**: più engagement predice costantemente un
completamento più alto, senza soglia né rendimenti decrescenti. Questo significa che il
-segnale è utile lungo tutto il suo range, non solo agli estremi.
+segnale è utile lungo tutto il suo intervallo di valori, non solo agli estremi.
-
+
*Tasso di completamento per quartile del segnale per i 3 predittori principali. La
relazione è graduata, non binaria.*
-Due insight aggiuntivi rafforzano il portafoglio di segnali. La **submission delle
-valutazioni** è un potente predittore binario: gli studenti che hanno consegnato almeno
-una valutazione nei primi 28 giorni completano a tassi sostanzialmente più alti dei
-non-submitter. E la **costanza batte l'intensità**: login giornalieri regolari predicono
-il completamento più fortemente di burst ad alto numero di click per sessione.
+Due insight aggiuntivi rafforzano il portafoglio di segnali:
+
+- la **consegna delle valutazioni** è un potente predittore binario: gli studenti che hanno consegnato almeno
+una valutazione nei primi 28 giorni completano a tassi sostanzialmente più alti di chi non ha consegnato nulla
+
-Questi segnali comportamentali sono forti. Ma sono semplicemente proxy della demografia?
+- la **costanza batte l'intensità**: accessi giornalieri regolari predicono il completamento più fortemente
+di poche sessioni concentrate con moltissimi click
+
+È vero che questi segnali comportamentali sono molto forti. Ma sono semplicemente un riflesso della demografia?
---
+
+
## BQ3: Cosa conta di più, demografia o comportamento?
-> **Risultato chiave:** Il comportamento domina. Gli effect size comportamentali sono
+> **Risultato chiave:** il **comportamento domina**. Gli effect size comportamentali sono
> multipli rispetto a quelli demografici. All'interno di ogni livello di istruzione,
-> l'engagement alto supera drammaticamente l'engagement basso.
+> l'engagement alto supera nettamente l'engagement basso.
+
+Rispetto all'esito finale del corso (completato o non completato), abbiamo testato:
+- **6 variabili demografiche categoriche**:
+ - genere
+ - fascia d'età
+ - livello di istruzione
+ - fascia IMD (Index of Multiple Deprivation)
+ - disabilità
+ - regione
+
+
+- **2 variabili demografiche numeriche**:
+ - tentativi precedenti
+ - crediti studiati
+
+Il risultato finale è stato che **tutte** le 8 sono **statisticamente significative** dopo correzione
+**Benjamini-Hochberg** ([approfondimento su Wikipedia, in inglese](https://en.wikipedia.org/wiki/False_discovery_rate)),
+ma i loro **effect size** sono **uniformemente deboli**.
+Il predittore demografico più forte (livello di istruzione più alto) raggiunge una V di Cramér di circa **0,15**.
+Segue la fascia IMD con **0,13** e tutte le altre variabili demografiche restano sotto **0,09**.
-Abbiamo testato 6 variabili demografiche categoriche (genere, fascia d'età, livello di
-istruzione, fascia IMD (Index of Multiple Deprivation), disabilità, regione) e 2
-variabili demografiche numeriche
-(tentativi precedenti, crediti studiati) contro l'esito di completamento. Tutte le 8 sono
-statisticamente significative dopo correzione Benjamini-Hochberg, ma i loro effect size
-sono uniformemente deboli. Il predittore demografico più forte (livello di istruzione
-più alto) raggiunge una V di Cramér di circa **0,15**; segue la fascia IMD con **0,13**,
-e tutte le altre variabili demografiche restano sotto **0,09**.
+Di contro, le **variabili comportamentali** (giorni attivi, click totali, consegna delle valutazioni, intensità
+dei click) mostrano **effect size** diverse volte **superiori**.
-Per contro, le variabili comportamentali (giorni attivi, click totali, submission
-valutazioni, intensità click) mostrano effect size diverse volte superiori. Il divario è
-netto: i segnali comportamentali predicono l'esito molto più fortemente di qualsiasi
-variabile demografica.
+Il divario è netto: **i segnali comportamentali predicono l'esito molto più fortemente di qualsiasi
+variabile demografica**.
-
+
*Confronto diretto degli effect size demografici e comportamentali. Il divario è
sostanziale: i segnali comportamentali sono costantemente più forti.*
-Il test critico: l'engagement riflette semplicemente la demografia? Il grafico di
+Il test critico: l'engagement riflette semplicemente la demografia? Il grafico di
interazione sottostante mostra che all'interno di **ogni livello di istruzione**, gli
-studenti ad alto engagement superano drammaticamente quelli a basso engagement. Uno
+studenti ad alto engagement superano nettamente quelli a basso engagement. Uno
studente con un livello di istruzione formale inferiore ma alto engagement ha più
probabilità di completare rispetto a uno studente altamente istruito che non interagisce
con la piattaforma.
-
+
*All'interno di ogni livello di istruzione, il gap di engagement sovrasta il gap
-educativo. Il comportamento è il fattore determinante, non il background.*
+educativo. Il **comportamento** è il **fattore determinante**, non il background.*
-Questo risultato ha una **dimensione etica**: i segnali comportamentali sono sia
-statisticamente più forti *sia* azionabili. La demografia non può essere cambiata; il
-comportamento può essere influenzato attraverso il design della piattaforma. Targetizzare
-il comportamento evita le preoccupazioni di equità insite nel profiling demografico.
+Questo risultato ha anche una **dimensione etica**: i segnali comportamentali non sono soltanto i più forti
+statisticamente, sono anche quelli su cui la piattaforma può davvero agire.
+La demografia di uno studente non si può cambiare; il suo comportamento sì, attraverso il design della piattaforma.
+Concentrare gli interventi sul comportamento evita inoltre le preoccupazioni di equità insite nella profilazione
+demografica.
-Il design del corso stesso influenza i livelli di engagement?
+Alla luce di questo, la domanda successiva nasce spontanea: *è il design del corso stesso a influenzare i livelli
+di engagement?*
---
+
+
## BQ4: Come le caratteristiche dei corsi influenzano la retention?
-> **Risultato chiave:** I tassi di completamento variano sostanzialmente tra i 7 moduli:
-> dal **37,4%** (CCC) al **70,9%** (AAA), un gap di **33,5 punti percentuali**. Pattern
-> suggestivi emergono intorno alla densità delle valutazioni e alla durata del corso, ma
-> con soli 7 punti dati non è possibile alcuna conclusione inferenziale.
+> **Risultato chiave:** i tassi di completamento variano sostanzialmente tra i 7 moduli, andando dal **37,4%** (CCC)
+> al **70,9%** (AAA), ovvero un gap di **33,5 punti percentuali**.
+> Pattern suggestivi emergono intorno alla densità delle valutazioni e alla durata del corso, ma
+> con soli 7 corsi (7 punti dati) non è possibile alcuna conclusione inferenziale.
-Il grafico di ranking sottostante mostra l'intera distribuzione. Il modulo AAA trattiene
-quasi tre quarti dei suoi studenti; il modulo CCC ne perde quasi due terzi.
+Il grafico sottostante mostra la classifica completa. Il modulo AAA trattiene
+quasi tre quarti dei suoi studenti, mentre il modulo CCC ne perde quasi due terzi.
-
+
*I tassi di completamento vanno dal 37,4% al 70,9% tra i 7 moduli OULAD.*
Gli scatter plot esplorativi rivelano pattern suggestivi tra le caratteristiche del design
-del corso (densità valutazioni, durata) e i tassi di completamento. Tuttavia, con n = 7,
-qualsiasi correlazione è descrittiva, non inferenziale: la correlazione di Spearman
-richiede |rho| > 0,79 per la significatività a questa dimensione campionaria.
+del corso (densità delle valutazioni, durata) e i tassi di completamento. Tuttavia, come detto, con n = 7
+qualsiasi correlazione è descrittiva, non inferenziale: la correlazione di Spearman\* richiede infatti
+|rho| > 0,79 per la significatività con un campione così piccolo.
+
+\* La correlazione di Spearman misura quanto due grandezze si muovono insieme guardando l'ordine dei valori
+(le posizioni in classifica) invece dei valori esatti: rho = 1 quando al crescere dell'una cresce sempre anche
+l'altra, rho = 0 quando non c'è alcun legame, rho = -1 quando il legame è perfettamente inverso. Con soli 7 corsi,
+solo un legame quasi perfetto (|rho| > 0,79) si può distinguere dal puro caso. Approfondimento:
+[Spearman's rank correlation coefficient](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient)
+(Wikipedia, in inglese).
-
+
*La densità delle valutazioni e la durata del corso mostrano associazioni suggestive con
il completamento. Ogni punto è un modulo (mediato sulle sue presentazioni).*
-**Avvertenze critiche:** Questi pattern sono confusi da almeno tre fattori: (1) difficoltà
-della materia, perché alcuni moduli insegnano contenuti intrinsecamente più difficili; (2)
-auto-selezione degli studenti, perché studenti più motivati potrebbero scegliere
-determinati corsi; (3) investimento istituzionale, perché l'allocazione delle risorse
-varia tra i dipartimenti. Il design del corso è una leva che vale la pena studiare, ma richiede più
-dati (più corsi, o variazione sperimentale) per trarre conclusioni.
+**Avvertenze critiche.** Questi pattern si intrecciano con almeno tre fattori che i dati non permettono di separare:
+- difficoltà della materia, perché alcuni moduli insegnano contenuti intrinsecamente più difficili
+- auto-selezione degli studenti, perché studenti più motivati potrebbero scegliere determinati corsi
+- investimento istituzionale, perché l'allocazione delle risorse varia tra i dipartimenti
+
+Il design del corso è quindi una leva che vale la pena studiare, ma richiede più dati (più corsi, o variazione
+sperimentale) per trarre conclusioni interessanti e di reale supporto alle decisioni.
-Attingendo da tutte e quattro le analisi precedenti, proponiamo ora tre interventi concreti.
+Attingendo da tutte e quattro le analisi precedenti, **si propongono** ora **tre interventi concreti**.
---
+
+
## BQ5: Top 3 interventi raccomandati
-> **Risultato chiave:** Tre interventi basati sul comportamento, ordinati per rapporto
-> impatto/costo, coprono insieme la maggioranza degli studenti a rischio. Poiché i
-> segmenti si sovrappongono significativamente, un rollout sequenziato evita outreach
-> ridondanti.
+> **Risultato chiave:** tre interventi basati sul comportamento, ordinati per rapporto impatto/costo,
+> coprono insieme la maggioranza degli studenti a rischio.
+> Poiché i segmenti si sovrappongono in modo significativo, un'attivazione scaglionata degli interventi
+> (rollout sequenziato) evita di contattare più volte gli stessi studenti con messaggi ridondanti.
### Segmenti target
-La query BQ5 dimensiona tre segmenti studenteschi definiti da criteri osservabili e
-azionabili, non demografici. Tutte le definizioni utilizzano dati comportamentali dei
-primi 28 giorni.
+La query BQ5 dimensiona tre segmenti studenteschi definiti da criteri osservabili e su cui la piattaforma può
+agire direttamente, non demografici.
+Tutte le definizioni utilizzano dati comportamentali dei primi 28 giorni.
| Segmento | Definizione | Dimensione | Tasso di non completamento |
|----------|------------|------------|---------------------------|
@@ -311,91 +374,103 @@ primi 28 giorni.
| **Non-submitter** | Nessuna valutazione consegnata nei primi 28 giorni | **11.494** (35,3%) | **71,8%** |
| **Early disengager** | Attività nei giorni 0–14, zero nei giorni 15–28 | **2.213** (6,8%) | **77,8%** |
-Tutti e tre i segmenti mostrano tassi di non completamento molto superiori al baseline
-della piattaforma (~53%). Gli studenti ghost completano appena nel 7,7% dei casi, contro
-una media di piattaforma del 47,2%.
+Nota sulla metrica: la tabella riporta il tasso di **non completamento** (bocciature e ritiri insieme), non il solo
+tasso di ritiro usato in BQ1. Per chi progetta gli interventi contano entrambi gli esiti negativi: uno studente che
+arriva in fondo al corso e viene bocciato è comunque uno studente che la piattaforma non è riuscita a portare al
+traguardo.
+
+Tutti e tre i segmenti mostrano tassi di non completamento molto superiori al baseline della piattaforma (~53%).
+Gli studenti ghost completano appena nel 7,7% dei casi, contro una media di piattaforma del 47,2%.
+
+
+
+*Dimensione dei tre segmenti target e relativo tasso di non completamento, a confronto con il valore complessivo
+riscontrato in piattaforma.*
### I tre interventi
-| | Attivazione Ghost | Checkpoint Valutazioni | Re-engagement Settimana 3 |
-|---|---|---|---|
-| **Priorità** | 1: Quick win | 2: Costruire dopo | 3: Investire quando pronti |
-| **Trigger** | Zero attività VLE\* entro il giorno 3 | 3 giorni prima della prima scadenza, non consegnato | 3+ giorni consecutivi di inattività dopo attività iniziale |
-| **Azione** | Sequenza email: benvenuto giorno 3 + follow-up giorno 7 con link al primo step | Promemoria con anteprima della valutazione e stima del tempo | Email "Ci manchi" con riepilogo progressi e confronto con i pari |
-| **Costo** | **Basso** (solo automazione email) | **Medio** (trigger consapevoli delle scadenze + calendario del corso) | **Medio-Alto** (tracciamento attività in tempo reale + personalizzazione) |
-| **Evidenza** | BQ2: l'engagement precoce è il predittore più forte; BQ3: comportamento > demografia | BQ2: la submission è un segnale binario chiave; BQ1: cliff alle scadenze | BQ1: cliff di abbandono a metà corso alle settimane 3–4; BQ2: predittore ultimo-giorno-attivo |
-| **Stima impatto** | Maggiore: divario più ampio tra segmento e tasso della piattaforma | Medio: divario sostanziale submitter vs non-submitter | Medio: targetizza un failure mode distinto dai ghost |
-
-\* VLE = Virtual Learning Environment, la piattaforma online dove vivono i
-contenuti del corso.
-
-**Approccio alla stima dell'impatto:** Per ogni intervento, modelliamo scenari di
-conversione conservativi (10–25% degli studenti targetizzati cambiano comportamento).
-Gli studenti ghost convertiti si assumono raggiungere il tasso medio di completamento
-della piattaforma, non quello degli studenti attivi. Gli studenti re-ingaggiati si
-assumono raggiungere un tasso a metà tra disimpegnati e costanti. Queste sono assunzioni
-deliberatamente conservative.
+| | Attivazione Ghost | Checkpoint Valutazioni | Re-engagement Settimana 3 |
+|---|--------------------------------------------------------------------------------------|--------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------|
+| **Priorità** | 1: Quick win | 2: Costruire dopo | 3: Investire quando pronti |
+| **Trigger** | Zero attività VLE\* entro il giorno 3 | 3 giorni prima della prima scadenza, non consegnato | 3+ giorni consecutivi di inattività dopo attività iniziale |
+| **Azione** | Sequenza email: benvenuto giorno 3 e follow-up giorno 7 con link al primo step | Promemoria con anteprima della valutazione e stima del tempo | Email "Ci manchi" con riepilogo progressi e confronto con i pari |
+| **Costo** | **Basso** (solo automazione email) | **Medio** (trigger consapevoli delle scadenze e calendario del corso) | **Medio-Alto** (tracciamento attività in tempo reale e personalizzazione) |
+| **Evidenza** | BQ2: l'engagement precoce è il predittore più forte; BQ3: comportamento > demografia | BQ2: la consegna è un segnale binario chiave; BQ1: cliff alle scadenze | BQ1: cliff di abbandono a metà corso alle settimane 3-4; BQ2: predittore ultimo giorno attivo |
+| **Stima impatto** | Maggiore: divario più ampio tra segmento e tasso della piattaforma | Medio: divario sostanziale submitter vs non-submitter | Medio: intercetta una modalità di abbandono diversa da quella dei ghost |
+
+\* VLE = Virtual Learning Environment, la piattaforma online dove vivono i contenuti del corso.
+
+**Approccio alla stima dell'impatto:** per ogni intervento modelliamo scenari di conversione prudenti, in cui si
+ipotizza che cambi comportamento solo il 10–25% degli studenti raggiunti.
+Si assume inoltre che gli studenti ghost convertiti possano raggiungere il tasso medio di completamento della
+piattaforma (non quello degli studenti attivi) e che gli studenti re-ingaggiati si fermino a un tasso a metà strada
+tra disimpegnati e costanti.
+Queste sono assunzioni deliberatamente conservative.
### Sovrapposizione dei segmenti
-Gli studenti ghost e i non-submitter si **sovrappongono fortemente**: uno studente con
-zero accesso VLE non può consegnare una valutazione. Questo significa che gli interventi
-1 e 2 targetizzano in larga misura la stessa popolazione da angolazioni diverse; il loro
-impatto non va sommato ingenuamente. Gli early disengager, per definizione, hanno avuto
-attività iniziale: si sovrappongono meno con i ghost, rendendo l'intervento 3 una leva
-indipendente che targetizza un failure mode diverso.
+Gli studenti ghost e i non-submitter si **sovrappongono fortemente**: uno studente con zero accessi al VLE non può
+consegnare una valutazione. Questo significa che gli interventi 1 e 2 raggiungono in larga misura la stessa
+popolazione da angolazioni diverse. Il loro impatto non va quindi ingenuamente sommato.
+Gli early disengager, per definizione, hanno invece avuto un'attività iniziale: si sovrappongono meno con i ghost,
+e questo rende il terzo intervento ([Re-engagement Settimana 3](#i-tre-interventi)) una leva indipendente, che
+intercetta una modalità di abbandono diversa.
-
+
-*Matrice priorità impatto-vs-costo. L'Attivazione Ghost è il chiaro quick win:
-segmento più grande, eccesso di non completamento più alto, costo più basso.*
+*Matrice priorità impatto-costo. L'Attivazione Ghost è il chiaro quick win (il massimo risultato con lo sforzo
+minimo): segmento più grande, eccesso di non completamento più alto e costo più basso.*
-
+
*Analisi della sovrapposizione dei segmenti. Le barre grigie mostrano studenti
appartenenti a più segmenti. La sovrapposizione ghost–non-submitter è sostanziale.*
---
+
+
## Limitazioni e avvertenze
-- **Solo dati osservazionali.** Tutti gli effect size e le differenze nei tassi di
- completamento sono associazioni, non relazioni causali. Gli studenti più attivi
- potrebbero essere intrinsecamente più motivati: l'engagement potrebbe essere un
- proxy, non una causa.
-- **Dati storici.** OULAD copre le coorti 2013–2014 della Open University (UK).
- I comportamenti degli studenti e le piattaforme di apprendimento online sono
- cambiati significativamente da allora.
-- **BQ4 limitato da n = 7.** Con soli 7 moduli, nessuna statistica inferenziale è
- possibile per l'analisi a livello di corso. I pattern sulle caratteristiche del
- design sono ipotesi, non conclusioni.
-- **Le stime di impatto sono assunzioni.** I tassi di conversione (10–25%) sono
- proiezioni plausibili basate su benchmark di settore, non su risultati misurati.
- Non esistono dati di A/B testing nel dataset.
-- **Nessun dato sui costi.** Le stime dei costi di implementazione (Basso / Medio /
- Medio-Alto) sono qualitative. Lo sforzo ingegneristico effettivo dipende
- dall'infrastruttura della piattaforma esistente.
-- **Nota etica.** Tutti gli interventi targetizzano il comportamento, non la
- demografia. L'outreach automatizzato dovrebbe includere meccanismi di opt-out
- per rispettare l'autonomia degli studenti.
+- **Solo dati osservazionali.** Tutti gli effect size e le differenze nei tassi di completamento sono associazioni,
+non relazioni causali. Gli studenti più attivi potrebbero essere intrinsecamente più motivati:
+l'engagement potrebbe essere un proxy (un indicatore che riflette altro, per esempio la motivazione), non una causa.
+
+
+- **Dati storici.** OULAD copre le coorti 2013–2014 della Open University (UK). I comportamenti degli studenti e le
+piattaforme di apprendimento online sono cambiati significativamente da allora.
+
+
+- **BQ4 limitato da n = 7.** Con soli 7 moduli, nessuna statistica inferenziale è possibile per l'analisi a livello di
+corso. I pattern sulle caratteristiche del design sono ipotesi, non conclusioni.
----
-## Provenienza dei grafici
+- **Le stime di impatto sono assunzioni.** I tassi di conversione (10–25%) sono proiezioni plausibili basate su
+benchmark di settore, non su risultati misurati. Non esistono dati di A/B testing nel dataset.
+
+
+- **Nessun dato sui costi.** Le stime dei costi di implementazione (Basso / Medio / Medio-Alto) sono qualitative.
+Lo sforzo ingegneristico effettivo dipende dall'infrastruttura della piattaforma esistente.
+
+
+- **Nota etica.** Tutti gli interventi agiscono sul comportamento, non sulla demografia. Le comunicazioni
+automatiche verso gli studenti dovrebbero sempre includere un meccanismo di cancellazione (opt-out), per
+rispettare l'autonomia degli studenti.
+
+---
-Tutte le figure di questo report sono generate dai 7 notebook di analisi in
-[`notebooks/`](../../notebooks/): il prefisso numerico del file immagine corrisponde
-al notebook che lo produce (per esempio `03_dropout_curves_overlaid.png` nasce da
-`03_bq1_dropout_timing.ipynb`). I notebook leggono i CSV esportati dalla pipeline in
-`data/analysis/` (cartella locale, non versionata) e salvano i grafici in
-`reports/figures/`.
+
-Per rigenerare le figure da un clone del repository:
+## Appendice: provenienza dei grafici e dei numeri
-1. eseguire la pipeline: `python -m run_pipeline`
-2. eseguire i notebook in ordine (da 01 a 07): ogni notebook riesporta le proprie figure
+Tutte le figure di questo report sono generate dai 7 notebook di analisi in [`notebooks/`](../../notebooks/):
+il prefisso numerico del file immagine corrisponde al notebook che lo produce (per esempio
+`03_dropout_curves_overlaid.png` nasce da `03_bq1_dropout_timing.ipynb`).
+I notebook leggono i CSV esportati dalla pipeline e salvano ogni figura in due lingue, con lo stesso nome file:
+inglese in `reports/figures/` (usata dai documenti EN), italiano in `reports/figures/it/` (usata da questo report
+e dal README italiano).
-Le istruzioni complete di setup sono nel [README](../../it/README.md), sezione
-Avvio Rapido. I numeri citati nel testo sono verificati sugli stessi CSV esportati
-dalla pipeline, in particolare sugli export statistici (`stats_*.csv`) per effect
-size e intervalli di confidenza.
+I numeri citati nel testo sono verificati sugli stessi CSV esportati dalla pipeline, in particolare sugli export
+statistici (`stats_*.csv`) per effect size e intervalli di confidenza.
+Le istruzioni per rigenerare figure e dati da un clone del repository sono nel [README](../../it/README.md),
+sezione Avvio Progetto.
diff --git a/tests/test_notebooks.py b/tests/test_notebooks.py
new file mode 100644
index 0000000..383bff9
--- /dev/null
+++ b/tests/test_notebooks.py
@@ -0,0 +1,170 @@
+"""Static checks on the analysis notebooks.
+
+The test suite never executes the notebooks: they require the full DuckDB
+pipeline and would regenerate figures. These tests parse the .ipynb JSON
+instead, locking two things that would otherwise only fail at notebook
+runtime:
+
+- repository hygiene: stripped outputs, pure-Python code cells, no local
+ absolute paths leaking into public artifacts;
+- the NB07 display-label contract: segment, overlap-category and
+ recommendation names are constants used as dictionary keys shared across
+ cells (palette, DataFrames, L10N maps). A typo in one of those keys would
+ surface as a KeyError (or a silently gray bar) only when the notebook
+ runs; here it fails fast in CI.
+"""
+
+import ast
+import json
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Any
+
+import pytest
+
+from src.config import PROJECT_ROOT
+
+NOTEBOOKS_DIR: Path = PROJECT_ROOT / "notebooks"
+NOTEBOOK_PATHS: list[Path] = sorted(NOTEBOOKS_DIR.glob("*.ipynb"))
+NB07_PATH: Path = NOTEBOOKS_DIR / "07_bq5_recommendations_synthesis.ipynb"
+
+
+def _load_notebook(path: Path) -> dict[str, Any]:
+ """Parse a notebook as plain JSON: no nbformat dependency needed."""
+ return json.loads(path.read_text(encoding="utf-8"))
+
+
+def _code_cells(notebook: dict[str, Any]) -> list[dict[str, Any]]:
+ return [cell for cell in notebook["cells"] if cell["cell_type"] == "code"]
+
+
+def _source(cell: dict[str, Any]) -> str:
+ # .ipynb stores each cell source as a list of lines with trailing newlines
+ return "".join(cell["source"])
+
+
+def _find_code_cell(notebook: dict[str, Any], marker: str) -> str:
+ """Return the source of the single code cell containing marker.
+
+ Uniqueness is asserted so a copy-pasted definition in a second cell
+ (which would shadow the first at runtime) fails the lookup loudly.
+ """
+ matches: list[str] = [
+ _source(cell) for cell in _code_cells(notebook) if marker in _source(cell)
+ ]
+ assert len(matches) == 1, (
+ f"Expected exactly one code cell containing {marker!r}, "
+ f"found {len(matches)}"
+ )
+ return matches[0]
+
+
+def test_notebooks_are_discovered() -> None:
+ """Guard for the parametrized tests below: an empty glob would make
+ them all pass vacuously (e.g. after a directory rename)."""
+ assert len(NOTEBOOK_PATHS) >= 7
+
+
+@pytest.mark.parametrize("path", NOTEBOOK_PATHS, ids=lambda p: p.name)
+class TestNotebookHygiene:
+ """Hygiene rules every committed notebook must satisfy."""
+
+ def test_outputs_are_stripped(self, path: Path) -> None:
+ """nbstripout must have removed outputs and execution counts."""
+ for index, cell in enumerate(_code_cells(_load_notebook(path))):
+ assert cell["outputs"] == [], f"code cell {index} has outputs"
+ assert (
+ cell["execution_count"] is None
+ ), f"code cell {index} has an execution count"
+
+ def test_code_cells_are_pure_python(self, path: Path) -> None:
+ """Every code cell must parse with ast (no IPython magics or shell
+ escapes): the notebooks are read as plain Python by tooling
+ (ruff, static analysis) and by these very tests."""
+ for index, cell in enumerate(_code_cells(_load_notebook(path))):
+ try:
+ ast.parse(_source(cell))
+ except SyntaxError as exc:
+ pytest.fail(f"{path.name} code cell {index} is not pure Python: {exc}")
+
+ def test_no_absolute_local_paths(self, path: Path) -> None:
+ """Public artifacts must not leak absolute paths from a local machine."""
+ for index, cell in enumerate(_load_notebook(path)["cells"]):
+ assert "/Users/" not in _source(
+ cell
+ ), f"cell {index} contains a local absolute path"
+
+
+class TestNb07DisplayLabelContract:
+ """Cross-check NB07's display-label constants against its L10N maps.
+
+ The two cells are executed in isolation: the labels cell is pure by
+ design, and the localization cell only needs three names from the
+ setup cell (FIGURES_DIR, FIG_DPI, plt), stubbed here so no matplotlib
+ import or real figures directory is involved.
+ """
+
+ @pytest.fixture(scope="class")
+ def nb07_namespace(
+ self, tmp_path_factory: pytest.TempPathFactory
+ ) -> dict[str, Any]:
+ notebook = _load_notebook(NB07_PATH)
+ namespace: dict[str, Any] = {
+ "FIGURES_DIR": tmp_path_factory.mktemp("figures"),
+ "FIG_DPI": 150,
+ "plt": SimpleNamespace(show=lambda: None, close=lambda fig: None),
+ }
+ # exec() is deliberate: it is the only way to run notebook cells
+ # without a kernel, and the source is our own repository content.
+ exec(_find_code_cell(notebook, "PALETTE_SEGMENT = {"), namespace)
+ exec(_find_code_cell(notebook, "L10N = {"), namespace)
+ return namespace
+
+ @pytest.mark.parametrize("lang", ["en", "it"])
+ def test_l10n_covers_all_segments(
+ self, nb07_namespace: dict[str, Any], lang: str
+ ) -> None:
+ display = nb07_namespace["L10N"][lang]["segment_display"]
+ assert set(display) == set(nb07_namespace["SEGMENT_ORDER"])
+
+ @pytest.mark.parametrize("lang", ["en", "it"])
+ def test_l10n_covers_all_overlap_categories(
+ self, nb07_namespace: dict[str, Any], lang: str
+ ) -> None:
+ display = nb07_namespace["L10N"][lang]["category_display"]
+ assert set(display) == set(nb07_namespace["OVERLAP_CATEGORIES"])
+
+ @pytest.mark.parametrize("lang", ["en", "it"])
+ def test_l10n_covers_all_recommendations(
+ self, nb07_namespace: dict[str, Any], lang: str
+ ) -> None:
+ display = nb07_namespace["L10N"][lang]["rec_display"]
+ assert set(display) == set(nb07_namespace["RECOMMENDATIONS"])
+
+ def test_palette_matches_segment_order(
+ self, nb07_namespace: dict[str, Any]
+ ) -> None:
+ """SEGMENT_ORDER is derived from the palette keys; if that ever
+ changes, the per-segment colors would silently shuffle."""
+ assert list(nb07_namespace["PALETTE_SEGMENT"]) == list(
+ nb07_namespace["SEGMENT_ORDER"]
+ )
+
+ def test_one_recommendation_per_segment(
+ self, nb07_namespace: dict[str, Any]
+ ) -> None:
+ """The priority matrix zips RECOMMENDATIONS with SEGMENT_ORDER
+ positionally, so their lengths must match."""
+ assert len(nb07_namespace["RECOMMENDATIONS"]) == len(
+ nb07_namespace["SEGMENT_ORDER"]
+ )
+
+ def test_l10n_languages_are_structurally_identical(
+ self, nb07_namespace: dict[str, Any]
+ ) -> None:
+ """EN and IT must expose the same furniture keys and the same
+ number of cost ticks: a key present in one language only would
+ crash the figure loop for the other."""
+ l10n = nb07_namespace["L10N"]
+ assert set(l10n["en"]) == set(l10n["it"])
+ assert len(l10n["en"]["cost_ticks"]) == len(l10n["it"]["cost_ticks"])