feat(pipeline): statistical export, verified claims, and i18n restructuring

README.md

@@ -1,4 +1,4 @@
-# Learning Retention Analytics 🇬🇧 [🇮🇹](README_IT.md)
+# Learning Retention Analytics 🇬🇧 [🇮🇹](it/README.md)
 
 [![Test & Coverage](https://github.com/aleattene/learning-retention-analytics/actions/workflows/test.yml/badge.svg)](https://github.com/aleattene/learning-retention-analytics/actions/workflows/test.yml)
 [![Code Quality](https://github.com/aleattene/learning-retention-analytics/actions/workflows/code_quality.yml/badge.svg)](https://github.com/aleattene/learning-retention-analytics/actions/workflows/code_quality.yml)
@@ -83,7 +83,8 @@ project_root/
 │   ├── pipeline/
 │   │   ├── step_01_ingest.py           # CSV OULAD → raw DuckDB tables
 │   │   ├── step_02_transform.py        # Raw tables → analytical views
-│   │   └── step_03_export.py           # Views → CSV + optional Sheets push
+│   │   ├── step_03_export.py           # Views → CSV + optional Sheets push
+│   │   └── step_04_stats.py            # BQ2/BQ3 statistical tests → CSV
 │   ├── stats/tests.py                  # Statistical test wrappers
 │   ├── sheets/push.py                  # Google Sheets integration
 │   └── utils/                          # Logging, runtime utilities
@@ -211,7 +212,7 @@ In summary:
 | [Methodology](docs/METHODOLOGY.md) | Statistical approach, design choices, trade-offs |
 | [Transferability](docs/TRANSFERABILITY.md) | Pattern portability to SaaS, subscriptions, fitness |
 | [Cloud Migration](docs/MIGRATION.md) | DuckDB to BigQuery path, gaps and checklist |
-| [ADR](docs/ADR.md) | 7 architectural decisions with rationale |
+| [ADR](docs/ADR.md) | Architectural decisions with rationale |
 | [Testing](docs/TESTING.md) | Test architecture, strategy, and decisions |
 
 ---

README_IT.md → it/README.md

@@ -1,10 +1,10 @@
-# Analisi della Retention nell'Apprendimento 🇮🇹 [🇬🇧](README.md)
+# Analisi della Retention nell'Apprendimento 🇮🇹 [🇬🇧](../README.md)
 
 [![Test & Coverage](https://github.com/aleattene/learning-retention-analytics/actions/workflows/test.yml/badge.svg)](https://github.com/aleattene/learning-retention-analytics/actions/workflows/test.yml)
 [![Code Quality](https://github.com/aleattene/learning-retention-analytics/actions/workflows/code_quality.yml/badge.svg)](https://github.com/aleattene/learning-retention-analytics/actions/workflows/code_quality.yml)
 [![codecov](https://codecov.io/gh/aleattene/learning-retention-analytics/graph/badge.svg?token=LS2ASS9Z6K)](https://codecov.io/gh/aleattene/learning-retention-analytics)
 [![Python 3.13+](https://img.shields.io/badge/python-3.13%2B-blue.svg)](https://www.python.org/downloads/)
-[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](../LICENSE)
 [![Dataset: OULAD](https://img.shields.io/badge/dataset-OULAD-orange.svg)](https://analyse.kmi.open.ac.uk/open_dataset)
 
 ---
@@ -83,7 +83,8 @@ project_root/
 │   ├── pipeline/
 │   │   ├── step_01_ingest.py           # CSV OULAD → tabelle raw DuckDB
 │   │   ├── step_02_transform.py        # Tabelle raw → viste analitiche
-│   │   └── step_03_export.py           # Viste → CSV + push opzionale su Sheets
+│   │   ├── step_03_export.py           # Viste → CSV + push opzionale su Sheets
+│   │   └── step_04_stats.py            # Test statistici BQ2/BQ3 → CSV
 │   ├── stats/tests.py                  # Wrapper per test statistici
 │   ├── sheets/push.py                  # Integrazione Google Sheets
 │   └── utils/                          # Logging, utilità runtime
@@ -185,7 +186,7 @@ binarizzata come Completato (Pass + Distinction) vs Non completato (Fail + Withd
 
 ## Risultati Principali
 
-L'analisi completa è disponibile nel [Report Esecutivo](reports/REPORT_IT.md).
+L'analisi completa è disponibile nel [Report Esecutivo](../reports/it/REPORT.md).
 In sintesi:
 
 - **BQ1**: circa 1 iscrizione su 3 termina con il ritiro esplicito; il dropout
@@ -208,18 +209,18 @@ In sintesi:
 
 | Documento | Contenuto |
 |-----------|-----------|
-| [Report Esecutivo](reports/REPORT_IT.md) | Analisi completa BQ1–BQ5 con figure e numeri |
-| [Metodologia](docs/METHODOLOGY_IT.md) | Approccio statistico, scelte progettuali, trade-off |
-| [Trasferibilità](docs/TRANSFERABILITY_IT.md) | Portabilità dei pattern a SaaS, abbonamenti, fitness |
-| [Migrazione Cloud](docs/MIGRATION_IT.md) | Percorso da DuckDB a BigQuery, gap e checklist |
-| [ADR](docs/ADR_IT.md) | 7 decisioni architetturali con razionale |
-| [Testing](docs/TESTING_IT.md) | Architettura di test, strategia e decisioni |
+| [Report Esecutivo](../reports/it/REPORT.md) | Analisi completa BQ1–BQ5 con figure e numeri |
+| [Metodologia](../docs/it/METHODOLOGY.md) | Approccio statistico, scelte progettuali, trade-off |
+| [Trasferibilità](../docs/it/TRANSFERABILITY.md) | Portabilità dei pattern a SaaS, abbonamenti, fitness |
+| [Migrazione Cloud](../docs/it/MIGRATION.md) | Percorso da DuckDB a BigQuery, gap e checklist |
+| [ADR](../docs/it/ADR.md) | Decisioni architetturali con razionale |
+| [Testing](../docs/it/TESTING.md) | Architettura di test, strategia e decisioni |
 
 ---
 
 ## Licenza
 
-Questo progetto è distribuito con [Licenza MIT](LICENSE).
+Questo progetto è distribuito con [Licenza MIT](../LICENSE).
 
 Il dataset OULAD è distribuito con licenza
 [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) - vedi citazione sopra.

notebooks/04_bq2_early_signals.ipynb

@@ -528,7 +528,7 @@
    "id": "16",
    "metadata": {},
    "source": [
-    "> **Interpretation:** With a large dataset (~32K enrollments), most signals remain significant even after Bonferroni correction. This confirms that the differences are real — but statistical significance alone does not tell us which signals are *practically meaningful*. That is what the effect size ranking (next section) addresses."
+    "> **Interpretation:** With a large dataset (~32K enrollments), all 8 signals remain significant even after the conservative Bonferroni correction (8/8 under both Bonferroni and BH). This confirms that the differences are real — but statistical significance alone does not tell us which signals are *practically meaningful*. That is what the effect size ranking (next section) addresses."
    ]
   },
   {
@@ -607,7 +607,7 @@
    "id": "19",
    "metadata": {},
    "source": [
-    "> **Key finding:** The forest plot reveals a clear ranking of early behavioral signals. The strongest predictors of completion are likely engagement-volume metrics (active days, total clicks), while assessment-based signals and registration timing provide complementary information.\n",
+    "> **Key finding:** The forest plot reveals a clear ranking of early behavioral signals. Engagement-volume metrics lead: within-course engagement decile (d = 0.97), active days (d = 0.90), and total clicks (d = 0.63); the remaining signals (last active day, click intensity, first score and timing) fall between |d| = 0.11 and |d| = 0.55 and provide complementary information.\n",
     ">\n",
     "> **Practical implication:** An early warning system should prioritize the top-ranked signals. These are the metrics most worth monitoring in the first 28 days to identify at-risk students."
    ]
@@ -873,10 +873,12 @@
    "id": "28",
    "metadata": {},
    "source": [
-    "> **Key finding:** The gap between ghost and active students is enormous and the 95% bootstrap confidence intervals do not overlap. Ghost students — those with zero VLE activity in the first 28 days — have a near-zero completion rate.\n",
+    "> **Key finding:** The gap between ghost and active students is enormous and the 95% bootstrap confidence intervals do not overlap: 4.8% completion for ghosts (CI 4.2-5.4%) versus 54.3% for active students (CI 53.7-54.9%). Ghost students — those with zero VLE activity in the first 28 days — have a near-zero completion rate.\n",
     ">\n",
     "> This is the **strongest single signal** in the dataset: if a student has not clicked on any VLE resource within the first 4 weeks, their probability of completing the course is negligible.\n",
     ">\n",
+    "> **Definition note:** in this notebook \"ghost\" means zero VLE activity in the first 28 days (n = 4685). The BQ5 segment sizing uses a broader operational threshold instead (at most 1 active day and fewer than 10 clicks), which yields a larger segment with a 7.7% completion rate: the strict definition isolates the pure signal, the broad one sizes the intervention target.\n",
+    ">\n",
     "> **Intervention priority:** Ghost students are the lowest-hanging fruit. They do not need better content — they need to be *activated*. In SaaS terms, this is the onboarding gap: users who signed up but never experienced the core product value."
    ]
   },
@@ -891,17 +893,17 @@
     "\n",
     "1. **All 8 early signals show statistically significant differences** between completers and non-completers. With ~32K enrollments, even modest differences reach significance — which is why effect size (Cohen's d) is the primary ranking criterion.\n",
     "\n",
-    "2. **The strongest behavioral predictors** are engagement-volume metrics: active days, total clicks, and engagement decile. These signals capture both frequency and intensity of platform interaction in the first 28 days.\n",
+    "2. **The strongest behavioral predictors** are engagement-volume metrics: engagement decile (d = 0.97), active days (d = 0.90), and total clicks (d = 0.63). These signals capture both frequency and intensity of platform interaction in the first 28 days.\n",
     "\n",
-    "3. **Multiple comparison correction confirms robustness.** Most signals remain significant after both Bonferroni and Benjamini-Hochberg correction, indicating that the associations are real, not artifacts of multiple testing.\n",
+    "3. **Multiple comparison correction confirms robustness.** All 8 signals remain significant after both Bonferroni and Benjamini-Hochberg correction, indicating that the associations are real, not artifacts of multiple testing.\n",
     "\n",
     "4. **Dose-response relationships are monotonic.** More engagement consistently predicts higher completion rates across all signal quartiles. There are no obvious thresholds or diminishing returns — the relationship is graded.\n",
     "\n",
     "5. **Assessment submission is a binary predictor.** Whether or not a student submitted any assessment in the first 28 days is itself a powerful signal, independent of the score achieved.\n",
     "\n",
     "6. **Ghost students are the extreme case.** Zero VLE activity in the first 28 days predicts near-certain non-completion. This is the clearest actionable segment for intervention.\n",
     "\n",
-    "7. **Effect sizes are small-to-medium** by Cohen's conventions. This is typical for behavioral data: individual signals explain a modest share of outcome variance. The practical value comes from combining multiple signals into an engagement score (BQ5).\n",
+    "7. **Effect sizes span Cohen's full scale.** The two leading volume signals reach the \"large\" threshold (d = 0.90 and 0.97), most others sit in the medium band (|d| ≈ 0.5-0.6), and the timing signals remain small (|d| ≈ 0.1-0.2). Individual signals still explain only part of the outcome variance: the practical value comes from combining multiple signals into an engagement score (BQ5).\n",
     "\n",
     "8. **No causal claims.** All findings are associations. Motivated students may both engage more and complete more — engagement could be a proxy for motivation, not a cause of success.\n",
     "\n",

notebooks/05_bq3_demographics_vs_behavior.ipynb

@@ -536,7 +536,7 @@
    "id": "13",
    "metadata": {},
    "source": [
-    "> **Part A Summary:** All 8 demographic features show statistically significant associations with completion after multiple comparison correction. However, the effect sizes are uniformly **small**: Cramér's V values are below 0.15 and |Cohen's d| values for numeric demographics are below 0.2. Demographics tell us *who is slightly more likely* to complete, but they lack the discriminative power to identify at-risk students with confidence."
+    "> **Part A Summary:** All 8 demographic features show statistically significant associations with completion after multiple comparison correction. However, the effect sizes remain **small to modest**: Cramér's V peaks at about 0.15 (highest education 0.150, IMD band 0.134, all others below 0.09) and |Cohen's d| for the numeric demographics stays below 0.3 (studied credits 0.28, previous attempts 0.21). Demographics tell us *who is slightly more likely* to complete, but they lack the discriminative power to identify at-risk students with confidence."
    ]
   },
   {
@@ -614,7 +614,7 @@
    "id": "16",
    "metadata": {},
    "source": [
-    "> **Interpretation:** All 6 behavioral features show statistically significant associations with completion. More importantly, the **effect sizes are substantially larger** than the demographic ones. Several behavioral features reach medium effect sizes (|d| > 0.4), compared to the small demographic effects (|d| < 0.2, V < 0.15).\n",
+    "> **Interpretation:** All 6 behavioral features show statistically significant associations with completion. More importantly, the **effect sizes are substantially larger** than the demographic ones. Every behavioral feature reaches at least a medium effect size (|d| between 0.52 and 0.90), compared to the small-to-modest demographic effects (|d| up to 0.28, V up to 0.15).\n",
     ">\n",
     "> The strongest behavioral signals — engagement volume, activity frequency, and first assessment submission — provide far more discriminative information about eventual completion than any demographic variable.\n",
     ">\n",
@@ -672,7 +672,7 @@
    "id": "18",
    "metadata": {},
    "source": [
-    "> **Part B Summary:** Behavioral features consistently show medium effect sizes (|d| ≈ 0.3–0.6), with the strongest signals coming from engagement volume metrics and first assessment submission. These effects are 2–5× larger than the demographic effects in Part A."
+    "> **Part B Summary:** Behavioral features show medium-to-large effect sizes (|d| ≈ 0.5–0.9), with the strongest signals coming from engagement volume metrics and first assessment submission. These effects are 2 to 4 times larger than the demographic effects in Part A (2.7× on average across comparable metrics)."
    ]
   },
   {
@@ -823,9 +823,9 @@
     "> **The Verdict: Behavior wins, decisively.**\n",
     ">\n",
     "> The comparison reveals a clear pattern:\n",
-    "> - **Behavioral features** (green) have effect sizes 2–5× larger than **demographic features** (blue)\n",
-    "> - Even the *weakest* behavioral signal is comparable to or stronger than the *strongest* demographic signal\n",
-    "> - Categorical demographics (Cramér's V) show uniformly small associations (V < 0.15)\n",
+    "> - **Behavioral features** (green) have effect sizes 2–4× larger than **demographic features** (blue)\n",
+    "> - Even the *weakest* behavioral signal (|d| = 0.52) is stronger than the *strongest* demographic signal (|d| = 0.28)\n",
+    "> - Categorical demographics (Cramér's V) show uniformly weak associations (V ≤ 0.15)\n",
     ">\n",
     "> **What this means for a platform operator:** Demographic profiling has limited predictive value. You cannot meaningfully identify at-risk students based on their age, gender, or education level alone. But monitoring their behavior in the first 28 days provides actionable early warning signals.\n",
     ">\n",
@@ -963,9 +963,9 @@
     "\n",
     "### What we learned\n",
     "\n",
-    "1. **All demographic features show statistically significant but weak associations** with completion. The largest Cramér's V values are below 0.15, and numeric demographic Cohen's d values are below 0.2. With ~32K enrollments, significance is easy to achieve — effect size is what matters.\n",
+    "1. **All demographic features show statistically significant but weak associations** with completion. The largest Cramér's V values peak at about 0.15 (education 0.150, IMD band 0.134), and numeric demographic Cohen's d values stay below 0.3. With ~32K enrollments, significance is easy to achieve — effect size is what matters.\n",
     "\n",
-    "2. **Behavioral features have 2–5× larger effect sizes** than demographic features. Engagement volume, activity frequency, and first assessment submission are far more informative about eventual completion.\n",
+    "2. **Behavioral features have 2–4× larger effect sizes** than demographic features. Engagement volume, activity frequency, and first assessment submission are far more informative about eventual completion.\n",
     "\n",
     "3. **Within every education level, engagement is the swing factor.** High-engagement students outperform low-engagement students regardless of their educational background. The within-group behavioral gap exceeds the between-group demographic gap.\n",
     "\n",