Echoes of the Crash: A Structural and Semantic Analysis of the Great Depression in American Newspapers

This repository contains the replication code for "Echoes of the Crash: A Structural and Semantic Analysis of the Great Depression in American Newspapers," a text-as-data analysis of 13.2 million newspaper articles from 1925 to 1934.

The project structure is based on the JSLab Template by Sameer Nair-Desai and Juan Carlos Cisneros.

🛠 Prerequisites

This workflow is designed for Linux and macOS systems. It relies on the following software, which will be installed locally by the setup script:

• micromamba for isolated package management
• tectonic for reproducible LaTeX compilation

✅ Quickstart

1. Run the setup script

   bash setup.sh

   This will:

   • Install micromamba locally (if not present)
   • Build the dedicated Python environment defined in environment.yml

2. Restart your shell Close your terminal and open a new instance to ensure the conda paths are correctly loaded.

3. Activate the environment You must activate the project environment before running any scripts:

   micromamba activate CrashText

4. Run the full analysis To execute the entire pipeline—from data cleaning and topic modeling to figure generation and paper compilation—run:

   bash run_all.sh

   This will execute the make.sh script in every module sequentially.

📂 Repository Structure

• 1_data/: Data ingestion and cleaning pipeline.

  • source/: Scripts to fetch and clean raw text (clean_data.py) and audit geographic coverage (audit_data.py).
  • output/: Stores the cleaned parquet files, the HuggingFace Arrow dataset, and descriptive audit plots (map_density_paper.png).

• 2_analysis/: Core computational analysis (Topic Modeling, RDD, GAM, SVO).

  • source/: Scripts for LDA (train_topics.py), structural breaks (structural_break.py), robustness checks (robustness_gam.py), and supervised learning (train_structural_classifier.py).
  • output/: Generated figures (figures/), LaTeX tables (tables/), and trained models (models/).

• 3_paper/: Paper compilation.

  • source/: LaTeX source files (paper.tex, references.bib).
  • output/: The final compiled manuscript (paper.pdf).

• lib/: Shell scripts and language-specific wrappers that power the build system.