Nicolas de Zamaroczy
The UN General Debate (UNGD) is the annual high-level gathering where each UN member state addresses the international community. Held each September at the start of the UN General Assembly session, it is one of the few forums where every country — from the largest powers to the smallest island nations — speaks on equal terms.
The UN General Debate Corpus (UNGDC) provides the English-language text of speeches delivered at the UNGD from 1946 to 2024: 10,952 speeches from over 190 countries. The dataset was originally created by Slava Jankin Mikhaylov, Alexander Baturo, and Niheer Dasandi, with articles published in Research & Politics (2017) and Journal of Peace Research (2024). Their replication materials are available on GitHub and Harvard Dataverse.
This repository applies NLP methods to the UNGDC to explore trends in global diplomacy. Four notebooks cover the full pipeline:
| Notebook | What it does |
|---|---|
| File Reader.ipynb | Ingests the raw speech text files, merges speaker and region metadata, and builds a SpaCy DocBin cache used by all downstream notebooks. |
| Semantic Similarity Analysis.ipynb | Computes cosine similarity between speeches using SpaCy word vectors, and explores what drives linguistic convergence and divergence across countries and time. |
| Conflict Regression Analysis.ipynb | Tests whether dyadic speech similarity predicts interstate conflict, using Correlates of War MID data and a series of logistic and ordinal regression models. |
| Topic Tracing.ipynb | Tracks mentions of specific topics (e.g. terrorism, climate change) across countries and over time using keyword matching and visualization. |
A named entity recognition (NER) notebook — visualizing the UNGDC as a network of countries, people, and organizations — is planned as a future addition.
1. Get the corpus
Download UNGDC_1946-2024.tar.gz from Harvard Dataverse and unzip it to a directory of your choice. Update the path in File Reader.ipynb to point to the folder containing the extracted .txt files.
2. Create the conda environment
conda env create -f environment.yml
conda activate ungdcThis installs all dependencies, including the en_core_web_lg SpaCy model.
3. Run the notebooks in order
Run File Reader first — it produces the SpaCy cache that the other notebooks depend on. Semantic Similarity Analysis and Topic Tracing can then run independently of each other. Conflict Regression Analysis depends on the similarity matrix produced by Semantic Similarity Analysis.
The computationally expensive steps (SpaCy processing, pairwise similarity, regression dataset assembly) are cached to disk. Once computed, subsequent runs load from cache automatically. Set FORCE_REPROCESS = True in the relevant cell to recompute from scratch.