Saturation-Driven Architecture for Qualitative Interview Campaigns — prototype

This repository is the software artefact accompanying the MSc Software Engineering thesis Saturation-Driven Architecture for Qualitative Interview Campaigns (University of Amsterdam, 2026). It contains the standalone Python prototype used to validate the architecture, together with the synthetic and anonymised real corpora the experiments consume.

The prototype implements a Saturation Signal Producer (SSP) that tracks thematic saturation in real time from interview transcripts, a 28-feature multinomial logistic-regression breadth classifier, and a post-hoc reinforcement-learning harness for evaluating exploration–exploitation strategies. Prototyping concluded at iteration 2.8, which is the canonical deployment; earlier iterations are kept for provenance.

Repository layout

prototype/
  src/                 # runtime package; src/v2/ is the canonical iter-2.8 path
  scripts/             # per-iteration reproduction pipelines (iter5 … iter9)
  tests/               # regression suite (178 tests)
  data/
    real_corpus/       # anonymised, English-translated WhatsApp interview corpus
    synthetic_corpus/  # synthetic corpora: gradual, fast_saturating, mixed_rate
    categories/        # inductive-code categories used by the extractor
  output/              # committed evidence for every iteration (v1 and v2)
  requirements.txt     # dependency list
  README.md            # prototype-level notes

Quick start

The prototype is a self-contained Python 3 package rooted at prototype/. Create a virtual environment and install the dependencies:

cd prototype
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

Run the offline self-check and the regression suite (no API key required):

.venv/bin/python3 -m scripts.iter8_run.formula_probe --self-check
.venv/bin/python3 -m pytest tests/

The regression suite reports 178 passing tests. The steps that call a language model (the synthetic sweep and the carbon-corpus reference run) require an OPENAI_API_KEY; see REPRODUCTION.md for the full recipe and its cost.

Headline results

On held-out leave-one-scenario-out evaluation the breadth classifier reaches 63.7 / 73.6 / 77.2 % alignment on synth-gradual, synth-fast, and synth-mixed; in-distribution deployment medians over three seeds are 72.9 / 89.6 / 75.0 %. Cross-distribution deployment on the carbon corpus requires per-corpus retraining; this generalisation gap is documented as future work in the thesis.

Reproduction

REPRODUCTION.md specifies the environment, the artefact layout, and the command sequence that regenerates every committed artefact under prototype/output/v2-iteration-8/. The full run costs approximately US$6 in language-model API calls and about 15 minutes of sequential wall-clock time; the probe phase is offline and incurs no API cost.

Data

DATA_STATEMENT.md documents the provenance of both corpora and the anonymisation applied to the real WhatsApp interviews. The raw transcripts and the re-identification key are not included in this repository.

Licence

This repository, including the source code and the anonymised corpus under prototype/data/, is released as open source under the MIT licence in LICENSE. See DATA_STATEMENT.md for the provenance and anonymisation of the corpus.