Argumentation Mining with BERT

A sentence-level classifier that tags argumentative components in US presidential campaign debate transcripts. Fine-tunes bert-base-uncased on a labeled corpus of debate sentences and produces both predictions and token-level attributions (which words drove the model's decision).

The notebook walks through the full pipeline: data loading, label encoding, train/val/test split, tokenization, class-weighted fine-tuning, evaluation, and explainability with Integrated Gradients (Captum).

Task

Classify each debate sentence as one of three labels:

Label	Meaning	Example
Claim	An assertion that something is true / should be the case	"We need to invest in renewable energy."
Premise	A reason or evidence supporting a claim	"Last year, solar jobs grew by 5%."
O	Non-argumentative (procedural, small talk, etc.)	"Thank you for hosting this tonight."

Dataset

Source: Yes, we can! Mining Arguments in 50 Years of US Presidential Campaign Debates (Haddadan et al., ACL 2019).

The CSV shipped here (sentence_db_candidate.csv) is the candidate-debate subset. It has 18 columns; the notebook uses:

• Speech — the sentence text (input to BERT)
• Component — the label: Claim, Premise, or O

Class distribution (29,532 samples after dropping missing rows):

Class	Count	Share
Claim	11,964	40.5%
Premise	10,316	34.9%
O	7,252	24.6%

The class imbalance (especially the underrepresented O class) is why the notebook applies balanced class weights during fine-tuning.

Methodology

• Backbone: bert-base-uncased from Hugging Face Transformers, with a 3-class classification head.
• Tokenization: BertTokenizer (WordPiece, max length 128 by default in the notebook's Dataset class).
• Class weighting: compute_class_weight(class_weight="balanced") — the O class gets the highest weight (≈1.36), Claim the lowest (≈0.82), Premise ≈ 0.95.
• Loss: CrossEntropyLoss(weight=class_weights) — applied via a WeightedTrainer subclass that overrides the compute_loss method of HF's Trainer.
• Training: TrainingArguments with EarlyStoppingCallback(patience=2). ~2 epochs in the included run.
• Explainability: Integrated Gradients via Captum — produces per-token attribution scores for a chosen target class, showing which words pushed the model toward that prediction.

Results

From the classification report on the held-out test set (15% of the data):

Class	Precision	Recall	F1	Support
O (non-argumentative)	0.67	0.64	0.65	1,088
Claim	0.65	0.71	0.68	1,795
Premise	0.66	0.62	0.64	1,547

(Exact numbers may vary slightly if you re-run, but the model's F1 is in the 0.64–0.68 range per class — roughly balanced, no class collapsing.)

For the Integrated Gradients step, the notebook computes per-token attribution scores across the test set and reports the most-attributed tokens per class:

• Class 0 (O): 11,023 tokens attributed across the test set
• Class 1 (Claim): 35,326 tokens
• Class 2 (Premise): 28,778 tokens

Files

bert_argumentation_mining.ipynb   # the analysis notebook (Colab-ready)
sentence_db_candidate.csv         # debate sentences + labels (29,621 rows)
LICENSE                           # MIT

How to run

The notebook is designed for Google Colab. Easiest path:

1. Upload both files to Colab (or open from Google Drive / GitHub).
2. Run the cells in order.

The first two code cells install dependencies (transformers datasets scikit-learn, then captum). The rest reads the local CSV, fine-tunes BERT, evaluates on the test set, and runs the Integrated Gradients pass.

To run locally instead:

pip install transformers datasets scikit-learn captum torch
# then open the notebook in Jupyter

Training takes about 5 minutes per epoch on a Colab GPU. The Integrated Gradients pass on the full test set takes ~8 minutes (the notebook processes ~9 examples/second on Colab's CPU fallback path).

The fine-tuned model + tokenizer are saved to ./bert_argument_model/ and zipped into bert_argument_model.zip for download.

Project

• Source / issues: github.com/zaheralkaei/bert_argumentation_mining
• Author: Zaher Alkaei
• Dataset credit: Haddadan et al., Yes, we can! Mining Arguments in 50 Years of US Presidential Campaign Debates, ACL 2019.

License

MIT. See LICENSE. The dataset follows its own license — please refer to the ElecDeb60To16/Dataset repository for its terms.