A mechanistic interpretability study that locates the induction circuit in GPT-2 small and verifies it five independent ways — attention patterns, ablation, direct logit attribution, activation patching, and the raw weight matrices.
An induction head implements a two-token lookup. Seeing token X, it searches the context for
the previous occurrence of X and predicts whatever followed it: [A][B] ... [A] -> [B]. It is one
of the best-understood mechanisms behind in-context learning, and it emerges in essentially every
transformer trained on sequence data.
Built with TransformerLens. Every notebook in this repository is committed with its outputs, so all figures and numbers below are readable without running anything.
The circuit is L4H11 (previous-token) → {L5H5, L5H1, L6H9, L7H10, L7H2} (induction) → logits, composed through the induction heads' keys.
| Claim | Method | Evidence | Notebook |
|---|---|---|---|
| GPT-2 does in-context lookup | per-position loss on [BOS, rand, rand] |
loss 13.25 → 0.48 at the repeat boundary (27x) | 2 |
| Five heads attend like induction heads | induction-stripe score over all 144 heads | 0.82–0.91, vs <0.51 for every other head | 2 |
| They are necessary | zero and mean ablation | second-copy loss 0.48 → 5.4 (11x); both ablations agree | 3 |
| An upstream head feeds them | ablate L4H11, which is not an induction head | 0.48 → 0.71; control head L3H0 only reaches 0.54 | 3 |
| They write the answer | direct logit attribution | L7H2, L7H10, L6H9 lead the direct logit contribution | 3 |
| They causally carry the signal | activation patching, clean vs corrupted | same heads top the recovery ranking | 3 |
| The wiring is in the weights | K-composition score | 0.087–0.104 to the induction heads vs 0.048 baseline | 3 |
| The induction head copies | OV circuit W_E W_OV W_U (toy model) |
77% top-1 copying accuracy vs 0.002% chance | 4 |
| The prev-token head's behaviour is positional | QK circuit W_pos W_QK W_posᵀ (toy model) |
clean i-1 stripe, mean ≈ 1.0 |
4 |
| Composition is specifically through keys | K vs Q vs V composition (toy model) | K 0.112 vs 0.039 baseline; Q and V sit at baseline | 4 |
Two negative results are reported rather than hidden, because both are informative:
- Single-head zero ablation is a poor localiser. Sweeping all 144 heads individually ranks layer-0 heads (L0H8, L0H9) alongside the real circuit. Deleting an early head corrupts the representation every later layer reads, and zero is off-distribution, so the metric partly measures collateral damage. DLA and patching — which respectively skip downstream effects and keep the model on distribution — are much cleaner on identical inputs.
- GPT-2's weights are not legible in the token basis. The same OV copying circuit that scores 77% on a clean 2-layer model scores under 1% on GPT-2, and two of the five heads do not clear chance. LayerNorm, MLPs and superposition break the linear token→logit path the analysis assumes. GPT-2 provably runs an induction circuit, yet its weights will not reveal it — which is a fair summary of why weight-level interpretability is harder than behavioural interpretability, and roughly what work on superposition and sparse autoencoders exists to fix.
| # | Notebook | Question | Key techniques |
|---|---|---|---|
| 1 | 01_attention_patterns.ipynb | What do GPT-2's 144 heads do? | run_with_cache, circuitsvis, collapsing a [dest, src] pattern to a per-head score |
| 2 | 02_induction_detection.ipynb | Which heads do induction? | repeated random tokens, per-position loss, the induction stripe at offset -(T-1) |
| 3 | 03_causal_analysis.ipynb | Is the circuit real, and where does the signal flow? | zero/mean ablation, direct logit attribution, activation patching, K-composition |
| 4 | 04_qk_ov_circuits.ipynb | What do the weights alone say? | FactoredMatrix, OV copying circuit, positional QK circuit, K/Q/V composition |
Read them in order — each notebook's limitations are what motivate the next one. Notebook 4 switches from GPT-2 to a 2-layer attention-only model, for the reason given above.
Shared helpers live in induction_utils.py: model loading, repeated-token generation, per-head scoring, ablation and patching hook builders, weight-circuit products, and the layers × heads heatmap used throughout.
The probe input. All detection work uses [BOS, rand, rand] — T=50 uniformly random token ids,
repeated verbatim. Random tokens mean no learned linguistic knowledge helps on the first copy, so any
loss drop on the second copy is in-context learning in isolation.
Where an induction head must look. For a destination d in the second copy holding token X,
the first occurrence is at d - T and the token to predict sits at d - T + 1. So the head attends
from d to d - T + 1 — a stripe at constant offset src - dest = -(T - 1). Averaging attention
along that diagonal gives one score per head.
Patching design. clean = [BOS, seq, seq] versus corrupt = [BOS, seq_bad, seq]. Both share an
identical second copy, so the prediction targets match exactly and only the mechanism differs.
Recovery is reported as the fraction of the clean/corrupt metric gap that one patched head restores.
Why induction heads look inert on prose. On an ordinary sentence they have no repeated pattern to look up, so softmax forces their attention onto BOS as a no-op slot. In notebook 1 the top three attention-sink heads are L5H1, L7H2 and L7H10 — the very heads notebook 2 identifies as induction heads. Judging a head from one sentence is how you miss the interesting ones.
pip install -r requirements.txt
jupyter labNotebook 4 downloads a small model from HuggingFace on first run
(callummcdougall/attn_only_2L_half, from the ARENA course).
Verified on Python 3.13, PyTorch 2.11 + CUDA 12.8, TransformerLens 3.x, on a CUDA GPU. It will run
on CPU, but the two 144-head sweeps in notebook 3 are slow there. All notebooks fix
torch.manual_seed(0) and disable gradients, so the numbers reproduce exactly.
- Elhage et al., A Mathematical Framework for Transformer Circuits (2021) — QK/OV factorisation, composition scores
- Olsson et al., In-context Learning and Induction Heads (2022)
- TransformerLens — the hook and cache API used throughout
- ARENA — source of the 2-layer attention-only model in notebook 4