residual-stream

Mechanistic experiments on how LLMs represent and compute arithmetic internally, with strict no-parser controls, reproducible audits, and an interactive article.

Lightweight residual-stream adapters for frozen LLMs: match LoRA at equal parameters, add inference-time steering, and run many behaviors per token on a single model.

Hyperprobe is the Python implementation of the framework proposed in the paper "Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures".

通过权重修改，让开源模型讯飞星火Spark-X2.5-4B彻底相信自己是一只傲娇的AI鲸鱼

Model forensics through geometry: residual-stream projections, activation traces, and attention routing.

Configurable character-level transformer training suite with built-in mechanistic interpretability toolkit — scale to 150M+ parameters and beyond, no ceilings, only hardware limits. Inspect attention weights, hidden states, and head specialisation across all layers. Documented circuit findings included.

An empirical study probing the residual stream vectors of google/gemma-2-2b-it during contrastive corporate finance compliance constraints. Features a native PyTorch layer-tracking notebook and an interactive 4-column Streamlit web dashboard analyzing geometric activation divergence deltas.

PDSF: prediction-anchored decomposition of transformer residual streams — pipeline, prompt sets, and non-regenerable data for Guda (2026)

Model Flow: following texts through a language model's layers

A hypothesis of depth-indexed state regimes in pre-RMSNorm language models.

Reproducing Anthropic's Golden Gate Claude on an open model: extracting a concept direction from Qwen3-1.7B's residual stream with TransformerLens, then steering generation with a single vector addition — no fine-tuning, no gradients.

Measurement harness and raw data: a cheap robust statistic, sigma_med*sqrt(d), predicts the tolerance boundary of additive interventions in language models. Kappa stays in [0.2, 1.2] across 11 checkpoints and 6 architectures, while raw boundaries span ~4600x.

Unsupervised latent-work and residual-stream interference profiling for language models

Experimental Recurrent Residual Stream for GPT-2

Empirical evidence for predictive coding tendencies in the GPT-2 family: residual stream convergence, activation patching, MLP transform analysis, zero-ablation, and logit lens across 7 languages.