Research archive
A navigable record of the ideas, artifacts, and institutions that formed the modern intelligence stack—reviewed as evidence, not arranged as a reading list.
211reviewed records
22research domains
1843–2025year range
Technical report
The essay lays out three claims: networks contain features (directions that detect meaningful concepts), features are connected by weighted circuits that implement algorithms, and analogous features and circuits recur across models. Working through vision networks, it shows curve detectors and other units that can be read and tested. It set the agenda and vocabulary for the mechanistic-interpretability program that followed.
Technical report
The report decomposes a Transformer into a residual stream that attention heads read from and write to, and shows how heads compose across layers to move and combine information. It introduces tools like the QK and OV circuits and identifies simple mechanisms in small models. This gave mechanistic interpretability a concrete framework for studying language models.
Technical report
By tracking model behavior over training, the authors found that a phase change in in-context learning coincides with the formation of induction heads that look back for a previous occurrence of the current token and copy what followed. They present converging evidence that these heads drive much of a model’s ability to learn from its prompt. It connected a specific circuit to an emergent capability.
Technical report
Using small toy models, the authors demonstrate that when features are sparse a network represents many of them as overlapping directions rather than one-per-neuron, trading interference for capacity. They map when superposition appears and how it organizes into geometric structures. This reframed why interpretability is hard and motivated methods to pull features apart.
Technical report
The work trains a sparse autoencoder on the activations of a small language model and recovers thousands of features that each correspond to a single, nameable concept, resolving the superposition problem in practice. The features are interpretable, can be steered, and generalize. It became the template for the sparse-autoencoder interpretability work now applied to frontier models.