Glossary Entry

Mechanistic Interpretability

The research program of reverse-engineering the internal computations of neural networks into human-understandable components such as features, directions, and circuits.

Interpretability LLMs

Also called: mech interp, mechanistic interp

Seed source: Transformer Circuits Thread

Mechanistic interpretability treats a trained neural network as an object to be reverse-engineered rather than a black box to be benchmarked: the goal is to identify the internal features a model represents, the directions in activation space that carry them, and the circuits that connect them into behaviour. The field’s core toolkit includes linear probes, the logit lens, sparse autoencoders, activation patching, and attribution graphs.

Beyond scientific curiosity, the practical motivation is safety and trust: a faithful map of a model’s internals offers a way to audit what it is actually computing (plans, deceptions, silent intermediate steps) independently of what its visible output claims.