An intro to transformers, along the papers
Nine ideas, nine papers, one visual language. Everything below animates in the same rendering the live inspector uses on real models — mock data on this page, forward hooks in the app. Read top to bottom, then open the chat and watch every one of these ideas happen on a real model, live.
Attention Is All You Need
Before 2017, sequence models were recurrent: information crawled position by position through a bottleneck state. This paper removed recurrence entirely. Every position emits a query, a key and a value; a position's query is scored against every key (q·k/√d), softmax turns the scores into weights, and the values blend in proportion. Any position can reach any other in one step, and the whole thing is parallel across the sequence during training.
Run several of these side by side with different learned projections and you get multi-head attention — each head free to specialize in its own relation: syntax, coreference, position, copying.
live: click any token in the chat — its real attention arcs; in viz · attn, cells subdivide into per-head fingerprints.
GPT-2 — language modeling is enough
Decoder-only transformers trained on a single objective — predict the next token — turned out to pick up translation, summarization and QA as side effects of getting good at prediction. The engine is autoregression: score the whole vocabulary, sample one token, append it, run again. Earlier keys/values are cached, so each new token costs one forward pass.
Every capability question since — reasoning, tool use, agents — is ultimately a question about what this loop learned to represent.
live: send a message in the chat; the top-5 distribution and the sampled token stream in per step.
The logit lens
A blog post, not a paper — still one of interpretability's most used tools. Because layers only add to the residual stream (§05), you can take the stream at any depth and push it through the final norm and unembedding, as if the model stopped there. The next-token prediction usually forms gradually: noise early, plausible guesses mid-depth, settled late.
Where it settles is a per-token difficulty signal: easy continuations settle early, hard ones keep flipping until the last layers — and some wrong answers never settle at all.
live: the dots on the chat heatmap are real lens flips — red where the prediction settles on the token actually emitted.
Feed-forward layers are key–value memories
Roughly two-thirds of a transformer's parameters sit in the MLP blocks, and this paper explained what they do: the expand projection acts as thousands of keys that match patterns in the residual stream; the contract projection holds values written back when a key fires. Factual recall largely lives here.
The division of labour: attention decides where to look, MLPs supply what is known. A wrong fact emitted with a big mid-depth MLP write is a wrong memory — a different failure from a wrong plan, and it needs a different fix.
live: viz · mlp in the chat is exactly this block's write strength, per layer × token.
The residual stream view
Anthropic's framework paper recast the architecture: don't read a transformer as a pipeline of layers — read it as one shared residual stream that every attention head and every MLP reads from and writes into. x ← x + block(x), dozens of times. Blocks communicate by leaving vectors in the stream for later blocks to find.
Because the structure is additive, interpretability becomes accounting: measure who wrote what, where, and how much. That accounting is precisely what model-graph streams.
live: every heatmap row in the dashboard is one of these blocks; hot cells are big writes into the stream.
Induction heads & in-context learning
Why can a frozen model learn from its prompt? A two-head circuit: one head tags every token with what preceded it; the second — the induction head — finds the previous occurrence of the current token and copies whatever came next. "…The cat sat … The cat" → " sat".
These heads appear in a sharp phase change during training, at the same moment in-context learning ability jumps — one of the cleanest mechanism-to-capability links found so far.
live: repeat a phrase in the chat and watch vertical stripes recur in the attn heatmap — induction-style copying, visible.
Attention sinks
Softmax must sum to 1 — even when a head has nothing useful to say. The spare probability parks on the first tokens: the attention sink. Keep those first tokens plus a sliding window and models stream indefinitely (StreamingLLM); drop them and quality collapses.
Sinks matter for debugging too. In one of our recorded runs, a failed arithmetic answer spent 40% of its attention on the sink token instead of the numbers — the model wasn't looking at the problem at all.
live: select a token in the chat — the "◂ context" share often IS the sink; heads that fire on every token are usually parked there.
Mixture of experts — scale width, not compute
Replace one big MLP with N small experts and a router that sends each token to its top-k: parameter count scales with N, compute only with k. Most frontier models now run on this trade.
The router's choices are structure you can watch — experts specialize by token type, and utilization per expert across a run tells you whether the routing collapsed or spread.
live: on MoE models the chat cut-out panel streams real router weights and per-run expert utilization.
Beyond attention — SSMs and hybrids
Attention's n² cost and ever-growing KV cache motivated a return to recurrence — done right this time. Selective state spaces (Mamba) and gated short convolutions carry a fixed-size state: long context stops being expensive. Hybrids keep a few real attention layers for global recall and use cheap local mixers everywhere else.
model-graph's default model, LFM2.5-1.2B, is exactly such a hybrid — the C rows and A rows in every heatmap are this design decision made visible, layer by layer.
live: compare C-row and A-row writes in the chat — local mixing vs global retrieval, per token.
Papers are the map. The running model is the territory.
Every animation above is mock data in a real renderer. The app runs the same visual language on live forward passes — norms, heads, router flow, lens, arcs, trajectories — for any HF causal LM.