The logit lens, live: watching a 1.2B model settle on its answer
The logit lens is one of interpretability's simplest, most striking tools: take the residual stream after each layer, push it through the model's final norm and unembedding matrix, and ask — if the network stopped here, what would it predict?
model-graph computes it for every generated token and renders it two ways: dots on the heatmap wherever a layer's prediction changes (red where it flips to the token actually emitted — the settle layer), and a full per-token ladder in the inspector.
A real ladder
Asking LFM2.5-1.2B "why is the sea salty?", the token
" because" looks like this on its way down the network:
L00 "ty" 100% ← echo of the current token L01 " salt" 25% ← topic surfaces L02 "盐" 7% ← salt, in Chinese L04 " salinity" 0% L07 " undert" 1% ← noise floor L10 " because" 79% ← settles L12 " because" 95% L15 " because" 100%
Semantic content appears in the middle of the network in whatever surface form is handy (a Chinese token, a morpheme), and the final layers mostly commit and calibrate.
Settle depth is a difficulty meter
Echoed and grammatical tokens settle in the first third of the layers; genuine choices stay contested. That's per-token difficulty, measured rather than guessed — and it's the intuition behind early-exit and speculative decoding: if the lens settles at L10 of 16, the last six layers were mostly polish. Watching entropy collapse at the same step the evidence token arrives closes the loop: you can see where in depth and when in time the model made up its mind.