rows = layers (depth, L00 first) · columns = generated tokens (time). each cell is the L2 norm of a block's output for that token — how strongly the block wrote into the residual stream at that step.
colors normalize per row against that layer's running max: absolute norms grow with depth, so raw values would drown the early layers. compare color along a row (time), not down a column.
norm of the MLP / feed-forward output — the block that transforms features within one position.
why this signal: MLPs hold most of the parameters and act like key→value memories, so this is the closest single proxy for "how much knowledge got injected here". look for: bursts on content words vs quiet function words · hot mid-depth rows where recall happens · what happens on the eos column.
what it teaches: the "MLPs as key→value memories" result (Geva et al. 2021) becomes visible — factual tokens trigger big writes where a lookup fires, function words barely disturb the stream. If a wrong fact is emitted with a big mid-depth MLP write, the model retrieved a wrong memory, not a wrong plan — a genuinely different failure to fix.
norm of the attention output — on C rows (hybrid models
like LFM2.5) it is the gated short-conv mixer instead.
why this signal: the token mixer is the only place
information moves between positions — spikes mean this token
pulled hard on context (pronouns, closing quotes, repeated names).
compare A vs C rows to see the hybrid's
division of labour: conv mixes locally, global attention does
long-range retrieval.
in this mode A cells subdivide into
per-head panels (2×n grid, head h0 top-left, row-major): the
interior is the head fingerprint — each panel normalized
against the cell's hottest head, so the pattern shows which
heads did the work — while the framing border keeps the aggregate
attn norm, i.e. how much work. hover a panel to identify the
head; recurring fingerprints across columns = specialised heads.
what it teaches: heads specialise. Vertical stripes that recur when text repeats are induction-style behavior (copying earlier patterns forward); a head that fires on almost every token is often just parking attention on the first tokens — the attention sink. Distinguishing "moved information" from "parked attention" is the core skill of reading attention data.
server runs also capture where each token looked: select any A cell and its top attention sources draw as arcs over the token strip (blue ◂ = mass on the prompt/context); the cut-out lists them with weights. per-token uncertainty (entropy + top-1 margin) rides on every frame — the raw material for reasoning analysis: watch entropy spike at forks and attention snap to the tokens that resolve them.
norm of the full hidden state after the layer — the accumulated representation that every block writes into.
why this signal: the residual stream is the model's working memory. growth with depth is expected (the harness checks L0 < L15), so the information is in deviations — sudden per-token jumps (one layer dominating), unusually flat columns, odd eos behaviour.
what it teaches: the residual stream is the shared bus every block reads from and writes into — features coexist superposed in the same dimensions. A layer whose write suddenly dominates the stream is where a decision got made; chase it with the lens ladder and the attention arcs on that column.
each token's residual vector (2048-d) plotted as one labeled point,
projected into 3d (or 2d — press d) and connected in
generation order. the blue dotted trace
is the full context — every prompt position's prefill state,
linked to where generation takes over. the right-edge
minimap shows all layers' trajectories at once — click a
thumb (or ↑/↓) to switch depth. drag rotates · wheel / pinch zooms ·
t cycles labels auto→all→off (auto hides colliding
labels, recent tokens win) · c toggles context · click
a point for its cut-out.
why this projection: a fixed random orthonormal basis (seeded) instead of PCA — no refitting while streaming, so the shape never drifts under you and runs are comparable. relative geometry survives approximately: nearby points = similar hidden states. look for: drift as context accumulates · loops on repeated structure (lists!) · jumps at topic shifts · run starts ringed in ink, eos in red · compare early vs late layers (↑/↓) to watch depth untangle the space.
what it teaches: LM hidden states are famously
anisotropic — they occupy a narrow cone, and the cloud's
shape shows it. Special tokens (<|im_start|>)
sit as huge-norm outliers: attention sinks in embedding form.
The step replay (space) shows the path being walked: tight
loops = the model cycling similar states (lists, repetition),
long jumps = genuine semantic movement. Generation orbiting the
context cluster = grounded; drifting far from it can be
creativity — or hallucination.
each layer's residual is decoded through the final norm + unembedding as if the network stopped there (the "logit lens"). a dot = this layer's top-1 prediction differs from the layer below; red = it flipped to the token that was actually emitted — the settle layer.
why: settle depth is a per-token difficulty meter — echoed and grammatical tokens settle early, semantic choices settle late. click a cell for the full ladder.
what it teaches: depth is compute. If the lens settles at L10 of 16, the last six layers mostly polished an already-made decision — the intuition behind early-exit and speculative decoding. Watch a hard token stay contested to the final layers while an easy one locks in a third of the way down: per-token difficulty, measured rather than guessed.
the emitted token above each column (red = eos, ↵ = newline). hover for the top-5 alternatives — this is the actual sampling distribution, so at T=0 the winner saturates; raise T to see real spread.
what it teaches: entropy in bits ≈ log₂ of the number of plausible continuations — 1 bit ≈ a coin flip, 0.01 bits ≈ certainty. Entropy spikes mark decision forks; the margin (p₁−p₂) shows decisiveness. On reasoning chains, watch entropy collapse exactly when the evidence token arrives — or fail to, which predicts a shaky answer.
server api runs the hooked python model — full internals (norms, lens, projections). browser engines run a curated small model entirely in your browser: weights stream directly from huggingface.co and inference runs on WebGPU (wasm fallback) via transformers.js.
browser runs DO feed the viz with what is honestly observable client-side: the token strip, run boundaries, and real top-k prediction probabilities (tapped from the logits each step) — hover the strip or click a token. per-layer cells stay pale because a compiled ONNX graph exposes no hidden states — no place to hang hooks.
the observable browser model closes most of that gap: its graph was re-exported (export_observable.py) with every layer's residual stream as a declared output, so resid norms, the full embedding trajectory + context trace, and predictions are all computed client-side from real tensors. mlp/attn write norms, per-head norms and the logit lens still need the server engine.
everything streams from forward hooks over
ws://localhost:8765 (v1 protocol) — the same feed the
full dashboard consumes.
clicking a mode card above switches the heatmap to it.