One concept
Each exhibit is designed around one mental model and one observable learning outcome.
Interactive explanations of language models
Don’t just read it. Peek inside.
See how language models work through focused, interactive visual explanations. No setup. No long video. Just one concept in one HTML file.
Conceptual preview · Attention
animal
street
crossed
it
The animal crossed the street because it… Which earlier word matters?
Concepts
The collection so far.
Concept 01 · Start here
Tokenization
Watch the real GPT-2 tokenizer cut text into tokens — as guided slides, a single-screen studio, or a free-scroll playground.
Concept 02Embeddings
Where integer IDs become vectors the model can compute with — orbit 4,000 real GPT-2 word vectors in a 3D studio.
Concept 03Positional Encoding
Give vectors a sense of order — rotate query and key dimension pairs, then shift both positions and watch relative distance survive.
Concept 04Attention
How tokens look at each other — hover real GPT-2 attention maps across 12 layers and 12 heads.
Concept 05Logits & Sampling
How the model turns 50,257 scores into one token — bend real GPT-2 logits with temperature, top-k, and top-p, then throw the dart.
Concept 06KV Cache
Decode one token at a time, compare full-history K/V recomputation with append-only reuse, and see how MHA, GQA, and MQA change cache memory.
Training collection
See how the model learns.
Training alternates between computing a prediction and changing the parameters that produced it. Explore the update rule, trace the forward computation it improves, then follow the error as it travels backward.
Training 01
Gradient Descent
Follow a point down a computed loss landscape, inspect one update, and push the learning rate from cautious convergence into overshoot and divergence.
Training 02Forward Pass
Shape a tiny hidden state, follow every weighted contribution through ReLU, and watch three token logits become one probability distribution.
Training 03Backpropagation
Send the cross-entropy error backward through the same tiny MLP — p − y at the logits, through the ReLU gate, to a gradient for every weight.
The format
Small, portable experiments for big ideas.
One HTML
Everything needed to learn and interact lives inside a single offline-friendly file.
Learn by doing
Change inputs, replay steps, and see cause and effect instead of memorizing a diagram.