THE MODEL OBSERVATORY
Give it a memory. Trace the information. Cut a head and see what it forgets.
Here, red = fox binds a colour to an animal; the final red asks it to recall the animal. This model uses a controlled 15-token vocabulary.
red blue green gold pink cyan · fox owl cat elk bee yak · = ;
A <bos> token is added automatically. Training uses 2–4 unique colour/animal pairs separated by semicolons, followed by a known colour alone. Other valid token sequences run too, but may fail. Maximum: 31 input tokens.
Loading the tiny model from this site…
FIELD NOTES
Red has no fixed relationship to fox. Every training example randomly reassigns colours to animals. If you replace fox with owl, the model must read the current context rather than rely on a memorized red–fox association. This controlled task is called associative recall. Run the first two examples: the structure barely changes, but the correct answer does.
Each token becomes 32 numbers plus a position vector: the leftmost matrix. In each block, attention mixes information from earlier positions, then an MLP transforms each position; both updates are added to the residual stream. Each plane is a token × channel matrix. Mint is positive, coral is negative, with one shared scale across all three planes. The white outline follows your selected token.
The paths show softmax(QKᵀ/√8): how much a head reads from each position. A large weight does not establish that a position determines the answer; values, output projections and other heads matter too. Ablation zeros one head’s AV output and recomputes later layers. A probability drop establishes an effect of this intervention on this input, not universal importance across tasks.
This is a two-block, four-head, 32-dimensional pre-norm causal Transformer trained from scratch for this site, with learned positional embeddings, GELU MLPs and a next-token output head. It is not a chatbot and has no natural-language-understanding evaluation. Auxiliary attention supervision explicitly guides L1H1 to read an animal’s colour and L2H1 to locate the answer animal. A training-only colour probe also guides the first block to retain the binding. This is a guided teaching model, not evidence of an emergent circuit. Your browser loads the weights from this site and computes every layer in JavaScript. The motion traces those results; it does not depict actual hardware timing.