Loading
Three matrices, one dot product, one softmax. The row always sums to 1 — which is a constraint the model must satisfy, not a finding it reports.
Open the instrumentAttention gets explained with metaphors — the model "looks at" a word, "focuses on" a phrase, "decides what matters." Every one of those verbs smuggles in an intention that is not there.
Here is the whole mechanism, with nothing borrowed from psychology.
The decisive idea: attention is a weighted average. The weights come from a dot product and a softmax, and the softmax guarantees they sum to 1 whether or not there is anything worth averaging.
Each token arrives at a head as a vector of 256 numbers. The head immediately makes three different versions of it by multiplying through three learned matrices:
Query and key are the matchmaking pair; value is the cargo. In bert-mini each head works in 64 dimensions, because the 256-wide stream is split four ways so that four heads can specialise independently.
The score between token i and token j is the dot product of i's query with j's key, divided by 8 — the square root of 64. That division is not decoration. Without it the dot products grow with dimension, the softmax saturates, and gradients vanish during training.
Then every row of scores goes through a softmax, which exponentiates and normalises. And that is the step worth staring at.
A row summing to 1 is a constraint, not a finding. The model must distribute exactly one unit of attention somewhere, even when it has nothing useful to say.
Real weights from the shipped forward pass. Pick a word and the row shown is that word's attention across the sentence — it always sums to 1, because a softmax cannot do anything else.
Softmax cannot output "nothing here." It takes whatever scores it is handed and returns a distribution: all positive, summing to exactly 1.
So when you look at an attention row and see weight spread across nine words, you are not seeing the model report nine relationships. You are seeing the model discharge an obligation. It had one unit of attention to distribute and no option to abstain.
This is why "the model attends 0.499 to keys" is a weaker statement than it sounds. The right question is never how much — it is how much compared to what the row would look like if the head had nothing to say.
That comparison has a name here. Divide the largest weight by the uniform share, 1/n. A ratio of 1.0 means the winner is no better than picking at random. On a nine-word sentence, uniform is 0.111, so that peak of 0.499 is 4.5× uniform — a real preference. The same word read through head-averaged rollout peaks at 0.141, which is 1.27× uniform: noise wearing a ranking.
The same word, read through two different heads. Ratio is the largest weight divided by the uniform share: 1.0 means the row says nothing, 9 means one word takes almost everything.
Switch heads in the figure above and the answer changes completely. This is the part that surprises people, and it is the most important structural fact about transformers.
There is no single attention. There are sixteen of them in this model — four heads in each of four layers — and they were never trained to agree. Each head has its own query, key and value matrices, so each one asks a different question about the same sentence. One head may reliably answer "the word immediately after." Another may find the subject of a verb four positions back.
When a visualisation shows you "the attention" for a sentence, it has quietly made a choice among sixteen answers, or averaged them into one. Both are interpretation. Neither is wrong, but you should know which you are looking at — and the next post in this series is about how badly that choice can go if you make it on confidence alone.
The mechanism above is a hundred lines of arithmetic. Three matrix multiplies to get Q, K and V, one more to score every pair, a softmax over each row, then a weighted sum of the value vectors.
No memory. No search. No decision. A token's output is a blend of every token's value vector, mixed in proportions computed from dot products.
Everything transformers are good at emerges from stacking that operation and letting the matrices learn. Which is genuinely remarkable — and it is remarkable because the primitive is this plain, not despite it.
The honest reading of an attention weight is narrow. It tells you the proportion in which one position's value vector entered another position's output, in one head, at one layer. It does not tell you the model understood a relationship, and it does not tell you the weight mattered to the final answer.
That last gap — between attention paid and contribution made — is measurable, and Part 9 measures it.
Next: sixteen heads, and why five of them only ever say "the next word."
More to read
Nine entries in the vocabulary could not survive being typed. The bug was invisible for weeks because the output looked completely reasonable.
Choosing the most confident head picked a positional one every time. It reported “it → was, 0.962” — the answer that head gives for every word in every sentence.
Naive int8 held attention parity at 2.97e-2 — thirty times worse than the bar. Where you spend precision is the whole engineering story.
