01 / 10
AISep 2026
Mind Map: The Sentence You Type Is Not What the Model Reads
Nine entries in the vocabulary could not survive being typed. The bug was invisible for weeks because the output looked completely reasonable.
7 min read
Project 03 · A ten-part field notebook
Inside Mind Map
A real BERT encoder runs in your browser, and its attention becomes a diagram you can interrogate. Read the tokenizer, the forward pass, the measurement crisis and the negative result — in the order they became understandable.
02 / 10
AISep 2026
Mind Map: What One Attention Head Actually Computes
Three matrices, one dot product, one softmax. The row always sums to 1 — which is a constraint the model must satisfy, not a finding it reports.
4 min read
03 / 10
AISep 2026
Mind Map: Sixteen Heads, and Five of Them Only Say “Next Word”
Choosing the most confident head picked a positional one every time. It reported “it → was, 0.962” — the answer that head gives for every word in every sentence.
5 min read
04 / 10
AISep 2026
Mind Map: Running BERT in a Browser With No Inference Engine
Naive int8 held attention parity at 2.97e-2 — thirty times worse than the bar. Where you spend precision is the whole engineering story.
4 min read
05 / 10
AISep 2026
Mind Map: Downloading Twenty Rows of a Thirty-One Megabyte Table
The word-embedding table is 31,254,528 bytes. A twelve-word sentence needs about 20 KB of it. HTTP Range makes that the only part that travels.
4 min read
06 / 10
AISep 2026
Mind Map: The Weight That Was a Share of Something Else
A median 24% of every displayed row had already been deleted before you saw it. In the worst case, 95%. The arithmetic was right the whole time.
4 min read
07 / 10
AISep 2026
Mind Map: Attention Rollout Is Correct, and Almost Useless
Every rollout row in the sentence sits between 0.935 and 0.971 normalized entropy. That is a uniform distribution wearing a ranking, and it shipped as the default view.
4 min read
08 / 10
AISep 2026
Mind Map: How Do You Test a Model You Cannot Eyeball?
I wrote the code and the tests from the same wrong idea, and they agreed with each other perfectly. Everything was green.
5 min read
09 / 10
AISep 2026
Mind Map: We Tried the Published Fix and Measured What It Was Worth
It halves the problem and does not dissolve it. [CLS] carries 0.63× the value norm of a content token — not the near-zero the paper reports. Four layers is not twelve.
4 min read
10 / 10
AISep 2026
Mind Map: A Diagnostic, Not a Mind Reader
Swap the one adjective that decides what “it” refers to. A reader's answer flips. Not one of the sixteen heads changes its winner.
5 min read
Runs in your browser
Particle Life — Primordial Soup
Thousands of particles. Simple rules. Complex life-like behavior emerges on its own.
Live simulationLaunch Simulation
PhysicsWebGPUSimulation
Visualizing the Invisible
Built with TypeScript and WebGPU — no structure is programmed, emergence happens naturally from a simple attraction matrix
Status:Live
View Project