An interactive playground

Attention
Visualization

A Vision Transformer turns every patch into a query, a key, and a value. This playground lets you watch how those Q · K · V spaces take shape — layer by layer, head by head — and how a query's attention flows to its keys and on to their values.

DINOv3-L · 24 layers · 16 heads · 261 tokens (5 prefix + 256 patches)

explore ↓
01 — interactive

Attention Visualization

I wanted to see how each token's query, key, and value get projected into their own spaces, and how that geometry ties back to attention. So I lay out the per-(layer, head) Q·K·V in 3D and trace a query to its keys and on to their values. Doing this makes one thing clear: every layer and every head learns its own distinct projection — and that is exactly what gives each one a different role, and why the attention map changes from one to the next.

Layer 0
Head 0
Query —
Q K V │ — Q·K attention ┄ K→V
input image
query → patch attention
Attention map
query
key
attention weight low high
Drag the Query slider or click a Q node to see its attention.
02 — features

Layer Visualization

To probe what each layer itself is doing, I additionally took the features after every block's MLP and projected them to PCA-RGB. Reading them across depth, I can see that what each layer looks at changes layer by layer — early layers latch onto position and low-level texture, deeper layers organize into object parts and semantics. Pick a sample below to see all 24 layers (left → right, top → bottom).