Self-Attention Explorer

Visualize how Transformers process language by weighing the relevance of different words.

Controls

Mode

Temperature ($\tau$): 1.0

Vector Dim ($d_k$): 2

$$ \text{Attention}(Q, K) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) $$

Drag the colored bars to adjust vectors.

Weighted Sum of Values ($O_i = \sum w_{ij} V_j$)

The output is a blend of all Value vectors, weighted by relevance.