RoPE Studio — Peek LLM

Studio · Computed RoPE geometry

Fig. 1

Relative distance

Start here. Move K and Q apart; each dimension pair turns at a different speed.

Key position n5

012

Query position m6

012

RoPE rotates every 2D pair by position × frequency before Q meets K.

Select a plane here or tap it in the visualization.

Both absolute angles change. Their relative phase does not.

query Q at position m key K at position n relative phase between them

Computed teaching vectors · three representative dimension pairs

Sequence positions $Δ = m - n = + 1$

Relative rotation. The dot product can now respond to distance. Tap a plane to inspect its frequency.

+1relative offset Δ

0.845mean paired cosine

6.000fast Q angle · rad

5.000fast K angle · rad

The score is a compact teaching measure across the three displayed pairs, not a trained model attention weight.

d_{0}, d_{1}

Fast rotation

$θ_{0} = 1.00$ rad / position

relative phase +1.000 rad

Fast pair. Nearby positions separate quickly, making local order easy to distinguish.

{(R_{m} q)}^{T} (R_{n} k) = q^{T} R_{n - m} k

before shift $m - n = 1$

after both +1 $(m + 1) - (n + 1) = 1$

Same relative phase · same score

Core idea. RoPE does not store distance in one vector. Distance appears when a rotated query is compared with a rotated key.