Positional Encoding

A token vector has meaning but no address. RoPE rotates pairs of query and key dimensions so their dot product can feel relative distance — without attaching a permanent position label.

1 exhibit Computed RoPE example Offline, single file

StudioStart hereComputed
RoPE Studio

Move a query and key through three rotating dimension pairs, inspect their relative phase, then shift both positions together and watch the comparison remain unchanged.
Single screen · 2.5D Canvas + MathML→

Positional Encoding

RoPE Studio