KV Cache

Autoregressive decoding revisits the same history one token at a time. Keep every past key and value, append one new pair, and stop projecting the same tokens again.

1 exhibit Computed decoding trace Offline, single file

StudioStart hereComputed
KV Cache Studio

Decode one token at a time while two synchronized tracks compare full-history recomputation with a growing K/V cache. Then switch MHA, GQA, and MQA to see the memory tradeoff.
Single screen · DOM + MathML→

KV Cache

KV Cache Studio