Forward Pass
A model turns activations into new activations by repeating a small set of operations. Trace one tiny MLP from weighted sums through ReLU, token logits, and softmax probabilities.
Training 02 · Forward Pass
A model turns activations into new activations by repeating a small set of operations. Trace one tiny MLP from weighted sums through ReLU, token logits, and softmax probabilities.