Predicting the next word
A transformer reads "The cat sat on the" from the bottom up. The words become vectors, are refined layer by layer, and the top produces a probability for every possible next word.
Tokens
First the text is split into tokens, here one per word. Each token is just an ID number in the model's vocabulary.
Embeddings
Each token ID is looked up in a table and becomes a long vector of numbers that captures its meaning. A position signal is added so the model knows the word order.
Self-attention
In each layer, every word looks back at the words before it and decides how much each one matters. The last "the" pays most attention to "on", "sat" and "cat" (thicker lines) and mixes their information into its own vector.
Stacked layers
The same kind of block is repeated many times. Lower layers pick up simple relations like grammar, higher layers build richer meaning: a cat is sitting on some surface.
Next-word probabilities
The top vector at the last position is scored against the whole vocabulary, and softmax turns the scores into probabilities. "mat" is the most likely next word at 42%.