← All explainers
Machine Learning
How LLMs Work
From next-token prediction to transformers — text, interaction, and video together.
1. It's all next-token prediction
Everything an LLM does reduces to one trick: given some text, predict what comes next. Repeat, and you get essays, code, and conversations.
2. Attention is how it uses context
To predict well, the model must look back at the right earlier words. That mechanism is attention — and it's the heart of the transformer.
3. Putting it together
Stack attention + feed-forward layers, train on a lot of text, and next-token prediction starts to look like reasoning.