Foundations
Foundations
Essential concepts for understanding LLMs
7 Articles
23 Papers Referenced
~56 min Reading Time
Recommended Reading Order
1
Tokenization: How Models See Text
Tokens, vocabularies, BPE intuition, and engineering trade-offs.
2
Attention: Choosing the Relevant Context
Attention weights, Q/K/V, multi-head self-attention, and compute costs.
3
Sampling and Decoding: From Probabilities to Text
Temperature, top-k, top-p, and the choices made at inference time.
4
Transformer Architecture: The Skeleton of Modern LLMs
The Transformer block, decoder-only LLMs, and major design trade-offs.
5
Embeddings: Putting Discrete Symbols into Continuous Space
Word vectors, contextual representations, and the foundation of semantic retrieval.
6
Positional Encoding: Where Does Order Come From
Absolute positions, relative positions, and RoPE.
7
Why LLMs Emerge Abilities
A careful look at scale, data, compute, and emergence.