Skip to content

LLM · Foundations

First distinguish training objectives from model architecture, then trace a request through Tokenizer → Transformer → output distribution. Knowing the names is not enough: you need to calculate tensor shapes and KV cache size, and explain causal masks, positional extrapolation, and tokenization boundaries.

Chapters

  1. Chapter 1: How Large Language Models Differ from Traditional NLP
  2. Chapter 2: Transformer Architecture
  3. Chapter 3: MHA Limitations, MQA, GQA, and FlashAttention
  4. Chapter 4: Positional Encoding
  5. Chapter 5: Tokenizers

Return to LLM Topics.