The math I need for LLM inference engineering, and nothing more. I'm learning one topic at a time. When I finish a topic, its lesson is added here and linked below.
Each lesson has the same parts: why it matters for inference, a real-life example, a small worked example, the inference connection, key takeaways and a quick check with answers.
✅ = done, 🟡 = in progress, ⬜ = to do, ⭐ = very important
Progress: 34 done, 3 left, 37 total
- ✅ Vectors and dot product ⭐
- ✅ Matrices and matrix multiplication ⭐
- ✅ Matrix shapes and counting FLOPs ⭐
- ✅ Linear transformations (+ ReLU filter)
- ✅ Transpose and identity matrix
- ✅ Tensors:
[batch, tokens, hidden]⭐ - ✅ Splitting matrices across GPUs + Megatron trick ⭐
- ✅ Rank and low-rank (LoRA), already known
- ✅ SVD (idea only)
- ✅ Bits and bytes
- ✅ Binary integers (INT8, INT4)
- ✅ Floating point: FP32, FP16, BF16 ⭐
- ✅ Quantization basics: scale, round, dequantize ⭐
- ✅ Rounding errors and overflow
- ✅ FP8 ⭐
- ✅ What gets quantized: weights, activations, KV cache ⭐
I learned the KV cache properly before going on. I already know transformers, so attention isn't repeated here.
⚡ Quick revision: the whole KV cache on one page
- ✅ How an LLM writes: one word at a time, looking back at all old words
- ✅ Attention basics: Q, K, V
- ✅ Without cache vs with cache: make K and V once and save them (about 500× less work for 1,000 words)
2. Core ⭐ ✅
- ✅ Why we save K and V, but not Q
- ✅ Prefill vs decode
- ✅ KV cache size formula:
2 × layers × KV heads × head size × bytes × words × users - ✅ Why decode is slow: limited by memory speed, not maths
- ✅ MHA → MQA → GQA
- ✅ FP8 KV cache
- ✅ Sliding window
4. Managing it on a real server ⭐ ✅
- ✅ The waste problem (fragmentation)
- ✅ PagedAttention (vLLM): small pages, given only when needed, stored anywhere
- ✅ Continuous batching: when one user finishes, the next takes their seat right away
- ✅ Prefix caching: reuse K,V of the same start (saves the question, not the answer)
- ✅ Offloading to CPU memory: move unused K,V out of the GPU, bring it back later
- ✅ Eviction: which old notes to throw away (LRU: used longest ago goes first)
- ✅ Disaggregation: prefill and decode on different GPUs (the KV cache is sent between them)
- ✅ Attention without a cache in PyTorch (code)
- ✅ Attention with a cache, then compare the speed: same answer, 2–24× faster (code)
- ✅ A simple paged KV cache (code)
- ✅ Exponent (eˣ) and logarithm (log)
- ✅ Softmax ⭐
- ✅ Activation functions: GELU, SiLU, SwiGLU
- ✅ Sin and cos (RoPE)
- ✅ Probability basics ⭐
- ✅ Mean and variance (LayerNorm / RMSNorm)
- ✅ Sampling: temperature, top-k, top-p ⭐
- ✅ Entropy, cross-entropy, perplexity
- ✅ KL divergence
I already know transformers, so these are revision notes from the inference side: shapes, parameter counts, FLOPs and how each part is served.
- ✅ Embeddings (word → vector) and lm_head
- ✅ Attention formula:
softmax(QKᵀ / √d) × V⭐ - ✅ Multi-head attention and GQA
- ✅ Feed-forward layers and MoE (SwiGLU in 3.3)
- ✅ KV cache and its size formula ⭐ (see the KV Cache deep dive above)
- ✅ Big-O
- ✅ Counting FLOPs ⭐
- ✅ Counting memory in bytes ⭐
- ✅ Memory bandwidth vs compute speed
- ⬜ Arithmetic intensity and the roofline model ⭐
- ⬜ Latency vs throughput ⭐
- ⬜ Basic queuing math
Only needed for training, not inference.
Matrix shape rule: (m × k) · (k × n) = (m × n)
FLOPs (matmul): 2 × m × n × k
FLOPs per token: ≈ 2 × parameters
Model size (bytes): parameters × bytes per number
Read time: model size ÷ memory bandwidth
Scale: scale = biggest number ÷ biggest whole number allowed
Quantize: stored = round(number ÷ scale)
Dequantize: back = stored × scale
GPU memory needed: weights + KV cache + about 10% extra
KV cache per word: 2 × layers × KV heads × head size × bytes
KV cache total: KV cache per word × words × users
Users on a GPU: (GPU memory − weights − extra) ÷ KV per user
W4A16: 4-bit weights, 16-bit activations
Log rule: log(a × b) = log a + log b
Softmax: eˣⁱ ÷ sum of all eˣ (subtract the max first)
Temperature: softmax(scores ÷ T)
SwiGLU FFN: Down( SiLU(Gate·x) × Up·x ) → 3 matrices
Rotate [x, y] by θ: [x·cos θ − y·sin θ, x·sin θ + y·cos θ]
AND / OR: P(A and B) = P(A) × P(B), P(A or B) = P(A) + P(B)
Expected value: sum of (chance × result)
Std: √(average of (x − mean)²)
RMSNorm: x ÷ √(average of x²) × weight
Min-p cut-off: min_p × top chance
Spec decode accept: min(1, target chance ÷ draft chance)
Surprise: −log(chance)
Perplexity: e^(average surprise on real text) (lower = better)
KL(P ‖ Q): sum of P × log(P ÷ Q) (0 = same)
Tokens: 1 token ≈ 0.75 English words
lm_head FLOPs: 2 × hidden × vocab per token
Attention: softmax(QKᵀ / √d) × V
Decode attn FLOPs: 4 × context length × hidden, per layer
Heads: hidden ÷ head size (4,096 ÷ 128 = 32)
FFN params (SwiGLU): 3 × hidden × FFN size, per layer
MoE: memory = all parameters, compute = active parameters
Big-O (n = tokens): decode attention O(n), prefill attention O(n²), FFN O(1), KV cache O(n)
Prefill FLOPs: 2 × parameters × prompt tokens (+ 2 × n² × hidden × layers for attention)
Decode FLOPs/step: 2 × parameters × batch size
Time (compute): FLOPs ÷ (GPU FLOPS × MFU)
Users per GPU: (GPU memory − weights − extra) ÷ KV per user
Bytes read per step: weights + KV per user × users
Time (memory): bytes read ÷ memory speed
GPUs needed: total memory ÷ one GPU's memory, rounded up
GPU ratio: GPU FLOPS ÷ bandwidth (H100 ≈ 500 FLOPs per byte)
Real time: the bigger of memory time and maths time
- How a GPU works (HBM, SMs, Tensor Cores, NVLink)
- Inference techniques (PagedAttention, continuous batching, speculative decoding, FlashAttention, quantization methods like GPTQ, AWQ and SmoothQuant, parallelism)
- Serving frameworks (vLLM, SGLang, TensorRT-LLM)
- Measuring and profiling (TTFT, TPOT, throughput, Nsight)
- Projects