Huiyu Chen

Meta · Multimodal LLMs

← All learning paths阅读中文版 →

PATH 01 · 8 CHAPTERS

LLMs, starting with one token

Tensors, attention and training objectives, then real model I/O, retrieval, MoE and inference-time compute.

  1. 01

    How does text become a tensor?

    Understand vocabulary IDs, embedding lookup and padding through the four dimensions B, T, D and V.

  2. 02

    Compute attention: Q, K, V and the causal mask

    Use four tokens to understand scores, normalization and aggregation, and why masking precedes softmax.

  3. 03

    Beyond attention: residuals, normalization, FFNs and position

    Trace a decoder block and distinguish token mixing from feature transformation.

  4. 04

    From pretraining to alignment: what is optimized?

    Separate next-token loss, SFT, LoRA and preference optimization; inspect shifted labels and supervision masks.

  5. 05

    From logits to text: sampling, KV cache and stopping

    Compute softmax, temperature and top-p; distinguish decoding policy from model distribution and inspect cache reuse.

  6. 06

    A real-model lab: tokens, hidden states, attention and embeddings

    Load MiniLM in the browser and GPT-2 in a notebook; inspect actual tensors to distinguish encoders, decoders and pooling.

  7. 07

    Embeddings, RAG and evaluation: retrieval is not an answer

    Measure retrieval, reranking, context assembly and answer attribution separately, then improve the failing stage.

  8. 08

    Toward the frontier: MoE, long context and verified reasoning

    Understand frontier designs through compute, memory, communication and verification rather than model names alone.