# Module 0: Start Here: What an LLM Does > What a large language model actually computes, from prompt to next-token probability, and how this free 30-module transformer course is organised. - Orientation · core track · Foundational · 45 minutes - By the end you can describe in one paragraph what happens between typing a prompt and seeing the next word, and know how to use the course. - Web page (interactive figures, labs and equation explainers): https://transformers-and-math.drmhse.com - From *Transformers & Math* by Mike Chumba: https://transformers-and-math.drmhse.com A large language model does one thing: given some text, it gives a probability to every possible next piece of text. Chat assistants and code completion are built by calling that operation over and over. This course takes it apart, one piece of mathematics at a time, until you can follow every number from your prompt to the word that comes back. This module gives you the whole picture first, so every later module has a place to fit. Nothing here needs to make full sense yet; each step has a module of its own. ## 0.1 From a Prompt to a Next-Token Probability Take the prompt `The cat sat on the`. Here is what happens before the model offers `mat`. 1. **Split the text into tokens.** A *tokenizer* cuts the text into pieces from a fixed list, its *vocabulary*. Common words are usually one token each; rare words are split into several. Each token is then replaced by its position in the list, a whole number. (Module 9.) 2. **Turn each token into a vector.** The model keeps a table with one row of numbers per vocabulary entry, and looks up the row for each token. A token is now a list of a few hundred or a few thousand numbers. (Module 1.) 3. **Mark where each token sits.** On its own, the model cannot tell the first word from the last, so position information is mixed in. (Modules 6, 7 and 15.) 4. **Let the tokens look at each other.** In each *attention* layer, every token works out which earlier tokens matter to it and pulls in information from them. This is how `the` at the end comes to carry the fact that something `sat on` it. (Module 6.) 5. **Process each token on its own.** A small network applied to each token separately transforms what attention gathered. (Module 7.) 6. **Repeat.** Steps 4 and 5 form one *block*. A model stacks dozens of blocks, each adding to a running vector per token. (Module 7.) 7. **Score every possible next token.** The last token's final vector is compared with every vocabulary entry, giving one score each, and the scores are turned into probabilities that add up to 1. (Modules 4 and 7.) 8. **Choose one.** A *decoding* rule picks the next token from those probabilities, appends it to the text, and the whole process runs again for the token after. (Module 12.) The model's numbers come from *training*: showing it enormous amounts of text and nudging every number so that it gives higher probability to the token that actually came next (Modules 2, 4 and 10). An assistant is made from that raw predictor by further training on examples of good answers and on people's preferences (Modules 20 to 22). ```mermaid graph LR Text["Prompt text"] --> Tok["Tokenizer: text to token ids"] Tok --> Emb["Embedding lookup: ids to vectors"] Emb --> Blocks["Stack of blocks: attention, then a per-token network, repeated"] Blocks --> Out["Output layer: one score per vocabulary token"] Out --> Probs["Softmax: scores to probabilities"] Probs --> Pick["Decoding: choose the next token"] Pick -->|"append and repeat"| Tok ``` > **In words.** Tokens become vectors, the vectors pass through a stack of blocks, the output layer scores every vocabulary entry, softmax turns scores into probabilities, and a decoding rule picks one. The chosen token is appended and the loop runs again. ## 0.2 Watching the Loop Run The figure below runs the whole loop with a real, very small model, trained on Shakespeare, in your browser. It works one character at a time rather than one subword token, and it is far too small to write well, but it has learned enough to make sensible guesses. Its four panels follow the stages of section 0.1 for the current text, and the buttons run the loop: score, choose, append, repeat. > **Interactive figure** `token-tracer`, on the web page: https://transformers-and-math.drmhse.com#figure-token-tracer > > **Try:** The prompt is "to be or not to". Before pressing anything, guess the model's favourite next character. Then switch *Choose* to *most likely* and press *Append 20*: what do you expect the text to do? > > **What happens:** A space, at about 0.76: after "to", a word almost always ends. Always taking the most likely character soon locks into a loop, "to be or not to the the the the", because the likeliest continuation of "the" is a space and the likeliest word after a space is "the". Reset and sample instead, and the randomness avoids the loop (section 12.3), though once the sixteen characters the model reads are all "the the the", even sampling rarely escapes. The text it writes looks Shakespearean and means nothing; a model millions of times larger writes text that makes sense, using exactly this loop. > **In words.** A prompt box and the text written so far, with the model's characters in red and the last sixteen characters, all the model can read, shaded. Below, four panels show the current step: the characters as numbered tokens, their vectors, the vectors after two blocks with the same shape, and the five likeliest next characters. Append one or append twenty runs the loop, choosing either the likeliest character or a random one weighted by probability, with a temperature slider. ## 0.3 What You Need, and How the Course Is Organized ### What You Should Already Know - **High-school algebra**, including the rules for exponents and logarithms: $e^{a}e^{b} = e^{a+b}$ and $\log(ab) = \log a + \log b$. - **Sine and cosine, and angles in radians.** A full turn is $2\pi$ radians. - **Summation notation.** $\sum_{i=1}^{3} x_i$ means $x_1 + x_2 + x_3$. Module 1 goes over it again. - **Basic Python**: functions, lists and loops, enough to read short code samples. Helpful but not required: derivatives from a first calculus course (Module 2 teaches what the course uses), and some NumPy or PyTorch (Modules 1 and 4 introduce the calls the course uses). You do **not** need linear algebra beyond vectors, probability, machine learning, reinforcement learning or anything about GPUs. Each is taught just before it is needed. ### The Parts and the Two Tracks | Part | Modules | What it covers | |---|---|---| | 0 · Orientation | 0 | This page | | I · Foundations | 1–5 | The maths and machine learning the rest is built from | | II · The Transformer | 6–8 | Attention, the block, and the three families of model | | III · Building an LLM | 9–14 | Tokens, pretraining, training at scale, generation, evaluation, and what the model learns | | IV · The Modern Architecture | 15–19 | The LLaMA-style block and the designs that followed it | | V · From Model to Assistant | 20–22 | Fine-tuning, preference learning, and reasoning models | | VI · Deployment | 23–24 | Serving and quantization | | VII · Beyond Text | 25–28 | Vision, vision-language models, OCR and speech | | VIII · Capstone | 29 | Build, upgrade, measure and ship a tiny GPT | Every module is marked **Core** or **Elective**. The **core track** is Modules 0 to 15, 21 to 23, and the capstone. Electives go deeper on one topic and can be taken in any order once the modules they build on (named in each header) are done. ### How Each Module Works - **A header** with the goal, the modules it builds on, its track and an estimated time. - **Sections** that start with small numbers you can check by hand, then give the general formula. - **Figures** you can change. Most carry a **Predict, then try** box: commit to a guess *before* touching the figure. The prediction is what makes the figure teach. - **Exercises** in three tiers: **Check** (recall), **Apply** (compute by hand) and **Extend** (reason further, or write code). Numeric answers are checked in the page, and every exercise has a worked solution. ### Reading the Notation Click any equation for a plain-English explanation of each part. Symbols such as $d_{\text{model}}$ or $\mathcal{L}$ show a gloss on hover; a toggle at the top of each module turns these marks off. The [glossary](/glossary) lists every symbol and term in the order you meet them. ## 0.4 How the Labs Work Wherever a claim can be tested, it is, and the result appears in one of three boxes: - **Measured**: run as an experiment on the course's reference implementation. The box names the experiment. - **Not measured**: from a paper or model card, not reproduced here. - **Caveat**: what a result does *not* show, such as a measurement on a model too small to generalize from. The reference implementation is [tiny-transformers](https://huggingface.co/spaces/zeguru/tiny-transformers), a small, readable transformer written for this course. Every lab panel shows the code it ran and the command to rerun it on your machine. Small labs run in your browser with a **Run** button; labs that train a model run in the terminal. ### Companion Notebooks Modules 1 to 29 each have a companion notebook that runs free on Colab's CPU in a minute or two: the button at the top of each module opens it, and sections it covers carry a **Run this section** badge. The notebooks rebuild the module's ideas in code you can change. A good habit throughout: when you read a number, ask which kind of box it came from. **Exercise (check).** In which order do these happen when a model produces the next token: (a) softmax over the vocabulary, (b) embedding lookup, (c) tokenization, (d) the stack of blocks? *Solution:* (c) tokenization, (b) embedding lookup, (d) the stack of blocks, (a) softmax over the vocabulary. The text becomes token ids, the ids become vectors, the blocks transform the vectors, and only at the very end are the final scores turned into probabilities. **Exercise (check).** A sentence in a module says a technique "roughly doubles generation speed", inside a **Not measured** box. What does that tell you about the number? *Solution:* That it comes from a paper, a model card or a vendor, and the course has not reproduced it. It may well be true, but it has not been checked on the reference implementation, and the conditions under which it was measured (hardware, model size, batch size) may differ from yours. **Exercise (extend).** Generation produces one token per pass through the model. A model writes a 300-token answer. How many passes through the full stack of blocks does that take, at minimum, and what does that suggest about where the cost of generation goes? *Answer:* 300 *Solution:* At least 300 passes, one per generated token, since each token depends on all the ones before it. (Processing the prompt itself can be done in one pass, as Module 12 explains.) So the cost of a long answer is dominated by repeating the whole model once per token, which is why Module 12 introduces a cache that avoids recomputing the earlier tokens, and Module 23 is devoted to making each of those passes cheaper. ### Core Academic References - **Vaswani et al. (2017)**: *Attention Is All You Need*. NeurIPS 2017. [arXiv:1706.03762](https://arxiv.org/abs/1706.03762) - **Radford et al., OpenAI (2019)**: *Language Models are Unsupervised Multitask Learners* (GPT-2). Technical Report.