ℝ — the real numbers
Read "R" or "the reals". It is the set of all ordinary numbers: whole, negative, fractions and decimals.
The double-struck ("blackboard bold") is the set of real numbers, such as , , , and . It does not include complex numbers. Every weight and activation in a model is a real number; a computer's floating-point numbers are a finite approximation of them.
Related sets you will meet: (counting numbers or depending on the author), (all integers, including negatives), and (only zero and one, as in a one-hot vector).
Example
w is a single real number, such as 0.37 or -12.
Where the course uses it
- 1.1 Scalars, Vectors, Matrices, and Tensors · Module 1, Shapes and Linear Algebrafirst use
- 1.5 Embeddings: From a Token to a Vector · Module 1, Shapes and Linear Algebra
- 1.6 Linear Projections: One Vector Into Three · Module 1, Shapes and Linear Algebra
- 6.1 The Question Attention Answers · Module 6, Attention
- 6.6 Multi-Head Attention · Module 6, Attention
- 7.4 The Feed-Forward Network · Module 7, The Block and the Full Model
- 7.5 The Output Layer · Module 7, The Block and the Full Model
- 8.1 The 2017 Encoder-Decoder and Cross-Attention · Module 8, Three Families: Encoder, Encoder-Decoder, Decoder
- 12.2 The KV Cache · Module 12, Generation
- 25.2 The Vision Transformer · Module 25, Vision Transformers and Contrastive Learning
- 26.1 Connecting Vision to Language · Module 26, Vision-Language Models
- 26.4 Training on Images and Text Together · Module 26, Vision-Language Models
- 28.3 Vector Quantization and Residual VQ · Module 28, Generating Audio: Codecs and TTS