Var — variance
Read "the variance of X". It measures how spread out a random quantity is around its average. Its square root is the standard deviation.
: the average squared distance from the mean. Two rules are used repeatedly in the course:
- For independent quantities, variances add: .
- Scaling by a number scales the variance by : .
If the entries of and are independent with mean 0 and variance 1, each product has variance 1, so the first rule gives . The second rule then shows that dividing by brings it back to 1.
Example
The mean is 0 and X² is always 1.
Where the course uses it
Related terms
- 𝔼 — expected valueRead "the expected value of X". It is the average value a random quantity takes if you draw it many, many times.
- The 1/√d_k scalingRead "one over root d k". Attention divides its scores by the square root of the head width so they keep a spread of about 1, however wide the head is.
- σ — standard deviation, singular value, or sigmoidRead "sigma". Usually it is the standard deviation: a typical distance of values from their average. With a numbered subscript (section 20.2) it is a singular value, and written as a function, σ(z), it is the sigmoid.