Var⁡(⋅)\operatorname{Var}(\cdot)

Var — variance

Read "the variance of X". It measures how spread out a random quantity is around its average. Its square root is the standard deviation.

Var⁡(X)=E[X2]−(E[X])2\operatorname{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2: the average squared distance from the mean. Two rules are used repeatedly in the course:

  • For independent quantities, variances add: Var⁡(X+Y)=Var⁡(X)+Var⁡(Y)\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y).
  • Scaling by a number cc scales the variance by c2c^2: Var⁡(cX)=c2Var⁡(X)\operatorname{Var}(cX) = c^2 \operatorname{Var}(X).

If the entries of q\mathbf{q} and k\mathbf{k} are independent with mean 0 and variance 1, each product qikiq_i k_i has variance 1, so the first rule gives Var⁡(q⊤k)=dk\operatorname{Var}(\mathbf{q}^\top\mathbf{k}) = d_k. The second rule then shows that dividing by dk\sqrt{d_k} brings it back to 1.

Example

X∈{−1,+1} equally likely  ⟹  Var⁡(X)=1−02=1X \in \{-1, +1\} \text{ equally likely} \;\Longrightarrow\; \operatorname{Var}(X) = 1 - 0^2 = 1

The mean is 0 and X² is always 1.

Where the course uses it

Related terms