∏\prod

∏ — product

Read "the product of". The big pi works like ∑, but it multiplies the terms instead of adding them.

∏t=1Tat=a1×a2×⋯×aT\prod_{t=1}^{T} a_t = a_1 \times a_2 \times \cdots \times a_T.

It appears in two places in the course. First, chaining probabilities: a sentence's probability is the product of each token's probability given the tokens before it. Second, chaining derivatives through many steps. This is why RNN gradients vanish: multiply TT numbers that are each below 1 and the result shrinks toward zero, faster the larger TT is.

Example

∏t=130.5=0.5×0.5×0.5=0.125\prod_{t=1}^{3} 0.5 = 0.5 \times 0.5 \times 0.5 = 0.125

Three factors of one half already give one eighth; fifty would give about 0.000000000000001.

Where the course uses it

Related terms