\mathbf{e}_i

eᵢ — a one-hot vector

Read "e i". It is a vector with a 1 in position i and 0 everywhere else: token number i, written as a vector.

$\mathbf{e}_i \in \{0,1\}^{|V|}$ has $|V|$ entries and a single 1, at position $i$ . Multiplying it by a matrix picks out row $i$ of that matrix: $\mathbf{e}_i^\top W_E = W_E[i,:]$ .

No real implementation does that multiplication; it just reads row $i$ from memory (nn.Embedding in PyTorch). Writing it as a multiplication shows that the embedding step is an ordinary matrix product like every other layer. It also makes the gradient easy to see: only row $i$ of $W_E$ gets updated.

Example

|V| = 4,\; i = 2 \;\Longrightarrow\; \mathbf{e}_2 = [0, 0, 1, 0]

Positions are counted from 0 here, so position 2 is the third slot.

Where the course uses it

1.5 Embeddings: From a Token to a Vector · Module 1, Shapes and Linear Algebrafirst use
4.3 Logits, Softmax, and Cross-Entropy · Module 4, How a Model Learns From Data
16.4 Sandwich Normalization and Embedding Scaling · Module 16, Efficient and Long Attention

Related terms

|V| — size of the vocabularyRead "the size of V". Single bars around a set mean "how many items it has", so |V| is the number of tokens in the vocabulary.
{ } — a setCurly braces list the members of a set: a collection of values with no order and no repeats. Read {0, 1} as "the set of 0 and 1".
W_E — the embedding matrixRead "W E". It is a learned table with one row per token in the vocabulary; each row is that token's vector of d_model numbers.