\nabla

∇ — gradient

Read "nabla" or "the gradient of". It is the list of every partial derivative at once, and it points in the direction that increases the function fastest.

$\nabla_\theta \mathcal{L}$ collects $\partial\mathcal{L}/\partial\theta_k$ for every parameter $\theta_k$ into one vector, the same shape as the parameters. Training takes a small step in the opposite direction, $\theta \leftarrow \theta - \eta\,\nabla_\theta \mathcal{L}$ with a small step size $\eta$ , because the fastest way uphill, reversed, is the fastest way down.

Example

\mathcal{L} = a^2 + 3b \;\Longrightarrow\; \nabla \mathcal{L} = \left[\tfrac{\partial \mathcal{L}}{\partial a},\, \tfrac{\partial \mathcal{L}}{\partial b}\right] = [2a,\, 3]

One partial derivative per parameter, stacked into a vector.

Where the course uses it

2.2 Partial Derivatives and the Gradient · Module 2, Calculus and Optimizationfirst use
2.4 Gradient Descent · Module 2, Calculus and Optimization
21.2 Policy Gradients · Module 21, Preference Learning: RL, RLHF and DPO

Related terms

∂ — partial derivativeRead "partial L by partial θ". It is how much the top quantity changes when you nudge the bottom one slightly, keeping every other input fixed.
𝓛 — the lossRead "L" or "the loss". The curly L is the single number that measures how wrong the model is. Training adjusts the weights to make it small.