\hat{x}

Hat and tilde — an estimate or modified version

Read x̂ as "x hat" and x̃ as "x tilde". Both mark a changed copy of x: most often an estimate or a rounded version (hat), or a transformed version (tilde).

$\hat{x}$ is what you get back after quantizing and then dequantizing $x$ . $\hat{\mathbf{z}}$ is a codec's reconstruction of $\mathbf{z}$ . $\hat{w}$ is a rounded weight. The gap between $\hat{x}$ and $x$ is the error the method has to keep small.

The convention is loose, so read the surrounding sentence: in section 24.2 $\hat{X}$ and $\hat{W}$ are simply the rescaled matrices, not estimates. A tilde is used in a similar way for "the altered version of", such as $\tilde{\mathbf{q}}$ for a query after RoPE has rotated it.

Example

x = 0.137 \;\to\; \hat{x} = 0.14

The stored value after rounding to two decimal places. The error is 0.003.

Where the course uses it

10.2 AdamW · Module 10, Pretrainingfirst use
15.1 Rotary Position Embedding (RoPE) · Module 15, The LLaMA Block
16.5 Logit Soft-Capping and QK-Norm · Module 16, Efficient and Long Attention
24.2 Uniform Integer Quantization · Module 24, Quantization and Number Formats
24.3 Outlier Channels and SmoothQuant · Module 24, Quantization and Number Formats
24.4 Post-Training Quantization: AWQ and GPTQ · Module 24, Quantization and Number Formats
28.2 Autoencoders and Latent Spaces · Module 28, Generating Audio: Codecs and TTS
28.3 Vector Quantization and Residual VQ · Module 28, Generating Audio: Codecs and TTS

Related terms

≈, ∼, ∝ — approximately, distributed as, proportional to≈ reads "approximately equals". ∼ reads "is distributed as" or "is roughly of the order of". ∝ reads "is proportional to".
Integer quantizationConverts a real number x into a small integer q: divide by a scale s, round to the nearest integer, add a zero-point z, and clip to the allowed range.