Curriculum Checklist & TOC

Mastery Track0%

0 of 161 Topics Mastered30 Modules

Orientation

Part I · Foundations

Part II · The Transformer

Part III · Building an LLM

Part IV · The Modern Architecture

Part V · From Model to Assistant

Part VI · Deployment

Part VII · Beyond Text

Part VIII · Capstone

Superposition

Superposition — more features than dimensions

Representing many more features than there are dimensions by giving each a nearly perpendicular direction, which works because only a few features are active at once.

A cost is that single neurons respond to several unrelated features (section 14.5).

Where the course uses it

14.5 Superposition · Module 14, What the Model Learnsfirst use

Related terms

Orthogonal matrixA matrix that only rotates (or mirrors) vectors: it keeps every length and every angle, and its transpose undoes it. R^⊤R = I says exactly that.