Welcome to
Fundamentals of transformers - Live Workshop
Course Syllabus and Content
Module 1
What are LLMs?
2 Lessons
Demystifying terminology behind LLMs
- 01ChatGPT is to LLM, as Kleenex is to tissueSneak Peek
- 02Model, Data, Algorithms, OptimizationSneak Peek
Module 2
What LLMs predict
2 Lessons
Introduction to Autoregressive Decoding
- 01Conditional generationSneak Peek
- 02Demo; Manual LLM inferenceSneak Peek
Module 3
How LLMs predict
3 Lessons
The architecture for a Large Language Model
- 01Vectors, intuitivelySneak Peek
- 02Word embeddingsSneak Peek
- 03Nearest neighborsSneak Peek
Module 4
How Transformers predict
3 Lessons
The innards of a transformer layer
- 01Self-attention adds contextSneak Peek
- 02Matrix multiplies, intuitivelySneak Peek
- 03MLP transformsSneak Peek
Module 5
How LLMs attend
4 Lessons
How to find the needle in the haystack
- 01Forward facing attentionSneak Peek
- 02Sneak Peek
- 03Multi-head attentionSneak Peek
- 04Grouped-Query AttentionSneak Peek
Module 6
Modern LLM connection to papers
3 Lessons
Connection to papers
- 01Academic transformer diagramsSneak Peek
- 02Modern-day transformer diagramsSneak Peek
- 03Bottlenecks in LLMs todaySneak Peek