Welcome to

Fundamentals of transformers - Live Workshop

Course Syllabus and Content

Module 1

What are LLMs?

2 Lessons

Demystifying terminology behind LLMs

  • 01ChatGPT is to LLM, as Kleenex is to tissue
    Sneak Peek 
  • 02Model, Data, Algorithms, Optimization
    Sneak Peek 
Module 2

What LLMs predict

2 Lessons

Introduction to Autoregressive Decoding

  • 01Conditional generation
    Sneak Peek 
  • 02Demo; Manual LLM inference
    Sneak Peek 
Module 3

How LLMs predict

3 Lessons

The architecture for a Large Language Model

  • 01Vectors, intuitively
    Sneak Peek 
  • 02Word embeddings
    Sneak Peek 
  • 03Nearest neighbors
    Sneak Peek 
Module 4

How Transformers predict

3 Lessons

The innards of a transformer layer

  • 01Self-attention adds context
    Sneak Peek 
  • 02Matrix multiplies, intuitively
    Sneak Peek 
  • 03MLP transforms
    Sneak Peek 
Module 5

How LLMs attend

4 Lessons

How to find the needle in the haystack

  • 01Forward facing attention
    Sneak Peek 
  • 02
    Sneak Peek 
  • 03Multi-head attention
    Sneak Peek 
  • 04Grouped-Query Attention
    Sneak Peek 
Module 6

Modern LLM connection to papers

3 Lessons

Connection to papers

  • 01Academic transformer diagrams
    Sneak Peek 
  • 02Modern-day transformer diagrams
    Sneak Peek 
  • 03Bottlenecks in LLMs today
    Sneak Peek