Welcome to

AI Bootcamp

Course Syllabus and Content

Week 1

Introduction to AI applications

8 Lessons

This module introduces foundational concepts and practical workflows for working with Large Language Models (LLMs). Topics include terminology (e.g., ChatGPT vs. LLM, inference phases, training stages, and model compression techniques), the LLM ecosystem (vector databases, inference APIs, and fine-tuning libraries), and the model lifecycle. Participants will build a simple LLM-based system from scratch, starting with “Hello World” inference using Hugging Face, and deploy an LLM API using Modal for serverless deployment.

  • 01Introduction to AI applications
    Sneak Peek 
  • 02Intro to AI modalities
    Sneak Peek 
  • 03Examples of AI applications
    Sneak Peek 
  • 04Technologies
    Sneak Peek 
  • 05How to brainstorm AI applications
    Sneak Peek 
  • 06Vertical applications and their example technologies
    Sneak Peek 
  • 07Indie hacker examples
    Sneak Peek 
  • 08Venture funded LLM examples
    Sneak Peek 
Week 2

Building a Shakespearean Language Model

7 Lessons

Building a Shakespearean Language Model

  • 01What is a tokenizer? What makes a tokenizer "good"?
    Sneak Peek 
  • 02Build a baseline 'word-based' tokenizer
    Sneak Peek 
  • 03Build a byte-pair encoding
    Sneak Peek 
  • 04Compare with state-of-the-art tokenizers (e.g., Llama’s)
    Sneak Peek 
  • 05What is an embedding? What makes an embedding "good"?
    Sneak Peek 
  • 06Understand the semantic meaning of word embeddings
    Sneak Peek 
  • 07Train a word embedding on Shakespearean tokens
    Sneak Peek 
Week 3

Building an n-gram language model

5 Lessons

Building an n-gram language model

  • 01What is an n-gram?
    Sneak Peek 
  • 02Train a 2-gram model
    Sneak Peek 
  • 03Predict using 2-gram model
    Sneak Peek 
  • 04Train an n-gram model
    Sneak Peek 
  • 05Predict using n-gram model, in an autoregressive manner
    Sneak Peek 
Week 4

Building self-attention

2 Lessons

Building self-attention

  • 01A minimal version of self-attention
    Sneak Peek 
  • 02Build a batched version of self-attention
    Sneak Peek 
Week 5

Building the feed-forward neural network

3 Lessons

Building the feed-forward neural network

  • 01A minimal version of the feed-forward neural network (e.g., MLP)
    Sneak Peek 
  • 02Build a batched version of the MLP
    Sneak Peek 
  • 03“Train” on sanity check data
    Sneak Peek 
Week 6

Assembling the transformer-based language model

3 Lessons

Assembling the transformer-based language model

  • 01Add the remaining components (Skip connections, norms, positional encodings)
    Sneak Peek 
  • 02Use black box optimizer to pretrain
    Sneak Peek 
  • 03Predict using transformer-based language model, autoregressively
    Sneak Peek 
Week 7

Evaluating and deploying a transformer-based language model

4 Lessons

Evaluating and deploying a transformer-based language model

  • 01Run unsupervised evaluation using perplexity
    Sneak Peek 
  • 02Export the model using torchscript
    Sneak Peek 
  • 03Plug in our custom model into our LLM API
    Sneak Peek 
  • 04Open-sourced pre-trained models include Llama, Phi-3, Mixtral etc.
    Sneak Peek 
Week 8

Datasets

3 Lessons

Datasets

  • 01Pre-curated pretraining datasets
    Sneak Peek 
  • 02Common tools for data curation (mech turks, vis tools)
    Sneak Peek 
  • 03how to add guardrails
    Sneak Peek 
Week 9

Low-Rank Adapters for Instruction Tuning

4 Lessons

Low-Rank Adapters for Instruction Tuning

  • 01Build low-rank adapters for Shakespearean model
    Sneak Peek 
  • 02Download instruction tuning dataset
    Sneak Peek 
  • 03Run low-rank adapter fine-tuning
    Sneak Peek 
  • 04Open-sourced instruction-tuned models include DBRX, Pythia, Cerebrus etc.
    Sneak Peek 
Week 10

Retrieval-Augmented Generation (RAG)

5 Lessons

Retrieval-Augmented Generation (RAG)

  • 01What is RAG?
    Sneak Peek 
  • 02Implement RAG
    Sneak Peek 
  • 03Apply RAG to Shakespearean model
    Sneak Peek 
  • 04Apply RAG to pre-trained Large Language Models
    Sneak Peek 
  • 05Integrate RAG into our API
    Sneak Peek 
Week 11

The Future of Large Language Models

5 Lessons

The Future of Large Language Models

  • 01How to use LlamaIndex to chain tools together
    Sneak Peek 
  • 02The data wall
    Sneak Peek 
  • 03The ultimate test of time (Use fundamentals to understand and use time to filter)
    Sneak Peek 
  • 04How to keep to date
    Sneak Peek 
  • 05Multi modal large language models
    Sneak Peek 
Week 12

Machine learning operations

4 Lessons

Machine learning operations

  • 01project scoping
    Sneak Peek 
  • 02data needs
    Sneak Peek 
  • 03modeling strategies
    Sneak Peek 
  • 04deployment requirements
    Sneak Peek 
Week 13

Agents

2 Lessons

Agents

  • 01What are agents?
    Sneak Peek 
  • 02Design patterns
    Sneak Peek