🧬
🧬 Hands-on companion

Build Your Own LLM in Python

8 lessons: tokenizer, bigram table, weighted die with temperature, loss, gradient training, n-grams and overfitting — on all of Shakespeare's plays, in pure Python right in the browser. The hands-on companion to "How AI Works".

What you'll build
🔤Your own tokenizer
🎲A text generator with temperature
📉Loss 4.17 → 2.5 with your own eyes
🧪Overfitting caught live
📘 8 lessons1-2 evenings🎓 Intermediate
🎯 Mission
Build and train a language model with your own hands
+130XP
🏆CertificatePDF
📜Your model writes Shakespeare
Progress0 / 8 · 0%
Next up: The book in memory: counting symbols

Course program

  1. 01

    The book in memory: counting symbols

    Loading textset and sortedA 65-symbol vocabulary⚡ +10
    ▶ Now
  2. 02

    Tokenizer: letters to numbers and back

    stoi and itosenumerateencode / decodeRound-trip check⚡ +10
    Open
  3. 03

    Questions and answers: splitting the book

    Encoding the whole book(context, answer) pairstrain / val 90 / 10⚡ +10
    Open
  4. 04

    The first model: a "who follows whom" table

    A 65 × 65 tablezip over neighboursThe probability pieBigrams⚡ +10
    🔒 Pro
  5. 05

    The die and the temperature: the model writes

    random.choicesThe generation loopTemperaturerandom.seed⚡ +10
    🔒 Pro
  6. 06

    Measuring surprise: the loss

    −log pAverage surpriseAdd-one smoothingtrain vs val⚡ +10
    🔒 Pro
  7. 07

    Turning the knobs: training by gradient

    logits and softmaxBatchesThe p − y gradientLearning rate⚡ +10
    🔒 Pro
  8. 08

    Looking further back: context and overfitting

    n-gramstuple as a keyOverfitting liveWhat attention adds
    Your model writes Shakespeare
    🔒 Pro