CS480/680: INTRODUCTION TO MACHINE LEARNING, Spring 2026, University of Waterloo

Overview

This course focuses on the introduction of machine leanring. However, we will cover what you are particularly interested in, e.g., technical details of how to train your own ChatGPT.

Pre-requisite:

Format:

Graded Student Work for CS480/680: Homeworks (We do not accept hand-written submission. Typeset using LaTeX is recommended. HW solutions will not be released. Please attend TA's office hour for a solution.): Homework Policy: Completed assignments will be submitted through LEARN. Submit early and often! You must write your solutions independently and individually, and you should always acknowledge any help you get (book, friend, internet, etc.). Using AI to write homeworks is prohibited. We may use tools to detect your submission. Mark appeals should be requested within two weeks of receiving the mark. The appeal could go either ways, so request only if you truly believe something is wrong.

Late Policy: We do NOT accept any late submissions, unless you have a legitimate reason with a formal proof (e.g., hospitalization, family urgency, etc.). The proof date should be within 7 days before your homework deadline. Traveling, being busy with other stuff, internet disconnection, or simply forgetting to submit, are not considered legitimate. Without a proof, your score will be 0 as long as you are late, even for 1min (LEARN submission portal will be closed on time. We DO NOT accept homework submission by email.). With a proof and instructor's approval, you can get a 7-day homework extension. According to the school policy, undergraduate students are allowed to use short-term absence once per term. Please inform the TA head xxx (xxx@uwaterloo.ca) and provide a screenshot if you have submitted an application to Quest for a 2-day extension. Failing to do so (e.g., only informing instructors or other TAs) will make your application invalid, and your delayed homework will still be marked as late.

Textbook:

There is no required textbook, but the following fine texts are recommended.

Schedule (tentative)

Date Category Topic Slides Suggested Readings
Lecture 1 May 11 Introduction
Link Deep Learning, Section 1
Lecture 2 May 13 Classic ML Perceptron
Link Patterns, Predictions, and Actions, Page 37
May 20 Classic ML Perceptron - Cont'
Link Patterns, Predictions, and Actions, Page 37
Lecture 3 May 25 Classic ML Linear Regression
Link Probabilistic Machine Learning: An Introduction, Page 363
Lecture 4 May 27 Classic ML
  • Linear Regression - Cont'
  • Logistic Regression
  • Link
  • Link
    		• Probabilistic Machine Learning: An Introduction, Page 333
    Lecture 5 May 26 Classic ML Hard-Margin SVM
    		Link The Elements of Statistical Learning, Section 12.3
    Lecture 6 June 1 Classic ML Soft-Margin SVM
    		Link The Elements of Statistical Learning, Section 12.3
    Lecture 7 June 3 Classic ML
  • Soft-Margin SVM - Cont'
  • Reproducing Kernels
  • Link
  • Link
    		• The Elements of Statistical Learning, Section 12.3
    Lecture 8 June 8 Classic ML Gradient Descent
    		Link Convex Optimization, Section 9.3
    Lecture 9 June 10 Neural Nets
  • Gradient Descent - Cont'
  • Fully Connected NNs
  • Link
  • Link
    		• Deep Learning, Section 6
    June 15 Neural Nets Fully Connected NNs - Cont'
    		Link Deep Learning, Section 6
    Lecture 10 June 17 Neural Nets Convolutional NNs
    		Link Deep Learning, Section 9
    June 22 Neural Nets Convolutional NNs - Cont'
    		Link Deep Learning, Section 9
    Lecture 11 June 24 Neural Nets Transformer
    		Link
  • “Attention Is All You Need”. Vaswani et al. 2017 link
    		•
    June 29 Neural Nets Transformer - Cont'
    		Link
  • “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”. Devlin et al. 2018 link
  • (GPT-1) “Improving Language Understanding by Generative Pre-training”. Radford et al. 2018 link
  • (talk by Andrej Karpathy) State of GPT link
    • Lecture 12 July 6 Modern ML Paradigms Large Language Models
    		Link
  • (GPT-2) “Language Models are Unsupervised Multitask Learners”. Radford et al. 2019 link
    • July 8 Modern ML Paradigms Large Language Models - Cont'
    		Link
  • (GPT-3) “Language Models are Few-Shot Learners”. Brown et al. 2020 link
  • (GPT-3.5) “Training Language Models to follow Instructions with Human Feedbacks”. Ouyang et al. 2022 link
  • (GPT-4) “GPT-4 Technical Report”. OpenAI 2023 link
    • Lecture 13 July 13 Modern ML Paradigms Speculative Decoding
    		Link
  • Fast Inference from Transformers via Speculative Decoding
  • Accelerating Large Language Model Decoding with Speculative Sampling
  • Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
    • July 15 Modern ML Paradigms Speculative Decoding - Cont' Link
  • EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
  • EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
  • EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
    • Lecture 14 July 20 Modern ML Paradigms GANs
    		Link
  • Generative Adversarial Networks
    • Lecture 15 July 22 Modern ML Paradigms Self-supervised Learning Link
  • A Simple Framework for Contrastive Learning of Visual Representations
  • Learning Transferable Visual Models From Natural Language Supervision
    • Lecture 16 July 27 Trustworthy ML Adversarial Attacks
    		Link
  • DeepFool: a simple and accurate method to fool deep neural networks
    • Lecture 17 July 29 Trustworthy ML Adversarial Robustness
    		Link
  • Towards Deep Learning Models Resistant to Adversarial Attacks
  • Theoretically Principled Trade-off between Robustness and Accuracy
    • Lecture 18 Aug 5 Trustworthy ML Certified Robustness Link
  • Towards Deep Learning Models Resistant to Adversarial Attacks
  • Theoretically Principled Trade-off between Robustness and Accuracy

    • Mental Health: If you or anyone you know experiences any academic stress, difficult life events, or feelings like anxiety or depression, we strongly encourage you to seek support.

    On-campus Resources

    Off-campus Resources

    Diversity: It is our intent that students from all diverse backgrounds and perspectives be well served by this course, and that students’ learning needs be addressed both in and out of class. We recognize the immense value of the diversity in identities, perspectives, and contributions that students bring, and the benefit it has on our educational environment. Your suggestions are encouraged and appreciated. Please let us know ways to improve the effectiveness of the course for you personally or for other students or student groups. In particular: