This page will provide information on what is covered in each lecture and will be updated as the class progresses.

Course slides are available on Moodle.

Date Topics Reading

18/07/2011

Overview of the course

  • Basic course information and administrative details
  • Supervised and unsupervised learning
  • Learning task, instances, features, labels, reward/loss, training, testing
Lecture slides
Chapter 1 of SS17

20/07/2011

Classification

  • Overview of classification: setup, training, test, validation dataset, overfitting.
  • Classification families: linear discriminative, non-linear discriminative, decision trees, probabilistic (conditional and generative), nearest neighbor.
Slides

20/07/2018 - 03/08/2018

Decision tree classification,

  • Purity, Gini index, entropy
  • Algorithms for constructing a decision tree
  • Pruning methods to avoid over-fitting
  • Regression trees
slides
Chapter 3 of Mitchell97

27/7/2018

Quiz 1,

  • Quiz on basics of probability and statistics and decision trees.
Sample reading: sections 3.1 to 3.9 here

01/08/2018 -- 17/08/2018

Probabilistic classifiers

  • Basics assumed: probability, axioms of probability, random variables, common distributions, means, variance and other moments, joint distributions, and conditional distributions.
    • You are assumed to know this material. For revising you can look at any standard textbook. Here are some handy references: chapters 1--6 from PRS, Chapter 1 and 2 of Bis07). Multivariate Gaussian (normal) distribution, covariance
  • Generative classifiers: LDA, QDA
  • Generative classifiers: Naive Bayes classification
  • Conditional classifier: Logistic
Chapters 4.2, 4.3.2 of Bis07
Example: naive Bayes
Lecture notes PDF and OneNote Link
Additional reading:
Mitchell's chapter

17/8/2018

Quiz 2,

Probabilistic classifiers
24/08/2018

Hyperplane classifiers,

  • Loss-regularization framework for classification
  • Loss functions: 0/1 ("true"), square, perceptron, logistic, hinge
  • Regularizers (Chapter 1.1 and 3.1.4 of Bis07)

Lecture notes
29/08/2018, 31/08/2018

Convex Optimization (Review),

  • Review of convex function and optimization of unconstrained functions. (
    • Definition and properties of convex function (Chapters 3.1.1 to 3.1.5, 3.2 of BV)
    • Unconstrained optimization algorithms: zero-th order , first order (Chapters 9.1 to 9.3 of BV excluding convergence proofs)
    • second order (Chapter 9.5 of BV excluding convergence proofs)

Convex functions notes
Optimization notes

06/09/2018,
19/09/2018--,
26/09/2018

Feedforward Neural networks ,

  • Feedforward networks
  • Backpropagation Algorithm

Lecture slides, Slides as pdf
Chapter 6 of Deep Learning book

28/09/2018

Convolutional Neural Networks


03/10/2018, 05/10/2018, 10/10/2018

Recurrent Neural Networks

  • Basic RNNs
  • Back propagation along time
  • Application time series forecastingm, language modeling (with word embeddings)
  • Encoder-decoder model with attention for sequence to sequence learning.

  • Lecture slides
    Slides in pdf
    Chapter 10.0 to 10.4 of Deep Learning book

    10/10/2018, 12/10/2018, 17/10/2018

    Clustering

    Lecture notes

    24/10/2018, 26/10/2018
    Combining models Lecture notes: bagging Lecture notes: boosting

    31/10/2018, 02/10/2018
    Support vector machines (Chapter 7 of Bis07)
    • Max margin motivation: low density, high stability
    • Margin geometry to primal SVM formulation for separable training data (demo)
    • Dual formulation and role of alpha in a form of sparse local regression
    • Inseparable data, slack variables, hinge loss, upper bound on 0/1 training loss (demo)
    • Handling non-linear regression by lifting data points to higher dimension (demo)
    • Polynomial, Gaussian, RBF kernels
    • Sequential minimal optimization (SMO) algorithm
    Lecture notes Wikipedia

    07/11/2018
    Overview of graphical models
    09/11/2018
    Overview of Markov Decision Process and Reinforcement Learning Lecture by Sabyasachi Ghosh