Course Description

Welcome to "Introduction to Machine Learning 419(M)".
In this undergraduate-level course, you will be introduced to the foundations of machine learning along with a slew of popular machine learning techniques. This will also give you insights on how to apply machine learning to solve a new problem.

This course is open to any non-CSE undergraduate student who wants to do a minor in CSE. There are no prerequisites.

Course Info

Time: Wednesdays, Fridays, 9.30 am to 10.55 am
Venue: CC 103
Instructor: Preethi Jyothi. You can email me at pjyothi [at] cse [dot] iitb [dot] ac [dot] in


TAs:
Saurabh Garg (email: saurabhgarg [at] cse.iitb.ac.in)
Tanmay Parekh (email: tanmayb [at] cse.iitb.ac.in)
Sunandini Sanyal (email: sunandinis [at] cse.iitb.ac.in)
Anupama Vijjapu (email: anupamav [at] cse.iitb.ac.in)
Himanshu Agarwal (email: aghimanshu [at] cse.iitb.ac.in)
Aniket Muiri (email: 163050059 [at] iitb.ac.in)
Pooja Palod (email: 163050026 [at] iitb.ac.in)

Instructor office hours (in CC 221): 7 to 8 pm on Wednesdays, 5 pm to 6 pm on Fridays

TA office hours:
Himanshu, Aniket and Pooja (3 pm to 4 pm on Wednesdays, KreSIT library)
Saurabh and Tanmay (7 to 8 pm on Thursdays, SL3 in CC building)
Sunandini and Anupama (10.45 am to 11.45 am on Mondays, KreSIT library)

Course grading

All assignments should be completed individually. No form of collaboration is allowed unless explicitly permitted by the instructor. Anyone deviating from these standards of academic integrity will be reported to the department's disciplinary committee.

  1. Four assignment sets (40%)
  2. Midsem exam (15%)
  3. Project (15%)
  4. Final exam (25%)
  5. Participation (5%)

Syllabus

Course syllabus includes basic classification/regression techniques such as Naive Bayes', decision trees, SVMs, boosting/bagging and linear/logistic regression, maximum likelihood estimates, regularization, basics of statistical learning theory, perceptron rule/multi-layer perceptrons, backpropagation, brief introduction to deep learning models, dimensionality reduction techniques like PCA and LDA, unsupervised learning: k-means clustering, gaussian mixture models, selected topics from natural/spoken language processing, computer vision, etc. [The syllabus is subject to minor changes depending on how the course proceeds.]

Schedule

Date Title Summary slides Reading
Jan 5, 2018 Machine Learning: What and why? Lecture1.pdf Chapter 1 of [SS-2017]
Jan 10, 2018 Linear Regression (Part I) Lecture2.pdf Chapter 3.1,3.2.1 of [TH-2009]
Jan 12, 2018 Linear Regression (Part II) Lecture3.pdf Notes by Cosma Shalizi
Jan 16, 2018 Assignment 1 released:
Due Jan 24, 2018
Assignment 1 -
Jan 17, 2018 Linear Classification:
Perceptron Algorithm
- Notes by Avrim Blum
Jan 19, 2018 Decision Trees Lecture5.pdf Shared via Moodle
Jan 24, 2018 "Some cases of pathology diagnostics using ML" Guest lecture by Prof. Amit Sethi -
Jan 31, 2018 Bias Variance Tradeoff Lecture6.pdf Notes by Andrew Ng
Feb 2, 2018 Generalization errors + model selection Lecture7.pdf Notes on VC dimension (Section 11.1)
Feb 7, 2018 Assignment 2 released:
Due Feb 16, 2018
Assignment 2 -
Feb 7, 2018 MLE/MAP/Naive Bayes Lecture8.pdf Peter Robinson's slides on MLE vs. MAP
Feb 9, 2018 Logistic Regression Lecture9.pdf Chapter 3 from "Machine Learning" by Tom Mitchell, September 2017.
Feb 14, 2018 Regularization Lecture10.pdf Chapter 3.4 (everything before Eqn 3.45) of [TH-2009] and Section 3.3 from Tom Mitchell's book chapter referred above.
Feb 16 & 21, 2018 SVMs and Kernel Methods Lecture12.pdf Andrew Moore's slides on SVMs
Feb 28 & March 2, 2018 Midsem week Lecture13.pdf Midsem practice problems shared via Moodle
March 7 & 9, 2018 Clustering, mixture models + EM algorithm Lecture14.pdf
Lecture15.pdf
Chapter 9 of [CB-2006]
March 14, 2018 Dimensionality Reduction - A Tutorial on PCA by Jonathon Shlens
March 16, 2018 Introduction to Neural Networks + Backpropagation Lecture17.pdf Neural Networks and Deep Learning, Chapters 1 & 2
March 19, 2018 Assignment 3 released:
Due March 28, 2018
Assignment 3 -
March 21, 2018 Neural Networks continued - Notes on multi-layer NNs and backpropagation
March 23, 2018 Guest lecture by Prof. Arjun Jain: Deep Learning for Computer Vision Slides -
March 28, 2018 Deep Learning for Speech and Language Processing Lecture19.pdf -
April 4, 2018 Ensemble learning: Boosting and Bagging - Chapter 14 of [CB-2006]
April 6, 2018 Ensemble learning continued - -
April 11, 2018 Hidden Markov Models Lecture22.pdf Chapter 9 of Jurafsky & Martin, 2016
April 11, 2018 Assignment 4 released:
Due May 2, 2018
Assignment 4 -
April 13, 2018 Sequence models continued - Chapter 9 of Jurafsky & Martin, 2016
April 11, 2018 Last lecture Lecture24.pdf Slides from David Silver's course on RL

More reading

Topic Additional reading
Linear Regression Chapters 3.1-3.3 of [CB-2006]
Linear models for classification Chapter 4.1 of [CB-2006]
Statistical Learning Theory Vapnik, V., An Overview of Statistical Learning Theory, IEEE Transactions on Neural Networks, Vol. 10, pp. 988-999, 1999.
Discriminative (LR) vs. Generative (NB) models Ng, A., On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes, NIPS, 2001.
Support Vector Machines and Kernel Methods Christopher Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, 1998.
NNs as Universal Approximators Kurt Hornik, Maxwell Stinchcombe and Halbert White, Multilayer Feedforward Networks are Universal Approximators, Neural Networks, 1989.
Deep Learning Goodfellow, I., Bengio, Y., Courville, A., Deep Learning, Part II, MIT Press, 2016.

Resources

Most of the suggested readings will be freely available online. Here are some textbooks to refer to:
  1. Understanding Machine Learning. Shai Shalev-Shwartz and Shai Ben-David. Cambridge University Press. 2017. [SS-2017]
  2. The Elements of Statistical Learning. Trevor Hastie, Robert Tibshirani and Jerome Friedman. Second Edition. 2009. [TH-2009]
  3. Foundations of Data Science. Avrim Blum, John Hopcroft and Ravindran Kannan. January 2017. [AB-2017]
  4. Pattern Recognition and Machine Learning. Christopher Bishop. Springer. 2006. [CB-2006]

Website credit: This is based on a Jekyll template.