Welcome to *"Introduction to Machine Learning 419(M)". *

In this undergraduate-level course, you will be introduced to the foundations of machine learning along with a slew of popular machine learning techniques. This will also give you insights on how to apply machine learning to solve a new problem.

This course is open to any non-CSE undergraduate student who wants to do a minor in CSE. There are no prerequisites.

**Time:** Wednesdays, Fridays, 9.30 am to 10.55 am

** Venue:** CC 103

** Instructor: ** Preethi Jyothi. You can email me at pjyothi [at] cse [dot] iitb [dot] ac [dot] in

Saurabh Garg (email: saurabhgarg [at] cse.iitb.ac.in)

Tanmay Parekh (email: tanmayb [at] cse.iitb.ac.in)

Sunandini Sanyal (email: sunandinis [at] cse.iitb.ac.in)

Anupama Vijjapu (email: anupamav [at] cse.iitb.ac.in)

Himanshu Agarwal (email: aghimanshu [at] cse.iitb.ac.in)

Aniket Muiri (email: 163050059 [at] iitb.ac.in)

Pooja Palod (email: 163050026 [at] iitb.ac.in)

Himanshu, Aniket and Pooja (3 pm to 4 pm on Wednesdays,

Saurabh and Tanmay (7 to 8 pm on Thursdays,

Sunandini and Anupama (10.45 am to 11.45 am on Mondays,

All assignments should be completed individually. No form of collaboration is allowed unless explicitly permitted by the instructor. Anyone deviating from these standards of academic integrity will be reported to the department's disciplinary committee.

- Four assignment sets (
**40%**) - Midsem exam (
**15%**) - Project (
**15%**) - Final exam (
**25%**) - Participation (
**5%**)

Course syllabus includes basic classification/regression techniques such as Naive Bayes', decision trees, SVMs, boosting/bagging and linear/logistic regression, maximum likelihood estimates, regularization, basics of statistical learning theory, perceptron rule/multi-layer perceptrons, backpropagation, brief introduction to deep learning models, dimensionality reduction techniques like PCA and LDA, unsupervised learning: k-means clustering, gaussian mixture models, selected topics from natural/spoken language processing, computer vision, etc. *[The syllabus is subject to minor changes depending on how the course proceeds.] *

Date | Title | Summary slides | Reading |
---|---|---|---|

Jan 5, 2018 | Machine Learning: What and why? | Lecture1.pdf | Chapter 1 of [SS-2017] |

Jan 10, 2018 | Linear Regression (Part I) | Lecture2.pdf | Chapter 3.1,3.2.1 of [TH-2009] |

Jan 12, 2018 | Linear Regression (Part II) | Lecture3.pdf | Notes by Cosma Shalizi |

Jan 16, 2018 | Assignment 1 released: Due Jan 24, 2018 |
Assignment 1 | - |

Jan 17, 2018 | Linear Classification: Perceptron Algorithm |
- | Notes by Avrim Blum |

Jan 19, 2018 | Decision Trees | Lecture5.pdf | Shared via Moodle |

Jan 24, 2018 | "Some cases of pathology diagnostics using ML" | Guest lecture by Prof. Amit Sethi | - |

Jan 31, 2018 | Bias Variance Tradeoff | Lecture6.pdf | Notes by Andrew Ng |

Feb 2, 2018 | Generalization errors + model selection | Lecture7.pdf | Notes on VC dimension (Section 11.1) |

Feb 7, 2018 | Assignment 2 released: Due Feb 16, 2018 |
Assignment 2 | - |

Feb 7, 2018 | MLE/MAP/Naive Bayes | Lecture8.pdf | Peter Robinson's slides on MLE vs. MAP |

Feb 9, 2018 | Logistic Regression | Lecture9.pdf | Chapter 3 from "Machine Learning" by Tom Mitchell, September 2017. |

Feb 14, 2018 | Regularization | Lecture10.pdf | Chapter 3.4 (everything before Eqn 3.45) of [TH-2009] and Section 3.3 from Tom Mitchell's book chapter referred above. |

Feb 16 & 21, 2018 | SVMs and Kernel Methods | Lecture12.pdf | Andrew Moore's slides on SVMs |

Feb 28 & March 2, 2018 | Midsem week | Lecture13.pdf | Midsem practice problems shared via Moodle |

March 7 & 9, 2018 | Clustering, mixture models + EM algorithm | Lecture14.pdf Lecture15.pdf |
Chapter 9 of [CB-2006] |

March 14, 2018 | Dimensionality Reduction | - | A Tutorial on PCA by Jonathon Shlens |

March 16, 2018 | Introduction to Neural Networks + Backpropagation | Lecture17.pdf | Neural Networks and Deep Learning, Chapters 1 & 2 |

March 19, 2018 | Assignment 3 released: Due March 28, 2018 |
Assignment 3 | - |

March 21, 2018 | Neural Networks continued | - | Notes on multi-layer NNs and backpropagation |

March 23, 2018 | Guest lecture by Prof. Arjun Jain: Deep Learning for Computer Vision | Slides | - |

March 28, 2018 | Deep Learning for Speech and Language Processing | Lecture19.pdf | - |

April 4, 2018 | Ensemble learning: Boosting and Bagging | - | Chapter 14 of [CB-2006] |

April 6, 2018 | Ensemble learning continued | - | - |

April 11, 2018 | Hidden Markov Models | Lecture22.pdf | Chapter 9 of Jurafsky & Martin, 2016 |

April 11, 2018 | Assignment 4 released: Due May 2, 2018 |
Assignment 4 | - |

April 13, 2018 | Sequence models continued | - | Chapter 9 of Jurafsky & Martin, 2016 |

April 11, 2018 | Last lecture | Lecture24.pdf | Slides from David Silver's course on RL |

Topic | Additional reading |
---|---|

Linear Regression | Chapters 3.1-3.3 of [CB-2006] |

Linear models for classification | Chapter 4.1 of [CB-2006] |

Statistical Learning Theory | Vapnik, V., An Overview of Statistical Learning Theory, IEEE Transactions on Neural Networks, Vol. 10, pp. 988-999, 1999. |

Discriminative (LR) vs. Generative (NB) models | Ng, A., On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes, NIPS, 2001. |

Support Vector Machines and Kernel Methods | Christopher Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, 1998. |

NNs as Universal Approximators | Kurt Hornik, Maxwell Stinchcombe and Halbert White, Multilayer Feedforward Networks are Universal Approximators, Neural Networks, 1989. |

Deep Learning | Goodfellow, I., Bengio, Y., Courville, A., Deep Learning, Part II, MIT Press, 2016. |

- Understanding Machine Learning. Shai Shalev-Shwartz and Shai Ben-David. Cambridge University Press. 2017. [SS-2017]
- The Elements of Statistical Learning. Trevor Hastie, Robert Tibshirani and Jerome Friedman. Second Edition. 2009. [TH-2009]
- Foundations of Data Science. Avrim Blum, John Hopcroft and Ravindran Kannan. January 2017. [AB-2017]
- Pattern Recognition and Machine Learning. Christopher Bishop. Springer. 2006. [CB-2006]

**Website credit:** This is based on a Jekyll template.