CS 726: Advanced Machine Learning (Fall 2016)

Course description

• Probabilistic Graphical models: In this part we will cover how to build supervised and unsupervised models involving a large number of unknowns. The unknowns are modeled probabilistically using graphical models, which include Bayesian networks and Markov Random Fields. We will discuss these models in the context of applications like NLP, image processing, and speech recognition.
• Deep learning: In this part we will introduce the topic of deep learning using neural networks for various supervised learning tasks. We will see how modern deep learning networks remove the need for feature engineering in the context of some of the same applications as above.

The course assumes basic knowledge of probability and statistics and linear algebra. A quiz will be administered within the course drop deadline to help you assess if you have the requisite background for the course. Chapters 2 and 3 of the Deep-learning book are a good place to refresh the necessary required background. Also, the course assumes basic background in machine learning, for example as covered in Chapter 5 of the Deep-learning book.

The course is open to CS MTechs, PhD, DD and BTech students. Students of other departments should approach for permission only if they meet the necessary pre-requisites. Third year BTech students need to take prior permission from the instructor for enrolling in the course.

Approximate credit structure

• 25% Mid-semester exam
• 25-30% End semester exam
• 12% Graded paper homeworks
• 10% Quizzes
• 13-18% Project + programming homework

Reading List

Probabilistic Graphical Models: Principles and Techniques, by Daphne Koller and Nir Friedman, MIT Press, 2009.

Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, MIT Press, 2016.

The course calendar will provide links to other relevant papers and book chapters for specific topics.
Other relevant text books:

• A Primer on neural networks for natural language processing, by Yaov Goldbeg.
• R. G. Cowell, A. P. Dawid, S. L. Lauritzen and D. J. Spiegelhalter. "Probabilistic Networks and Expert Systems". Springer-Verlag. 1999.
• M. I. Jordan (ed). "Learning in Graphical Models". MIT Press. 1998. Collection of papers. These appear collated here.
• J. Pearl. "Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference." Morgan Kaufmann. 1988.
• Graphical models by Lauritzen, Oxford science publications
• F. V. Jensen. "Bayesian Networks and Decision Graphs". Springer. 2001.
• Neural Networks and Deep Learning by Michael Nilson

• Vincent Vanhouck's course on Udacity
• Stanford's course on Convolutional Neural Networks for Visual Recognition
• Hinton's coursera offering
• Lecun's course on Deep Learning at NYU