CS 725: Foundations of Machine Learning (Autumn 2023)

An upper-level undergraduate course(s) in algorithms and data structures, a basic course on probability and statistics, basic understanding of linear algebra and multivariate calculus. If you feel you need to revise these topics, you can find some resources below. Chapters 2,3,and 4 of the Deep learning book is a compact resource. Homework assignments will require programming in Python. This is a first course on machine learning and no prior knowledge of machine learning is assumed.

The course is open only to CS and CMInDS Masters and PhD students. PG students of other departments can write for permission if they meet the necessary pre-requisites. UGs are not permitted. Please do not write for special permission.

Approximate credit structure

• 25% Mid-semester exam
• 35% End semester exam
• 12% Three programming homeworks (Before midsems)
• 15% Class project (After midsems)
• 10% In-class quizzes (best n-2 of n quizzes used for grading. There will be no compensation for missed quizzes)
• Scribe class notes (3%)

Audit Students: Audit students will get a pass if they attend at least 90% of the lectures, and get at least 70% marks in the in-class online quizzes. Audit students are also allowed to participate in the first programming assignment. If they do, then their 70% cutoff will be calculated based on the combination of SAFE quiz and marks in that assignment. Due to our limited TA support, we will *not* grade all assignments or exams of audit students. However, audit students are welcome to do the assignments and projects on their own.

Reading Material

Weekwise course calendar and classnotes (if any) will be made available on Moodle. Other reference books appear below.

Reference books

[ KM22]

``Probabilistic Machine Learning'' by Kevin Murphy. MIT Press, Mar 2022, Available online.

[ SS17]

Understanding Machine Learning. Shai Shalev-Shwartz and Shai Ben-David. Cambridge University Press. 2017. Available online.

[ Bis07]

Pattern recognition and machine learning by Christopher Bishop, Springer Verlag, 2006. website

[ HTF]

Hastie, Tibshirani, Friedman The elements of Statistical Learning Springer Verlag

[Mit97]

[DDL2019]

Dive into Deep Learning Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola, 2019

Supplementary books

[ PRS ]

Probability, Random Variables and Stochastic processes by Papoulis and Pillai, 4th Edition, Tata McGraw Hill Edition.

[ BV ]

Boyd and Vandenberghe Convex optimization Book available online:

GBC16

Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, MIT Press, 2016.

[ GS ]

Linear Algebra and Its Applications by Gilbert Strand. Thompson Books.

Other useful resources

• Andrew Ng's offering on Coursera
• Lecture notes from some of the previous offerings of the class.
• Demo applets for various machine learning operations