Systems for Machine Learning

Mythili Vutukuru
Department of Computer Science and Engineering, IIT Bombay

This page contains the course material for CS 794 (Systems for Machine Learning). In this course, you will learn about the infrastructure that underpins modern deep learning systems. The topics covered in this course include GPU architecture and networking, ML programming using CUDA and frameworks like PyTorch, hardware-aware performance optimizations, distributed training, and LLM inference optimizations.

Pre-requisites: Students should have taken a basic course in machine learning, and be familiar with concepts related to neural network training and inference. The course will involve a significant programming component in CUDA and PyTorch, so students must be comfortable with C++ and Python programming in a Linux environment. Students are expected to use free GPU resources available on cloud platforms to solve the take-home assignments.

News: We are working on a textbook for Systems of Machine Learning. Please find below the draft chapters, with accompanying slides and programming assignments. You can check back here for more updates soon.

Chapter#	Title	Chapter PDF	Slides	Programming Assignments
1	Introduction to SysML
2	Review of Deep Learning Concepts
3	Programming AI Hardware	pdf	link	GitHub link
4	Hardware-aware Optimizations
5	Machine Learning Programming Frameworks
6	Distributed Training
7	Networking Optimizations
8	LLM Inference Optimizations

The course material for the Spring 2026 offering of this course is archived below.

Lecture#	Topics	Slides	References	Programming Assignments
0	Introduction to the course	slides	Machine Learning Systems Ch. 1	PA0: Kaggle setup
1	Overview of deep learning concepts	slides	Machine Learning Systems Ch. 4 The Hundred Page Language Model Book Ch. 4 Youtube Playlist on Neural Networks	PA1: KV caching for GPT model in PyTorch
2	Hardware for AI acceleration	slides	Machine Learning Systems Ch. 11 How to Scale Your Model Part 2, 12
3	Hardware-aware performance optimizations	slides	Machine Learning Systems Ch. 10, 11 How to Scale Your Model Part 2 FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
4	CUDA programming	slides	CUDA programming guide Eunomia Basic Cuda Tutorial You can use a Kaggle GPU-accelerated notebook, or leetgpu.com playground to practice CUDA programming	PA2: Optimizations to Matrix Multiplication PA3: Optimizations to a simple MLP PA4: Flash Attention
5	High-level ML programming frameworks	slides	Machine Learning Systems Ch. 7
6	Distributed training	slides	Machine Learning Systems Ch. 8 The Ultra-Scale Playbook by Hugging Face
7	LLM inference optimizations	slides	Efficient Memory Management for Large Language Model Serving with PagedAttention Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
8	Networking optimizations	slides	Alibaba HPN: A Data Center Network for Large Language Model Training Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms

References: The following textbooks (available online) provide a good background on several topics covered in the course. I am grateful to the authors for permitting me to use content from these books in my slides for the Spring 2026 offering.

Machine Learning Systems by Prof. Vijay Janapa Reddi.
The Hundred-Page Language Models Book for an overview of LLMs and The Hundred-Page Machine Learning Book for a recap of machine learning broadly, both by Andriy Burkov

Systems for Machine Learning

Mythili Vutukuru Department of Computer Science and Engineering, IIT Bombay

Mythili Vutukuru
Department of Computer Science and Engineering, IIT Bombay