CS 754: Advanced Image Processing(aka Inverse Problems in Image Processing), Spring 2021

CS 754 - Advanced Image Processing (aka Inverse Problems in Image Processing)

Course Information

Instructor: Ajit Rajwade
Office: SIA-218, KReSIT Building
Email: ajitvr AT cse DOT iitb DOT ac DOT in
Lecture Venue: MSTeams
Lecture Timings: Slot 14, i.e. Tuesday and Friday 5:30 to 7:00 pm. (I will share pre-recorded lecture videos.)
Instructor Office Hours: Immediately AFTER class on MSTeams (Details will be announced later). I may also hold periodic in-person doubt-clearing sessions.
Teaching Assistants: Sabyasachi Ghosh, Nitish Gangwar, Saurabh Telure, Vishal Sanoria, Krushnakant Bhattad

Course Description

Besides being just two- or three-dimensional arrays containing numbers (pixel intensity values), the images that are seen and acquired in everyday life have a number of very interesting properties. For example, small patches from different spatial regions of an image are quite similar to each other. Another example is that these images when transformed by certain kinds of matrix operations (eg: wavelet or discrete cosine transform) produce coefficients that are very sparse (i.e. most of them are zero or close to zero in value). These properties are a key to developing efficient and accurate algorithms for a variety of applications such as image noise removal, blur removal, image separation, compression and even image-based forensics. Not just that, these properties have also inspired the design of new sensors to acquire images much faster (a technology called compressed sensing) which is really crucial in tasks involving video or some types of images in medicine (such as magnetic resonance imaging - MRI). This course will systematically study several such techniques motivated by these interesting properties of natural images. The course will explore some key theory (including the proofs of some truly beautiful theorems), algorithms, as well as many applications. It will expose students to a broad range of modern, state-of-the-art techniques in image processing. A more detailed (tentative) syllabus can be accessed here. Many of the problems we will deal with in this class fall in the category of inverse problems, i.e. problems where the number of unknowns seems to exceed the number of knowns, at superficial glance, and there is also noise or system matrix contamination.

Many of the techniques studied in this course are very relevant in the era of BIG DATA, and they focus on intelligent acquisition of data. Can we acquire data in a compressed format so as to save resources: time, radiation given to a patient in a CT scanner, electrical power? Surprisingly, though we study these techniques in the context of image processing, the techniques are very generic and applicable in a variety of other domains. Last year at the beginning of the pandemic, many students in this course helped me and a colleague in developing techniques for saving resources used in COVID-19 RTPCR testing!

Need for the course

This course will cover some state-of-the-art techniques for several applications in image processing and provide some rigorous theoretical foundation behind those.
It will be of interest to students working in allied areas such as machine learning, statistics or signal processing, as well.
Besides CSE and EE, this course will be of interest to students from various disciplines such as remote sensing (CSRE), earth sciences, computational biology, computational physics, etc.

Image Processing Applications

The course will cover principled techniques that can be applied to some interesting image processing problems:

Denoising with different noise models, deblurring with different models of blur (motion/defocus)
Tomographic reconstruction (in computed tomography machines, as well as in biology/virology)
Reflection removal
Image-based forensics
Image compression
Compressive reconstruction - including reconstruction of video, Magnetic resonance imaging (MRI) and hyperspectral images
Some cutting-edge problems in image processing and machine learning: low rank matrix completion/recovery, phase retrieval, robust principal components analysis (RPCA) and compressive RPCA
Time permitting, we will study the application of different types of convolution neural networks (CNNs) for solving inverse problems -- this is a relatively less explored facet of CNNs. We will also try to explore theoretical aspects of neural networks.

Intended Audience and Pre-requisites

Intended for UG students from the second year onward, DDP students, and all PG students (MTech/PhD) from CSE/EE/EP/CSRE/Earth Sciences. You must have taken CS 663 or EE 610 or CS 725, otherwise you must discuss with me whether you have suitable background for this course.

Textbooks

"Natural Image Statistics", by Aapo Hyvarinen, Jarmo Hurri and Patrick Hoyer, Springer Verlag 2009, freely downloadable online
"A Mathematical Introduction to Compressive Sensing", by Simon Foucart and Holger Rauhut, Birkhauser, 2013
Statistical Learning with Sparsity: The Lasso and Generalizations", by Hastie, Tibshirani and Wainwright, CRC press (freely downloadable online)
Various research papers on key topics will be circulated

Resources

MATLAB at IITB
MATLAB Tutorial: here or here
MATLAB Image Processing Toolbox tutorial: here
Matlab Tutorial
The MathWorks - MATLAB Tutorial
Matlab Primer
On-line Matlab Help
Writing Fast Matlab Code (pdf)
Code Vectorization Guide
Matlab Programmin Style Guidelines (pdf)

Grading scheme (tentative)

Midsem: 10%
Cumulative Endsem: 15%
About 6-7 assignments involving programs, written problems and Google searches (yes you read that right -- but there's some (not so) fine print attached :-)) 50% (individually or in pairs)
Course project with demo and viva 25% (individually or in pairs). You are expected to put in significant effort in the course project (in terms of reading literature as well as implementation and mathematics) and assignments.
100% attendance is expected - the only exception is documented emergencies or participation in conferences. Students with less than 80% attendance may get DX grade.
Audit students must (1) write both exams and submit all assignments + project, and (2) have an aggregate score of at least 50%, for the AU grade.
I may put up optional quizzes online for you to check your understanding. These will not be graded.

Webpages of Previous Offerings

Course project

Project Topics and Instructions

Detailed Schedule

Date
Content of the Lecture
Assignments/Readings/Notes

Lecture 1: 4th Jan (Tue)

Course overview: intro to compressed sensing, tomography, dictionary and transform learning, low rank matrix recovery

Slides (check moodle)

Lecture 2: 7th Jan (Fri)

Compressed sensing (CS): introduction and motivation

Review of DFT and DCT, Review of JPEG, representation of a signal/image as a linear combination of basis vectors

Sparsity of natural images in transform bases

Slides (check moodle)

Lecture 3: 7th Jan (Fri)

Candes, Romberg, Tao: puzzling experiment; Basic optimization problem for CS involving the total variation (i.e. sum total gradient magnitude of the image)

Concept of incoherence between sensing matrix and representation basis

Slides (check moodle)

Lecture 4: 11th Jan (Tue)

Theorem 1 for reconstruction guarantees

Whittaker-Shannon sampling theorem

Comparison between Theorem 1 and Shannon's sampling theorem.

Slides (check moodle)

Lecture 5: 11th Jan (Tue)

Theorem 1 for reconstruction guarantees

Whittaker-Shannon sampling theorem

Comparison between Theorem 1 and Shannon's sampling theorem.

Slides (check moodle)

Date	Content of the Lecture	Assignments/Readings/Notes
Lecture 1: 4th Jan (Tue)	Course overview: intro to compressed sensing, tomography, dictionary and transform learning, low rank matrix recovery	Slides (check moodle)
Lecture 2: 7th Jan (Fri)	Compressed sensing (CS): introduction and motivation Review of DFT and DCT, Review of JPEG, representation of a signal/image as a linear combination of basis vectors Sparsity of natural images in transform bases	Slides (check moodle)
Lecture 3: 7th Jan (Fri)	Candes, Romberg, Tao: puzzling experiment; Basic optimization problem for CS involving the total variation (i.e. sum total gradient magnitude of the image) Concept of incoherence between sensing matrix and representation basis	Slides (check moodle)
Lecture 4: 11th Jan (Tue)	Theorem 1 for reconstruction guarantees Whittaker-Shannon sampling theorem Comparison between Theorem 1 and Shannon's sampling theorem.	Slides (check moodle)
Lecture 5: 11th Jan (Tue)	Theorem 1 for reconstruction guarantees Whittaker-Shannon sampling theorem Comparison between Theorem 1 and Shannon's sampling theorem.	Slides (check moodle)