CS 754 - Advanced Image Processing (aka Inverse Problems in Image Processing)



Course Information

Course Description

Besides being just two- or three-dimensional arrays containing numbers (pixel intensity values), the images that are seen and acquired in everyday life have a number of very interesting properties. For example, small patches from different spatial regions of an image are quite similar to each other. Another example is that these images when transformed by certain kinds of matrix operations (eg: wavelet or discrete cosine transform) produce coefficients that are very sparse (i.e. most of them are zero or close to zero in value). These properties are a key to developing efficient and accurate algorithms for a variety of applications such as image noise removal, blur removal, image separation, compression and even image-based forensics. Not just that, these properties have also inspired the design of new sensors to acquire images much faster (a technology called compressed sensing) which is really crucial in tasks involving video or some types of images in medicine (such as magnetic resonance imaging - MRI). This course will systematically study several such techniques motivated by these interesting properties of natural images. The course will explore some key theory (including the proofs of some truly beautiful theorems), algorithms, as well as many applications. It will expose students to a broad range of modern, state-of-the-art techniques in image processing. A more detailed (tentative) syllabus can be accessed here. Many of the problems we will deal with in this class fall in the category of inverse problems, i.e. problems where the number of unknowns seems to exceed the number of knowns, at superficial glance, and there is also noise or system matrix contamination.

Many of the techniques studied in this course are very relevant in the era of BIG DATA, and they focus on intelligent acquisition of data. Can we acquire data in a compressed format so as to save resources: time, radiation given to a patient in a CT scanner, electrical power? Surprisingly, though we study these techniques in the context of image processing, the techniques are very generic and applicable in a variety of other domains. Last year at the beginning of the pandemic, many students in this course helped me and a colleague in developing techniques for saving resources used in COVID-19 RTPCR testing!

Need for the course Image Processing Applications

The course will cover principled techniques that can be applied to some interesting image processing problems:

Intended Audience and Pre-requisites

Intended for UG students from the second year onward, DDP students, and all PG students (MTech/PhD) from CSE/EE/EP/CSRE/Earth Sciences/CMiNDS/KCDH/IEOR/SysCon. You must have taken CS 663 or EE 610 or CS 725, otherwise you must discuss with me whether you have suitable background for this course.

Textbooks

Resources


Grading scheme (tentative)

Webpages of Previous Offerings


Course project

Project Topics and Instructions

Detailed Schedule

Date

Content of the Lecture

Assignments/Readings/Notes

Lecture 1: 5/1 (Mon)
  • Course overview
  • Intro to compressed sensing, tomography, dictionary learning, low rank matrix recovery
Lecture 2: 8/1 (Thu)
  • Compressed sensing (CS): introduction and motivation
  • Review of DFT and DCT, Review of JPEG, representation of a signal/image as a linear combination of basis vectors
  • Sparsity of natural images in transform bases
  • Candes, Romberg, Tao: puzzling experiment; Basic optimization problem for CS involving the total variation (i.e. sum total gradient magnitude of the image)
Lecture 3: 12/1 (Mon)
  • Whittaker-Shannon sampling theorem
  • Concept of incoherence between sensing matrix and representation basis
  • L0 and L1 norm optimization problems for compressed sensing; proof of uniqueness of the solution of the L0 problem
Lecture 4: 15/1 (Thu)
  • Theorem 1 and its relation to Shannon's sampling theorem; examples of bad/incompatible pairs of signal-support and measurement-subset
  • Intuition behind incoherence
  • Concept of restricted isometry property (RIP) and its relation to the nullspace property
Lecture 5: 19/1 (Mon)
  • Theorems 2 and 3 and comments on them
  • Motivation for use of L1 norm instead of L2 norm for compressed sensing
  • Interpretation of L1 norm optimization as a linear programming problem
Lecture 6: 22/1 (Thu)
  • Sample results for compressed sensing
  • CS for piecewise constant signals: Theorem 4
  • Algorithms for compressed sensing: matching pursuit (MP) and orthogonal matching pursuit (OMP)
  • Iterative Shrinkage and Thresholding Algorithm (ISTA)
Lecture 7: 29/1 (Thu)
  • Algorithms for compressed sensing: iterated shrinkage and thresholding algorithm (ISTA)
  • Rice single pixel camera, and its block-based version
Lecture 8: 2/2 (Mon)
  • Rice single pixel camera in video mode: separate frame-by-frame reconstruction, coupled reconstruction
  • CASSI camera for compressive hyperspectral imaging
  • Color filter arrays and color image demosaicing
Lecture 9: 7/2 (Sat)
  • Concept of tradeoff between spatial and temporal resolution in image acquisition
  • Video snapshot compressive sensing: concept, hardware description, results
  • Introduction to compressed sensing for pooled testing: noise model in RTPCR
Lecture 10: 9/2 (Mon)
  • Introduction to compressed sensing for pooled testing: Dorfman's algorithm, design of pooling matrices, use of family and contact tracing information, group sparsity
Lecture 11: 12/2 (Thu)
  • Proof sketch for theorem 3: use of Cauchy Schwartz Inequality, Triangle Inequality, Reverse Triangle Inequality, relationship between different norms
  • Theorem 6: use of RIC of order s, instead of order 2s
  • Theorem 5: problem P1 analyzed using mutual coherence
  • Mutual coherence versus RIC; Gershgorin's disc theorem to derive the relationship between them
  • Logan's result regarding recovery of a bandlimited signal given impulse noise
Lecture 12: 16/2 (Mon)
  • Sensing matrix design
  • Compressive classification
  • CS in MRI
Lecture 13: 19/2 (Thu)
    Dictionary Learning
    • Concept of dictionary learning and dictionary coefficients
    • Orthonormal and overcomplete dictionaries; overcompleteness and sparsity; sparse coding for orthonormal and overcomplete dictionaries
    • KMeans as a special case of dictionary learning
    • Dictionary learning using Olshausen's method, Method of optimal directions
    • Introduction to KSVD method for dictionary learning