Course Information

• Instructor: Ajit Rajwade
• Office: SIA-218, KReSIT Building
• Email: ajitvr@cse DOT iitb DOT ac DOT in
• Lecture Venue: SIC-201 (2nd floor of KReSIT Building - even though ASC says CC 103)
• Lecture Timings: Monday and Thursday 7:00 pm to 8:25 pm (slot 13)
• Instructor Office Hours (in room SIA-218): TBD
• Teaching Assistants: Deepak Gupta and Dinesh Kumar Meena (deepgupta,dkmeena) @ cse DOT iitb DOT ac DOT in

Course Description

• This course will cover some state-of-the-art techniques for several applications in image processing and provide some rigorous theoretical foundation behind those.
• It will be of interest to students working in allied areas such as machine learning, statistics or signal processing, as well.

• Denoising under different situations
• Tomographic reconstruction
• Removal of motion blur
• Reflection removal
• Image-based forensics
• Image compression
• Compressive reconstruction - including reconstruction of video, MRI and hyperspectral images

Intended Audience and Pre-requisites

Textbooks

• "Natural Image Statistics", by Aapo Hyvarinen, Jarmo Hurri and Patrick Hoyer, Springer Verlag 2009, freely downloadable online
• "A Mathematical Introduction to Compressive Sensing", by Simon Foucart and Holger Rauhut, Birkhauser, 2013
  

Resources

• MATLAB at IITB
• MATLAB Tutorial: here or here
• MATLAB Image Processing Toolbox tutorial: here
• Matlab Tutorial
• The MathWorks - MATLAB Tutorial
• Matlab Primer
• On-line Matlab Help
• Writing Fast Matlab Code (pdf)
• Code Vectorization Guide
• Matlab Programmin Style Guidelines (pdf)

Grading scheme (tentative)

• Midsem 10%
• Cumulative Endsem 10%
• About 6-7 assignments involving programs and written problems 55% (individually or in pairs)
• Course project with demo and viva 25% (individually or in pairs). You are expected to put in significant effort in the course project and assignments.
• 100% attendance is expected - the only exception is documented emergencies or participation in conferences. Students with less than 80% attendance may get DX grade.
• Audit students must (1) write both exams and submit all assignments + project, and (2) have an aggregate score of at least 50%, for the AU grade

Date	Content of the Lecture	Assignments/Readings/Notes
2nd Jan	Course overview	Slides (check moodle)
5th Jan	Statistics of natural images Power law Correlation between a pixel and its neighbors Sparsity of DCT coefficients - Laplacian model Sparsity of wavelet coefficients, dependencies between wavelet coefficients in different sub-bands Bayesian models: likelihood and prior probability or probability density with examples	Slides (check moodle)
9th Jan	Bayesian models: likelihood and prior probability or probability density with examples Denoising or deblurring using a Laplacian signal prior; derivation of the ISTA algorithm in detail Denoising or deblurring using a Gaussian signal prior - leading to the Wiener filter	Slides (check moodle)
12th Jan	Genesis of the Laplacian model for DCT coefficients of natural images Lindeberg's central limit theorem, exponential distribution for patch variances in natural images	Slides (check moodle) Paper on Laplacian model for DCT coefficients
16th Jan	Denoising using dependencies between wavelet coefficients, and a modified Wiener filter update	Slides (check moodle) E. Simoncelli, "Bayesian denoising of visual images in the wavelet domain"
19th Jan	Semi-automated method for reflection removal using statistical properties of natural images Iterative Reweighted Least squares Algorithm (IRLS)	Slides (check moodle) A. Levin and Y. Weiss, "User assisted separation of reflections from a single image using a sparsity prior", associated code HW1 out
23rd Jan	Compressed Sensing Conventional sensing versus compressed sensing Application areas of compressed sensing: MRI, video, CT, hyperspectral images Shannon's sampling theorem and its limitations Candes' puzzling experiment The role of sparsity Concept of sensing matrix, representation matrix and incoherence between the two The key optimization problem in compressed sensing using L0 norm and its softening to L1 norm Number of independent columns of a sensing matrix	Slides (check moodle) HW1 out
30th Jan	Softening to L1 norm: linear programming Theorem by Candes, Romberg, Tao involving incoherence and sparsity Corollary to the theorem involving Fourier sensing matrix and signals sparse in canonical basis: comparison to Shannon's sampling theorem Intuition behind incoherence Concept of the restricted isometry property	Slides (check moodle) HW1 out
2nd Feb	Concept of the restricted isometry property Sufficient condition for compressive reconstruction of compressible signals with and without noise (theorem 3 and theorem 2 in the slides) Comparison between Theorems 1, 2, 3 Random sensing matrices and the restricted isometry property L1 versus L2 norm in compressed sensing	Slides (check moodle) HW2 out
6th Feb	L1 versus L2 norm in compressed sensing Candes' experiments and its results in terms of theorem 1 (leading to theorem 4): reconstruction of piecewise constant signals/images Concept of mutual coherence Theorem 5: sufficient conditions for compressive recovery using mutual coherence Gershgorin's disc theorem: relation between restricted isometry constant (RIC) and mutual coherence; comparison between the two Greedy algorithms for compressive reconstruction: matching pursuit (MP) and orthogonal matching pursuit (OMP)	Slides (check moodle) HW2 out
9th Feb	Rice single pixel camera (SPC) Rice (SPC) in video mode Architecture of compressive camera by El Gamal Video compressive sensing based on coded snapshots	Slides (check moodle) HW2 out
13th Feb	Video compressive sensing based on coded snapshots CASSI camera for hyperspectral imaging	Slides (check moodle) HW2 out
16th Feb	Applications of CS in Magnetic Resonance Imaging Discussion of project topics	Slides (check moodle) HW2 out
27th Feb	Midterm paper distribution
2nd March	Sketch of proof of key theorem on CS (theorem 3 in the slides) Associated lemmas on the RIP and other simple vector properties for the aforementioned proof Statement of improved version of theorem 3 - theorem 6, with RIP of order s instead of 2s (for s-sparse signals)	Slides (check moodle) HW2 out
13th March	Designing of compressed sensing matrices by minimization of mutual coherence: applications to the Hitomi video camera and demosaicing Method of Duarte-Carvajalino et al	Slides (check moodle) HW3 out
16th March	Dictionary Learning Dictionary learning: problem definition, sparse coding: problem definition Principal Components Analysis (PCA): derivation Application of PCA to face recognition (eigenfaces), image compression; PCA on natural image patches and its relation to DCT	Slides (check moodle) HW3 out
20th March	Motivation for overcomplete dictionaries Method of Olshausen and Field: sparsity constraints on sparse codes and gradient descent based dictionary updates Method of Optimal Directions (MOD) for Overcomplete dictionary learning	Slides (check moodle) HW3 out
23rd March	KSVD algorithm: sparse coding through OMP, dictionary update using Eckhart Young theorem for rank 1 approximations Applications of KSVD: compression, denoising, inpainting	Slides (check moodle) HW3 out
27th March	Blind compressed sensing: inferring KSVD dictionaries directly from compressive measurements Non-negative matrix factorization (NMF) and Non-negative sparse coding (NNSC) Poisson noise in images, Applications of NNSC in removal of Poisson noise	Slides (check moodle) HW4 out
30th March	Method of union of orthonormal basis Orthogonal procrustes for inferring orthonormal bases (applications of SVD) Tomographic Rconstruction Problem statement and definition Concept of radon transform and its relationship to tomographic projections Back-projection for tomography and its limitations Applications of tomography, Beer's law, 1st to 4th generation CT	Slides (check moodle) HW4 out
3rd April	Filtered backprojection: detailed derivation and use of Ram-Lak filter; relation between backprojection and the "true" Radon inverse Comparison between filtered backprojection and backprojection Tomography as a compressed sensing problem: empirical comparison to FBP Limitations in theory: Radon matrix does not obey RIP, incoherence properties Coupled tomographic reconstruction of similar slices	Slides (check moodle) HW5 out
6th April	Tomography under unknown angles: application scenarios Concept of image and projection moments, relation between image and projection moments and the angles of projection (in parallel beam tomography) Fundamental rotational ambiguity tomography under unknown angles Moment-based method for estimating projection angles and image moments from tomographic projections under unknown angles Ordering based method for tomography under unknown angles, assuming known distribution of the unknown angles - nearest neighbor algorithm (due to Basu and Bresler)	Slides (check moodle) HW5 out
10th April	Ordering based method for tomography under unknown angles, assuming known distribution of the unknown angles - nearest neighbor algorithm (due to Basu and Bresler) Comparison between ordering-based and moment based methods Laplacian eigenmaps for dimensionality reduction, with toy examples Application of Laplacian Eigenmaps for tomography under unknown angles	Slides (check moodle) HW5 out
13th April	PCA-based denoising for tomography from noisy measurements under unknown angles Compressive classification Classification from compressive measurements - maximum likelihood classifier, generalized maximum likelihood classifier, matched filter, smashed filter	Slides (check moodle) Slides for compressive classification (check moodle) HW5 out