Learning Materials and Textbooks

• Lecture slides that will be regularly posted
• "Introductory Techniques for 3D Computer Vision", Emanuele Trucco and Alessandro Verri, Prentice Hall.
• Robot Vision, by B. K. P. Horn, MIT Press (Cambridge).
• Computer Vision: Algorithms and Applications, by Richard Szeliski (freely downloadable!)
• Computer Vision: A Modern Approach, Forsyth and Ponce, Pearson Education.

Computational Resources

• MATLAB at IITB
• MATLAB Tutorial: here or here
• MATLAB Image Processing Toolbox tutorial: here
• Matlab Tutorial
• The MathWorks - MATLAB Tutorial
• Matlab Primer
• On-line Matlab Help
• Writing Fast Matlab Code (pdf)
• Code Vectorization Guide
• Matlab Programmin Style Guidelines (pdf)

Grading Policy

• Mid-sem exam: 20%
• Final exam (cumulative): 25%
• Programming or written assignments (5 or 6) to be done in groups of two students: total 50%. There may be an oral exam based on the assignments at the end of the semester (to ensure that each student has put effort in every assignment).
• Attendance and class participation: 5%
• If you are a super-keen student who wants to do a course project, you may do so. In that case, you may do just k-2 assignments (k = 5 or 6), and concentrate on a course project. Instructor approval is a must for this! This option is especially recommended (though not compulsory) for graduate students (masters/PhD) who are interested in the broad areas of computer vision/graphics/image processing.

Other Policies

• Attendance is mandatory. Students with less than 80% attendance may be given a DX grade.
• Assignments will be given out (typically) once every two or three weeks. They must be submitted on or before the deadline. No late assignments will be accepted. The programming components of the assignments will typically involve MATLAB, so you must be willing to learn it quickly.
• We will adopt a zero-tolerance policy against any forms of plagiarism or any other form of cheating. Just don't do it! In cases of plagiarism, givers and takers will both be considered equally responsible.
• This course is (inherently) cumulative. The syllabus for the final exam will include everything taught during the semester.

Lecture Schedule:

Date	Content of the Lecture	Assignments/Readings/Notes
06/01 (Mon)	Introduction, course overview	Intro
07/01 (Tue)	Camera geometry: Transformations of points and vectors in 2D (translation, rotation, scaling, shearing, reflection - affine transformation), concept of homogeneous coordinates, composition of affine transformations: example - reflection across an arbitrary line; Transformation of points and vectors in 3D: translation, rotation about X,Y,Z axis, rotation about an arbitrary axis	Slides Read a standard computer graphics textbook (eg: Foley van Dam) for transformation matrices in 2D and 3D.
09/01 (Thurs)	Affine transformations in 3D: rotation and reflection matrices, reflection about an arbitrary plane, properties of orthonormal matrices, composition of affine transforms Camera geometry - pinhole, perspective projections, relationship between image and object coordinats, weak-perspective and orthographic projections, vanishing points	Slides Chapter 2 (ignore section 2.2.3 and 2.5) of Trucco and Verri.
13/01 (Mon)	Camera calibration: concept of camera coordinate system and world coordinate system, concept of extrinsic and intrinsic parameters Intro to calibration algorithm	Slides Chapter 2 (section 2.4) and Chapter 6 of Trucco and Verri.
16/01 (Thurs)	Camera calibration algorithm Use of SVD in solving equations of the form Av = 0, where v is an unknown vector and A is a known matrix Use of a calibrated camera system Vanishing points and orthocenter property	Slides Chapter 2 (section 2.4) and Chapter 6 of Trucco and Verri.
20/01 (Mon)	Camera calibration 2nd algorithm Use of SVD in solving equations of the form Av = 0, where v is an unknown vector and A is a known matrix Use of a calibrated camera system: binocular system, finding height of an object using cross-ratio Planar homography	Slides Chapter 2 (section 2.4) and Chapter 6 of Trucco and Verri. Decide your homework partner for the semester and send me an email by the end of the week!
21/01 (Tue)	Planar homography Determining planar homography Camera with a lens - focal length, aperture width, depth of field, human eye as camera Singular decomposition (SVD) - its applications in linear algebra	Slides Slides for SVD Chapter 2 (section 2.4) and Chapter 6 of Trucco and Verri. Decide your homework partner for the semester and send me an email by the end of the week! Homework 1 is posted. Due 5th Feb before 11:59 pm
21/01 (Thurs)	Calculus of variations - concept of functional Derivation of the Euler Lagrange equation for a functional of the form J(y) = \int_{x_0}^{x_1} F(x,y(x),y'(x)) dx Example of length of a curve as a functional	Notes from here (ignore the subsection on "alternate derivation"). Local copy here. I will post some additional notes later. Read appendix A.6 of B. K. P. Horn Decide your homework partner for the semester and send me an email by the end of the week! Homework 1 is posted. Due 5th Feb before 11:59 pm
27/01 (Mon)	Calculus of variations - concept of functional Example of a projectile E-L equations for functional of the form J(y) = \int_{x_0}^{x_1} F(x,y(x),y'(x),z(x),z'(x)), i.e. one independent variable (x) and multiple dependent variables (z(x) and y(x)) E-L equation for functional of the form J(z) = \int \int_{D} F(x,y,z,z_x, z_y), i.e. multiple independent variables (x and y) and one dependent variable (z(x,y)) Example of the previous functional for image denoising	Notes from here (ignore the subsection on "alternate derivation"). Local copy here. I will post some additional notes later. Read appendix A.6 of B. K. P. Horn Optional but highly recommended reading: "Principle of Least Action - a special lecture almost verbatim" by Richard Feynmann (read upto the part where he talks about relativity). Local copy here.
28/01 (Tue)	Example on image denoising - concept of data fidelity term, regularizer and regularization parameter E-L equation with higher order derivatives Application in optics based on Fermat's principle that light takes the least-time path of travel	Class notes on variational calculus Read appendix A.6 of B. K. P. Horn Optional but highly recommended reading: "Principle of Least Action - a special lecture almost verbatim" by Richard Feynmann (read upto but not including the part on relativity). Local copy here.
30/01 (Thurs)	Shape from Shading (SfS): problem statement Scene radiance, image irradiance, surface reflectance model Lambertian reflectance model, Lambert's cosine law Variational formulation for SfS	Slides Sections 2.2.3 and 9.1 to 9.4 of Trucco and Verri (skip section 9.3.1)
3/2 (Mon)	Variational formulation for SfS Euler Lagrange and method to solve it Violation of integrability constraint	Slides Sections 2.2.3 and 9.1 to 9.4 of Trucco and Verri (skip section 9.3.1)
4/2 (Tue)	Violation of integrability constraint Methods to overcome or mitigate the violation of integrability constraint Photometric stereo - estimating surface normals of a Lambertian object given n >= 3 images in the same pose but different lighting directions	Slides Sections 2.2.3 and 9.1 to 9.4 of Trucco and Verri (skip section 9.3.1)
6/2 (Thurs)	Photometric stereo - detecting shadows and specularities, applications in relighting, advantages of a 3D object model over a 2D image Estimating depth from shading by Tsai and Shah's method	Slides Matlab demo - photometric stereo Sections 2.2.3 and 9.1 to 9.4 of Trucco and Verri (skip section 9.3.1) Optional reading for curious souls: Variational approach to shape from shading (by Brooks and Horn) Optional reading for curious souls: Approach of Tsai and Shah Homework 2 is posted. Due 12th feb and 27th Feb before 11:59 pm
10/2 (Mon)	Optical flow - concept and examples Brightness constancy equation, Aperture problem Horn and Shunck method (global technique) Lucas-Kanade method (local technique)	Slides Online notes by Ying Wu from Northwestern University (read only sections 1 and 2). Local copy here Homework 2 is posted. Due 12th feb and 27th Feb before 11:59 pm Homework 1 solutions are up Optional reading for curious souls: "Determining Optical Flow",Horn and Shunck, Artificial Intelligence, 1981
11/2 (Tue)	Discussion of Horn and Shunck method Discussion of Lucas-Kanade method, multiresolution version of Lucas-Kanade Barberpole illusion	Slides Online notes by Ying Wu from Northwestern University (read only sections 1 and 2). Local copy here Optional reading for curious souls: "Determining Optical Flow",Horn and Shunck, Artificial Intelligence, 1981 Optional reading for curious souls: "Horn and Shunck meets Lucas Kanade: Combining Local and Global Optic Flow Methods ", Bruhn, Weickert and Schnorr, International Journal of Computer Vision, 2005 Midsem exam on 18/2 (Tue) from 8:30-10:30 am in SIC 201/301
13/2 (Thurs)	Midterm revision
24/2 (Mon)	Distribution of midterm answer papers and discussion of solutions	Midterm solutions
25/2 (Tue)	Applications of Optical Flow Stereo - introduction, stereopsis in the brain Stereo reconstruction with aligned cameras	Slides Chapter 7 of Trucco and Verri, Chapter 13 of B K P Horn
27/2 (Thurs)	Stereo reconstruction with aligned cameras Intrinsic and extrinsic parameters of a stereo system Stereo reconstruction with non-aligned cameras but known parameters Epipolar geometry, epipolar constraint, essential matrix and fundamental matrix	Slides [See moodle] Chapter 7 of Trucco and Verri, Chapter 13 of B K P Horn
3/3 (Mon)	Epipolar geometry, epipolar constraint, essential matrix and fundamental matrix Eight-point algorithm to determine essential/fundamental matrix; location of epipoles from fundamental matrix Stereo triangulation under noise - 3D reconstruction when the intrinsic/extrinsic parameters of a stereo system are known 3D reconstruction when only the intrinsic parameters of a stereo system are known	Slides [See moodle] Chapter 7 of Trucco and Verri, Chapter 13 of B K P Horn HW2 solutions are up
4/3 (Tue)	3D reconstruction when only the intrinsic parameters of a stereo system are known 3D reconstruction when the intrinsic and extrinsic parameters of a stereo system are unknown stereo rectification stereo correspondence algorithms: ordering constraint, cross-correlation, normalized cross-correlation, sum of squared differences, [Dynamic programming algorithm to be covered later]	Slides [See moodle] Chapter 7 of Trucco and Verri, Chapter 13 of B K P Horn Homework 3 is out, due 13th March before 11:59 pm.
6/3 (Thurs)	HW2 solutions are up Feature point tracking (Kanade-Lucas-Tomasi tracker) Motion models, detection of occlusions Applications (brief discussion): mosaicing, stabilization, structure from motion	Slides Matlab code to understand the eigenvalues of the structure tensor Technical report (must read): "Good Features to Track" by Shi and Tomasi (Cornell University, 1993) Homework 3 is out, due 13th March before 11:59 pm.
10/3 (Mon)	(Lecture by Prof. Sharat Chandran) Structure from motion (SfM): problem statement and introduction Overview of the Tomasi-Kanade SVD-based factorization algorithm	Slides (courtesy: Prof. Sharat Chandran) Section 8.5.1 of Trucco and Verri Paper: Tomasi and Kanade, "Shape and Motion from Image Streams under Orthogrpahy: a Factorization Approach", IJCV 1991. (Read upto section 4)
12/3 (Tue)	(Lecture by Prof. Sharat Chandran) Overview of the Tomasi-Kanade SVD-based factorization algorithm for SfM Intuition and proof of the algorithm	Slides Section 8.5.1 of Trucco and Verri Paper: Tomasi and Kanade, "Shape and Motion from Image Streams under Orthogrpahy: a Factorization Approach", IJCV 1991. (Read upto section 4)
13/3 (Thurs)	(Lecture by Prof. Sharat Chandran) Intuition and proof of the Tomasi-Kanade factorization algorithm Mathematical issues	Slides Section 8.5.1 of Trucco and Verri Paper: Tomasi and Kanade, "Shape and Motion from Image Streams under Orthogrpahy: a Factorization Approach", IJCV 1991. (Read upto section 4)
18/3 (Tue)	Robust fitting in computer vision: motivation - least squares fitting as a maximum likelihood problem Effect of outliers on least squares solutions Examples of outliers in computer vision problems	Slides Appendix A.7 of Trucco and Verri Paper by Peter Meer: Robust parameter estimation in computer vision. HW3 solutions are up Homework 4 is out - due 1st April before 11:59 pm.
20/3 (Thurs)	Least Median squares technique for robust fitting RANSAC (RAndom SAmple Consensus) for robust fitting Applications of RANSAC: line fitting, homography estimation, fundamental matrix estimation	Slides Appendix A.7 of Trucco and Verri Paper by Peter Meer: Robust parameter estimation in computer vision. Homework 4 is out - due 1st April before 11:59 pm.
24/3 (Mon)	Discussion about HW4 in detail	Slides Appendix A.7 of Trucco and Verri Homework 4 is out - due 1st April before 11:59 pm.
25/3 (Tue)	SIFT algorithm: intuitive notion of a salient point, applications of salient point detection Step (1) of the SIFT algorithm: difference of Gaussians at different scales, detection of extrema in Difference of Gaussians pyramid, scale selection	Slides SIFT paper by David Lowe Homework 4 is out - due 1st April before 11:59 pm.
27/3 (Thurs)	Four major steps of SIFT: (1) Initial keypoint estimation, (2) Refinement of keypoint location and scale, elimination oif weak keypoints and keypoints on edges, (3) Keypoint orientation estimation, (4) Creation of SIFT descriptor Discussion of following properties of SIFT descriptors: rotation and scale invariance, illumination invariance	Slides SIFT paper by David Lowe Homework 4 is out - due 1st April before 11:59 pm.
1/4 (Tue)	Discussion of following properties of SIFT descriptors: rotation and scale invariance, illumination invariance Detailed discussion of step 1: concept of scale space (only introductory), Laplacian of Gaussian filter (LoG) and its relation to DoG filter, use of LoG/DoG in detection of blobs of various radii	Slides SIFT paper by David Lowe Homework 4 is out - due 1st April before 11:59 pm.
3/4 (Thurs)	Application of SIFT for object detection in cluttered scenes Discussion on the limitations of the video stabilization algorithm in HW4	Slides SIFT paper by David Lowe Homework 4 solutions posted.
7/4 (Mon)	Face detection: problem statement Introduction to the Adaboost algorithm	Slides (will be updated) Paper on face detection using Adaboost Homework 5 posted - due 16th April before 11:59 pm (No extensions).
8/4 (Tue)	Adaboost algorithm Statement of theorem about its training errors Concept of overfitting in machine learning Viola and Jones face detector: feature selection	Slides (will be updated) Paper on face detection using Adaboost Homework 5 posted - due 16th April before 11:59 pm (No extensions). Final exam timetable
10/4 (Thurs)	Viola and Jones face detector: feature selection Viola and Jones face detector: cascade of detectors; false positive rates, false negative rates, detection rates	Slides (will be updated) Paper on face detection using Adaboost Homework 5 posted - due 16th April before 11:59 pm (No extensions). Final exam timetable
12/4 (Sat)	Theoretical treatment of Adaboost: Upper bound on training error, Adaboost as a coordinate descent, Optimizing for alpha (weight of weak classifier), Optimizing for the weak classifier itself, Upper bound on training error as number of Adaboost rounds increase [This part is not on the final exam] Generalization capabilities of Adaboost (only summary of results): concept of classifier margins	Slides (will be updated) Paper on face detection using Adaboost Homework 5 posted - due 16th April before 11:59 pm (No extensions). Final exam timetable Study questions Homework solutions in one place: HW1, HW2, HW3, HW4, HW5.