Learning Materials and Textbooks

• Lecture slides that will be regularly posted
• "Introductory Techniques for 3D Computer Vision", Emanuele Trucco and Alessandro Verri, Prentice Hall.
• Robot Vision, by B. K. P. Horn, MIT Press (Cambridge).
• Computer Vision: Algorithms and Applications, by Richard Szeliski (freely downloadable!)
• Computer Vision: A Modern Approach, Forsyth and Ponce, Pearson Education.

Computational Resources

• MATLAB at IITB (see readme file in the folder, for installation instructions)
• MATLAB Tutorial: here or here
• MATLAB Image Processing Toolbox tutorial: here
• Matlab Tutorial
• The MathWorks - MATLAB Tutorial
• Matlab Primer
• On-line Matlab Help
• Writing Fast Matlab Code (pdf)
• Code Vectorization Guide
• Matlab Programmin Style Guidelines (pdf)

Grading Policy

• Mid-sem exam: 20%
• Final exam (cumulative): 20%
• Programming assignments (five or six): 35% (all to be done in groups of 2-3 students)
• Course project: 20% (to be done in the same group of 2-3 students)
• Attendance and class participation: 5%
• Course project work will be presented by the student group during a viva at the end of the course. During this viva, each student in the group will be separately questioned, not only on the project work, but also the assignments. Each student is expected to contribute to each and every assignment and the course project. Out of the 35% set aside for assignments, the viva about the assignments will count for 5%.
• Audit requirements: You must write both exams, submit all assignments and the project, and score at least 40% to get an AU.

Other Policies

• Attendance is mandatory. Students with less than 80% attendance may be given a DX grade.
• Assignments will be given out (typically) once every two or three weeks. They must be submitted on or before the deadline. No late assignments will be accepted. The programming components of the assignments will typically involve MATLAB, so you must be willing to learn it quickly.
• We will adopt a zero-tolerance policy against any forms of plagiarism or any other form of cheating. Just don't do it! In cases of plagiarism, givers and takers will both be considered equally responsible.
• This course is (inherently) cumulative. The syllabus for the final exam will include everything taught during the semester.

Course Projects

Lecture Schedule:

Date	Content of the Lecture	Assignments/Readings/Notes	Interesting Extra Readings (not for exam)
3rd Jan	Course overview	Slides
7th Jan	Camera Geometry Transformations in 2D: translation, rotation, scaling, shearing; affine and rigid transformations Transformations in 3D: translation, rotation about X,Y,Z axis, rotation about arbitrary axis, 3D affine, number of degrees of freedom Composition of transformations in 2D and 3D with examples; concept og homogeneous coordinates in 2D and 3D Concept of pinhole camera, need for pinhole, geometry of perspective projection through pinhole camera	Slides
10th Jan	Concept of pinhole camera, need for pinhole, geometry of perspective projection through pinhole camera Weak perspective projection and orthographic projection Concept of image coordinate system and camera coordinate system; intrinsic camera parameters Concept of world coordinate system and its relationship to camera and image coordinate systems; extrinsic camera parameters Concept of camera calibration and basic aim of camera calibration	Slides	Telecentric lenses: image-space Telecentric lenses: object-space
14th Jan	Concept of camera calibration and basic aim of camera calibration Algorithm for derivation of camera matrix (size 3 x 4) due to Faugeras and Toscani; derivation of camera parameters from the camera matrix Motivation for camera calibration - implications for 3D reconstruction using two calibrated cameras Perspective invariant - cross-ratio	Slides	Cross-ratio at cut-the-knot A nice app for a metrology application
17th Jan	Perspective invariant - cross-ratio - proof of cross-ratio being a perspective invariant Use of cross-ratio and vanishing points in metrology - two different scenarios Introduction to planar homography	Slides	Cross-ratio at cut-the-knot A nice app for a metrology application
21st Jan	Introduction to planar homography Derivation for planar homography; algorithm for homography estimation given N pairs of corresponding points from two images of a planar scene Another camera calibration algorithm; orthocenter theorem for vanishing points	Slides HW1 out	Computer vision and football
24th Jan	Clarifications about homography Image Alignment Problem statement: physically and digitally corresponding points Motion models and degrees of freedom; non-rigid/deformable/non-parametric image alignment Control point based image alignment using least squares - derivation for pseudo-inverse Introduction to the SIFT algorithm Applications of image alignment: Google art project	Slides HW1 out	Google art project
27th Jan	Forward and reverse image warping - bilinear and nearest-neighbor interpolation Image alignment using image similarity measures: mean squared error, normalized cross-correlation Concept of field of view in image alignment using image similarity measures Monomodal and multimodal image alignment Concept of joint histograms and behaviour of joint histograms in multi-modal image alignment	Slides HW1 out
31st Jan	Concept of joint histograms and behaviour of joint histograms in multi-modal image alignment Concept of entropy and joint entropy, algorithm for multimodal registration by minimizing joint entropy Aspects of image registration: 2D/3D, motion model, monomodal or multimodal Application scenarios for image alignment: template matching, video stabilization, panorama generation, face recognition, 3D to 2D alignment	Slides HW2 out
3rd Feb	Least squares algorithm for determining orthonormal transformation between corresponding pairs of points: the orthogonal procrustes problem Robust Methods in Computer Vision Least squares problems and their relation to the Gaussian distribution on the noise Examples of outliers in computer vision Explanation of why the Gaussian distribution is unsuited to handling outliers Introduction to the Laplacian distribution	Slides Notes on the orthogonal procrustes problem Slides about the singular value decomposition HW2 out
7th Feb	Introduction to the Laplacian distribution and Generalized Gaussian distribution The importance of heavy-tailed distributions in robust statistics Mean versus median: L2 fit versus L1 fit Least median of squares algorithm (LMedS) RanSaC (random sample consensus) algorithm Use of RanSaC in robust determination of planar homographies Variants of RanSaC	Slides HW2 out
10th Feb	Structure from Motion Motion as a cue to inference of 3D structure from images Motion factorization algorithm by Tomasi and Kanade for inference of (sparse) 3D structure of a fixed object being observed by a moving orthographic camera (or a rigidly moving object, being observed by a fixed orthographic camera) Aspects of the above algorithm: Eckhart Young theorem in SVD, metric constraints for inference of motion parameters and 3D structure SVD: concept of SVD as a weighted summation of rank-one matrices	Slides Slides about the singular value decomposition HW2 out
17th Feb	Midsem review session
28th Feb	Optical Flow Dealing with the aperture problem: regularization Horn and Shunck method: algorithm using discrete formulation, steps of Jacobi's method for matrix inversion, and comments about limitations	Slides about Optical Flow	Original Paper by Horn and Shunck Simplified explanation of the above paper
3rd March	Distribution of midsem papers
14th March	Lucas-Kanade algorithm for optical flow Multi-scale Lucas-Kanade algorithm Comparison of Horn-Shunck and Lucas-Kanade algorithms Applications of optical flow	Slides about Optical Flow
17th March	Feature Point Tracking Feature point tracking: Kanade-Lucas-Kanade tracker Motion models: patch-wise translation and patch-wise affine Concept of a good feature point based on saliency (similar to criteria in Lucas-Kanade optical flow algorithm) Tracking of salient feature points: using translation and affine models Some results of KLT tracker Applications of feature point tracking: mosaicing, video stabilization, structure from motion	Slides Good features to track, Cornell University Report by Shi and Tomasi Detection and Tracking of Feature Points, Cornell University Report by Tomasi and Kanade
21st March	Adaboost Machine learning 101 jargon Outline of Adaboost algorithm for binary classification - weak and strong classifiers Concept of weight of weak classifier, weight of sample point in Adaboost Concept of family of weak classifiers	Slides
28th March	Theory behind Adaboost: objective function for Adaboost, and Adaboost as coordinate descent on this objective function Derivation of weights of weak classifiers and weights of training samples Comments on the generalization error of Adaboost	Slides
4th April	Adaboost for face detection Computation of Haar-like features Concept of classifier cascade for pruning away negative samples, concept of false positive rate and detection rate	Slides Robust Real-time Face Detection, paper by Viola and Jones, International Journal of Computer Vision (2004).
11th April	Concept of (binocular geometric) stereo, stereo baseline, stereo disparity Simplest case of stereo with aligned coordinate systems of the two cameras: inverse relation between depth and disparity Parameters of a stereo system Epipolar geometry: epipolar plane, left and right epipoles, left and right epipolar lines Fully calibrated stereo with unaligned coordinate systems Essential and fundamental matrices in a stereo system; eight point algorithm	Slides (upto and including slide 28 only) Read sections 7.1 and 7.3 from Trucco and Verri