PhD Thesis

* The draft of my PhD thesis synopsis is available here [synopsis.pdf ~ 19MB]
* The draft of my full PhD thesis is available here [thesis.pdf ~ 28MB]

Optimisation Methods for Structure from Motion

The problem of recovering the shape and motion of 3D deformable objects from monocular video sequences (popularly called structure from motion) is extremely challenging and hard to solve.Integrating ideas from diverse fields such as differential geometry, machine learning, non-linear and global optimisation theory, we have proposed novel optimisation algorithms for solving the structure from motion problem.

We use variations of the elegant Motion Factorisation framework to solve the structure from motion problem. Since our problem contest is more practical, and challenging, finding the correct factorisation is difficult problem using existing methods. The prevalent closed from solutions are known to be ill-posed and seldom work well for challenging datasets. We overcome this hardship using a special iterative non-linear optimisation scheme.

Summarised below is the few of the advancements pertaining to optimisation techniques for motion factorisation problem that we were able to achieve.

Optimisation on Manifolds

The majority of the optimisation techniques found in computer vision literature are myopic to geometric properties of the underlying data or the process which generates the data. For example, in the problem we are handling the parameter space consists of geometric objects like rotation matrices which have non-Euclidean structures. The classical optimisation routines were designed with computational convenience in mind and do not explicitly handle the geometrical constraints placed on the data or the parameter space. Recent work in numerical techniques have shown how geometric properties of the data and the parameters can be preserved, whilst not hampering the performance of the optimiser. In our work we show how such methods can be adapted for motion factorisation problems. The key observation was that geometrical parameters resides in manifolds whose analytical structure has been well-understood. Moreover, these analytical structures can be used to preserve the geometrical properties during the optimisation i


This video sequence shows the 3D reconstruction we obtain of a synthetic dataset consisting of a 3D animation of a shark. The top row shows the actual 2D input point tracks, whereas the bottom row gives the plot of our reconstruction juxtaposed with the ground truth. The scene is viewed from a camera directly above the shark

3D reconstruction for a motion capture dataset. The top row shows the actual 2D input point tracks, middle row is the reconstructed shape rendered from a novel viewpoint, whereas the bottom row gives the plot of our reconstruction juxtaposed with the ground truth. The scene is viewed from a camera placed directly above the head

Relevant Publications
  1. Appu Shaji, Sharat Chandran and David Suter Manifold Optimisation for Motion FactorisationTo appear in the proceedings of International Conference on Pattern Recognition, 2008 [pdf - 840K]
  2. Appu Shaji and Sharat Chandran Riemannian Manifold Optimisation for Non-rigid Structure from Motion Proceedings of IEEE CVPR workshop, NORDIA 2008, Anchorage [pdf - 300K]

Data Driven Priors

The recent advancements in computational science has not only given us better, faster and more precise numerical and modelling techniques, but also enabled us to derive richer statistics from a wide and mammoth corpus of data (using machine learning techniques). This is further supplemented with availability of high-volume, high-quality databases (e.g: HumanEva, CMU datasets). In our work, we show that combining a non-rigid factorisation algorithm with learnt statistical models from archival motion capture database is useful in general, and particularly useful when only sparse number of features are available.

Rendering of the recovered 3d pose from novel viewpoints. The top row shows the raw frames with features overlayed. The middle and bottom shows the recovered 3d pose rendered from two novel view points. The front view is identical and not shown.

Relevant Publication
  1. Appu Shaji, Behjat Siddiquie, Sharat Chandran and David Suter Human Pose Extraction from Moncoular Videos using Non-rigid Factorization Proceedings of the British Machine Vision Conference, 2007 [pdf- 2.2MB]

Global Optimisation

Searching for a global optimal exhaustively through the entire parameter space is a near impossible task, especially for a high dimensional problem like ours, which is why iterative schemes based on motion factorisation have been used. We show that we can reach a solution which is at most $\epsilon$ away from the global minima for our cost function. We obtain this in tractable time by using a branch and bound optimisation scheme. At the top-level the proposed branch and bound scheme is a divide and conquer scheme where the original problem is split into easier problems over a smaller sub-domains of the search space. These easier problems provide lower and upper bounds to the original problem and can be used to guide the search effectively and non-exhaustively. In our work we are able to give a good approximation to the original problem using some recent work in convex approximations to trilinear systems.
Sheetal Joseph