Research Summary
A long standing goal of computer graphics has been to create
"interesting" images. Traditionally the input to the graphics system
is a scene consisiting of geometric primitives, composed of different
materials and a set of light sources. A relatively recent and
alternate technique, Image based Modeling and Rendering (IBMR)
has emerged for creation of photorealistic images. The goal of
image-based modeling and rendering is to capture a scene from
pre-acquired imagery and construct a sufficient model to allow
realistic rendering of novel views. There are several advantages to
this approach:
- The display algorithms for Image based Rendering require modest
computational resources and are thus suiatble for real-time
implementation on workstations and personal computers.
- The cost of interactively viewing the scene is independent of the
scene complexity.
- The source of the pre-acquired imagery can be real or virtual
environment, i.e. from digitized images or rendered models. In fact
the two can be mixed together.
My research interests lie at this exciting confluence of
Computer Graphics and Computer Vision, where they
reinforce each other to achieve realism in both analyzing and
synthesizing the beautiful visual world surrounding us. I am
particularly interested to understand the complex way in which the
appearance of real or virtual world objects changes under
variable illumination, viewpoint and pose. To this end, I
strive to develop novel and efficient algorithms to better extract,
represent and regenerate visual information from
photographs/images.
Thesis Abstract
The inherent rigidity of Image-based Rendering (IBR) techniques,
i.e., images once captured cannot be modified is a major limitation
which cannot satisfy computer graphics needs. Our goal is to push this
limit and provide novel scene synthesis beginning only with images of
the scene as input.
To this end, we propose techniques to introduce dynamics into
Image based Rendering techniques. In any IBR model, there exists two
aspects, the scene (whose image is captured) and the viewpoint (from
which the image is captured). The possibility of novelty in "scene
synthesis" therefore lies in both these two aspects. Specifically,
- Image-based Relighting (IBRL): IBRL techniques capture
reference images of a scene/object under a set of illumination
conditions; and then use them to synthetically generate an image of
the scene under novel illumination configurations. However, to ensure
visual fidelity, a huge number of images need to be captured resulting
in a drastic increase in the storage and computational
requirements. We propose an efficient novel two-stage algorithm which
produces realistic renderings.
- Environment Matting (EM): EM refers to discovering the
complex illumination effects emanating from the interaction of the
environment with a transparent object. Prior techniques use a large
number of (monochrome/two-tone) probing images to extract the
matte. We propose an efficient EM technique which uses multiple colors
as cues, and uses a holistic cube as the environment.
- Object Pose: Traditionally, creating animations using only
images either meant 2D sketch animation or, synthesizing, from videos
of a moving object, a similar motion sequence of the object in any
environment. The knowledge of the desired motion of the object is a
prerequisite. We present a technique to create realistic animation
sequences of an object moving along any arbitrary path in any
specified environment given only images of the static object.
- Viewpoint: Generating detailed views of a scene from novel
viewpoints, once authentic imagery of the scene has been acquired
using a camera gantry, or a handheld camera, is a standard requirement
for computer graphics. Moving from static novel images, a walkthrough
or a camera walk through the implied virtual world is often
desirable, but the repeated access of the huge acquired image data
makes the task increasingly computationally and memory intensive. We
focus on providing real-time walkthroughs using an IBR technique,
Light Field Rendering.
We hope our work will take the CG community a step closer towards
realizing
images as a CG primitive.
Pre-Synopsis
