Abstract: In the literature, environment matting refers to the complex process of discovering how light in an environment interacts with an object; notably, it may transfer through a transparent object, and undergo scattering. Modeling the 3D geometry, and the index of refraction, of chipped, non-uniformly dirty optically active substances is intractable; therefore image-based frameworks are useful. The most convincing techniques use a large number of (monochrome, or two-tone) probing images to extract the matte.

In this talk, after an introduction to the classic opaque matting (chroma key) problem, we describe the use of the frequency spectrum, wavelet, and color space decomposition as cues.

As a departure from existing techniques, our specific work (at IITB) uses environment mapping, and solves the problem with fewer number of backdrop images. Thus, while exhibiting sophisticated illumination effects of refraction, reflection, attenuation and scattering, we have minimal memory and computational requirements. Applications of this work include the relighting of objects for virtual and augmented reality.

(Joint work with Biswarup Choudhury and Deepali Singla)