Environment Matting Revisited

Abstract

The most convincing environment matting (EM) techniques use a large number of (monochrome, or two-tone) probing images to extract the matte. In this paper, we use \emph{multiple colors} as cues, noting that color encodes several bits of information. Compared to earlier works, we use substantially smaller number of preprocessed images in the form of a color cube environment. If $c$ colors are used for the cube each of whose faces is $k\times k$, the number of images we needed is $\lceil \log_{c}6k^2 \rceil$. The number six comes in becomes we want to emphasize that the problem needs to be solved in a complete environment. Further, we also provide a version in which only 1 image is used for real-time imaging purposes.

We show quantitative as well qualitative results in the form of images that exhibit sophisticated illumination effects. The rendered objects show effects such as highlights, simultaneous refraction and reflection (instead of pure specular refraction). Objects my be colored, and influence the attenuation of light. We have minimal memory and computational requirements. Applications of this work include the relighting for virtual and augmented reality.