Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance

Introduction

We consider the problem of decomposing an image into its material-dependent properties, known as reflectance or albedo, and its light-dependent properties, such as shading, shadows, specular highlights, and inter-reflectance. An example is shown in the next image, which includes `turtle' from the MIT dataset for intrinsic images [2].

At the top of the image you can see the input to our algorithm with the ground truth images from the dataset underneath it.

Abstract

We address the challenging task of decoupling material properties from lighting properties given a single image. In the last two decades virtually all works have concentrated on exploiting edge information to address this problem. We take a different route by introducing a new prior on reflectance, that models reflectance values as being drawn from a sparse set of basis colors. This results in a Random Field model with global, latent variables (basis colors) and pixel-accurate output reflectance values. We show that without edge information high-quality results can be achieved, that are on par with methods exploiting this source of information. Finally, we are able to improve on state-of-the-art results by integrating edge information into our model. We believe that our new approach is an excellent starting point for future developments in this field.

Results

Using the `turtle' image as input to our algorithm, you get the reflectance and shading image shown below (we set the parameter to image optimal; please see the paper for details).

All other results are contained in the results archive.

Publications

Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance
P. Gehler, C. Rother, M. Kiefel, L. Zhang, B. Schölkopf
Twenty-Fifth Annual Conference on Neural Information Processing Systems (NIPS) 2011 (corresponding poster)

Code

Please feel free to download the code from our publication to reproduce the results. The code is written in MATLAB. The files come with `minimize.m' [1] and `MPIKmeans' [3].

Our code is released under the BSD3 license for research purposes only.

Version 1.2 (tar.bz2)

Links

[1] www.gatsby.ucl.ac.uk/~edward/code/minimize

[2] MIT dataset

[3] MPIKmeans

Contact

Please feel free to contact us per mail: Martin Kiefel.