Shape and Reflectance from RGB-D Images using Time Sequential Illumination

VISAPP 2016

M Hudon, A Gruson, Paul Kerbiriou, R Cozot, K Bouatouch

Abstract

In this paper we propose a method for recovering the shape (geometry) and the diffuse reflectance from an image (or video) using a hybrid setup consisting of a depth sensor (Kinect), a consumer camera and a partially controlled illumination (using a flash). The objective is to show how combining RGB-D acquisition with a sequential illumination is useful for shape and reflectance recovery. A pair of two images are captured: one non flashed (image under ambient illumination) and a flashed one. A pure flash image is computed by subtracting the non flashed image from the flashed image. We propose an novel and near real-time algorithm, based on a local illumination model of our flash and the pure flash image, to enhance geometry (from the noisy depth map) and recover reflectance information.

[Paper] [bibtex]

Videos

Shape reconstruction:

From left to right: kinect depth, bilateral filter, our result

Additional Notes: Erratum

In the paper, the image alignement from [Or-El et al. 2015] was made on CPU. Moreover the method described in [Or-El et al. 2015] does not necessarily require a mask.