Reflection Removal

This project was done as the course project for the Digital Image Processing course and was mentored by Prof. Vineet Gandhi.

Reflection in images is a big problem in photography. In this project, we implement and evaluate one the recent algorithms described in literature that attempt to computationally remove reflections. Multiple images were tested on the algorithm and various test cases have been shown towards the end. All scripts written by me for this project are available on gitlab, here.

Photographs taken through glass windows often contain both the desired scene and
undesired reflections. In this project, we use ghosting cues that arise from shifted double reflections of the reflected scene off the glass surface to exploit asymmetry between the transmission and reflections layers of the surface.

 

screenshot-from-2017-01-13-013540

Inferring from the Ghosting Cues

We model Ghosting caused due to double paned reflection (R1, R2) as a double impulse kernel. Here we consider only the first two reflections and ignore higher order reflections.

screenshot-from-2017-01-12-231313

screenshot-from-2017-01-12-232007

screenshot-from-2017-01-13-012628

Layer Separation Algorithm

We model the layer separation as an Optimization problem which consists of minimizing the Reconstruction Cost and Maximizing the Patch Priors of the Reflection and Transmission layers subject to the non negativity constraints. We use the Estimated Patch Prior Log Likelihood to determine the patch priors for the  Transmission and Reflection layers. Such a cost function is non convex because of the presence of GMM priors. Hence, the cost function is modeled as a half Quadratic Optimization Problem.

screenshot-from-2017-01-13-013844

Post Processing

In order to preserve the color palette in the original image,  we rescale the color intensities in the transmitted image to retain the original flavor of the image.

Screenshot from 2017-01-12 23:05:00.png

Results:

References:

  1. D. Zoran and Y. Weiss. From learning models of natural image patches to whole image restoration. In IEEE International Conference on Computer Vision, pages 479–486, 2011.
  2. Y. Diamant and Y. Y. Schechner. Overcoming visual reverberations. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  3. Shih, Y., Krishnan, D., Durand, F., Freeman, W.T.: Reflection removal using
    ghosting cues. In: Computer Vision and Pattern Recognition (CVPR), 2015 IEEE
    Conference on, IEEE (2015) 3193–3201
Advertisements