Deep Image Matting
About
Image matting is a fundamental computer vision problem and has many applications. Previous algorithms have poor performance when an image has similar foreground and background colors or complicated textures. The main reasons are prior methods 1) only use low-level features and 2) lack high-level context. In this paper, we propose a novel deep learning based algorithm that can tackle both these problems. Our deep model has two parts. The first part is a deep convolutional encoder-decoder network that takes an image and the corresponding trimap as inputs and predict the alpha matte of the image. The second part is a small convolutional network that refines the alpha matte predictions of the first network to have more accurate alpha values and sharper edges. In addition, we also create a large-scale image matting dataset including 49300 training images and 1000 testing images. We evaluate our algorithm on the image matting benchmark, our testing set, and a wide variety of real images. Experimental results clearly demonstrate the superiority of our algorithm over previous methods.
Key insights
What problem does deep image matting solve?
Image matting estimates foreground transparency (alpha) from an image and trimap. Prior methods rely on color, failing when foreground and background colors overlap. This paper uses a two-stage deep network to learn structure directly, plus a large synthetic dataset.
How does this approach differ from prior matting methods?
Unlike prior deep methods that refine existing matting outputs, this network learns alpha end-to-end from image and trimap. On the new Composition-1k test set (1000 images), the full model achieves 50.4 SAD versus 161.4 for DCNN Matting.
What practical use does this matting model have?
The paper documents matting as key technology for image editing and film production workflows. A user study on 31 real images via Amazon Mechanical Turk showed users preferred this method over seven prior methods, demonstrating generalization beyond benchmarks.
What limitations and uncertainties do the authors note?
The authors note it is unclear whether ranking changes between benchmark and real-image tests reflect overfitting to alphamatting.com or failure of standard error metrics to capture human perception. The synthetic dataset's composited nature raised initial bias concerns.