Joint Depth and Alpha Matte Optimization via Stereo

Junlei Ma, Dianle Zhou, Chen Chen, Wei Wang


This study presents a novel iterative algorithm of joint depth and alpha matte optimization via stereo (JDMOS). This algorithm realizes simultaneous estimation of depth map and matting image to obtain final convergence. The depth map provides depth information to realize automatic image matting, whereas the border details generated from the image matting can refine the depth map in boundary areas. Compared with monocular matting methods, another advantage offered by JDMOS is that the image matting process is completely automatic, and the result is significantly more robust when depth information is introduced. The major contribution of JDMOS is adding image matting information to the cost function, thereby refining the depth map, especially in the scene boundary. Similarly, optimized disparity information is stitched into the matting algorithm as prior knowledge to make the foreground–background segmentation more accurate. Experimental results on Middlebury datasets demonstrate the effectiveness of JDMOS.


  1. A. Levin, D. Lischinski, and Y.Weiss.2008. A closed form solution to natural image matting. IEEE Trans. on PAMI, 30(2): 228-242.
  2. Y. Chuang, B. Curless, D.Salesin, and R.Szeliski. 2001. A Bayesian approach to digital matting. In CVPR (2), pages 264-271, 2001.
  3. D. Scharstein and R. Szeliski, 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, vol. 47, no. 1-3, pp.7-42.
  4. S. Mattoccia, S. Giardino, and A. Gambini, 2010. Accurate and efficient cost aggregation strategy for stereo correspondence based on approximated joint bilateral filtering,” in Proceedings of the Asian Conference on Computer Vision (ACCV 7810), vol. 38, pp. 371-380, Xi'an, China.
  5. L. Xu, O. C. Au, W. Sun et al., 2013. Stereo matching by adaptive weighting selection based cost aggregation. in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS'13), pp.1420-1423, Beijing, China.
  6. A. Arranz, A. Sanchez, and M. Alvar. 2012. Multi resolution energy minimisation framework for stereo matching. IET Computer Vision, vol. 6, no. 5, pp. 425-434.
  7. Y. Boykov, O. Veksler, R. Zabin. 2001. Fast Approximate Energy Minimization via Graph Cuts [J], IEEE Trans. on PAMI, 23(1):1222-1239.
  8. J. Wang and M. Cohen. 2007. Simultaneous matting and compositing. In CVPR.
  9. S. Baker, R. Szeliski, and P. Anandan. 1998. A layered approach to stereo reconstruction. In CVPR, pages 434-441.
  10. R. Szeliski and P. Golland. 1998. Stereo matching with transparency and matting. In ICCV, pages 517-525.
  11. L. Zitnick, S. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. 2004. High-quality video view interpolation using a layered representation. ACM Transaction on Graphics, 23(3):600-608.
  12. Ruigang Yang, Jiejie Zhu, Miao Liao, Zhigeng Pan. 2013. Joint depth and alpha matte optimization via fusion of stereo and time-of-flight sensor. IEEE Conference on Computer Vision and Pattern Recognition, vol. 00, no. , pp. 453-460, 2009,
  13. T. Matsuo, S. Fujita, N. Fukushima, and Y. Ishibashi. 2015. Efficient edge-awareness propagation via single-map filtering for edge-preserving stereo matching, in Three-Dimensional Image Processing, Measurement (3DIPM), and Applications, vol. 9393 of Proceedings of SPIE, International Society for Optics and Photonics, San Francisco, Calif, USA.
  14. J. Liu, X. Sang, C. Jia, N. Guo, Y. Liu, and G. Shi. 2014. Efficient stereo matching algorithm with edgedetecting, in Optoelectronic Imaging and Multimedia Technology III, vol. 9273 of Proceedings of SPIE, p. 7.
  15. B. J. Tippetts, D.-J. Lee, J. K. Archibald, and K. D. Lillywhite. 2011. Dense disparity real-time stereo vision algorithm for resource limited systems, IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 10, pp. 1547-1555.
  16. D. Chen, M. Ardabilian, and L. Chen. 2015. A fast trilateral filter based adaptive support weight method for stereo matching, IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 5, pp. 730-743.
  17. Rostam Affendi Hamzah and Haidi Ibrahim. 2016. Literature Survey on Stereo Vision Disparity Map Algorithms. Journal of Sensors, vol. 2016, Article ID 8742920, 23 pages, doi:10.1155/2016/8742920.
  18. Dheeraj Singaraju, Rene Vidal. 2011. Estimation of Alpha Mattes for Multiple Image Layers, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.33 n.7, p.1295-130

Paper Citation

in Harvard Style

Ma J., Zhou D., Chen C. and Wang W. (2017). Joint Depth and Alpha Matte Optimization via Stereo . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 426-433. DOI: 10.5220/0006191304260433

in Bibtex Style

author={Junlei Ma and Dianle Zhou and Chen Chen and Wei Wang},
title={Joint Depth and Alpha Matte Optimization via Stereo},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},

in EndNote Style

JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Joint Depth and Alpha Matte Optimization via Stereo
SN - 978-989-758-222-6
AU - Ma J.
AU - Zhou D.
AU - Chen C.
AU - Wang W.
PY - 2017
SP - 426
EP - 433
DO - 10.5220/0006191304260433