FACE RECONSTRUCTION WITH STRUCTURED LIGHT

John Congote, Iñigo Barandiaran, Javier Barandiaran, Marcos Nieto, Oscar Ruiz

Abstract

This article presents a methodology for reconstruction of 3D faces which is based on stereoscopic images of the scene using active and passive surface reconstruction. A sequence of Gray patterns is generated, which are projected onto the scene and their projection recorded by a pair of stereo cameras. The images are rectified to make coincident their epipolar planes and so to generate a stereo map of the scene. An algorithm for stereo matching is applied, whose result is a bijective mapping between subsets of the pixels of the images. A particular connected subset of the images (e.g. the face) is selected by a segmentation algorithm. The stereo mapping is applied to such a subset and enables the triangulation of the two image readings therefore rendering the (x;y; z) points of the face, which in turn allow the reconstruction of the triangular mesh of the face. Since the surface might have holes, bilateral filters are applied to have the holes filled. The algorithms are tested in real conditions and we evaluate their performance with virtual datasets. Our results show a good reconstruction of the faces and an improvement of the results of passive systems.

References

  1. Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. (2009). The digital emily project: photoreal facial modeling and animation. In ACM SIGGRAPH 2009 Courses, SIGGRAPH 7809, pages 12:1- 12:15, New York, NY, USA. ACM.
  2. Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. (2010). High-quality single-shot capture of facial geometry. ACM Trans. on Graphics (Proc. SIGGRAPH), 29(3).
  3. Chen, S., Li, Y., and Zhang, J. (2008). Vision processing for realtime 3-D data acquisition based on coded structured light. Image Processing, IEEE Transactions on, 17(2):167-176.
  4. Greg Turk, M. L. (1994). Zippered polygon meshes from range images. In ACM SIGGRAPH. Computer Graphics Proceedings, Annual Conference Series, pages 311-318.
  5. Haro, G. and Pardís, M. (2010). Shape from incomplete silhouettes based on the reprojection error. Image Vision Comput., 28:1354-1368.
  6. Hartley, R. I. and Zisserman, A. (2004). Multiple view geometry in computer vision. Cambridge University Press.
  7. Leclercq, P., Liu, J., Woodward, A., and Delmas, P. (2005). Which stereo matching algorithm for accurate 3d face creation. In Klette, R. and Zunic, J., editors, Combinatorial Image Analysis, volume 3322 of Lecture Notes in Computer Science, pages 690-704. Springer Berlin - Heidelberg. 10.1007/978-3-540-30503-3 53.
  8. Marras, S., Ganovelli, F., Cignoni, P., Scateni, R., and Scopigno, R. (2010). Controlled and adaptive mesh zippering. In GRAPP - International Conference in Computer Graphics Theory and Applications.
  9. Onofrio, D., Tubaro, S., Rama, A., and Tarres, F. (2005). 3D Face Reconstruction with a four camera acquisition system. In Int'l Workshop on Very Low Bit-Rate Video Coding.
  10. Pighin, F. and Lewis, J. P. (2005). Introduction. In ACM SIGGRAPH 2005 Courses, SIGGRAPH 7805, New York, NY, USA. ACM.
  11. Ronda, J. I., Valdés, A., and Gallego, G. (2008). Line geometry and camera autocalibration. J. Math. Imaging Vis., 32:193-214.
  12. Salvi, J., Fernandez, S., Pribanic, T., and Llado, X. (2010). A state of the art in structured light patterns for surface profilometry. Pattern Recognition, 43(8):2666 - 2680.
  13. Scharstein, D. and Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision, 47(1-3):7-42.
  14. Scharstein, D. and Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, volume 1, pages I-195 - I-202 vol.1.
  15. Shen, C., O'brien, J., and Shewchuk, J. (2004). Interpolating and approximating implicit surfaces from polygon soup. In ACM Transactions on Graphics, pages 896- 904. ACM Press.
  16. Stylianou, G. and Lanitis, A. (2009). Image based 3d face reconstruction: A survey. International Journal of Image and Graphics, 9(2):217-250.
  17. Tsalakanidou, F., Forster, F., Malassiotis, S., and Strintzis, M. G. (2005). Real-time acquisition of depth and color images using structured light and its application to 3d face recognition. Real-Time Imaging, 11(5-6):358 - 369. Special Issue on Multi-Dimensional Image Processing.
  18. Weise, T., Leibe, B., and Gool, L. V. (2007). Fast 3d scanning with automatic motion compensation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07).
  19. Zhang, S., Royer, D., and Yau, S.-T. (2006). Highresolution, real-time-geometry video acquisition. In ACM SIGGRAPH 2006 Sketches, SIGGRAPH 7806, New York, NY, USA. ACM.
  20. Zhao, W., Chellappa, R., Phillips, P. J., and Rosenfeld, A. (2003). Face recognition: A literature survey. ACM Comput. Surv., 35:399-458.
Download


Paper Citation


in Harvard Style

Congote J., Barandiaran I., Barandiaran J., Nieto M. and Ruiz O. (2011). FACE RECONSTRUCTION WITH STRUCTURED LIGHT . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2011) ISBN 978-989-8425-47-8, pages 149-155. DOI: 10.5220/0003371401490155


in Bibtex Style

@conference{visapp11,
author={John Congote and Iñigo Barandiaran and Javier Barandiaran and Marcos Nieto and Oscar Ruiz},
title={FACE RECONSTRUCTION WITH STRUCTURED LIGHT},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2011)},
year={2011},
pages={149-155},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003371401490155},
isbn={978-989-8425-47-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2011)
TI - FACE RECONSTRUCTION WITH STRUCTURED LIGHT
SN - 978-989-8425-47-8
AU - Congote J.
AU - Barandiaran I.
AU - Barandiaran J.
AU - Nieto M.
AU - Ruiz O.
PY - 2011
SP - 149
EP - 155
DO - 10.5220/0003371401490155