VIEW-BASED APPEARANCE MODEL ONLINE LEARNING FOR 3D DEFORMABLE FACE TRACKING

Stéphanie Lefèvre, Jean-Marc Odobez

2010

Abstract

In this paper we address the issue of joint estimation of head pose and facial actions. We propose a method that can robustly track both subtle and extreme movements by combining two types of features: structural features observed at characteristic points of the face, and intensity features sampled from the facial texture. To handle the processing of extreme poses, we propose two innovations. The first one is to extend the deformable 3D face model Candide so that we can collect appearance information from the head sides as well as from the face. The second and main one is to exploit a set of view-based templates learned online to model the head appearance. This allows us to handle the appearance variation problem, inherent to intensity features and accentuated by the coarse geometry of our 3D head model. Experiments on the Boston University Face Tracking dataset show that the method can track common head movements with an accuracy of 3.2º, outperforming some state-of-the-art methods. More importantly, the ability of the system to robustly track natural/faked facial actions and challenging head movements is demonstrated on several long video sequences.

References

  1. Ahlberg, J. (2001). Candide 3 - an updated parameterised face. Technical Report LiTH-ISY-R-2326, Linköping University, Sweden.
  2. Cascia, M. L., Sclaroff, S., and Athitsos, V. (2000). Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3d models. In IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), volume 22.
  3. Chen, Y. and Davoine, F. (2006). Simultaneous tracking of rigid head motion and non-rigid facial animation by analyzing local features statistically. In British Machine Vision Conf. (BMVC), volume 2.
  4. Cootes, T., Edwards, G., and Taylor, C. (1998). Active appearance models. In European Conf. Computer Vision (ECCV), volume 2.
  5. Cootes, T., Taylor, C., Cooper, D., and Graham, J. (1995). Active shape models - their training and application. Computer Vision and Image Understanding, 61(1):38-59.
  6. DeCarlo, D. and Metaxas, D. (2000). Optical flow constraints on deformable models with applications to face tracking. Int. Journal of Computer Vision, 38(2):99-127.
  7. Dornaika, F. and Davoine, F. (2006). On appearance based face and facial action tracking. In IEEE Trans. On Circuits And Systems For Video Technology, volume 16.
  8. Gross, R., Matthews, I., and Baker, S. (2006). Active appearance models with occlusion. Image and Vision Computing Journal, 24(6):593-604.
  9. Jepson, A., Fleet, D., and El-Maraghi, T. (2003). Robust online appearance models for visual tracking. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), 25(10):1296- 1311.
  10. Lefèvre, S. and Odobez, J. (2009). Structure and appearance features for robust 3d facial actions tracking. In Int. Conf. on Multimedia & Expo.
  11. Li, Y. (2004). On incremental and robust subspace learning. Pattern Recognition, 37(7):1509-1518.
  12. Matthews, I., Ishikawa, T., and Baker, S. (2004). The template update problem. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), 26(6):810 - 815.
  13. Morency, L.-P., Whitehill, J., and Movellan, J. (2008). Generalized adaptive view-based appearance model: Integrated framework for monocular head pose estimation. In IEEE Int. Conf. on Automatic Face and Gesture Recognition (FG).
  14. Tu, J., Tao, H., and Huang, T. (2009). Online updating appearance generative mixture model for meanshift tracking. Machine Vision and Applications, 20(3):163-173.
  15. Vacchetti, L., Lepetit, V., and Fua, P. (2004). Stable realtime 3d tracking using online and offline information. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), 26(10):1385-1391.
  16. Xiao, J., Baker, S., and Matthews, I. (2004). Real-time combined 2d+3d active appearance models. In IEEE Conf. Computer Vision and Pattern Recognition (CVPR).
  17. Xiao, J., Moriyama, T., Kanade, T., and Cohn, J. (2003). Robust full-motion recovery of head by dynamic templates and re-registration techniques. Int. Journal of Imaging Systems and Technology, 13(1):85-94.
Download


Paper Citation


in Harvard Style

Lefèvre S. and Odobez J. (2010). VIEW-BASED APPEARANCE MODEL ONLINE LEARNING FOR 3D DEFORMABLE FACE TRACKING . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010) ISBN 978-989-674-028-3, pages 223-230. DOI: 10.5220/0002836002230230


in Bibtex Style

@conference{visapp10,
author={Stéphanie Lefèvre and Jean-Marc Odobez},
title={VIEW-BASED APPEARANCE MODEL ONLINE LEARNING FOR 3D DEFORMABLE FACE TRACKING},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010)},
year={2010},
pages={223-230},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002836002230230},
isbn={978-989-674-028-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010)
TI - VIEW-BASED APPEARANCE MODEL ONLINE LEARNING FOR 3D DEFORMABLE FACE TRACKING
SN - 978-989-674-028-3
AU - Lefèvre S.
AU - Odobez J.
PY - 2010
SP - 223
EP - 230
DO - 10.5220/0002836002230230