Chii-Yuan Chuang, I-Chen Lin, Yung-Sheng Lo, Chao-Chih Lin



Producing a life-like 3D facial expression is usually a labor-intensive process. In movie and game industries, motion capture and 3D scanning techniques, acquiring motion data from real persons, are used to speed up the production. However, acquiring dynamic and subtle details, such as wrinkles, on a face are still difficult or expensive. In this paper, we propose a feature-point-driven approach to synthesize novel expressions with details. Our work can be divided into two main parts: acquisition of 3D facial details and expression synthesis. 3D facial details are estimated from sample images by a shape-from-shading technique. While employing relation between specific feature points and facial surfaces in prototype images, our system provides an intuitive editing tool to synthesize 3D geometry and corresponding 2D textures or 3D detailed normals of novel expressions. Besides expression editing, the proposed method can also be extended to enhance existing motion capture data with facial details.


  1. Beier, T., and Neely, S. “Feature-based Image Metamorphosis”, Proc. ACM SIGGRAPH'92, Pages 35-42, 1992.
  2. Blanz, V., Basso, C., Poggio, T., and Vetter, T. “Reanimating Faces in Images and Video”, Computer Graphics Forum 22 (3), Pages 641 - 650, 2003.
  3. Blanz, V., and Vetter, T. “A Morphable Model for the Synthesis of 3D Faces”, Proc. ACM SIGGRAPH'99, Pages 187-194, 1999.
  4. Breger, C., Covell, M., and Slaney, M. “Video rewrite: Driven Visual Speech with Audio”, Proc. ACM SIGGRAPH'97, Page 353-360, 1997.
  5. Deng, Z., Chiang, P-Y., Fox, P., and Neumann, U. “Animating Blendshape Face by Cross-Mapping Motion Capture Data”, Proceedings of the 2006 Symposium on Interactive 3D Graphics and Games, Pages 43-48, 2006.
  6. Ezzat, T., Geiger, G., and Poggio, T. “Trainable Videorealistic Speech Animation”, Proc. ACMSIGGRAPH'02, Pages 388-398, 2002.
  7. Fang, H., and Hart, J. C. “Textureshop: Texture Synthesis as a Photograph Editing Tool”, ACM Trans. on Graphics., Volume 23, Issue 3 (August 2004), Pages 354-359, 2004.
  8. Fletcher, R. “Practical Methods of Optimization”, Vol. 1, John Wiley & Sons, 1980.
  9. Forsyth. Ponce “Computer Vision: a Modern Approach”, Prentice Hall, 2003.
  10. Golovinskiy, A., Matusik, W., and Pfister, H. “A Statistical model for Synthesis of Detailed Facial Geometry”, ACM Trans. on Graphics, Volume 25, Issue 3, Pages: 1025-1034, 2006.
  11. Guenter, B. K., Grimm, C., Wood, D., Malvar, H. S., and Pighin, F. H., “Making Faces”, Proc.ACM SIGGRAPH'98, Pages 55-66, 1998.
  12. Horn, B.K. 1990. “Height and Gradient from Shading”, International Journal of Computer Vision, Vol. 5(1), Pages 37-75, 1990.
  13. Laurence, B., Gergo, K., Nadia M. T., Prem K., “Simulation of Skin Aging and Wrinkles with Cosmetics Insight”, Computer Animation and Simulation, Pagers 15-27, 2000
  14. Lin, I.-C., Yeh, J.-S., and Ouhyoung, M. “Extracting 3D Facial Animation Parameters from Multiview Video Clips”, IEEE Computer Graphics and Applications, Vol. 22(6), Pages 72-80, 2002.
  15. Liu, Z., Shan, Y., and Zhang, Z. “Expressive Expression Mapping with Ratio Images”, Proc. ACM SIGGRAPH'01, Pages 271-276, 2001.
  16. Nielson, G. M. “Scattered Data Modeling” IEEE Computer Graphics and Applications, Vol.13, Issue 1, Pages 60- 70, 1993.
  17. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. H. “Synthesizing Realistic Facial Expressions from Photographs”, Proc. ACM SIGGRAPH'98, Pages 75-84, 1998.
  18. Sifakis, E., Neverov, I. and Fedkiw, R., “Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data”, ACM Trans. on Graphics, Vol. 24, Issue 3, Pages 417-425, 2005.
  19. Tu, P-H., Lin, I-C., Yeh, J-S., Liang, R-H., and Ouhyoung, M. “Surface Detail Capturing for Realistic Facial Animation”, Journal of Computer Science and Technology, Vol. 19(5), Pages 618-625, 2004.
  20. Waters, K. “A Muscle Model for Animating ThreeDimensional Facial Expression”, Proc. ACMSIGGRAPH'87, Pages 17-24, 1987.
  21. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. “Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination”, ACM Trans. on Graphics, Vol. 24, Issue 3, Pages 756-764, 2005.
  22. Weyrich T., Matusik W., Pfister H., Lee J., Ngan A., Jensen W. And Gross M., “Analysis of Human Faces using a Measurement-Based Skin Reflectance Model” Proc. ACMSIGGRAPH'06, Pages 1013-1024, 2006.
  23. Yosuke B., Takaaki K., and Tomoyuki N. , “A Simple Method for Modeling Wrinkles on Human Skin”, Pacific Graphics 02, Pages 166-175, 2002.
  24. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. “Spacetime Faces: High-Resolution Capture for Modeling and Animation”, ACM Trans. on Graphics, Vol. 23, Issue 3, Pages 548-558, 2004.
  25. Zhang, Q., Liu, Z., Guo, B., Terzopoulos, D., and Shum, H. “Geometry-Driven Photorealistic Facial Expression Synthesis”, IEEE Trans. On Visualization and Computer Graphics, Vol. 12(1), Pages 48-60, 2006.
  26. Zhang, R., Tsai, P.-S., Cryer, J., and Shah, M. “Shape from Shading: A Survey”, IEEE Trans. On Pattern Analysis and Machine Intelligence, Vol. 21(8), Pages 690-706, 1999.
  27. Zhu L, Lee W.-S., "Facial Expression via Genetic Algorithms", Computer Animation and Social Agents , 2006

Paper Citation

in Harvard Style

Chuang C., Lin I., Lo Y. and Lin C. (2007). FEATURE-POINT DRIVEN 3D EXPRESSION EDITING . In Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP, ISBN 978-972-8865-72-6, pages 165-170. DOI: 10.5220/0002074201650170

in Bibtex Style

author={Chii-Yuan Chuang and I-Chen Lin and Yung-Sheng Lo and Chao-Chih Lin},
booktitle={Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,},

in EndNote Style

JO - Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,
SN - 978-972-8865-72-6
AU - Chuang C.
AU - Lin I.
AU - Lo Y.
AU - Lin C.
PY - 2007
SP - 165
EP - 170
DO - 10.5220/0002074201650170