Parts-Based Implicit 3D Face Modeling

Yajie Gu, Nick Pears

2024

Abstract

Previous 3D face analysis has focussed on 3D facial identity, expression and pose disentanglement. However, the independent control of different facial parts and the ability to learn explainable parts-based latent shape embeddings for implicit surfaces remain as open problems. We propose a method for 3D face modeling that learns a continuous parts-based deformation field that maps the various semantic parts of a subject’s face to a template. By swapping affine-mapped facial features among different individuals from predefined regions we achieve significant parts-based training data augmentation. Moreover, by sequentially morphing the surface points of these parts, we learn corresponding latent representations, shape deformation fields, and the signed distance function of a template shape. This gives improved shape controllability and better interpretability of the face latent space, while retaining all of the known advantages of implicit surface modelling. Unlike previous works that generated new faces based on full-identity latent representations, our approach enables independent control of different facial parts, i.e. nose, mouth, eyes and also the remaining surface and yet generates new faces with high reconstruction quality. Evaluations demonstrate both facial expression and parts disentanglement, independent control of those facial parts, as well as state-of-the art facial parts reconstruction, when evaluated on FaceScape and Headspace datasets.

Paper Citation

in Harvard Style

Gu Y. and Pears N. (2024). Parts-Based Implicit 3D Face Modeling. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 201-212. DOI: 10.5220/0012423200003660

in Bibtex Style

@conference{visapp24,
author={Yajie Gu and Nick Pears},
title={Parts-Based Implicit 3D Face Modeling},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2024},
pages={201-212},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012423200003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Parts-Based Implicit 3D Face Modeling
SN - 978-989-758-679-8
AU - Gu Y.
AU - Pears N.
PY - 2024
SP - 201
EP - 212
DO - 10.5220/0012423200003660
PB - SciTePress