Let’s Get the FACS Straight: Reconstructing Obstructed Facial Features

Tim Büchner, Sven Sickert, Gerd Volk, Christoph Anders, Orlando Guntinas-Lichius, Joachim Denzler

2023

Abstract

The human face is one of the most crucial parts in interhuman communication. Even when parts of the face are hidden or obstructed the underlying facial movements can be understood. Machine learning approaches often fail in that regard due to the complexity of the facial structures. To alleviate this problem a common approach is to fine-tune a model for such a specific application. However, this is computational intensive and might have to be repeated for each desired analysis task. In this paper, we propose to reconstruct obstructed facial parts to avoid the task of repeated fine-tuning. As a result, existing facial analysis methods can be used without further changes with respect to the data. In our approach, the restoration of facial features is interpreted as a style transfer task between different recording setups. By using the CycleGAN architecture the requirement of matched pairs, which is often hard to fullfill, can be eliminated. To proof the viability of our approach, we compare our reconstructions with real unobstructed recordings. We created a novel data set in which 36 test subjects were recorded both with and without 62 surface electromyography sensors attached to their faces. In our evaluation, we feature typical facial analysis tasks, like the computation of Facial Action Units and the detection of emotions. To further assess the quality of the restoration, we also compare perceptional distances. We can show, that scores similar to the videos without obstructing sensors can be achieved.

Download


Paper Citation


in Harvard Style

Büchner T., Sickert S., Volk G., Anders C., Guntinas-Lichius O. and Denzler J. (2023). Let’s Get the FACS Straight: Reconstructing Obstructed Facial Features. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 727-736. DOI: 10.5220/0011619900003417


in Bibtex Style

@conference{visapp23,
author={Tim Büchner and Sven Sickert and Gerd Volk and Christoph Anders and Orlando Guntinas-Lichius and Joachim Denzler},
title={Let’s Get the FACS Straight: Reconstructing Obstructed Facial Features},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP},
year={2023},
pages={727-736},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011619900003417},
isbn={978-989-758-634-7},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP
TI - Let’s Get the FACS Straight: Reconstructing Obstructed Facial Features
SN - 978-989-758-634-7
AU - Büchner T.
AU - Sickert S.
AU - Volk G.
AU - Anders C.
AU - Guntinas-Lichius O.
AU - Denzler J.
PY - 2023
SP - 727
EP - 736
DO - 10.5220/0011619900003417
PB - SciTePress