Altering Facial Expression based on Textual Emotion

Mohammad Imrul Jubair, Md. Masud Rana, Md. Amir Hamza, Mohsena Ashraf, Fahim Ahsan Khan, Ahnaf Tahseen Prince

2022

Abstract

Faces and their expressions are one of the potent subjects for digital images. Detecting emotions from images is an ancient task in the field of computer vision; however, performing its reverse—synthesizing facial expressions from images—is quite new. Such operations of regenerating images with different facial expressions, or altering an existing expression in an image require the Generative Adversarial Network (GAN). In this paper, we aim to change the facial expression in an image using GAN, where the input image with an initial expression (i.e., happy) is altered to a different expression (i.e., disgusted) for the same person. We used StarGAN techniques on a modified version of the MUG dataset to accomplish this objective. Moreover, we extended our work further by remodeling facial expressions in an image indicated by the emotion from a given text. As a result, we applied a Long Short-Term Memory (LSTM) method to extract emotion from the text and forwarded it to our expression-altering module. As a demonstration of our working pipeline, we also create an application prototype of a blog that regenerates the profile picture with different expressions based on the user’s textual emotion.

Paper Citation

in Harvard Style

Jubair M., Rana M., Hamza M., Ashraf M., Khan F. and Prince A. (2022). Altering Facial Expression based on Textual Emotion. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 917-924. DOI: 10.5220/0010813100003124

in Bibtex Style

@conference{visapp22,
author={Mohammad Imrul Jubair and Md. Masud Rana and Md. Amir Hamza and Mohsena Ashraf and Fahim Ahsan Khan and Ahnaf Tahseen Prince},
title={Altering Facial Expression based on Textual Emotion},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP},
year={2022},
pages={917-924},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010813100003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP
TI - Altering Facial Expression based on Textual Emotion
SN - 978-989-758-555-5
AU - Jubair M.
AU - Rana M.
AU - Hamza M.
AU - Ashraf M.
AU - Khan F.
AU - Prince A.
PY - 2022
SP - 917
EP - 924
DO - 10.5220/0010813100003124
PB - SciTePress