Authors:
Adrian Lubitz
1
;
Matias Valdenegro-Toro
2
and
Frank Kirchner
1
;
3
Affiliations:
1
Department of Computer Science, University of Bremen, 28359 Bremen, Germany
;
2
Department of AI, University of Groningen, 9747 AG Groningen, The Netherlands
;
3
Robotics Innovation Center, German Research Center for Artificial Intelligence, Bremen, Germany
Keyword(s):
Human-Robot Interaction, Perception, Dataset, Deep Learning.
Abstract:
Robots are becoming everyday devices, increasing their interaction with humans. To make human-machine interaction more natural, cognitive features like Visual Voice Activity Detection (VVAD), which can detect whether a person is speaking or not, given visual input of a camera, need to be implemented. Neural networks are state of the art for tasks in Image Processing, Time Series Prediction, Natural Language Processing and other domains. Those Networks require large quantities of labeled data. Currently there are not many datasets for the task of VVAD. In this work we created a large scale dataset called the VVAD-LRS3 dataset, derived by automatic annotations from the LRS3 dataset. The VVAD-LRS3 dataset contains over 44K samples, over three times the next competitive dataset (WildVVAD). We evaluate different baselines on four kinds of features: facial and lip images, and facial and lip landmark features. With a Convolutional Neural Network Long Short Term Memory (CNN LSTM) on facial i
mages an accuracy of 92% was reached on the test set. A study with humans showed that they reach an accuracy of 87.93% on the test set.
(More)