loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Authors: Samar Daou 1 ; Ahmed Rekik 1 ; 2 ; Achraf Ben-Hamadou 1 ; 2 and Abdelaziz Kallel 1 ; 2

Affiliations: 1 Laboratory of Signals, systeMs, aRtificial Intelligence and neTworkS, Technopark of Sfax, Sakiet Ezzit, 3021 Sfax, Tunisia ; 2 Digital Research Centre of Sfax, Technopark of Sfax, Sakiet Ezzit, 3021 Sfax, Tunisia

Keyword(s): Lipreading, Audiovisual Dataset, Human-Machine Interaction, Graph Neural Networks.

Abstract: In this paper, we propose a new lipreading approach for driver-car interaction in a cockpit monitoring environment. Furthermore, we introduce and release the first lipreading dataset dedicated to intuitive driver-car interaction using near-infrared driver monitoring cameras. In this paper, we propose a two-stream deep learning architecture that combines both geometric and global visual features extracted from the mouth region to improve the performance of lipreading based only on visual cues. Geometric features are extracted by a graph convolutional network applied to a series of 2D facial landmarks, while a 2D-3D convolutional network is used to extract the global visual features from the near-infrared frame sequence. These features are then decoded based on a multi-scale temporal convolutional network to generate the output word sequence classification. Our proposed model achieved high accuracy for both training scenarios overlapped speaker and unseen speaker with 98.5% and 92.2% r espectively. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.233.242.67

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Daou, S.; Rekik, A.; Ben-Hamadou, A. and Kallel, A. (2023). Near-infrared Lipreading System for Driver-Car Interaction. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP; ISBN 978-989-758-634-7; ISSN 2184-4321, SciTePress, pages 631-638. DOI: 10.5220/0011692300003417

@conference{visapp23,
author={Samar Daou. and Ahmed Rekik. and Achraf Ben{-}Hamadou. and Abdelaziz Kallel.},
title={Near-infrared Lipreading System for Driver-Car Interaction},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP},
year={2023},
pages={631-638},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011692300003417},
isbn={978-989-758-634-7},
issn={2184-4321},
}

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP
TI - Near-infrared Lipreading System for Driver-Car Interaction
SN - 978-989-758-634-7
IS - 2184-4321
AU - Daou, S.
AU - Rekik, A.
AU - Ben-Hamadou, A.
AU - Kallel, A.
PY - 2023
SP - 631
EP - 638
DO - 10.5220/0011692300003417
PB - SciTePress