loading
Papers Papers/2020

Research.Publish.Connect.

Paper

Authors: Shimeng Peng 1 ; Shigeki Ohira 2 and Katashi Nagao 1

Affiliations: 1 Department of Intelligent Systems, Graduate School of Informatics, Nagoya University, Nagoya, Japan ; 2 Information Technology Center, Nagoya University, Nagoya, Japan

ISBN: 978-989-758-417-6

ISSN: 2184-5026

Keyword(s): Mental States Recognition in Conversation, Multimodal Feature Analysis, Educational Data Analysis, Facial Data, Heart Rate, Machine Learning, Learning Analytics.

Abstract: Students’ mental states have been widely acknowledged as crucial components for inferring their learning processes and are closely linked with learning outcomes. Understanding students’ complex mental states including concentration, confusion, frustration, and boredom in teacher-student conversation could benefit a human teacher’s perceptual and real-time decision-making capability in providing personalized and adaptive support in coaching activities. Many lines of research have explored the automatic measurement of students’ mental states in pre-designed human-computer tasks. It still remains a challenge to detect the complex mental states of students in real teacher-student conversation. In this study, we made such an attempt by describing a system for predicting the complex mental states of students from multiple perspectives: facial and physiological (heart rate) cues in real student-teacher conversation scenarios. We developed an advanced multi-sensor-based system and applied it in small-scale meetings to collect students’ multimodal conversation data. We demonstrate a multimodal analysis framework. Machine learning models were built by using extracted interpretable proxy features at a fine-grained level to validate their predictive ability regarding students’ multiple mental states. Our results provide evidence of the potential value of fusing multimodal data to understand students’ multiple mental states in real-world student-teacher conversation. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.236.170.171

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Peng, S.; Ohira, S. and Nagao, K. (2020). Reading Students’ Multiple Mental States in Conversation from Facial and Heart Rate Cues. In Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-417-6; ISSN 2184-5026, pages 68-76. DOI: 10.5220/0009564000680076

@conference{csedu20,
author={Shimeng Peng. and Shigeki Ohira. and Katashi Nagao.},
title={Reading Students’ Multiple Mental States in Conversation from Facial and Heart Rate Cues},
booktitle={Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2020},
pages={68-76},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009564000680076},
isbn={978-989-758-417-6},
issn={2184-5026},
}

TY - CONF

JO - Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Reading Students’ Multiple Mental States in Conversation from Facial and Heart Rate Cues
SN - 978-989-758-417-6
IS - 2184-5026
AU - Peng, S.
AU - Ohira, S.
AU - Nagao, K.
PY - 2020
SP - 68
EP - 76
DO - 10.5220/0009564000680076

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.