Drivers Pressures States Recognition based on Heart Rate Variability

Kongjian Qin

, Hongwei Liu

, Mingjun Zhang

and Jinchong Zhang

CATARC Automotive Test Center (Tianjin) Co.,Ltd, Tianjin, China

China Intelligent and Connected Vehicles (Beijing) Research Institute Co.,Ltd., Beijing, China

Keywords: Drowsy Driving Detection, Driving Performance and Activity, Heart Rate Variability Analysis.

Abstract: Drivers pressures are major causes of road accidents, and thus drivers’ pressures states recognition become

an important topic in Advanced Driver Assistant System (ADAS). Physiological signals provide information

about the internal functioning of human body and thereby provide accurate, reliable and robust information

on the driver’s state. In this work, the several features, which are 8 heart rate variability features and 10

mathematical features, are trained using three classifiers: Support Vector Machine (SVM), K-nearest-

neighbor (KNN) and Ensemble. The algorithms based pNN5 and LF/HF achieved best performance in HRV

linear features evaluation, and the accuracy (AC), sensitivity (SE), specificity (SP) for Stress Recognition in

Automobile Drivers data are 89.0%, 91.8% and 77.3% respectively. The mathematical features result in

98.6%,99.1% and 91.5% for accuracy (AC), sensitivity (SE), specificity, respectively.

1 INTRODUCTION

It is easy for drivers to have mental stress during the

driving process, due to monotonous driving behavior.

For example, long-time traffic jams or driving on

heavily congested roads will increase the risk of

driver accidents. It is found potential hazards caused

by various driver pressures (Gibson 2000). However,

the recognition and classification of driver pressure

levels can be used as a monitoring and early warning

technology for ADAS, which has developed rapidly

in recent years.

In the selection of driver's physiological

parameters, the mental state recognition method

based on EEG has been proposed (Su 2008, C 2010,

F 2012, Hashemi 2014), but it is difficult to put into

actual use, due to the poor noise immunity and

difficulty of deployment of EEG acquisition in the

vehicle scene. It is proved that drivers’ skin

conductance and heart rate parameters are more

clearly related to their stress levels, according to

experiments (Singh 2014). In addition, driving

fatigue state detection has also been proposed, based

on analysis method of facial image and vehicle

driving data (Mbouna 2013, Jo 2014, Cyganek 2104).

However, these methods require special equipment to

be installed in the vehicle, such as a camera for facial

image collection or a data recording device for

accessing vehicle driving data.

Based on the driver's heart, the pressure detection

method has also attracted much attention. According

to the principle that sleep affects the driver's

autonomic nervous system (ANS) and heart activity,

fatigue detection is carried out, based on the

physiological parameters of the heart. In 2005, It is

proposed the most practical method to detect the

driver’s condition during actual driving based on

heart rate detection (Healey 2005). In 2016, it is (Chui

2016) et al. proposed a fatigue detection method

based on driver's electrocardiogram. The

psychological impact of the road traffic environment

on drivers and the resulting physiological burden and

changes in driving behavior were studied (Domestic

2001). In addition, some researchers detect fatigue

driving behavior based on Photo Plethysmo Graphy

(PPG) signals (Lee 2011). Although experiments

show that this method can obtain good performance,

it is difficult to stably obtain good ECG or PPG

signals, due to the influence of car motion.

Nevertheless, the heart rate variability (HRV)

extracted from ECG and PPG signals has a strong

anti-noise ability, which is an effective sign to

identify the internal state of the human body.

Based on the MIT-BIH autopilot pressure

recognition data set, this paper carried out research on

the pressure state recognition algorithm by using the

driver's HRV. After the ECG signal is preprocessed,

8 kinds of HRV parameters are extracted, and then 10

Qin, K., Liu, H., Zhang, M. and Zhang, J.

Drivers Pressures States Recognition based on Heart Rate Variability.

DOI: 10.5220/0011158800003444

In Proceedings of the 2nd Conference on Artiﬁcial Intelligence and Healthcare (CAIH 2021), pages 29-33

ISBN: 978-989-758-594-4

mathematical features with statistical significance are

obtained. After sorting all the features, support vector

machine (SVM) and K nearest neighbor (KNN) are

trained in two classifiers.

The structure of this article is as follows: in the

second section, we will introduce the new system

structure. The third section introduces the analysis

methods of the studied variables. The experimental

results are shown in the fourth section. Finally, this

article reviews the main conclusions and discusses

future work in the fifth section.

2 DRIVER STATE FEATURE

EXTRACTION BASED ON

HEART RATE VARIABILITY

2.1 Heart Rate Variability

The R wave is the highest peak in the ECG, and the

RR interval (RRI) is defined as the interval between

the R wave and the next R wave. HRV is the

fluctuation of RRI, a physiological phenomenon that

reflects the activity of the cardiac autonomic nervous

system. Therefore, HRV analysis is used to monitor

stress and cardiovascular disease. Although there are

two features of HRV: linear finite element features

and nonlinear features, this study mainly uses linear

finite element features, as the extraction of non-linear

features requires long-term RRI measurement to keep

the output features stable, which can’t be used in real

time, for example, fatigue driving early warning.

Linear HRV features are divided into time domain

features and frequency domain features.

Time-domain features include:

 MeanNN: the average value of RRI.

 SDNN: standard deviation of RRI.

 RMSSD: root mean square error of adjacent

RRI.

 TP: total power change of RRI.

 pNN50: the number of sample pairs where

the difference between adjacent RRIs is

greater than 50ms in a given measurement

time.

Frequency-domain features include as follows.

The first one is Low Frequency (LF), which is the

power in the low frequency band of the PSD (0.04

Hz-0.15 Hz). LF mainly reflects the regulation of

sympathetic nerves, with the main function of

sympathetic nerves to strengthen the heartbeat and

muscle work ability. The sympathetic nerve has an

inhibitory effect on the smooth muscle of the

bronchioles, making the bronchi dilate, which is

conducive to lung breathing. Sympathetic nerve

activity increases when the body is under tension and

requires intense ventilation.

The second one is High Frequency (HF), which is

the power in the high frequency band of the PSD

(0.15 Hz–0.4 Hz). HF mainly reflects the adjustment

of the parasympathetic nerve to the body. The

function of the parasympathetic nerve is opposite to

the sympathetic nerve. The two jointly regulate the

body's heart rate, respiration, glandular secretion, and

the blood flow distribution of important organs, such

as the liver and adrenal glands, which can slow the

body's heartbeat, lower blood pressure and shrink the

bronchi, so as to reduce unnecessary energy

consumption and reflect the activities of the

parasympathetic nervous system.

The third one is LF/HF, that is, LF to HF ratio,

which shows the balance between the activities of the

sympathetic nervous system and the parasympathetic

nervous system. When the body holds still, the

activity of the parasympathetic nerve increases.

However, it may cause the body to fatigue after a long

time. LF/HF must change continuously within a

certain range, so as to maintain human health.

2.2 Mathematical Characteristics of

Heart Rate Variability

After the ECG signal is filtered, the mathematical

features of each linear feature are obtained based on

linear HRV feature extraction. There are 12 time-

domain linear features, including mean, median,

standard deviation (SD), variance, maximum,

minimum, skewness, kurtosis, power, root mean

square (RMS), approximate entropy and Hurst

exponent.

3 DATA SET OF DRIVER HEART

RATE

3.1 Date Set

This research uses MIT-BIH’s Stress Recognition in

Automobile Drivers data set. Each sample in the data

set uses electrocardiogram (ECG), electromyography

(EMG), skin conductance (EDA) and respiration rate.

The sampling frequency of ECG is 496 Hz, the

sampling frequency of skin conductance and

respiration is 31 Hz and the sampling frequency of

EMG is 15.5 Hz, with a total of 17 driving tests. The

labeled information comes from a questionnaire of all

drivers, including the perception of low, medium, and

CAIH 2021 - Conference on Artiﬁcial Intelligence and Healthcare

high stress during rest, highway and city driving, with

two scoring methods, free scoring and mandatory

ranking. Drivers score the driving event amid the free

scoring method, with the scoring standard from "1" to

"5", where "1" represents the feeling of "no pressure"

while "5" represents the feeling of "high pressure".

Mandatory ranking requires drivers to rank events on

a scale from 1 to 7, where "1" is assigned to the least

stressful driving event, while "7" is assigned to the

most stressful driving event. Drivers are asked to rate

events by using this scale, including encounters with

toll booths, mergers and exits and other city, and

highway driving tasks. The values of the two stress

levels in each questionnaire are standardized, and

then the average and standard deviation are calculated

and inversely transformed. The test analysis of all

categories shows that the overall score and the

comparison score are significantly different

(p>0.001), which supports the rationality of the data

set.

3.2 Data Preprocessing

The original ECG data collected from the driver

includes noise caused by various reasons. First, it is

needed to use a filter with a cutoff frequency of 3-100

Hz to eliminate noise. Then the heartbeat is detected

by QRS complex scanner using Pan and Tompkins

algorithm (Karegar 2017). The HRV signal is

obtained by accurately measuring the R peak value

from the ECG signal based on the wavelet transform

technique (Zhao 2012).

3.3 Feature Extraction

The preprocessed ECG signal is first subjected to

HRV linear feature extraction to generate 8 HRV

features, including MeanNN, SDNN, RMSSD, TP,

NN50, LF, HF and LF/HF. The time-domain features

are extracted by algorithms to generate mathematical

features, with a total of 5*10+3=53 types of features.

The significant difference values of the mathematical

characteristics of the sample categories are shown in

Table 1.

Table 1: Significant differences of varies characteristics.

Time-frequency

characteristics

Time-frequency

characteristics

Mean Skewness

Median Kurtosis

Standard variance power

variance Root Mean Square (RMS)

Hurst index Approximate entropy

3.4 Classifier

SVM, KNN and ensemble classifiers are used to

classify features in the experiment. 75% of the data is

used for training the classifier, while 25% of the data

is used for testing. Support vector machine is a classic

binary classification algorithm, which seeks the

optimal linear decision surface between classes by

minimizing structural risks (Zhang 2015). KNN is a

super machine learning algorithm that uses

autoregressive features and each form to classify

various low alert states, which is better than quadratic

discriminant analysis (QDA) and linear discriminant

analysis (LDA) (Bhuvaneswari 2015).

3.5 Evaluation Method

We can calculate true positive (TP), false negative

(FN), true negative (TN) and false positive (FP), so

we can calculate performances of accuracy (AC),

sensitivity (SE) and specificity (SP). For example:

(%) 100

TP TN

TP TN FP FN

=×

+++

(1)

(%) 100

TP FN

=×

(2)

(%) 100

TN FP

=×

(3)

In addition, because our test data is biased (the

pressure-free interval is more than the pressure

interval), it is important to have two parameters,

namely, the balance accuracy (BA) and the geometric

mean (GM), such as

(%) ( ) 100

TP TN

TP FN TN FP

=+×

(4)

(%) 100

TP TN

TP FN TN FP

=+×

(5)

4 THE RECOGNITION OF

DRIVER'S STRESS STATE

This research carried out the following experiments.

the first one is to extract 5 kinds of heart rate

variability features, and then calculate 20 kinds of

time-frequency domain features, and then input the

heart rate variability features as training data into the

classifier for training and evaluation. The second one

is to extract 5 kinds of heart rate variability features,

and then calculate 20 kinds of time-frequency domain

features, and then input each time-frequency domain

feature as training data into the classifier for training

and evaluation. The difference between the two

experiments is to focus on the characteristics of heart

Drivers Pressures States Recognition based on Heart Rate Variability

rate variability and time-frequency domain

characteristics. And the contribution of various

features to the performance of the classifier is tested

through the experiments.

Experiment 1 uses various HRV features

extracted by the heart rate variability analysis

method, carries out training modeling and obtains

experimental results. It can be seen from Table 2 that

SDNN and RMSSD can get better recognition results,

while RRI and pNN50 are slightly less effective, and

features got by TP are the worst. This reflects that the

sensitivity of these features has a high degree of

discrimination for identifying the driver's stress state.

Table 2: Driver status recognition results based on the characteristics of heart rate variability.

HRV Classifier

Accuracy

AC (%)

Sensitivity

SE (%)

Specificity

SP (%)

Balance

accuracy

(

)

Geometric

mean

(

)

MeanNN

SVM 69.0 77.2 32.6 54.9 104.8

KNN 75.0 81.9 41.2 61.6 111.0

SDNN

SVM 78.0 84.1 47.6 65.9 114.8

KNN 77.0 83.1 49.5 66.3 115.1

RMSSD

SVM 72.0 78.7 47.2 62.9 112.2

KNN 79.5 84.8 57.3 71.0 119.2

SVM 69.0 75.4 49.2 62.3 111.6

KNN 69.0 75.1 51.2 63.1 112.4

pNN50

SVM 88.0 91.1 74.5 82.8 128.7

KNN 89.0 91.8 77.3 84.6 130.1

SVM 74.3 79.1 60.9 70.0 118.3

KNN

72.7 77.3 60.9 69.1 117.5

SVM

71.5 75.8 61.2 68.5 117.0

KNN

75.6 79.4 66.1 72.8 120.6

LF/HF

SVM 89.0 91.3 82.0 86.6 131.6

KNN 92.0 93.7 86.8 90.2 134.3

Experiment 2 uses various HRV features extracted

by the time-frequency analysis method, carries out

training modeling and obtains experimental results. It

can be seen from Table 3 that the model recognition

accuracy of features such as mean, variance, mean

square deviation, and maximum value is higher,

which reflects that these features contain more

discernable

information about the driver's state.

However, the Hurst index, skewness, kurtosis, Q1

and other parameters have little effect on the

performance of the recognizer, with the recognition

rate at the range of 50%±5.

Table 2: Driver status recognition results based on time-frequency characteristics

Mathematical

characteristics

Classifier

Accuracy

AC (%)

Sensitivity

SE (%)

Specificity

SP (%)

Balance

accuracy

BA (%)

Geometric

mean

GM (%)

features

SVM 98.6 99.1 91.5 95.3 138.0

KNN 98.2 98.8 90.7 94.8 137.7

mean

SVM 96.0 97.4 78.9 88.2 132.8

KNN 92.0 94.7 68.6 81.6 127.8

median

SVM 92.0 94.6 71.4 83.0 128.8

KNN 95.0 96.6 81.8 89.2 133.6

standard

deviation (SD)

SVM 94.0 95.9 80.6 88.3 132.9

KNN 95.0 96.6 84.6 90.6 134.6

variance

SVM 91.0 93.6 76.9 85.3 130.6

KNN 92.0 94.3 80.2 87.3 132.1

Hurst index

SVM 85.0 88.8 70.0 79.4 126.0

KNN 86.5 89.8 73.5 81.7 127.8

CAIH 2021 - Conference on Artiﬁcial Intelligence and Healthcare

Skewness

SVM 89.0 91.7 78.4 85.1 130.4

KNN 88.9 91.5 79.3 85.4 130.7

Kurtosis

SVM 87.1 89.9 77.7 83.8 129.5

KNN 89.2 91.5 81.5 86.5 131.5

power

SVM 68.0 71.4 61.0 66.2 115.1

KNN 70.5 73.7 64.0 68.8 117.3

Root mean

square(RMS)

SVM 93.5 94.8 89.4 92.1 135.7

KNN 94.4 95.5 91.1 93.3 136.6

Approximate

entropy

SVM 84.3 86.5 79.3 82.9 128.7

KNN 85.4 87.3 81.1 84.2 129.8

5 CONCLUSION

This study uses the ECG physiological signal data set

to study the method of identifying the driver's stress.

The results show that the features perform better in

detecting the three categories of low pressure,

medium pressure and high pressure, according to the

results of the classifier, with classification accuracy

rates at 93.1%, 96.6%, and 96.6%, respectively. With

the improvement of ECG performance, other

physiological signals can also be combined to

improve the detection accuracy of low vigilance. In

the future, vehicle and behavior-based methods can

be combined with physiological methods to develop

reliable detection methods of driver state.

REFERENCES

A. Hashemi et al., Time driver’s drowsiness detection by

processing the eeg signals stimulated with external

flflickering light, Basic Clin. Neurosci., vol. 5, no. 1,

pp. 22–27, 2014.

B. Cyganek and S. Gruszczynski, Hybrid computer vision

system for drivers’ eye recognition and fatigue

monitoring, Neurocomputing, vol. 126, no. 27, pp. 78–

94, 2014.

B. G. Lee et al., Real-time physiological and vision

monitoring of vehicle driver for non-intrusive

drowsiness detection, IET Commun., vol. 5, no. 17, pp.

2461–2469, 2011.

Bhuvaneswari P, Kumar JS (2015) Influence of linear

features in nonlinear electroencephalography (EEG)

signals. Procedia Pro- cedia Comput Sci 47:229–236

C.-T. Lin et al., A real-time wireless brain–computer

interface system for drowsiness detection, IEEE Trans.

Biomed. Circuits Syst., vol. 4, no. 4, pp. 214–222, Aug.

2010.

F.-C. Lin et al., Generalized EEG-based drowsiness

prediction system by using a self-organizing neural

fuzzy system, IEEE Trans. Circuits Syst. I, Reg. Papers,

vol. 59, no. 9, pp. 2044–2055, Sep. 2012.

Gibson P M, Hennessy D, Wiesenthal D L. The Driving

Vengeance Questionnaire (DVQ): The Development of

a Scale to Measure Deviant Drivers' Attitudes [J].

Violence & Victims, 2000, 15(2)

H. Su and G. Zheng, A partial least squares regression-

based fusion model for predicting the trend in

drowsiness, IEEE Trans. Syst., Man, Cybern. A, Syst.

Humans, vol. 38, no. 5, pp. 1085–1092, 2008.

Healey, J.A., Picard, R.W., 2005. Detecting stress during

real-world driving tasks using physiological sensors

Intelligent Transportation Systems. IEEE Trans. 6 (2),

156–166

J. Jo et al., Detecting driver drowsiness using feature-level

fusion and user-specific classification, Expert Syst.

Appl., vol. 41, no. 4, pp. 1139–1152, 2014.

K. T. Chui et al., An accurate ECG-based transportation

safety drowsiness detection scheme, IEEE Trans. Ind.

Inform., vol. 12, no. 4, pp. 1438–1452, Aug. 2016.

Karegar FP, Fallah A, Rashidi S (2017) ECG based human

authen- tication with using generalized hurst exponent.

Iran Conf Electr Eng 17:34–38

M. Singh, A. B. Queyam, A novel method of stress

detection using physiological measurements of

automobile drivers, India, 2013.

Pan Xiaodong. Application of human body information

technology in road traffic environment and safety

evaluation [J]. China Journal of Highway and

Transport, 2001, 014(0z1):109-111,115.

R. O. Mbouna et al., Visual analysis of eye state and head

pose for driver alertness monitoring, IEEE Trans. Intell.

Transp. Syst., vol. 14, no. 3, pp. 1462–1469, Sep. 2013.

Zhang L, Liu FAN, Tang J (2015) Real-time system for

driver fatigue detection by RGB-D camera. ACM Trans

Intell Syst Tech- nol 6(2):1–17

Zhao C, Zhao M, Liu J, Zheng C (2012)

Electroencephalogram and electrocardiograph

assessment of mental fatigue in a driving simulator.

Accid Anal Prev 45:83–90

Drivers Pressures States Recognition based on Heart Rate Variability