Contactless Heart Rate Measurements using RGB-camera and Radar

Manola Ricciuti

, Gianluca Ciattaglia, Adelmo De Santis

, Ennio Gambi

and Linda Senigagliesi

Dipartimento Di Ingegneria dell’Informazione, Universit

a Politecnica delle Marche,

Via Brecce Bianche 12, Ancona 60131, Italy

Keywords:

Contactless Sensor, Heart Rate, mmWave Radar.

Abstract:

The detection of vital parameters with traditional approaches, as the electrocardiograph, requires to appro-

priately place electrodes in direct contact with patients’ skin, often causing irritation. On the other hand,

contactless measurement of physiological parameters provides an unobtrusive and comfortable instrument for

subjects’ conditions monitoring, with application to home monitoring of aging people and in particular to

those suffering of heart disease. In this paper two contactless techniques are proposed, based on radar technol-

ogy and on video processing from an RGB camera. In order to validate their precision, the proposed methods

are compared with three wearable low cost devices, taken as a reference for the outcomes. The developed ap-

proaches prove to achieve excellent performances, with an estimated mean relative error of 0.55% with respect

to a commercial cardiac strap device.

1 INTRODUCTION

The Heart Rate (HR) measurement represents an im-

portant indicator of people’s health condition. In the

last years, research about contactless HR estimation

methods gained an increasing attention, especially

since they can represent a non-invasive alternative to

conventional devices to detect different pathologies

related to cardiac problems before getting a clinical

diagnosis. Clinical devices in fact are often expensive

and usually require the physical presence of patients

in the hospital environment, which may be a difﬁculty

especially for the elder people. Furthermore, clinical

examination are equipped with electrodes that may re-

act with the patient’s skin and in some case cause sig-

niﬁcant damages. In addition other available contact

sensors require exact positioning, while contactless

methods allow to obtain a precise measurement also

for patients that are not completely autonomous. Ag-

ing population represents the category of people who

can take the most advantage of the use of contactless

HR monitoring devices directly in their homes, where

a complete electrocardiogram (ECG) recording may

not be easy to perform. A continuous HR contactless

https://orcid.org/0000-0003-4870-0914

https://orcid.org/0000-0002-9084-536X

https://orcid.org/0000-0001-6852-8483

https://orcid.org/0000-0002-8798-4588

monitoring can ﬁnd an application in several contexts,

for example in driving conditions (Lee et al., 2018), or

to indicate a mental stress state (Kumar et al., 2007).

The design of contactless HR detection algorithms

with a high accuracy is still an open problem. As

a benchmark, the reliability of their performance

is usually compared with those derived from stan-

dard methods, such as electrocardiography (ECG), or

from other provably reliable non-invasive technolo-

gies, such as wearable devices (Georgiou et al., 2018).

Several methodologies for contactless HR have

been proposed over the years. Consolidated sys-

tems exploit the use of the Microwave Impulse Radar

(MIR) (Michahelles et al., 2004) or the Ultra Wide-

band (UWB) radar (Paulson et al., 2005; Zetik et al.,

2007), which are non invasive and non contact meth-

ods based on very short radar impulses. These tech-

niques use a wider bandwidth than conventional radar

systems, enabling a positive impact on the informa-

tion content of the received signal, proportional to the

bandwidth of the low power transmitted signal. An

alternative solution is given by the use of automotive

radars. These sensors work in the mmWave range,

with very high bandwidth and low cost. Also thanks

to their operating frequencies, automotive Radars

have a very small dimension, thus allowing to eas-

ily use them for healthcare applications (Wang et al.,

2015; Hsu and Tseng, 2016).

Ricciuti, M., Ciattaglia, G., De Santis, A., Gambi, E. and Senigagliesi, L.

Contactless Heart Rate Measurements using RGB-camera and Radar.

DOI: 10.5220/0009793201210129

In Proceedings of the 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE 2020), pages 121-129

ISBN: 978-989-758-420-6

121

Another well-known short-range contactless HR

estimation approach is represented by image and

video processing (Li et al., 2014; Rouast et al., 2018;

Sanyal and Nundy, 2018). It is mainly based on the

detection of changes in the skin color due to the blood

ﬂow, which is usually imperceptible to the human eye.

Since people’s faces are an uncovered area, the face

acquisition with an RGB camera is suitable for HR

extraction by video processing. The Eulerian Video

Magniﬁcation (EVM) method allows to enhance the

skin color due to the blood ﬂow, and it is a widely

used approach in several studies (Alghoul et al., 2017;

Gambi et al., 2017; Bennett et al., 2017; Aubakir

et al., 2016).

Starting from the above considerations, this paper

develops and deepens the work shown in (Ciattaglia

et al., 2019), by the proposition of two methods to

estimate HR through different contactless technolo-

gies. In particular, we consider signals obtained from

an automotive radar and from video, which are elabo-

rated to detect the HR of subjects with different char-

acteristics. For the implementation of the proposed

methodologies we exploit low cost commercial de-

vices originally designed for other purposes, show-

ing that we are able to achieve good precision perfor-

mances. In order to estimate their accuracy, we com-

pare the results obtained with those measured with

more standard and provably precise contactless tech-

nologies, such as a pulse oximeter, a Polar wearable

sensor and a Garmin smartwatch, which can be easily

self-applied in a domestic context. Besides the mea-

surement of the average value of the HR in a ﬁxed

interval, we also prove that by exploiting the consid-

ered contactless methodologies it is possible to assess

the time variation of the HR, achieving a small and in

some case negligible error with respect to more con-

ventional technologies.

2 RELATED WORK

The use of radar systems in contexts other than the

original one, such as the ambient assisted living, has

seen an important technological advancement in the

last years. Recently, these systems have attracted the

attention of the medical ﬁeld, thanks to their ability

to provide high precision measurements at a reduced

cost. In particular, a special interest has raised about

the remote monitoring of vital parameters (Pisa et al.,

2016). In (Wang et al., 2013; Lin et al., 1979) vital

signs are extracted by using a Continuous Wave (CW)

radar, while the same analysis is performed through

an impulsive Ultra wideband (UWB) radar in (Ren

et al., 2016) and a Frequency-Modulated Continuous-

Wave (FMCW) radar in (Wang et al., 2015; Mu

noz-

Ferreras et al., 2019). The operational frequency of

this kind of radar is in the range 24-80 GHz (De Ponte

uller, 2017); the use of such high frequencies allows

measurements with good resolution, thus making it

possible to analyze variations in very small displace-

ments, such as those produced by the heartbeat.

Examples of remote monitoring of the heart pulse

through the Photoplethysmography (PPG) are given

in (Li et al., 2014; Rouast et al., 2018; Verkruysse

et al., 2008). The PPG signal can be extracted through

a pulse oximeter by exploiting a lightweight optical

device (Tremper, 1989) for measuring oxygen satura-

tion and heart beats in clinical use. Recently has been

proved that PPG signal can be measured using a dig-

ital camera and the ambient light as an illumination

source. When the digital camera is used to acquire

a video, Videoplethysmography (VPG) (Rumi

nski,

2016; Couderc et al., 2015; Gambi et al., 2017) is

deﬁned as the signal derived by the RGB frame se-

quences processing.

In the paper here presented, we describe the VPG

signal extracted from the subjects’ face, using a face

detection process (Viola and Jones, 2004), through

the luminance component obtained by transforming

the RGB in YIQ (Y-luminance, IQ chrominance)

space color. The Regions of Interest Region of Inter-

ests (ROIs) are selected manually and they represent

a percentage of a detected face; these ROIs are ap-

plied for every subjects under tests in the same way.

Deep learning is exploited in (Hsu et al., 2017; Sebe

et al., 2019) to track the better facial ROI by using a

set of landmarks, but, differently from this work, the

authors use an approach based on Independent Com-

ponent Analysis (ICA) (Alghoul et al., 2017) or by

extracting the sample mean sequences of the R, G,

and B channels. On the contrary, in this study we

adopt the EVM approach to highlight the blood ﬂow

after the heart pumping. The EVM method allows to

amplify the color values assumed by the pixels in a

speciﬁc frequency band. This means that our algo-

rithm is able to recognize a periodic variation in a se-

quence, and ampliﬁes that variations in the frequency

range selected in the input.

The rest of the paper is organized as follows. In

Sec. 3 radar setup and signal processing are de-

scribed, also considering a Multiple-Input Multiple-

Output (MIMO) conﬁguration, while in Sec. 4 the

video signal processing developed is presented. Sec.

5 contains the results obtained and the comparison be-

tween the considered measurement techniques. Final

considerations and remarks are provided in Sec. 6.

ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health

122

3 HEART RATE ESTIMATION

FROM RADAR

The use of radar systems to determine the physiolog-

ical parameters of a subject is implemented with the

application of a technology that comes from the au-

tomotive world. Automotive radars are in fact capa-

ble of achieving extremely high precision at reduced

costs. In our tests we exploit the AWR1843 radar

provided by Texas Instruments (Instruments, 2019), a

MIMO radar with a fully integrated system containing

a RF section, a DSP processor, an analog-to-digital

converter (ADC) and a micro-controller; 4 receivers

and 3 transmitters are available. A scheme of the de-

vice is reported in Fig. 1.

DSP

and

MCU

ADCs

Signal

Generator

BUS LVDS

Receivers

Transmitters

Figure 1: AWR1843 scheme.

A high precision in the deﬁnition of the target position

is needed for the subject’s HR detection, since, as de-

scribed in the following, the heart activity is studied

through the analysis of the phase of the signal Fast

Fourier Transform (FFT). Accuracy can be improved

by exploiting MIMO. Moreover, as discussed in (Lee

et al., 2019), MIMO turns out to be extremely effec-

tive for the simultaneous monitoring of the HR of two

subjects, thanks of the possibility of identifying the

angular dimension of the subject.

A description of the MIMO technology applied to

radar systems is contained in (Fishler et al., 2004).

Differently from classic communication systems ap-

plication, the main goal in a radar context is to in-

crease the angular resolution without adding com-

plexity to the system. As stated in (Jian and Stoica,

2009), the result is a virtual receiving antenna, whose

characteristics depend on the position of the transmit-

ter and the receivers.

3.1 Radar Signal Processing

As described in (Ciattaglia et al., 2019), the HR ex-

traction procedure from radar works as follows. The

radar transmits a sequence of chirps divided into

frames. Each chirp is designed to be able to pre-

cisely detect the position of a subject. It is possible

to use only one frame for each chirp, thus not cre-

ating a data overhead during the processing phase.

It is important to carefully set the duration of each

chirp and the number of the contained frames within

it, since these values directly impact on the sample

rate of the HR. From (Instruments, 2019) the sam-

pling frequency along the slow time can be written as

sampling[HR]

periodicity

, (1)

where k is the number of chirps inside one frame and

periodicity

represents the frame duration.

For a better target position identiﬁcation the

MIMO technology is applied. Using this technique

is possible to identify not only the distance but also

the angle position, this is an improvement of the algo-

rithm accuracy (Ding et al., 2016; Xiong et al., 2018).

A Field Programmable Gate Array (FPGA) is

used to collect the samples of the beat signals from

the ADCs. These signals come from four receivers’

lines, and it is possible to store the samples in a data

cube. This cube is depicted in Fig. 2. Samples of

a single chirps are stored along the fast time, sam-

ples of different chirps are stored along the slow time,

while samples of different receivers are stored along

the spatial sampling. The fast time is used to detect

the range distance of the subject, the spatial sampling

and the slow time detect the angle and the velocity,

respectively.

Fast Time

Spatial Sampling

Slow Time

Figure 2: MIMO Data Cube.

By performing a bi-dimensional FFT on the Fast Time

- Spatial Sampling plane, we obtain information about

the subject position for each transmitted chirp. In our

tests the target is static and there is no great variation

in the FFT of the different chirps. In Fig. 3 we show

an example of this FFT, where the red square repre-

sents the subject.

Once evaluated the target position, it is possible to ap-

ply the algorithm for the extraction of the HR. The

displacement variation due to the heart contraction

is far below the radar range resolution, thus making

the phase analysis the only way of extracting the HR.

The HR in fact produces a phase modulation of the

FFT signal corresponding to the target position, as de-

scribed in (Ding et al., 2016). After this identiﬁcation

is possible to extract the signal of interest along the

slow time. The information is inside the phase, for

Contactless Heart Rate Measurements using RGB-camera and Radar

123

Figure 3: Subject position identiﬁcation.

Table 1: Filters parameters.

Parameter Value

Filter Type FIR Equiripple

stop1

0.8 Hz

pass1

1.1 Hz

stop2

2.5 Hz

pass2

2.8 Hz

this reason a phase extraction and unwrap operation

are needed. After having obtained the phase signal

we can ﬁlter it in the range of the heartbeat, i.e. be-

tween 60 beats/min and 150 beats/min. The values

used in our setting are reported in Tab. 1.

4 HEART RATE ESTIMATION

FROM VIDEO

This section describes the techniques and algorithms

applied for HR extraction through the video process-

ing of a subject’s face, by exploiting the face detection

and the slight variations in skin color, generated by

the blood ﬂow in the tissues, ampliﬁed with the EVM

method. The entire algorithm is brieﬂy presented in

Fig. 4.

The video is captured with GoPro Hero 6 with a

frame rate of 60 fps (2400 frames for a sequence of

40 seconds) and resolution 1920 × 1080 pixels. Each

frame composing the video is converted from RGB

to YIQ space color to process only the luminance sig-

nal (Y), which constitutes the Videoplethysmographic

(VPG) signal.

The face detection algorithm is derived by Vi-

ola and Jones approach. A face detector of at most

300 × 300 pixel is applied in order to extract the face

of the subjects under test. The variations in skin color,

Figure 4: Main scheme of the video signal processing algo-

rithm.

Figure 5: ROI selection on a subject’s face. F stand for

Forehead, C for Cheek and N for Neck.

imperceptible to human eye, reveal the subject HR

thanks to the support of technological innovation. In

particular, in each frame of the acquired video, we

applied the EVM method (Wu et al., 2012), which al-

lows to amplify skin color with the blood ﬂow follow-

ing the pumping of the heart, by deploying advanced

image processing techniques. The algorithm analy-

ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health

124

ses the sequences of frames by performing a spatial

decomposition and a time ﬁltering to determine the

skin color changes in a speciﬁc frequency range. In

the spatial decomposition, realized with four levels of

a Gaussian pyramid in which the higher level is the

smallest one, each image is down-sampled so that,

starting with a level of 300 × 300 px, the higher level

has 18x18 pixel size. The time ﬁltering is developed

on each spatial band with Inﬁnite Impulse Response

(IIR) band-pass ﬁlters to extract the frequency bands

of interest.

As regards the emphasis on the skin colours varia-

tion, the main principle underlying the EVM consists

in the ampliﬁcation of the color values assumed by

the pixels in a selected area or ROIs, realized by a

speciﬁc parameter, i.e. the magniﬁcation factor or α-

factor, set to 50. The selected ROIs are constituted by

the forehead, cheeks and neck. The mean luminances

extracted in each ROI must be added up to realize a

total ROI and to process the VPG signal. As shown

in Fig. 5 ROIs are chosen by setting the percentages

of the box identiﬁed by the face detector and the ROIs

size are respectively 90 ×30, 30 ×30 and 30× 26 pix-

els. The VPG signal corresponds to the average val-

ues of the Y-component of all the pixels values inside

the selected ROIs. We are able to extract the subject

HR by setting the plausible cut-off frequencies for a

healthy subject, which in this work correspond to (50

bpm/60) Hz for the lower and to (174 bpm/60) Hz for

the higher, and applying ﬁrst a third order Butterworth

band-pass ﬁlter and then a FFT.

5 EXPERIMENTAL RESULTS

Experimental tests were conducted at the ICT Labo-

ratory of the Polytechnic University of Marche on a

set of 15 Caucasian and Asian people of different age

and weight. Radar and GoPro Hero 6 were placed

on two tripods, radar at a distance of about 20 cm

from the subject’s chest and GoPro at about 50 cm

from the subject’s face. The subjects under test were

in a standing position. The used measurement setup

is depicted in Fig. 6. During the video acquisition,

the subjects were asked minimize their movements in

order to avoid noisy signals or face tracking errors.

Videos are captured in indoor conditions, hence we

used a standard lamp (see Fig. 6) in addition to the

ceiling light to better illuminate the subjects’ faces.

Radar conﬁguration parameters are reported in

Tab. 2; it is possible to note that the heartbeat of the

subject is sampled with a frequency of 20 Hz, which

is a much larger value than Nyquist’s limit.

In Tab. 3 we report the characteristics of the sub-

Figure 6: Measurement setup.

Table 2: RADAR parameters.

Parameter Value

chirp

15.7 ms

Bandwidth 3.99 GHz

periodicity

50 ms

No. chirps in frame 1

No. samples in chirp 128

sampling

4 MSps

max

4.2184 m

samplingHR

20 Hz

Used TX and RX TX1/RX1-RX4

jects participating to the tests and in Tab. 4 the results

of the corresponding average heartbeat obtained with

both the considered methods, radar (indicated as R)

and video (V) processing, with respect to values given

by Pulse Oximeter (POx). Since the EVM method

ampliﬁes the skin color, the subjects’ characteristics

are described to highlight any possible inﬂuence on

the HR estimation. People with beard or make-up

may generate a greater error in the VPG processing.

The average heartbeat has been evaluated by consid-

ering the peak value obtained after applying a FFT to

the entire signal. The peak value in frequency is then

multiplied by 60 to convert it in bpm.

In Figs. 7 and 8 we show an example of the am-

plitude (or luminance) evolution of the VPG signal as

a function of time, where 40 seconds of acquisition

are considered, and frequency for subject 14 in Tab.

4. Fig. 8 underlines the presence of a peak, which

corresponds to the maximum value of the HR, that is

equal to 92 bpm.

The relative percent errors Er% between different

methods shown in Tab. 4 are calculated as follows

Er% =

|Re f − A|

Re f

· 100, (2)

where Re f is the reference value and A is the value

Contactless Heart Rate Measurements using RGB-camera and Radar

125

Table 3: Characteristics of the subjects under test.

S Age Weight Characteristics

1 22 66 Caucasian

2 22 85 Caucasian, Beard

3 37 75 Caucasian, Make-up

4 28 58 Caucasian, Beard

5 27 54 Caucasian, Make-up

6 26 54 Caucasian

7 37 75 Caucasian, Tanned skin

8 25 74 Caucasian, Beard

9 24 75 Asian

10 25 64 Caucasian

11 45 70 Caucasian

12 31 70 Caucasian, Beard

13 29 63 Caucasian

14 29 63 Caucasian

15 36 58 Caucasian, Make-up

16 29 50 Caucasian

Table 4: Tests Results.

S V POx R Er

|P−R|

% Er

|P−V|

1 77 76 77 1.05 1.32

2 99 102 98.4 3.53 2.94

3 96 93 91 1.94 3.23

4 65 67 70 3.88 2.99

5 61 64 64 0.62 4.69

6 72 73 73 0.27 1.37

7 90 93 92.4 0.65 3.23

8 77 72 71 1.67 6.94

9 74 73 72 1.37 1.37

10 79 81 83 2.22 2.47

11 79 78 79 1.54 1.28

12 103 103 104 0.97 0.00

13 72 72 67 6.67 0.00

14 92 93 82 12.26 1.08

15 81 83 80 3.13 2.41

16 70 71 72 1.41 1.41

estimated using the contactless approach. The Mean

Relative Errors (MREs) (Srivastava, 2014), computed

by averaging the relative percentage errors Er%, be-

tween Radar and Pulse Oximeter and VPG and Pulse

Oximeter, are reported in Tab. 5, showing that errors

obtained are very similar. The small errors measured

also prove that both contactless methods can be used

as a valid alternative to the Pulse Oximeter.

Table 5: Results and characteristics of the subjects under

test.

MRE POx-R MRE POx-V

2.76% 2.35%

Figure 7: VPG signal extracted in terms of amplitude as a

function of time.

Figure 8: VPG signal extracted in terms of amplitude as a

function of frequency. The peak at 1.526 Hz corresponds to

92 bpm.

A second kind of analysis is then carried out by

analysing the time evolution of the VPG and Radar

signals. As a reference, we considered the results

obtained from two additional devices, a Polar H7

Bluetooth smart heart rate sensor, in association with

the smartphone application Heart Rate Monitors, and

a Garmin Vivoactive 3 smartwatch. They are both

cheap but highly reliable HR monitoring devices.

The variability of HR is captured in the entire tem-

poral sequence according the approach described in

the following. Acquisitions of 40s were taken by si-

multaneously placing the Radar and the GoPro, wear-

ing the Polar using a chest strap and the Garmin

smartwatch. We investigated the FFT in the frequency

domain by segmenting the VPG and Radar signals in

the time domain in portions of 10 seconds, with 1

second of overlapping windows. The result is then

interpolated and ﬁltered using a Savitzky-Golayand

(Theodor, 1996) ﬁlter, a Finite Impulse Response

(FIR) optimal ﬁlter that minimizes the least-squares

error in a polynomial ﬁtting applied to noisy data. Fi-

nally, the outcome is compared to those extracted us-

ing the wearable devices.

The beats values resulted from the time domain

analysis are shown in Fig. 9. All the examined curves

lead to similar trends in their second part (which cor-

responds to 20-40 seconds of acquisition), although

the radar exhibits a maximum value slightly shifted

ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health

126

with respect to the other methods. However, we also

observe that radar trend signiﬁcantly differs from the

others in the ﬁrst 20 seconds of observation, and this

is probably due to the fact that the algorithm used for

radar signal processing is sensitive to the inevitable

micro oscillations of subjects in standing position. On

the other hand, the error measured on the radar is lim-

ited with respect to the Polar, which represents our

main and most precise reference, as shown in Tab. 6.

Figure 9: Comparison between contactless measurements

and wearable devices.

Tab. 6 also reports MREs calculated comparing dif-

ferent methods, where P stands for Polar sensor and G

for Garmin smartwatch. The minimum error achieved

between Polar and VPG signal proves that the video

contactless HR estimation represents a very precise

approach. Radar shows a greater error with respect to

other methods, but even in the worst case it remains

below the threshold of 3%, thus demonstrating good

overall performances.

Table 6: MRE between Polar (P), Garmin (G), VPG (V)

and Radar (R) estimated values. The ﬁrst letter indicates

the reference device.

Methods compared MRE value

P-G 1.74%

P-V 0.55%

G-V 1.93%

P-R 2.09%

G-R 2.92%

V-R 1.93%

5.1 Discussion

The proposed contactless approaches for HR mea-

surement have proved to be valid and effective meth-

ods, with the advantage of an ease-of-installation in

a home environment, thus allowing the monitoring

of the elderly people in a simple way to reproduce.

The cheapness of the commercial devices, such as the

Go Pro RGB camera or the radar used in this work,

makes also easy to reproduce the setup in different

rooms of the house, in order to extend the HR mon-

itoring to a domestic environment. The main error

sources of the considered measurements are proba-

bly represented by the noise caused by the subjects’

movements during the HR detection and by the dis-

tances between people and contactless devices. These

distances can be further reduced, especially in a do-

mestic scenario. In addition, the radar captures other

signals which overlap to the HR, being focused on an

area not sufﬁciently small. However, this problem can

be avoided by applying a proper ﬁltering.

6 CONCLUSIONS AND FUTURE

WORK

Different contactless methodologies to estimate heart

rate have been proposed. The application of these

methodologies has relevance especially in contexts

where the user is not able to properly handle the

measurement devices, as the non-contact measure-

ment provides for the user a simple positioning, even

seated. We exploited low cost devices, in particular

an automotive radar and a commercial camera, to ex-

tract physiological parameters from a set of 15 sub-

jects with various characteristics. In order to validate

our results, we have compared them with provably re-

liable technologies and we have shown that it is pos-

sible to assess the time variation of the heart rate with

a small and in some case negligible error. The errors

of 0.55% obtained with VPG approach and 2.09% us-

ing radar, compared with the Polar H7, taken as ref-

erence device, have proven the system precision and

reliability of the developed methods. Further analy-

ses could involve the evaluation of the proposed ap-

proaches by using an extended data-set comprising of

different characteristics of the subjects, including eth-

nicity and lifestyle.

REFERENCES

Alghoul, K., Alharthi, S., Al Osman, H., and El Sad-

dik, A. (2017). Heart rate variability extraction from

videos signals: Ica vs. evm comparison. IEEE Access,

5:4711–4719.

Aubakir, B., Nurimbetov, B., Tursynbek, I., and Varol,

H. A. (2016). Vital sign monitoring utilizing eule-

rian video magniﬁcation and thermography. In 2016

38th Annual International Conference of the IEEE En-

gineering in Medicine and Biology Society (EMBC),

pages 3527–3530. IEEE.

Contactless Heart Rate Measurements using RGB-camera and Radar

127

Bennett, S., El Harake, T. N., Goubran, R., and Knoefel, F.

(2017). Adaptive eulerian video processing of thermal

video: An experimental analysis. IEEE Transactions

on Instrumentation and Measurement, 66(10):2516–

2524.

Ciattaglia, G., Senigagliesi, L., De Santis, A., and Ricciuti,

M. (2019). Contactless measurement of physiological

parameters. In 2019 IEEE 9th International Confer-

ence on Consumer Electronics (ICCE-Berlin), pages

22–26.

Couderc, J.-P., Kyal, S., Mestha, L. K., Xu, B., Peterson,

D. R., Xia, X., and Hall, B. (2015). Detection of atrial

ﬁbrillation using contactless facial video monitoring.

Heart Rhythm, 12(1):195–201.

De Ponte M

uller, F. (2017). Survey on ranging sensors and

cooperative techniques for relative positioning of ve-

hicles. Sensor, 17:271.

Ding, L., Ali, M., Patole, S., and Dabak, A. (2016). Vibra-

tion parameter estimation using fmcw radar. In 2016

IEEE International Conference on Acoustics, Speech

and Signal Processing (ICASSP), pages 2224–2228.

Fishler, E., Haimovich, A., Blum, R., Chizhik, D., Cimini,

L., and Valenzuela, R. (2004). Mimo radar: an idea

whose time has come. In Proceedings of the 2004

IEEE Radar Conference (IEEE Cat. No.04CH37509),

pages 71–78.

Gambi, E., Agostinelli, A., Belli, A., Burattini, L., Cip-

pitelli, E., Fioretti, S., Pierleoni, P., Ricciuti, M.,

Sbrollini, A., and Spinsante, S. (2017). Heart rate de-

tection using microsoft kinect: Validation and com-

parison to wearable devices. Sensors, 17(8):1776.

Georgiou, K., Larentzakis, A. V., Khamis, N. N., Al-

suhaibani, G. I., Alaska, Y. A., and Giallafos, E. J.

(2018). Can wearable devices accurately measure

heart rate variability? a systematic review. Folia med-

ica, 60(1):7–20.

Hsu, G.-S., Ambikapathi, A., and Chen, M.-S. (2017). Deep

learning with time-frequency representation for pulse

estimation from facial videos. In 2017 IEEE Interna-

tional Joint Conference on Biometrics (IJCB), pages

383–389. IEEE.

Hsu, T.-W. and Tseng, C.-H. (2016). Compact 24-ghz

doppler radar module for non-contact human vital-

sign detection. In 2016 International Symposium on

Antennas and Propagation (ISAP), pages 994–995.

IEEE.

Instruments, T. (2019). Awr1843 single-chip 76-ghz to 81-

ghz automotive radar sensor evaluation module. http:

//www.ti.com/tool/AWR1843BOOST.

Jian, L. and Stoica, P. (2009). MIMO Radar Signal Process-

ing. Wiley-IEEE Press, ﬁrst edition.

Kumar, M., Weippert, M., Vilbrandt, R., Kreuzfeld, S., and

Stoll, R. (2007). Fuzzy evaluation of heart rate signals

for mental stress assessment. IEEE Transactions on

fuzzy systems, 15(5):791–808.

Lee, H., Kim, B.-H., Park, J.-K., and Yook, J.-G. (2019).

A novel vital-sign sensing algorithm for multiple sub-

jects based on 24-ghz fmcw doppler radar. Remote

Sensing, 11:1237.

Lee, K., Park, D., Cho, C., and Ko, H. (2018). Robust re-

mote heart rate estimation in car driving environment.

In 2018 IEEE International Conference on Consumer

Electronics (ICCE), pages 1–2.

Li, X., Chen, J., Zhao, G., and Pietik

ainen, M. (2014). Re-

mote heart rate measurement from face videos under

realistic situations. In 2014 IEEE Conference on Com-

puter Vision and Pattern Recognition, pages 4264–

4271.

Lin, J. C., Kiernicki, J., Kiernicki, M., and Wollschlaeger,

P. B. (1979). Microwave apexcardiography. IEEE

Transactions on Microwave Theory and Techniques,

27(6):618–620.

Michahelles, F., Wicki, R., and Schiele, B. (2004). Less

contact: Heart-rate detection without even touching

the user. In Eighth International Symposium on Wear-

able Computers, volume 1, pages 4–7. IEEE.

noz-Ferreras, J., Wang, J., Peng, Z., Li, C., and G

omez-

Garc

ıa, R. (2019). Fmcw-radar-based vital-sign mon-

itoring of multiple patients. In 2019 IEEE MTT-S In-

ternational Microwave Biomedical Conference (IM-

BioC), volume 1, pages 1–3.

Paulson, C. N., Chang, J. T., Romero, C. E., Watson, J.,

Pearce, F. J., and Levin, N. (2005). Ultra-wideband

radar methods and techniques of medical sensing and

imaging. In Smart Medical and Biomedical Sensor

Technology III, volume 6007, page 60070L. Interna-

tional Society for Optics and Photonics.

Pisa, S., Pittella, E., and Piuzzi, E. (2016). A survey of radar

systems for medical applications. IEEE Aerospace

and Electronic Systems Magazine, 31(11):64–81.

Ren, L., Wang, H., Naishadham, K., Kilic, O., and Fathy,

A. E. (2016). Phase-based methods for heart rate

detection using uwb impulse doppler radar. IEEE

Transactions on Microwave Theory and Techniques,

64(10):3319–3331.

Rouast, P. V., Adam, M. T., Chiong, R., Cornforth, D., and

Lux, E. (2018). Remote heart rate measurement using

low-cost rgb face video: a technical literature review.

Frontiers of Computer Science, 12(5):858–872.

Rumi

nski, J. (2016). Reliability of pulse measurements in

videoplethysmography. Metrology and Measurement

Systems, 23(3):359–371.

Sanyal, S. and Nundy, K. K. (2018). Algorithms for moni-

toring heart rate and respiratory rate from the video of

a user’s face. IEEE journal of translational engineer-

ing in health and medicine, 6:1–11.

Sebe, N., Alameda-Pineda, X., Tulyakov, S., Ricci, E., Yin,

L., and Cohn, J. F. (2019). Self-adaptive matrix com-

pletion for heart rate estimation from face videos un-

der realistic conditions. US Patent App. 10/335,045.

Srivastava, S. (2014). Weka: a tool for data preprocess-

ing, classiﬁcation, ensemble, clustering and associa-

tion rule mining. International Journal of Computer

Applications, 88(10).

Theodor, P. (1996). Introduction to signal processing, by s.j.

orfanidis, prentice hall signal processing series, pren-

tice hall, upper saddle river, new jersey, 1996 - book

review. Control Engineering Practice, 4:1771–1772.

ICT4AWE 2020 - 6th International Conference on Information and Communication Technologies for Ageing Well and e-Health

128

Tremper, K. K. (1989). Pulse oximetry. Chest, 95(4):713–

715.

Verkruysse, W., Svaasand, L. O., and Nelson, J. S. (2008).

Remote plethysmographic imaging using ambient

light. Optics express, 16(26):21434–21445.

Viola, P. and Jones, M. J. (2004). Robust real-time face

detection. International journal of computer vision,

57(2):137–154.

Wang, S., Pohl, A., Jaeschke, T., Czaplik, M., K

ony, M.,

Leonhardt, S., and Pohl, N. (2015). A novel ultra-

wideband 80 ghz fmcw radar system for contactless

monitoring of vital signs. In 2015 37th Annual In-

ternational Conference of the IEEE Engineering in

Medicine and Biology Society (EMBC), pages 4978–

4981.

Wang, S., Pohl, A., Jaeschke, T., Czaplik, M., K

ony, M.,

Leonhardt, S., and Pohl, N. (2015). A novel ultra-

wideband 80 ghz fmcw radar system for contactless

monitoring of vital signs. In 2015 37th Annual In-

ternational Conference of the IEEE Engineering in

Medicine and Biology Society (EMBC), pages 4978–

4981. IEEE.

Wang, Y., Liu, Q., and Fathy, A. E. (2013). Cw and

pulse–doppler radar processing based on fpga for hu-

man sensing applications. IEEE Transactions on Geo-

science and Remote Sensing, 51(5):3097–3107.

Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F.,

and Freeman, W. (2012). Eulerian video magniﬁca-

tion for revealing subtle changes in the world. ACM

Transactions on Graphics, 31(4):1–8.

Xiong, Y., Peng, Z., Xing, G., Zhang, W., and Meng, G.

(2018). Accurate and robust displacement measure-

ment for fmcw radar vibration monitoring. IEEE Sen-

sors Journal, 18(3):1131–1139.

Zetik, R., Sachs, J., and Thoma, R. S. (2007). Uwb short-

range radar sensing-the architecture of a baseband,

pseudo-noise uwb radar sensor. IEEE Instrumentation

& Measurement Magazine, 10(2):39–45.

Contactless Heart Rate Measurements using RGB-camera and Radar

129