Deep Visio-PhotoPlethysmoGraphy Reconstruction Pipeline for

Non-invasive Cuff-less Blood Pressure Estimation

Francesco Rundo

1 a

, Francesca Trenta

2 b

, Roberto Leotta

and Sebastiano Battiato

2 c

STMicroelectronics, ADG Central R&D, Catania, Italy

Department of Mathematics and Computer Science, University of Catania, Catania, Italy

Keywords:

Deep Learning, Computer Vision, PPG (PhotoPlethysmoGraphy).

Abstract:

In medical ﬁeld, many cardiovascular and correlated diseases can be early treated by monitoring and analyzing

the subject’s blood pressure (BP). However, the measurement of blood pressure requires the use of invasive

medical and health equipment, including the classical sphygmomanometer or the digital pressure meter. In

this paper, we proposed an innovative algorithmic pipeline to properly estimate the systolic and diastolic

blood pressure of a subject through the visio-reconstruction of the PhotoPlethysmoGraphic (PPG) signal. By

means of an innovative method of face-motion magniﬁcation through Deep Learning, it is possible to visio-

reconstruct speciﬁc points of the PPG signal in order to extract features related to the pressure level of the

analyzed subject. The proposed approach can be used effectively in healthcare facilities for the fast and non-

invasive monitoring of the pressure level of subjects or in other similar applications. We compared our results

using a classic cuff-less blood pressure device with encouraging results that reach 92% in accuracy.

1 INTRODUCTION

Monitoring the systolic and diastolic pressure of both

healthy and hypertensive subjects is certainly one of

the most important aspects for safeguarding subject’s

health. Many cardiovascular diseases are strictly

caused by pressure dysfunctions and therefore can

be easily treated if the blood pressure level is kept

under control (Wu et al., 2015). In recent years,

there has been an increasing interest in measuring

blood pressure by taking advantage of simple and

non-invasive approaches, some of these are based

on the use of the PhotoPlethysmoGraphic (PPG) sig-

nal (Dastjerdi et al., 2017). Photoplethysmography is

a simple optical technique that can be used to detect

changes in the volume of blood in the microvascular

bed of tissues (Rundo et al., 2018c). It is not inva-

sive since it makes measurements on the surface of

the skin. Based on these assumptions, we designed a

Deep Learning (DL) architecture to perform the visio-

reconstruction of the PPG signal from the face-motion

analysis of the subject, with the aim to extract ad-hoc

features correlated to the level of systolic and dias-

https://orcid.org/0000-0003-1766-3065

https://orcid.org/0000-0003-2524-3837

https://orcid.org/0000-0001-6127-2470

tolic pressure. The proposed pipeline can be imple-

mented as an embedded ﬁrmware in any smartphone

equipped with a video camera and PPG sensing-

device (classical Light Emitting Diodes LEDs with

Silicon PhotoMultiplier (SiPM) photo-detector de-

vice (Rundo et al., 2018c)) and therefore can be ap-

plied daily and simply by any subject (Rundo et al.,

2018c). In Figure 1, we illustrated the method used

to sample the PPG signal as well as the physiological

correlation with heart activity and them with blood

pressure (Vinciguerra et al., 2018). As shown in Fig-

ure 1, the PPG sensing framework (PPG Sensing De-

vice) is composed by a coupled LEDs with SiPM de-

vice (photo-detector) with a STM32 based microcon-

troller for pre-processing the sampled physiological

raw data (Vinciguerra et al., 2018). More details

in (Rundo et al., 2018c; Vinciguerra et al., 2018).

In Figure 1, it is also evident how the PPG signal is

strongly correlated to the cardiac activity of a subject

(by measurement of changes in blood volume) and

then with blood pressure. The remainder of the pa-

per is structured as follows. In Section II we present

the related works. In Section III we describe the pro-

posed pipeline while in Section IV we report the ex-

periments we made for validating the proposed ap-

proach. Finally, in Section V we report the conclu-

sions.

Trenta, F., Rundo, F., Leotta, R. and Battiato, S.

Deep Visio-PhotoPlethysmoGraphy Reconstruction Pipeline for Non-invasive Cuff-less Blood Pressure Estimation.

DOI: 10.5220/0010380900750080

In Proceedings of the International Conference on Image Processing and Vision Engineering (IMPROVE 2021), pages 75-80

ISBN: 978-989-758-511-1

Figure 1: The PPG signal sampling pipeline.

2 RELATED WORKS

A considerable amount of literature focused on esti-

mating blood pressure or arterial stiffness taking ad-

vantage of Deep Learning (DL) methods. In (Monte-

Moreno, 2011), the author proposed a non-invasive

approach to estimate the systolic and diastolic blood

pressure by using PPG. The experimental results shed

light on relationship between blood pressure, glu-

cose levels and PPG waveform conﬁrming the ef-

fectiveness of Machine Learning (ML)-based tech-

niques. In addition to these techniques, special-

ized ML-based methods have been proposed with

the recent emergence of Deep Learning. Slapnicar

et al. (Slapni

car et al., 2019) investigated the prob-

lem of detecting Blood Pressure (BP) using a ML-

based architecture. In order to overcome limitations

derived from using cuff-based devices, the authors

used PPG and its ﬁrst and second derivative to feed a

novel spectro-temporal Deep Neural Network (DNN).

Finally, they performed a leave-one-subject-out ex-

periments conﬁrming the advantage of the proposed

model in computing dependency between PPG wave-

forms and blood pressure. In (Alty et al., 2007),

the authors proposed a ML-based pipeline to predict

arterial stiffness i.e. and an indicator correlated to

blood pressure. It is reliable for the classiﬁcation of

subjects into high and low aortic pulse wave veloc-

ity (PWV) aiming to analyze cardiovascular disease.

Their work highlights the effective results achieved

by Support Vector Machine (SVM) not only for clas-

siﬁcation purpose but also in performing multilinear

regression. In (Rundo et al., 2018b), the authors de-

scribed an innovative approach to estimate cardiovas-

cular disease risk via blood pressure. The proposed

method measures BP by analyzing the PPG signal

without requiring any user calibration. In (Huynh

et al., 2018), the authors proposed an interesting ap-

proach to estimate the blood pressure by using aver-

aging Impedance Plethysmography (IPG) for detec-

Figure 2: Example of a LSTM cell.

tion of Pulse Transit Time (PTT). The experimen-

tal estimation of blood pressure (BP) provided very

interesting results (RMSE: 8.47 ± 0.91 mmHg and

5.02 ± 0.73mmHg for systolic and diastolic level, re-

spectively). Despite providing effective results, the

aforementioned approaches require the use of inva-

sive medical devices throughout the measurement of

blood pressure levels. In this work, the authors pro-

posed a less-invasive and cuff-less approach for mea-

suring blood pressure.

3 BACKGROUND AND THE

PROPOSED PIPELINE

In this paper, a novel Deep Long Short-Term Mem-

ory (LSTM) based pipeline is presented. The

LSTM Vanilla architecture was originally proposed

by Hochreiter and Schmidhuber (Hochreiter and

Schmidhuber, 1997) for preventing the Vanishing gra-

dient problem which affects Recurrent Neural Net-

works (RNNs). The LSTM cell is able to select

what information to discard or store. In order to

produce effective results in real-applications, this se-

lective method requires three different mechanism to

read, store and discard information by taking advan-

tage of speciﬁc vectors called “gates”. Basically, the

“input gate”, “output gate” and “forget gate” decision

is implemented via activation functions which deﬁne

if a given information is relevant or not. More in de-

tails, given x

as input vector, h

t−1

as previous cell

output, C

t−1

previous cell memory, h

as current cell

output and C

as current cell memory, we deﬁne Equa-

tions (1)-(3) to determine what information to store.

Finally, we generate the output of LSTM by updat-

ing old cell state as per Equation (4) and merging the

previous output, the input and the bias vector as Equa-

tions (5)-(6).

= σ(W

◦ [h

t−1

, x

] + b

(1)

= σ(W

◦ [h

t−1

, x

] + b

(2)

IMPROVE 2021 - International Conference on Image Processing and Vision Engineering

Figure 3: The proposed Deep LSTMs pipeline.

= tanh(W

◦ [h

t−1

] + b

(3)

= f

∗C

t−1

+ i

∗

(4)

= σ(W

◦ [h

t−1

, x

] + b

(5)

= o

∗ tanh(C

)

(6)

In Figure 2, a prototype of LSTM cell is reported.

In recent years, several LSTM-based approaches have

been developed with promising results. For instance,

LSTM architectures have been largely employed in

automotive ﬁeld to visio-reconstruct such part of the

car-driver PPG signal (sampled by ad-hoc sensors

placed in the steering) when this physiological sig-

nal was no longer available (Trenta et al., 2019). In-

spired by recent literature, we designed a Deep LSTM

pipeline aiming to better visio-reconstruct the PPG

waveforms of a subject in order to estimate the cor-

related blood pressure. The proposed system archi-

tecture consists of a physical signal-acquisition mod-

ule (PPG sensing device) needed for preliminary cal-

ibration as well as a vision module to extract effec-

tive facial descriptors and a Machine Learning frame-

work that reconstruct such part of the PPG wave-

forms (extremes points such as minimum, maximum

and so on) in order estimate the blood pressure (BP).

During the calibration phase of the proposed system,

the PPG signal is collected by using a coupled LED-

SiPM sensing system available in several medical de-

vices or smartphones and able to generate the PPG

raw data (Rundo et al., 2018a; Rundo et al., 2017).

Then, we apply the patented Bi2PRS algorithm to ob-

tain the ﬁltered compliant PPG signal. The Bi2PRS

algorithm (Rundo et al., 2018a) is able to properly

ﬁlter the collected PPG raw data (by means of an ad-

hoc Butterworth band-pass ﬁlter) and then to deter-

mine, in the ﬁltered PPG waveforms, the right value

of the extreme points such as max, min, etc. More

details on Bi2PRS in (Rundo et al., 2018a; Rundo

et al., 2017). During the PPG sampling session, we

recorded a video sequence of the subject by using a

smartphone having a camera device with 30 fps as

frame-rate, under high light condition. As widely

known in scientiﬁc literature (Oh et al., 2018), the

face of the subject performs visual micro-movements

imperceptible at naked eye and closely related to the

cardiac pumping activity. These micro-movements

are strongly correlated to PPG signal as a cardiac re-

lated signal. Inspired by the work of Wu et al. (Wu

et al., 2012), in which the authors introduced Video

Magniﬁcation to amplify facial micro movements for

revealing the ﬂow of blood, we proposed an approach

in which a group of video frames (face sequences of

the subject) are extracted and then analyzed in or-

der to identify such facial landmarks to be tracked

(through the pixel intensities over time). Speciﬁcally,

we designed input layer of the proposed Deep LSTM

model for processing facial landmarks of both eyes.

In Figure 3, we have summarized the overall scheme

of our proposed pipeline. More details in the next

paragraphs. The framework of the proposed Deep

LSTM architecture is composed by one input layer,

three hidden layers, and one output layer. Speciﬁ-

cally, we designed three hidden layers which include

two regular layers and one dropout layer appointed to

boost the overall performance. The model was trained

with an initial learning rate of 10

−3

. The batch size

was set to 512 and the maximum number of train-

Deep Visio-PhotoPlethysmoGraphy Reconstruction Pipeline for Non-invasive Cuff-less Blood Pressure Estimation

ing epochs was set to 100. During the calibration

phase, we trained the designed Deep LSTM to ana-

lyze the correlation between the facial time-evolution

selected landmarks of the subjects with correlated ex-

treme points of the PPG signal. The output of the

LSTM pipeline represents predicted extreme points

of the PPG signal considering the facial landmarks

time series. When the proposed Deep LSTM frame-

work has learned the correlation between facial land-

marks and extreme points of the subject’s PPG sig-

nal, the calibration phase will be dropped and there-

fore the system will operate feed-forward in the vi-

sion part only linked to the acquisition of facial land-

marks only. The calibration phase of the device re-

quires few minutes of acquisition of the PPG signal

(and corresponding visual data) and it will be neces-

sary to perform it only once. On the other hand, the

trained feed-forward system (only Visio-based Deep

LSTMs pipeline followed by reconstructed PPG ex-

treme point(s) classiﬁer, as described in the section

IV) is able to generate the output (blood pressure es-

timation) in a near real-time context (few seconds).

The so designed pipeline was ported to the STM32

architecture through the STM32-CUBE AI software

platform. In Figure 4, a graph reporting the esti-

mated extreme points by the proposed trained Deep

LSTM super-imposed with original source PPG sig-

nal is shown. Once we have collected the set of char-

acteristic extreme points of the waveforms of the PPG

signal, we are able to characterize the subject’s car-

diac activity with regard to the two phases of the car-

diac cycle i.e. systole and diastole, on which the level

depends on blood pressure. In Figure 5, we reported

a classical PPG waveform and the corresponding dy-

namic correlated to the cardiac cycle. By means of the

characteristic PPG visio-reconstructed extreme points

(m1, m2, m3, m4), we are able to estimate the heart

systole and diastole phases, therefore, the pressure

levels related to them. For each pair of PPG wave-

forms (PPG

, PPG

j+1

) we deﬁne the following indi-

cators:

ϕ = [m

, m

,dx

,dy

,mAI

]

∀ j = 1..N

PPG

− 1 ; i = 1,2, 3, 4

(7)

= (x

)

∀ j = 1...N

PPG

− 1 ; i = 1,2, 3, 4

(8)

= x

j+1

− x

∀ j = 1...N

PPG

− 1 ; i = 1,2, 3, 4

(9)

= y

j+1

− y

∀ j = 1...N

PPG

− 1 ; i = 1,2, 3, 4

(10)

Figure 4: The reconstructed extreme points super-imposed

to the corresponding source PPG signal.

mAI

= ((y

− y

) − y

)/(y

− y

)

(11)

where mAI

is a modiﬁed version of the so-called

Augmentation Index usually computed for measur-

ing the arterial stiffness (Gonzalez et al., 2012) while

NPPG represents the number of PPG waveforms. The

other indicators reported in the Equations (8)-(10) al-

low us to characterize cardiac cycles (and therefore

the relative pressure levels) according to the PPG sig-

nal mapping reported in Figure 5. The elements of the

vector ϕ represents the input of a machine learning

framework (Fully Connected Multi-Layers Network

with binary output) designed to learn the correlation

between the so computed input elements and the cor-

responding value of the systolic and diastolic blood

pressure. The output of the machine learning frame-

work is a binary value which can be considered as a

discriminating ﬂag to indicate if the subject presents

normal pressure values (0) or not (1). The set 120/80,

which indicates 120 mmHg for systolic pressure and

80 mmHg for diastolic pressure, has been considered

as normal blood pressure values. Under the supervi-

sion of a team of physiologists, we have deﬁned all

pressure values less than or equal to 120/80 as ac-

ceptable blood pressure while higher values are con-

sidered anomalous and as such must be signaled and

monitored. It should be noted that the proposed sys-

tem can monitor and discriminate even different pres-

sure levels (with respect to the classic 120/80 mmHg)

requiring a different and adequate calibration. The

proposed pipeline has been tested and validated as de-

scribed in the following paragraph.

4 EXPERIMENTS

In order to train and validate the proposed system, un-

der the supervision of the physiologists who collabo-

rated with us in this work, we have collected a dataset

of subjects to perform systolic and diastolic pressure

measurements simultaneously with the acquisition of

IMPROVE 2021 - International Conference on Image Processing and Vision Engineering

Figure 5: The reconstructed extreme points super-imposed

to the corresponding source PPG signal.

the PPG signal and contextually the video sequence

reporting the subject’s face. The blood pressure mea-

surements were performed using a classic certiﬁed

medical sphygmomanometer. All procedures were

carried out under the supervision of the physiologists

and after we have received the informed consent of

each patient and having acquired the consent from the

Ethical Committee CT1 (authorization n.113 / 2018 /

PO), which were conducted in accordance with the

Declaration of Helsinki. The dataset is composed of

56 both healthy and hypertensive subjects including

both males and females individuals. The minimum

age of the subjects in the dataset is 21 years while the

maximum age is 70 years. The collected minimum

pressure value is around 110/75 while the maximum

pressure value is 140/85. For each subject, systolic

and diastolic pressure was acquired as well as a few

minutes (5 min) acquisition of PPG signal and video

signal (subject face), as above described. The acquisi-

tions of the PPG signal, needed for the preliminar cal-

ibration and training of the whole pipeline, were per-

formed at a sampling frequency of 1 kHz and by us-

ing the system described in (Rundo et al., 2018a). We

designed a Fully Connected Network (FCN) trained

with the Scaled Conjugate Gradient backpropagation

(SCG) algorithm described in (Møller, 1993) with a

unique hidden layer of 500 neurons. The designed

FCN learns as input the visio-reconstructed PPG ex-

treme points while the output is a binary ﬂag con-

ﬁrming if the corresponding blood pressure is normal

(0) or not (1). In Figure 6, we reported the learn-

ing error dynamic of the proposed machine learning

framework as well as the ROC curves both in train-

ing and validation. The dataset has been divided as

follow: 70% of the data has been used for the train-

ing activity while the remaining 30% for testing and

validation. In order to robustly validate the proposed

method, we calibrated the proposed pipeline for all

subjects of the training and validation dataset and

then we created tests consisting of multiple frames

Figure 6: (a) Learning error dynamic histogram of the Ma-

chine Learning classiﬁer (b) ROC curves both in validation

and training dataset.

of the subjects that are sequentially and randomly

fed to the Machine Learning system (for each sub-

ject we initialize the machine learning system with

related weights computed during calibration). For

each patient, we reconstituted the characteristic ex-

treme points of the PPG signal which were then fed

to the machine learning framework for classifying the

corresponding blood pressure level. The total accu-

racy (in test set) of the proposed system in discrim-

inating normal-pressure subjects from subjects with

blood pressure higher than normal or hypertensive is

91.7%.

Deep Visio-PhotoPlethysmoGraphy Reconstruction Pipeline for Non-invasive Cuff-less Blood Pressure Estimation

5 CONCLUSIONS

The obtained results are very promising in the ﬁeld

of medical-health applications for the early preven-

tion of cardiovascular pathologies. The main beneﬁt

of the proposed system is the non-invasive and effec-

tive estimation of the subject’s blood pressure level

in few seconds. The experimental results allow us to

be conﬁdent about the applicability of this approach

in different applications in the medical ﬁeld. Future

works will focus on collecting more data in order to

improve the effectiveness of the proposed approach

as well as to implement a robust pipelines for moni-

toring the response to certain oncological treatments

(such as chemotherapy and immunotherapy) as many

anti-neoplastic drugs are known to produce abnor-

mal increases in blood pressure which therefore re-

quires continuous monitoring and within acceptable

times (Banna et al., 2018; Rundo et al., 2019).

REFERENCES

Alty, S. R., Angarita-Jaimes, N., Millasseau, S. C., and

Chowienczyk, P. J. (2007). Predicting arterial stiffness

from the digital volume pulse waveform. IEEE Trans-

actions on Biomedical Engineering, 54(12):2268–

2275.

Banna, G. L., Camerini, A., Bronte, G., Anile, G., Ad-

deo, A., Rundo, F., Zanghi, G., Lal, R., and Li-

bra, M. (2018). Oral metronomic vinorelbine in

advanced non-small cell lung cancer patients unﬁt

for chemotherapy. Anticancer research, 38(6):3689–

3697.

Dastjerdi, A. E., Kachuee, M., and Shabany, M. (2017).

Non-invasive blood pressure estimation using phono-

cardiogram. In 2017 IEEE International Symposium

on Circuits and Systems (ISCAS), pages 1–4. IEEE.

Gonzalez, R., Manzo, A., Delgado, J., Gomis-Tena, J., and

Saiz, J. (2012). Photoplethysmographic augmentation

index using the signal fourth derivative. In 2012 Com-

puting in Cardiology, pages 821–824. IEEE.

Hochreiter, S. and Schmidhuber, J. (1997). Long short-term

memory. Neural computation, 9(8):1735–1780.

Huynh, T. H., Jafari, R., and Chung, W.-Y. (2018). Nonin-

vasive cufﬂess blood pressure estimation using pulse

transit time and impedance plethysmography. IEEE

Transactions on Biomedical Engineering, 66(4):967–

976.

Møller, M. F. (1993). A scaled conjugate gradient algo-

rithm for fast supervised learning. Neural networks,

6(4):525–533.

Monte-Moreno, E. (2011). Non-invasive estimate of blood

glucose and blood pressure from a photoplethysmo-

graph by means of machine learning techniques. Arti-

ﬁcial intelligence in medicine, 53(2):127–138.

Oh, T.-H., Jaroensri, R., Kim, C., Elgharib, M., Du-

rand, F., Freeman, W. T., and Matusik, W. (2018).

Learning-based video motion magniﬁcation. In Pro-

ceedings of the European Conference on Computer Vi-

sion (ECCV), pages 633–648.

Rundo, F., Conoci, S., Fallica, P. G., and Petralia, S. (2017).

Processing of electrophysiological signals.

Rundo, F., Conoci, S., Ortis, A., and Battiato, S.

(2018a). An advanced bio-inspired photoplethysmog-

raphy (ppg) and ecg pattern recognition system for

medical assessment. Sensors, 18(2):405.

Rundo, F., Ortis, A., Battiato, S., and Conoci, S. (2018b).

Advanced bio-inspired system for noninvasive cuff-

less blood pressure estimation from physiological sig-

nal analysis. Computation, 6(3):46.

Rundo, F., Petralia, S., Fallica, G., and Conoci, S. (2018c).

A nonlinear pattern recognition pipeline for ppg/ecg

medical assessments. In Convegno Nazionale Sensori,

pages 473–480. Springer.

Rundo, F., Spampinato, C., Banna, G. L., and Conoci, S.

(2019). Advanced deep learning embedded motion

radiomics pipeline for predicting anti-pd-1/pd-l1 im-

munotherapy response in the treatment of bladder can-

cer: Preliminary results. Electronics, 8(10):1134.

Slapni

car, G., Mlakar, N., and Lu

strek, M. (2019). Blood

pressure estimation from photoplethysmogram using

a spectro-temporal deep neural network. Sensors,

19(15):3420.

Trenta, F., Conoci, S., Rundo, F., and Battiato, S. (2019).

Advanced motion-tracking system with multi-layers

deep learning framework for innovative car-driver

drowsiness monitoring. In 2019 14th IEEE Inter-

national Conference on Automatic Face & Gesture

Recognition (FG 2019), pages 1–5. IEEE.

Vinciguerra, V., Ambra, E., Maddiona, L., Romeo, M.,

Mazzillo, M., Rundo, F., Fallica, G., di Pompeo,

F., Chiarelli, A. M., Zappasodi, F., et al. (2018).

Ppg/ecg multisite combo system based on sipm tech-

nology. In Convegno Nazionale Sensori, pages 353–

360. Springer.

Wu, C.-Y., Hu, H.-Y., Chou, Y.-J., Huang, N., Chou,

Y.-C., and Li, C.-P. (2015). High blood pressure

and all-cause and cardiovascular disease mortalities in

community-dwelling older adults. Medicine, 94(47).

Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F.,

and Freeman, W. (2012). Eulerian video magniﬁca-

tion for revealing subtle changes in the world. ACM

transactions on graphics (TOG), 31(4):1–8.

IMPROVE 2021 - International Conference on Image Processing and Vision Engineering