A Mobile Application for Physical Activity Recognition using

Acceleration Data from Wearable Sensors for Cardiac Rehabilitation

M. Chaari

1,2

, M. Abid

2,3

, Y. Ouakrim

2,3

, M. Lahami

and N. Mezghani

2,3

National School of Engineers of Sfax, Sfax University, Tunisia

LICEF Research Center, TELUQ, Montreal, Canada

Laboratoire de Recherche en Imagerie et Orthop

edie (LIO), CRCHUM, Montreal, Canada

Keywords:

mHealth, Mobile Application, Cardiac Rehabilitation, Human Activity Recognition (HAR), Wearable

Sensors, Classiﬁcation.

Abstract:

mHealth applications are an ever-expanding frontier in today’s use of technology. They allow a user to record

health data and contact their doctor from the convenience of a smartphone. This paper presents a ﬁrst ver-

sion release of a mobile application that aims to assess compliance of cardiovascular diseased patients with

home-based cardiac rehabilitation, by monitoring physical activities using wearable sensors. The application

generates reports for both the patient and the doctor through an interactive dashboard, as initial proposal, that

provides feedback of physical activities of daily living undertaken by the patient. The application integrates a

human activity recognition system, which learns a support vector machine algorithm to identify 10 different

daily activities, such as walking, going upstairs, sitting and lying, from accelerometer data using a connected

textile including movement sensors. Our early deployment and execution results are promising since they are

showing good accuracy for recognizing all the ten daily living activities.

1 INTRODUCTION

Cardiac rehabilitation (CR) is a systematic model of

chronic vascular disease care that pro-actively moni-

tors these conditions using a multi-faceted approach.

This approach includes behavior change strategies re-

lated to a sustainable lifestyle and adherence to phar-

macological treatment as well as therapeutic exercises

and physical activity programs to improve secondary

prevention outcomes in patients with cardiovascular

disease or recovering from surgery. CR reduces to-

tal mortality and cardiac mortality by 20 to 25% (Cyr

et al., 2018). It may also reduce the number of

hospitalizations related to heart disease and the need

for new revascularization procedures in patients with

coronary artery disease. However, only a minority of

eligible patients participate in CR programs. Home-

based cardiac rehabilitation (HBCR) is deﬁnitely one

of the new urgently needed strategies to improve the

participation rate. It uses remote coaching with indi-

rect exercise supervision and helps limit hospital or

clinic visits (Thomas et al., 2019). In recent decades,

CR has evolved from simple surveillance aimed at a

safe return to physical activity, to a multidisciplinary

approach focused on patient education, personalized

physical training, changing risk factors and the well-

being of cardiac patients (Mampuya, 2012). More-

over, recent advances in information and communica-

tion technologies have been used to enhance HBCR

programs (Varnﬁeld et al., 2011). Besides improv-

ing the quality of measures, wearable devices and

portable medical sensors have also proven effective

in monitoring a greater number of patients in pre-

vention and rehabilitation programs in a personalized

manner. As a result, and thanks to recent tools, the

use of home-based mHealth programs has been in-

creasing, achieving good control over vital signs and

physical activities (Medina et al., 2017). Recent re-

search studies in human activity recognition (HAR)

have focused on sensor-based home monitoring sys-

tems. HAR is deﬁned as the ability of an intelligent

system to infer temporally contextualized knowledge

regarding the state of the user and to classify a set of

human activities on the basis of a set of sensor read-

ings. Its role, extremely important in the burgeon-

ing healthcare ﬁeld, is to provide all the necessary

data on the patient’s health and well-being outside a

hospital setting. Technological advances have made

HAR a rapidly growing area, thanks to the use of af-

fordable mobile platforms such as smart phones and

other personal tracking devices (Damasevicius et al.,

2016), which, together with body-worn sensors, can

solve the cardiac patient monitoring problem. There

are countless examples of applications that use hu-

Chaari, M., Abid, M., Ouakrim, Y., Lahami, M. and Mezghani, N.

A Mobile Application for Physical Activity Recognition using Acceleration Data from Wearable Sensors for Cardiac Rehabilitation.

DOI: 10.5220/0009118706250632

In Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 5: HEALTHINF, pages 625-632

ISBN: 978-989-758-398-8; ISSN: 2184-4305

625

man activity recognition to indicate the health status

of humans. Fitbit, for example, offers smart watches

and ﬁtness bands to allow healthcare professionals

to track individuals and monitor their activity trends

over time (Paul et al., 2015). Tactio Health, Samsung

Health and many more are able to track the number of

steps, sleep patterns, passive periods, etc (Voicu et al.,

2019). They are designed to facilitate monitoring pa-

tients efﬁciently so doctors can have better control of

their chronic conditions. Although these apps recog-

nize a very limited number of activities, they have ex-

cellent results, being extremely accurate in detecting

the type of activity performed.

Our study aims to develop a mobile application for

monitoring physical activities by using wearable sen-

sors as part of a HBCR program which is designed for

patients who are unable to attend the traditional fa-

cility based cardiac rehabilitation. The application’s

goal is to help clinicians perform remote follow-up

on their patients. It displays their daily routine activi-

ties on a simple dashboard, providing doctors with the

necessary information to check patient’s health condi-

tion. So this allows patients to exercise in their own

homes while being supervised by one of the clinicians

in cardiac rehabilitation all done using the mobile ap-

plication. Patients also have access to the application

so they can also check their improvement.

This work will be presented as follows: Section 2

analyzes the methods used in this project. In Section

3, the system architecture is explained, with speciﬁca-

tion of the technologies chosen for the application de-

velopment. Section 4 focuses on presenting the main

results, and, ﬁnally, the conclusion is given in Section

2 METHODS AND MATERIALS

The work presented here was carried out as a joint

academic-industry research project designed to mea-

sure patient adherence to HBCR programs. In the

present study, a mobile application is developed for

activity monitoring based on data acquired by the

Hexoskin intelligent textile shirt

, developed by Carr

Technologies (Montreal, Canada). The Hexoskin is

an easily donned, comfortable stretch shirt that can

be used in any ambient environment. Data acquisi-

tion by the Hexoskin is non-invasive and can be per-

formed continuously without hampering the move-

ments of the person wearning it. Hexoskin offers the

only clinically validated system that allows precise

ECG cardiac monitoring with lung function and ac-

https://www.hexoskin.com/

tivity monitoring over the long-term (Banerjee et al.,

2015). Its measurements were found to be reliable in

many research studies. This system offers a recording

capacity of over 42,000 data points of scientiﬁc infor-

mation per minute. The sensors are placed away from

the chest area so users can safely engage in contact

sports (Cherif et al., 2018a). A cable running from

the shirt is plugged into the companion device that is

secured in a waist pocket (Figure 1).

Figure 1: Hexoskin shirt, smart device and USB cable.

Once reception of body metrics data begins, the

Hexoskin device can wirelessly stream the data to a

mobile device in real-time, or store the information

until the user is able to transfer it via the supplied

USB cable. In practice, the Hexoskin enables real-

time monitoring of cardiac, respiratory and 3D accel-

eration data via cardiac, breathing and movement em-

bedded sensors, respectively (Figure 2). In this paper,

we focus on movement sensors that calculate 3-axis

accelerations.

(a) (b) (c)

Figure 2: Hexoskin embedded sensors : (a) cardiac, (b)

breathing and (c) movement sensors. The cardiac sensors

allow to track the heart rate and display the electrocardio-

gram (ECG) in real time. The breathing sensors indicate the

breathing capacity for a given activity. The movement sen-

sors calculate 3-axis accelerations, activity levels and steps.

This study extends previous research (Cherif et al.,

2018b), by implementing a HAR framework on a

HEALTHINF 2020 - 13th International Conference on Health Informatics

626

mobile phone using 3D acceleration data. As illus-

trated in Figure 3, the HAR framework consists of

two phases: the ofﬂine data training and the online

classiﬁcation.

Figure 3: The HAR framework includes an ofﬂine learning

and an online classiﬁcation.

2.1 Data Collection for Ofﬂine Learning

The data collection has been approved by institu-

tional ethics committees and all subjects provided

written informed consent before participating. Data

collection was performed at the research center of

the University of Montreal Hospital Center (Mon-

treal, Canada) and recruited participants were in good

health and didn’t have chronic pain or known loco-

motor or heart problems. Fourteen more healthy vol-

unteers have been included in the pre-established data

base with the same data collection protocol. A to-

tal of 40 voluntary participants wore the smart textile

Hexoskin and performed 5 repeats of a sequence of

10 tasks: going up the stairs, going down the stairs,

walking, running, sitting, falling right, falling left,

falling backward, falling forward and lying down.

The choice of these selected activities was made to

represent the majority of everyday living activities

(Attal et al., 2015). To ensure a good recording sig-

nal, patients were instructed to moisten the three gray

electrodes inside the shirt, connect the Hexoskin de-

vice to the shirt connector and insert it horizontally

into the shirt pocket with the wire upward and the

light outward.

The accelerations were collected from the 3-axis

sensors integrated into the Hexoskin. In total, the

acceleration data of 40 healthy volunteers perform-

ing the same data collection protocol was collected

and considered in the classiﬁcation model. A video

recording was made during the whole session. Its se-

quences were used later to label the different classes

of physical activities.

Figure 4: A scenario of data collection with a participant

wearing the Hexoskin. Each sub-image corresponds to a

task, i.e., an activity.

2.2 Data Transmission for Online

Classiﬁcation

When the shirt is unplugged, data recording ceases

and the device automatically shuts down after 60 sec-

onds. The administrator logs in to the participant’s

Hexoskin account, connects the device to the com-

puter with the USB cable and launches the HxSer-

vices application. The data start to synchronize auto-

matically and are saved on the Hexoskin server.

In a real-life context, the patient would perform

the data transmission daily using his own computer

to transfer his records from the smart device to the

Hexoskin server. Then, he can launch the mobile ap-

plication. Once he does, a request is sent to the Hex-

oskin server via the Application Programming Inter-

face (API)

(Application Programming Interface) to

get the patient’s data. Upon receiving the request, the

Hexoskin server sends a response.

2.3 Classiﬁcation

The collected acceleration data are trained, during the

ofﬂine phase, using a machine learning classiﬁcation

algorithm which returns the resulting activities in la-

bel form.

As shown in Figure 5, the machine learning clas-

siﬁcation algorithm consists of the following steps:

First, the acceleration norm is computed from the

three axis acceleration signal. The sequence of the

acceleration vector norm is then segmented into short

time sequences using a rectangular window around

each detected peak. More precisely, peaks are de-

tected by thresholding the norm. Time-domain and

frequency domain features are computed on 1s-length

windows from the 3D acceleration data. The Relief-F

algorithm is then used to rank individual features ac-

https://api.hexoskin.com/docs/page/oauth2/

A Mobile Application for Physical Activity Recognition using Acceleration Data from Wearable Sensors for Cardiac Rehabilitation

627

cording to their relevance scores. After the selection

process, the top ten features were retained to be used

for the classiﬁcation. The next step is a physical ac-

tivity classiﬁcation system developed using Matlab.

The classiﬁcation uses the Support Vector Machine

(SVM) classiﬁer. It is a supervised machine learn-

ing algorithm that aims to ﬁnd the optimal separating

decision hyperplanes between classes with the max-

imum margin between patterns of each class. The

tests were done according to the process of Leave-

one-subject-out cross validation for the SVM model.

Tests of its performance, evaluated in terms of correct

classiﬁcation rates, have demonstrated the reliability

of the approach with an accuracy of 95.4%.

There has been numerous studies to investigate dif-

ferent machine learning models for physical human

activity recognition using wearable sensors for accel-

eration signals. In (Attal et al., 2015), the authors

proposed a classiﬁcation methodology to recognize,

using acceleration data, different classes of daily liv-

ing human activities, by comparing different machine

learning techniques namely, k-Nearest Neighbor (k-

NN), SVM, Gaussian Mixture Models (GMM), and

Random Forest (RF). In our paper, the choice of the

feature extraction, selection and classiﬁcation meth-

ods was made just for deploying the HAR framework

on mobile devices. The next step, which is our con-

cern, is the implementation of the mobile application,

and the deployment of our own custom model for

HAR. We’ll be detailing this step in the next section.

Figure 5: Block diagram of the proposed physical activity

classiﬁcation system as proposed in (Cherif et al., 2018b).

In their work, the authors tested an SVM and a KNN clas-

siﬁer.

3 APPLICATION

ARCHITECTURE AND

TECHNOLOGIES

The objective of this project part was to develop a

mobile application for physical activity classiﬁcation

from data collected by the Hexoskin textile. The ap-

plication will be used to implement a tool to measure

compliance with home rehabilitation treatments. The

physiological classiﬁcation and analysis systems as

Figure 6: System scenario: data reception, HAR classiﬁ-

cation and local visualization. The Data is located in the

Hexoskin server. The ofﬂine data training is performed in

the backend server. Only the results are transplanted onto

the mobile device dashboard.

well as the compliance measurement tool will be in-

tegrated in the future into the Hexoskin platform.

The collected data, analysis and patient local inter-

face of the project was implemented in a smartphone

application compatible with Android or iOS. This ap-

plication as show in Figure 6 has the following main

functions:

• Data reception (steps 1, 2 and 3): After data trans-

mission from the Hexoskin shirt to the server, the

application has to receive Hexoskin information

packages from the server using an API. Then, the

system has to extract the acceleration, cardiac and

respiratory data.

• HAR classiﬁer (step 4): The app uses a classiﬁer

algorithm to build a recognition model to estimate

the current activity.

• Local visualization (step 5): All the latest sensor

data can be seen in the mobile screen, in conjunc-

tion with the latest recognized activity.

Figure 7 presents the different parts of our appli-

cation. The frontend comprises the presentation layer

based on an Ionic 4 (Dunka et al., 2017), Angular

and Cordova

framework to produce a multiplat-

form application imitating the native behavior, group-

ing the Man-Machine interfaces and scripts that com-

municate with the backend based on the protocol Hy-

pertext Transfer Protocol (HTTP) and Java Script Ob-

ject Notation (JSON) exchange format.

The backend, corresponding to the second part,

comprises the application server developed using

Python

technology, since the classiﬁcation model

https://angular.io/

https://cordova.apache.org/

https://www.python.org/

HEALTHINF 2020 - 13th International Conference on Health Informatics

628

Figure 7: Global architecture of the application.

was converted from Matlab to Python in order to facil-

itate the connection with the front end. We used the

Flask

framework provided with a lightweight Web

server, which requires minimal conﬁguration, can be

controlled via Python code and can easily exchange

information with MongoDB

, in which the data is

stored.

The third part, the Hexoskin Web server (Webster

et al., 2017), contains the data for the patient wearing

http://ﬂask.palletsprojects.com/en/1.1.x/

https://www.mongodb.com/fr

the connected textile, and it is the Hexoskin Rest API

and the classiﬁcation system that make it possible to

interact and manipulate the Hexoskin data via HTTP

requests. This includes access to biometrics and user

information, but also data annotation, advanced report

visualization, metrics retrieval, and more.

4 RESULTS

This mobile application is developed for an mHealth

context, that may be used for communication be-

tween a patient undertaken a HBCR and a clinician.

However, we will only present screenshots and func-

tionalities to illustrate the dashboard dedicated to the

doctor. The patient’s perspective will be considered

in future work.

Authentication. Authentication is a process that

performs identity veriﬁcation. It is an important

phase in order to secure the application. To access

the various features of the application, the user must

type his credential (login and password) into the

corresponding ﬁelds (Figure 8).

Patient’s Proﬁle. The proﬁle interface, as shown

in Figure 8, is the ﬁrst interface the user sees

and interacts with. It provides a variety of pa-

tient information such as name, birth date, heart

rate, weight, height,etc. The user can then access the

main menu to discover the features of our application.

Dashboard. The dashboard depicted in Figure 9

shows the activities corresponding to the chosen

record. Each activity is described by its name and

duration. The patient’s advancement is also recorded

in the form of progress bars, allowing the doctor to

check whether the patient is following the exercice

program.

Medical Report Generation. After consulting the

dashboard, the doctor can write his medical report.

To do so, he clicks on ”Write a report” to access

the report form. He indicates his patient’s state

(improved, not improved or deteriorated), and then

writes his notes concerning the records he has seen.

Medical Reports List. Each report is saved in the

list (Figure 8) where it is indexed by date, content,

doctor’s name, record id and duration.

https://api.hexoskin.com/docs/index.html#api-keys-

and-oauth-requests

A Mobile Application for Physical Activity Recognition using Acceleration Data from Wearable Sensors for Cardiac Rehabilitation

629

Figure 8: Login, patients proﬁle and medical reports pages.

Patient’s Program. The doctor can set out an exer-

cice program for his patients, setting the performance

durations of the individual activities as goals to reach.

This mHealth solution gathers information from

the wearable sensors incorporated in an intelligent

textile and process it using a HAR approach accessi-

ble via a mobile application. These kinds of applica-

tions provide a novel approach to the health care sys-

tem, resulting in new and better services such as con-

stant monitoring of patients and remote consultation

services. Together, these services yield a faster and

more reliable health care industry with almost zero

waiting time, changing the classical approach of the

health care system as a reactive service to that of a

preemptive industry. In this project, the main focus is

to improve the health care services for patients who,

while their health status is not critical, still need con-

stant monitoring. We have used the Hexoskin as an

easy-to-wear washable textile that tracks heart rate,

breathing rate and acceleration and has been used in

several research studies showing its efﬁciency (Villar

et al., 2015). Moreover, having all the data recorded

in the server allows it to be accessed rapidly when-

ever needed, which is proﬁtable in terms of energy

and time. The classiﬁcation model has an overall ac-

curacy of 95.4% and a very short response time. In

addition, we have developed an interactive interface

which allows the doctor to monitor his patient’s health

easily from home.

The proposed HAR system is used for training and

recognition of human activities in daily living, and in-

tegrated in a mobile device as a ﬁrst release version.

An extensive work will be accomplished for train-

ing and recognition of physical excercices in rehabil-

itation process. The key requirements for the HAR

system designed to support the rehabilitation process

will be speciﬁed in conjunction with the rehabilitation

team.

5 CONCLUSIONS

In this paper, we have presented a multi-platform

HAR-based mobile application for HBCR. We devel-

oped an application that allows doctors to monitor

targeted patients at their homes, using the Hexoskin

intelligent-textile shirt. The shirt records accelera-

tion signals which are then sent to our application for

analysis and classiﬁcation, in order to recognize one

of the following activities: walking, running, falling

right, falling left, falling backward, falling forward,

going up stairs, going down stairs, sitting and lying.

The application allows the doctor to visualize the tex-

tile recordings, each representing wearing of the Hex-

oskin for one day, and to write reports about these

records. The idea was to design an archive of medi-

cal reports as a reference the doctor can use to better

judge a patient’s condition. We aim to increase the

patients particpating rate in HBCR programs thanks

to this system, so that they can lead a normal and safe

life without having to worry about their health. The

work process involved several stages in order to ac-

complish the development and implementation of the

HEALTHINF 2020 - 13th International Conference on Health Informatics

630

Figure 9: The dashboard page.

application and to collect the data. The collection of

data for the new database was also an important phase

in developing the classiﬁcation system. However, this

application is not limited to integrated functionalities

such as monitoring patient’s activity and writing re-

ports.

In future work, we aim to add several types of

exercises that are included in a cardiac rehabilitation

program according to the guidelines of the American

College of Sports Medicine (ACSM). In conjuction

with a cardiac rehabilitation team, we will work on

gathering empirical data from a set of patients with

heart failure following a cardiac rehabilitation pro-

gram. While, the activities we have examinated so far

are quite simple, many of them could in fact be part

of more complex routines or behaviors. For instance,

swimming and cycling are composed of several in-

stances of walking, running, and sitting (among other

activities) characterized by a certain logical sequence

and duration. Moreover, integrating cardiac and res-

piratory signals into the medical decision and the clas-

siﬁcation system would be important for a better clas-

siﬁcation rate. Future expanded datasets should also

include more participants, especially patients in reha-

bilitation. We intend also to conduct an extensive set

of experiments to compare different machine learn-

ing and deep learning approaches, especially ensem-

ble methods, for human activity recognition using the

acceleration data from the Hexoskin.

REFERENCES

Attal, F., Mohammed, S., Dedabrishvili, M., Chamroukhi,

F., Oukhellou, L., and Amirat, Y. (2015). Physical Hu-

man Activity Recognition Using Wearable Sensors.

Sensors, 15(12):31314–31338.

Banerjee, T., Anantharam, P., Romine, W., Lawhorne, L.,

and Sheth, A. (2015). Evaluating a potential com-

mercial tool for healthcare application for people with

dementia. In the International Conference on Health

Informatics and Medical Systems.

Cherif, N., Mezghani, N., Gaudreault, N., Ouakrim, Y.,

Mouzoune, I., and Boulay, P. (2018a). Physiologi-

cal data validation of the hexoskin smart textile. In

BIODEVICES, pages 150–156.

Cherif, N., Ouakrim, Y., Benazza-Benyahia, A., and

Mezghani, N. (2018b). Physical activity classiﬁcation

using a smart textile. In 2018 IEEE Life Sciences Con-

ference (LSC), pages 175–178.

Cyr, C., Robert, S., and Sylvie, P. (2018). Mod

eles de

eadaptation cardiaque alternatifs utilisant les struc-

tures de gestion de cas et des maladies chroniques.

40:4–6.

Damasevicius, R., Vasiljevas, M., Salkevicius, J., and

zniak, M. (2016). Human activity recognition in

aal environments using random projections. Com-

putational and Mathematical Methods in Medicine,

2016:1–17.

Dunka, B., Emmanuel, E., and Oyerinde, Y. (2017). Hybrid

mobile application based on ionic framework tech-

nologies. International Journal of Recent Advances

in Multidisciplinary Research, 04:3121–3130.

Mampuya, W. (2012). Cardiac rehabilitation past, present

A Mobile Application for Physical Activity Recognition using Acceleration Data from Wearable Sensors for Cardiac Rehabilitation

631

and future: An overview. Cardiovascular Diagnosis

Therapeutics, 2:38.

Medina, J., Olmo, M., Aguilera, M., and Espinilla, M.

(2017). Real-time monitoring in home-based car-

diac rehabilitation using wrist-worn heart rate devices.

Sensors, 17.

Paul, S. S., Tiedemann, A., Hassett, L. M., Ramsay, E.,

Kirkham, C., Chagpar, S., and Sherrington, C. (2015).

Validity of the ﬁtbit activity tracker for measuring

steps in community-dwelling older adults. BMJ Open

Sport & Exercise Medicine, 1(1).

Thomas, R., Beatty, A., Beckie, T., Brewer, L., Brown,

T., Forman, D., Franklin, B., Keteyian, S., Kitzman,

D., Regensteiner, J., Sanderson, B., and Whooley, M.

(2019). Home-based cardiac rehabilitation: A scien-

tiﬁc statement from the american association of car-

diovascular and pulmonary rehabilitation, the ameri-

can heart association, and the american college of car-

diology. Journal of Cardiopulmonary Rehabilitation

and Prevention, 39:1.

Varnﬁeld, M., Karunanithi, M., S

arel

a, A., Garcia, E., Fair-

full, A., Oldenburg, B., and Walters, D. (2011). Up-

take of a technology-assisted home-care cardiac reha-

bilitation program. The Medical journal of Australia,

194:15–19.

Villar, R., Beltrame, T., and Hughson, R. (2015). Valida-

tion of the hexoskin wearable vest during lying, sit-

ting, standing and walking activities. Applied Physi-

ology, Nutrition, and Metabolism, 40:1019–24.

Voicu, R.-A., Dobre, C., Bajenaru, L., and Ciobanu, R.-

I. (2019). Human physical activity recognition using

smartphone sensors. Sensors, 19:458.

Webster, E., Sukaviriya, N., Chang, H.-Y., and Kozloski,

J. (2017). Predicting cognitive states from wearable

recordings of autonomic function. IBM Journal of Re-

search and Development, 61:2:1–2:11.

HEALTHINF 2020 - 13th International Conference on Health Informatics

632