Human Interface for a Neuroprothesis Remotely Control

Lucas M. Argentim

, Maria Claudia F. Castro

and Plinio A. Tomaz

Electrical Engineering Department, University Center FEI, Humberto de Alencar Castelo Branco,

Sao Bernardo do Campo, Brazil

Computer Science Department, University Center FEI, Humberto de Alencar Castelo Branco,

Sao Bernardo do Campo, Brazil

Keywords:

Electrical Stimulation, Electromyography, User Interface, Stroke, Hemiplegia.

Abstract:

Neuromuscular Electrical Stimulation (NMES) and Surface Electromyography (sEMG) have been widely ex-

plored by the scientiﬁc community for the rehabilitation of individuals with motor deﬁcits due to stroke. The

literature shows the beneﬁts of sEMG-activated NMES use in both motor rehabilitation and neural plasticity

stimulation. Currently, there is a strong tendency to expand the clinical environment, and the internet can be

used by healthcare professionals to do detailed follow-up and interact with their patients remotely. This work

presents a neuroprothesis activated by sEMG that allows conﬁguration and monitoring of usage parameters re-

motely. Two control platforms were developed for different user proﬁles; health professionals (Web Interface)

and neuroprosthesis users (Smartphone Application).

1 INTRODUCTION

Stroke consists of a neurological deﬁcit that occurs

when there is a lack of adequate blood ﬂow in a par-

ticular brain region, either by obstruction (ischemic,

of 85% of the cases) or by rupture (hemorrhagic) of

the vessels resulting in a cerebral infarction (Sacco

et al., 2013). As a consequence, spasticity as a change

in skeletal muscle control, caused by an imbalance of

signals from the Central Nervous System to the nerve

endings of muscles, and a muscular hypertonia are

veriﬁed. As an example, the ﬂexor pattern of the el-

bow, wrist and ﬁngers is a commonly observed con-

dition in these individuals, which makes it impossible

to perform ordinary everyday tasks that use the move-

ment of these joints independently, such as combing

the hair, brushing teeth, typing in keyboard of a com-

puter, etc. (Carmo et al., 2015).

Approximately 15 million people worldwide have

a stroke each year; even where advanced technology

and facilities are available, 60% of those who suffer

a stroke die or become dependent, causing immediate

and devastating changes both in their lives and those

closest to them. Although the incidence of stroke

is declining in many developed countries, the abso-

lute number of strokes continues to increase because

of the ageing population. By 2050, the proportion

of the world’s population over 60 years is estimated

to be 22% reaching 2 billion, 80% living in low-

and middle-income countries (Mackay and Mensah,

2004a,b; WHO, 2015). It is important that such care

is taken to control, prevent and manage these events.

The techniques for rehabilitation of individuals

with motor deﬁciency, due to Stroke, are widely ex-

plored by the scientiﬁc community. The Neuromus-

cular Electrical Stimulation (NMES) is a widely used

technique in rehabilitation of individuals with mo-

tor dysfunctions. Over the years, works such those

developed by Kralj et al. (1993), Hara (2008), Shin

et al. (2008), Lin and Yan (2011), Hu et al. (2012),

Meadmore et al. (2013) and Hara (2013) have demon-

strated the effectiveness of this technique for reduce

the time needed to re-establish patients’ condition

from a stroke.

Kralj et al. (1993) showed that, since the 1970s,

a NMES have been applied as a therapeutic means

of effective treatment to improve the range of wrist

and ﬁnger extension motions and to prevent the con-

tractures caused by ﬂexor spasticity. Hara (2008)

and Hara (2013) presented a home-based rehabilita-

tion program with power-assisted FES. However, the

system did not allowed conﬁguration and monitoring

of usage parameters remotely by health professionals.

Instead, patients received training for equipment con-

ﬁguration and electrode positioning and the exercise

protocol that they should accomplish. However, they

attained, using a multi-channel near-infrared spec-

troscopy (NIRS) study, a greater cerebral blood ﬂow

Argentim, L., Castro, M. and Tomaz, P.

Human Interface for a Neuroprothesis Remotely Control.

DOI: 10.5220/0006719002470253

In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 1: BIODEVICES, pages 247-253

ISBN: 978-989-758-277-6

247

during an EMG-controlled NMES therapy. Shin et al.

(2008) showed, in a random experiment, that the use

of an EMG-triggered NMES to the wrist extensor

for two sessions (30 min/session) a day, ﬁve times

per week for 10 weeks could results in a signiﬁ-

cant improvement of motor function and also in a re-

organization of the sensorimotor cortex. Lin and Yan

(2011) also applied NMES with sessions in the same

frequency, but for a period of only 3 weeks. They

observed not only signiﬁcant improvement of motor

functions, but also the maintenance of it for at least 6

months. Promising results were also achieved by Hu

et al. (2012) using an EMG-driven electromechani-

cal robot system integrated with NMES and by Mead-

more et al. (2013) using a multi-channel NMES sys-

tem in goal-orientated tasks.

Other studies conﬁrm that this practice not only

promotes functional recovery but also stimulates the

process of neurogenesis, that is, the sequence of

events leading to the formation of the nervous sys-

tem after traumatic events that may have damaged

it (Huang et al., 2015). The effects depends on the

severity of the impairments, the time since stroke, the

intensiveness of the NMES use and more than this the

chalenges for the Nervous System. However, there

is a trend towards the advantage of sEMG-triggered

NMES over cycling NMES, a better motor enhance-

ment due to task-oriented training, and patients re-

taining some degree of ﬁnger extension tend to shift

towards a focused activity in the ipsilesional corti-

cal site after NMES induced activity, whereas patients

who did not regain ﬁnger extension showed enhanced

involvement of the contralesional cortical site (Kem-

permann et al., 2000; Rushton, 2003; Schaechter,

2004; Shin et al., 2008; Quandt and Hummel, 2014).

In addition, a strong tendency to expand the clin-

ical setting for treatment of individuals may be noted

in recent years (Piron et al., 2004, 2009). The lit-

erature presents works such as Zhang et al. (2008)

and Buick et al. (2016) in which the Internet allows

healthcare professionals to make detailed follow-up

and interact with their patients remotely from their

homes.

Based on this context, this work presents a neu-

roprothesis activated by sEMG that allows conﬁgu-

ration and monitoring of usage parameters remotely.

Two control platforms were developed for different

user proﬁles; health professionals (Web Interface) and

neuroprosthesis users (Smartphone Application).

2 MATERIALS AND METHODS

2.1 Neurostim

Neurostim is a custom made neuroprothesis, based on

the application of Neuromuscular Electrical Stimula-

tion (NMES) with surface electrodes activated by sur-

face myoelectric signal (sEMG). Its use aims for the

rehabilitation of the upper limbs (hands and wrists)

of patients with hemiplegia due to stroke. Both

NMES and sEMG technologies are widely known,

widespread and used in the clinical area.

The differentials correspond to the proposal of

daily use in the home environment, and for this pur-

pose two control platforms was developed; one for the

user of the neuroprosthesis and another platform for

remote monitoring, allowing the health professional

to communicate with the device and to conﬁgure the

parameters of use, physiotherapy programs, and ﬂow

monitoring.

2.2 Interface for the Neuroprosthesis

User

The user interface, in the form of a Android smart-

phone application was developed using Android Stu-

dio 2.3.3 is meant to be installed in Android versions

4.0 and superior. The application has the primary pur-

pose of conﬁguring and controlling the neuroprosthe-

sis with the necessary instructions so that, with the use

of electric current, the extension of wrist and ﬁngers

is promoted when the intent of movement by the user

is captured via sEMG.

As a secondary objective, the application should

be able to become a communication channel between

the user and the healthcare professional responsible

for the rehabilitation program so that, adjustments of

the parameters of use and information sending about

the usage can be done remotely through the internet.

Users who will be using the application have been

victims of a stroke. As a consequence, the motor

functions of one side of his body were affected, caus-

ing them to partially lose the movements of the wrist

and ﬁngers. In the smartphone applications, users

must be able to:

• Use the neuroprosthesis simply and comfortably

• Use a smartphone application that has a simple

interface and allows navigation between features

with only one hand, which is possibly the non-

dominant hand.

Functionalities and tasks to be performed in the

application include:

BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices

248

• Log in to a proﬁle that saves personal information

about your device’s usage Screen Layout

• Pair the neuroprosthesis with the application to be

able to communicate and exchange data

• Consult historical information on past uses of the

neuroprosthesis

• Choose mode of operation: rehabilitation pro-

gram or functional mode

• Setting the parameters of Intensity of stimulation

of neuroprosthesis

• Conﬁgure sEMG threshold parameters of neuro-

prosthesis activation

• Start the operation with exercises or daily activi-

ties

• Reconﬁgure the parameters, if necessary, during

use

• Send the data collected by the neuroprosthesis in

an automatic and transparent way to the health-

care professional responsible for their rehabilita-

tion program

• Receive updates of usage parameters from the

neuroprosthesis in an automatic and transparent

way, to be able to adapt the usage of the device ac-

cording to its evolutionary framework during the

rehabilitation program

All data must be stored locally, specially if the

smartphone does not have an internet connection

at the end of procedure and synchronization can

not happen at that time (will be trigger later on

whenever a stable connection is available).

2.3 Interface for the Health Professional

The interface for the health professional consists of

a web platform programmed in HTML5, PHP and

JavaScript to analyze and interact remotely with the

neuroprosthesis. The web interface should be able to

become a two-way communication instrument with

the neuroprotesis, promoting the necessary informa-

tion to conﬁgure its use, as well as an instrument of

interaction between the user and the health profes-

sional and data analysis. By synchronizing the cap-

tured neuroprosthesis data on the respective smart-

phones, the healthcare professional will get the de-

vice’s usage history information so progression anal-

ysis from a clinical standpoint can be done and re-

motely perform the necessary adjustments, as all the

data will be stored in a MySQL database.

The web interface must become a bi-directional

communication instrument with the neuroprosthesis,

promoting, through data packets, the necessary con-

ﬁgurations so that, with the use of electric current, the

extension of wrists and ﬁngers can be promoted when

the intention of movement by the user is captured via

sEMG. The interface should also be able to become an

instrument of data analysis and interaction between

the user and the healthcare professional responsible

for the rehabilitation program so that, adjustments of

the parameters of use and the sending of information

about the use can be done remotely.

The interface should be able to conﬁgure the neu-

roprosthesis to the most diverse types of user char-

acteristics. For this, a database will store the pa-

rameters referring to each patient and their respec-

tive neuroprosthesis. When patients synchronize in-

formation with their smartphones, the healthcare pro-

fessional will get historical devices usage information

so that, analyzes of the clinical picture progression

can be made and the necessary adjustments are made

remotely.

Users who will use the web interface are health-

care professionals responsible for the rehabilitation

program for patients who have been victims of a

stroke. With the Web Interface, they must be able to:

• Log into a proﬁle that has saved your personal in-

formation using the interface

With the users of the neuroprosthesis under

their responsibility registered

• Be able to register new users under your responsi-

bility

Name

Last name

Age

E-mail

Social Security Number

General address data

Contact

Patient ID

Rankin Scale score

Diagnosis

Responsible Health Professional ID

• Edit the existing user registry under their respon-

sibility

• Conﬁgure the neuroprosthesis remotely

Rise Time

Down time

Plateau Time

Activation threshold of sEMG

Intensity of the stimulus

Execution time or number of repetitions

Frequency of stimulus

Human Interface for a Neuroprothesis Remotely Control

249

• Analyze the usage data history sent from the

smartphone to the cloud database, shared between

both applications in a web interface

Dates of use

Time of use

Repetition count

Sampling of sEMG sensor readings

Time stamp of user parameter changes

• Set remotely usage schedule for conﬁgurable pa-

rameters

Relevant information can be printed and exported

from the web interface to further analysis on com-

plementary software, whenever is needed.

3 RESULTS

3.1 Neurostim

Neurostim uses an ATmega32u4 (Atmel Corporation)

microcontroller to produce symmetrical biphasic con-

stant current pulses with ﬁxed frequency at 20 Hz,

two pulse width options 300 µs and 600 µs, and ad-

justable amplitude from 0 to 40 mA (resistive load of

1k Ω). These parameters have already been studied

by the scientiﬁc community and are considered to be

the most adequate for obtaining muscle contraction

patterns (Quandt and Hummel, 2014).

The stimulator is small in size and attached to an

armband where there are also the electrodes needed

for stimulation and sEMG, with the possibility of re-

location of position to allow customization and per-

sonalization according to the individualized motor re-

sponse. The stimulation electrodes are conductive

rubber.

The triggering of the stimulation depends on an

initial effort of the user captured by a sEMG device

called MyoWare (Advander Technologies LCC), de-

ﬁned through the conﬁguration of a threshold of the

myoelectric signal. Once the threshold is reached,

the stimulation is triggered, following an increase in

the amplitude of the graded stimulus to the conﬁg-

ured maximum (ramp modulation), remaining for the

pre-established time and then ceasing, also with the

gradual decrease of the stimulus. A new cycle will

only be started if the threshold of the sEMG signal is

again reached.

3.2 Interface for the Neuroprosthesis

User

Upon start, the application synchronizes all data avail-

able at the server side for the logged in user. All in-

formation regarding historical data and agenda of us-

age will be available for consult, as shown in Figure

1. Depending on the setups made by the health pro-

fessional, the user will have the possibility to choose

between modes of operation, maximum intensity, etc.

Figure 1: Application screen showing historical usage data.

The communication with the smartphone applica-

tion is made through Bluetooth using the nRF51822

(Nordic Semiconductor) module, where the device

can receive and send information in real time accord-

ing to the setups made on the app (change of threshold

and intensity by the user), as shown in Figure 2.

3.3 Interface for the Health Professional

User can login with its credentials at the login page,

having access to the list of all the patients under its

responsibility as shown in Figure 3. Whenever is

needed, new patients can be added.

For each patient, the health professional can setup

an agenda containing the usage parameters accord-

ing to the rehabilitation program strategy, as shown

in Figure 4.

BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices

250

Figure 2: Application screen showing the control functions

and readings from the neuroprothesis.

Also, the health professional can visualize data

from all the procedures executed by each patient. In-

formation regarding sEMG and usage parameter will

be shown in a graph (Figure 5), along with a summary

with relevant information about the respective proce-

dure (Figure 6).

Relevant information can be printed and exported

from the web interface to further analysis on comple-

mentary software, whenever is needed.

3.4 Interfaces and System Evaluation

While awaiting the approval of the ethics commit-

tee to perform the usability and clinical performance

evaluation of the system, a pilot test was performed to

verify if users are able to use the interfaces and if the

data storage and communication are well performed.

Results were as expected, where an user was able to

successfully control the neuropothesis, as well as re-

trieve and send data to the server on the cloud contain-

ing usage setups and procedures informations auto-

matically. Additionaly, another user was able to suc-

cessfully setup remotely the neuropothesis, as well as

retrieve data containing usage setups and procedures

information automatically so further analysis and ad-

justments could be done without having the patient

physically present to do so.

4 CONCLUSION

The neuroprosthesis should be versatile enough for

the user to feel comfortable using it as a physiother-

apeutic element either in the rehabilitation program,

or in a functional way to assist in daily tasks. The

effectiveness of user interaction is a key to device ac-

ceptance.

The smartphone application was developed in a

way that simple interfaces allow the patient to per-

form all expected activities with ease, preventing er-

rors during the stimulation procedures and avoiding

non expected behaviors from the electric stimulation.

The web interface was also developed so the

health professionals can have signiﬁcant information

about usage and perform analysis to execute eventual

adjustments to maximize effectivity to patients treat-

ment.

As the next steps usability and clinical evaluations

trials will be performed.

REFERENCES

Buick, A. R., Kowalczewski, J., Carson, R. G., and Proc-

hazka, A. (2016). Tele-supervised fes-assisted exer-

cise for hemiplegic upper limb. IEEE Transactions

on Neural Systems and Rehabilitation Engineering,

24(1):79–87.

Carmo, J. F., Morelato, R. L., Pinto, H. P., and Oliveira,

E. R. A. (2015). Disability after stroke: a systematic

review. Fisioterapia em Movimento, 28(2):407–418.

Hara, Y. (2008). Neurorehabilitation with new functional

electrical stimulation for hemiparetic upper extremity

in stroke patients. Journal of Nippon Medical School,

75(1):4–14.

Hara, Y. (2013). Rehabilitation with functional electrical

stimulation in stroke patients. International Journal

of Physical Medicine & Rehabilitation, 1:147.

Hu, X., Tong, K., Li, R., Xue, J., Ho, S., and Chen, P.

(2012). The effects of electromechanical wrist robot

assistive system with neuromuscular electrical stimu-

lation for stroke rehabilitation. Journal of Electromyo-

graphy and Kinesiology, 22:431–439.

Huang, Y., YeE Li, J. C., Zhou, H., and Tan, S. (2015). Elec-

trical stimulation elicits neural stem cells activation:

new perspectives in cns repair. Frontiers in human

neuroscience, 9.

Kempermann, G., van Praag, H., and Gage, F. H. (2000).

Activity-dependent regulation of neuronal plasticity

and self repair. Progress in brain research, 127:35–

48.

Kralj, A., Amovi

c, R., and Stani, U. (1993). Enhancement

of hemiplegic patient rehabilitation by means of func-

tional electrical stimulation. Prosthetics and Orthotics

International, 17(2):107–114.

Human Interface for a Neuroprothesis Remotely Control

251

Figure 3: Web application screen showing list of Health Professionals patients.

Figure 4: Web application screen showing usage parameters agenda for a patient.

Figure 5: Web application screen showing sEMG and usage information of a procedure.

Figure 6: Web application screen showing summary information of a procedure.

BIODEVICES 2018 - 11th International Conference on Biomedical Electronics and Devices

252

Lin, Z. and Yan, T. (2011). Long-term effectiveness of neu-

romuscular electrical stimulation for promoting motor

recovery of the upper extremity after stroke. Journal

of rehabilitation medicine, 43(6):506–510.

Mackay, J. and Mensah, G. (2004a). Section 15: Global

burden of stroke. (Accessed on Oct/2017).

Mackay, J. and Mensah, G. (2004b). Section 16: Deaths

from stroke. (Accessed on Oct/2017).

Meadmore, K., Exell, T., Freeman, C., Kutlu, M., Rogers,

E., Hughes, A.-M., Hallewell, E., and Burridge, J.

(2013). Electrical stimulation and iterative learning

control for functional recovery in the upper limb post-

stroke. In 2013 IEEE International Conference on

Rehabilitation Robotics, Seattle, Washington USA.

IEEE.

Piron, L., Tonin, P., Trivello, E., Battistin, L., and Dam,

M. (2004). Motor tele-rehabilitation in post-stroke

patients. Medical informatics and the Internet in

medicine, 29(2):119–125.

Piron, L., Turolla, A., Agostini, M., Zucconi, C., Cortese,

F., Zampolini, M., Zannini, M., Dam, M., Ventura,

L., Battauz, M., et al. (2009). Exercises for paretic

upper limb after stroke: a combined virtual-reality

and telemedicine approach. Journal of Rehabilitation

Medicine, 41(12):1016–1020.

Quandt, F. and Hummel, F. C. (2014). The inﬂuence of

functional electrical stimulation on hand motor re-

covery in stroke patients: a review. Experimental &

Translational Stroke Medicine, 6:9.

Rushton, D. (2003). Functional electrical stimulation and

rehabilitationan hypothesis. Medical engineering &

physics, 25(1):75–78.

Sacco, R. L., Kasner, S. E., Broderick, J. P., Caplan, L. R.,

Connors, J. B., Culebras, A., Elkind, M. S., George,

M. G., Hamdan, A. D., Higashida, R. T., Hoh, B. L.,

Janis, L. S., Kase, C. S., Kleindorfer, D. O., Lee, J.-

M., Moseley, M. E., Peterson, E. D., Turan, T. N.,

Valderrama, A. L., and Vinters, H. V. (2013). An up-

dated deﬁnition of stroke for the 21st century. Stroke,

44(7):2064–2089.

Schaechter, J. D. (2004). Motor rehabilitation and brain

plasticity after hemiparetic stroke. Progress in Neuro-

biology, 73:61–72.

Shin, H. K., Cho, S. H., Jeon, H.-s., Lee, Y.-H., Song,

J. C., Jang, S. H., Lee, C.-H., and Kwon, Y. H.

(2008). Cortical effect and functional recovery by

the electromyography-triggered neuromuscular stim-

ulation in chronic stroke patients. Neuroscience let-

ters, 442(3):174–179.

WHO (2015). Ageing and health. (Accessed on Oct/2017).

Zhang, S., Hu, H., and Zhou, H. (2008). An interactive

internet-based system for tracking upper limb motion

in home-based rehabilitation. Medical & Biological

Eengineering & Computing, 46(3):241–249.

Human Interface for a Neuroprothesis Remotely Control

253