Four Gesture Recognization of a Robotic Hand using EMG
Maryam Arshad
1
, Nimra Iftikhar
1
and Noman Naseer
2
1
Department of Electrical and Computer Engineering, Air University, E9, Islamabad, Pakistan
2
Department of Mechatronics Engineering and BioMedical, Air University, Islamabad, Pakistan
Keywords: EMG, Indirect BCI, Myo Armband.
Abstract: Electromyography (EMG) measures muscle response when nerves are stimulated. The objective of this
research paper is to present a work on control of a robotic hand using Electromyography. CAD model was
selected using various open sources. The structure is printed with poly laic acid (PLA) material with the help
of a 3D printer. EMG signals were acquired by wearing the eight channel Myo Armband, placed on the
forearm muscles of 10 subjects. Then, these signals were filtered to remove noise. Different features are
applied on noise free acquired signal and KNN is used for classification. From the KNN classifier, we
achieved 98.9% accuracy.
Gestures exhibited were Victory, Thumbs Up, Open Hand and Grasp. The
classified signals are used to control the robotic hand.
1 INTRODUCTION
A physical disability is a condition that affects a
person's mobility or physical capacity. Therefore we
uses prosthesis which is an artificial device,
developed to replace the function of a lost limb.
Prosthesis are classified as exoskeleton and
endoskeleton. Exoskeleton prosthesis gain its
structural strength from outer laminated strength
whereas, endoskeleton prosthesis gain its structural
integrity from the inner endoskeleton.
There are two major types of exoskeleton: Upper-
extremity prostheses and Lower-extremity
prostheses. Upper-extremity prostheses further have
following types transradial amputation, trans-
humeral amputation, wrist dis-articulation, elbow dis-
articulation and shoulder dis-articulation. Whereas,
lower-extremity prosthesis includes prostheses for
transfemoral amputation, knee disarticulation, hip
disarticulation, ankle disarticulation, transtibial
amputation and partial foot amputation.
Depending on the use of external power they are
divided into Active and Passive techniques (Windrich
et al, 2016). An active prosthesis powered devices
externally and consist of sensors in contact with the
skin, which then pick up the signals from the arm and
sequentially operate the actuators, which in order
controls the movement(Windrich et al, 2016).
Different techniques are used to developed control
for brain controlled interface (BCI) (Mattia, 2016) for
example Surface Electromyography (Anil and
Sreeletha, 2019), Electroencephalography (EEG),
Functional Magnetic Resonance Imaging (FMRI),
(Bright et al, 2016), Force myography (FMG) (Cho et
al, 2016) and Targeted Muscle Reinnervation (TMR)
(Cheesborough et al, 2015). From which sEMG,
EEG, FMG and FMRI are non-invasive techniques
whereas, TMR are invasive techniques.
Electromyography (EMG) measures muscle
response when nerves are stimulated (Anis, 2019).
Electromyography (EMG) is a symptomatic program
used to survey the wellbeing of muscles and the nerve
cells that control them. These nerve cells are called
engine neurons ( Shi, 2018). A brief description of the
different EMG strategies signal separation and
comparison of different strategies by analyzing the
EMG signals, in relation to their execution was
proposed (Raez, 2016).
EMG is performed by an instrument called an
electromyograph, which recognizes electrical signals
generated by muscles. Electromyography (EMG) can
be assessed utilizing conductive parts or anodes on
the outside of the skin, or it tends to be evaluated
utilizing an obtrusive technique (coordinating the
sensor into the muscle) (M. Ali, 2020). Surface EMG
is the most common techniques (Del Vecchio et al,
2017).
EMG is a non-invasive method of indirect BCI
which gradually grows in number of applications,
such as EEG (Mattia, 2016) (Gandhi, 2010).
80
Arshad, M., Iftikhar, N. and Naseer, N.
Four Gesture Recognization of a Robotic Hand using EMG.
DOI: 10.5220/0010289100800087
In Proceedings of the International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical (HIMBEP 2020), pages 80-87
ISBN: 978-989-758-500-5
Copyright
c
2021 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
Including biomedical, a prosthesis, a combination of
a human machine. In any case, there is noise EMG
symptoms are important barriers to overcoming them
achieves the best performance of any based EMG
application known as Human Machine Identifier
(HMI)( Zimenko, 2013).
A robotic hand is an electro-mechanical system. It
is composed of different parts. The main parts are the
electrical components and mechanical structures that
allow movement. In many such configurations, the
kinematics of the robot are the same as those of a
human hand, and the joints of each finger can be
controlled independently (Naseer, 2018).
Virtual prototyping technology is used to create
designs for 3D printed Myo robotic hands. CAD
models were selected using a variety of open source.
It can be easily accessed on the Internet. This hand
uses a small motor to control the movement of the
fingers of the hand. Use a 3D printer to print small
parts of the hand, and finally assemble all the parts
into a robot. The design is tightly based on the
function of the tendon, which is how the hand works.
By using servo motors to help open and close the
clamps, threads are used to manipulate the structure.
Print structures with polyacrylic material (PLA) using
a 3D printer.
Our robot-based manipulator is controlled by the
movement of the hand. Its working principle is EMG
sensor based. Sensor record hand movements and
then transmit the information to the encoder, which
prepares the encoder for receiver-side transmission.
This information is received via Bluetooth and then
transmitted to the Arduino. The micro-controller
makes various decisions based on the information
received. These decisions are passed to the motor
driver, which triggers the motor in different
configurations, causing the robotic hand to move the
Myo Robotic Hand to a certain direction. Depending
on the application, the robotic hand can be designed
to perform all required tasks such as welding,
gripping, rotating, etc.
The task has numerous utilizations in the
territories of wellbeing, apply autonomy and bio-
robots, just as an individual deadened with a couple
of hands. This structure can be utilized to distinguish
high radiation objects. This plan can be utilized for
prostheses with a high number of degrees of
opportunity. With some modifications, it can be used
in heavy industry to collect and position various
objects with real-time signals. The objects that can be
treated can be, for example, toxic chemicals,
radioactive materials. It can also be used on assembly
lines in the automotive industry
This research has several applications in different
field such as Robotics and Bio-medical Industry.
Such as, to help people with wrist disarticulation so
that they become independent and return to normal
life.
2 METHODOLOGY
A methodology was evolved in which research is
divided into different part or stages.
Fig .1 shows the experimental setup of this
research. CAD model was selected using various
open sources. Then the structure is printed with poly
Figure 1: Experimental setup.
Four Gesture Recognization of a Robotic Hand using EMG
81
laic acid (PLA) material with the help of a 3D
printer. EMG signals were acquired by wearing the
eight channel Myo Armband, placed on the forearm
muscles of 10 subjects. Different features are applied
on noise free acquired signal and KNN is used for
classification. From the KNN classifier, we achieved
98.9% accuracy.
2.1 3D Printing of Robotic Hand
Rather than structuring a CAD model which isn’t our
project goal. We looked through CAD models of
hand on different open sources. After all the
examination and study we finished a streamlined plan
which comprise of four moveable fingers, a
moveable and contradicting thumb, improved
gripping design, simple and reliable.
The following point must be considered while
choosing the cad model:
A cover over all the electrical components to
protect them from environment.
Close resemblance with natural human hand.
Entire weigh of hand including electronics
not more than 600g
Fingers actuation using threads attached
through motors.
Fig. 2 shows the 3d print/design of the robotic hand.
Figure 2: 3D hand design
For 3D printing we used Poly Lactic Acid which
is a thermoplastic and is derived from sugar. It is
carbon free material. PLA when heated above 180° C
become mold-able and upon cooling hard. It is used
with 3D Printer because it is reusable, light, has
strength than produces products with high quality.
Another material that can be used is Acrylonitrile-
Butadiene-Styrene (ABS). It is made from monomers
Acrylonitrile, 1,3-Butadiene and Styrene. It is strong
and durable but it need a hot printing bed above 200C.
It also produces poisonous gases.
After receiving 3d printed robotic hand, it was
assembled. Electronic circuits, controller and motors
are placed inside the
3D Printed Myoelectric Hand. 5 Servo Motors
and fishing line are used to actuate the fingers. Device
is attached to the patients disabled with upper limb.
As our project is robotic hand therefore no need for
exact dimension.
The components/hardware used for regulating the
robotic hand are as follow: Actuator, battery,
microcontroller and wires. Actuator is a device due to
which a machine or any other device can operate.
They are characterize on the basis of their rotation and
the power they use. The actuator we are using in our
project is servo motors. Servo Motor is DC engine
with a mechanism known as feed-back mechanism.
Figure 3: Servo Motor Flow Diagram
This motor run on electricity with the help of a
battery and spin at high RPM. Servo Motor (MG995).
Fig.3 shows Servo Motor Flow Diagram.The reason
for selecting MG995 is that they are providing us the
required Torque as well as they are small in size and
HIMBEP 2020 - International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical
82
inexpensive in comparison to other DC Motor
available in the market.
RPM stands for rotation per minute but put out
very low torque (a rotational effect used to work).
Servo motors have different types.
Their main feature is to control the position of
shaft which is inside it. Servo DC Motor are of small
size, relatively low in price and provide torque from
9.4kg/cm to about 11kg/cm. They are widely used in
robotic applications. Servo Motor attract current
relative to the mechanical burden applied and has
capacity to turn to explicit precise position, which
depend on the model utilized. Fig.4 shows servo
motor.
Figure 4: Servo motor
In this project 4 buck converters (LM2596) are
used. One with each servo motor. The LM2596 is a
series of buck converter basically regulator which
provide us with the functions of a step down (buck)
controller. They are able to drive 3A load easily that
is enough for the project. Some other specification of
this regulator is given below:
Input Voltage (min-max): 4.5V-40V
Input Voltage (min-max): 3.3V-37V
Arduino is an open-source and open-authorized
programming (IDE) which make it easy to form the
code and to move it on to the Arduino Board.
Utilizing Arduino MATLAB library, Arduino board
goes about as the fundamental correspondence load
up between the constant development of client’s
finger and the mechanical hand giving every actuator
the ideal PWM. An Arduino as a microcontroller was
utilized to process the information gained from the
sensor of Myo Armband. Arduino sheets depend on
equipment and programming. Arduino Uno is used.
After considering all the requirements, Lithium-
ion polymer battery (LiPO) Battery 5000 mAh is
used. Table 1 shows the battery calculation, on the
basis of which, battery is selected.
Table 1: Battery calculation
I
n
d
e
x
Battery Calculation
Component
Quanti
ty
Curren
t
Volta
ge
Power Total
Power
1
Servo
Mg995
4 0.45A*
6V 2.7W 2.7x4=10.8
W
2
Micro
Controller
1 0.05A
7V 0.35W 0.35x1=0.35
W
3
PSU
Circuitry
1 1A
10V 10W 10x1=10W
4
Miscellaneo
us
1 0.5A
5V 2.5W 2.5x1=2.5W
5
Servo
Sg90
1 0.36A*
5V 1.8W 1.8x1=1.8W
Total Power = 10.8+0.35+10+2.5+1.8
= 31.65 W (Power)
Assuming operation for 1hr = 31.65Wh (Energy)
Battery Capacity = 2851.35mAh (Wh x1000/V)
Battery Voltage = 11.1 V
Advantages of using lipo battery is as follow
High Discharge Current
Light Weight
More Charge Cycle
11.1 V
5000mAh
Step by step procedure to assemble the hand:
Step1: All fingers except thumb are assembled
from fingertip, joint and knuckle. String are used to
joint these parts. Thumb is assembled from thumb
joint, thumb tip and joint.
Step 2: Four Servo motors are placed in the
Forearm region and tightened with a screw provided
with the servo motor. Threads are connected to the
servo on the horns. The infant finger and the ring
finger are connected to a similar servo, as they
provide the same functionality.
Four Gesture Recognization of a Robotic Hand using EMG
83
Step 3: Cut ten 20 inches fishing line. Two
fishing line per finger are used. Crimp them by
feeding the fishing line from one side. Feed the
fishing line from the finger tips till the fishing line
exists from the palm.
Step 4: Now the fishing line are attached to the
corresponding servo motor. The fishing line should
be tensioned. To check the fishing line is tension’s
move. The motor counter clockwise with hand and
the finger will move inward. By rotating it clockwise
the finger should open.
Step 5: Now wear Myo Armband and then
connect Arduino with motors and place the forearm
cover on forearm body.
EMG sensor is placed on the forearm muscle
using Myo Armband. EMG is powered by a separate
battery providing 18V. Robotic hand was also tested
by using arduino.
As we are done with the assembling of Myo
Robotic hand, after that we integrated Myo Armband
with the arduino and perform the following four
gestures of victory, thumbsup, open and close using
IDE (open licensed software). Figure 5 shows the
gestures performed by a robotic hand.
Figure 5: Gestures performed by robotic hand
2.2 Data Acquisition
For signal acquisition Myo Armband was utilized. It
is a light weighted armband with a heap of just 93g
and expandable between 7.5–13inches. It contains 8
Medical Grade Stainless Steel EMG sensors,
fundamentally delicate, 9 turn Internal Measurement
Unit (IMU), and there is whirligig, accelerometer and
magnetometer all of 3-center point. It involves ARM
Cortex M4 Processor.
It interacts with computer by methods of
Bluetooth Smart Wireless Technology. Fig.6 shows
Myo Armband sensor.
Figure 6: Myo Armband
A subject wore it on the forearm that continuously
read the muscle data and sent it via the in-built
Bluetooth to the laptop present right next to the
subject in form of a vector. Figure 7 shows the
position of sensor.
Figure 7: Location of electrodes on muscles (frontal view.)
The laptop’s Bluetooth received the incoming
data and passed it to the MATLAB.
The experimental setup decided for this project is
as follows:
• Number of classes (actions) = 4
• Number of subjects = 10(4 male and 6 female)
• Number of male subjects = 4
HIMBEP 2020 - International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical
84
• Number of female subjects = 6
• Age group = 20-25 years
• Number of activities per subject per trial= 10
• Total time for each trial = 40s
• All subjects are healthy.
2.3 Methods of Detecting Waves
Myo Armband is an advance band that read the
electrical activity of muscles and movement of arm
and signals are filtered by the Myo Armband on
sampling frequency of 200Hz, in order to detect wave
two methods are as follow:
• Detecting wave by Peak.
Detecting wave by Peak is one of those best
solution in which the required patch of the signal
can be separated from the whole signal and can be
utilize in the work. This method used the data of
the indices of peaks by separating then-point to
the left of each index and same n-point to the right
of each index.
• Detecting wave by Slope.
Wave detection can also be found by a method in
which the required signal can be separated
through the whole signal. In this technique the
separation of the point begins when slope starts to
increase till the time when it start decreasing.
2.3.1 Features Extraction
Signal provided by using Myo Armband is a filtered
signal, Savitzky Golay filter was applied to smoothen
the signal (Christov, 2018).
The features were extracted from filtered EMG
signal. This features extraction help to reduce the data
in beneficial way.
In order to get higher end result of classification,
features vector must be cautiously selected. We
extracted 4 features for each electrodes making a total
of 32 features (4 features * 8 electrodes). The best
results were found to be Standard Deviation (STD),
Waveform Length (WL), Root Mean Square (RMS)
and Mean Absolute Value (MAV) (Hong, Khan and
Hong, 2018). Fig.8 shows unfiltered EMG signal
whereas Fig.9 shows filter EMG signal.
Figure 8: Un-filtered EMG Signal
Figure 9: Filtered EMG Signal
2.4 Classification
Classification is the main part of the project and it is
used to classify the signal as it is known as pattern-
recognition in simple language.
We skilled three classifiers on extracted features,
the classifiers have been Linear Discriminant
Analysis (LDA), K-Nearest Neighbors (KNN),
Support Vector Machine (SVM). The classifiers such
as LDA, SVM and KNN were trained separately on
each of the above feature. Maximum accuracy was
achieved using Root Mean Square as a feature on
KNN.
2.5 Interfacing
Next step of this project is to build the interface. At
first Myo-Armband and laptop was interface,
followed by laptop to robotic hand. Myo Armband can
be also interfaced with any other devices through
Four Gesture Recognization of a Robotic Hand using EMG
85
Myo-script. To write a script for Myo-armband and to
control any device, myo-script was used. Myo-scripts
coding language is lua, which is commonly used for
coding a lot of games or software. Notepad ++ for lua
is used because Notepad++ supports different
programming languages. It is also a source code for
free.
Myo Armband was interfaced with the laptop,
next step was to again interface the laptop with the
robotic hand (L. F. Sanchez, 2017)
3 EXPERIMENTATION
Using the essential examination approaches for
grouping we utilized three classifier which are as
follow:
• Linear Discriminant Analysis (LDA)
• K-Nearest Neighbors (KNN)
• Support Vector Machine (SVM)
One of the non-parametric method of classification
is K-Nearest Neighbors (KNN) works by using the
fact of ‘feature similarity’. The data will be assigned
in that category whose feature are more likely similar
with the incoming data points (Altın and Er, 2016).
98.9% accuracy was achieved by using KNN.
Another technique for arrangement Support
Vector Machine (SVM) which is a watch over
machine calculation by placing every information
thing in a form of a points in n-dimensional space, n
is known as numbers of features you have. After that
a hyperplane was form between the data which
actually
separates the classes and the new data is
divided into classes on the basis of side gap (Alkan
and Günay, 2012). With the each feature value being
the value of a particular coordinate, giving us the
exactness of 94.7 Percent the element of Mean
Absolute value (MAV). Fig 9 presents the Percentage
accuracy versus features on three different classifiers.
The basic research technique is LDA. It works or
identify different classes by calculating the
probability using the Bayes Theorem (Alam and
Arefin, 2018). 94.5% accuracy was achieved by using
LDA from the feature of Mean Absolute value
(MAV).
4 CONCLUSIONS
The hardware including the mechanical and electrical
parts was completed within the designated timeline.
Final application of LDA, SVM and KNN algorithms
with maximum accuracy are completed.
A classifier model of LDA, SVM and KNN was
trained using 400 movements (100 for each class). As
the resulting data points after feature extraction was
already well separated, the accuracy achieved was
(accuracy of all classifier)
Although final accuracy will be calculated by
taking account some more observation. The classifier
was trained using 4 feature of data
• RMS (accuracy)
• WL (accuracy)
•STD (accuracy)
•Mav (accuracy)
Figure 10: Percentage accuracy versus features on three different classifiers
HIMBEP 2020 - International Conference on Health Informatics, Medical, Biological Engineering, and Pharmaceutical
86
Real time classification was checked by performing
the gesture on real time and seeing if the predicted
response is same as the performing one. Testing of the
entire system was done step by step.
This project was started with an aim help the
deprived and disabled people with upper limb
disability. We applied our engineering knowledge
and developed a Myo-robotic hand for this noble
cause. The work done so far in Pakistan for the
rehabilitation cause is not such as should have been.
This motivated us to work on Myo-robotic hand.
REFERENCES
Windrich, M. et al. (2016) ‘Active lower limb prosthetics:
A systematic review of design issues and solutions’,
BioMedical Engineering Online. BioMed Central,
15(3), pp. 5–19. doi: 10.1186/s12938-016- 0284-9.
D. Mattia, L. Astolfi, J. Toppi, “Interfacing brain and
computer in neuro-rehabilitation,” 2016 4th
International Winter Conference on Brain-Computer
Interface (BCI) 22-24 Feb. 2016.
Anil, N. and Sreeletha, S. H. (2019) ‘EMG Based Gesture
Recognition Using Machine Learning’, Proceedings of
the 2nd International Conference on Intelligent
Computing and Control Systems, ICICCS 2018. IEEE,
(Iciccs), pp. 1560–1564. doi:
10.1109/ICCONS.2018.8662987.
Del Vecchio, A. et al. (2017) ‘Associations between motor
unit action potential parameters and surface EMG
features’, Journal of Applied Physiology, 123(4), pp.
835–843. doi: 10.1152/japplphysiol.00482.2017.
Bright, D. et al. (2016) ‘EEG-based brain controlled
prosthetic arm’, Conference on Advances in Signal
Processing, CASP 2016, pp. 479–483. doi:
10.1109/CASP.2016.7746219.
L. F. Sanchez, H. Abaunza, and P. Castillo, “Safe
navigation control for a quadcopter using user’s arm
commands,” International Conference on Unmanned
Aircraft Systems (ICUAS) June 13-16, 2017, Miami,
FL, USA, 2017
Cho, E. et al. (2016) ‘Force myography to control robotic
upper extremity prostheses: A feasibility study’,
Frontiers in Bioengineering and Biotechnology,
4(MAR), pp. 1–12. doi: 10.3389/fbioe.2016.00018.
Cheesborough, J. E. et al. (2015) ‘Targeted muscle
reinnervation and advanced prosthetic arms’, Seminars
in Plastic Surgery, 29(1), pp. 62–72. doi: 10.1055/s-
0035-1544166.
M. B. I. Raez, M. S. Hussain and F. M. Yasin, “Techniques
of EMG signal analysis: detection, processing,
classification and applications,” Biological procedures
Online, 2016.
K. Zimenko, A. Margun, and A. Kremlev EMG, “Real-
Time Classification for Robotics and HMI,” 18th
International Conference on Methods & Models in
Automation & Robotics (MMAR) 2013.
Choksawatdikorn, shutterstock, human hand muscles of
education. Large.5168x3448 pixels and
43.8x29.2cm.300DPI.JPEG
Christov, I., Raikova, R. and Angelova, S. (2018)
‘Separation of electrocardiographic from
electromyographic signals using dynamic filtration’,
Medical Engineering and Physics. Elsevier Ltd, 57, pp.
1–10. doi: 10.1016/j.medengphy.2018.04.007.
Hong, K. S., Khan, M. J. and Hong, M. J. (2018) ‘Feature
Extraction and Classification Methods for Hybrid
fNIRS-EEG Brain-Computer Interfaces’, Frontiers in
Human Neuroscience, 12(June), pp. 1– 25. doi:
10.3389/fnhum.2018.00246.
Altın, C. and Er, O. (2016) ‘Comparison of Different Time
and Frequency Domain Feature Extraction Methods on
Elbow Gesture’s EMG’, European Journal of
Interdisciplinary Studies, 5(1), p. 35. doi:
10.26417/ejis.v5i1.p35-44.
M. Ali, A. Riaz, W. U. Usmani and N. Naseer, "EMG Based
Control of a Quadcopter," 2020 3rd International
Conference on Mechanical, Electronics, Computer,
and Industrial Technology (MECnIT), Medan,
Indonesia, 2020, pp. 250-254, doi:
10.1109/MECnIT48290.2020.9166603.
Alkan, A. and Günay, M. (2012) ‘Identification of EMG
signals using discriminant analysis and SVM
classifier’, Expert Systems with Applications. Elsevier
Ltd, 39(1), pp. 44–47. doi:
10.1016/j.eswa.2011.06.043.
Alam, M. S. and Arefin, A. S. (2018) ‘Real-Time
Classification of Multi-Channel Forearm EMG to
Recognize Hand Movements using Effective Feature
Combination and LDA Classifier’, Bangladesh Journal
of Medical Physics, 10(1), pp. 25–39. doi:
10.3329/bjmp.v10i1.39148.
D. Mattia, L. Astolfi, J. Toppi, “Interfacing brain and
computer in neuro-rehabilitation,” 2016 4th
International Winter Conference on Brain-Computer
Interface (BCI) 22-24 Feb. 2016.
J. Shi, Z. Dai, “Research on Gesture Recognition Method
Based on EMG Signal and Design of Rehabilitation
Training System,” IEEE 3rd Advanced Information
Technology, Electronic and Automation Control
Conference (IAEAC) 12-14 Oct. 2018.
N. Naseer, F. Ali, S. Ahmed, S. Iftikhar, R. A. Khan and H.
Nazeer, "EMG Based Control of Individual Fingers of
Robotic Hand," 2018 International Conference on
Sustainable Information Engineering and Technology
(SIET), Malang, Indonesia, 2018, pp. 6-9, doi:
10.1109/SIET.2018.8693177.
T. Gandhi, A. Jena, A. B. Pal, Novel approach for BCI,
2010 First International Conference on Integrated
Intelligent Computing, 5- 7 Aug. 2010.
Anis, A.; Irshad, M.; Hamza, S.; Naseer, N.; Nazeer, H. and
Andrian, . (2019). EMG based Control of Transtibial
Prosthesis.In Proceedings of the International
Conference on Health Informatics and Medical
Application Technology - Volume 1: ICHIMAT, ISBN
978-989-758-460-2, pages 74-81. DOI:
10.5220/0009464200740081
Four Gesture Recognization of a Robotic Hand using EMG
87