Virtual Application for Preventing Repetitive Strain Injuries on

Hands: First Insights

Helder Freitas

, Vitor Carvalho

2,3

, Filomena Soares

1,2

and Demetrio Matos

3,4

Department of Industrial Electronics, University of Minho, Guimarães, Portugal

R&D Centre Algoritmi, University of Minho, Guimarães, Portugal

Polytechnic Institute of Cavado and Ave, School of Technology, Barcelos, Portugal

MEtRICs Research Centre, University of Minho, Guimarães, Portugal

Keywords 3D Sensor, Image Processing, Unity, Classifiers, Hand Injuries Prevention, Repetitive Strain, Muscle

Articular Exercises.

Abstract: This paper is focused on the problem of repetitive strain injuries in hands. These injuries are mostly related

to professional activities, especially in people that work at high levels of industrialization, patchwork or in

use of advanced technology in productive process, as they are subject to a high rate of work where the

performed tasks often lead to repetitive actions. The objective of this paper is the development of an

application for hands detection and their movements to develop a game for preventing strain injuries. The

activity will be developed in the software Unity, where it will be associated a 3D sensor, Intel RealSense 3D

Camera F200. During the activity, the user will execute several prevention/relaxation exercises. With the

activity implementation in companies and establishments, the employees will be able to exercise their hands,

thus reducing the risk of being affected by strain injuries, contributing to the decrease or even elimination of

the subsequent related costs with these injuries.

1 INTRODUCTION

This paper presents the problem associated to

repetitive strain injuries in hands. These injuries

happen due to the repetition of a certain movement,

which leads to the excessive use of certain joints and

muscles of the body (Moreira, 2015).

The main injuries in hands due to repetitive efforts

are carpal tunnel syndrome, tendonitis and

Tenosynovitis (DeQuervain disease). These injuries

are responsible for pain and functional incapacity on

upper limbs, which involve tendons, muscles, joints,

nerves and blood vessels (Aptel et al., 2002).

Nowadays, strain injuries affect more and more

people and are mostly related to professional

activities, especially in people that work at high levels

of industrialization or in use of advanced technology

in productive process (Oliveira and Barreto, 1997), as

they are subjected to a high rate of work where the

tasks they perform often lead to repetitive actions.

Strain injuries negatively influence the employee

performance since it leads to a reduced productivity

as they have to perform their tasks with pain and

discomfort on extreme cases. Moreover, they can

even lead to the interruption of the professional career

(Moreira, 2015).

The lack of awareness for prevention often leads

the employers to believe that costs associated to

prevention are unnecessary expenditure. However, in

long term and with onset of these injuries on their

employees the associated expenses become higher

than expenses in prevention. Sometimes the treatment

implies leads to periods of temporary incapacity for

work and, occasionally, to early retirement. This

situation is difficult to manage for businesses,

because prolonged absence of an employee requires a

distribution of his/her duties by other employee or the

temporary hiring of a new employee, being necessary

to invest in his/her instruction (Serranheira et al.,

2003). This leads to production breaks and brings

some setbacks that may condition the company goals.

The objective of this paper is the development of

a game application for hands detection and their

movements for preventing strain injuries in hands.

This is a current and pertinent case that addresses the

needs of prevention to people who are typically

subject to strain injuries. After literature reviewing, it

is verified that several authors argue that performing

Freitas H., Carvalho V., Soares F. and Matos D.

Virtual Application for Preventing Repetitive Strain Injuries on Hands: First Insights.

DOI: 10.5220/0006295102370244

In Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2017), pages 237-244

ISBN: 978-989-758-216-5

237

a set of muscle stretching and warming exercises

daily prevents hand injuries. Thus, a future developed

game will provide several mandatory

prevention/relaxation exercises to make possible to

progress in the game.



The users will perform the exercises at least twice

a day, this is, at the beginning and at the end of their

daily work (Moreira, 2015).

The activity will be developed in the software

Unity, the software recognized as the engine for

developing cross-platform games (Unity n.d.). It will

be employed a 3D sensor, Intel RealSense 3D Camera

F200, which is responsible for the image capture,

allowing hands recognition and movements caption.

Implementing this game in companies and

establishments, the employees will be able to exercise

their hands and preventing future strain injuries on

hands. It is the authors´ believe that, with the goal of

improving their scores in the game, the employees

will voluntarily perform the exercises that will be

beneficial to their well-being.

Following this trend, this paper is divided into

seven sections. In the second section, Hand Injuries

from Repetitive Strain, there are mentioned the

repetitive strain injuries in the hands due repetitive

efforts. The third section, Related Work, presents the

work referred in the literature in this topic. The fourth

section, Movements Description, details the

stretching and warm-up exercises to prevent

repetitive strain injuries on hands. Then, the fifth

section, Implementation, presents the development of

the application. The sixth section, Results, presents

the obtained results. And finally, in section seven,

Final Comments, some conclusions and future

perspectives of work are described.

2 HAND INJURIES FROM

REPETITIVE STRAIN

The occurrence of hand injuries due to repetitive

strain has proved to be a problem in industrialized

countries, because many work activities are

characterized by repetitive movements and required

great effort to perform them (Silva, 2015).

There are several musculoskeletal injuries that

affect hands due repetitive strain, where the most

common are the Carpal Tunnel Syndrome, Tendinitis

and Tenosynovitis.

2.1 Carpal Tunnel Syndrome

Carpal tunnel syndrome is a peripheral neuropathy

caused by incarceration in medial nerve by

compression, stretching, friction or angulation in a

confined space (carpal tunnel) (Serranheira et al.,

2003).

Figure 1 shows a hand with the median nerve,

indicating the fingers this nerve affects. Also, it

shows a cross section to differentiate a healthy wrist

from a wrist with carpal tunnel syndrome.

The carpal tunnel syndrome injury mainly occurs

due repetitive wrist movements and his major

symptoms are pain, numbness, tingling, burning

sensation and muscle weakness in the lower part of

the thumb (Nunes & Bush, 2011).

The carpal tunnel syndrome affects about 0.6 to

2% of working age population (Palmer, 2011).

Figure 1: Carpal Tunnel Syndrome, adapted from (Move

Forward n.d.).

2.2 Tendinitis

Tendonitis is an inflammation of the tendon sheaths

around a joint (Nunes and Bush, 2011). Tendons are

structures that attach muscles to the skeleton.

The excessive use of the wrist allows an underload

in tendons, which generates microscopic lesions.

Thus, the body will try to recover from these micro

lesions. However, if overuse persists, it can occur

inflammation through swelling, making the injured

tendon more vulnerable to overload, causing

tendinitis (Simoneau et al. n.d.). The major symptoms

of tendonitis are pain, weakness, swelling, burning

sensation in the affected areas (Nunes and Bush,

2011).

2.3 Tenosynovitis

Tenosynovitis is the simultaneous inflammation of a

tendon and surrounding synovial sheath, that is, when

a tendonitis occurs the tendon swells and compresses

the sheath, causing the inflammation (Simoneau et

BIODEVICES 2017 - 10th International Conference on Biomedical Electronics and Devices

238

al., n.d.).

Figure 2 shows a hand affected by DeQuervain's

disease, which is one of the most known tenosynovitis

(Nunes and Bush, 2011).

The tenosynovitis injury affects the tendons and

sheaths of the wrist at the base of the thumb and has

symptoms like tingling, numbness, swelling and pain

in the lower part of the thumb (Nunes & Bush 2011).

The occurrence of this injury can lead to the

development of carpal tunnel syndrome.

Figure 2: DeQuervain Tenosynovitis, adapted from

(Chiropractic, 2015).

3 RELATED WORK

There is research on repetitive strain injuries in

several fields. However, in addition to the

recommendation of physical exercise there is no

method for the prevention of these injuries

(Serranheira et al., 2003; Silva, 2015; Aptel et al.,

2002; Nunes and Bush, 2011; Simoneau et al., n.d.).

In order to develop an application to the

prevention of these injuries, it is necessary a sensor

capable of detecting hands in 3D. The literature refers

to sensors as Kinect, Leap Motion and Intel

RealSense 3D Camera F200.

The Leap Motion is a new gesture and position

tracking system with the accuracy level of sub-

millimeter (Weichert et al., 2013).

There are already some applications developed

with Leap Motion: the activity developed for

enhancing motor skills of children with autism (Zhu

et al., 2015), the development of an interactive

application for learning Portuguese Sign Language

(Torres et.al, 2016), and also the activity developed to

physiotherapy for people suffering Parkinson's

disease (Colgan, 2015), among others.

Intel has already hand recognition modules, and

also developed recognition algorithms for some

gestures (IntelRealSense 2014). There are also other

applications developed using Intel cameras, for

example, applications for emotion recognition (Silva

et. al., 2016), medicine rehabilitation, defending

penalty games or play drums games (Han et al.,

2015).

Moreover, there are also several applications

developed with Kinect, among them the recognition

of sign language (Miguel and Sotelo, 2014) or

applications for rehabilitation of upper limbs (Calin

et al., 2011) and both limbs (Martins et. al., 2016).

Although, from the literature review undertaken,

no solution was identified for the problem proposed

in this paper.

4 MOVEMENTS DESCRIPTION

In this section, it will be discussed the various warm-

up and stretching exercises for the hand. The

execution of these exercises are important, since the

main reason for the existence of repetitive strain

injuries results from that the part of the body has not

been sufficiently exercised to perform a task

(Aparício and Silva, 2014).

To prevent the injuries indicated in the previous

section there are two sets of exercises that can be

implemented: warming up and stretching off

exercises, to the wrists and fingers for each hand.

4.1 Stretching Exercises

Stretching exercises should be performed at least

twice a day, at the beginning and end of daily

functions (Moreira, 2015).

4.1.1 Wrist

Being the wrist where all the nerves, tendons and

blood vessels pass to the hand it is very important to

perform an appropriate stretching.

Figure 3: Stretch exercise, wrist bend upward.

Virtual Application for Preventing Repetitive Strain Injuries on Hands: First Insights

239

Figures 3 and 4 demonstrate the execution of the

stretching exercises for the wrist. To stretch the wrist,

it is necessary to keep the arm extended and bend the

wrist upward, figure 3, with the help of the other

hand. Then, it is required to perform the same

exercise, but this time it is necessary to bend the wrist

down, figure 4.

Figure 4: Stretch exercise, wrist bend downward.

4.1.2 Fingers

The stretching of the thumb, figure 5, the index and

middle fingers, figure 6, are very important, since

they are the most used fingers on day to day work.

Also, these fingers are affected by the median nerve,

which passes through the carpal tunnel.

Figure 5: Thumb stretch exercise.

Figure 6: Index and middle finger stretch exercise.

Figure 5 shows the execution of the stretching

exercise of the thumb.

Performing regular stretching exercises increases

flexibility, and it can promote injury prevention

improving the recovery ability, in case the worker

eventually suffers from any injury (Branquinho &

Fragelli, 2012).

4.2 Warm-up Exercises

The heating of muscles and joints will prepare the

user for intense muscular activities (Moreira, 2015).

4.2.1 Wrist

There are several exercises to warm-up all the

muscles and joints of the hand.

Figure 7 represents a warm-up exercise. To

perform this exercise. It is necessary to keep the arm

extended and slowly rotate the hand in a circle.

Starting in figure 7a and rotating passing through

figure 7b, 7c and 7d, ending in position 7a, then repeat

the same exercising process a couple times.

Figure 7: Warm-up exercise, wrist rotation.

Figure 8 shows a pulse heating exercise. To

perform this exercise, the arm should be extended and

still, where only the hand should perform horizontal

movements, moving the hand from position in figure

8a to position in figure 8.b then go back to the position

in figure 8.a and repeat the same movement few

times.

Figure 8: Wrist warm-up exercise, horizontal movements.

Figure 9 shows another warming-up exercise for

the wrist. This exercise is similar to the previous one,

but this time the hand should perform vertical

movements, going from position 9a to 9b and coming

back to position 9a, and repeat.

b a

d c

BIODEVICES 2017 - 10th International Conference on Biomedical Electronics and Devices

240

Figure 9: Wrist warm-up exercise, vertical movement.

4.2.2 Fingers

The following exercises are performed to warm up

fingers.

The exercise in figure 10 consists on gather to the

thumb each of the remaining fingers, one at a time. In

figure 10a the thumb is gathered to the index finger,

in 10b it is gathered to the middle finger, in 10c it is

gathered to the ring finger and in figure 10d it is

gathered to the pinkie finger.

Figure 10: Warm-up exercise, gather to the thumb each of

the remaining finger.

The represented exercise in figure 11 consists in a

random movement of the fingers. Figure 11a, 11b,

11c and 11d represent the different random positions

of the fingers.

Figure 11: Warm-up exercise, random movement of the

fingers.

Finally, in figure 12, there is another heating

exercise consisting of stretching, figure 12a, and

flexing off, figure 12b, all the fingers at the same time

then repeat the process a couple of times.

Summarizing, the warm-up exercises activate

blood circulation and warm up muscles and joints

(Marques, 2011).

Figure 12: Warm-up exercise, Stretching and flexion

exercises.

5 IMPLEMENTATION

In this chapter, it will be discussed the system

developed for the activity, the Unity programming

and the main features of the Intel RealSense 3D

Camera F200.

The use of the Intel RealSense 3D Camera F200

is essential for the activity, once it is responsible for

the 3D detection of hands. The Intel RealSense 3D

Camera F200 has a colour camera, infrared laser

projector and a depth camera. The effective range for

gesture capture is between 20 and 60 centimetres

(IntelRealSense 2014). Table 1 presents the main

camera specs.

Figure 13 show all tracking points tracked from

the hand by camera. However, on the application only

the red tracking points are tracked.

Table 1: Camera Specs, adapted from (IntelRealSense

2014).

Color Camera Depth (IR) Camera

Resolution

Up to 1080p@30FPS

Up to 640x480@ 60FPS

(VGA)

or 120FPS (IR).

HVGA@120FPS.

Active Pixels

1920x1080 (2M) 640 x 480 (VGA)

Frame Rate

30/60/120 FPS*

30/60/120 FPS (Depth),

120FPS (IR)

Field of View (DxVxH)

77° x 43° x 70° (Cone)

90° x 59° x 73° (Cone)

IR Projector FOV- N/A x 56°

x 72° (Pyramid)

b a

Virtual Application for Preventing Repetitive Strain Injuries on Hands: First Insights

241

Figure 13: Hand Tracking Points, adapted from

(IntelRealSense 2014).

The implemented system contains the Intel

RealSense 3D Camera F200 and a computer. The user

is placed in front of the camera to perform the

movements to be tracked. The application is

developed in Unity software, an editor which allows

game development. It is also compatible with various

image, audio, video and text formats. In addition, it

allows to target many devices more easily (Unity

Editor n.d.; Unity n.d.).

Figure 14 represents the general flowchart of the

application.

Figure 14: Implementation flowchart.

Starting the activity, the camera will be activated

and a frame will be captured. From this frame, it will

be verified the hands presence. If they are not

detected, the system captures a new frame, otherwise

the subroutine Tracking action will be performed.

Then, until the End game command is given the

application will continue to run.

The sub-routine Tracking action, figure 15, is

responsible to move the game objects to the position

where hands and fingers are placed on a 3D space.

Figure 15: Tracking action flowchart.

6 RESULTS

In this section, there are shown how hands are

detected, for several stretch and warm-up exercises

referred in the third section.

Figure 16 shows the warm-up exercise tracking.

This exercise consists in gathering to the thumb each

of the remaining fingers, as previously referred.

Figure 16: Tracking result, worm-up exercise gather to the

thumb each of the remaining finger.

Figure 17 shows the tracking result of some wrist

warm-up exercises, in figure 17a and 17c are

presented the wrist warm-up exercise, horizontal

movements, and in figure 17b and 17d are presented

the wrist warm-up exercise, vertical movements.

BIODEVICES 2017 - 10th International Conference on Biomedical Electronics and Devices

242

Figures 18 and 19 show the tracking result when

stretching exercises are performed. Figure 18 presents

the tracking result for wrist stretch exercises. In figure

18a the wrist is bend downward and in figure 18b the

wrist is bend upward. Figure 19 presents the tracking

result for thumb, figure 19a, and index and middle

finger, figure 19b, stretch exercise.

Figure 17: Tracking result for wrist warm-up exercises with

horizontal and vertical movements.

Figure 18: Tracking result from wrist stretching.

Figure 19: Tracking result from stretching thumb, index and

middle finger.

7 FINAL COMMENTS

The goal of this paper is the development of an

application for the detection of hands and their

movements, in order to develop a game application

for preventing strain injuries in hands. The present

activity was developed in Unity software using the

RealSense F200 camera from Intel. Considering the

results obtained, it is observed that it is possible to

detect hands and their movements.

As future work, it is necessary to improve the

efficiency in the detection of the exercises, namely

the recognition of the stretching exercises.

The strain injuries are mainly related to

professional activities, so the implementation of this

game in companies would be an added value, since

this game could decrease or even eliminate the arise

of these injuries in employees, reducing the

subsequent related costs.

ACKNOWLEDGEMENTS

The authors would like to express their

acknowledgments to COMPETE: POCI-01-0145-

FEDER-007043 and FCT – Fundação para a Ciência

e Tecnologia within the Project Scope:

UID/CEC/00319/2013.

REFERENCES

Aparício, L.N. & Silva, A., 2014. Postura, Dor e Perceção

de Esforço na Aprendizagem do Acordeão. (in

Portuguese).

Aptel, M., Aublet-Cuvelier, A. & Cnockaert, J.C., 2002.

Work-related musculoskeletal disorders of the upper

limb. Joint, bone, spine : revue du rhumatisme, 69(6),

pp.546–55. Available at: http://www.ncbi.nlm.

nih.gov/pubmed/12537261 (Accessed November 30,

2016).

Branquinho, T. & Fragelli, O., 2012. Abordagem ecológica

para avaliação dos determinantes de comportamentos

preventivos: proposta de inventário aplicado aos

músicos. , 25, pp.73–84. (in Portuguese).

Calin, A. et al., 2011. Mira – Upper Limb Rehabilitation

System Using Microsoft Kinect. Studia Univ. Babes¸

Bolyai, Informatica, 54.

Chiropractic, W.F., 2015. Screen-Shot-2015-10-22-at-

11.22.59-AM.png (487×341). Available at: http://

chiro352.com/ wordpress/wp-content/ uploads/2015/

10/Screen-Shot-2015-10-22 -at- 11.22.59-AM. png

(Accessed November 29, 2016).

Colgan, A., 2015. Changing How People Look at Physical

Therapy. Leap Motion. Available at:

Virtual Application for Preventing Repetitive Strain Injuries on Hands: First Insights

243

http://blog.leapmotion.com/ changing- people- look-

physical-therapy/(Accessed October 28, 2016).

Han, D. et al., 2015. Rehabilitation Training System for the

Motor Deficit of Upper Extremity. 2015 IEEE Twelfth

International Symposium on Autonomous

Decentralized Systems, pp.162–166. Available at:

http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?

arnumber=7098253.

IntelRealSense, 2014. SDK Design Guidelines. Available

at: http:// www.intel.com/ realsense (Accessed

November 28, 2016).

Marques, R.M.M., 2011. Identificação dos fatores de risco

determinantes da prevalência lesões músculo-

esqueléticas nos membros superiores e coluna vertebral

nos músicos profissionais em Portugal. (in Portuguese).

Martins, T., Carvalho, V., Soares, F., Tracking of

Physiotherapy Exercises Using Image Processing

Techniques, CONTROLO16, 12th Portuguese

Conference on Automatic Control, Guimarães,

Portugal, 14-16 September, 2016.

Miguel, C. & Sotelo, F., 2014. Portuguese Sign Language

Recognition From Depth Sensing Human Gesture And

Motion Capture.

Moreira, J.A., 2015. A prevenção de Lesões por Esforço

Repetitivo (LER) nas aulas de saxofone. Universidade

do Minho. (in Portuguese).

Move Forward, image.axd (700×532). Available at:

http://www.moveforwardpt.com/image.axd?id=ee55b4

95-e5bb-4f5d-bf15-

1e8414ec3f65&t=634796061214070000 (Accessed

November 29, 2016).

Nunes, I.L. & Bush, P.M., 2011. Work-Related

Musculoskeletal Disorders Assessment and Prevention.

Ergonomics-A system Approach, pp.1–31.

Oliveira, E.M. De & Barreto, M., 1997. Engendrando

Gênero na Compreensão das Lesões Por Esforços

Repetitivos.

Palmer, K.T., 2011. Carpal tunnel syndrome: the role of

occupational factors. Best practice & research. Clinical

rheumatology, 25(1), pp.15–29. Available at:

http://www.ncbi.nlm.nih.gov/pubmed/21663847

(Accessed November 30, 2016).

Serranheira, F., Lopes, F. & sousa Uva, A., 2003. Lesões

Músculo-Esqueléticas (LME) e Trabalho: Uma

associação muito frequente.

Silva, L.M.O., 2015. Estudo de casos de Lesões

Musculosqueléticas Relacionadas com o Trabalho dos

Membros Superiores existentes numa empresa de

componentes automóveis.

Silva, V., Soares, F., Esteves, J. S., Mirroring Emotion

System – On-line Synthesizing Facial Expressions on a

Robot Face, 8th International Congress on Ultra

Modern Telecommunications and Control Systems and

Workshops (ICUMT), Lisboa, Portugal, 18-20, 2016.

Simoneau, S., St-Vincent, M. & Chicoine, D., Work-

Related Musculoskeletal Disorders (WMSDs)

Association paritaire pour la santé et la sécurité du

travail Secteur fabrication de produits en métal et de

produits électriques.

Smartphonegurus, 2015. F200-Creative_label.JPG.7ff2d

d6b0bc69858788430f94557964f.JPG (408×252).

Available at: http://www.smartphonegurus.com/

forums/uploads/monthly_2015_07/F200-

Creative_label.JPG.7ff2dd6b0bc69858788430f945579

64f.JPG (Accessed November 29, 2016).

Torres, M., Carvalho, V., Soares, F., “iLearnPSL” –

Development of an interactive application for learning

Portuguese Sign Language: first insight, CISPEE16,

Vila Real, Portugal, 19-21 October, 2016.

Unity, Unity - Game engine, tools and multiplatform.

Available at: https://unity3d.com/pt/unity (Accessed

December 9, 2016).

Unity Editor, Unity - Editor. Available at: https://

unity3d.com/pt/unity/editor (Accessed December 9,

2016).

Weichert, F. et al., 2013. Analysis of the Accuracy and

Robustness of the Leap Motion Controller. Sensors,

13(5), pp.6380–6393. Available at: http://

www.mdpi.com/1424-8220/13/5/6380/ (Accessed

December 12, 2016).

Zhu, G. et al., 2015. A Series of leap motion-based

matching games for enhancing the fine motor skills of

children with autism. Proceedings - IEEE 15th

International Conference on Advanced Learning

Technologies: Advanced Technologies for Supporting

Open Access to Formal and Informal Learning, ICALT

2015, pp.430–431.

BIODEVICES 2017 - 10th International Conference on Biomedical Electronics and Devices

244