Keywords: Wearable Device, Real Time Worker Monitoring, Shipyard Fabrication Worker.

Abstract: In many areas of shipbuilding supervisors are presently assigned to monitor and control the

worker’s productivity. This monitoring and controlling activities are not effective as the

productivity can only be measured after activities are executed. To improve the worker

productivity, a real time monitoring and controlling system is required and developed using the

advances of wearable device technology. In this first phase, the work will be focused in the

areas of shipyard fabrication workers. The wearable device is equipped with motion sensor

consisting of 2 (two) types of motion sensors namely accelerometer and gyroscope sensor that

are integrated using Inertial Measurement Unit (IMU) system with Arduino as Microcontroller

Unit (MCU. The typical motion gesture of fabrication workers can be recognized and used as

data of productivity monitoring of a worker. In the laboratory experiment, it shows that typical

gesture of fabrication worker can be seen by the value of root mean square of the resulting

monitoring data from gyroscope censor worn at the lower right hand and accelerometer censor

worn at the lower back spin.

1 INTRODUCTION

Indonesian shipyards commonly have similar

problems with the European SME shipyards (Hubler

& Frank, 2016). They are mostly categorized as

small and medium sized (SME) shipyards facing

specific challenges. Limited financial capabilities

and resources reduce their possibilities to invest in

new technologies and production facilities. In

addition, typical problems such as less disciplined

workers, and lack of professional skills that are

experienced in general by shipyards in developed

countries reduce the ability to use innovative

technologies for improving their productivity. In

order to be competitive, such shipyards must

optimize the use of available resources. Monitoring

the use of human resources both the shipyards and

outsourcing companies is a key important. The

present advances in real time worker monitoring

system is one solution to tackle these opportunities.

Applying real time data monitoring system, worker

productivity of shipyards can be monitored and then

using various strategies of performance monitoring

system overall productivity can be maintained. The

application of this system will also enable the

shipyards to identify scenarios to optimize the use of

available resources such as performance based

salary system. It further gives insight in the options

to take up and manage a larger number of projects

simultaneously and effectively.

2 LITERATURE

2.1 Performance Monitoring

Monitoring is an activity to observe carefully a

situation or condition, including certain behaviors or

activities, with the aim that all input data or

information obtained from the results of these

observations can be the basis for making decisions

about the next actions needed.

Performance monitoring has several objectives,

among others shipyards (Hubler & Frank, 2016):

1. Compliance

2. Auditing

3. Explanation

Application of Wearable Device for Real Time Monitoring System of

Shipyard’s Fabrication Workers

Triwilaswandio Wuruk Pribadi

1,2

, Takeshi Shinoda

Department of Naval Architecture Sepuluh Nopember Institute of Technology, Surabaya 60111 Indonesia

Graduate School of Engineering, Kyushu University, 744 Motooka Nishi-ku Fukuoka 819-0395, Japan

Department of Marine Systems Engineering, Faculty of Engineering, Kyushu University, 744 Motooka Nishi-ku Fukoaka

Pribadi, T. and Shinoda, T.

Application of Wearable Device for Real Time Monitoring System of Shipyard’s Fabrication Workers.

DOI: 10.5220/0008542200630072

In Proceedings of the 3rd International Conference on Marine Technology (SENTA 2018), pages 63-72

ISBN: 978-989-758-436-7

2.2 Wearable Device

Wearable Device is a device that can be used on

parts of the human body related to computer

operations and the latest technology, and uses the

principle of "Wearable Technology", namely

technology that can be used also implemented in

everyday life based on their aesthetics and functions.

The features and sensors that can be used

presently are (Uslu et al, 2013):

1. Geofences in mobile applications only

2. Location Information

3. Maps and Maps Service

4. Device Sensors

5. Activity Recognition

2.3 Classification of Wearable

Technology Devices

The wearable device classification can be classified

according to function, appearance, proximity to

humans, and other parameters (Chatterjee et al,

2016).

2.3.1 Smartwatch

A smart watch is a computerized device or small

computer that is intended to be worn on the wrist

and has expanded the functions often associated with

communication (Mortazavi et al, 2015). Most of the

smartwatch models are currently based on cellular

operating systems.

2.3.2 Smart Eyewear

Other categories of wearable devices, smart glasses

that can be used for various applications in optical

head-mounted displays (OHMD), heads-up displays

(HUDs), Virtual Reality (VR), Augmented Reality

(AR), Mixed Reality (MR) and smart contact lenses

(Mardonova et al, 2018).

2.3.3 Fitness Tracker

Fitness trackers, also known as activity trackers, are

typically worn on the wrist, chest, or ear, and are

designed to monitor and track outdoor sports

activities and measure skills related to skills, such as

speed and distance running, breathing, pulse, and

sleep habits (Cadmus-Bertram, 2017).

2.3.4 Smart Clothing

Smart clothing is similar to other types of wearable

devices that are used to monitor the physical

condition of the wearer, ranging from sports clothing

and consumer clothing (smart shirts and body wear)

(Hanuska et al, 2016).

2.3.5 Wearable Camera

The appeal of this camera is very suitable for

making videos and photos in real time. Two types of

cameras can be used: a small camera that can be

attached to the body or clothes, or even can be used

on the ear, and a larger camera with an attachment to

attach to a hat or helmet (Hanuska et al, 2016).

2.4 Other References

This reference is used to determine the appropriate

method that will be used in the study of gesture

motion of worker monitoring systems. The method

and system are presently used for various purposes.

2.4.1 Microcontroller Unit (MCU)

Microcontroller is a computer system that all or

most of its elements are packaged in an IC chip, so it

is often called a single chip microcomputer

(Gridling, 2007). Wearable devices that are currently

used to do things for specific purposes are as

follows.

a. Sensor-Based Intelligent Positioning and

Monitoring System

The industry is currently pursing towards a

production environment automatically. The position

of workers who do work for production lines and

monitor their movements is very important. For this

purpose, the sensor consisting of 3 axis

accelerometer and 3 axis gyroscope which is often

called an inertial measuring unit sensor (IMU) is a

good choice to do this (Edvardsen et al, 2017).

b. Self-contained Position Tracking of Human

Movement Using Small Inertial/Magnetic

Sensors Module

This position tracking using the Arduino

microcontroller and small inertial/magnetic sensors

which is used to track people who walk (Yun et al,

2007). This system is a system that is almost the

same as the IMU system (Inertial Measurement

Unit). The location of the difference is on the use of

3 types of sensors consisting of an accelerometer

sensor, gyroscope sensor and magnetometer sensor.

c. Essential Tremor Measurement and Analysis

Computer sensor networks were developed to

monitor hand position with essential tremor

conditions, nervous system disorders that cause

uncontrolled shaking, especially in the hands and

SENTA 2018 - The 3rd International Conference on Marine Technology

upper body. The network collects 3D position data

using two ZX Distance and Gesture Sensors, an

Arduino Uno Board, and Raspberry Pi (Burt et al,

2017).

2.4.2 Android and IOS Device

Designing a system with android to determine a

person's condition when he falls. This system is used

in the world of health which is very helpful for

tracking someone who is sick (Casilari et al, 2015).

An iOS application that runs on the iPhone, which

communicates with two Bluetooth low-energy

modular sensors (BLE) (containing an accelerometer

and 3-axis gyroscope and magnetometer).

2.4.3 Smartwatch

In particular, this device can track activity for the

clinical environment and how to guarantee that the

user does the desired activity. This device can track

activity in real-time with sensors mounted on the

user's wrist in the form of a smartwatch (Mortazavi

et al, 2015). This system can be used to record and

track data in large periods of time to provide a

classification of movements from users.

Furthermore, the identification carried out is the

state of the user's posture which consists of sitting,

standing, and lying down. In addition, this device

can also identify the transition from all three user

postures.

2.4.4 Position and Location Tracking

Sensors

Location sensors and position tracking (GPS,

altimeters, magnetometers, compass, and

accelerometers) are the most common types of

sensors in wearable devices, such as activity

trackers, smartwatches, and even medical clothing

where they are used for examines the physical

activity and health of the patient (Khoa, 2015).

3 METHODOLOGY

A research methodology was developed in order to

achieve the research objective. This can be seen in

Figure 1 as a flowchart of activities and each of

them described below.

Figure 1: Flowchart of Research Methodology.

3.1 Literature Review

Literature review is the study of theories and

critically review the existing systems that will be

used in completing the research objective and to

better understand the problems. References to work

on this research are obtained from books, scientific

journals, papers and browsing from the internet that

is competent and accountable

3.2 Observation

The focus of the observations made is the scope of

work (scopes of work) and the movement of body

parts of fabrication workers in carrying out activities

as well as making observations on the worker

monitoring system is presently used.

3.3 Determination of Sensor Location

The tool used to capture, play back, and process

video recordings from several cameras and sensor

modules is the Microsoft Kinect camera. The

process of recording video simulation activities or

work activities using additional software

applications, namely iPi Recorder. While the

Application of Wearable Device for Real Time Monitoring System of Shipyard’s Fabrication Workers

software application that is used to perform motion

analysis is the iPi Mocap Studio application.

3.4 Hardware Design

The tool plan that will be used in this research is the

Arduino microcontroller with accelerometer sensor

module and gyroscope.

3.5 Determining Data Retrieval

Activities

This stage is carried out to determine which body

parts are dominant when fabrication worker

performs related activities. The activities or

activities are marking with bamboo mall aids in

squatting position, straight cutting with a squat

position skater, and a weld tag with a standing

position.

3.6 Analysis of Work Movement

This analysis is done with Kinect tools with iPi

recorder and iPi Mocap Studio software

applications. The purpose of this movement analysis

is to get the right place to put sensors from the tools

that have been made.

3.7 Data Retrieval

Data retrieval flow as in Figure 2 as follows.

Figure 2: Flow of Data Retrieval.

3.8 Data Analysis

This stage is a stage to analyze the data obtained

from the sensor module used. Data output can be

used as a graph which will show a pattern of

activities. Besides it can also be analyzed to identify

the dominant axis when fabrication worker performs

work activities.

3.9 Application System Design

This stage is the stage for creating a web server that

functions as a data storage database. The output of

the sensor module used will be stored in the

database. In addition, there is also a plan to

download data from sensor readings. So that the real

time monitoring system process can be

implemented. The aim is to assist in the performance

monitoring process so that the process is more

practical, easy, effective and efficient.

3.10 Simulation

The simulation phase is carried out to find out

whether the designed tools and systems can be used

to perform gesture motion monitoring. This means

that the tools and systems made can recognize that

the fabrication worker is carrying out a work activity

properly.

4 FABRICATION WORK

ACTIVITIES

4.1 Scope of Works

Fabrication is the initial stage of the ship building

process. This fabrication process is carried out in the

fabrication workshop where the production of this

process is the components for the new building of

the ship.

4.1.1 Marking

 Straight Marking

 Curve Marking

4.1.2 Cutting

 Cutting using a semi-automatic machine.

 Cutting using a blander machine.

 Cutting using a CNC (Computer Numerical

Control) machine.

4.2 Hardware Devices

The type of microcontroller used is Arduino. The

device is an ATMega328 microcontroller issued by

Atmel which has a RISC (Reduction Instruction Set

SENTA 2018 - The 3rd International Conference on Marine Technology

Computer) board architecture. Figure 3 is the

material and equipment used to realize the

equipment to be used.

(a) (b)

Figure 3: Equipment Components: (a) Arduino Uno (b)

MPU 6050 Gyroscope & Accelerometer Sensor Module

and equipped with Male to Female Jumper Cable and USB

2.0 Cable.

4.2.1 Arduino Board

Arduino Uno is one of the products labeled Arduino

which is an electronic board that contains an

ATMega328 microcontroller (a chip that

functionally acts like a computer). This tool can be

used to assembly electronic circuits from simple to

complex.

4.2.2 Sensor Module

The following is a description of the wiring that is

carried out between the Arduino Uno board and the

sensor module used.

a) The VCC pin on the Sensor is connected with

a 3.3 V pin on Arduino Uno R3;

b) The GND pin on the Sensor is connected to

the GND pin on Arduino Uno R3;

c) The SCL pin on the Sensor is connected to the

A5 Analog In pin on the Arduino Uno R3;

d) The SDA pin on the sensor is connected to the

Analog A4 pin on Arduino Uno R3.

Figure 4: Arduino Wiring Board with Sensor Module.

Next step is to do the wiring between the

Arduino Uno board and the ESP 8266 Wi fi shield.

The wiring is shown in Figure 5.

a) VCC pin on Wi-Fi Shield ESP 8266 connected

to pin 5 V on Arduino Uno R3;

b) The GND pin on Wi-Fi Shield ESP 8266 is

connected to the GND pin on Arduino Uno R3;

c) Pin TX on Sensor connected with RX Analog

In pin on Arduino Uno R3;

d) The RX pin on the sensor is connected to the

Analog TX In pin on Arduino Uno R3.

Figure 5: Arduino Uno Wiring Board with Wi Fi Shield

ESP 8266.

4.3 Determination of Censor Location

Determination of sensor placement is done by

making observations and observations in the real

work location. Figure 6 is a diagram shows activities

to determine the location of sensors used.

Figure 6: Activities for Sensor Determination.

4.3.1 Movement Simulation

The iPi Recorder software application can be used to

simulate movements of activities carried out by a

pre-determined fabrication worker.

4.3.2 Data Analysis

Figure 7 shows the result of the recording that is

done using iPi Recorder, then the analysis will be

done using the help of a software application, iPi

Mocap Studio. The goal is to make it easier to carry

out movement analysis so that a dominant body part

when carrying out the activities can be identified.

Application of Wearable Device for Real Time Monitoring System of Shipyard’s Fabrication Workers

(a)

(b)

(c)

Figure 7: Changes to the movement carried out when

carrying out straight cutting activities using semi-

automatic machine: (a) first movement, (b) second

movement, (c) third movement.

4.3.3 Censor Location

The next step is to analyze the body parts showing

the typical gesture motion of a fabrication workers.

Figure 8: Information obtained from iPi Mocap Studio.

In Figure 8, the output for conducting motion

analysis was shown. It was identified that the

location of the body parts showing the most

significant movement during activities performed by

fabrication workers was the Right Fore Arm to

record hand gesture motion using gyroscope and

Lower Back Spine to record linier movement of the

body using accelerometer as shown in Figure 9.

Figure 9: Location of Censor and Arduino Uno.

4.4 Application Design

Observations made on the monitoring system in

the real shipyard situation are used to design an

application system. Simulations are carried out

starting from the log-in page carried out by the user,

registration or registration, up to log-out as in Figure

10, Figure 11, and Figure 12.

From this application data will be displayed for

every 60 seconds from each sensor, namely the

accelerometer sensor and gyroscope sensor. The data

will be stored in the server database and can be

downloaded to the server computer.

Figure 10: Display of Application Log-In System.

(a)

SENTA 2018 - The 3rd International Conference on Marine Technology

(b)

Figure 11: Display of Data Output (a) Accelerometer, (b)

Gyroscope.

Figure 12: Export Display of Monitoring Results Data.

5 RESULTS

The experiment was carried out with the MPU 6050

sensor module with the output of 3 axis

(coordinates) namely the x axis, y axis and z axis.

The microcontroller unit (MCU) used is Arduino

Uno. Data retrieval is carried out for 60 seconds

with the result of 2000 data for each axis on each

sensor. The specified time delay is 0.03 seconds.

5.1 Marking

During the experiment, activity of marking by using

bamboo mall tools as shown in Figure 13 the

fabrication worker is obtained data graph as shown

in Figure 14.

Figure 13: Characteristic of marking movement.

Figure 14: Output Data Module Sensor Accelerometer and

Gyroscope on Marking Activities.

5.2 Cutting

In the experiment the activity or activity of cutting

work straight by using a semi-automatic gas cutting

as shown in Figure 15 is obtained the data graph as

shown in Figure 16.

Figure 15: Cutting Movement Characteristics (Cutting)

5.3 Tag Weld

In the experiment the activity or activity of

welding point (tag weld) as shown in Figure 17 by a

fabrication worker obtained data graph as shown in

Figure 18.

Figure 17: Characteristics of Weld Tag Movement (Point

Welding).

Application of Wearable Device for Real Time Monitoring System of Shipyard’s Fabrication Workers

Figure 18: Output Data Module Output Sensor

Accelerometer and Gyroscope on Tag Weld Activity.

Figure 16: Output of Data Module Accelerometer and

Gyroscope on Cutting Activity.

5.4 Summary of Experimental Data

Table 1 below shows an example of summary of

experimental data.

Table 1: Summary of Experimental Data.

True Marking 1,847 2,846 1,766 0,100 0,228 0,013

False Marking 12,426 12,858 12,881 1,300 1,528 1,013

True Cutting 17,847 6,846 4,766 43,856 42,744 50,900

False Cutiing 0,100 0,228 0,013 12,900 34,123 0,837

True Tag Weld 2,336 2,495 1,313 13,885 8,115 10,418

False Tag Weld

0,064 0,111 0,137 3,148 3,891 3,477

Accelerometer Gyroscope

Activity

6 TRIAL SIMULATION

Trial simulations are carried out to determine the

accuracy of the designed system. Comparative

analysis of the value of MSE (Mean Squared Error)

between the simulation data of proper activities and

the correct simulation with the wrong activities will

determine whether the hardware and application

system can be used to justify the fabrication workers

do the activities properly. Table 2 is the result of a

comparison of the MSE scores from the experiments

carried out.

Table 2: Summary of Mean Square Error (MSE) Value.

XYZ X Y Z

Minimum Data -2320 6946 -4588 -13440 -22637 -15108

Maximum Data 10932 21346 10244 14219 19404 25466

Average Data 3891 15750 2351 -574 407 398

Minimum Data -9452 2048 -7856 -834 -1038 -264

Maximum Data 10724 22984 4668 584 1481 884

Average Data 1320 15368 -3226 -117 -19 253

Minimum Data -1884 10704 6524 -7176 -10906 -3040

Maximum Data 8740 16212 10132 5584 10117 6816

Average Data

3856 12250 7642 -97 60 55

Turning

Marking Squat

Position

Straight

Cutting Squat

Position

Standing Tag

Weld Position

Activity

Accelerometer Gyroscope

Data

Character

7 DISCUSSION

In this paper, a key important of real time data

monitoring system for improving worker

productivity in a shipyard of developed countries has

been addressed. In such a shipyard the worker

SENTA 2018 - The 3rd International Conference on Marine Technology

productivity is monitored only by the presence of

workers in the location and then by recording the

resulting interim products executed by supervisors.

While supervisors have difficulties to closely

monitoring the real performance of workers,

especially to differentiate between workers with

acceptable performance and under performance

during they are doing the activities. This will finally

make shipyard management difficult to estimate the

overall project performance and frequently will

affect the ship delivery time.

Further problems will arise if new approach in

salary system based on real performance of workers

is implemented in order to improve the overall

productivity and finally the profit of shipyards. This

new approach will require the real time monitoring

system as the basis of performance measurement

activities of workers. Many skills and competences

of workers involve in the process of shipbuilding

from fabrication, sub assembly, assembly and

erection. It was observed during the research that

every worker has typical gesture motion in doing

their activities. If such typical gesture motion of

workers can be identified and recognized by the

developed system, it will make possible to improve

significantly the overall shipyard productivity.

In this first phase of the research, an observation

was focused on the development of the real time

monitoring system to record, to identify and to

recognize of gesture motion of fabrication workers.

It was also identified that the most important part of

the system is censor location in the body of workers.

The censors must be located in the part of bodies

that moves dominantly to express typical motion

gesture. This has been executed by doing video

recording to the fabrication activities. Further this

was followed by simulating and captioning the

typical motion by using Microsoft Kinect and IPi

Motion Capture Studio Software.

An analysis using IPI Motion Capture Software

can then be executed to determine the dominant

parts of bodies that can show the typical gesture

motion of worker. It was identified that the position

or location of the body parts showing the most

significant movement for all activities performed by

fabrication workers was the Right Fore Arm to

record hand gesture motion using gyroscope and

Lower Spine to record linier movement of the body

using accelerometer.

A prototype of the developed system based on

wearable devices consisted of Arduino

microcontroller and two sensors accelerometer and

gyroscope has been explained clearly in the previous

paragraphs. This is then followed by trying the

system prototype to the workers in the laboratory in

order to evaluate the performance of the system.

During the system trial, various configuration of

fabrication activities of workers has been tried and

the resulting gesture motion of workers has been

recorded by two censors simultaneously. The

gyroscope censor records the gesture motions of

right hand of the worker and accelerometer censors

records the linear gesture motions of lower spines of

the worker. The two censors record the motions in

the three directions X, Y, Z.

A graph showing the gesture of worker motion

and its calculated MSE (Mean Squared Error) were

obtained from each work activity performed. The

application system will recognize the typical proper

work activities through the MSE values generated by

the recording data. If the MSE value on three each

axis X, Y, Z has a small value, the application

system will recognize as a proper work activity.

However, if the MSE value produced is large, the

system will state that it is improper work activity.

8 CONCLUSION

From the facts and discussion above, it can be

concluded as follows.

 The dominant body part when fabrication

worker performs typical activities is right

forearm signing as hand gesture motion and

lower spin showing linier movement of body.

 A prototype of proposed system developing a

combination of the IMU (Inertial Movement

Unit) system with the accelerometer and

gyroscope sensor modules and the Arduino

Uno microcontroller can be used to perform

motion capture and monitor the gesture motion.

 A graph showing the gesture of worker motion

and its MSE (Mean Squared Error) were

obtained from each work activity performed.

 Proper work activities can be recognized by the

MSE values generated by recorded motion

data. If the MSE value on three axis (X, Y, Z)

has a small value, then it will be recognized as

a proper work activity. On the other hand, if

the MSE value produced is large, the activity

will be recognized as improper work activity.

 It was recognized the factors that influence the

recording data is because of noise data

generated by accuracy of censor readings.

Application of Wearable Device for Real Time Monitoring System of Shipyard’s Fabrication Workers

REFERENCES

Burt, e. a., 2017. Essential Tremor Measurement and

Analysis. New Jersey Governor's School of

Engineering and Technology. New Jersey, USA.

Cadmus-Betram, L., 2017. Using Fitness Trackers in

Clinical Research: What Nurse Practitioners Need to

Know. J. Nurse Pract. Finlandia.

Casilari, e. a., 2015. Analysis of Android Device-Based

Solutions for Fall Detection. Universidad de Malaga.

Malaga.

Chatterjee, e. a., 2016. Classification of Wearable

Computing: A Survey of Electronic Assistive

Technology and Future Design. Proceedings of the

2016 Second International Conference on Research in

Computational Intelligence and Communication

Networks (ICRCICN), India, pp. 23 - 25.

Edvardsen, e. a., 2017. Sensor-Based Intelligent

Positioning and Monitoring System. Uppsala

Universitet. Uppsala.

Gridling, G., 2007. Introduction to Microcontrollers,

Courses 182.064, s.l.: Institute of Computer

Engineering, Vienna University of Technology.

Hanuska, e. a., 2016. Smart Clothing Market Analysis. J.

Nurse Pract. Finlandia.

Hubler, M. & Frank Roland, 2016. Developing Smart

Technologies for Productivity Improvement of

European Small and Medium Sized Shipyards.

European SMARTYards Consortium. Hamburg.

Khoa, T., 2015. Wearable Smart Technologies: New Era

of Technology. Lapland University of Applied

Sciences. Lapland.

Mardonova, e. a., 2018. Review of Wearable Device

Technology and Its Application to the Mining

Industry. Pukyong National University. Busan.

Mortazavi, e. a., 2015. Can Smartwatches Replace

Smartphones for Posture Tracking?. University of

California. Los Angeles, USA.

Uslu, e. a., 2013. Human Activity Monitoring with

Wearable Sensors and Hybrid Classifiers. Computer

Engineering. London.

Yun, e. a., 2007. Self-contained Position Tracking Human

Movement Using Small Inertial/Magnetic Sensor

Module. IEEE International Conference on Robotics

and Automation Roma. Italy.

SENTA 2018 - The 3rd International Conference on Marine Technology