ACTIVITY INFORMATION SHARING SYSTEM WITH VIDEO
AND ACCELERATION DATA
Go Hirakawa
Network Application Engineering Laboratories Ltd, Fukuoka, Japan
Yuichi Hattori, Masato Nakamura, Sozo Inoue
Kyusyu Institute of Technology, Fukuoka, Japan
Keywords: Activity information, Accelerometer, Web service, Smart phone.
Abstract: In this paper, we introduce the large-scale activity information sharing system ALKAN2. ALKAN2 gathers
users' activity information, which consists of three-axis accelerometer and video data using smart phones
and web service. The information can be shared with other users, and their evaluations can also be gathered.
In designing ALKAN2, we challenged 1) to diversify the types of activities, 2) to gather massive amount of
activities, 3) and to motivate users to provide activity information.
1 INTRODUCTION
As pervasive computing evolves, human activity
recognition technology is expected for various
application fields, such as entertainment, healthcare,
and agriculture. In entertainment, recognizing
choreographies can help players’ proficient. In
healthcare, measuring lifestyle behavior of patients
can be of help to prevent lifestyle-related diseases.
In agriculture, recording work log of farmers can
lead to improve workflow of cultivating process.
Previously, recognizing human activities
required complicated equipment. However, the
widespread usages of three-axis accelerometer
equipped with mobile phones have opened the
possibility of easy-to-deploy activity recognition.
Improving activity recognition requires a large
number of activity data for various types of
activities. However, gathering these data comes with
difficulty, because these always conflict with
motivating participants to perform activities and
provide their information. Due to this difficulty,
current human activity recognition technologies only
support limited kinds of activities. (Bao and Initille,
2004).
In this paper, we introduce the large scale
activity information sharing system ALKAN2,
which gathers users’ activity information with three-
axis accelerometer on smart phones and video data,
while providing participants’ functionalities to share,
mimic, and evaluate activity information.
In ALKAN2 system, we classify the user roles
into 3 types, though a user can switch from any to
any: “provider”, “mimic” and “viewer”. Providers
upload activity data, which consists of accelerometer
and video data to the server. Mimic mimics an
activity that is uploaded by a Provider, and uploads
the mimicking acceleration data to the server.
Viewers watch and evaluate activity data of other
users.
By these features, ALKAN2 has the following
advantages:
1) It diversifies the target activity data by
allowing participants to define activity classes by
themselves. Expert-participants in various fields
such as dance, medical work and agricultural work
would create customized activities.
2) It inspires motivation of participants by being
mimicked for Providers, automatically scored by
similarity for Mimics, and evaluations by Viewers
for both.
3) As a result of 1) and 2), it gathers massive
amount of activity data including accelerometer and
video data. In the remainder of this paper, we show
related work in Section II, ALKAN2 system in
Section III, discussion in Section IV, and conclude
in Section V.
557
Hirakawa G., Hattori Y., Nakamura M. and Inoue S..
ACTIVITY INFORMATION SHARING SYSTEM WITH VIDEO AND ACCELERATION DATA.
DOI: 10.5220/0003411505570561
In Proceedings of the 1st International Conference on Pervasive and Embedded Computing and Communication Systems (SIMIE-2011), pages 557-561
ISBN: 978-989-8425-48-5
Copyright
c
 2011 SCITEPRESS (Science and Technology Publications, Lda.)

2 RELATED WORK
Activity recognition requires a large amount of
training data such as labels and accelerometer data.
There is already a work for gathering large-scale
activity information by using the previous ALKAN.
(Hattori et al., 2010). ALKAN can collect a large
amount of accelerometer data and label data from
mobile device such as iPhone/iPod Touch to the
server. However, ALKAN does not bring a benefit
to user, because ALKAN only has a feature of data
gathering. In ALKAN2 system, every user can enjoy
contents include movies, and contents provider can
get response and reputation from other users, like as
other CGM (Consumer Generated Media) system
represented by YouTube or other video sharing
system. Additionally, ALKAN cannot gather a broad
spectrum of activities, because a user can only select
from pre-set activities. In ALKAN2 system, we
permit users to define actions that they want or
require. This is because ALKAN2 can gather a
broad spectrum of activities.
In the literature, Berchtold analyzes ten type of
activity using mobile devices and a server. However,
they analyze only three activities per attachment
position. Berchtold et al. proposes a system for
activity recognition service using mobile phones and
cloud computers. (Berchtold and Buddle, 2010). The
proposed system provides activity recognition
service to mobile phones, as well as it evolves the
recognition model gathering the data to the cloud
computer.
Although our system does not provide activity
recognition so far, we propose another kind of
reasons to use the system for users by mimics of
actions. Moreover, our system can also be extended
to activity recognition service when the data are
gathered massively. Thus, our system can be
positioned as a new approach to gather training data
for evolving recognition models.
3 ALKAN2 SYSTEM
Fig.1 shows ALKAN2 system overview. The system
consists of smart phone software and information
gathering server. In ALKAN2 system, users collect
and upload acceleration sensor data with smart
phones. They also record and upload video data
through web browsers on client PCs. Users also
watch activity data and make evaluations on them.
In ALKAN2, we gather information of where the
smartphone
is, such as waist, held in hand or in a
Figure 1: ALKAN2 system overview.
pocket. If we gather an adequate amount of location
information, we can recognise the location of
smartphone from sensor data. Furthermore, we can
recognise human activity regardless of its location.
ALKAN2 system is suitable for gathering
characteristic activity information, such as step of
dances, forms of sports and other gestures, because
everyone can valuate such activities with watching
video of them. We apply ALKAN2 system to
training of darts, practice of dance and training of
manners. We also expect that we can apply
ALKAN2 system to job training such as medical
care, agricultural work and guidance of health care.
3.1 User Roles
In ALKAN2, we classify the user roles into the
following 3 types, in which any user can switch to
any role:
3.1.1 Provider
A user with role “provider” creates a new activity.
Firstly, Providers collect sensor data using a
smart phone and records video using a camera.
Then, uploads them to the server, and finally, binds
the sensor and video data as activity information the
Providers can receive evaluations of for the activity
from Viewers.
We consider sports trainer, work instructor, and
dancer who wants to bring her/his dance into vogue
as Providers. They are valued their activity from
many people, mimicked by Mimics as described
later. Eventually, they can earn a reputation, and
lead the trend in users’ community.
Provider
Mimic
Viewer
Internet
ALKAN
Server
Sensordata&moviedata
Sensordata&moviedata
EvaluationfromViewer
EvaluationfromViewer
Evaluationscore
Activitycontents
Evaluation
PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems
558

3.1.2 Mimic
A user with role “mimic” mimics activities uploaded
by Providers.
Firstly, Mimics collect sensor data using a smart
phone. Video data can be optionally recorded. Then,
s/he uploads and binds the data as other activity
information. S/he can also receive evaluations from
Viewers.
We consider sports trainee, intern, amateur
dancer, and follower of Providers as Mimics. They
watch contents provided by Providers, which include
example video of activities, and mimic the activities
with recording accelerometer data. Eventually, they
can recognize their progress in these activities.
3.1.3 Viewer
A user with role “viewer” watches activity data on
the web browser on a client PC.
S/he gives numerical and/or commentative
evaluation to shared activities.
We consider people who just enjoy contents with
activity video as Viewers. The more ALKAN2
system gathers attractive contents, the more this
system gains the Viewer. As a result, Providers and
Mimics get response and reputation for their
contents, which motivate them to create new
contents. We also expect Viewer to grow up to
Mimics or Provider.
3.2 Data Binding
In ALKAN2 system, an activity information data
accepts multiple accelerometer data and video data
to support variety of recording data situations such
as multiple cameras and multiple sensors. Therefore,
the system provides the functionality to “bind”
multiple data as single activity information.
3.3 Smart Phone Software
Smart phone software runs on iOS which supports
iPod Touch, iPhone, and iPad, and on Android OS.
Fig.2-3 shows sensing interface with the smart
phone software. At first, a user selects a type of
activity, then selects the sensor position, and then
starts sensing with performing activity. The sensor
data and the metadata are stored in the memory on
the smart phone. Finally, the smart phone software
sends these data to the server when it becomes
online.
Figure 2: sensing interface. Figure 3: Selecting activities.
3.4 Server
As shown on Fig. 4, information gathering server
has the following features.
3.4.1 Activity Definition
Providers create new activity definition using web
browser on client PCs. The term “activity” consists
of activity name, description, and value of METS
(metabolic equivalents).
3.4.2 Activity Management
Providers and Mimics upload and associate video
data and accelerometer data with activity definition
using web browser on client PCs. They also create
list of activity, and allow accessible to other users.
3.4.3 Activity Information Storage
Mimics record and send mimicked sensor data to
server software using smart phone software. Server
software stores activity data on database.
3.4.4 User Management
All users register their authorization data using web
browser on client PCs. Smart phone software also
use this authorization data for personal verification.
Information gathering server consists of Apache
as web server and MySQL as database server. Fig.5
shows the view of activity information with client
PC. The browser view of activity information in
ALKAN2 shows video data, accelerometer, and
evaluations by Viewers.
ACTIVITY INFORMATION SHARING SYSTEM WITH VIDEO AND ACCELERATION DATA
559

Figure 4: Feature of Information Gathering Server.
Figure 5: View of activity information with client PC.
4 DISCUSSION
In ALKAN2 system, we classify the system user
into 3 roles. Providers can create an activity of
her/his interest, or specialties. This leads not only to
diversify the type of activities, but also to upgrade
the quality of activity data.
Viewers can browse many activity data with
video data uploaded by Providers and Mimics for
free. High-quality activity data will attract Viewers,
and will receive good reputation. These reputation
and response from Viewers motivate Providers and
Mimics to upload activity data continuously. (Bruke,
Marlow and Lento, 2009). As a result, ALKAN2
system benefits massive amount of activities
uploaded by Providers and Mimics. Consequently,
we suppose this cycle brings positive growth in
gathering activity data. Thus, ALKAN2 system fits
in gathering large-number of activity data for
various types of activities.
On the other hand, ALKAN2 system is suited to
gathering characteristic activity information, such as
dances, sports and gestures, rather than daily habits
of activity information, because ALKAN2 system
motivates users to upload activity data by reputation
and response from other users. Gathering activity
information of such daily habits is an issue in the
future.
ALKAN2 system requires a PC client as well as
smart phone client to register activity data, to upload
video data, and to associate them. Registering
activity information without PC clients is also an
issue in the future.
5 CONCLUSIONS
We introduced the large-scale activity information
sharing system ALKAN2, which can share and
evaluate human activity information. Moreover, we
challenged to diversify the types of activities, to
motivate users to provide activity information, and
accordingly, to gather massive amount of activities.
We have already started experiments using the
evaluation system consists of more than 200
ALKAN2 clients installed on iPod Touch. Moreover,
we are releasing ALKAN2 system to everyone, and
willing to expand experiments globally.
ACKNOWLEDGEMENTS
This work is supported by 'Information Explosion IT
Platform' (21013038) by the Grand-in-Aid for
Scientific Research, Grant-in-Aid for Young
Scientists (A) (21680009) of JSPS, and Strategic
Information and Communications R&D Promotion
Programme (092110002) of MIC, Japan. We are
grateful for their support.
REFERENCES
Ling Bao and Stephen S. Intille. (2004) ‘Activity
Recognition from User- Annotated Acceleration Data’,
International Conference on Pervasive 2004, pp.1–17.
Nishkam Ravi, Nikhil Dandekar, Preetham Mysore, and
Michael L. Littman. Activity recognition from
accelerometer data. In AAAI, pages 1541–1546,
2005.
Yuichi Hattori, Masaki Takemori, Sozo Inoue, Go
Hirakawa, Osamu Sudo, “Operation and Basline
ServerSoftware
Activity
Definition
Activity
Manager
Activity
Information
Storage
User
Manager
DB
User
Create NewActivity
AssociatewithMovie
CreateActivityList
MimicActivity
SendSensorData
Authorization
PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems
560

Assesment Large Scale Activity Gathering System by
Mobile Device”, DICOMO2010, pp.583-590, Jul,
2010.
Yuichi Hattori, Sozo Inoue, Go Hirakawa, Osamu Sudo.
“Gathering Large Scale Human Activity Information
Using Mobile Sensor Devices”, International
Workshop on Network Traffic Control, Analysis and
Applications (NTCAA-2010), pp.708- 713, Fukuoka,
Japan, 2010.
J. Lester, T. Choudhury, N. Kern, G. Borriello, and B.
Hannaford. A hybrid discriminative-generative
approach for modeling human activities. In Proc. of
the International Joint Conference on Artificial
Intelligence (IJCAI), 2005.
M. Berchtold, M. Budde, D. Gordon, H. R. Schmidtke,
and M. Beigl. “Actiserv: Activity recognition service
for mobile phones”, In Proceedings of the Fourteenth
International Symposium on Wearable Computers
(ISWC 2010), 2010.
Burke, M., Marlow, C., and Lento, T. Feed me: motivating
newcomer contribution in social network sites. In
Proc. CHI ‘09. ACM Press (2009), 945-954
ACTIVITY INFORMATION SHARING SYSTEM WITH VIDEO AND ACCELERATION DATA
561