Perception of Human Activities
A Means to Support Connectedness Between the Elderly and Their Caregivers
Kadian Davis
1
, Evans Owusu
2
, Carlo Regazzoni
3
, Lucio Marcenaro
3
, Loe Feijs
1
and Jun Hu
1
1
Department of Industrial Design, Eindhoven University of Technology, Den Dolech 2, 5612 AZ, Eindhoven,
The Netherlands
2
Independent Researcher, Eindhoven, The Netherlands
3
Electrical, Electronics and Telecommunication Engineering and Naval Architecture Department, University of Genoa,
Via Opera Pia 11a, 16145, Genova, Italy
Keywords:
Activity Recognition, Monitoring, Sensors, Smart-phone, Perception, Connectedness.
Abstract:
This position paper describes a smart-phone based activity recognition system for improving social connect-
edness between caregivers and their elderly relatives. Sensing technologies can enable real-time monitoring
to provide activity recognition in order to support health and safety among the elderly who are living indepen-
dently. However, most existing activity recognition systems are focused on using sensors for unidirectional
monitoring of emergency cases in particular, fall detection. Motivated by the desire to utilize bidirectional
activity recognition to improve connectedness between an ageing population and their caregivers, we describe
our planned approach to investigate how the perception of a caregiver’s activities by a senior citizen and
vice versa can aid in improving social connectedness. To investigate this perception, activity states will be
transformed into an information visualization into the caregiver’s home and vice versa without overt commu-
nication from participants. Findings are expected to provide further insight on the extent to which perception
of human activities increase social connectedness.
1 INTRODUCTION
With the growth of demographic ageing within the
European Union, formal and informal care services
are increasingly becoming concerned about the fiscal
burden and demands of the ageing population. Conse-
quently, these institutions are in favour of home care
solutions. As discussed in Davis et al. (2015), there
are inadequate resources such as nurses, which induce
cost; and consequently independent living is forced.
In contrast, despite wanting to live ‘separate’ lives
from their elderly counterparts, caregivers often ex-
perience worry or uncertainty about their elderly rel-
ative’s health and well-being.
For many years, a number of scholars have as-
sociated social isolation and loneliness with old age
(Sheldon et al., 1948; Halmos, 2013). In general,
retirement, mobility-impairment, increased isolation,
death of loved ones and kin-separation due to glob-
alization may cause loneliness in ageing societies.
Furthermore, research has found a number of health
risks associated with loneliness and social isolation
including high mortality risks, cardiovascular and in-
fectious diseases, cognitive deterioration and depres-
sion (Becker et al., 1998; Stafford et al., 2011; Steptoe
et al., 2013). However, epidemiological research sug-
gests that strong social ties play a critical role in en-
hancing the elderly’s psychological and physiological
well-being (Umberson and Montez, 2010).
Social participation has been incorporated into the
research and public policies of ageing societies as
shown in (Landabaso and Letter, 2013). As a result,
this enables social cohesion through active involve-
ment in volunteering services, clubs and associations.
In turn, this increases a sense of belongingness and
also influences quality of life, which is critical for the
sustainable development of ageing societies.
Human activity recognition (HAR) systems can
help in reducing the challenges of supporting el-
derly independent living and reducing the burden of
their caregivers. Moreover, HAR systems provide
a plethora of opportunities for automatic recognition
of activities of daily living (ALDs) in the context of
health and elderly care such as fall detection (Bourke
et al., 2007), classifying activities of Parkinson’s dis-
ease patients (Rodriguez-Martin et al., 2013) and en-
couraging physical activity (Consolvo et al., 2008).
However, most of the existing applications are fo-
194
Davis K., Owusu E., Regazzoni C., Marcenaro L., Feijs L. and Hu J..
Perception of Human Activities - A Means to Support Connectedness Between the Elderly and Their Caregivers.
DOI: 10.5220/0005490601940199
In Proceedings of the 1st International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AgeingWell-
2015), pages 194-199
ISBN: 978-989-758-102-1
Copyright
c
2015 SCITEPRESS (Science and Technology Publications, Lda.)
cused on detecting potentially dangerous situations
and generating automatic alarms in the case of an
emergency (Rashidi and Mihailidis, 2013).
1.1 Research Goals
Social Connectedness can be described as a sense
of belongingness based on the appraisal of having
enough close contacts (Van Bel et al., 2008). In this
position paper, we hope to improve social connectiv-
ity through the perception of activity states generated
from smart-phone sensors. This work forms part of
the proposed experiment of Davis et al. (2015), which
aims to induce social presence through subtle aware-
ness of activity and emotional states between the care-
givers and their elderly relatives.
The ubiquitous cell-phone provides a rich plat-
form for human activity recognition as shown in (An-
guita et al., 2012). The inclusion of sensors namely
the camera, accelerometer and gyroscope, its unobtru-
siveness and communication features (WIFI, 3G and
Bluetooth) add to the smart-phone’s appeal for en-
abling human activity recognition (Ustev et al., 2013).
For the purposes of this research, we hope to apply the
notion of abstraction (i.e. highlighting key informa-
tion contained in the data and suppressing irrelevant
details as discussed in (Cavaco, 2014) to investigate
how the elderly and their caregivers will perceive raw
activity data versus classified data.
2 RELATED WORK
In perceiving human activity, we rely on several in-
formation sources from various modalities including
sensory, motor and affective processes (Blake and
Shiffrar, 2007). Specifically, the human visual sys-
tem seemingly plays a significant role in understand-
ing human actions and intentions. Furthermore, it is
extremely sensitive to human movement and as such
it is able to extract socially relevant details (Troje,
2002). This implies that human motion could pro-
vide reliable information for discerning affect (Blake
and Shiffrar, 2007). For instance, Kaye et al. (2005)
confirmed that virtual intimate objects gave a sense
of peripheral presence and activity awareness; thus
enhancing intimacy in long distance relationships.
Therefore, we believe that the perception of human
activities can improve social connectedness. For
instance, imagine you have an information display
showing your distant elderly parent’s activities in the
periphery of your living room, which is placed in
your living room and you suddenly discern that some-
thing is wrong and you are prompted to call your
mother. Could you recognize specific patterns in the
daily activities of your mother? Could you recognize
that something was wrong based on the dynamics of
the visualization in the absence of direct communi-
cation? Would this improve social connectedness be-
tween yourself and your elderly parent? We believe
that this is possible and in this section we will explore
a number of studies focused on awareness systems for
elderly care.
2.1 Awareness Systems Supporting
Elderly Care
Naturally, awareness of others and their activities can
directly influence social connectedness by addressing
social and emotional needs and possibly initiate com-
munication through other media (Markopoulos and
Mackay, 2009; Romero et al., 2007). Early concep-
tions of awareness systems to support ageing in place
were introduced through the Digital Family Portrait
(Mynatt et al., 2001) and the CareNetDisplay (Con-
solvo et al., 2004). The Digital Family Portrait used
sensors to collect information (eg., weather and ac-
tivities) to display a qualitative sense of a senior citi-
zen’s daily activities to their families over a mediated
environment. The CareNetDisplay was an extension
of the previous ideas of the Digital Family Portrait
and consequently; the researchers amplified an older
person’s photograph with details about their daily life
including activities, moods, medication, falls, meals
and outings (Consolvo et al., 2004). Later, the Daily
Activities Diarist sought to address the weaknesses of
the previous awareness systems through its narrative
presentation of awareness information (Metaxas et al.,
2007). These earlier systems demonstrated great po-
tential for facilitating awareness for distant family
members who were worried about their elderly rel-
atives or members of an elderly care network offer-
ing day-to-day care. However, like the Digital Family
Portrait and CareNetDisplay, the Diarist reflected an
unbalanced communication channel and in most cases
portrayed unidirectional monitoring where the care-
giver observed their elderly relatives for notification
of alarming situations. Unidirectional monitoring in-
volves the deployment of wearable devices and other
intrusive sensors all around a senior’s home. This
is invasive and consequently violates the privacy and
dignity rights of the elderly population.
More recent communication mediated systems
that support peripheral awareness for connecting re-
mote families include the following: the ASTRA -
awareness system for connecting households and mo-
bile family members Romero et al. (2007) and SoP-
resent - an awareness system for connecting remote
PerceptionofHumanActivities-AMeanstoSupportConnectednessBetweentheElderlyandTheirCaregivers
195
households Dadlani et al. (2014). In sum, both Astra
and SoPresent attempted to address the asymmetrical
communication concerns of the previously discussed
research by sharing moments and experiences. How-
ever, they were designed to support connectivity be-
tween families of remote households, which though
related is not the specific context of this research. In
our research, we are interested in understanding how
the elderly and their caregivers will perceive the ac-
tivity data presented in the periphery and how this ac-
quired knowledge improves social connectivity.
3 PROPOSED ACTIVITY
RECOGNITION PLAN
We have chosen an Android based platform as a
means for activity recognition because of its em-
bedded sensors such as the accelerometer and gy-
roscope. In addition, Android’s openness provides
existing resources (source modifications and open
tools), which facilitate the inexpensive development
of context-awareness tools. Therefore, we have devel-
oped the Accelerometer Gyrometer Logger (available
on Google Play) to collect accelerometric and gyro-
scopic sensor readings at a frequency of 50Hz. Spe-
cific properties will be extracted (eg. mean, standard
deviation, entropy etc.) from this data to detect the
following activities: walking, standing, laying, sit-
ting, walking upstairs and downstairs, running and cy-
cling for both the elderly and their caregivers.
Several studies have shown that the Support Vec-
tor machine (SVM) is the most popular classification
method in comparison to the quadratic classifier, k-
nearest neighbor algorithm and artificial neural net-
works. For instance, Ravi et al. (2005) demonstrated
that SVM gives one of the best accuracies for rec-
ognizing human activities achieving over 99.4% ac-
curacy for boosted SVM. Moreover, Anguita et al.
(2012) adapted the standard SVM to exploit fixed-
point arithmetic to reduce computational complex-
ity. This approach enabled them to deploy the smart-
phone inbuilt accelerometer and gyrometer sensors
while maintaining normal battery lifespans for other
shared resources on the phone. The adapted multi-
class SVM obtained similar accuracy compared with
the traditional SVM. In this position paper, we pro-
pose to adopt the multi-class SVM algorithm inspired
by Anguita et al. (2012) to train and classify our data
for recognizing the activities we have mentioned ear-
lier.
3.1 Data Collection
Anguita et al. (2012) have made publicly available,
a HAR training dataset (accelerometer and gyrome-
ter) of 30 volunteers in the age range of 19-48 years.
In their experiment, each volunteer was instructed
to perform the following activities: sitting, standing,
walking, lying, walking upstairs and walking down-
stairs while wearing a smart-phone on their waist.
The experiments were video recorded to enable them
to label the data. We will adopt a similar approach
to collect additional datasets to include activities such
as running and cycling. Our dataset together with the
existing dataset will be used to train the multi-class
SVM classifier for activity recognition. However, our
study will include a larger sample space with our par-
ticipants ranging between 18-90 years, who are living
mostly in the Netherlands. Two sets of training data
will be used, one for participants aged 18-59 years
and the other for participants aged 60-90 years. We
assume the elderly population will exhibit a slower
speed of motion in comparison to their caregivers.
The autographer wearable camera will be mounted
unto the participant’s bodies to capture daily activi-
ties for labelling and interpreting the data. Also, we
will utilize the caregivers and their elderly relatives
own homes as test-beds for our data collection. For
the purposes of this experiment, we assume that the
homes of the caregivers and their elderly relatives are
furnished with staircases, which is typical of Dutch
family homes. The selected activities include typ-
ical activities of daily living, which are commonly
found in validated scales such as (Reisberg et al.,
2001). Also, we used the following articles (Chavar-
riaga et al., 2013; Cook et al., 2009) as references for
compiling our scenario.
3.1.1 Scenario Script
For our experiment, we will ask each participant to
perform the following activities in sequence.
Telephone Use (Sitting): The subject will dial a
specific number listed on an instruction sheet and
write down the instructions given on the recorded
message. The stationary and cellular phone will
be located on the dining room table.
Cleaning (Walking): Participants will be asked to
vacuum the floor of their living rooms.
Exercising (Running and Cycling): Outside of
their own homes, able bodied participants will be
asked to jog on the spot for a few minutes. More-
over, those who are able to cycle will be asked to
ride a bicycle for a few minutes. Note well that
this scenario is more applicable to the caregivers.
ICT4AgeingWell2015-InternationalConferenceonInformationandCommunicationTechnologiesforAgeingWelland
e-Health
196
Hand Washing (Standing): Participants will be
asked to wash their hands in the bathroom face
basin using their own toiletries.
Collect Ingredients (Walking Upstairs and Down-
stairs): Each Participant will be asked to collect
some ingredients, which will be placed on the
stair-head.
Preparing a Snack (Walking and Standing): Par-
ticipants will be asked to prepare a snack and
beverage in accordance with the recorded instruc-
tions.
Resting (Lying): Participants will be asked to lay
on the couch in their living rooms.
Moreover, the data sets generated from the above
mentioned activities will serve as ground truth for val-
idating the perceptions of both the caregivers and their
elderly relatives.
4 THE EXPERIMENT
In this paper, we will investigate the extent to which
the perception of the activities of the elderly by their
caregivers and vice versa help to improve bonding.
Moreover, we seek to understand how much hidden
information would be perceived from the unclassified
accelerometer data and the classified data, which has
a direct correlation between the level of activity and
the jagged edges of the output graph.
4.1 Proposed Methodology
We will conduct a preliminary study applying the
within subjects design to a group comprising of ten
caregivers and ten elderly participants. To reduce the
carry-over effects of this experimental design, we will
divide the group into two groups A and B. This im-
plies, that we will show Group A the classified ac-
tivity display and subsequently show the unclassified
activity display. For Group B, we will first show the
unclassified activity display and then the classified ac-
tivity display. Figures 1 and 2 summarize the experi-
mental approaches.
In the classification approach, we will extract and
select these features (standard deviation, mean, en-
tropy etc), which will be used to train the multi-class
SVM classifier. Signals from the both the elderly and
the caregivers will be streamed to a server, which will
perform feature extraction and selection, and subse-
quently classify the activities. The activities will be
grouped into low, medium and high levels according
to table one. These levels of activities will be relayed
to android tablets, positioned like a photo-frame in
Figure 1: Set-up I showing the classification approach.
Figure 2: Set-up II showing the unclassified approach.
Table 1: Table showing activity classification levels.
Activities Activity Levels
lying low
sitting low
standing medium
walking medium
walking downstairs medium
running high
cycling high
walking upstairs high
the homes of both the caregivers and their elderly rel-
atives and will be rendered using simple line graphs
(Figure 3).
Figure 3: Line graph showing the activity levels.
The dual display of the activities of both the care-
giver and their elderly relatives shows a bidirectional
communication channel, which we believe is a nec-
essary component for reducing the ‘Big Brother’ role
of caregivers. As shown in set-up II, the experiment
is repeated but this time, the original raw accelerom-
eter data is filtered, smoothened and approximated to
a similar line graph as displayed in the set-up I.
Our choice of simple line graphs to display the
activities is not random. Line graphs are one of the
most common means of visualizing time-series data.
PerceptionofHumanActivities-AMeanstoSupportConnectednessBetweentheElderlyandTheirCaregivers
197
They are able to easily display data, facilitate com-
parison and reveal trends within the data. As shown
in Figure 3, we believe it will be easier to associate
low, medium and high slopes with low, medium and
high activity levels respectively. Therefore, this visual
rendering will portray the same aesthetic quality and
visual complexity for both displays.
4.2 Evaluation
To determine the usefulness of the proposed approach
we hope to investigate the following:
How does the experimental subjects perceive both
displays and their preference?
Did the elderly participants recognize certain pat-
terns in the activities of their caregivers and vice
versa?
When they observe the display, what are their per-
ceptions of their caregiver’s activities?
Will the application trigger social behaviour? For
instance, will we see an increase in phone calls?
Furthermore, to investigate social connectedness we
will utilize the validated scales such as the Inclusion
of Other in the Self scale (IOS) as discussed in (Aron
et al., 1992) and the IPO Social Presence question-
naire detailed in (De Greef and Ijsselsteijn, 2001).
5 CONCLUSIONS AND FUTURE
WORK
We have presented our proposed approach for per-
ception of activities from smart-phone generated data
to increase bonding relations between the elderly and
their caregivers. Furthermore, we have described two
techniques for investigating the participants’ percep-
tions of the activity displays and how their percep-
tions would support social connectivity between the
generations. Firstly, we proposed the popular SVM
technique for the activity classification display and
secondly we proposed an abstraction of the original
raw accelerometric and gyroscopic data for the un-
classified display. Provided that the results of this
experiment are favourable, we have a variety of ac-
tivity displays in mind and we hope to implement
them in the near future. Our major long term goal
is to improve bonding and care by designing a system
in which the elderly and their caregivers can interact
through other signals (mainly physiological signals),
among them unconscious signals. In subsequent ex-
periments, we will implement a covert lighting appli-
cation, which will provide contextual awareness of
activity and emotional states of both the caregivers
and their elderly counterparts. Therefore, we believe
this subtle contextual awareness will facilitate the un-
derstanding of the affective and activity states of both
parties and in turn will help them to respond with the
appropriate behaviour. Consequently, this could con-
tribute to improving social connectedness.
ACKNOWLEDGEMENTS
This work was supported in part by the Erasmus
Mundus Joint Doctorate (EMJD) in Interactive and
Cognitive Environments (ICE), which is funded by
Erasmus Mundus under the FPA no. 2010-2012.
REFERENCES
Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-
Ortiz, J. L. (2012). Human activity recognition
on smartphones using a multiclass hardware-friendly
support vector machine. In Ambient assisted living
and home care, pages 216–223. Springer.
Aron, A., Aron, E. N., and Smollan, D. (1992). Inclusion of
other in the self scale and the structure of interpersonal
closeness. Journal of personality and social psychol-
ogy, 63(4):596.
Becker, T., Leese, M., Clarkson, P., Taylor, R., Turner, D.,
Kleckham, J., and Thornicroft, G. (1998). Links be-
tween social networks and quality of life: an epidemi-
ologically representative study of psychotic patients in
south london. Social Psychiatry and Psychiatric Epi-
demiology, 33(7):299–304.
Blake, R. and Shiffrar, M. (2007). Perception of human
motion. Annu. Rev. Psychol., 58:47–73.
Bourke, A., O’brien, J., and Lyons, G. (2007). Evaluation of
a threshold-based tri-axial accelerometer fall detection
algorithm. Gait & posture, 26(2):194–199.
Cavaco, C. A. d. Q. S. (2014). New visualization model for
large scale biosignals analysis.
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T.,
Tröster, G., Millán, J. d. R., and Roggen, D. (2013).
The opportunity challenge: A benchmark database for
on-body sensor-based activity recognition. Pattern
Recognition Letters, 34(15):2033–2042.
Consolvo, S., McDonald, D. W., Toscos, T., Chen, M. Y.,
Froehlich, J., Harrison, B., Klasnja, P., LaMarca, A.,
LeGrand, L., Libby, R., et al. (2008). Activity sensing
in the wild: a field trial of ubifit garden. In Proceed-
ings of the SIGCHI Conference on Human Factors in
Computing Systems, pages 1797–1806. ACM.
Consolvo, S., Roessler, P., and Shelton, B. E. (2004). The
carenet display: lessons learned from an in home eval-
uation of an ambient display. In UbiComp 2004:
Ubiquitous Computing, pages 1–17. Springer.
ICT4AgeingWell2015-InternationalConferenceonInformationandCommunicationTechnologiesforAgeingWelland
e-Health
198
Cook, D., Schmitter-Edgecombe, M., Crandall, A.,
Sanders, C., and Thomas, B. (2009). Collecting and
disseminating smart home sensor data in the casas
project. In Proceedings of the CHI Workshop on De-
veloping Shared Home Behavior Datasets to Advance
HCI and Ubiquitous Computing Research.
Dadlani, P., Gritti, T., Shan, C., de Ruyter, B., and
Markopoulos, P. (2014). Sopresent: An awareness
system for connecting remote households. Ambient
Intelligence, page 67.
Davis, K., Hu, J., Feijs, L., and Owusu, E. B. (2015). So-
cial hue: A subtle awareness system for connecting
the elderly and their caregivers. In Pervasive Com-
puting and Communications Workshops (PERCOM
Workshops), 2015 IEEE International Conference on.
IEEE. In press.
De Greef, P. and Ijsselsteijn, W. A. (2001). Social pres-
ence in a home tele-application. CyberPsychology &
Behavior, 4(2):307–315.
Halmos, P. (2013). Solitude and privacy: A study of social
isolation, its causes and therapy. Routledge.
Kaye, J., Levitt, M. K., Nevins, J., Golden, J., and Schmidt,
V. (2005). Communicating intimacy one bit at a time.
In CHI’05 extended abstracts on Human factors in
computing systems, pages 1529–1532. ACM.
Landabaso, M. and Letter, L. D. (2013). Guide to social
innovation. page 71.
Markopoulos, P. and Mackay, W. (2009). Awareness sys-
tems: Advances in theory, methodology and design.
Springer Science & Business Media.
Metaxas, G., Metin, B., Schneider, J., Markopoulos, P., and
De Ruyter, B. (2007). Daily activities diarist: sup-
porting aging in place with semantically enriched nar-
ratives. In Human-Computer Interaction–INTERACT
2007, pages 390–403. Springer.
Mynatt, E. D., Rowan, J., Craighill, S., and Jacobs, A.
(2001). Digital family portraits: supporting peace of
mind for extended family members. In Proceedings of
the SIGCHI conference on Human factors in comput-
ing systems, pages 333–340. ACM.
Rashidi, P. and Mihailidis, A. (2013). A survey on ambient-
assisted living tools for older adults. IEEE journal of
biomedical and health informatics, 17(3):579–590.
Ravi, N., Dandekar, N., Mysore, P., and Littman, M. L.
(2005). Activity recognition from accelerometer data.
In AAAI, volume 5, pages 1541–1546.
Reisberg, B., Finkel, S., Overall, J., Schmidt-Gollas, N.,
Kanowski, S., Lehfeld, H., Hulla, F., Sclan, S. G.,
Wilms, H.-U., Heininger, K., et al. (2001). The
alzheimer’s disease activities of daily living interna-
tional scale (adl-is). International Psychogeriatrics,
13(02):163–181.
Rodriguez-Martin, D., Sama, A., Perez-Lopez, C., Catala,
A., Cabestany, J., and Rodriguez-Molinero, A. (2013).
Svm-based posture identification with a single waist-
located triaxial accelerometer. Expert Systems with
Applications, 40(18):7203–7211.
Romero, N., Markopoulos, P., Van Baren, J., De Ruyter, B.,
Ijsselsteijn, W., and Farshchian, B. (2007). Connect-
ing the family with awareness systems. Personal and
Ubiquitous Computing, 11(4):299–312.
Sheldon, J. H. et al. (1948). The social medicine of old age.
report of an enquiry in wolverhampton. The Social
Medicine of Old Age. Report of an Enquiry in Wolver-
hampton.
Stafford, M., McMunn, A., Zaninotto, P., and Nazroo, J.
(2011). Positive and negative exchanges in social re-
lationships as predictors of depression: evidence from
the english longitudinal study of aging. Journal of Ag-
ing and Health.
Steptoe, A., Shankar, A., Demakakos, P., and Wardle, J.
(2013). Social isolation, loneliness, and all-cause mor-
tality in older men and women. Proceedings of the
National Academy of Sciences, 110(15):5797–5801.
Troje, N. F. (2002). Decomposing biological motion: A
framework for analysis and synthesis of human gait
patterns. Journal of vision, 2(5):2.
Umberson, D. and Montez, J. K. (2010). Social relation-
ships and health a flashpoint for health policy. Journal
of health and social behavior, 51(1 suppl):S54–S66.
Ustev, Y. E., Durmaz Incel, O., and Ersoy, C. (2013).
User, device and orientation independent human ac-
tivity recognition on mobile phones: challenges and
a proposal. In Proceedings of the 2013 ACM confer-
ence on Pervasive and ubiquitous computing adjunct
publication, pages 1427–1436. ACM.
Van Bel, D. T., IJsselsteijn, W. A., and de Kort, Y. A.
(2008). Interpersonal connectedness: conceptualiza-
tion and directions for a measurement instrument. In
CHI’08 extended abstracts on Human factors in com-
puting systems, pages 3129–3134. ACM.
PerceptionofHumanActivities-AMeanstoSupportConnectednessBetweentheElderlyandTheirCaregivers
199