On Visualizing Movements and Activities of Healthy Seniors

An Overview

Shahram Payandeh

Experimental Robotics and Graphics Laboratory, Simon Fraser University,

8888 University Drive, Burnaby, British Columbia, V5A 1S6, Canada

Keywords:

Active Seniors, Movement Sensing, Activity Sensing, Movement Visualization, Activity Visualization,

Cluster of Active Seniors.

Abstract:

Increased life expectancy and motivational support for the pursuit of independent living for active seniors have

been introducing a number of challenges for the families and care monitoring personnel. In general, this class

of aging seniors are healthy enough that they can afford to continue to have the independent living life styles

but also are old enough that one still needs to have some levels of monitoring of their movements and activi-

ties in their private dwellings. These motions and activities can be monitored through a network of connected

sensors at each individual’s dwelling where they can be networked to form a monitoring cluster. The objec-

tive is then to be able to present and visualize such movements and activities information to the monitoring

personnel. This paper reviews various sensing modalities which can be used in monitoring movements and

activities of seniors and presents an overview of visualization techniques which have been utilized in various

similar applications. In addition the paper highlights a conceptual framework which can be used to visualize

movements and activities in a cluster of such sensor network.

1 INTRODUCTION

Visual monitoring of elderly through cameras in ei-

ther independent living dwellings or senior care giv-

ing facilities has been a center of attention for a num-

ber of years (Zouba, N. et. al (2010)). One of the ben-

eﬁts of such monitoring set-up is the direct viewing

access where family members and care giving staff

can visually observed movements and activities of the

seniors (NiScanaill, S. (2006)). Another very impor-

tant beneﬁt of visual monitoring of movements and

activities is through visual analytic where it is pos-

sible to predict any future abnormalities using long-

term historical movements and activities data of se-

niors (Forkan, A. at. el (2014)). Such early antici-

pation of anomalies can improve the rate of disease

prevention.

One of the key challenges in effectively utilizing

the existing video technology in common living ar-

eas is its lack of protecting privacy of seniors while

individual is being viewed through video camera net-

work (Islam, R. et. al (2009)). The other main chal-

lenge of deploying the automated version of the cur-

rent video technology is the lack of effective infor-

mation display and visualization associated with the

movements and activities where seniors can feel com-

fortable to share and also informative enough for the

family members and care giving staff to view (Lu, Y.

and Payandeh, S. (2008)). Other types of technology

have also been introduced which require the senior to

wear various sensors and monitoring devices (Maki,

H. et. al. (2011)). Although in comparison with the

visual monitoring through cameras, these alternative

monitoring technologies can preserve privacy of se-

niors in their dwellings; it requires them to wear these

sensors which can introduce other types of inconve-

nient and discomfort to the aging population. Sim-

ilar to home security system, on/off and proximity

switches and sensors can also be deployed in the pri-

vate residence of seniors. These sensors can be inte-

grated in a design of smart homes (e.g. beclose.com)

that can give a crude information in regard to, for ex-

ample, the opening and closing of doors or presence

or absence of the senior in a room. Other newer multi-

modal technologies are also being evaluated which in

addition to allowing visual monitoring of the scene

through the video camera, it can also supply infor-

mation about the relative location of the object with

respect to the camera and sense the source of audio

inputs (e.g. Kinect-II from Microsoft).

In deploying a network of sensors for monitor-

ing the movements and activities of seniors, a num-

517

Payandeh S..

On Visualizing Movements and Activities of Healthy Seniors - An Overview.

DOI: 10.5220/0005362805170522

In Proceedings of the 10th International Conference on Computer Graphics Theory and Applications (GRAPP-2015), pages 517-522

ISBN: 978-989-758-087-1

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

ber of questions should be concurrently addressed.

For example, by explaining the features of the sens-

ing technology to seniors, the ﬁrst question which can

be asked should be: a) what would the senior and

their family consider adequate collection of informa-

tion when using a proposed sensing technology?; b) to

the care giving personnel, what would be considered

as an adequate amount of graphical display informa-

tion in regard to monitoring movements and activi-

ties of the elderly?; c) what main features of a sensor

like Kinect-II technology can be utilized in order to

explore approaches for collecting and monitoring the

movement and activities of the elderly and d) what are

various approaches which can be used based on the

sensing technology to visually update and display the

monitoring information of the seniors to the care giv-

ing staff. For example, the sensing technology such

as Kinect II (sometimes it also referred to as ambient

sensing) can be used to identify some unsafe behav-

iors such as leaving the bed or chair. Bed and chair

alarms sensing devices can be used in private living

areas of some seniors but these alarms are costly and

not very reliable due to existence of large number of

false alarms.

2 SENSING THE MOVEMENTS

AND ACTIVITIES

Similar to standard home security system, various

on/off and proximity swicthes can be deployed and

be integrated at various locations of the senior’s

dwelling. These sensors can be used to detect open-

ing and closing of doors, sitting on various furniture

and detect presence and absence of the seniors in var-

ious rooms (Figure 1). Similar to home security, these

sensors can be networked and dispatched to a central

monitoring stations.

Figure 1: Schematics of living area of a senior. Circular

icons are used to show various locations where on/off or

proximity switches can be integrated into the living area.

Another sensing modality which has been very

popular for direct video capture of the seniors in their

dwellings is through usage of video cameras. Unlike

Figure 2: A monitoring station consisting of a number of

cameras distributed within a cluster of a city block.

Figure 3: (a) The motion history image (MHI) of a walking

person. (b) The person’s position and size are indicated by

the green solid rectangle, which is the weighted sum of N

red dashed rectangles.

the integration of the switches and proximity sensors,

this technology do not require to equip the furniture

and appliances with sensors. The monitoring person-

als can directly observe the movements and activities

of the seniors at a given central station (Figure 2). In

some cases it is possible to utilized various image pro-

cessing algorithms in order to assist the monitoring

personnel through the overlaid visual analytic to ac-

cess the monitoring environment. For example, Fig-

ure 3 shows an example of how the movements of a

person can be tracked over a number of video frames.

Figure 4 shows an example of how by proper location

of multiple cameras (one stationary and the other with

pan/tilt/zoom capabilities), the fall of a person can be

detected and the event can be trigged other cameras

for collecting increased levels of details. In another

example, Figure 5 shows how movements of one per-

son away from the simple crowd can be identiﬁed and

be tracked which can then trigger a visual alarm for

the monitoring personnel.

Other recently introduced sensors can offer more

versatile and economically feasible solution to mon-

itoring movements and activities of seniors. These

sensors can address visual privacy issues associated

GRAPP2015-InternationalConferenceonComputerGraphicsTheoryandApplications

518

Figure 4: (a) Trajectories and sizes of people over last 30

frames till falling is detected in the stationary camera (b)

Screen shot when fall happens (c) The 2-D 20 × 20 hue-

saturation histogram of the fallen person (d) Samples gen-

erated in the active camera (e) Target’s location and size.

Figure 5: (a) Trajectories and sizes of people over last 30

frames till wandering is detected in the stationary camera

(b) Screen shot when wandering happens (c) The 2-D 20×

20 hue-saturation histogram of the wanderer (d) Samples

generated in the active camera (e) Target’s location and size.

Figure 6: An example of sensing the movements of people

through Kinect II sensor and various examples of the visu-

alization of such movements.

with conventionalvideo camera systems. Sensors like

Kinect II offers a number of sensing modalities in-

cluding the IR sensors, depth sensors, positional mi-

crophone sensor and also a video camera. For exam-

ple, some of the privacy issues regarding the direct

visual monitoring of the seniors through video cam-

era can be addressed through utilization of the depth

sensing. Figure 6 shows an example of monitoring

the movements through the IR sensor and also shows

various reconstruction of sense data for visualization.

One of the main challenges of the current deployment

of this sensing technology is the restriction it intro-

duces in regard to the placement of people with re-

spect to a single sensor and association of the move-

ments of seniors between a network of such sensors.

3 MOVEMENT AND ACTIVITY

VISUALIZATION

Visualization of the movements and activities of se-

niors in their private dwelling shares some similarities

with other related areas of information visualizations.

The overall objectives are to convey to the care giving

staff, who are in charge of monitoring a cluster of se-

nior dwellings, information which at various levels of

detail about the movement status and activity health

of each individual. Figure 7 shows and example of

visualization of 96 known activities of a person show-

ing one per ring (Zhao, J. et. al. (2008)). The author

proposed that generalization of such movements con-

sists of three operations; spatial, temporal and activ-

ity abstraction. Three types of techniques are distin-

guished: suppression, aggregation and codiﬁcations.

Here, suppression is deﬁned as a way of reducing di-

mensionality by retaining certain important compo-

nents or dimensions from the original data set. Ag-

gregation refers to operations that merge, combine or

summarize similar or related objects/elements into a

new and higher-level abstraction. This can further be

divided into two types: spatial aggregation and tem-

poral aggregation.

Figure 8 shows another example that illustrates

the movements of an individual during one week

(Kang, C. et. at. (2008)). Every individual has a

daily activity program consisting of a number of out-

of-home activities, including activities that are spa-

tially and/or temporally ﬁxed for certain individuals

and others that can be undertaken at various locations

or times of the day.

(Buono, P. et. al (2014)) also extended the notion

of ring and time space to present a visualization tech-

nique for people’s activities. They proposed the use

of graphical representation of the boxplot with differ-

ent semantics for easier user interface. For example,

in Figure 9 , the external circumference has 24 h indi-

cated on it. Six stripes are visualized which represent

the users and color histograms for each members rep-

resents their activities after, before and the expected

activities after the current time.

An interesting approach was proposed for repre-

senting activities using the density map (Wang, S. and

OnVisualizingMovementsandActivitiesofHealthySeniors-AnOverview

519

Figure 7: Daily visualization of activities of a person as

concentric ring topology.

Figure 8: A week-long individual travel-activity path.

Skubic, M. (2008)). Here, different colors are used

to represent different levels of density in the motion

sensor data. The density is computed as the number

of all motion sensors during an hour divided by time

at home during that hour. Figure 10 shows an exam-

ple such visualization technique where the x-axis is

the hours in a day and the y-axis represents days in

the month of an active lifestyle. The color bar on the

right of the ﬁgure shows the colors of different densi-

ties. Black represents time away from home. White

means that no sensor activated. Colors change from

light grey, yellow, green, light blue to dark blue as the

density per hour increases. The dark blue color rep-

resents the highest density of 550 or more events per

hour. The sedentary life style map is much less col-

orful, green is the darkest color in the map, and the

corresponding density is around 300 times per hour.

The black areas of cover much less time than the den-

sity map for the active life style.

An integration of a single ﬁsh-eye camera for de-

Figure 9: An example of iOS developers in a team of six

are visualized, starting from the time shown in the center

(15:24) up to the next 24 hours. The activities of the de-

veloper at the top of the list are represented in the outmost

stripe, the others are represented in the more internal stripes

according to the list order. Gray zones in a stripe show per-

son unavailability.

Figure 10: An example of density map showing an active

and sedentary life style.

termining the speed and activities of a senior was pro-

posed by (Zhou, Z. et. al. (2008)). Through sin-

gle camera image processing, they proposed a coarse

classiﬁcation of human actions based on their phys-

ical locations and speeds. The objective here was

to develop a low-complexity, efﬁcient, and robust

scheme for action recognition. Various actions such

as walking, sitting on the couch, standing up, sit-

ting at the dinning room table, preparing meals in

the kitchen, visiting bathroom and going outdoors are

classiﬁed. Figure 11 shows an example of senior ac-

tivities classiﬁcation based on the physical location

and speed. The physical location in the room provides

important contextual information for action recogni-

tion.Figure 12 displays the sequence of actions per-

formed by two persons over a one hour period. It can

be seen that person B is much more active than A.

GRAPP2015-InternationalConferenceonComputerGraphicsTheoryandApplications

520

Figure 11: Recognizing major activities of daily living us-

ing location and speed.

Figure 12: Action sequences of two persons over a one-hour

period.

4 VISUALIZATION

CHALLENGES

Similar to any visualization approaches of real phys-

ical events that rely on sensors, visualization of the

movements and activities of seniors is also depen-

dent on the types and modalities of such sensors. In

contrast to most cases of monitoring people in pub-

lic places, viewing and monitoring seniors in their

private dwellings introduced additional constraints on

the monitoring signals that encompasses the privacy

considerations of the individual. In addition, costs

and integration of any monitoring sensors can play a

vital key in overall acceptance of monitoring systems.

The long term objectives of any monitoring and

visualization techniques of healthy seniors is predic-

tion of future abnormalities. This can be accom-

plished by using long-term historical data through

context aware monitoring system in order to detect

progressive changes in human health and behavioral

patterns. Failing to detect symptoms early can result

in severe disease or other serious consequences. For

the case of healthy seniors, the monitoring of gait pat-

terns and hand manipulation patterns can also signi-

ﬁes an on-set of some muscular disease or disorder.

Existing sensing system which is similar to home

security set-up consists of on-off switches which

needs to be integrated with the existing house-hold

items. These sensor can only supply low level moni-

toring information in regard to presence and absence

of movements and activities. Deployment of video

cameras in private dwellings for direct video moni-

toring has been always face privacy challenges. Im-

age processing of live video streams can be carried to

allow some levels of view distortion for the monitor-

ing personnel. However, detecting the gait patterns

and even establishing a robust image processing al-

gorithm for monitoring movements of seniors in vari-

ous lighting conditions have been a major challenges.

Other new type of sensors such as Kinect II can ad-

dress some of the limitation of the above mentioned

sensing modalities. For example, depth sensor can

be used to estimate both presence, movements and

gait patterns of the seniors which can also mask and

protect the privacy of the individual. However, the

network of these sensors needs to be deployed and

calibrated in the living space of the seniors. In addi-

tion to the standard camera (i.e. RGB sensor), these

sensors also are equipped with the IR type imaging

which is less dependent on the illumination condi-

tions. Kinect II also offers a directional microphone

which can be exploited (in addition to the depth sens-

ing) to trigger various visualization tool in the event

of presence/absence of movements and activities.

Given such state-of-the-art in sensing technology

it can be seen that there exist various limitations in

their deployments such as inaccuracy in their meas-

rement/processing or their effective range and accept-

ability. Such limitations can also be extended to the

visualization of the sensed information for the mon-

itoring personnel. In particular when such individ-

uals is in charge of monitoring the movements and

activities of a number of seniors located within their

monitoring clusters. The following are some work-

ing proposal which can be followed in order to design

and developed an effecting visualization techniques.

a) The visualization of movements and activities of

seniors in a monitoring clusters should be structured

in an increased-levels of detail; b) at the highest level,

the living space of each senior can be normalized and

mapped to a patch on a sphere; c) the movements of

the seniors can be mapped to a point in its correspond-

ing patch; d) the activities of the senior can be mapped

OnVisualizingMovementsandActivitiesofHealthySeniors-AnOverview

521

to an animated ﬁgure similar to the image of Vitruvian

Man where the positions of the hands and legs can

correspond to certain activity ; e) color can be associ-

ated to a point on a patch representing the position of

the seniors and to each limbs of the ﬁgure represent-

ing the relative health of the activities the seniors with

respect to the historical normal activities.

5 CONCLUSIONS

The paper gives an overview of the state-of-the-art in

the sensing technology which can be used to monitor

the movements and activities of the seniors. In ad-

dition this paper presents some overview of various

visualization techniques which have been proposed in

the literature for addressing problems similar to pro-

pose monitoring environment. Being able to monitor

the movements and activities of seniors located in a

cluster of connected dwellings is a major issue which

needs to be effectively address. This is due to the in-

creased life expectancy and motivational support for

the seniors to continue and pursuit independent liv-

ing life-style . The overall objective is to enable the

monitoring personnel to effectively visualize the sta-

tus of a senior, within a network, at various levels of

details. For this a 3D textured graphical sphere where

the senior activities and movements can be mapped to

and can be interfaced by the personnel through vari-

ous user interface devices (such intel RealSense Tech-

nology) can play a dominant role in effectively visual-

izing and interacting with the displayed information.

REFERENCES

Buono, P., Costabile, M. and Lanzilotti, R. (2014). ”A cir-

cular visulization of people’s activities in distributed

teams”, Journal of Visual Languages and Computing,

online

Forkan, A., Khalil, I., Tari, Z., Foufou, S. and Bouras, A.

(2014). ”A context-aware approach for long-term be-

havioural change detection and abnormality predic-

tion in ambient assisted living”, Pattern Recognition,

on-line.

Islam, R., Ahamed, S., Hassan, C, and Tanviruzzaman, M.,

(2009). ”Toward Universal Access to Home Monitor-

ing for Assisted Living Environment”, Universal Ac-

cess in HCI, Part II, Ed. C. Stephanidis, Lecture Notes

in Computer Science, pp. 189-198.

Kang, C.,Gai, S., Lin, Xing and Xiao, Y. (2010). ”Ana-

lyzing and geo-visulizating individual human mobility

patterns using mobile call records”, In Proceedings of

International Conference on Geo-informatics, pp. 1-7

Lu, Y. and Payandeh, S., (2008). ”Cooperative hybrid multi-

camera tracking for people surveillance”, Proceedings

of Canadian Journal of Electrical and Computer En-

gineering, Vol.33 , No.3 , pp.145-152.

Maki, H., Ogawa, H., Matsuoka, S., Yonezawa, Y. and

Caldwell, W., (2011). ”A Daily Living Activity Re-

mote Monitoring System for Solitary Elderly People”,

Proc. of IEEE Inter. Conf. on Engineering In Medicine

and Biology , pp. 5608-5611.

NiScanaill, C., Carwe, S., Barralon, O., Noury, N., Lyons,

D. and Lyons, G. (2006). ”A Review of Approaches to

Mobility Telemonitoring of the Elderly in Their Liv-

ing Environment”, Annals of Biomedical Engineering,

Vol. 24, No. 4, pp. 547-563.

Zhao, J., Forer, P. and Harvey, A. (2008). ” Activi-

ties, Ringmaps and Geovisulization of Large Human

Movements Fields”. Information Visualization, Vol. 7,

pp. 198-209.

Wang, S. and Skubic, M. (2008). ”Density map visulization

from motion sensors for monitoring activity level”, In

IET 4th International Conference on Intelligent Envi-

ronment ,pp. 1-8.

Zhou, Z. , Chen, X., Chung, Y., He, Z., Han, T. and Keller, J.

(2008). ”Activity analysis, summarization, and visu-

alization for indoor human activity monitoring”, IEEE

Transaction on Circuits and Systems for Video Tech-

nology, Vol. 18, No. 11, pp. 1489-1498.

Zouba, N., Bremond, F. and Thonnat, M. (2010). ”An Ac-

tivity Monitoring System for Real Elderly at Home:

Validation Study”, Proc. of IEEE Inter. Conf. on Ad-

vanced Video and Signal Surveillance, pp. 278-285.

GRAPP2015-InternationalConferenceonComputerGraphicsTheoryandApplications

522