HERMES: A PERVASIVE SYSTEM FOR MEMORY SUPPORT

AND AMBIENT ASSISTED LIVING

Alex Conconi, Fabio Cattaneo

TXT e-solutions, Via Friga 27, 20126, Milano, Italy

Aristodemos Pnevmatikakis, John Soldatos

AIT, 0,8km Markopoulo Ave., 19002, Peania, Greece

Sebastian Prost, Manfred Tscheligi

CURE, Modecenterstraße 17, Businesspark Marximum, Objekt 2, 1110, Vienna, Austria

Keywords: Pervasive computing, Ubiquitous computing, Ambient assisted living, Ageing well, Middleware, Surface

computing.

Abstract: As sensors and other pervasive computing technologies are increasingly penetrating ambient assisted living

applications researchers, engineers and application architects are starving for tools, techniques and

frameworks for building and integrating added-value applications. In this paper we present HERMES, the

architecture and implementation of a pervasive computing platform, which supports the development of

ambient assisted living applications for the ageing society, with a particular emphasis on memory aids and

cognitive training. The platform is integrated in the sense that it enables combined support for memory aid

and cognitive training applications. It is also accessible via ergonomic interfaces developed on top of multi-

touch surface devices. Furthermore, the platform is modular and extensible since it provides Application

Programming Interfaces (APIs) for the flexible development of additional applications. Along with the

middleware architecture of the platform we also present representative trial deployments, which manifest

that the presented platform is an attractive option for building pervasive applications for the ageing society.

1 INTRODUCTION

Two decades after the introduction of Mark Weiser’s

ubiquitous and pervasive computing vision (Weiser,

1991), we are increasingly witnessing the

deployment of pervasive computing applications in

fields like manufacturing, health care, smart housing

and ambient assisted living (AAL). Recently,

pervasive applications for ambient assisted living

and ageing well are proliferating both in research

and the enterprise (Stanford, 2002). This

proliferation is mainly a result of the rising longevity

and falling mortality phenomena, which maximizes

the societal impact of these pervasive computing

applications. As argued in (Waibel, 2009) and

(Petersen, 2001) pervasive computing applications

that can alleviate the elderly cognitive decline have a

prominent position among AAL applications for

ageing well.

As sensors and other pervasive computing

technologies are increasingly penetrating AAL

applications for elderly cognitive support,

researchers, engineers and application architects are

starving for tools, techniques and frameworks for

building and integrating added-value applications.

This is because development of applications for

ageing well must confront a dual challenge: First

they have to address the conventional integration

complexity issues associated with the distribution

and heterogeneity of pervasive computing

applications (Soldatos, 2007); at the same time they

must become tailored to particular requirements of

elderly users (Mylonakis, 2008). Towards

addressing the first challenge, application architects

can adopt legacy pervasive computing architectures

dealing with context-acquisition, context-awareness,

105

Conconi A., Cattaneo F., Pnevmatikakis A., Soldatos J., Prost S. and Tscheligi M..

HERMES: A PERVASIVE SYSTEM FOR MEMORY SUPPORT AND AMBIENT ASSISTED LIVING.

DOI: 10.5220/0003447001050114

In Proceedings of the 6th International Conference on Software and Database Technologies (ICSOFT-2011), pages 105-114

ISBN: 978-989-8425-76-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

sensor and actuator control, semantic services, as

well as bridging of disaggregated systems (Dimakis,

2008). Such legacy architectures must be

appropriately augmented or modified in order to

address requirements of aged users. In this article,

we identify such particular requirements along

several complementary axes, namely integrated

support for various cognitive support applications,

ergonomics and ease of use, as well as modularity

and extensibility.

Each of these axes poses additional requirements

to the development of pervasive computing

applications. In particular:

• In terms of integrated application support, tools

and techniques for pervasive cognitive support

applications must support the synergetic action

of applications dealing with memory support,

cognitive training, as well as social interaction.

This integration shall result in a combined

approach for enhancing the elderly mental

ability and social interaction. At the same time

integration allows for individualization of

applications according to the end-user mental

state.

• In terms of ergonomics and easy of use,

applications must feature novel ergonomic

interfaces providing end user with comfort and

flexibility. Such interfaces include novel

interfaces with large buttons and other controls,

as well as multi-touch surfaces offering a

mixed reality experience.

In this paper we present the architecture and

implementation of a pervasive computing platform,

which supports the development of ambient assisted

living applications for the ageing society, with a

particular emphasis on memory aids and cognitive

training. The platform addresses several of the

challenges outlined above focusing on memory

support and cognitive training as the most important

issues, with special attention to ergonomics and ease

of use for the elderly users. Specifically, it is

integrated in the sense that it enables combined

support for memory aid and cognitive training

applications. It is accessible via ergonomic

interfaces developed on top of multi-touch surface

devices. Furthermore, the platform is modular and

extensible since it provides Application

Programming Interfaces (APIs) for the flexible

development of additional applications. In terms of

core middleware technology, the platform is built

over the reference architecture for pervasive systems

developed in the scope of the CHIL project (Waibel,

2009). Several enhancements over this architecture

are however made in order to address requirements

that are peculiar to the ageing support tasks at hand.

In the scope of the paper we present related

middleware architectures and customizations in

order to meet the target goals. Overall the presented

architecture is an attractive option for building

pervasive applications for the ageing society.

The paper is structured as follows: Following

this introductory section, section 2 introduces the

main problems that have to be addressed by the

architecture along with related work on middleware

architectures and technologies for pervasive

computing systems. Section 3 introduces the

pervasive platform for ageing applications, including

its main hardware, software and middleware

elements. Section 4 illustrates the deployment of the

introduced architecture in the scope of realistic trial

settings. Finally, section 5 concludes the paper.

2 MOTIVATION AND RELATED

WORK

2.1 Application Requirements:

Memory Support for Elderly

People

“Ageing well”, namely enabling elderly people to

live longer and independently in their homes has

become a desirable objective, and AAL applications

can be instrumental in achieving it (European

Commission, 2006). Our solution focuses on

cognitive assistance and memory support in

particular. Existing research on the relationship

between elderly and technology carried out by

Burdick (2004), Hirsch (2000) and Zajicek (2005)

was a starting point for our design work. When

discussing requirements with user groups some

further important indications emerged:

• system should be as easy as possible to

understand and operate, while not appearing a

“dumbed down”;

• user interface should be accessible taking into

account physical and cognitive impairments;

• system should be unobtrusive;

• memory support application should be available

both at home and on the go;

• user should always be in control of the system

(and not vice versa): users appreciate gentle

reminders but they hate being told what to do.

As a general principle we can state that elderly

users are not keen on using a system “badged” as

designed for the elderly, no matter how useful its

features. This principle is challenging for designers,

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

106

as a trade-off is needed between attractiveness and

usability principles (e.g. accessibility and

simplicity). Another challenge, particularly

important in the case of pervasive system, is the

trade-off between integration with the environment

and unobtrusiveness. In the next section we discuss

the technical impact of those considerations.

2.2 Technical Challenges

Based on the above requirements, the technical

implementation of non-trivial systems for ageing-

well is associated with a host of technical

challenges. One of the primary challenges related to

the need for integrating a host of hardware, software

and middleware components, which are in most

cases distributed and heterogeneous. For in-door

environments, a non-obtrusive sensing infrastructure

along with perceptual signal processing algorithms

that extract the user’s context have to be integrated.

At the same time, mobile devices are needed in

order to support outdoor context acquisition for

roaming elderly users.

Another technical challenge relates to the

implementation and integration of cutting edge

signal processing algorithms that can credibly

extract the user’s context, e.g. as in (Pnevmatikakis,

2007). The integration of such algorithms (in a smart

spaces environment) can boost the required non-

intrusive nature of the system, since it obviates the

need for the more invasive tagged/tab based

approaches. Note that the algorithms to be used

must be robust in order to enable accurate context

acquisition enabling efficient recording of important

moments/situations, which facilitates memory

support. The system must also enable and support

modeling and tracking of situations within both

indoor and outdoor environments. Situation

modeling facilitates tracking of complex contextual

states beyond what single perceptual components

can provide.

The system should also blend different elderly-

oriented applications such as context-aware memory

support, cognitive training, as well as services

facilitating the elderly’s communication and

activation. This represents another technical

challenge that asks for flexible access to context and

data acquired by the system in order to adapt the

respective applications.

Later paragraphs review related pervasive

systems that address these challenges.

2.3 Related Work on Pervasive

Systems Architectures

For over a decade major pervasive and ubiquitous

computing projects have developed middleware

infrastructures facilitating components integration,

context-composition, coordination of devices,

orchestration of modalities and adaptation to

context, as well as development and deployment of

pervasive context-aware services. As prominent

example the Interactive Workspaces project at the

Stanford University (Johanson, 2002) has developed

the Interactive Room Operating System iROS

(Ponnekanti, 2003), which provides a reusable,

robust and extensible software infrastructure

enabling the deployment of component based

ubiquitous computing environments. Also, the

Oxygen project at MIT produced the robust

MetaGlue (Coen, 1999) agent infrastructure,

enabling agents to run autonomously from

individual applications so they are always available

to service multiple applications. MetaGlue has been

augmented with context-awareness features based on

the GOALS architecture (Saif, 2003). Other projects

such as EasyLiving have emphasized on the

coordination of the devices, as well as fusion of

contextual information. EasyLiving (Shafer, 2000)

focuses on computer vision technologies for person-

tracking and visual user interaction (Brumitt, 2000).

Also, there have been architectures that take into

account wearable systems and wireless

communications between components, towards

human centric applications, such as the architecture

of the Aura project at the Carnegie Mellon

University (Garlan, 2002). Aura monitors

applications and perform dynamic changes to it,

based on various resources and parameters such as

user mobility, changing user needs and system

faults.

Building on the basic principles and best

practices of these early projects, later projects such

as CoBRA (Chen, 2004) and later CHIL (Dimakis,

2008) have provided more versatile plug and play

architectures. For example, CoBRA enables the

integration of semantic structures (i.e. ontologies)

including the SOUPA standard ontology for

pervasive computing (Chen, 2004).

These projects lay out a set of basic middleware

principles for building complex heterogeneous and

highly distributed systems. However, they are not

customized to meet the elderly needs, in terms of

supporting multiple devices, supporting mobility,

providing ergonomic interfaces and integrating

cutting edge perceptual processing components for

HERMES: A PERVASIVE SYSTEM FOR MEMORY SUPPORT AND AMBIENT ASSISTED LIVING

107

non-trivial context acquisition. Our HERMES

system extends the above works across several axes,

including the integration with multiple user devices

(i.e. surface devices and mobile terminals), the

support for location-aware services, as well as the

integration of robust components for in-door person

tracking and activities of daily life detection. These

unique characteristics of the HERMES system are

elaborated in the following sections.

3 HERMES ARCHITECTURE

3.1 Overview

The HERMES architecture was designed to be

modular, flexible and interoperable.

Figure 1: The HERMES architecture.

It includes several technologies and accommodates

components for capturing and processing the user’s

context and to manage the user interaction with the

system, namely:

• Environmental hardware sensors (cameras,

microphones, etc.)

• Processing modules that extract conceptual

information (e.g.: a person’s identity entering

the monitored user area) from the above

sensors. Such modules process visual and audio

information and are able to generate a series of

events relating to the senior user’s context.

• Knowledge Module (Metadata Processing,

Semantic and Reasoning), which consists of

two major components namely: an ontology

model conceptualizing knowledge about the

users’ surrounding environment as well as a

rule engine for validating and interpreting

events as acquired by the sensor and processing

modules.

• Data Access Middleware, which is the major

component that allows the communication and

information exchange of the components. It

collects events from the processing modules

and validates them utilizing the Knowledge

base. It leverages the Web Services middleware

library in order to access information from the

low-level processing modules (i.e. perceptual

components).

• Content Repository, a multimedia storage for

managing raw content like video and audio,

along with a database that provides access to

both contents and events regarding the

cognitive-support applications.

• Application Layer (Cognitive Support

Applications) – composed by different GUIs,

specific to the kind of cognitive support

provided, and the related back-end. The GUIs

are meant to run on different devices such as:

standard PCs, touch-sensitive surfaces (for

Cognitive Games) and mobile devices.

Based on these components, the illustration of the

high level architecture presents also the main

connections supporting the integration workflow.

3.2 Sensing and Processing

Environment

The HERMES sensing and processing system

utilises audio-visual sensors either located in the

monitored space, or on the mobile device to capture

audio and visual signals from important aspects of

the life of the user. On the mobile device, the

microphone is used to record important

conversations when the user is not at the monitored

space. The device is placed between the speakers

and the audio recordings are hence medium-field.

These recordings are transcribed when the user

returns home, as they are transferred to the main

HERMES computers. Reverberation and noise in

such recordings render them difficult for

transcription. This is addressed by an advanced

algorithm for transcription detailed in (Aronowitz,

2010).

In the monitored space there is a microphone

array comprising seven microphones and two

cameras with overlapping field of view. This audio-

visual setup allows for tracking and recognition of

people in the monitored space. The perceptual

algorithms operating on these signals are split into

on-line (operating in real-time) and off-line.

The aim of on-line algorithms is to monitor the

state of the room and trigger recordings and

reminders. These are image-based 2D face tracking

(Pnevmatikakis, 2010) and face recognition

(Pnevmatikakis, 2009). A third algorithm can also be

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

108

on-line. This is 3D visual person tracking (Andersen,

2010), but since its performance deteriorates in

setups with just two cameras, it is only deployed in

the lab with four cameras and not in the space of the

elderly.

Off-line algorithms provide further processing of

the audio-visual signals. There are two such

algorithms. A 3D audio-visual tracker

(Pnevmatikakis and Talantzis, 2010) provides the

track of the active speaker and an estimate of the

presence of speech based solely on geometrical

criteria. Then, a beamformer utilises the 3D location

of the active speaker to improve on the audio

signals, generating something more suitable for

transcription. Our preliminary results with just two

microphones indicate a moderate improvement of

the word error rate.

3.3 Information Persistence and

Context Modeling

As already outlined the HERMES system provides

two main structures and associated data structures

for persisting information, namely an XML database

and an ontological knowledge base. The HERMES

XML Database ensures the persistence of

information in the system. It stores details that are

directly accessible by the user, for example the

shopping basket, and others automatically created by

the system, such as audiovisual recordings and

results of perceptual components. In order to ensure

that it can be extended to cover any future additions,

while at the same time facilitating data exchange, the

information is stored as XML data. This allows the

system to adapt to future changes, as the API for

accessing information will remain the same.

Furthermore, this also allows for backwards

compatibility with other modules, as any additional

information in the XML messages can be easily

ignored if the said module does not

recognize/support it.

On top of the XML Database runs the

Knowledge Base and Reasoning module; this is

responsible for combining the outputs of perceptual

components with information existing in the XML

Database, in order to deduce the system state and to

initiate the required actions. The Knowledge Base

comprises ontology models, which describe the

classes, properties and their taxonomies. An

ontology model can be defined and inserted to the

system either programmatically via a programming

API such as the Jena framework or by using one of

the available ontology editors (such as Protégé,

Swoop and OntoEdit) (Cardoso, 2007). In addition

to ontology models, the knowledge base comprises

reasoners (notably RacerPro, OntoBroker), which

allow the deduction of implicit knowledge by

processing the knowledge that has explicitly been

stated to the system.

3.4 Context Modeling

Context acquisition in HERMES is primarily based

on the processing of sensor streams, notably audio

and video streams captured from cameras and

microphones within the HERMES in-door

environment, as well as GPS sensor streams

stemming from the HERMES mobile devices (i.e.

PDAs). Audio and video streams are processed by

perceptual components (i.e. signal processing

technologies), which provide functionalities such as

detecting faces, identifying people/faces within

closed sets of people/faces, tracking people

movement within the smart room, as well as

identifying voice activity. The processing and

combination of their outputs can lead to a richer and

more sophisticated set of contextual information, for

example the situations associated with self-care,

identification of people walking in the smart spaces,

identification of situations associated with the

elderly domestic life, etc.

The HERMES context modeling and

identification approach relies on processing of the

output of multiple perceptual components in order to

compose and identify such more sophisticated

situations. To this end, the network of situations

approach emphasizing the combination of perceptual

components outputs on the basis of a graph of

situations (Soldatos, 2007) was integrated in the

system.

In terms of the timescale of the context

identification, the HERMES system distinguishes

between real-time and non real-time context

identification. Real-time (or even semi real-time)

identification of context hinges on combining

perceptual components output at fine timescales.

However, the HERMES system can also persist and

index information in the knowledge base (described

in the previous paragraph), with a view to reasoning

over contextual information in coarser timescales.

The latter approach leverages (historical)

information contained in the knowledge base and is

in principle a non real-time approach to context

processing and identification.

3.5 User Devices

The design of the end-user devices is based on two

guiding factors: (1) The special needs of elderly

HERMES: A PERVASIVE SYSTEM FOR MEMORY SUPPORT AND AMBIENT ASSISTED LIVING

109

users when interacting with a touch screen-based

device, and (2) the aim of reducing the complexity

of information retrieval tasks dealing with

potentially vast amounts of personal data.

HERMES comprises a number of applications

addressing different aspects of supporting episodic

and prospective memory. The main focus of the

mobile system is to give prospective memory

support while on the go and the desktop system is

the sole access point to recordings of past events due

the heavy processing requirements

3.5.1 Desktop

The desktop device provides the user with easy

access to six touch interaction-based applications.

The key applications Calendar (prospective memory

support), MyPast (episodic memory support), and

Cognitive Games (memory training) are

complemented by three applications to support and

extend functionality of their mobile counterparts, in

particular to take advantage of the larger touch

screen available on the desktop device. These

applications are Locations, People, and Shopping

List.

The Calendar (Figure 2) enables end users to

organise their future appointments and setting

custom reminders. The user has always the complete

overview of the application.

Figure 2: HERMES Calendar.

MyPast (Figure 3) allows retrieving of possibly

vast amounts of audio and video streams of events

that happened in the past. The specially designed

multi-touch-enabled interface aims at reducing

complexity of information retrieval process through

(1) an indirect search approach (Chau, 2008) and (2)

clustering of connected audio and video data within

so-called storylines. Indirect search is enabled by

multiple filters based on meta-data retrieved from

recorded material (time, people, and speech). The

concept of storylines is based on an event-based

segmentation of audio and video data. This data is

then allocated on a scroll- and zoom-able timeline of

events. The filters described above allow dynamic

inclusion or exclusion of events of the timeline.

Figure 3: MyPast – Setting a People filter.

Cognitive Games (Figure 7) offers three types of

cognitive games: First, the Maze game trains the

user’s memory of planned events by requiring him

or her to select two different appointment details

(such as its time and its description). Then, these

details, represented as blocks, have to be moved

through a maze simultaneously, training two-hand

coordination. Second, playing the Who-Is-Who

game trains person-related memory by matching

pictures of people with respective personal details.

Third, in the Puzzle game the user has to set together

scrambled puzzle parts of personally relevant

pictures of past events. Games can be played on the

Desktop or the Multi-Touch Surface (as described in

section 3.5.3)

The Locations application is complementing the

prospective memory support of the Calendar with

non-time-based, but location-based reminders. The

desktop application allows management of the

reminders for HERMES Mobile (see next section).

The People application (Figure 4) adds a third type

of prospective memory support. It allows access to

all people located in the database with a profile

picture either retrieved from video footage or taken

with the mobile device’s camera. It allows further to

set person-based reminders, i.e. reminders that are

not triggered by time or place, but the presence of a

given person in front of the HERMES cameras. As

an example, a user wants to be reminded to give

back a book borrowed from a friend, but forgets

when the person is present. This type reminder is

better associated with a person instead of time. As

soon as the person is visiting next time, a reminder

will be issued to give back the book.

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

110

Figure 4: HERMES People.

The Shopping List application (Figure 5)

complements memory support by helping users to

create shopping lists that can be read or changed

while on the go using HERMES Mobile.

Figure 5: Desktop Shopping List.

3.5.2 Mobile

The HERMES Mobile Application provides the user

with an advanced event manager and reminder

integrating geo-positional capabilities, people

browser, shopping list manager and, most important,

it is also the only sensor for outdoor scenarios,

supporting appointments dictation and conversations

recording.

Figure 6: HERMES Mobile Application: main menu.

The application runs on a PDA with touch screen

and built-in GPS device. This mobile device

provides an accessible and user-friendly interface for

elderly people. The user can type or dictate

appointments or notes. In supplementing the desktop

application MyPast, the mobile device provides

conversation support with the Conversations

application, turning the mobile device into an audio

sensor. The collected data is processed afterwards by

the system, upon synchronisation (typically when

the user comes home) as it is done for in-house

sensors. Specific features supporting particular

cognitive impairments have been added supporting

functionalities appearing in the desktop-based

applications.

3.5.3 Multi-touch Surface

A very low-cost multi-touch device has been

implemented to facilitate user interaction with the

system and the games. The hardware of the device

and its finger tracking algorithm are detailed in

(Petsatodis, 2009), while its use for the support of

effective ergonomic cognitive training for the

elderly is described in (Theodoreli, 2010).

The multi-touch device being used to play

cognitive games is shown in Figure 7.

Figure 7: Multi-touch surface of HERMES.

4 USER INVOLVEMENT AND

EVALUATION

4.1 Continuous User Involvement

Including the elderly users throughout all project

phases is of high importance for successful product

development (Demirbile, 2004). For HERMES, a

HERMES: A PERVASIVE SYSTEM FOR MEMORY SUPPORT AND AMBIENT ASSISTED LIVING

111

questionnaire, focus groups, cultural probes, diaries,

memory assessment and interviews were used for

assessing users’ needs. An initial mock-up was

evaluated with 8 users and led to additional

improvements and the development and deployment

of the first software prototypes that were

subsequently assessed under realistic conditions in

two trial sites in Austria and Spain with a total of 27

users. The system was personalized for each

(elderly) test-subject by populating HERMES with

personal data. Hence, each test subject was

confronted with his own data only. All trials were

carried out in a lab environment, yet the PDA

applications were tested in the field as well.

After an initial introduction participants carried

out a number of tasks with the system in order to

develop a feeling for it and its functionalities, while

also assessing usability issues. The study was carried

out in the presence of a researcher. This ensured

assistance for test-subjects (as needed) as well as

objective observation of test-subjects. The tasks

covered a wide range of scenarios, dealing with the

central use cases of the system (creating

appointments, setting reminders, searching for past

events, playing back videos etc.).

After completing the tasks, evaluation

questionnaires were filled in by the participants. The

questionnaires were kept consistent between the two

test locations to compare results and determine

differences based on cultural background.

After collecting feedback on usability and user

acceptance during the first user trials the interfaces

were further improved and extended.

4.2 Evaluation Research Questions

For the final evaluation of the integrated and

deployed prototype, the following three elements are

the focus of research:

1. User perception of performance of the

underlying components

2. Usability and user experience of the user

interfaces

3. Acceptance of the HERMES concepts for

cognitive support

The reason for performing this three-layered

evaluation is to be able to assess the origin of user

problems. This allows investigating if user

acceptance problems stem from concept rejection,

bad usability or just low system performance.

Performance evaluation focuses on the question

‘how good is “good enough” for the user’ for the

components underlying to the HERMES system, i.e.

which performance (e.g. speed, accuracy, and

relevance) of the various components is necessary

for the user to be a useful memory support.

The second research question focuses on

howwell the user experiences the HERMES system.

While usability problems and issues of interface

complexity were discovered to a large extent by

previous trials and heuristic evaluations, the focus of

the second trial is on user experience of the desktop

and mobile system as a whole, with an additional

focus on game experience of the cognitive games.

Furthermore, a learnability evaluation will assess

how elderly users are able to learn and improve their

individual use of HERMES according to speed and

subjective estimation.

The third layer targets the acceptance of the

HERMES technology and concepts and thus the

perceived benefits of the cognitive support provided

by HERMES. Existing technology acceptance

models, such as UTAUT (Venkatesh, 2003) have

proven to be inadequate for the context of ubiquitous

AAL technologies for various reasons, as argued by

Allouch (2009) and Arning (2009).:

• Context of use (no work-related context)

• Heterogeneity of the users (elderly people)

• Development status of the evaluated system

(prototype only)

These shortcomings are addressed by an

experimental evaluation design based on a proposal

by Allouch (2009). Specifically, rather than basing

acceptance on real usage, technology is assessed by

anticipating its adoption. The advantages of this

model are that technologies to be assessed do not

need to exist yet (or are in development phase) and

users do not have to have prior experience with

using them. Complementing this, pre- and post-

usage results are compared. The following

hypothesises are formulated:

1. The longer the duration of use, the higher the

Technology Acceptance (TA) of HERMES

2. The more previous use of technology, the

higher the TA of HERMES

3. The lower the memory functioning, the higher

the TA of HERMES

4. The higher the perceived user experience, the

higher the TA of HERMES

The first hypothesis will be assessed during the

home evaluation (see below). The second hypothesis

assumes that prior knowledge of general technology

will lower entrance barriers of AAL technology.

The third hypothesis stems from the question if users

with more memory problems will see higher

usefulness in the application. Finally, the forth

hypothesis seeks to confirm the influence of

emotional experience aspects during usage on the

overall acceptance of the system.

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

112

4.3 Evaluation Procedure

As time of writing, the final user evaluation is about

to be performed with a total of 59 participants in

Austria and Spain. 20 of those are evaluating

HERMES concepts and performance of underlying

the components. Usability and user experience

evaluation takes place not only in the lab (13 users

in Spain and 18 Users in Austria), but also in 8 real

homes of people for a period of two weeks per

home. The evaluation includes to a certain extent the

‘extended home’, which includes the people in the

direct surrounding of the study participant. Besides

interacting with the system as a single user, the

social interaction and context of interaction with

technology is assessed.

The evaluation procedure follows the three

evaluation layers described above: it will assess

performance of the system with a set of tasks

varying the performance of key HERMES

components (such as filters based on person

identification and transcription of speech). User

feedback on these varying conditions will be

gathered. Furthermore, user experience of the

system as a whole will be evaluated with a set of

tasks comparable to the ones used in the first trials.

Finally, the HERMES concepts are assessed using a

presentation of HERMES scenarios followed by the

adapted technology acceptance questionnaires.

5 CONCLUSIONS

Ambient assisted living and ageing well are

privileged fields for deploying pervasive context-

aware systems. Despite the proliferation of pervasive

system deployments, the challenges associated with

the elderly needs are not fully addressed in the scope

of legacy pervasive systems architectures. Such

challenges include the need for interacting with

users with multiple modalities and in a non-intrusive

way. Recent advancements in pervasive systems

render them appropriate for tacking these challenges.

In this paper we have illustrated a novel

pervasive system supporting non-trivial assistive

functionalities for elderly users, with a focus on

alleviating the cognitive decline. The system relies

on the integration of a wide array of components,

ranging from sensors and perceptual signal

processing algorithms, to touch devices and related

ergonomic software applications. Despite the

system’s complexity, early feedback from trials has

been very promising. Future work should emphasize

on enhancing deployment flexibility and ease, while

at the same time lowering the total cost of ownership

of the system. Furthermore, the complex task of

information retrieval needs continuous effort to ease

understanding of operational concepts for the very

heterogeneous group of elderly users. Such work

could serve as a basis for moving the next generation

of such assistive systems to the enterprise, as a

follow on to early (simpler) systems outlined in

(Stanford, 2002).

ACKNOWLEDGEMENTS

This work is part of the EU HERMES project (FP7-

216709), partially funded by the European

Commission in the scope of the 7

ICT Framework.

The authors acknowledge valuable help and

contributions from all partners of the project.

REFERENCES

Weiser, M., 1991. The Computer for the 21st Century.

Scientific American, 265(3).

Stanford, V. M., 2002. Pervasive computing: Applications

- using pervasive computing to deliver elder care.

IEEE Distributed Systems Online, 3.

Soldatos, J., Dimakis, N., Stamatis, K., and Polymenakos,

L. A Breadboard, 2007. Architecture for Pervasive

Context-Aware Services in Smart Spaces: Middleware

Components and Prototype Applications. Personal

and Ubiquitous Computing Journal, 11(3). Springer.

Mylonakis, V., Soldatos, J., Pnevmatikakis, A.,

Polymenakos, L., Sorin, A., and Aronowitz, H., 2008.

Using Robust Audio and Video Processing

Technologies to Alleviate the Elderly Cognitive

Decline. In: Proceedings of the 1st International

Conference on Pervasive Technologies Related to

Assistive Environments. Athens, Greece.

Dimakis, N., Soldatos, J., Polymenakos, L., Cuřín, J.,

Fleury, P., Kleindienst, J., 2008. Integrated

Development of Context-Aware Applications in Smart

Spaces. IEEE Pervasive Computing.

Waibel, A., and Stiefelhagen, R., 2009. Computers in the

Human Interaction Loop. Springer. London, 2

edition.

Petersen, R. C., Stevens, J. C., Ganguli, M., Tangalos, E.

G., Cummings, J. L., and DeKosky, S. T., 2001.

Practice parameter: Early Detection of dementia: Mild

cognitive impairment (an evidence-based review).

Report of the Quality Standards Subcommittee of the

American Academy of Neurology.

Petersen, R. C., Doody, R., Kurz, A., Mohs, R. C., Morris,

J. C., Rabins, P. V., Ritchie, K., Rossor, M., Thal, L.,

and Winblad, B., 2001. Current concepts in mild

cognitive impairment. Archives of Neurology, 58 (12).

Johanson, B., Fox, A., Winograd, T., 2002. The Interactive

Workspaces Project: Experiences with Ubiquitous

HERMES: A PERVASIVE SYSTEM FOR MEMORY SUPPORT AND AMBIENT ASSISTED LIVING

113

Computing Rooms. IEEE Pervasive Computing

Magazine, 1(2), pp.67-75.

Ponnekanti, S. R., Johanson, B., Kiciman, E., and Fox, A.,

2003. Portability, Extensibility and Robustness in

iROS. In Proceedings of the IEEE International

Conference on Pervasive Computing and

Communications. Dallas-Fort Worth, TX. March

2003.

Coen, M., Phillips, B., Warshawsky, N., Weisman, L.,

Peters, S., and Finin, P., 1999. Meeting the

Computational Needs of Intelligent Environments:

The Metaglue System. In: Proceedings of the 1st

International Workshop on Managing Interactions in

Smart Environments. Dublin, Ireland, 1999.

Saif, U., Pham, H., Mazzola, J. Paluska, Waterman, J.,

Terman, C., Ward, S., 2003. A Case for Goal-oriented

Programming Semantics. In:Proceedings of the System

Support for Ubiquitous Computing Workshop at the

5th Annual Conference on Ubiquitous Computing.

Shafer S., et al., 2000. The New EasyLiving Project at

Microsoft Research. In Proceedings of the 1998

DARPA/NIST Smart Spaces Workshop.

Brumitt, B., Krumm, J., Meyers, B., and Shafer, S., 2000.

Ubiquitous Computing and the Role of Geometry.

IEEE Personal Communications.

Garlan, D., Siewiorek, D., Smailagic, A., and Steenkiste,

P., 2002. Project Aura: Towards distraction-free

pervasive computing. IEEE Pervasive Computing, pp.

22-31.

Pnevmatikakis, A., Soldatos, J., Talantzis F., and

Polymenakos, L., 2007. Robust Multimodal Audio-

Visual Processing for Advanced Context Awareness in

Smart Spaces. Personal and Ubiquitous Computing

Journal. Springer.

Cardoso, J., 2007. The Semantic Web Vision: Where are

We? IEEE Intelligent Systems, 22(5), pp. 84–88.

Chen, H., et al., 2004. Semantic Web in in the Context

Broker Architecture. In: Proceedings of the Second

Annual IEEE International Conference on Pervasive

Computer and Communications. IEEE Press.

Chen, H., et al., 2004. SOUPA: Standard Ontology for

Ubiquitous and Pervasive Applications. In:

Proceedings of the International Conference on

Mobile and Ubiquitous Systems: Networking and

Services. IEEE Press.

European Commission, 2006. i2010: independent living

for the ageing society. Office for Official Publications

of the European Communities. Luxembourg 2006.

Burdick, D. C., and Kwon, S., 2004. Gerotechnology:

Research and practice in technology and aging.

Springer, New York.

Aronowitz, H., 2010. Unsupervised Compensation of

Intra-Session Intra-Speaker Variability for Speaker

Diarization. In: Proceedings of Odyssey 2010.

Pnevmatikakis, A., Katsarakis, N., Chippendale, P.,

Andreatta, C., Messelodi, S., Modena, C., and Tobia,

F., 2010. Tracking for Context Extraction in Athletic

Events. In Proceedings of the International Workshop

on Social, Adaptive and Personalized Multimedia

Interaction and Access. ACM Multimedia, Florence,

Italy, Oct. 2010.

Pnevmatikakis, A., and Polymenakos, L., 2009. Subclass

Linear Discriminant Analysis for Video-Based Face

Recognition. Journal of Visual Communication and

Image Representation, 20(8), pp. 543-551.

Andersen, M., Andersen, R., Katsarakis, N.,

Pnevmatikakis A., and Tan, Z.H., 2010. Three-

dimensional adaptive sensing of people in a multi-

camera setup. Special session on Person tracking for

assistive working and living environments, EUSIPCO

2010, Aalborg, Denmark, August 2010.

Pnevmatikakis, A., and Talantzis, F., 2010. Person

tracking in enhanced cognitive care: A particle

filtering approach. Special session on Person tracking

for assistive working and living environments,

EUSIPCO 2010, Aalborg, Denmark, August 2010.

Petsatodis, Th., Talantzis, F., Pnevmatikakis, A., and Diaz,

U., 2009. Interactive surfaces for enhanced cognitive

care. In: Proceedings of the 17

International

Conference on Digital Signal Processing. Santorini,

Greece, July 2009.

Theodoreli, V., Petsatodis, T., Soldatos, J., Talantzis, F.,

and Pnevmatikakis, A., 2010. A Low-Cost Multi-

Touch Surface Device supporting Effective

Ergonomic Cognitive Training for the Elderly.

International Journal of Ambient Computing and

Intelligence, 2(3), pp. 50-62, July-September 2010.

Allouch, S.B., Van Dijk, J.A.G.M., and Peters, O., 2009.

The acceptance of domestic ambient intelligence

appliances by prospective users. LNCS 5538, pp.77-

94, 2009.

Arning, Z., and Ziefle M., 2009. Different Perspectives on

Technology Acceptance: The role of technology type

and age. LNCS 5889, pp. 20-41, 2009.

Chau, D. H., Myers, B., & Faulring, A., 2008. What to do

when search fails: finding information by association.

In Proceedings of CHI '08, pp. 999-1008.

Venkatesh, V., Morris, M.G., Davis, F.D., and Davis,

G.B., 2003. User Acceptance of Information

Technology: Toward a Unified View. MIS Quarterly,

27, 2003, pp. 425-478.

Demirbilek, O. and Demirkan, H., 2004. Universal

product design involving elderly users: A participatory

design model. Applied Ergonomics, 35(4), pp. 361-

370, 2004.

Hirsch, T., Forlizzi, J., Hyder, E., Goetz, J., Stroback, J.,

and Kurtz, C., 2000. The ELDer Project: Social,

Emotional and Environmental Factors in the Design of

Eldercare Technologies, 2000. ACM Conference on

Universal Usability, 2000, pp. 72–79.

Zajicek, M., 2005. Older adults: Key factors in Design. In

Future interaction design. Springer. London, 2005.

ICSOFT 2011 - 6th International Conference on Software and Data Technologies

114