Jarmo Kalaoja, Julia Kantorovitch
VTT Technical Research Centre of Finlands,Kaitoväylä , P.O Box 1100 FI-90571, Oulu, Finland
Sara Carro, José María Miranda, Álvaro Ramos
Telefónica I+D, S.A. Parque Tecnológico Walqa, Huesca, Spain
Jorge Parra
Ikerlan Technological Research Centre, P.O.Box 146, 20500 Arrasate, Gipuzkoa, Spain
Keywords: Home networked services, service discovery, semantic modelling, requirements analysis, ontology
Abstract: With great advance in information technology and broadband networks, the interconnected networked home
devices are becoming increasingly popular. Number of heterogeneous networked devices and services
which belong to the traditionally separated functional islands such as PC (i.e. Internet), mobile, CE
broadcasting, and home automation, not working together can be found in our today’s home. Merging of
these devices and services would offer home individual residents user-friendly, intelligent, and meaningful
interfaces to handle home information and services. The semantic ontology based modelling of home
services can enable interoperability of heterogeneous services. The ontology may facilitate clear description
on how far each device is suitable for different kinds of information and different interaction demands. This
paper is presenting an analysis on the kind of vocabulary ontologies necessary in different functional
domains to cope with heterogeneity of service descriptions. The ontology based rich representation of
services will facilitate an efficient service discovery, integration and composition.
Merging of devices and services from the
traditionally separated domains presented in the
home would offer home individual residents user-
friendly, intelligent, and meaningful interfaces to
handle home information and services. To achieve
this, the heterogeneous networked services in the
home, where a wide variety of heterogeneous
networks, devices and software infrastructures,
coexist, should be discovered, integrated and
composed in a ubiquitous and seamless manner to
address the user requirements.
In Amigo project (Georgantas, 2005) we adapt
the semantic modelling approach for modelling of
the services to enable efficient automated execution
of the aforementioned tasks. In service oriented
software architectures (TINA, 1997) (WS, 2004),
service modelling is usually performed by the
operations that a service provides and requires from
other services in the architecture. Thus the Amigo
services are modelled based on the number of
required and provided capabilities. Each capability
specifies a number of inputs and outputs. The
capabilities along with their inputs and outputs can
be used for the service discovery process (i.e.
capabilities/requirements matching). Some other
properties of the service, often referred as non-
functional properties, such as quality of service
(QoS), context in which a service is executed, and
preferences specified in user profiles may assist to
dynamically select the services that best meet the
user needs.
In order to allow a rich representation of services
and thus facilitate efficient service discovery and
composition, functional capabilities, inputs, outputs
Kalaoja J., Kantorovitch J., Carro S., María Miranda J., Ramos Á. and Parra J. (2006).
In Proceedings of the Eighth International Conference on Enterprise Information Systems - ISAS, pages 461-466
DOI: 10.5220/0002487004610466
and non-functional attributes of the services can be
further semantically annotated using external
vocabulary ontologies. The use of ontologies enables
computational entities and services to have a
common set of concepts and vocabularies for
representing knowledge about a domain of interest,
while being able to interact with each other. By
using such ontologies, the relationships between
entities can be more clearly expressed and these
allow for better reasoning on their properties.
Ontologies are also beneficial for the re-usage of
knowledge, as several ontologies from various
sources can be integrated to describe the specific
domain. Finally, the deployment and customization
of such systems as Amigo in any home can be
considerably facilitated by using a common set of
concepts and vocabularies developed for the
networked home environment. The integration of
third-party applications and services can be much
eased using clear ontology classifications.
This paper elaborates on the vocabulary
ontologies for the semantic modelling of home
services. In the following, one possible realization
of home system is visualized in Section 2. Here the
approach and steps have been used to identify the
required vocabularies and classifications are
outlined. The initial analysis and results for different
functional domains (mobile, PC, domotic, CE) are
presented in Section 3. Section 4 concludes the
paper providing an overview of the current
contribution and the future work. As the work on
domains analysis and definition of different
ontologies is in progress, the objective of this paper
is to share and validate the approaches used, to
discuss the initial results obtained, and to verify the
directions for the future work.
The intensive user research (Rocker, 2005)
performed in Amigo project gives some ideas on
how the home system could interact with people in a
service-rich home environment. Based on this
research, a home system (we call it Amigo system)
can be visualized as handling a number of separate
aspects or application domains of the intelligent
home, demonstrating various properties and
functionalities that support the persons living in the
The Amigo system (in role of supporting gaming
and entertainment domain) can start a user's day
with playing music from preferred play list, showing
the personalized news or general summaries of hot
news topics. The content (music, film, TV/radio
program or game) can follow the user everywhere in
the house or even outside the home area using a
mobile or a portable device. It helps to select games
to play and supports the usage of various displays in
the home environment during the game session. It
downloads personal profiles and integrates game
devices when friends are coming over to the user’s
house for interactive multi-player game sessions.
The Amigo system (in role of supporting
household task assistance domain) can select the
correct settings for home appliances. It may even
know how to detect the presence of inappropriate
objects inside or near the appliances. It downloads
recipes and cooking programs to the kitchen and
displays them to facilitate the food preparation
The Amigo system (in role of supporting
ambience assistance domain) can configure the
home environment during game sessions, or while
the user is watching TV by adapting the light, sound
and video features/levels throughout the room. It
can adapt the home environment and create an all-
surround audio, light and video experience.
Amigo system cannot depend on the availability
of any specific set or types of devices in handling
these aspects. In order to support this, we need to
introduce an engineer role into the Amigo system
that supports higher level Amigo applications and
hides the necessary underlying technologies.
Examples of the support for this role can range from
controlling specific appliances to downloading
content to a PDA device. This "Engineer" role is
handled by the Amigo Intelligent services, and a
number of other services that are dynamically
registered using the service discovery functionality
of various appliances located in the Amigo house
and composed by the Amigo system when
The aforementioned scenes have been analysed
to gather the requirements for various domain
ontologies and vocabularies to cope with the
heterogeneity of services descriptions (i.e.
capabilities, inputs, outputs) and address the
requirement specifications. As an example: "John
and Robert start to play an interactive multi-player
adventure game in John's living room. The Amigo
system adjusts the ambience for the game experience
by controlling the lights and sound. Robert notices
that it is late so he has to leave, but he continues to
play the game under way..." Some examples of the
vocabularies related to the various Amigo domains
in this scene are:
From positional information and/or gesture
recognition output Amigo User Interfaces detect that
John and Robert want to play. Upon activation of
the game, the system queries the capabilities for
PositionControlInput User Interface Services provided
by the Amigo User interface. When controlling the
ambience in a room the Amigo system queries
services providing the domotic domain
and LightDetectors capabilities available in the
context of that room. TV monitor, when plugged in,
provides CE domain capability of
DigitalMediaRenderer for AVStream.
To be able to support the continuation of the
gaming session, John's Amigo system checks the
DeviceContext of Roberts PDA to match a most
suitable mobile client technology provided by game
console and uses the PDA’s
capability to load a game into it.
Other supporting services that are not that visible
in the example above are seamlessly involved. For
example, context management service will be used
to provide user location. The Amigo system may
ask the User modelling and profiling service for
relevant user context and preferences about favorite
adventure game to be automatically downloaded.
Different levels of ontologies (see Figure 1) can
be envisaged to model the entire Amigo home
environment appropriately in order to address the
service/application developer’s and user’s
requirements for efficient service discovery,
composition and invocation.
- Performance
- Security
- Privacy
- Physical
CE / PC / Mobile Domotic Devices
Data /
HW/ SW/ Communication Platforms
Abstract service Description
User Modeling
and Profiling
Figure 1: Different levels of services and ontologies.
To verify and validate this approach, the
following iterative steps have been used in order to
identify the required vocabularies and classifications
on which the ontologies will be based (Noy, 2001):
Step 1:
We gathered Amigo requirements for the
various domain ontologies. Some domain-specific
examples, based on user scenarios that can help to
determine what types of information are required
and what types of query support the ontology should
provide have been identified.
As an example, the mobile domain information
and services are useful in such cases as
communication (call, SMS), notification (alerts
about some events in the home), and entertainment
on the way. Examples of mobile platforms used in
home are a mobile phone, PDA, laptop, etc.
The PC domain is related to the “classical” view
that we have about computers. Personal computers,
Web cameras, peripherals such as printers, scanners,
etc, are included in this domain. The role of PC
domain is mainly related to storing, accessing and
processing information.
Analysis of the CE domain concerns audio,
video and other entertainment devices presented in
the home. The interaction of home individuals with
this domain mainly takes place during their leisure
time, thus making the CE domain a very attractive,
actively utilized and potentially profitable one.
Plenty of CE devices such as HiFi systems, TV
screens, video consoles, sound speakers, etc, can be
found in our homes. Especially the entertainment
content that is displayed and exchanged between
those CE devices is quite attractive and important
from the user’s point of view. This content includes
songs, films, still photos, etc, and can be seen as
indispensable attribute of the CE domain. In
addition, a many of the data linked to this content
(such as size, bit rate, and parental permissions) can
affect the user’s experience directly or indirectly.
The domotic domain is related to identifying the
requirements of home automation devices to Amigo
system. Some examples of such devices could be a
lighting system, a washing machine, a gas sensor,
etc. Energy cost savings, security of persons and
goods, comfort improvement are some of the
benefits of home automation. The role of domotic
domain is quite critical as it is also associated with
home care and safety. Safety and security of persons
and goods is increased with functions such as
detection of fire or gas leakage, tele-transmission of
alarm, or detection of intrusion.
Step 2
: The numerous available sources of
information that may provide classifications and
ontologies have been considered (FIPA, 2001,
FIPAAV, 2001, OMA, 2005, CCPP, 2004,
TvAnytime, MPEG7, 2004, MPEG21, 2002,
Shimizu, 2005, UPnP, 2003, DLNA, 2005). The
above enumerated sources provide suitable
information to model device capabilities to be useful
in adaptation of content information to particular
device mode, however they lack in providing service
functional descriptions that would allow the
automatic selection of particular devices based on
the services available, through the service capability
matching. Moreover, most of information is mobile
domain oriented and might give only some ideas for
service modelling in other domains. As regarding to
domotic domain, due to the great heterogeneity of
devices that can be found in the domain, there is no
available classification covering the entire home
automation world. Therefore the Amigo
classifications can be a starting point in modelling of
domotic information.
Step 3:
Here some draft lists of information (i.e.
vocabularies) for different domains based on various
sources enumerated have been created
Step 4:
Analysis and classifications. In this step
we analyse and filter the lists obtained from step 3
against the requirements from step 1 to identify
information needed to support the Amigo scenarios.
We are in the process of identifying the required
ontologies and relations between them for describing
the information of a particular domain. The
preliminary taxonomies and class hierarchies for the
ontology are initially defined and presented in the
The following set of taxonomies for different
functional domains (mobile, PC, domotic, CE) can
be identified for further analysis:
Taxonomy for the typical functional
capabilities of devices, provided as
software services
Taxonomy of functional capabilities of
software services which provide the
intelligence support in the home
Taxonomy for devices and platforms that
can be used to define device context and
facilitate the adaptive service discovery
Taxonomy of content including multimedia
When classifying the functional capabilities, we
focus on those capabilities of devices and services
that can be used for service discovery and further
service composition in Amigo. The various aspects
identified in the example are reflected in the
classification. Most of the analysed capabilities are
closely associated with the control of devices
discoverable in Amigo home (the engineering
aspect), but classifications that help modelling of
services associated with higher level aspects
supported by Amigo home (e.g. gaming and
entertainment domain support) are also analysed.
Figure 2 gives an idea of functional capabilities
class hierarchies in Amigo. The high level
FunctionalCapability of services found in Amigo
home can be described using three application
domains presented in Amigo home. These are
ExtendedHomeSupport, HomeCareAndSafetySupport, and
HomeInformationAndEntertainment. Naturally this level
of hierarchy can be easily extended if services
supporting other application domains functionality
will be developed in the future. Accordingly lower
level class hierarchies present the capabilities of the
services from four functional domains (mobile, PC,
domotic, CE) and intelligent user services which
support at least one of the above enumerated
application domains. In essence, the communication
with users (
UserComminication) by calling and
messaging is an inherent capability of the mobile
domain. In the PC domain which is much
overlapped with mobile domain, this may
additionally involve means for user
AudioVideoCommunication through the
Videoconference. MultimediaApplicationSupport may
be described by capabilities providing control to AV
devices in home, and
GameApplicationSupport by
support for game downloading and user
communities (not presented in the picture).
IntelligentHomeSupport can be viewed as the glue,
expanding through all application domains and
described by a number of services, which
capabilities support
UserIntefaceServices with devices
(GUI, speech, gesture services), provide
IntelligentSoftwareSupport through e.g.
ContextManagement, UserModellingAndProfilling, and
other intelligent services and control home devices
HomeDeviceControl). The Functional Capability
class hierarchies can be easily extended by service
developers with respect to application domains
possibly present in any future homes and also with
respect to particular domain of interest. For example,
HomeInformationAndEntertainment can be further
extended by
MusicNewsApplicationSuppport, etc. Accordingly
HomeCareAndSafetySupport will be further extended
in our future work with services capabilities specific
to devices from domotic domain such as e.g.
ControlLight, ControlBlinds, ControlTemperature, etc.
Figure 2: Partial class hierarchies of functional capabilities.
As previously mentioned, the device capability
profiles provide background for the devices and
platform taxonomy. A common problem with the
various standards considered is that they mostly
describe only the variable set of capabilities of a
device leaving out the standard set of capabilities for
the underlying platform technology. We think that
such assumptions should be explicit. As an
example, a partial draft of low level ontology based
on mobile platform taxonomy is illustrated in Figure
Figure 3: A mobile platform Taxonomy.
The proposed taxonomy is based on various
device property descriptions but is more generic in
order to cover also implicit information left out of
those classifications. Again, each class in this
taxonomy is a specialisation of its parent classes. For
clarity, the taxonomy is restricted to three levels in
which the first level defines a role of the platform,
the second is a sub-classification of platforms
provided in this role and the third level is usually a
specific technology with possibly several instances.
The vocabulary can describe for example the
communication properties of a mobile platform, or
classes of mobile devices with typical platform
properties. A mobile phone model can be specified
as an individual of class
MobilePhone with specific
properties. Similar ontologies can be created for the
devices and platforms from other functional domain
(PC, domotic, CE). The integration of those is part
of our future work.
The taxonomy for content is related to input and
output information provided by services and can be
used for the purpose of service selection/matching.
The types of content in mobile domain are related to
CE domain. Typical to these domains is that many
applications are fixed in device but others, like Java
games, are downloaded as content to device. Typical
application data e.g. in mobile and PC domains is
related to contact and calendar information:
Game / Calendar / Phonebook / CameraImaging
Bookmark / CalendarEntry / ContactInformation
Audio / Image / Video / Text
Stream / Still
QoS aspects in the service discovery may rise
some questions related to the format of used
multimedia content in particular situation/context,
i.e., which format for this content is the best suitable
for the current state of the network. Furthermore,
some capabilities of content-related services may
only apply to certain content formats or media. Thus
some explicit relation to these ontologies is
necessary. Device capabilities in CE are essentially
described by the way in which they behave in
relation to content. It is not our purpose to define a
content description language but rather to provide a
bounded taxonomy that will enable the modelling of
capabilities under the
class presented above. Therefore, after analysing the
aforementioned classifications and scenario
requirements, an essential classification may be:
ContentSize / Bitrate / NumberOfChannels
SpatialResolution / TemporalResolution
ADPCM / CS-ACELP / G.711 / MP3 / OGG / PCM /
WAV / Mono-Stereo
Plain / RTF
AVI / DivX / MPEG2 / MPEG4 / WMV / XVid /
A separate taxonomy for QoS characteristics is
also considered, but not yet elaborated, for mobile
and PC domains. This taxonomy is related mostly to
service usability because of restrictions of mobile
devices, selection of most suitable communication
channels, and determination whether a specific
content can be presented on a particular device.
This paper has presented the requirements analysis
and the initial results on ontology vocabularies and
classifications modelling for the services and
devices capabilities from four functional domains
(mobile, PC, domotic, CE) presented in the home
In our future work we are planning to iteratively
refine and extend the existing classifications with
generic services and devices capabilities found from
four functional domains and also with services
capabilities developed in duration of Amigo project
towards the support of the demonstrations for three
application domains of Home Information and
Entertainment, Home Care and Safety, and Extended
Home. Special attention will be paid to the
description of domotic devices and services with
their possible integration with domotic information
on devices states and events as most critical ones in
developing of services for the Home Care and Safety
application domain.
It is also planned to elaborate the ontology on
QoS related aspects that will contain the devices
related capabilities and the content related
information (the initial results are presented in this
research), and also network related QoS parameters
which would allow to tackle the interoperability
issues between different QoS schemas (e.g. UPnP
and RSVP) presented around the home.
We would like to thank IST, the European
research program. Amigo (IST project 004182) is a
European project in the Ambient Intelligence and
Pervasive Computing vision.
CCPP, 2004. Composite Capabilities / Preference Profiles
Deliverable, 2005. Amigo Consortium. Deliverable D1.1:
Report on user research
DLNA, 2005. DLNA information site.
FIPA, 2001. FIPA Device Ontology Specification. 2001.
FIPAAV, 2001. FIPA Audio/Visual Entertainment and
Broadcasting Specification
Georgantas, 2005. Georgantas, N., Ben Mokhtar, S.,
Bromberg Y.-D., Issarny, V., Kalaoja, J.,
Kantorovitch, J., Gérodolle, A., Mevissen, R. The
Amigo Service Architecture for the Open Networked
Home Environment. Short paper. In Proceedings of
5th Working IEEE/IFIP Conference on Software
Architecture (WICSA).
Noy, N., McGuinness, 2001. D.L. Ontology development
101: A Guide to Creating Your First Ontology.
Technical report, SMI-2001-0880
MPEG7, 2004. MPEG-7 Overview.
MPEG21, 2002. MPEG-21 Overview.
OMA, 2005. OMA User Agent
Rocker, C., Streitz, N., Jance., M, Portolan., N., 2005.
User Requirements for intelligent Home
Environments: A Scenario-Driven Approach and
Empirical Cross-Cultural Study. In SOC-EUSAI 2005
Smart Objects & Ambient Intelligence
Shimizu, N., Kitagawa, K., Wan, H., Ishikawa, N., Hjelm,
J., 2005. A Device and Service Description
Framework for Discovering and Reasoning in
Autonomous P2P Environment. MobEA III - Customer
Focused Mobile Services
TvAnytime. TV-Anytime Forum.
TINA, 1997. Service Architecture Specification.
UPnP, 2003. The standardized device and service
WS, 2004. Web Services Architecture, W3C Working
Group Note 11 February 2004.