USING MPEG-BASED TECHNOLOGIES FOR BUILDING
PERSONALIZED MULTIMEDIA INTERACTIVE
ENVIRONMENTS WITH CONTEXT-AWARENESS
REQUIREMENTS
Development of an application for Interactive Television
João Benedito dos Santos Junior, Iran Calixto Abrão and Thelma Virgínia Rodrigues
Pontifical Catholic University of Minas Gerais – PUC Minas, Poços de Caldas – MG, Brazil
Mario Guglielmo
Multimedia Division, Telecom Italia Laboratory, Turim, Italy
Keywords: New MPEG Standards; Interacti
ve Video and Audio; Context-Awareness
Abstract: We are using MPEG-4 technology to build applications to be used in real environments. One of these
applications allows for teacher to send real-time lessons to this/her students or to record them. The Tele-
Learning System under development includes: a) on the teacher side: a recording workstation with two
cameras, microphone, specific MPEG-4 software; b) an IP network or an MPEG-2 TS satellite link; c) on
the student side: a PC with special MPEG software, and a special board if receiving from satellite. This
research focuses on the broadcast scenario where a satellite board is used in a PC. Thus, the work covers
how to send the lesson even to a student that is not connected to the intranet, using a satellite link, either
over IP embedded in the MPEG-2 TS or directly over MPEG-2 TS. For the security part it may be necessary
to have a low-band return channel implemented, for example, through a mobile phone. The satellite
environment may require the redesign of the User Interface and the retargeting of the elementary streams
parameters in order to match specific requirements and features of the medium. At this point, new
interaction criteria have been established from distribution of MPEG-4 media objects and MPEG-7 scene
descriptions on network environments. Furthermore, context-awareness aspects are being added for
providing personalization on the teaching-learning environment and MPEG-21 is being studied for applying
to new multimedia requirements
.
1 INTRODUCTION
Universities (and high schools) and commercial
enterprises (manufacturers, service providers, ICT
companies, among others) generate an increasing
demand for more efficient means and supports to
teaching, training and upgrading activities. In the
different environments “efficient” has different
meaning: from the reduction of time required by a
student to get the subject of an academic course to
the minimization of training costs and unproductive
hours for the upgrading of a company personnel. In
this context, “e-Learning, “Distance Learning”,
“Corporate Training” are few of the names used to
indicate these kinds of services in several
combinations with the “multimedia” attribute.
It is widely accepted that the answer to this
dem
and can be provided by the new technologies,
such as MPEG-4 for media coding and streaming
and MPEG-7 for information description and
indexing (MPEG4, 2000a; MPEG-4, 2000b; MPEG-
7, 2000).
The common understanding is that by means of
th
ose technologies it will be possible to improve the
quality, at the moment often quite poor with respect
to content media, of the multimedia courses. The
way in which the content is selected, merged and
organized is the other key issue that imposes the
creation of new professional characters: the
180
Benedito dos Santos Junior J., Calixto Abrão I., Virgínia Rodrigues T. and Guglielmo M. (2005).
USING MPEG-BASED TECHNOLOGIES FOR BUILDING PERSONALIZED MULTIMEDIA INTERACTIVE ENVIRONMENTS WITH CONTEXT-
AWARENESS REQUIREMENTS - Development of an application for Interactive Television.
In Proceedings of the Seventh International Conference on Enterprise Information Systems, pages 180-183
DOI: 10.5220/0002526301800183
Copyright
c
SciTePress
instruction architects. Easy “navigation” of the
material and the actual possibility of accessing the
relevant piece of information by “few” clicks are
very important usability aspects that can determine
the spreading of the service and its success or, on the
contrary, its failure.
Availability and accessibility are important
aspects to consider beside the costs. Distance
learning platforms require suitable delivery media
solutions that can be based on both
telecommunication and broadcast infrastructures and
the different hypotheses are in principle able to
satisfy different requirements and could be a further
interoperability issue (Battista et al., 1999). The
management and protection systems proposed so far
have not been able to protect anything for long time
and new flexible and dynamically adaptable
solutions are required.
2 THE SYTEM ARCHITECTURE
In recent times, the growing demand of e-learning
systems brought to the choice of standard
multimedia technologies, to allow learning contents
being compatible with many other systems and
reaching flexibility and reusability.
In this context, MPEG-4 was adopted; MPEG-4
features make this standard really suitable for e-
learning applications; the main reasons are (MPEG-
4, 2000a; MPEG-4, 2000b): a) the compression ratio
reached by MPEG-4 Audio and Video encoders
allows to achieve good audio-video quality; b)
MPEG-4 Systems provide the splitting of the audio-
visual signals in elementary, synchronized objects;
c) a high security level is achieved, and it is
guaranteed by a key, and not by algorithms.
The system represented in Figure 1, is composed
of several parts: a) didactical sources: audio, video
(two analog or digital video sources) and teacher
slides; b) one or more PC (Control Stations) that
encode, compress and possibly encrypt the learning
elementary media; finally there are two software
modules, a “streamer”, that sends on different
channels (IP multicast, satellite, among others) and a
recorder that locally stores, in real-time, the whole
MPEG-4 scene; c) the network connection; d) a PC
for each student: it receives, decrypts and presents to
the user the multimedia interactive lesson.
We present now the system details for the
description of software modules, divided into three
categories, described separately: a) modules for the
creation of the real-time lesson; b) modules for the
network distribution; c) modules for the fruition of
the lesson.
Teacher
Students
Control
station
Network
Satellite
Figure 1: Diagram of the system.
The system tries to reproduce all these actions:
this is reached making use of both local and remote
interactivity from the teacher side and from the
student side.
3 FRUITION SYSTEM
The student position is conceptually the opposite of
the teacher one: the application installed on the
student PC is a special software that implements all
the necessary functions for the fruition of the lesson
(live or recorded).
This application from the user point of view is an
area in a page of a web browser, while internally is
composed of many blocks, listed below: a) the
Network Receiver accesses multimedia contents
from the appropriate device (LAN IP, satellite,
modem, among others); the output is made up of a
series of interactive streams; b) the MP4 file reader
accesses contents from local MP4 files; c) decoders
for audio, video, JPEG, among others, to decode
elementary streams; d) security modules (optional):
this module decrypts the lesson received; e)
compositor: it updates the student interface
according to the commands received; f) student
interface: the student can decide which stream of the
lesson he wants to see in full-screen mode, navigate
through the slides (among the slides already
transmitted), have a break during the lesson or
terminate it.
The scenario in which the student application
works is the following: the student is provided with
a multimedia PC and the special software to receive
the lesson in MPEG-4 format. Through a LAN or
satellite link he connects to the server that stores the
pre-recorded lessons or to follow the current live
lesson. After the beginning of the lesson, he can
interact with the scene: thanks to the particular
USING MPEG-BASED TECHNOLOGIES FOR BUILDING PERSONALIZED MULTIMEDIA INTERACTIVE
ENVIRONMENTS WITH CONTEXT-AWARENESS REQUIREMENTS: Development of an application for Interactive
Television
181
description of the graphic interface implemented
during the creation of the scene itself, the direction
of the lesson is left on the user interface: in
particular the student can select the stream he wants
to see, the slide showed. He can also interact with
the teacher with a chat system: when the student has
a question he can send it using the chat system, and
the teacher can choose the way of answering: text
reply to the single student, to all the students or by
microphone. The student has also the possibility to
download a file that the teacher has sent to
everybody (i.e. a document, a presentation, an
executable file, among others). Another way of
interaction is given by the questions that sometimes
the teacher can ask: the students have to answer in
real-time and send back the reply, so the teacher has
immediately the results of the test.
4 SYSTEM EVOLUTION
The system is expected to evolve in three directions:
the delivery to different kinds of networks, the
adaptation to new emerging standards, and the
context-awareness. In this scenario, the main aspects
of each one of these directions are presented in the
next sections.
The main problem in the direct use of e-learning
is not technical, and is due to the scarce penetration
of PC and related culture in these countries. In fact it
is a reality to rent bandwidth in a satellite link and to
send multimedia lessons over it using DVB-MPE
(Multi Protocol Encapsulation). This way IP packets
directly coming from the streaming engine are
encapsulated as they are onto MPEG-2 Transport
Packets (MPEG-2, 1994).
Interactivity can be further enhanced if the
terminal is equipped with large storage capacity.
This functionality can be implemented through
standard languages, and this takes us to another big
area of extensibility: MPEG-21 and standard
Learning Objects (MPEG-21, 2001).
In parallel, different bodies are moving towards
the definition of standardized languages, most
having in common XML (eXtensible Markup
Language) (Connoly, 1997) associated with different
Schemas or Document Type Definitions (they both
describe the syntactic structure of the document).
This common choice and the intrinsic extensibility
of the language facilitate the integration of the work
of different bodies.
4.1 Applying Requirements of
Context Awareness
Context can be viewed as being any information
that can be used for characterizing the status of an
entity in one specific case. One entity can be one
person, one place or one object relevant for any type
of interaction between user and application,
including user and application itself (Dey &
Abowd, 2000; Pascoe et al., 1999). Therefore, one
system is context-aware when it uses the context for
providing relevant information and/or services to
the user, and this relevance feature depends on
user’s tasks (Abowd, 1999; Schmidt, 2000).
These parameters can be obtained based on five
arguments: a) where is the user (WHERE); b) who
is the user (WHO); c) how the user works (HOW);
d) when it can do what (WHEN); e) what the user is
doing (WHAT). Context-awareness aspects can be
relevant when associated to environment, in which
the application is inserted, providing information
for adaptability. The words HOW, WHERE,
WHAT, WHO and WHEN denote semantics that
can be very complex. In this point, for example, the
semantic HOW is referring to “what resources are
being used” expresses, implicitly, “how the user is
using these resources”.
Figure 2: Basic model for integration of the new
interaction criteria.
At the level of implementation, a field
<TIMEDATE> of the HEADERSESSIONCHAT
xml_struct can be also divided into sub fields,
allowing more granularity in the synchronization
processes with the MPEG-4 elementary streams. For
example, a suggestion is the specification of the
following fields: <day>, <month>, <year>, <hour>,
<minute> and <second>. This additional
contribution is a proposal for content description and
Application_temporal_data
XML
structure
MPEG-4
data
Schema for
Synchronization
of the
Media Objects
and Chat Data
XML or MPEG-7
Approach
User
Applicatio
n
Time_stamp_data
ICEIS 2005 - HUMAN-COMPUTER INTERACTION
182
personalization of the teaching-learning
environment, using the further resources of the
technologies like MPEG-4 and MPEG-7.
As a concrete example, the XML structure,
generated by a chat application on the teacher side,
can be used for synchronizing the MPEG-4 objects
with the questions that have been formulated by the
students during the lesson. Thus, a publisher process
can be used for presenting statistics about the lesson,
considering the temporal relations described by chat
time parameters and time stamps of the MPEG-4
objects, as illustrated in Figure 2.
Furthermore, MPEG-7 scene descriptions can be
used for adding new personalization levels to the
teaching-learning environment (Nack & Lindsay,
1999; Santos Jr. et al., 2001). The integration of
MPEG-7 information with some DBMS (DataBase
Management System), through the XML scheme,
can provide new and interesting controls for both
student and teacher interfaces.
5 CONCLUSIONS
This work has presented how the new standards of
the MPEG family can be used of systematized form
on the development of the educational applications,
discussing the e-learning scenario developed in
TILAB currently.
MPEG-4 is a standard that builds on the success
of MPEG-1 and MPEG-2, two standards that have
changed the audio-visual landscape. The best
experts, provided by all industries with a stake in
multimedia, have developed MPEG-4. It is a
powerful standard, rich in functionality,
encompassing other successful standards.
Furthermore, MPEG-7 provides essential features
for scene description, allowing the development of
several personalized schemes for user interaction;
MPEG-21 is being investigated for building new
representation schemes for modeling of multimedia
objects as universal digital items, mainly in terms of
the learning-objects.
In the educational context, this paper presents a
contribution for content description and
personalization of the teaching-learning
environment, using the further resources of the
technologies MPEG-4 and MPEG-7, contributing to
the increase of the personalization level in teaching-
learning environments.
REFERENCES
(MPEG-4, 2000a): ISO/MPEG MPEG-4: MPEG-4
requirements, MPEG-4 applications and MPEG-4
development process. 2000.
http://mpeg.telecomitalialab.com/working_documents.ht
m.
(Battista et al., 1999): Battista, S. et al.: MPEG-4: A
Multimedia Standard for the Third Millennium, part
1, IEEE MultiMedia 6(4) 74–83. 1999.
(MPEG-4, 2000b): Information Technology – Coding of
Audio-visual Objects. MPEG-4 Systems. 2000.
(MPEG-7, 2001): ISO/MPEG N3445 MPEG-7:
Multimedia Description Schemes XM (v.7.0). 2001.
http://mpeg.telecomitalialab.com/working_documents
.htm
.
(Nack & Lindsay, 1999): Nack, F. and Lindsay, A. T.:
Everything You Wanted to Know About MPEG-7:
Part 1, IEEE Multimedia, pp.65 – 77, 1999.
(MPEG-2, 1994): Information Technology – Generic
Coding of Moving Pictures and Associated Audio.
MPEG-2 Systems. 1994.
(MPEG-21, 2001): ISO/MPEG - MPEG-21:
Requirements on Digital Item Usage Environment
Description.
http://mpeg.telecomitalialab.com/working_documents
.htm.
(Abowd, 1999): Abowd, G.D.: Software Engineering
Issues for Ubiquitous Computing; International
Conference on Software Engineering (ICSE'99),
pp.75-84. 1999.
(Dey & Abowd, 2000): Dey, A.K. and Abowd G.D.:
Towards a Better Understanding of Context and
Context-Awareness; CHI 2000 - Workshop on The
What, Who, Where, When, and How of Context-
Awareness. 2000.
(Pascoe et al., 1999): Pascoe, J. et al.: Issues in
Developing Context-Aware Computing. In Gellersen,
H. (ed.): Handheld and Ubiquitous Computing.
Lecture Notes in Computer Science, Vol. 1707.
Springer-Verlag, Berlin Heindelberg New York,
pp.208-221. 1999.
(Schmidt, 2000): Schmidt, A.: Implicit Human Computer
Interaction Through Context. Personal Technologies
Volume 4(2&3) June, pp.191-199. 2000.
(Santos Jr. et al., 2001): Santos Jr., J. B. dos; Goularte, R.;
Faria, G. B.; Moreira, E. S.: Modeling of User
Interaction in Context-Aware Interactive Television
Application on Distributed Environments.
Proceedings 1
st
Workshop on Personalization in
Future TV – 8
th
International Conference on User
Modeling. Sonthofen, Germany. July-2001.
http://icmc.sc.usp.br/~joao/futuretv.
(Connoly, 1997): Connoly, D.: XML Principles, Tools
and Techniques. World Wide Web Journal. O’Reilly,
V.2, Issue 4, 1997.
USING MPEG-BASED TECHNOLOGIES FOR BUILDING PERSONALIZED MULTIMEDIA INTERACTIVE
ENVIRONMENTS WITH CONTEXT-AWARENESS REQUIREMENTS: Development of an application for Interactive
Television
183