QUALITY LEARNING OBJETCS MANAGEMENT
A Proposal for E-learning Systems
Erla Morales, Ángela Barrón
Departament of Theory and History of Education, Salamanca University, Canalejas Street 169, Salamanca, Spain
Francisco García
Department of Computer Science, Salamanca University, Plaza de los Caídos Street, s/n, Salamanca,Spain
Keywords: Learning Objects, Evaluation, Knowledge Management.
Abstract: Web development is promoting important advantages for educational area specially e-learning systems. By
one side Learning Objects (LOs) aim the possibility to reuse specific information and by the other side they
can be interchanged though different context and platforms according to the user’s needs. However an
urgent necessity exists to guarantee the LOs quality content. There exists a plethora of quality criteria to
value digital sources but there are only a few suggestions about how to evaluate LOs to structure quality
courses. Our proposal consists on a system to evaluate LOs as a continued process taking into account
quality criteria related to metadata information, especially the educational category, together with a strategy
to ensure a continued LOs quality contents.
1 INTRODUCTION
In today’s world many efforts exists in development
of standards and specifications for management
information without interoperability problems. The
Learning Object (LO) concept is a set of resources
that could be used as independent and reusable units
of learning though different context and platforms.
To achieve this each one of LO may have
metadata (data about data) for their description and
administration. In this way it is possible to know
what kind of LO we are trying and if it is reusable
for new contexts of use.
However through metadata is possible to know
information about LOs but it doesn’t guarantee they
content quality.
Nowadays some efforts exist to evaluate LOs.
There are LOs repositories like MERLOT, CLOE
and DLNET (MERLOT, 2003) which propose three
dimensions of evaluation. However to guarantee an
optimal evaluation it is necessary to take into
account more things. Define criteria into different
kind of categories or principles aim to review
different points of view, for this reason we suggest
four categories with quality criteria. To ensure LOs
quality characteristics we propose to complete their
metadata information according to our quality
criteria.
There are a lot of LOs definitions and different
level of granularity. Our proposal is directed to LOs
as educational content, for this reason in section 2
we suggest our own LO definition that establish it
components and a knowledge model that establish a
relation between them.
One of the most famous instruments for
evaluating LOs is LORI (Learning Object Review
Instrument) (Nesbit et al, 2003) which aims to
evaluate LOs according to nine general items.
However items must to consider LOs metadata
information and made a relation between them
because metadata contains LOs information and in
some case they be able to be managed though their
metadata information only.
In section 3 we propose an evaluation instrument
which considers our four categories to evaluate LOs
from different points of view relating to quality
criteria with LOs metadata information.
Some LOs evaluation proposal suggest a
collaborative methodology taking into account
different kind of experts’ participation (Nesbit et al,
2002) and (Nesbit et al, 2004).
312
Morales E., Barrón Á. and García F. (2006).
QUALITY LEARNING OBJETCS MANAGEMENT - A Proposal for E-learning Systems.
In Proceedings of the Eighth International Conference on Enterprise Information Systems - AIDSS, pages 312-315
DOI: 10.5220/0002447603120315
Copyright
c
 SciTePress

According to this to promote a better LOs quality
section 4 proposes a LO evaluation strategy, which
combine our evaluation instrument with a
collaborative strategy executed by experts. To
improve and continuous LOs refeed we suggest a
user evaluation test.
2 NORMALIZATION OF
LEARNING OBJECTS
There are different LOs definitions (Wiley, 2000);
(Moreno and Bailly-Baillière, 2002); (IEEE, 2002);
(Polsani, 2003).
However taking into account LOs characteristics
and they management for e-learning system we
define a LO as a “unit with a learning objective,
together with digital and independent capabilities
and accessible through metadata to be reused in
different contexts and platforms” (Morales et al,
2005a).
LOs must have a learning objective because it
enables to direct the contents and material relating to
them. Ideally a LO must contain different types of
element which help to clarify the main idea.
Independent LOs means the possibility to teach
some topic by itself avoiding reusability problems.
Accessible through metadata capabilities deliver
the LOs characteristics providing different kind of
information about them. Our proposal is based on
IMS specifications for this reason we refer metadata
considering IMS LOM (Learning Object Metadata)
(IMS LOM, 2003).
Finally, LOs reusability means the possibility
that a LO could be reused many times independent
of software and platforms changes. This issue
reflects their interoperability and durability
characteristics.
However to achieve a suitable LOs management
it is necessary to take into account some pedagogical
issues related with their quality.
Educational objectives are related with cognitive
levels. Therefore, different kinds of taxonomies exist
that establish what it’s possible to learn into a
specific cognitive domain therefore some kind of
objectives, contents and difficulty level exist.
To ensure a suitable LOs management we
propose their normalization through a knowledge
model that classify LOs taking into account their
objectives, contents and difficulty level.
1.- Clasiffy LOs objectives according to their
complexity level. In this way it is easier knowing
about their compatibility for suitable new
educational situations. Then, we suggest Bloom’s
cognitive domain taxonomy (Bloom, 1956) because
it defines what and how to learn according to
complexity levels: low level (knowledge,
comprehension and application) and high level
(analysis, synthesis and evaluation).
2.- Define the difficulty level to each one of LOs, for
this issue we propose three kinds of complexity
levels: basic, medium and advanced because this
kind of classification would help teachers to select
the LO content according to their teaching
objectives.
3.- Classify the imported LOs into three kind of
content: data and concept, procedure or processes,
and reflection or attitude. This classification aims to
define the kind of content according to the learning
objectives.
LOs classification must be added to their
metadata information in this way it is possible to
manage them according to their characteristics. We
suggest to add this information into educational
metadata category specially description element it is
because our classification describe general issues to
take into account before a LO can be reused.
The provided LOs classifications for the
knowledge model allow teachers to find content
according to the subject area, type of content, and
level of difficulty.
Nevertheless, LOs normalization is not enough
to guarantee their quality. Next we suggest our own
LOs evaluation proposal.
3 EVALUATION INSTRUMENT
To evaluate LOs according to their characteristics
we propose to divide metadata information
especially educational section into four categories.
Each one of this categories have quality criteria to
evaluate their content, according to this it is possible
to evaluate them taking into account different points
of view.
Psychopedagogical category: This category
contains pedagogical criteria related to the
psychology of learning. This kind of criteria aims to
determine if the LO is suitable to promote learning.
Didactic-curricular category: This kind of
criteria aims to evaluate if an object is related to
curricular objectives according to the context in
which it will be applied.
Technical-Aesthetic category: Contain criteria
to evaluate technical issues like interface design and
metadata information.
Functional category: Contain criteria that aim to
verify a suitable LO functionality because if we have
QUALITY LEARNING OBJETCS MANAGEMENT - A Proposal for E-learning Systems
313

an object which does not work correctly it could
obstruct the learning process.
To facilitate LOs review evaluators can to see
metadata information file as well as LO URL
location. In this way users could to know a complete
information about the LO they are trying.
To ensure a suitable LO evaluation each one of
quality criteria have metrics for their evaluation that
aim to know what is they means. This information is
part of the instrument we are proposing.
The third column shows the range scale for
evaluation, if evaluators don’t know how to evaluate
it or if they have a doubt it is possible to select DN=
Don’t Know, otherwise they can to select the
following rate scale 1=very low, 2=low, 3=medium,
4=high and 5=very high.
Figure 1 shows some of the evaluation criteria
we are proposing. For example, into didactic-
curricular category there are quality criteria related
with LOs objectives, contents, and activities, each
one of this have a final score that aim to know their
individual scoring to reinforce them if it be
necessary. According to this, the final scoring of
each category is average out at the field “category
score” from this results it is possible to obtain a final
LO evaluation at the field “final evaluation results”.
In case of any doubt, critic or suggestions evaluators
have a comments section.
The final evaluation results could don’t have
whole numbers, for this reason LOs level of quality
will be interpreted taking into account the following
rating scale 1.0-1.9= very low, 2.0-2.9=low, 3.0-
3.9=regular, 4.0-4.5=good, 4.6-5.0=very good.
4 EVALUATION STRATEGY
We propose two modes of applying the instrument
in order to value the LO: individual and
collaborative. Individual evaluation provides us an
initial appreciation of the quality of the LO based on
the judgment of each participant.
According to (Vargo et al, 2003) we suggest a
collaborative evaluation to encourages not only
different points of view over the subject under
evaluation, but also a critical objectivity and a
reliable LOs evaluation.
The possibility of completing an evaluation
through collaborative method enables to contrast the
individual’s initial evaluation with the others
experts’ evaluations. It aims to share different points
of view to achieve an advanced and reliable
evaluation (Vargo et al, 2003).
To help teachers in this work by one side our tool
aim to analyze graphics that show statistics that
reflect individual an collaborative evaluation and by
the other side it provide a forum for discussions to
achieve an agreement for a final evaluation.
After LOs evaluation they must to be saved into
a repository that contains normalized and quality
contents. From this repository teachers could search
LOs to structure their courses offering quality
contents for their students. These contents will be
part of biggest units of learning like lesson, modules,
courses, etc. and they will be published by e-
learning system for their usability and be continually
evaluated to guarantee their quality.
Therefore a re-feeding process is needed which
taking into account students’ and teachers’
Figure 1: An example of Evaluation Instrument.
ICEIS 2006 - ARTIFICIAL INTELLIGENCE AND DECISION SUPPORT SYSTEMS
314

contributions to the LOs quality.
According to this, we propose a LOs re-
evaluation, which considers a learners’ experience
about the efficacy of the LO to improve their
knowledge (Morales and García, 2005); (Morales et
al, 2005b). For this reason once students have
finished their lesson they have to respond a little test
about their satisfaction with the contents. Each one
of this questions are related with LOs evaluation
instrument, in this way it is possible to contrast them
with previous experts evaluation.
Taking users responses, evaluators may have to
re-feed LOs to guarantee their continued quality.
5 CONCLUSIONS
To make suitable LO evaluation a fist thing we must
to consider is LO definition, we think our definition
may be suitable for LOs management because it
promotes a simple LOs contents that could help to
reuse them in an easy way.
Our normalization proposal helps to promote a
uniform LO level of granularity and the possibility
to increment LO reusability to another specific
context. It is because relating a LO to knowledge
domain aim to attend different educational situations
for different requirements.
LOs evaluation proposal is a way to evaluate
them according to their characteristics. LOs are
characterized for the separation between their
content and presentation. Therefore, the relation
presented between LOs metadata and quality criteria
is a concrete way to evaluate them.
Each one of evaluation categories aim to evaluate
this characteristics into a concrete set, providing
specific criteria to evaluate them. Metadata record
evaluation into technical category aim evaluators to
complete or correct metadata information and
evaluate the standard compliance.
Finally we think to achieve an integral LOs
evaluation is important not only to consider different
kind of experts evaluators but the possibility to
discuss their opinion though a collaborative strategy.
However an expert evaluation must be reinforced
with users’ evaluations, which might contribute their
experience and express their satisfaction.
Our future work is to implement this model in
order to make possible adjustments and
modifications.
ACKNOWLEDGEMENTS
This work was partly financed by Ministry of
Education and Science as well as FEDER Keops
project (TSI2005-00960).
REFERENCES
Bloom, B., 1956. Taxonomy of Educational Objectives:
Handbook I, Cognitive Domain. David McKay.
IMS LOM., 2003. Learning Resource Metadata
Specification.
http://www.imsglobal.org/metadata/index.html
MERLOT., 2003. Multimedia Educational Resources for
Learning and Online Teaching. http://merlot.org.
Morales, E. and García, F., 2005. Quality Content
Management for E-learning: General issues for a
decision support system, In ICEIS’05 7th international
conference on enterprise information system. ICEIS
Press.
Morales, E., García, F., Barrón, A., 2005a. Knowledge
Management System to Re-feed Learning Objects
Repository. In (m-ICTE’05) 3th international
conference on multimedia and information &
communication technologies in education.
Morales, E., García, F., Barrón, A., 2005b. Knowledge
Management for E-learning based on Learning Objects.
A qualitative focus, In ITHET’05, 6th international
conference on information technology based higher
education and training. IEEE CS Press.
Moreno, F., Bailly-Baillière, M., 2002. Diseño instructivo
de la formación on-line. Aproximación metodológica a
la elaboración de contenidos, Ariel Educación.
Nesbit, J., Belfer, K., Leacock, T., 2003. Learning Object
Review Instrument (LORI) User Manual E-Learning
Research and Assessment Network
Nesbit, J. C. & Li, J., 2004. Web-based tools for learning
object evaluation. International Conference on
Education and Information Systems: Technologies and
Applications Orlando, Florida.
Polsani, P., 2003. Use and abuse of reusable learning
objects. Journal of digital information.
http://jodi.ecs.soton.ac.uk/Articles/v03/i04/Polsani/
Vargo, J., Nesbit, J.C., Belfer, K., Archambault, A., 2003.
Learning object evaluation: computer-mediated
collaboration and inter-rater reliability. International
Journal of Computers and Applications, 25, 3..
http://www2.cstudies.ubc.ca/~belfer/Papers/202-1335.pdf.
Wiley, D. A., 2000. Learning object design and
sequencing theory, Unpublished Doctoral Dissertation,
Bringham YoungUniversity, Provo, UT,
http://reusability.org/read/chapters/wiley.doc, (2000).
QUALITY LEARNING OBJETCS MANAGEMENT - A Proposal for E-learning Systems
315