Francesco Colace, Massimo De Santo, Mario Vento
DIIIE-Università degli Studi di Salerno
Via Ponte Don Melillo, 1 84084 Fisciano (Sa)
Keywords: E-Learning, Assessment, Bayesian Networks, Metadata.
Abstract: In this paper we introduce a tutoring approach for E-Learning formative process. This approach is strictly
related to the assessment phase. Assessment in the context of education is the process of characterizing what
a student knows. The reasons to perform evaluation are quite varied, ranging from a need to informally
understand student learning progress in a course to a need to characterize student expertise in a subject.
Otherwise finding an appropriate assessment tool is a central challenge in designing a tutoring approach. In
this paper we propose an assessment method based on the use of ontologies and their representation through
a Bayesian Networks. The aim of our approach is the generation of adapted questionnaires in order to test
the student’s knowledge in every phase of learning process. Analyzing the results of the evaluation an
intelligent tutoring system can help students offering an effective support to learning process and adapting
their learning paths.
Our “information-oriented” society shows an
increasing demand of life-long learning. In such
framework, on-line learning is becoming a real
solution that allows flexibility and quality in the
learning process. In this field the assessment phase is
acquiring a strategic interest. In fact, assessment is
an important and difficult task in the whole teaching
and learning process (Dahlgren, 1984). In fact it can
furnish timely feedback through just-in-time
problem solving aid that turns the learning process
into a highly interactive learning experience. On the
other hand the implementation of an effectiveness
on-line evaluation strategy can be very difficult. In
fact experts consider the evaluation phase as the
weak point in an E-learning process. Many of
currently existing assessment systems focus on
simple assessment strategies, e.g. only on single or
multiple-choice questions with several answers, and
radio-buttons to select the correct answer.
Furthermore most of them are unable to support the
different needs of individual users and focus mainly
on the assessment of the “average user”. In this way
teachers can give only a mere quantitative evaluation
of students’ knowledge and can not fill in the gaps in
their learning approach. Assessment provides an
effective method to gather information about
student’s learning and it is a good starting point in
order to arrange feedback’s strategy. Finding an
appropriate assessment tool is a central challenge in
designing an assessment approach (Royer, 1993).
The difficulty arises because of the diversity of
learning objectives, the diversity in what counts as
evidence of learning, the diversity of tools available,
the varying resources available, and the varying
assessment contexts. One way to address these
various assessment goals and challenges is through
the use of concept maps (Turns, 2000). So starting
from this general framework in this paper we
describe our system for assessment and tutoring
based on Ontology formalism (the generalization of
concept maps) and metadata standards. In this paper
we represent ontology through Bayesian Networks
formalism because in this way we can depict and
estimate the preparatory links between the various
subjects belonging to knowledge domain. In this
way it is easier to understand the real knowledge
shortage of students (Colace, 2003). In this paper we
design and implement a tool that arranges the best
assessment strategy according to the information
inferred by the analysis of questionnaires. Our tool
can test the knowledge of students on every subject
of ontology adopting various approaches. The
Bayesian approach allows quantizing the probability
of correct answer of students in a particular subject.
In this way tool can propose to the student the
question with the lower (or higher) probabilities of
correct answer. At the end of the assessment
Colace F., De Santo M. and Vento M. (2006).
In Proceedings of the Eighth International Conference on Enterprise Information Systems - HCI, pages 157-160
DOI: 10.5220/0002457001570160
student’s profile is updated. The paper is organized
as follows: in Section 2, we provide the motivations
and the details of the proposed assessment tool. In
particular we give some details on ontologies and
their mapping through Bayesian Networks. In
section 3 we describe the proposed approach. In
section 4 the experimental results are reported.
Finally, in the last section we draw conclusions and
indicate future directions of our research.
Some of the tasks that an E-Learning platform
should carry out aim to allow people to find,
evaluate and acquire adapted learning objects. These
activities are common and easy to carry out in
traditional learning processes: we can not say the
same in distance learning field. In the designing and
organizing phase of a course, a teacher has to choose
the most appropriate contents: this selection shows
notable difficulties, also due to the huge amount of
information available, of which only a minimum
part really meets teachers’ needs. The possibility of
accessing to contents that could be useless or not
related to the subjects of interest is considerable. In
this framework a very important role is played by
the assessment phase. Assessment gives to the
learning environment the most direct information
about the student’s knowledge. The best assessment
approach provides questionnaires that are built
dynamically on the basis of the student model.
Questions have to cover the topics most recently
completed, as well as those that should be reviewed.
Each question has a level of difficulty, which is also
used in the updating phase of student model.
Correctly answering a harder question demonstrates
greater ability than correctly answering an easier one
(Shang, 2001). The assessment framework combines
researches from two major research disciplines:
adaptive educational hypermedia and semantic web
technologies. In this paper we focus our attention on
the assessment phase and in particular in the
designing of an adapted assessment generator and
assessment module.
2.1 Ontologies
The concept of ontology was taken from philosophy
where it means a systematic explanation of being. In
recent years, however, this concept has been
introduced and used in different contexts, thereby
playing a predominant role in knowledge
engineering and in artificial intelligence. Ontologies
could be represented as taxonomic trees of
conceptualizations: they are general and domain-
independent at a superior level, but become more
and more specific when one goes down the
hierarchy. In other words, when we move from the
highest taxonomic levels to the lowest ones,
characteristics and aspects typical of the domain
under examination are showed. In order to point out
this difference in literature we call them
heavyweight (deeper ontology) and lightweight
(advances taxonomy) ontology respectively. In this
paper we will adopt the last one approach keeping in
mind this definition of ontology: “ontology may take
a variety of forms, but it will necessarily include a
vocabulary of terms and some specification of their
2.2 Ontologies and Bayesian
Bayesian networks have been successfully used to
model knowledge under conditions of uncertainty
within expert systems, and methods have been
developed from data combination and expert system
knowledge in order to learn them. A Bayesian
network is a graph-based model encoding the joint
probability distribution of a set of random variables
X ={X
, …, X
). It consists of a directed acyclic
graph S where each node is associated with one
random variable Xi and each arc represents the
conditional dependence among the nodes that it
joints and a set P of local probability distributions,
each of which is associated with a random variable
Xi and conditioned by the variables corresponding to
the source nodes of the arcs entering the node with
which Xi is associated. The lack of an arc between
two nodes involves conditional independence. On
the other hand, the presence of an arc from the node
Xi to the node Xj represents that Xi is considered a
direct cause of Xj. In this paper we use an algorithm,
based on the formalism of the Bayesian networks,
able to infer propaedeutic relationships among
different subjects belonging to the knowledge
domain of a course (Colace, 2003). The first step of
this algorithm is the introduction of a mapping
between Ontology and Bayesian Network. In our
ontology model nodes represent the subjects
belonging to the course knowledge domain and the
arcs mean a propaedeutic relationship among the
nodes. We can map this ontology graph in a
Bayesian network in the following way: the
Bayesian networks nodes can model the subjects
belonging to the course Knowledge Domain and the
knowledge of subject by students while arcs in the
same way can mean the propaedeutic relationships
among the nodes. Given the previous mapping
strategy our aim is to use and update the ontology
used by teacher in his/her course. Obviously we
must define data type and data set for this approach.
As previously said the student’s answers to the end
course evaluation tests represent a source of implicit
evidence. In fact, teachers through the end-of-course
evaluation tests not only assess student’s knowledge
for every subject, but describe the course ontology
and outline the propaedeutic aspects that relate
subjects each other. In this way we can use an
updated ontology and we can measure the effective
propaedeutic links between the various topics of a
In this section we will describe in detail the
architecture of our tool (named Virtual Teacher) and
the proposed assessment tutoring strategy. As
previously said we aimed to design a tool for
assessment able to assist students and teachers in the
formative process. We designed our tool keeping in
mind the main needs of students and teachers. From
a technological point of view we designed the tool
according these constraints: Web based approach,
aesthetic and minimalist design, flexibility and
efficiency of use, help users recognize, diagnose,
and recover from errors. In the first phase of the
designing we pointed out the actors of the system
and the use cases. We identified three typologies of
actors in the system: Administrators, Teachers and
Students. Each of these figures has a well defined
role and tasks. In particular Administrators can
introduce new courses, describe new ontologies and
manage the accesses to the tool. Teachers can design
the reference ontology, describe the learning objects
and the questions linked to the nodes of ontology.
Teachers can also manage the reports of every
student in order to better supervise the learning
process. Students can use tool in three different
ways: Exam, Normal test, Bayesian test. In the
Exam way our tool arranges a classical final test
exam according to the teacher’s strategy. At the end
of the exam the system produces a report analyzing
the performance of student in every subject. The
normal test approach can be used during some
module of the course. The more interesting service
offered by our tool is the Bayesian test. This service
makes the most of the matching between ontology
and Bayesian network. In fact the first step is the
introduction of a mapping strategy between
Ontology and Bayesian Network. In our ontology
model nodes represent the subjects belonging to the
knowledge domain of the course and the arcs mean a
preparatory relationship among the nodes. In this
way we can map the ontology graph in a Bayesian
network in the following way: the nodes of Bayesian
Network model the subjects belonging to the course.
The states (two: yes and not) of nodes represent the
knowledge of student in the subject. The arcs mean
the propaedeutic relationships among the nodes. In
other words a node of Bayesian network-ontology
represents the Knowledge domain of a course and
quantizes student’s knowledge of this node. First of
all the system select a set of questions associated to
every network node. At the end of this first phase
system, through a Bayesian approach infers what
subjects the students knows better than others. In
fact through the Bayesian analysis the system can
measure the percentage of correct answer in a
subject. In particular it can predict the percentage of
correct answer to a subject after a correct (or not)
answer to questions related to propaedeutic subjects.
At this point it can apply various strategies: for
example it can select and propose to the student the
question with the smaller percentage of correct
answer. At the end of Bayesian test a detailed report
on the knowledge of student in the various subjects
is sent to teacher and to student himself. In particular
after the Bayesian test the system proposes to the
student some learning object for deepening some
subjects. At the end of Bayesian Test the system
updates the user profile of students.
In order to test the effectiveness of our tool we used
it during the course of Introduction to Computer
Science at Foreign Literature and Language Faculty
of University of Salerno. This course is composed
by seven modules: Introduction to PC Architecture,
Introduction to Operative System, Microsoft Word,
Microsoft Excel, Microsoft Access, Microsoft Power
Point, and Internet. On the basis of the
considerations of previous section, teacher designed
the reference ontology. Each node of the networks
has two states and shows the probability that a
generic learner knows the subject associated with the
same node. We have supposed that each node can
assume only the following two states: state ‘Yes’
complete knowledge of the subject and state ‘Not’
total ignorance on the subject. The student level of
knowledge could be evaluated on the basis of the
answers given to the questions (a set of questions is
proposed for each subject). At the end of the course
students have to get through a final examination’s
test composed by forty questions. The questions
belong to every subject of knowledge domain. The
number of student’s course was about 50 and at the
starting of the course we arranged them in two
groups (named blue and red). The first group had a
classical support to course activities and used only
the normal test approach while the latter <group
used also all functionalities of the tool as didactic
support. At the beginning of the course teachers
designed every module’s ontology in order to
organize the contents and an assessment test. The
results are in the table 1. The aim of this test is to
allow a first description of student through a
metadata structure. In this way teachers can obtain
information about the initial knowledge level of
students. This information is very useful in order to
describe for the first time the student profile. At this
point the system organized for the student of red
group a support material for every module of course.
In particular it proposes the most suitable contents
through a matching between the metadata of
learning objects and the description of the student.
As previously said during the course the students of
the two groups attended to the lessons and used the
virtual teacher tool. In particular students of red
group at the end of every module sustained a
Bayesian Test. At the end of course students had
their final course exam. In the table we depicted the
Table 1: Results of Assessment Tool. The meaning of
range are: [0-10]: inadequate, [11-15]: poor progress, [16-
20]: adequate, [21-25]: good, [26-30]: very good.
Blue Group Red Group
Students Assessment
0-10 10 0-10 12
11-15 11 11-15 10
16-20 9 16-20 8
21-25 3 21-25 3
26-30 3 26-30 2
Total 36 Total 35
Table 2: Results of Final Test.
Blue Group Red Group
Final Test Students FinalTest Students
0-10 4 0-10 3
11-15 9 11-15 5
16-20 8 16-20 6
21-25 10 21-25 12
26-30 5 26-30 9
Total 36 Total 35
If we analyze the difference between the assessment
and the final exam (table 1 and 2) we can note that
the percentage of students that get through the
assessment test is 37% in the red group and 42% in
the blue group while in the case of the final
examination the percentage is 77% in the red group
and 64% in the blue group. We can note as more
students of red group get through the final exam and
improve their performance respect the assessment
test (about 40%). In particular we can note that the
students of the blue group have a minor
improvement (about 22%) than the students of the
red group. At the same time the percentage of red
group’s students that have a mark in the range 26-30
is higher than in the case of blue group: 26% to 8%.
In order to collect more information about the
effectiveness of our tool at the end of course we
submitted a questionnaire to every student. In the
questionnaire we asked the effectiveness of
Bayesian test and of learning objects furnished by
system at the end of the test. The 87 % of students
said that the support of Virtual Teacher tool was
very important in the learning process. In particular
the 73% of students declared that the supporting
learning object helped them in a better knowledge of
the various subjects.
In this paper we proposed a tool for the assessment
and tutoring of students during a learning process.
This is based on the use of ontology and Bayesian
Network. In particular through the matching
between ontology and Bayesian Network our tool
allow an effective tutoring and a better adaptation of
learning path to demands of students. The
assessment based on Bayesian approach allows a
deeper analysis of student’s knowledge. The
experimental results seem to confirm our approach.
As a future step of our research we intend to
evaluate the performance of the proposed system
when new features for tracking strategies are used.
L. Dahlgren, Outcomes of learning, in The Experience of
Learning, F. Marton, D. J. Hounsell, and N. J.
Entwistle, Eds. Edinburgh, U.K.1984,
J. M. Royer, C. A. Cisero, and M. S. Carlo, Techniques
and procedures for assessing cognitive skills, Rev.
Educ. Res., vol. 63, no. 2, pp. 201–243, 1993.
J. Turns, C. J. Atman, and R. Adams, Concept Maps for
Engineering Education: A Cognitively Motivated Tool
Supporting Varied Assessment Functions, IEEE
Transactions On Education, Vol. 43, No. 2, 2000
F. Colace, M. De Santo, P. Foggia, M. Vento, Ontology
Learning Through Bayesian Networks, Proceedings of
ICEIS 2003 Conference, Angers, 2003
Yi Shang, Hongchi Shi, and Su-Shing Chen, An Intelligent
Distributed Environment for Active Learning, ACM
Journal of Educational Resources in Computing, Vol.
1, No. 2, 2001