Moodle Tools for Educational Analytics of the Use of Electronic
Resources of the University’s Portal
Olena G. Glazunova
1 a
, Maksym V. Mokriiev
1 b
, Olena H. Kuzminska
1 c
,
Valentyna I. Korolchuk
1 d
, Nataliia V. Morze
2 e
, Liliia O. Varchenko-Trotsenko
2 f
and
Roman A. Zolotukha
1 g
1
National University of Life and Environmental Sciences of Ukraine, 15 Heroyiv Oborony Str., Kyiv, 03041, Ukraine
2
Borys Grinchenko Kyiv University, 18/2 Bulvarno-Kudryavska Str., Kyiv, 04053, Ukraine
Keywords:
E-Learning, Learning Management Systems, Educational Analytics, E-Course, Analytics Tools CLMS,
Moodle, Higher Education.
Abstract:
The need for additional analysis of the effectiveness of e-learning implementation models and their resource
support in higher education institutions in the context of the COVID-19 pandemic has been actualized. An
overview of solutions and case studies in the context of selecting and analyzing the effectiveness of individual
services and learning management platforms is provided. It has been studied that in order to investigate the
effectiveness of using electronic resources to meet students’ educational needs, it is advisable to use quanti-
tative indicators in addition to student description results. This includes data from educational analytics on
the frequency and duration of students’ use of individual e-resources. Reviewed the functionality modules
“Course Comparison” of the Moodle LMS and “Statistics”, as well as the optional Analytics module. The
results of applying these modules to the analysis of e-learning courses of the National University of Life and
Environmental Sciences of Ukraine and Boris Grinchenko Kyiv University are presented. The reasons for
students’ low use of individual e-courses were investigated.
1 INTRODUCTION
The issue of the quality of e-learning resources in a
distance learning environment is extremely relevant
during the quarantine period associated with COVID-
19 (Vakaliuk et al., 2021). Despite the various qual-
ity assurance procedures for e-learning resources for
students, especially in forced distance learning set-
tings, it is often very difficult to assess the quality of
the resources used relying on these procedures, which
mainly include student surveys and peer review. How-
ever, due to the transition of higher education insti-
tutions to distance learning (Bobyliev and Vihrova,
2021) and in the context of the pandemic generated
a
https://orcid.org/0000-0002-0136-4936
b
https://orcid.org/0000-0002-6717-3884
c
https://orcid.org/0000-0002-8849-9648
d
https://orcid.org/0000-0002-3145-8802
e
https://orcid.org/0000-0002-0136-4936
f
https://orcid.org/0000-0003-0723-4195
g
https://orcid.org/0000-0003-3099-722X
by COVID-19, the problem of analyzing the quality
and adapting the design models of educational envi-
ronments (Morze et al., 2013; Glazunova and Shyshk-
ina, 2018), according to the types of institutions, ed-
ucational program, available resources, has become
more relevant and other (Edelhauser and Lupu-Dima,
2020).
Looking at e-learning quality indicators from
2000 to 2017, Silva et al. (Silva et al., 2018), based
on an analysis of scientific publications, identifies
that indicators that have the greatest weight can be
grouped into three categories: e-resources, data, pro-
cesses. In the context of this study, we can look at
the same resource provision with a focus on practical
cases and a combination of quantitative and qualita-
tive assessment.
For example, in distance learning for future health
professionals, digital imaging should be used ex-
tremely closely to real-life practice. Educational
institutions use specialized software, e.g. Clinical
Study Export (TCE) as a platform for extending the
PACS infrastructure by connecting educational func-
444
Glazunova, O., Mokriiev, M., Kuzminska, O., Korolchuk, V., Morze, N., Varchenko-Trotsenko, L. and Zolotukha, R.
Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal.
DOI: 10.5220/0010932700003364
In Proceedings of the 1st Symposium on Advances in Educational Technology (AET 2020) - Volume 2, pages 444-451
ISBN: 978-989-758-558-6
Copyright
c
2022 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
tions (Mildenberger et al., 2011).
Understanding the importance of communication
and cooperation in the implementation of distance
learning, the effectiveness of the use of different ser-
vices is the subject of analysis. For example, Biasutti
(Biasutti, 2017) presents the results of a comparative
analysis of forums and wikis as tools for online col-
laborative learning. However, as each service has to
be seen in the context of student training requirements
and educational goals, more and more researchers
are looking at integrated solutions to support self-
regulated educational strategies (Kraleva et al., 2019).
In this context, among the various learning Man-
agement Systems (LMS) on the market (Basaran and
Mohammed, 2020), LMS Moodle is the most pop-
ular in the implementation of distance and blended
learning in higher education (Oguguo et al., 2021;
Mintii, 2020; Abdula et al., 2020). According to re-
search by scientists from different countries, techno-
logical satisfaction about Moodle in higher education
is quite high the effect is equal to 0.78 with a 95%
confidence interval from 0.72 to 0.84 (Garc
´
ıa-Murillo
et al., 2020). The transition to distance learning and
increasing the frequency of interaction between teach-
ers and students through LMS has necessitated the
search for resources to bring online communication
closer to offline. One such solution is Moodle LMS
integration with Amazon Alexa for creating voice
content (Ochoa-Orihuel et al., 2020).
On the other hand, the success of a distance ed-
ucation program can be assessed, in addition to aca-
demic performance, by the level of student satisfac-
tion. However, there is a correlation between stu-
dent satisfaction with e-courses (e-resources and con-
tent) and readiness for online learning (Deveci Topal,
2016). The latter is usually determined by surveying
students, and is checked by data of educational an-
alytics which allow to analyze behavior of students
in LMS (Kadoi
´
c and Ore
ˇ
ski, 2018). Since the use
of educational analytics or analytics for student suc-
cess, according to Brown et al. (Brown et al., 2020)
is identified as one of the areas of educational tech-
nology, we consider it appropriate to use educational
analytics to determine the causes of student satisfac-
tion / dissatisfaction, followed by recommendations
for improving resources and methodological support
its application in higher education institutions. To do
this, it is necessary to explore the tools that can be
used to quickly analyze the effectiveness of resource
use, in particular, e-learning courses in disciplines,
and relevant indicators. Such analysis will allow a
rapid response to the problems that users of e-courses
are experiencing, allow for quick resolution of these
problems and make student learning more productive.
Research goal: justify the choice of tools for ed-
ucational analytics on the use of e-learning courses,
in particular to determine user activity, frequency and
duration of use of course resources in order to rec-
ommend to teachers to improve the quality of both
course materials and methods of using relevant re-
sources during distance learning.
Figure 1: Moodle e-course comparison module data model.
2 METHODS AND STUDY
MATERIALS
The study was conducted using data from the train-
ing portal of National University of Life and Envi-
ronmental Sciences of Ukraine (NULES) and Borys
Grinchenko Kyiv University (BGKU). Methods and
technologies of statistical analysis were used for the
research.
The university’s learning portal usually operates
on the basis of CLMS (Content Learning Manage-
ment System) platforms and is designed to support
the learning process with e-resources in the format
of e-books, web pages, lessons and video lessons,
test tasks, laboratory and practical, independent work.
For each discipline in the e-learning course can be
placed the above and other resources. It should be
noted that the use of e-learning courses in the ed-
ucational process should correspond to the working
curriculum of the discipline, and the resources should
contain relevant, popular information. The procedure
for attestation of electronic courses in universities in-
volves the implementation of a number of criteria,
Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal
445
which are usually spelled out in the relevant regula-
tions. In particular, in NULES, these criteria are di-
vided into structural-functional, scientific-substantive
and methodological. But the use of certified courses
is not uniform throughout the semester. At the same
time, students actively use part of the resources in the
courses, and some resources are not used at all. In or-
der to identify the reasons for this and to select tools
for quick analysis of course performance data and its
resources, it is necessary to analyze the relevant tools,
which are built-in or complementary.
Let us focus on the statistical and analytical tools
of CLMS Moodle. Course resource efficiency indi-
cators can be obtained through the use of the embed-
ded modules: “Course Comparison” and “Statistics”
modules, as well as the optional “Analytics” module.
2.1 Features of the Module
“Comparison of Courses”
The analysis modules in the base Moodle distribution
are not very powerful, but they are present too. One
of the first modules that is appropriate to use when
analyzing courses is Course Comparison. This can be
found under Manage Site Management Reports.
This module allows you to view four reports:
1. The most active courses (ranks courses by the
number of actions taken by their participants as
a whole).
2. Most active courses (weighted) (calculates the av-
erage performance per user in the course).
3. Participation rate (shows courses in descending
order of user participation rate).
4. Activity ratio (determined by the ratio of involve-
ment and participation of users in the course).
Some of these concepts need clarification.
The first analytical report gives us the opportunity
to see on which course there is active activity. Activ-
ity on a course is the total number of all views and
publications on the course during the period under
study. However, this is of little use as a large num-
ber of enrolled users will visually create more activity
compared to courses with a relatively small number
of participants.
Consequently, a weighted average of each user’s
activity on the course is already more informative and
will show everyone’s participation. Here you can see
how truly active courses with a small number of users
come to the fore. However, if there are few active
users on the course, the mass of enrolled but inactive
users will drag the course down in the rankings.
The following report can clarify this nuance. It
shows how many real active students are on the
course. The participation rate of active users is calcu-
lated as the share of active users to the total enrolment
in the course. Active users are defined as those who
had had activity during the period under study. But
here again the question arises, what are these users
doing on the course? Are they just reading (receiving
information) or are they active?
The fourth report gives us the answer to this ques-
tion. The publication and views activity ratio is calcu-
lated as the share of publications in relation to views.
Where views refer to any user going to another page
and reading’ it or downloading a file resource from
the course to their computer. Publications are defined
as any activity the user performs on a course, for ex-
ample, completing a quiz, completing a task (down-
loading a file or writing a text response), replying to a
forum post, and so on. That is publishing is not just a
forum post.
2.2 Features of the “Statistics” Module
Another auxiliary module for analyzing course per-
formance is the Statistics module, which operates at
the site level, providing some statistics on the activity
on courses as a whole, as well as in each individual
course. At the site level, the results of this module are
available to administrators and site managers. At the
level of each course, teachers can use it to generate
statistics within that course.
About the Course Comparison module analyzing
activities without dividing into teachers and students,
it is important to say exactly who generates such ac-
tivities the actual training of students or the active
creation of a teaching course by teachers. The Statis-
tics module brings clarification. Using this module,
we can see the activities of each individual role. It is
also possible to look separately at views, only publi-
cations, or only introductions. At a site-wide level,
these metrics plot all roles, while at a course level we
can get a separate graph for each role.
2.3 Possibilities of the “Analytics”
Module
Additional analytical reports can be obtained using
third-party modules. One such powerful addition is
the Analytics module. With its help it is possible to
analyze activity of each student both as a whole on a
course, and in each concrete resource of a course. We
are provided with such reports:
Valuation chart (shows the distribution of valua-
tions using a stock chart)
Work with content (shows the activities of stu-
AET 2020 - Symposium on Advances in Educational Technology
446
Figure 2: Courses usage statistics.
dents with each resource separately how much
they worked and how much they ignored)
Student activity (shows a consolidated distribu-
tion of student activities on the course in terms
of hours per day)
Execution of tasks (shows a diagram for all Tasks,
which demonstrates compliance with the dead-
lines)
Passing tests (shows a diagram for all tests, which
demonstrates compliance with the deadlines)
Distribution of views (shows the schedule of per-
sonal activities on the course of each student)
3 MAIN FINDINGS
Only certified electronic learning courses (ELC) were
chosen for the study, i.e. courses in which the struc-
ture and set of resources are correctly selected for the
implementation of the educational process. Thus in
NULES the certified courses operate within 5 years,
and in BGKU within 1 year. The number of certi-
fied courses at the end of 2020 in NULES was 1644
courses, in BGKU – 768 courses.
The first hundred most active courses (with high
average activity per user) can be obtained by using the
Course Comparison module and its “active courses
(weighted)” report (figure 2).
Based on the results (figure 2) it is possible to
identify courses with low user activity. The follow-
ing analysis of the content of such courses, didactic
features of the use of course resources will make it
possible to identify relevant problems with their use.
Usage statistics courses and the way they use
within the categories are also using the module
“Statistics”. For example, in BGKU such statistics
can be obtained by categories of departments (fig-
ure 3), and in NULES – by categories of specialties.
A direct query to the database of courses makes it
possible to obtain such data for all certified courses.
As a result, all courses can be divided into 3 cat-
egories: courses with low efficiency, sufficient and
high in terms of “activity per user”. For example,
NULES with a high degree of use has 12% of cer-
tified ELC, with sufficient – 57%, with low – 31%.
Analyzing ELCs with high efficiency, a number of
studies have been conducted on the use of resources
of these ELCs using an analytical module built into
Moodle.
For example, figure 4 reflects the students’ activ-
ity in the course “Computer Technologies and Pro-
gramming” during the last semester. The activity of
students in revising resources and publishing com-
pleted tasks or passing tests is uneven. The peak
falls towards the mid-term examinations and the end
of the semester, which may explain the need to com-
plete the quizzes. But we can conclude that the use
of the course at the beginning of the module is not
Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal
447
Figure 3: Courses usage statistics by category.
active enough and indicates that the laboratory and
self-study assignments are not completed on time and,
consequently, students are not working with the re-
sources.
To find out with which resources in e-courses, stu-
dents work actively, the function “Working with con-
tent” of the module Analytics” is used. For exam-
ple, figure 5 shows the activity of using theoretical
resources of the course “Information Technology”.
From this diagram we can conclude about the ex-
tremely low activity of students in the use of theo-
retical resources, in contrast to laboratory work.
Often students actively use electronic learning re-
sources in a discipline only because they have to take
a test every day and hand in work to be tested, but they
do not use theoretical resources because they are not
very informative. Another option is for students to ac-
tively use methodological materials in the discipline,
watching video tutorials, and to a lesser extent use re-
sources designed to monitor learning achievements.
An important task for universities is to obtain tools
to quickly assess the quality of electronic resources
by further using their content and methodology in the
teaching process.
Next, you need to analyze the content of edu-
cational material set out in theoretical resources, in
terms of structure, accessibility, relevance, practical
orientation. These are all tasks of scientific and sub-
stantive examination. Such an electronic course can
be reconsidered by the educational and methodical
commissions of the faculties regarding the possibility
of its use in the educational process.
As a result of using such tools, we have the oppor-
tunity to determine which e-courses contain:
an excessive number of tasks (exceeding the num-
ber of laboratory, modular and independent tasks)
AET 2020 - Symposium on Advances in Educational Technology
448
Figure 4: Analysis of electronic courses on the subject of general activity of students.
Figure 5: Analysis of e-courses on the use of resources.
that required the work of students with the course;
little informative, unstructured training materials
qualitatively presented theoretical materials and
methodical recommendations, which were ac-
tively used by students;
educational resources that were systematically
used during the semester.
To increase the efficiency of the use of ELC in
higher education, a number of steps can be taken to
use statistical and analytical tools Moodle (figure 6).
The first step should be to rank the courses by ac-
tivity per 1 user (weighted indicator). For all courses
that are actively used, the second step is performed
statistics of general activity in e-courses and analytics
of resource use in e-courses. The third step is to form
Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal
449
Figure 6: Scheme of using statistical and analytical tools Moodle to increase the efficiency of ELC.
conclusions and recommendations.
Identifying e-courses that are insufficiently used
in the educational process, provides an opportunity to
intensify work with teachers to improve their skills
with information support, to create e-courses, the use
of e-resources in the educational process. The e-
resources found not to be used by students in the
learning process should be reviewed according to sci-
entific peer review criteria. Built-in and additional
CLMS Moodle tools allow you to analyze the effec-
tiveness of e-courses in general and in terms of differ-
ent types of resources and, based on this analysis, to
form general recommendations for course teachers to
improve the use of e-courses in education.
4 CONCLUSIONS AND THE
RESEARCH PERSPECTIVE
The use of statistical and analytical tools in CLMS
Moodle to determine the effectiveness of the use of
e-courses contributes to the quality of the educational
process, in particular blended and distance learning.
By measuring weighted course user activity, overall
activity within the course and analyzing the use of
course resources, it is possible to identify the reasons
for the inefficient use of e-courses in the educational
process. Since the study was carried out on the ba-
sis of two higher education institutions, we can assert
general trends on the problems of using e-courses in
blended and distance learning. In the future, we see
the need to develop a model that provides automated
determination of levels of effectiveness of e-courses,
e-course resources, identification of factors that affect
the effectiveness of the use of courses, and specific
resources.
REFERENCES
Abdula, A. I., Baluta, H. A., Kozachenko, N. P., and Kas-
sim, D. A. (2020). Peculiarities of using of the Moo-
dle test tools in philosophy teaching. CEUR Workshop
Proceedings, 2643:306–320.
Basaran, S. and Mohammed, R. K. H. (2020). Usability
evaluation of open source learning management sys-
tems. International Journal of Advanced Computer
Science and Applications, 11(6):400–410.
Biasutti, M. (2017). A comparative analysis of forums and
wikis as tools for online collaborative learning. Com-
puters & Education, 111:158–171.
Bobyliev, D. Y. and Vihrova, E. V. (2021). Problems and
prospects of distance learning in teaching fundamental
subjects to future mathematics teachers. Journal of
Physics: Conference Series, 1840(1):012002.
Brown, M., McCormack, M., Reeves, J., Brooks, D. C., ,
Grajek, S., Alexander, B., Bali, M., Bulger, S., Dark,
S., Engelbert, N., Gannon, K., Gauthier, A., Gibson,
D., Gibson, R., Lundin, B., Veletsianos, G., and We-
ber, N. (2020). The 2020 EDUCAUSE Horizon Re-
port: Teaching and Learning Edition. EDUCAUSE,
Louisville, CO. https://library.educause.edu/-/media/
files/library/2020/3/2020 horizon report pdf.pdf.
Deveci Topal, A. (2016). Examination of university
students’ level of satisfaction and readiness for e-
courses and the relationship between them. Euro-
pean Journal of Contemporary Education, 15(1):7–
23. http://ejournal1.com/journals n/1459666234.pdf.
Edelhauser, E. and Lupu-Dima, L. (2020). Is Romania Pre-
pared for eLearning during the COVID-19 Pandemic?
Sustainability, 12(13). https://www.mdpi.com/2071-
1050/12/13/5438.
Garc
´
ıa-Murillo, G., Novoa-Hern
´
andez, P., and Rodr
´
ıguez,
R. S. (2020). Technological Satisfaction About Moo-
dle in Higher Education—A Meta-Analysis. IEEE Re-
vista Iberoamericana de Tecnologias del Aprendizaje,
15(4):281–290.
Glazunova, O. and Shyshkina, M. (2018). The concept,
principles of design and implementation of the univer-
AET 2020 - Symposium on Advances in Educational Technology
450
sity cloud-based learning and research environment.
CEUR Workshop Proceedings, 2104:332–347.
Kadoi
´
c, N. and Ore
ˇ
ski, D. (2018). Analysis of student
behavior and success based on logs in moodle. In
2018 41st International Convention on Information
and Communication Technology, Electronics and Mi-
croelectronics (MIPRO), pages 0654–0659.
Kraleva, R., Sabani, M., and Kralev, V. (2019). An analysis
of some learning management systems. International
Journal on Advanced Science, Engineering and In-
formation Technology, 9(4):1190–1198. http://ijaseit.
insightsociety.org/index.php?option=com content&
view=article&id=9&Itemid=1&article id=9437.
Mildenberger, P., Br
¨
uggemann, K., R
¨
osner, F., Koch, K.,
and Ahlers, C. (2011). PACS infrastructure sup-
porting e-learning. European Journal of Radiology,
78(2):234–238. From PACS to the clouds.
Mintii, I. S. (2020). Using Learning Content Management
System Moodle in Kryvyi Rih State Pedagogical Uni-
versity educational process. CEUR Workshop Pro-
ceedings, 2643:293–305.
Morze, N., Kuzminska, O., and Protsenko, G. (2013). Pub-
lic information environment of a modern university.
CEUR Workshop Proceedings, 1000:264–272.
Ochoa-Orihuel, J., Marticorena-S
´
anchez, R., and S
´
aiz-
Manzanares, M. C. (2020). Moodle LMS Integration
with Amazon Alexa: A Practical Experience. Ap-
plied Sciences, 10(19). https://www.mdpi.com/2076-
3417/10/19/6859.
Oguguo, B. C. E., Nannim, F. A., Agah, J. J., Ugwuanyi,
C. S., Ene, C. U., and Nzeadibe, A. C. (2021).
Effect of learning management system on student’s
performance in educational measurement and eval-
uation. Education and Information Technologies,
26(2):1471–1483.
Silva, J. C. S., Zambom, E., Rodrigues, R. L., Ramos,
J. L. C., and da Fonseca de Souza, F. (2018). Ef-
fects of learning analytics on students’ self-regulated
learning in flipped classroom. International Journal
of Information and Communication Technology Ed-
ucation, 14(3). https://www.igi-global.com/gateway/
article/205624.
Vakaliuk, T. A., Spirin, O. M., Lobanchykova, N. M., Mart-
seva, L. A., Novitska, I. V., and Kontsedailo, V. V.
(2021). Features of distance learning of cloud tech-
nologies for the organization educational process in
quarantine. Journal of Physics: Conference Series,
1840(1):012051.
Moodle Tools for Educational Analytics of the Use of Electronic Resources of the University’s Portal
451