Cloud Labs as a Tool for Learning Cisco CyberSecurity Operations and

DevNet Associate Fundamentals Courses

Nadiia R. Balyk

1 a

, Yaroslav Ph. Vasylenko

1 b

, Vasyl P. Oleksiuk

1,2 c

, Olesia R. Oleksiuk

3 d

and

Galina P. Shmyger

1 e

Ternopil Volodymyr Hnatiuk National Pedagogical University, 2 M. Kryvonosa Str., Ternopil, 46027, Ukraine

Institute for Digitalisation of Education of the National Academy of Educational Sciences of Ukraine, 9 M. Berlynskoho

Str., Kyiv, 04060, Ukraine

Ternopil Regional Municipal Institute of Postgraduate Education, 1 V. Hromnytskogo Str., Ternopil, 46027, Ukraine

Keywords:

ICT-Competence, Cloud Lab, Apache CloudStack, Computer Science Trainee Teachers, Rasch Model, Cisco

Network Academy.

Abstract:

The article is devoted to the study of the problem of using the corporate cloud of the university in the process

of studying some courses of the Cisco Network Academy. Today, many universities have similar academies,

while others can open them. Based on the free software platforms Apache CloudStack and EVE-NG Com-

munity Edition, the authors have developed and implemented 2 cloud labs. One of them is designed to teach

the course “CCNA CyberOperations”, and other is “DevNet Associate Fundamentals Courses”. Both labora-

tories work on the IaaS model. Thanks to the technology of built-in virtualization, the work of many virtual

machines, storage of their state, trafﬁc analysis and visualization of network topologies is supported. The

article describes the experience of teaching students majoring in “Secondary Education. Computer Science”.

The authors conducted a survey of students who studied in the courses. The purpose of the survey was to

determine how satisﬁed the learners were with the course. Statistical processing of the results was performed

based on the Rasch model using MiniSteps software and R language. Students highly rated on-line curriculum

materials, access to virtual machines, clear and easy to understand lessons, presenting information in multiple

ways.

1 INTRODUCTION

Currently, the problem of intensifying the training of

future professionals is relevant. This problem is es-

pecially relevant for the process of teaching computer

science teachers (Ponomareva, 2021). This is because

the effectiveness of this process is the basis for prepar-

ing future generations for life in the global digital

world.

Various studies prove that the improvement of

learning is possible through the use of e-learning sys-

tems (Kuzminska et al., 2019; Vlasenko et al., 2020).

However, these tools alone are not enough. Among

the factors inﬂuencing the low effectiveness of the

https://orcid.org/0000-0002-3121-7005

https://orcid.org/0000-0002-2520-4515

https://orcid.org/0000-0003-2206-8447

https://orcid.org/0000-0002-1454-0046

https://orcid.org/0000-0003-1578-0700

introduction of e-learning is the lack of independent

work of students.

Today, the development of computer systems and

networks provides universal access to educational re-

sources. This led to the emergence of the concept of

open education (Kukharenko and Oleinik, 2019). One

of its modern tools is massive open online courses

(MOOC – Massive Open Online Cources) (Zinovieva

et al., 2021).

One way to solve this problem is to study open

courses by students. Their advantages are as follows:

the opportunity to study at a convenient time; the abil-

ity to compare teaching styles and materials of dif-

ferent courses; the experience in discussing and peer

assessment; improving the skills of listening, reading

and writing English (or other); reﬂection of their own

pedagogical activity in the light of new ideas, the digi-

tal creativity and collaboration with other participants

(Markova et al., 2018).

Cisco offers similar courses within Cisco Network

308

Balyk, N., Vasylenko, Y., Oleksiuk, V., Oleksiuk, O. and Shmyger, G.

Cloud Labs as a Tool for Learning Cisco CyberSecurity Operations and DevNet Associate Fundamentals Courses.

DOI: 10.5220/0010924000003364

In Proceedings of the 1st Symposium on Advances in Educational Technology (AET 2020) - Volume 1, pages 308-318

ISBN: 978-989-758-558-6

Academy. Although these courses do not fully corre-

spond to the ideology of the MOOC, Cisco Network

Academy can be organized at any education institute.

Cisco Networking Academy, a Cisco Corporate So-

cial Responsibility Program, is an IT skills and ca-

reer building program available to educational institu-

tions and individuals worldwide. Today, the company

is professing a paradigm for providing free access to

some courses to a wide range of users.

For example, scientists at The Open University of

the United Kingdom have integrated some Cisco Net-

work Academy courses into the training process of

computer science bachelors. The researchers substan-

tiated the effectiveness of the developed environment

and identiﬁed the key role of the instructor in teach-

ing students. A constructivist approach and blended

learning model were applied during the design and

testing of the course. It has proven to be an effec-

tive way to conduct Cisco courses. Such conclusions

of authors are conﬁrmed by the positive feedback of

students and their academic achievements (Moss and

Smith, 2010).

These and other studies conﬁrm that Cisco Net-

work Academy courses can be used effectively in the

study of computer sciences. This raises the problem

of giving students access to the objects of study. This

problem is especially relevant in courses on cyber-

security and network applications development. The

solution to this problem is possible through the in-

troduction of cloud technologies for virtualization of

objects of study in the courses of the Cisco Network

Academy.

The goal of this article is to describe the model of

cloud labs as learning tools in Cisco CyberSecurity

Operations and DevNet Courses and to research the

feedbacks of students about such labs.

2 RESULTS

As the experience of a secondary school shows, a

teacher of informatics is the leading ICT specialist

(Kuzminska et al., 2019). In the context of providing

information security (Savchenko et al., 2020), he must

be able to balance the advantages and disadvantages

of using digital technologies in the learning process.

We suggest using Cisco Network Academy Courses

to improve the training process for future computer

science teachers. At the same time there are problems

with the provision of learning tools. Cisco Network

Academy offers several solutions to the problem, such

as:

• Use simulators like Cisco Packet Tracer. This

approach is usually offered in courses related to

the study of computer networks. The simulator is

quite a powerful and affordable tool, but it simu-

lates only the basic functionality of network de-

vices.

• Work with online and cloud services. For exam-

ple, this approach is used in programming courses

to access API functions. However, these services

may change. As a result, course authors need

to constantly monitor changes and adjust learning

objectives.

• Deployment virtual machines. In this case, the

training takes place in an artiﬁcially created en-

vironment, which is created speciﬁcally for this

course and contains all the necessary tools.

It should be noted that the Cisco Network

Academy courses use each of the approaches. In the

context of our study, we will examine the latter ap-

proach. We used virtual machines in CCNA Cyber

Operations and DevNet courses. Having analyzed the

available free courses, we chose CCNA Cyber Oper-

ations (Cisco, 2019) as a basic course for formation

teachers’ cybersecurity competences. By the end of

this course, the students will be able to:

• Install virtual machines to analyzing cybersecu-

rity threat events.

• Explain the role of the Cybersecurity Operations

Analyst in the enterprise.

• Explain the Windows and Linux OS features to

support cybersecurity analyses.

• Analyze the operation of network protocols and

services.

• Classify the various types of network attacks and

identify network security alerts.

• Use network monitoring tools to identify attacks

against network protocols.

• Use various methods to prevent malicious access

to computer networks.

• Analyze network intrusion data to verify potential

exploits.

• Apply incident response models to manage net-

work security incidents.

The course contains the following chapters: Cy-

bersecurity and the Security Operation Center, Win-

dows OS, Linux OS, Network Protocol and Services;

Network Infrastructure, Principles of Network Secu-

rity, Network Attacks: A Deep Look, Protection the

network, Cryptography and the Public Key Infrastruc-

ture, Endpoint Security and Analysis, Security Moni-

toring, Intrusion Data Analysis, Incident.

Cloud Labs as a Tool for Learning Cisco CyberSecurity Operations and DevNet Associate Fundamentals Courses

309

To our opinion, the material of some chapters can

be considered in other courses (Operation Systems,

Computer Networks, Cryptography, etc.). Another

approach is to include these chapters in the content

of mentioned courses.

Each chapter contains terms and concepts review,

quiz, labs and exam. In the process of teaching the

course, we met with the problem of organizing labo-

ratory works. Cisco Network Academy offers to run

them on virtual student machines. This approach is

justiﬁed, but it limits the universal and everywhere ac-

cess of students to study. The use of separate virtual

machines does not ensure the cooperation of students

between themselves and with the teacher.

An effective way to overcome these limitations is

to use the cloud technologies. Bykov and Shyshkina

(Bykov and Shyshkina, 2018) note that the develop-

ment of cloud computing technologies, adaptive in-

formation and communication networks services, vir-

tual and mobile learning facilities are the important

step towards solving the problems of accessibility and

quality of training. Application of cloud technologies

in professional activities should correspond the re-

quirements of fundamentalization of learning through

the inclusion in the content general both the theoret-

ical and the technological provisions, with demon-

stration of them on the concrete examples (Merzlykin

et al., 2017; Bondarenko et al., 2019; Lovianova et al.,

2019; Spirin et al., 2019). Glazunova and Shyshk-

ina (Glazunova and Shyshkina, 2018) distinguishes

the following levels of the University Cloud-based

Learning and Research Environment: physical, level

of the virtualization and virtual resource management,

as well as platforms and software levels.

We deployed a cloud-based environment accord-

ing to the IaaS model. In the environment, the public

and private cloud platforms are integrated. Since the

corporate cloud platforms are widely using the virtu-

alization technology, we see as possible the deploy-

ment of cloud laboratories on their basis.

After analyzing the interpretation of Bykov et al.

(Bykov et al., 2020), we note that the cloud labora-

tory is an information system in which network vir-

tual ICT objects are formed thanks to a special user

interface, which is supported by the system software

of the network setting. Such objects are an integral

part of a logical network infrastructure with a ﬂexible

architecture that, according to its structure and time,

corresponds to the personality needs of the user.

Cornetta et al. (Cornetta et al., 2019) have inves-

tigated how digital fabrication laboratories can lever-

age cloud technologies to enable resource sharing and

provide remote access to distributed expensive fab-

rication resources over the Internet. They deployed

a cloud lab according to the new Fabrication as a

Service (FaaS) model. Researchers have developed

ﬁrmware and software for monitoring equipment and

providing real-time communications.

Gillet and Li (Gillet and Li, 2015) explore the con-

cept of cloud laboratories as common spaces that in-

tegrate applications. Researchers are also studying

the problem of integrating MOOC into the learning

environment. They note that cloud labs can enable

the implementation of connectivist MOOCs, allowing

teachers or students to collect and monitor the use of

openly available learning resources.

Typically, in a cloud laboratory, information from

a subject ﬁeld is based on some facts, and therefore

limited by a set of predicted experiments. Another ap-

proach suggests that a pupil or student is able to carry

out any experiments, not limited to a previously pre-

pared set of results. It is thanks to the use of the virtu-

alization technology of operating systems, the last ap-

proach should be tried to implement in the designed

laboratory. Cloud virtualization technologies provide

unique opportunities for the learning organization of

the Cisco CyberSecurity Operations course.

The designed virtual laboratory was imple-

mented in the cloud-based learning environment of

Volodymyr Hnatiuk Ternopil National Pedagogical

University. Based on the comparative analysis [8], as

the program basis of the laboratory, we have chosen

the Apache CloudStack platform. Then we modiﬁed

the Cloud-based Learning Environment so that stu-

dents could create virtual networks. This networks

should not require changes in the topology of physi-

cal networks in the academic cloud. We divided the

trafﬁc transmitted between students’ virtual comput-

ers among 100 VLANs. So each student has an op-

portunity to store their virtual computers and other

devices in their personal or several guest networks.

As Apache CloudStack does not provide tools for

visualization of network structure, students often have

difﬁculty in designing and conﬁguring networks in

a cloud infrastructure. That fact prompted us to in-

tegrate into a virtual cloud laboratory a system that

makes it possible to visualize the process of network

design. It was vital that such system could work with

networks on Apache CloudStack virtual machines.

We analyzed relevant publications and compared sev-

eral platforms – Cisco packet tracer, Graphical Net-

work Simulator (GNS), Unetlab (EVE-NG). Despite

the beneﬁts of Cisco packet tracer, it did not provide

the performance of all tasks of the laboratory works.

Among the platforms of GNS and EVE-NG, we have

chosen the last.

Every student’s copy of ENE-NG platform is

a separate virtual machine in Apache CloudStack

AET 2020 - Symposium on Advances in Educational Technology

310

cloud. As each node of EVE-NG is itself a virtual

machine, hosts integrated in Apache CloudStack in-

frastructure have to support nested virtualization.

The laboratory works involves the use of such

virtual machines: CyberOps WorkStation (based on

Arch Linux); Kali Linux; Security Onion (based on

Ubuntu Linux); Metasploitable; Windows Client.

The students used a virtual cloud laboratory when

performing the following laboratory works:

1. Chapter 2: Windows Operating System. 2.0.1.2

Lab – Identify Running Processes; 2.1.2.10 Lab –

Exploring Processes, Threads, Handles, and Win-

dows Registry; 2.2.1.10 Lab – Create User Ac-

counts; 2.2.1.11 Lab – Using Windows Power-

Shell; 2.2.1.12 Lab – Windows Task Manager;

2.2.1.13 Lab – Monitor and Manage System Re-

sources in Windows.

2. Chapter 3: Linux Operating System. 3.1.2.6

Lab – Working with Text Files in the CLI; 3.1.2.7

Lab – Getting Familiar with the Linux Shell;

3.1.3.4 Lab – Linux Servers; 3.2.1.4 Lab – Locat-

ing Log Files; 3.2.2.4 Lab – Navigating the Linux

Filesystem and Permission Settings.

3. Chapter 4: Network Protocols and Services.

4.1.1.7 Lab – Tracing a Route; 4.1.2.10 Lab –

Introduction to Wireshark; 4.4.2.8 Lab – Using

Wireshark to Examine Ethernet Frames; 4.5.2.4

Lab – Using Wireshark to Observe the TCP 3-

Way Handshake; 4.5.2.10 Lab – Exploring Nmap;

4.6.2.7 Lab – Using Wireshark to Examine a UDP

DNS Capture; 4.6.4.3 Lab – Using Wireshark to

Examine TCP and UDP Captures; 4.6.6.5 Lab –

Using Wireshark to Examine HTTP and HTTPS;

4. Chapter 7: Network Attacks. 7.0.1.2 Lab – What

is Going On? 7.3.1.6 Lab – Exploring DNS Traf-

ﬁc; 7.3.2.4 Lab – Attacking a MySQL Database;

7.3.2.5 Lab – Reading Server Logs; Chapter 9:

Cryptography and the Public Key Infrastructure;

9.0.1.2 Lab – Creating Codes; 9.1.1.6 Lab – En-

crypting and Decrypting Data Using OpenSSL;

9.1.1.7 Lab – Encrypting and Decrypting Data us-

ing a Hacker Tool; 9.1.1.8 Lab – Examining Tel-

net and SSH in Wireshark; 9.1.2.5 Lab – Hashing

Things Out; 9.2.2.7 Lab – Certiﬁcate Authority

Stores;

5. Chapter 12: Intrusion Data Analysis. 12.1.1.7

Lab – Snort and Firewall Rules; 12.2.1.5 Lab –

Convert Data into a Universal Format; 12.2.2.9

Lab – Regular Expression Tutorial; 12.2.2.10

Lab – Extract an Executable from a PCAP;

12.4.1.1 Lab – Interpret HTTP and DNS Data

to Isolate Threat Actor; 12.4.1.2 Lab – Isolated

Compromised Host Using 5-Tuple.

A typical topology of the network for the labora-

tory works is showed in ﬁgure 1.

Each of these machines was available in a cloud-

based infrastructure. As a result, students could work

with virtual machines in the university’s local network

or through VPN. The course was taught in a mixed

methodology. It was dominated by independent dis-

tance work of students. The teacher’s consultations

were carried out at the classroom and online.

We have deployed a cloud lab for the Cisco De-

vNet Associate Fundamentals course. The course is

dedicated to the development of competencies for a

IT professionals, empowering organizations to em-

brace the potential of applications, infrastructure for

the network, Internet of Things (IoT), Webex, etc.

The course is also good because it can be completed

by students with low levels of programming skills The

DevNet course has the following modules:

1. Introduction. The module is devoted to the organi-

zation of the learning environment. Since students

will be working in a cloud lab, we have modiﬁed

this section a bit. In particular, they explained how

to deploy a VM, what its parameters should be

speciﬁed, how to connect to it remotely.

2. The DevNet developer environment. There are

opportunities to learn more through such features

as: learning labs, sandboxes, developers’ docu-

mentation and support.

3. Software Development and Design Content. The

software development life cycle is the main con-

cept of this module. A phases of this process are

also discussed in the module.

4. Understanding and Using APIs. In this mod-

ule, students study API Design and Architectural

Styles. The REST API is presented in more detail

5. Introduction to Network Fundamentals. The basic

concepts of computer networks based on models

OSI and TCP/IP are considered in this module.

6. Application Deployment and Security. Students

are introduced to application deployment models

such as virtual machines, containers, and server-

less computing.

7. Infrastructure and Automation. In this topic stu-

dents use a code to conﬁgure, deploy, and manage

applications together with the compute, storage,

and network infrastructures and services.

8. Cisco Platforms and Development. The module

will be useful for students to further their career

development. The topic describes Cisco Dev Cen-

ters. Those Dev Centers are a convenient way of

grouping technologies together.

Cloud Labs as a Tool for Learning Cisco CyberSecurity Operations and DevNet Associate Fundamentals Courses

311

Figure 1: The network topology for labs.

The course offers to use a virtual machine based

on free VirtualBox software. However, we modiﬁed

it and created a VM template for the Apache Cloud-

Stack platform.

The VM runs on Ubuntu Linux and includes the

following learning tools: interpreter of Python pro-

gramming language, Visual Studio Code IDE, Post-

man (The Collaboration Platform for API Develop-

ment), command-line utility Git, Cisco Packet Tracer,

etc.

For example, VM was used to create a chatbot in

the laboratory work “Integrating a REST API with

Python”. Students used the REST-API to work with

MapQuest, ISS Location and Webex Teams. Chatbot

read messages from the Webex Teams room in JSON

format, performed their parsing, found messages with

the name of the city. In the next step, the script called

the API of the MapQuest service to determine the ge-

ographical coordinates of the city. Another step was

to determine the nearest time for observation of the

International Space Station in this city. In the last

step, the chatbot sent a reply message to the Webex

Teams room.

After learning this courses, students completed the

ﬁnal exam. He contained 60 questions from all the

topics of the course, as well as the fragments of lab-

oratory works. 56 students majoring in “Secondary

education. Computer Science” passed the exam. Of

these, only 24 passed the exam successfully. This in-

dicator can be explained by the fact that the course

“Cyber Operations” was studied as optional and did

not affect the student’s rating at the university.

In addition to the ﬁnal exam students responded

to the questionnaire “Cyber Operations Course Feed-

back”. Questionnaire questions were formulated ac-

cording to the principle of the Likert scale (ﬁve re-

sponse categories) and grouped in 5 blocks (table 1).

3 STATISTICAL ANALYSIS OF

RESEARCH DATA

To evaluate the efﬁciency of the designed and de-

ployed cloud-based labs, a model with equally dis-

tributed responses of all indicators on the scale of the

latent variable was used. This is one of the models

of the Rasch’s family, which is used in the case of

polithomus indicators.

In the modern Item Response Theory (IRT),

Rasch’s model allows us to assess the meaning of la-

tent variables, to investigate the relationship between

them, and to identify factors that inﬂuence the behav-

ior of latent variables. IRT is based on the theory of

latent-structural analysis: the ﬁnal score is considered

as a result of the combined interaction of latent pa-

rameters – the true level of preparation of students

and the complexity of the questions (tasks). This ap-

proach to the evaluation of the studied features in IRT

theory differs signiﬁcantly from the classical test the-

ory, in which the result is the ﬁnal score in a particular

survey, corrected for error.

The Rasch’s model is interpreted as a model of

“objective measurements” that do not depend from

the respondents and measuring instruments. The

Rasch’s model is based on three assumptions (Bond

AET 2020 - Symposium on Advances in Educational Technology

312

Table 1: List of distractors (items) in questionnaire.

Distractor Code Description

Please rate your level

of satisfaction with

the following aspects

of this course

(Course Satisfaction)

CS1 On-line Curriculum Materials

CS2 Labs

CS3 Access to Equipment/Software

CS4 Classroom Instruction

CS5 Assessments

Please rate how

conﬁdent you feel in

your ability to do

each of the following

(Conﬁdent Ability)

CA1 Explain the role of the Cybersecurity Operations Analyst

CA2 Explain the Windows and Linux OS features and characteristics needed to

support cybersecurity analyses

CA3 Explain the operation of the network infrastructure and various types of net-

work attacks

CA4 Analyze the operation of network protocols and services, and identify attacks

against them

CA5 Use various methods to prevent malicious access to computer networks,

hosts, and data

CA6 Explain how to investigate endpoint vulnerabilities

CA7 Evaluate network security alerts and identify compromised hosts and vulner-

abilities

Compare your

instructor to other

instructors you have

had in terms of:

(Compare instructor)

CI1 Preparedness to teach the course

CI2 Clear and easy to understand lessons

CI3 Approachability with questions and ideas

CI4 Presenting information in multiple ways

CI5 Making the topic interesting

Please rate how

much you agree with

the next statements

(Course Content)

CC1 The lab activities helped me to achieve the stated course objectives

CC2 The exam scores reﬂected my understanding of the curriculum

CC3 Having access to equipment helped me learn

CC4 The course curriculum was technically accurate

To what extent did

this course help you

(Course Purpose)

CP1 Prepare for Certiﬁcation exam(s)

CP2 Learn skills that can be used in a future job

CP3 Increase your value in the job market

CP4 Obtain a new job or advance in your current job

et al., 2021):

1. The level of difﬁculty of tasks and the level of

preparedness of persons being tested can be mea-

sured in one scale, with a common standard unit

of measurement.

2. In the presence of such a scale the probability of

the correct answer of the tested person depends on

the difference between his level of preparedness

and the level of complexity of the test task.

3. The outcome of the confrontation of the tested

person with the test tasks can be predicted. If

the level of preparedness of the tested person is

higher, than the probability of his correct answer

to the task of a ﬁxed level of complexity should be

higher.

To measure the complexity of tasks and level of

knowledge, the unit of measurement, called logit, is

used. In our research, we used the WINSTEPS pro-

gram. The program is commercial, but its free version

called MINISTEP. It allows you to use all the capabil-

ities of WINSTEPS, but has a limit on the number of

questions in the test (25) and the number of people

(75) (WINSTEPS, 2019).

Standardized Residuals in the Rasch’s model are

modeled for normal distribution. Therefore, signiﬁ-

cant deviations from the value of “0” for the Mean

and the value “1” for the Standard Deviation (SD)

signal that the primary data do not correspond to the

Rasch’s model, which should correspond exactly to

the normal distribution. In our study, the values Mean

= -0.02 and SD = 1.03 are sufﬁciently satisfactory.

The classic indicator of reliability of the survey

scale is alpha Kronbach. Reliability is the consis-

tency of the results within a single test. Alpha Kro-

nbach points to the degree to which all items actu-

ally measure the same property (quality). It should be

noted that the high value of the coefﬁcient indicates

the existence of a general basis in the formulated set

of questions. Professionally designed tests must have

Cloud Labs as a Tool for Learning Cisco CyberSecurity Operations and DevNet Associate Fundamentals Courses

313

an internal consistency of at least 0.90. In our survey,

the Cronbach coefﬁcient α=0.96.

As can be seen from ﬁgures 2 and 3, informa-

tional and characteristic functions are acceptable for

IRT analysis.

Figure 2: Information function.

Figure 3: Characteristic function.

Person raw score-to-measure correlation = 1.00.

Cronbach Alpha (kr-20) person raw score “test”

reliability = 0.96, sem = 4.07.

Item raw score-to-measure correlation = -1.00.

In columns INFIT and OUTFIT of tables 2 and 3

speciﬁed parameters that characterize the correspon-

dence of the data to Rasch’s model. In the ﬁeld

MNSQ (mean-square statistic) the statistics of the

correspondence of the output data to the measuring

model are showed, obtained on the base of the aver-

age sums of the squares of the deviations of the the-

oretical values from the empirical ones. The MNSQ

values characterize the degree of “randomness” of the

results or the discrepancy of the data to the used mea-

surement model. Expected MNSQ values are near 1.

The high MNSQ OUTFIT values can be associated

with the “casual” respondents’ responses. The high

values of MNSQ INTFIT are usually interpreted as

an indicator of the low validity of the tool, that is, the

low suitability of the tool for the tasks for which it was

developed. The most qualitative and signiﬁcant (pro-

ductive) measurements are those for which the MNSQ

values lie in the range of 0.5 to 1.5. Higher values

(> 1.5) indicate uncertainty and “noise” in input data.

Too low values (<0.5) are also not very desirable be-

cause they indicate excessive, “information overload”

of the instrument. In the ZSTD ﬁeld, the standardized

MNSQ values are showed (with an average of 0 and a

standard deviation of 1). Valid value is -2.0 ≤ ZSTD

≤ +2.0.

For this survey, the match statistics for the mea-

surements of all items are in this range, so they can

all be used for further analysis.

Figure 4 shows the distribution of respondents and

their judgments on the same interval scale (efﬁciency

of the designed and deployed cloud-based environ-

ment). The content and composition of the questions

in the survey is satisfactory – this is evident from the

second bar graph on Figure 4. However, respondents’

answers to the questions posed are not balanced. This

means that some respondents answered randomly or

could not orient themselves with the choice of an ad-

equate response.

By analyzing table 4 in terms of the distractors in-

cluded in the poll, the following conclusions can be

drawn. Distractors with the lowest estimate of the ef-

ﬁciency of the proposed medium (Measure = -1.08,

Item = CC3) and with the highest estimate of the ef-

ﬁciency (Measure = 0.75, Item = CA6) are not pre-

sentational for this study, since, as noted above, on

the responses had an impact the factor of randomness

and the factor of reluctance of respondents to under-

stand the content of the questions deeply. The rest

of the distractors can be divided into three groups ac-

cording to the degree of inﬂuence on the overall ef-

ﬁciency: 1) with a small degree of inﬂuence on the

overall efﬁciency (Measure from -0.43 to -0.12, Items

= CP1, CP3, CP2, CS5, CA3, CP4, CI5); 2) with a

mediocre degree (Measure from -0.09 to 0.07, Items =

CS2, CS4, CI1, CC1, CI3, CA4, CA1, CA5); 2) with

a large degree of impact on overall efﬁciency (Mea-

sure from 0.13 to 0.41, Items = CA2, CC4, CI2, CC2,

CI4, CA7, CS3, CS1). The analysis of these distrac-

AET 2020 - Symposium on Advances in Educational Technology

314

Table 2: Output table “Summary Statistics” (summary of 56 measured person).

Total Score Count Measure Model S.E.

INFIT OUTFIT

MNSQ ZSTD MNSQ ZSTD

MEAN 72.1 25.0 -0.20 0.25 1.06 -0.13 1.07 -0.14

SEM 2.9 0 0.18 0.00 0.08 0.29 0.09 0.29

P.SD 21.5 0 1.30 0.03 0.62 2.27 0.65 2.14

S.SD 21.6 0 1.31 0.03 0.63 2.19 0.66 2.16

Table 3: Output table “Summary Statistics” (summary of 25 measured item).

Total Score Count Measure Model S.E.

INFIT OUTFIT

MNSQ ZSTD MNSQ ZSTD

MEAN 161.6 56.0 0.00 0.17 0.98 -1.13 1.07 -0.80

SEM 2.5 0 0.07 0.00 0.17 0.83 0.19 0.88

P.SD 12.2 0 0.34 0.00 0.84 4.08 0.95 4.32

S.SD 12.4 0 0.35 0.00 0.85 4.16 0.97 4.41

Figure 4: The relationship between the level of efﬁciency of

the designed and deployed cloud-based virtual lab and the

indicator variables.

tors at the content level will allow for the adjustment

of the structure, some components in design of vir-

tual cloud labs for the learning Cisco CyberSecurity

Operations.

To further analyze the study data, we used the

means of the R language in the RStudio environment.

Currently, the MIRT package (Full-Information Item

Factor Analysis (Multidimensional Item Response

Theory)) is one of the most effective means of the

R language to work with the Rasch model (Liu and

Chalmers, 2018). This is open-source software, use-

ful for real data analysis and research and provides

a didactic tool for teaching IRT. It has no limits on

the number of respondents or answers to questions.

We used the mirt function to process and visualize the

data. Here is the function call:

mod <− m i r t ( data = e x p da t a ,

i t e m t y p e = ” Rasch ” , model = 1)

where

• expdata is a data frame with students’ grades (link

was provided above).

• itemtype is a type of items to be modeled. A value

of ’Rasch’ means that a credit model will be built

by constraining slopes to 1 and freely estimating

the variance parameters.

• model is a model to be built. A value of “1” means

a unidimensional model.

To estimate the frequency of students’ grades on

all distractors, we constructed a histogram of response

frequencies (ﬁgure 5). To do this, we used the P-

function such as:

h i s t ( d , b r e a k s =c ( 0 : 5 ) , f r e q =TRUE,

c o l = ” b l u e ” ,

x l a b =” R e s p on se s ” ,

y l a b =” F r e q ue nc y ” ,

main= ” F re q u e nc y d i a gr am ” )

The vector d is obtained from the full dataframe by

extracting the header. That is, it contains columns of

data without distractors.

Figure 5 shows that the answers at levels 4 and 5

were given the least. We can explain this by the fact

that the proposed approach to the study of disciplines

is innovative. Therefore, there is vigilance and cau-

tion of students to use it in the learning process.

To assess how clear the content of the distractors

was for the respondents, we constructed a diagram us-

ing the next function.

p l o t ( mod1 , t y p e = ’ i n f o ’ ,

xl i m = c ( −4 , 4 ) , y l i m =c ( 0 , 4 0 ) )

Cloud Labs as a Tool for Learning Cisco CyberSecurity Operations and DevNet Associate Fundamentals Courses

315

Table 4: Item statistics: measure order.

Entry number Total Score Total Count Measure Model S.E. Item

11 135 56 0.75 0.17 CA6

1 147 56 0.41 0.17 CS1

3 149 56 0.35 0.17 CS3

12 151 56 0.29 0.17 CA7

16 151 56 0.29 0.17 CI4

19 154 56 0.21 0.17 CC2

14 155 56 0.18 0.17 CI2

21 156 56 0.15 0.17 CC4

7 157 56 0.13 0.17 CA2

10 159 56 0.07 0.17 CA5

6 160 56 0.04 0.17 CA1

9 160 56 0.04 0.17 CA4

15 160 56 0.04 0.17 CI3

18 160 56 0.04 0.17 CC1

13 161 56 0.02 0.17 CI1

4 162 56 -0.01 0.17 CS4

2 165 56 -0.09 0.17 CS2

17 166 56 -0.12 0.17 CI5

25 168 56 -0.18 0.17 CP4

8 169 56 -0.20 0.17 CA3

5 170 56 -0.23 0.17 CS5

23 172 56 -0.29 0.17 CP2

24 176 56 -0.40 0.17 CP3

22 177 56 -0.43 0.17 CP1

20 200 56 -1.08 0.17 CC3

Mean 161.60 56.00 0.00 0.17

P.SD 12.20 0.00 0.34 0.00

Figure 5: Histogram of response frequencies.

From the graph 6 of the information function we

can conclude that the tasks of the polytomy type are

the most informative for respondents with a level of

training from -1 to 2 logs. This suggests that for stu-

dents with an average level of preparation or slightly

higher, the formulated questions were the most infor-

Figure 6: Graph of the information function of the question-

naire.

mative. The shape of the information curve (bell-

shaped) indicates that the distractors were selected

correctly and their description was made correctly.

The ﬁgure 7 shows the graphs of the characteristic

functions of the responses to all distractors.

AET 2020 - Symposium on Advances in Educational Technology

316

Figure 7: Characteristic curves of response levels.

As can be seen from these graphs, the probabil-

ity of putting 1 point in students with a low level of

preparation and the probability of putting 5 points in

students with a high level of preparation was approx-

imately 0.9. This is the case for all distractors. The

probability of setting an average score by a student

with an average level of preparation is low. But this

is due to the higher frequency of averaging by most

students.

4 CONCLUSIONS

The problem of integrating cloud-based tools and

open online courses in the process of training future

computer science teachers is relevant and needs fur-

ther research. Cloud labs are one such form of in-

tegration. They ensure ubiquity and cooperation in

learning. In particular, the authors deployed cloud

labs to support training in Cisco CyberSecurity Oper-

ations and DevNet Associate Fundamentals Courses.

Learning the basics of cybersecurity is a topical

issue of ICT students training. The course “CCNA

Cyber Operations” of Cisco Network Academy pro-

vides an opportunity to organize such training. It con-

tains a lot of theoretical materials, quiz tasks, discus-

sion questions, labs, chapters exams and ﬁnal exam.

A virtual cloud laboratory was designed to carry out

laboratory works at the course. For this purpose, the

Apache CloudStack and EVE-NG Community Edi-

tion platforms were used. The virtual cloud labora-

tory provides the following possibilities: to create the

required number of virtual machines; to change the

computing power; to simulate the work of real com-

puters and networks; to visualize different network

topologies; to keep the state of virtual computers; to

work remotely through a virtual private network; to

combine separate virtual networks of students into

a single network; to help students and control their

learning outcomes.

DevNet Associate Fundamentals Courses is a very

successful integrated course. It gives students the op-

portunity to put into practice theoretical lessons in

networking and programming. It is also important to

teach students to work with modern APIs. So future

professionals will be able to create applications that

Cloud Labs as a Tool for Learning Cisco CyberSecurity Operations and DevNet Associate Fundamentals Courses

317

process data obtained from the clouds. The course

also demonstrates modern automation tools for the

deployment of network and cloud infrastructures. The

cloud lab also provides great learning opportunities in

the DevNet course. In it, students can run VMs with

basic development tools, run and test their application

for a long time.

The conducted research and its statistical process-

ing have limitations. They are associated with a small

number of students have participated in the experi-

ment. This sample size did not allow us to conduct a

qualitative experiment to verify the statistical differ-

ences between the control and experimental groups.

Nevertheless, statistical processing of the question-

naire “Course feedback” given by all students (even

those who did not pass the ﬁnal exam) indicates ef-

ﬁciency of the use of the deployed cloud laborato-

ries. Along with high-quality training materials from

the Cisco Network Academy, the students appreciated

highly the functional and widespread access to the vir-

tual objects of the cloud labs.

REFERENCES

Bond, T., Yan, Z., and Heene, M. (2021). Applying the

Rasch model: Fundamental measurement in the hu-

man sciences. 4 edition.

Bondarenko, O., Pakhomova, O., and Zaselskiy, V. (2019).

The use of cloud technologies when studying geogra-

phy by higher school students. CEUR Workshop Pro-

ceedings, 2433:377–390.

Bykov, V., Mikulowski, D., Moravcik, O., Svetsky, S., and

Shyshkina, M. (2020). The use of the cloud-based

open learning and research platform for collaboration

in virtual teams. Information Technologies and Learn-

ing Tools, 76(2):304–320. https://journal.iitta.gov.ua/

index.php/itlt/article/view/3706.

Bykov, V. Y. and Shyshkina, M. P. (2018). The con-

ceptual basis of the university cloud-based learning

and research environment formation and development

in view of the open science priorities. Information

Technologies and Learning Tools, 68(6):1–19. https:

//journal.iitta.gov.ua/index.php/itlt/article/view/2609.

Cisco (2019). Cisco CCNA Cyber Ops. https://www.cisco.

com/c/en/us/training-events/training-certiﬁcations/

certiﬁcations/associate/ccna-cyber-ops.html.

Cornetta, G., Mateos, J., Touhaﬁ, A., and Muntean, G.-M.

(2019). Design, simulation and testing of a cloud plat-

form for sharing digital fabrication resources for edu-

cation. Journal of Cloud Computing, 8(1):12.

Gillet, D. and Li, N. (2015). Case study 2: designing PLE

for higher education. In Kroop, S., Mikroyannidis, A.,

and Wolpers, M., editors, Responsive Open Learning

Environments: Outcomes of Research from the ROLE

Project, pages 115–133. Springer International Pub-

lishing, Cham.

Glazunova, O. and Shyshkina, M. (2018). The concept,

principles of design and implementation of the univer-

sity cloud-based learning and research environment.

CEUR Workshop Proceedings, 2104:332–347.

Kukharenko, V. and Oleinik, T. (2019). Open distance

learning for teachers. CEUR Workshop Proceedings,

2393:156–169.

Kuzminska, O., Mazorchuk, M., Morze, N., and Kobylin,

O. (2019). Attitude to the digital learning environment

in Ukrainian universities. CEUR Workshop Proceed-

ings, 2393:53–67.

Liu, C.-W. and Chalmers, R. P. (2018). Fitting item re-

sponse unfolding models to Likert-scale data using

mirt in R. PLOS ONE, 13(5):1–22.

Lovianova, I., Bobyliev, D., and Uchitel, A. (2019). Cloud

calculations within the optional course Optimization

Problems for 10th-11th graders. CEUR Workshop

Proceedings, 2433:459–471.

Markova, O. M., Semerikov, S. O., Striuk, A. M., Shalatska,

H. M., Nechypurenko, P. P., and Tron, V. V. (2018).

Implementation of cloud service models in training

of future information technology specialists. CEUR

Workshop Proceedings, 2433:413–428.

Merzlykin, P., Popel, M., and Shokaliuk, S. (2017). Ser-

vices of SageMathCloud environment and their di-

dactic potential in learning of informatics and math-

ematical disciplines. CEUR Workshop Proceedings,

2168:13–19.

Moss, N. and Smith, A. (2010). Large scale delivery of

cisco networking academy program by blended dis-

tance learning. In 2010 Sixth International Confer-

ence on Networking and Services, pages 329–334.

Ponomareva, N. S. (2021). Role and place of informatics in

the training of future teachers of mathematics. Journal

of Physics: Conference Series, 1840(1):012035.

Savchenko, S., Shekhavtsova, S., and Zaselskiy, V. (2020).

The development of students’ critical thinking in the

context of information security. CEUR Workshop Pro-

ceedings, 2731:383–399.

Spirin, O., Oleksiuk, V., Balyk, N., Lytvynova, S., and

Sydorenko, S. (2019). The blended methodology of

learning computer networks: Cloud-based approach.

CEUR Workshop Proceedings, 2393:68–80.

Vlasenko, K., Chumak, O., Achkan, V., Lovianova, I., and

Kondratyeva, O. (2020). Personal e-learning environ-

ment of a mathematics teacher. Universal Journal of

Educational Research, 8(8):3527–3535.

WINSTEPS (2019). WINSTEPS & Facets Rasch Software

. http://www.winsteps.com/index.

Zinovieva, I. S., Artemchuk, V. O., Iatsyshyn, A. V., Roma-

nenko, Y. O., Popov, O. O., Kovach, V. O., Taraduda,

D. V., and Iatsyshyn, A. V. (2021). The use of MOOCs

as additional tools for teaching NoSQL in blended and

distance learning mode. Journal of Physics: Confer-

ence Series, 1946(1):012011.

AET 2020 - Symposium on Advances in Educational Technology

318