360

◦

Photographic Panoramas as an Effective Multifunctional Aid for

Teaching Technology Subjects

Igor V. Barkatov

1 a

, Volodymyr S. Farafonov

1 b

, Valeriy O. Tiurin

1 c

, Serhiy S. Honcharuk

1 d

Andrei A. Lozko

1 e

, Volodymyr V. Marushchenko

1 f

, Kostyantyn V. Korytchenko

1 g

Vitaliy I. Barkatov

and Roman F. Muravlyov

National Technical University “Kharkiv Polytechnic Institute”, 2 Kyrpychova Str., Kharkiv, 61002, Ukraine

Innovative Distance Learning Systems Ltd., 30 Iuvileynyy Ave., Kharkiv, 61038, Ukraine

Keywords:

360

◦

Panorama, Virtual Tour, H5P Interactive Content, Teaching Video, VR Glasses, Simulator, Vehicle

Cabin.

Abstract:

Rapid development of modern machinery and its increasing complexity make high demands to the quality of

training of its users. Among the variety of kinds, the important one is vehicles, both civil and military. In the

teaching of associated subjects, there is an accepted hierarchy of teaching aids that includes common visual

aids (posters, videos, scale models etc.) on the ﬁrst stage, followed by simulators ranging in complexity, and

ﬁnishing at real vehicles. It allows achieving some balance between cost and efﬁciency of training by partial

replacement of more expensive and elaborated aids with the less expensive ones. However, the analysis of

teaching experience in the Institute reveals that this balance is still suboptimal: the quality and abundance of

common aids may be increased, and the simulators may be used more effective. This fact raises the problem of

extending the range and quality of available teaching aids for vehicle-related subjects, which is the aim of the

work. Beneﬁting from the modern information and visualization technologies, we present a collection of new

teaching aids, which are based on 360

◦

(spherical, 3D) photographic panoramas joined with H5P interactive

content framework or virtual reality devices. The nature of the aids, their potential applications, limitations

and beneﬁts in comparison to the common aids are discussed, and the practical recommendations about cre-

ating and implementing the aids are given. The proposed aids are shown to be cost-effective and proven to

increase efﬁciency of training, according to the results of a teaching experiment. For the implementation, a

tight collaboration between the Institute and an IT company was established. The authors conclude that the

proposed aids may signiﬁcantly improve the cost-efﬁciency balance of teaching technology subjects.

1 INTRODUCTION

Technology plays a vital role in modern world. At

present, most occupations and activities imply utiliza-

tion of some devices and equipment. Among the va-

riety of classes, an important representative is vehi-

cles. A wide assortment is designed and extensively

used in civil (transport, building, service etc.), mili-

https://orcid.org/0000-0003-2605-574X

https://orcid.org/0000-0003-0785-9582

https://orcid.org/0000-0003-3311-9043

https://orcid.org/0000-0001-5607-1033

https://orcid.org/0000-0002-3868-9064

https://orcid.org/0000-0002-5965-6384

https://orcid.org/0000-0002-1005-7778

tary, and paramilitary (emergency, police) ﬁelds. The

following features are typical for modern vehicles and

their exploitation process:

• increasing complexity of the chassis itself and the

installed equipment;

• fast development, resulting in frequent appear-

ance of upgraded and novel models;

• often, hard use conditions (especially for military

and paramilitary vehicles);

• high costs of repairing and replacement of broken

samples.

Consequently, extensive knowledge about the

proper exploitation of the vehicle and related skills

must be delivered to trainees during education in or-

der for them to become qualiﬁed users.

Barkatov, I., Farafonov, V., Tiurin, V., Honcharuk, S., Lozko, A., Marushchenko, V., Korytchenko, K., Barkatov, V. and Muravlyov, R.

360 Photographic Panoramas as an Effective Multifunctional Aid for Teaching Technology Subjects.

DOI: 10.5220/0010927500003364

In Proceedings of the 1st Symposium on Advances in Educational Technology (AET 2020) - Volume 1, pages 691-701

ISBN: 978-989-758-558-6

691

2 RELATED WORK

In the teaching of vehicle-related subjects, there is an

established and accepted hierarchy of teaching meth-

ods and corresponding aids (Course, 2019). It is sum-

marized in table 1.

Table 1: The accepted hierarchy of teaching aids in teaching

vehicle-related subjects.

Stage Teaching

aids

Goals

1 Common

visual aids

(posters,

animations,

videos, scale

models etc.)

Provide the knowledge about

the constitution, functioning,

appearance, and exploitation of

the vehicle. No skill developing

is assumed.

2 Simulators The purpose is two-fold.

Firstly, providing informa-

tion about appearance and

exploitation of the vehicle.

Secondly, a more or less wide

range of skills may be trained,

depending on the class of

simulator.

3 Real vehicles Providing real-world driving

experience and developing ex-

ploitation skills.

An extensive literature, both pedagogical and

technical, is available about the problems of design

and use of simulators (Rusilo, 2010; Rudkovs’kyy,

2013; Prihodko, 2009; Vakaliuk et al., 2020). The

transition from the ﬁrst-stage aids to the last-stage

ones is characterized by two trends. On one hand,

the trainee’s experience becomes more relevant to the

real-world use experience. On the other hand, expen-

diture for material resources and time per one trainee

increases, as well. The reasons are manifold:

• a vehicle and, to a lesser extent, a simulator are

expensive to obtain and maintain;

• exploiting vehicles is resource-expensive;

• each vehicle or simulator is able to accommo-

date a single trainee only and, thus, have very low

throughput: each trainee has to enter and leave it

one by one.

The above hierarchy is aimed to balance the qual-

ity and cost of training, which is to provide the best

training for a given budget, by partial replacement of

more expensive aids with the less expensive ones. The

stated aim is actually achieved.

3 STATEMENT OF PROBLEMS

Nevertheless, the analysis of teaching experience col-

lected in the Institute reveals that the reached balance

is still suboptimal.

The ﬁrst identiﬁed deﬁciency is the extensive use

of simulators just as advanced visual aids, when they

act simply as 1:1 scale models. Their purpose here is

just familiarizing the trainees, who are already taught

with textbooks, posters, videos, with the actual ap-

pearance of the vehicle cabin (location of controls,

indicators, handles etc.). “Familiarizing” here means

establishing the connection between the remembered

ﬂat two-dimensional pictures of the cabin with its ac-

tual spatial three-dimensional appearance and work-

ing out the head, arms, hands movements needed to

activate the learned controls. This fact leads to the

next problems:

• Trainees are able to occupy the simulator one by

one only, extending the duration of the class (i.e.

the throughput is very low).

• The time available for using the simulator at

its full capacity for developing skills by other

trainees is, thus, reduced.

• In education establishments, which do not possess

a simulator, the trainees are unable to receive this

kind of training.

Let us consider a simple example. In a group of

15 trainees and one teacher during a 75 minutes class

each person will receive just 5 minutes of experienc-

ing the simulator in the best case (i.e. no prelimi-

nary instruction is needed, entering and leaving the

cabin occur rapidly etc.). Importantly, the teacher

is focused on the single trainee sitting in the cabin

and, thus, cannot perform teaching with the rest of

the group. Simultaneous utilization of 3–4 simulators

may improve the situation but requires corresponding

expenses. The reason of such unpractical use of simu-

lators is the absence of other teaching aids, which can

be employed instead. In other words, there is a pro-

nounced gap between the ﬁrst and second hierarchy

positions, which is forcedly ﬁlled by simulators. It is

illustrated in ﬁgure 1 where the particular sensational

features provided by the discussed teaching aids are

shown (Barkatov et al., 2020).

In the described situation only the two basic sensa-

tional features of simulators are used out of four that

is evidently suboptimal.

The second identiﬁed deﬁciency is while the basic

teaching aids (posters and videos) are generally avail-

able at present, their abundance is still limited, and

technical level often does not meet the standards of

AET 2020 - Symposium on Advances in Educational Technology

692

Fig. 1. Sensational features provided by the teaching aids in the accepted hierarchy. The com-

plexity and cost of implementation increase top to bottom and left to right. The question mark

represents the gap.

In the described situation only the two basic sensational features of simulators are

used out of four that is evidently suboptimal.

The second identified deficiency is while the basic teaching aids (posters and vid-

eos) are generally available at present, their abundance is still limited, and technical

level often does not meet the standards of modern technology. Videos and animations

are valuable means for teaching technical disciplines, which study the structure and

functioning of mechanisms and systems. If the former is relatively easy to learn by

means of static posters, drawings, diagrams, the latter implies understanding some

processes occurring in time and, thus, demands animated visualizations for effective

learning. The same is true for studying exploitation of devices, including vehicles,

where the object to learn is the procedure (sequence of actions) that must be per-

formed by operator (trainee) to get some result. Importantly, the role of such aids

strongly increases in conditions of distance learning when the access of trainees to

simulators may be limited. Therefore, the problem of improving their quality and

abundance deserved thorough attention.

To create a teaching video about vehicle one needs corresponding visual content,

which at present is obtained in either of two ways:

Manual drawing a flat or three-dimensional (3D) model of the vehicle from scratch

using specialized graphics or engineering software (3D Studio MAX, SolidWorks,

AutoCAD etc.), with further animating it.

Recording the video of performing operations on a real vehicle using camera.

Unfortunately, both ways are associated with significant difficulties, which limit their

application in practice:

 The first way is very labor-intensive. It requires having highly detailed information

about the vehicle (dimensions and appearance of its elements). Further, the model

Audiovisual

perception

Audiovisual

perception

Immersion

experience

Immersion

experience

Motor, tactile

experience

Motor, tactile

experience

Feedback

(limited)

Audiovisual

perception

1. Poster, video

2. Simulator

3. Real vehicle

Figure 1: Sensational features provided by the teaching aids in the accepted hierarchy. The complexity and cost of implemen-

tation increase top to bottom and left to right. The question mark represents the gap.

modern technology. Videos and animations are valu-

able means for teaching technical disciplines, which

study the structure and functioning of mechanisms

and systems. If the former is relatively easy to learn

by means of static posters, drawings, diagrams, the

latter implies understanding some processes occur-

ring in time and, thus, demands animated visualiza-

tions for effective learning. The same is true for

studying exploitation of devices, including vehicles,

where the object to learn is the procedure (sequence of

actions) that must be performed by operator (trainee)

to get some result. Importantly, the role of such aids

strongly increases in conditions of distance learning

when the access of trainees to simulators may be lim-

ited. Therefore, the problem of improving their qual-

ity and abundance deserved thorough attention.

To create a teaching video about vehicle one needs

corresponding visual content, which at present is ob-

tained in either of two ways:

1. Manual drawing a ﬂat or three-dimensional (3D)

model of the vehicle from scratch using special-

ized graphics or engineering software (3D Studio

MAX, SolidWorks, AutoCAD etc.), with further

animating it.

2. Recording the video of performing operations on

a real vehicle using camera.

Unfortunately, both ways are associated with sig-

niﬁcant difﬁculties, which limit their application in

practice:

• The ﬁrst way is very labor-intensive. It requires

having highly detailed information about the vehi-

cle (dimensions and appearance of its elements).

Further, the model prepared in such a way looks

noticeably idealized compared with the appear-

ance of the real vehicle. Also, rather high skills

are required for creation.

• For the second way, video recording in conﬁned

premises like vehicle cabins is difﬁcult and re-

quires appropriate skills and equipment. More-

over, the collected video material cannot be re-

used: each sequence of operations in vehicle

must be recorded individually. In total these fac-

tors make the process time-consuming and labor-

intensive, too.

Summarizing, the available range of teaching aids

is markedly incomplete, and their creation (posters,

videos) or use (simulators) is connected with severe

difﬁculties. This reduces the quality of teaching

vehicle-related subjects. The goal of our work is to

introduce new teaching aids in order to increase the

stated quality.

4 OVERVIEW OF USED

TECHNOLOGIES

We propose a collection of teaching aids built around

360

◦

photographic panoramas possibly connected

with H5P framework or virtual reality (VR) glasses.

In this chapter we shortly summarize these technolo-

gies.

360

◦

panorama (also called spherical or 3D

panorama) is an image that covers and contains the

full horizontal and vertical ﬁeld of view around a ﬁxed

point. It may be either artiﬁcial (i.e. drawn manually

or computer-generated) or photographic. The photo-

graphic ones are created by processing a number of

ordinary photographs (each having ﬁeld of view less

than 180

◦

) shot from the same position to all the di-

rections around (Wikipedia, 2020; Grinev, 2019). The

principle is shown in ﬁgure 2. Specialized software is

used for this sake, which generates smooth and con-

360 Photographic Panoramas as an Effective Multifunctional Aid for Teaching Technology Subjects

693

tinuous transitions between neighboring shots. Un-

like ordinary images, 360

◦

panoramas obviously can-

not be viewed as a whole without slicing, therefore,

when viewed on displays they are scrolled to the po-

sition of interest using computer mouse or other input

device. If the target comprises of several sections then

an individual panorama should be shot and fabricated

for each of them. The user has to switch between

them at viewing. At present, they are mostly used

for advertisement and entertaining purposes, however,

some educational use is also made (e.g., panoramas of

museum interiors) (Authentic Ukraine, 2020).

Figure 2: The principle of composing a 360

◦

photographic

panorama. The camera is located in the center of the ﬁeld

of view. A single photograph shot is shown explicitly, while

for the rest only borders are shown.

H5P (acronym of HTML5 Package) is a frame-

work allowing users create and demonstrate multifar-

ious interactive multimedia content, such as quizzes,

interactive videos and presentations etc. The frame-

work is designed such that its use is simple and does

not demand special training in information technolo-

gies. It is based on a set of Web-related technolo-

gies, hence, the content can be created and viewed

using any modern Web browser without the need of

additional software. Further, it is well-integrated with

learning management systems providing the opportu-

nity to enrich distance courses by incorporating inter-

active content (Scherbyna, 2016).

Virtual reality is a system designed to create the

effect of the person’s presence in some environment,

either having a real counterpart or missing it, by spe-

ciﬁc affecting their sensors (eyes, hears, skin etc.) via

special equipment and software. This effect is usually

called “immersion” in the literature. The central com-

ponent of the equipment is a head-mounted display

(called VR glasses), which form the person’s ﬁeld of

view by displaying the picture that is provided by VR

software. The most important feature of the glasses is

interactivity: the movements of the person’s head are

monitored, transferred to the software and processed

by it for the sake of updating the image in accordance

with the new direction of the head (Singh and Singh,

2017). This turns a passive spectator into an active

viewer who is able to look around.

In general, the possibilities of utilizing VR in

teaching various subjects have been actively dis-

cussed for several decades, and different software and

hardware solutions were proposed. Mostly, the ﬁelds

where practical study involves large hazard or ex-

penses were worked out, for example, medicine (Sa-

tava, 1993; Rowe and Cohen, 2002; Gallagher and

Cates, 2004), technology and ﬁre safety (Ren et al.,

2006; Ooi et al., 2019; Zhao and Lucas, 2015; Sam-

paio et al., 2010; Wang et al., 2018), driving (Bayarri

et al., 1996; Kang et al., 2004). At present, it became

an accepted aid, particularly, in medicine and mili-

tary training (Bhagat et al., 2016; Lele, 2013), while

in other ﬁelds its usefulness is still discussed. For de-

tailed reviews of the place of VR in education, the

reader is referred to subject papers (Wang et al., 2018;

Lavrentieva et al., 2020; He et al., 2017; Pantelidis,

2009; Osipova et al., 2019; Kramarenko et al., 2020).

5 PROPOSED SOLUTIONS

Basing on a above stated technologies we propose a

collection of teaching aids, which we think ﬁlls the

identiﬁed deﬁciencies well. It is represented in ﬁg-

ure 3. Now we will discuss each aid individually.

5.1 Teaching Video

Yet, ﬁrstly we describe the possible application

of 360

◦

panoramas for making traditional teaching

videos. We argue that they can readily serve as a

source of visual content for this sake, hence eliminat-

ing several related difﬁculties (see chapter 3):

• Fabricating a photographic panorama is much less

labor-intensive and time-consuming than drawing

a high-quality poster or 3D model because it does

not involve manual reconstruction of the vehicle

appearance from scratch. Instead, making a series

of photographs is a routine process, and compos-

ing the panorama is almost completely automated

by software.

• In contrast to recording videos, less skills and

simpler equipment are needed for photography.

Further, a panorama contains complete informa-

tion about appearance of the vehicle or its cabin.

Therefore, once manufactured, it can be repeat-

edly used to make a video on any procedure in-

volving this vehicle.

AET 2020 - Symposium on Advances in Educational Technology

694

Fig. 3. Sensational features provided by the collection of proposed teaching aids, which fill the

gap (represented as the question mark).

5.1 Teaching video

Yet, firstly we describe the possible application of 360° panoramas for making tradi-

tional teaching videos. We argue that they can readily serve as a source of visual con-

tent for this sake, hence eliminating several related difficulties (see chapter 3):

 Fabricating a photographic panorama is much less labor-intensive and time-

consuming than drawing a high-quality poster or 3D model because it does not in-

volve manual reconstruction of the vehicle appearance from scratch. Instead, mak-

ing a series of photographs is a routine process, and composing the panorama is

almost completely automated by software.

 In contrast to recording videos, less skills and simpler equipment are needed for

photography. Further, a panorama contains complete information about appearance

of the vehicle or its cabin. Therefore, once manufactured, it can be repeatedly used

to make a video on any procedure involving this vehicle.

We propose the following algorithm of making videos basing on 360° panoramas:

 Display the needed panorama on the computer screen.

 Start screen capture together with sound recording from microphone using the

corresponding software.

 Consider the first action of the chosen procedure.

 Scroll the panorama to show the view corresponding to the current action.

 Point, which control has to be activated or which indicator should be noticed, by

drawing remarks on the screen using corresponding software.

 Voice the performed action and accompanying commentaries.

 Repeat steps 4–6 for the next action until the procedure is finished.

 Stop screen capture.

1. Interactive

teaching video

Audiovisual

perception

Audiovisual

perception

Immersion

experience

Motor

experience

Interaction

with trainee

2. Virtual tour

4. Extended VR

simulator

Audiovisual

perception

Interaction

with trainee

Audiovisual

perception

Immersion

experience

Motor expe-

rience (limited)

3. Virtual reality

simulator

Figure 3: Sensational features provided by the collection of proposed teaching aids, which ﬁll the gap (represented as the

question mark).

We propose the following algorithm of making

videos basing on 360

◦

panoramas:

• Display the needed panorama on the computer

screen.

• Start screen capture together with sound recording

from microphone using the corresponding soft-

ware.

• Consider the ﬁrst action of the chosen procedure.

• Scroll the panorama to show the view correspond-

ing to the current action.

• Point, which control has to be activated or which

indicator should be noticed, by drawing remarks

on the screen using corresponding software.

• Voice the performed action and accompanying

commentaries.

• Repeat steps 4–6 for the next action until the pro-

cedure is ﬁnished.

• Stop screen capture.

Among others, we made a teaching video on pro-

cedure “Preparation to starting-up and starting-up of

the engine” using 360

◦

panorama of BTR-4E shot by

us previously. It describes the sequence of 19 actions

that trainee in the role of driver has to perform.

5.2 Interactive Video Lesson

As far as 360

◦

panoramas facilitate making traditional

teaching videos, they as well assist making the H5P

interactive video lessons. The feature of this kind

of multimedia content is that it allows incorporating

various interactive elements, such as appearing ex-

planations, images, tables, bookmarks, links. During

watching H5P interactive video lesson, the user visu-

ally and audibly receives information about the pro-

cedure they needs to perform. Various test tasks can

be added, as well. At a certain point, the video stops

playing, and Moodle expects the trainee to perform

some action within this video. The playback will con-

tinue once the trainee selects the correct action.

The main advantage of this interactive approach is

that the trainee takes an active part in the lesson in-

stead of being a passive spectator, which forces them

to be more attentive and focused during watching. Im-

portantly, fabrication of such lessons is rather non-

complicated and does not require additional software

(they are developed in a Web browser). Such videos

can be integrated easily into courses created within

Moodle learning management system (Polhun et al.,

2021).

Based on the previously mentioned panorama-

based teaching video (chapter 5.1) we have made an

interactive video lesson, which was then integrated

into the distance course “Structure and exploitation

basics of BTR-4E”. The course is published on the In-

stitute’s Moodle platform (dl.khpi.edu.ua, 2020). The

next interactivity was added: algorithm of actions in

text format, the values shown by indicators during

procedure, labels indicating the location of the main

devices and controls of the BTR, control questions the

trainee has to answer.

5.3 Virtual Tour

Apart from using just as a source of visual content

for making videos, the 360

◦

photographic panoramas

themselves are excellent teaching aids that surpass

other sorts of multimedia:

• The complete information of the vehicle appear-

ance is structured and collected in a single piece

360 Photographic Panoramas as an Effective Multifunctional Aid for Teaching Technology Subjects

695

of multimedia that can be explored in a natural

way.

• The exact correspondence between the panorama

and actual appearance of vehicle elements makes

them recognized easily when a trainee enters the

real cabin.

• Smooth and continuous exploration of a

panorama creates a solid and coherent im-

age of the vehicle interior or exterior in the

trainee’s mind.

A 360

◦

panorama that can be interactively viewed

(scrolled) is called “virtual tour”. Optionally virtual

tours include some additional content (texts, multi-

media) and links to related panoramas (e.g. those

of other rooms in an apartment or sections in a vehi-

cle). Displaying and hiding this content and transition

between panoramas is done by activating markers,

which are placed at chosen points of the panorama.

Demonstration of virtual tours does not require

special software and is done in a Web browser. This

appears an essential advantage because allows native

integration of the panoramas with Web-based learn-

ing management systems, such as Moodle. It is done

using H5P framework. For Moodle, teacher ﬁrstly

has to open Content bank and add a content item of

type “Virtual Tour (360)”, then upload the fabricated

panorama there. After this, the described markers for

trainee’s interaction with the panorama can be placed.

The starting position and direction of the view should

be set. We recommend ﬁrstly load all the panora-

mas of the vehicle (each its section, maybe its exte-

rior, as well) and then connect them with transitions

by means of placing such markers.

We turned the abovementioned 360

◦

panorama of

BTR-4E into a virtual tour and integrated it into the

distance course (dl.khpi.edu.ua, 2020). Numerous

teaching information was added by means of H5P:

• labels of cabin elements and controls to speed up

memorization of their location and main purpose;

• schemes to explain the internals of the cabin el-

ements and their connection and interaction with

each other;

• video clips to animate the panorama and show the

controls and indicators in action;

• control questions to allow teacher check the mem-

orization of material by trainees.

5.4 Virtual Reality Simulator

Despite the rich opportunities provided by software,

the full potential of 360

◦

panoramas may be utilized

only if they occupy the whole ﬁeld of view of a trainee

instead of being viewed on a distant display. Virtual

reality is the technology that makes this possible. It

allows imitate the presence in a real premise depicted

on the panorama and achieve the effect of “immer-

sion”. Hence, it becomes advantageous to employ VR

glasses for viewing 360

◦

panoramas. The next bene-

ﬁts can be reached:

• The image of the 360

◦

panorama appears com-

pletely surrounding the trainee, convincingly imi-

tating staying inside the premise.

• The panorama becomes interactive, i.e. respond-

ing to the look-up movements of the trainee’s

head.

Also, the trainee becomes able to perform move-

ments of arms and hands in order to imitate using con-

trols seen in the ﬁeld of view. Although the former

are not visible in VR glasses (without involving ad-

ditional elaborate VR equipment), this is still useful

and provides correct (through incomplete) motor ex-

perience because the location of controls in the ﬁeld

of view displayed by VR glasses is the same as in-

side the vehicle cabin. This feature even more dis-

tinguishes this teaching aid from common visual aids

(posters etc.), which are unable to provide reasonable

motor experience.

Summarizing, the combination of a 360

◦

panorama and VR glasses is able to provide visual

perception plus both immersion and limited motional

experiences. Hence, it is located in between posters

and simulators in the diagram (ﬁgure 1) ﬁlling the

gap described in chapter 2.

Yet, there are several difﬁculties arising during

practical use of VR at classes.

Firstly, while the teaching aids stated above do

not require special software for use, here, a dedicated

software must be developed that receives the data

about head rotations from VR glasses and performs

scrolling the panorama. We solved this problem by

means of a collaboration with IT company (chapter

5.6).

Secondly, the teacher is unable to monitor the ac-

tions of the trainee because they is unable to see the

image on the VR display. Therefore, in order to allow

controlling the trainee’s actions, a supplementary dis-

play is needed, whose purpose is to demonstrate the

image that is displayed by VR glasses at the moment.

The third difﬁculty is the very limited ability of

the trainee wearing VR glasses to answer teacher’s

questions about the image seen (e.g. “indicate the but-

ton named X on the control panel”). It is caused by

the fact that by default, the environment is interactive

just to some extent: it responds to the rotations of the

user’s head, but the user is unable to affect it in any

AET 2020 - Symposium on Advances in Educational Technology

696

other way. To solve this problem, the trainee has to be

provided with a separate device called “controller”.

Its purpose is to receive the user’s input and affect the

image seen in the VR glasses and, as a result, on the

teacher’s computer display. For example, the trainee

would draw lines and other marks on the displayed

panorama, in such a way answering teacher’s ques-

tions. A simplest controller is computer mouse.

Taking all the above into account, the proposed

VR simulator consists of four components, which are

depicted in ﬁgure 4. It may be implemented using

a range of hardware; the authors’ choice is stated in

chapter 5.6 (Barkatov et al., 2020).

5.5 Extended Virtual Reality Simulator

As it has been mentioned in the previous section,

the VR simulator in the present implementation lacks

some useful features. Firstly, the trainee does not see

their hands, therefore, the arms movements imitating

reaching vehicle controls has to be made blindly. Sec-

ondly, there is no tactile experience: the movements

of hands and ﬁngers imitating activating vehicle con-

trols are made in empty space that does not provide

tactile feedback (the sense of touching a solid object,

the resistance force appearing during its activation).

Introducing there features (at least, approximate) can

improve the immersion effect.

At present, there is a technical mean available that

can achieve this goal, namely, VR gloves. This de-

vice connects to the VR glasses and performs two-

side data exchange. Firstly, the data about current lo-

cation in space and pose of the hand is collected from

sensors and transmitted to the VR glasses. VR soft-

ware can process this data and draw a 3D model of

hand in the correct pose and location with respect to

the virtual environment. So that, VR gloves can serve

as a controller allowing user point some places of

the displayed panorama or perform hands and ﬁngers

movements to answer teacher’s questions. Secondly,

the VR software continuously checks the contacts be-

tween the 3D model of hand and the virtual environ-

ment; if any, it sends the information about the loca-

tion and character of the contact to VR gloves. The

gloves process this data and activate matching vibrat-

ing motors to make the user feeling virtual objects.

In total, these features help to correctly perform and

remember complex operations, even if they require

ﬁne motor skills. This is rather actual for vehicle-

related subjects because the panels in vehicle cabins

or in mounted equipment are full of diverse buttons,

toggles, levers, and other controls.

Yet, there are obstacles hindering wide applica-

tion of this aid. The functionality for proper display-

ing the 3D model of hands must be implemented in

the used VR software, which is a non-trivial problem.

Moreover, the cost of VR gloves is considerable at

present. Still, we expect that the advances in the ﬁeld

will facilitate solving these difﬁculties and make this

aid much more feasible in the near future.

5.6 Details of Implementation of VR

Simulator

For the implementation of the software part, a tight

collaboration between the Institute and IT company

“Innovative Distance Learning Systems Limited” was

established. The Company developed the viewing

software, and the experts of the Company performed

photographing the interiors and exteriors of vehicles,

fabrication of 360

◦

panoramas, and loading them to

the viewing software. The Institute took part in de-

veloping the content, carried out approbation, and de-

veloped methods for the most efﬁcient application of

the product in teaching.

The hardware part was chosen in accordance with

the following considerations.

The VR glasses are of two kinds. The ﬁrst kind

comprises a built-in display; such glasses must by

connected to a source of video signal, which is usu-

ally a computer running VR software. The second

kind of glasses is called “VR boxes”. There, the role

of display is played by a smartphone, which has to be

installed (reversibly) into the VR box. In this case, the

source of video signal is the same smartphone, which

runs VR software.

The teacher’s display must be connected either to

the computer that generates the image for VR glasses

of the ﬁrst kind, or to the supplementary computer

that receives the image from the smartphone installed

in VR glasses of the second kind.

The simplest kind of controller is computer

mouse: when connected, a cursor is shown on the im-

age, and the trainee is able to move it and set to the

needed position (e.g., to the position, at which some

control is seen at the moment).

Taking this into account we used the next hard-

ware in our implementation (ﬁgure 5):

• a smartphone running Android operating system

where VR software is installed and 360

◦

panora-

mas are uploaded;

• a VR box because it does not require a computer

to work, is much cheaper than VR glasses, and

still provides the ability to view 360

◦

panoramas;

• a wireless computer mouse as a controller because

it is a common and inexpensive device requiring

no adaptation for trainees;

360 Photographic Panoramas as an Effective Multifunctional Aid for Teaching Technology Subjects

697

it is located in between posters and simulators in the diagram (Fig. 1) filling the gap

described in chapter 2.

Yet, there are several difficulties arising during practical use of VR at classes.

Firstly, while the teaching aids stated above do not require special software for use,

here, a dedicated software must be developed that receives the data about head rota-

tions from VR glasses and performs scrolling the panorama. We solved this problem

by means of a collaboration with IT company (chapter 5.6).

Secondly, the teacher is unable to monitor the actions of the trainee because they is

unable to see the image on the VR display. Therefore, in order to allow controlling the

trainee’s actions, a supplementary display is needed, whose purpose is to demonstrate

the image that is displayed by VR glasses at the moment.

The third difficulty is the very limited ability of the trainee wearing VR glasses to

answer teacher’s questions about the image seen (e.g. "indicate the button named X

on the control panel"). It is caused by the fact that by default, the environment is in-

teractive just to some extent: it responds to the rotations of the user’s head, but the

user is unable to affect it in any other way. To solve this problem, the trainee has to be

provided with a separate device called "controller". Its purpose is to receive the user’s

input and affect the image seen in the VR glasses and, as a result, on the teacher’s

computer display. For example, the trainee would draw lines and other marks on the

displayed panorama, in such a way answering teacher's questions. A simplest control-

ler is computer mouse.

Taking all the above into account, the proposed VR simulator consists of four

components, which are depicted in Fig. 4. It may be implemented using a range of

hardware; the authors’ choice is stated in chapter 5.6. [5]

Fig. 4. Composition of the proposed virtual reality simulator.

5.5 Extended virtual reality simulator

As it has been mentioned in the previous section, the VR simulator in the present

implementation lacks some useful features. Firstly, the trainee does not see their

hands, therefore, the arms movements imitating reaching vehicle controls has to be

made blindly. Secondly, there is no tactile experience: the movements of hands and

fingers imitating activating vehicle controls are made in empty space that does not

provide tactile feedback (the sense of touching a solid object, the resistance force

appearing during its activation). Introducing there features (at least, approximate) can

improve the immersion effect.

(Extended) Virtual reality simulator

Software

showing 360°

panorama

VR glasses

Controller

Computer

Figure 4: Composition of the proposed virtual reality simulator.

• a laptop running Windows 10 operating system

because the needed conﬁguration is relatively

simple and the image from VR glasses may be re-

ceived wirelessly via Miracast technology.

 a smartphone running Android operating system where VR software is installed

and 360° panoramas are uploaded;

 a VR box because it does not require a computer to work, is much cheaper than VR

glasses, and still provides the ability to view 360° panoramas;

 a wireless computer mouse as a controller because it is a common and inexpensive

device requiring no adaptation for trainees;

 a laptop running Windows 10 operating system because the needed configuration

is relatively simple and the image from VR glasses may be received wirelessly via

Miracast technology.

Fig. 5. Authors' choice for the hardware implementation of proposed virtual reality simulator.

Considering the extended VR simulator that includes VR gloves as a controller in-

stead of computer mouse, we note that the gloves must meet several requirements to

provide the best immersion experience during training:

 complete finger tracking;

 usage of moisture-resistant material;

 tracking the shoulder joints and wrists;

 wireless implementation with low latency;

 vibrating motors for tactile feedback.

Finally, to maximize the immersion experience, the VR software should accompany

all trainee's actions on switching on/off the controls by accurate sounds.

5.7 Application in Teaching

The 360° photographic panoramas are well-suited for demonstrating confined premis-

es where all the points of interest (labels, controls, indicators etc.) are located at a

similar distance from the viewer. Actually, this is a case for most vehicle cabins and

portable equipment, transported by vehicles (e.g. portable chemical laboratories).

Therefore, the aid is most suitable for teaching subjects considering the above things.

VR box

Wireless

mouse

Windows

10 laptop

Android

smart-

phone

360°

panora-

Figure 5: Authors’ choice for the hardware implementation

of proposed virtual reality simulator.

Considering the extended VR simulator that in-

cludes VR gloves as a controller instead of computer

mouse, we note that the gloves must meet several re-

quirements to provide the best immersion experience

during training:

• complete ﬁnger tracking;

• usage of moisture-resistant material;

• tracking the shoulder joints and wrists;

• wireless implementation with low latency;

• vibrating motors for tactile feedback.

Finally, to maximize the immersion experience,

the VR software should accompany all trainee’s ac-

tions on switching on/off the controls by accurate

sounds.

5.7 Application in Teaching

The 360

◦

photographic panoramas are well-suited for

demonstrating conﬁned premises where all the points

of interest (labels, controls, indicators etc.) are lo-

cated at a similar distance from the viewer. Actually,

this is a case for most vehicle cabins and portable

equipment, transported by vehicles (e.g. portable

chemical laboratories). Therefore, the aid is most

suitable for teaching subjects considering the above

things.

Considering the ﬁnancial side, the total price of

the aid is very limited (hundred to thousand times less

than a price of a single simulator), which makes fea-

sible for educational establishments to obtain the aid.

Moreover, the price makes readily possible equip-

ping specialized classes for groups of 10–20 trainees.

This option proportionally increases the time a trainee

spends in the teaching aid and, thus, further improves

teaching quality.

Importantly, the ﬁrst three aids (360

◦

panorama-

based traditional and interactive videos, virtual tours)

are ready for use at distance learning, as well. More-

over, we expect that in perspective VR glasses will

become even more affordable, and it would be pos-

sible for trainees to equip themselves with proposed

VR simulators and attend the corresponding classes

distantly. This would improve their learning experi-

ence even more.

Summarizing, 360

◦

photographic panoramas ap-

pear as a multifunctional content that can be turned

into a range of teaching aids of different character.

5.8 Test of Effectiveness

In order to test the efﬁciency of the proposed aids, a

teaching experiment was carried out at the Institute.

The points #1 “Location and operation rules of con-

trols and indicators” and #2 “Preparation of the vehi-

cle to engine starting-up and movement” of the prac-

tice lesson “Training at simulators on preparatory ex-

ercise #1” belonging to the credit module “Driving

basics” of the subject “Basics of driving combat ve-

hicles” were chosen. The exercise #1 in this module

is “Preparation to starting-up and starting-up of the

engine” (further called “the exercise”), its procedure

contains 19 steps.

The experimental class was held at an experi-

mental multifunctional room. 360

◦

panorama-based

teaching videos, virtual tours, and VR simulators

AET 2020 - Symposium on Advances in Educational Technology

698

were employed. For comparison of efﬁciency, the ref-

erence class on the same lesson was held in a tradi-

tional way using common teaching aids (posters, con-

ventional simulators).

Both classes started with learning the general

structure of the BTR-4E transporter, its cabins, con-

trols, and exploitation basics. The trainees were pro-

vided with the general information about the purpose

of the control cabin and the driver operating proce-

dure by means of either posters and textbooks (ref-

erence group) or distance course “Structure and ex-

ploitation basics of BTR-4E” (experimental group)

containing 360

◦

panorama-based teaching videos and

virtual tours (dl.khpi.edu.ua, 2020).

Then, the training was continued with the help

of either a conventional simulator (in the reference

group) or the proposed teaching aid further called “a

VR simulator of the driver cabin” (in the experimen-

tal group). In the latter case, the procedure was as

follows. The teacher divided the group to pairs, and

in each pair trainees were assigned with #1 and #2.

Then, the following tasks were speciﬁed to #1 and #2:

Actions of #1: Help #2 to wear a VR simulator

and take a controller. Read the text of the exercise

procedure step by step making pauses after each step

to allow #2 ﬁnd the needed control. Check the cor-

rectness of #2’s actions by watching the laptop, do

corresponding notes and write down the results of

training into the control sheet.

Actions of #2: Wear a VR simulator and take a

controller, repeat the steps read by #1, ﬁnd the needed

control and point it with the controller. Then, pro-

nounce each step of the procedure by memory, ﬁnd

the needed control and point it with the controller.

When the actions were completed the trainees #1

and #2 exchanged their roles.

The success of teaching during the experimental

and reference classes was assessed by the results of

the next class, when both groups of trainees had to

execute the exercise at a conventional BTR-4E simu-

lator. This class has been carried out identically with

both groups. The main results are as follows.

The marks for completing the exercise are sum-

marized in table 2. Here, the mark is determined by

the consumed time: “excellent”, “good”, “satisfac-

tory” corresponds to no longer than 1 min 30 sec, 2

min, 2 min 30 sec, respectively. It is seen that both

groups have similar distribution of marks that indi-

cates they received equivalent training. This proves

that the collection of proposed teaching aids is able to

successfully augment the traditional teaching aids and

replace full-size simulators in the task of familiarizing

trainees with vehicle cabin.

Further, the occupation of the conventional sim-

Table 2: The success rates of the two groups of trainees.

Control

group

Experimental

group

Total trainees 25 25

”Excellent” marks 10 (40%) 11 (44%)

”Good” marks 8 (32%) 6 (24%)

”Satisfactory” marks 7 (28%) 8 (32%)

”Unsatisfactory” marks — —

ulator by the experimental group was 4 times lower

than by the reference one. Hence, application of the

proposed aid allows free substantial amount of the

simulator time, which then can be allocated to other

classes, which employ full range of its capabilities.

6 CONCLUSIONS

Deﬁciencies in the available range of teaching aids

for vehicle-related subjects, which decrease the teach-

ing quality of the subjects, were identiﬁed. Firstly, it

is desirable to rise the amount, quality, and technical

level of such basic teaching aids as teaching videos.

Secondly, there is a high demand for a teaching aid

that possesses such feature of the simulators as im-

mersion but is less expensive. Both these deﬁciencies

are proposed to ﬁll by means of an introduced collec-

tion of new teaching aids based on 360

◦

photographic

panoramas.

Considering the ﬁrst deﬁciency, the panoramas

can readily serve as a content for making teach-

ing videos, including the H5P interactive ones. Im-

portantly, the process of fabrication and the needed

equipment are much simpler than those for traditional

posters and videos. In addition, the panoramas them-

selves act as an interactive teaching aid with function-

ality inaccessible for traditional posters, drawings,

videos that can be naturally integrated into learning

management systems like Moodle.

The second deﬁciency is solved by a teaching aid

constituting the 360

◦

photographic panorama viewed

in virtual reality glasses. Its main feature is the pos-

sibility to provide visual sensation, immersion and

motor experiences, which are approaching to the pro-

vided by conventional simulators. At the same time,

the equipment is rather affordable to obtain by educa-

tional establishments, and its usage is straightforward

and does not demand special skills.

The efﬁciency of the aids was proved by a teach-

ing experiment that showed they can serve as an alter-

native to conventional posters and simulators on the

stage of trainee’s familiarizing with vehicle cabin ap-

pearance.

Summarizing, we think the proposed collection of

360 Photographic Panoramas as an Effective Multifunctional Aid for Teaching Technology Subjects

699

aids may signiﬁcantly improve the technology level

and cost-efﬁciency balance of teaching the subjects

where vehicles or mobile equipment are considered,

and may receive wide application in civil and mili-

tary education establishments, emergency and mili-

tary units, enterprises using special equipment. At

present, we have completed the 360

◦

photographic

panoramas of a series of armored (BM “Oplot”, BMP-

2, BTR-4E) and emergency (ﬁre engine) vehicles.

REFERENCES

Authentic Ukraine (2020). Ukrainian open-air museums.

https://museums.authenticukraine.com.ua/en/.

Barkatov, I. V., Farafonov, V. S., Tiurin, V. O., Honcharuk,

S. S., Barkatov, V. I., and Kravtsov, H. M. (2020).

New effective aid for teaching technology subjects: 3d

spherical panoramas joined with virtual reality. CEUR

Workshop Proceedings, 2731:163–175.

Bayarri, S., Fernandez, M., and Perez, M. (1996). Vir-

tual reality for driving simulation. Commun. ACM,

39(5):72–76.

Bhagat, K. K., Liou, W.-K., and Chang, C.-Y. (2016). A

cost-effective interactive 3d virtual reality system ap-

plied to military live ﬁring training. Virtual Reality,

20(2):127–140.

Course (2019). Course of driving combat vehicles of Ar-

mored forces of Ukraine. Varta, Kyiv.

dl.khpi.edu.ua (2020). Distance course “Struc-

ture and exploitation basics of BTR-4E”.

http://dl.khpi.edu.ua/course/view.php?id=576.

Gallagher, A. G. and Cates, G. U. (2004). Virtual reality

training for the operating room and cardiac catheteri-

sation laboratory. The Lancet, 364(9444):1538–1540.

Grinev, R. (2019). What is a 360 degree panorama.

https://truevirtualtours.com/ru/article/what-is-a-360-

degree-panorama.

He, J., Han, P., Liu, H., Men, S., Ju, L., Zhen, P., and

Wang, T. (2017). The research and application of

the augmented reality technology. In 2017 IEEE 2nd

Information Technology, Networking, Electronic and

Automation Control Conference (ITNEC), pages 496–

501.

Kang, H. S., Jalil, M. K. A., and Mailah, M. (2004). A PC-

based driving simulator using virtual reality technol-

ogy. In Proceedings of the 2004 ACM SIGGRAPH In-

ternational Conference on Virtual Reality Continuum

and Its Applications in Industry, VRCAI ’04, page

273–277, New York, NY, USA. Association for Com-

puting Machinery.

Kramarenko, T., Pylypenko, O., and Zaselskiy, V. (2020).

Prospects of using the augmented reality application

in STEM-based Mathematics teaching. CEUR Work-

shop Proceedings, 2547:130–144.

Lavrentieva, O., Arkhypov, I., Kuchma, O., and Uchitel,

A. (2020). Use of simulators together with virtual and

augmented reality in the system of welders’ vocational

training: Past, present, and future. CEUR Workshop

Proceedings, 2547:201–216.

Lele, A. (2013). Virtual reality and its military utility. Jour-

nal of Ambient Intelligence and Humanized Comput-

ing, 4(1):17–26.

Ooi, S., Tanimoto, T., and Sano, M. (2019). Virtual real-

ity ﬁre disaster training system for improving disas-

ter awareness. In Proceedings of the 2019 8th Inter-

national Conference on Educational and Information

Technology, ICEIT 2019, page 301–307, New York,

NY, USA. Association for Computing Machinery.

Osipova, N., Kravtsov, H., Hniedkova, O., Lishchuk, T., and

Davidenko, K. (2019). Technologies of virtual and

augmented reality for high education and secondary

school. CEUR Workshop Proceedings, 2393:121–131.

Pantelidis, V. S. (2009). Reasons to use virtual real-

ity in education and training courses and a model

to determine when to use virtual reality. Themes

in science and technology education, 2(1-2):59–70.

https://ﬁles.eric.ed.gov/fulltext/EJ1131313.pdf.

Polhun, K., Kramarenko, T., Maloivan, M., and Tomilina,

A. (2021). Shift from blended learning to distance one

during the lockdown period using Moodle: test con-

trol of students’ academic achievement and analysis

of its results. Journal of Physics: Conference Series,

1840(1):012053.

Prihodko, A. (2009). Ukrainian simulation industry: prob-

lems and solutions. Defence Express, (1–2):52–56.

Ren, A., Chen, C., Shi, J., and Zou, L. (2006). Application

of virtual reality technology to evacuation simulation

in ﬁre disaster. In Arabnia, H. R., editor, Proceed-

ings of the 2006 International Conference on Com-

puter Graphics & Virtual Reality, CGVR 2006, Las

Vegas, Nevada, USA, June 26-29, 2006, pages 15–21.

CSREA Press.

Rowe, R. and Cohen, R. A. (2002). An evaluation of a vir-

tual reality airway simulator. Anesthesia & Analgesia,

95(1):62–66.

Rudkovs’kyy, A. (2013). Integration of system of trainers

is in process of combat training of subdivisions of the

army. Military Technical Collection, (2):99–104.

Rusilo, P. A. (2010). Problem questions in relation to the

state and prospects of development of educational-

trainer facilities for mechanized and tank units. Sys-

tems of Arms and Military Equipment, (2):61–64.

Sampaio, A. Z., Ferreira, M. M., Ros

ario, D. P., and Mar-

tins, O. P. (2010). 3D and VR models in Civil En-

gineering education: Construction, rehabilitation and

maintenance. Automation in Construction, 19(7):819–

828.

Satava, R. M. (1993). Virtual reality surgical simulator. Sur-

gical Endoscopy, 7(3):203–205.

Scherbyna, A. A. (2016). H5P – a new tool for multime-

dia interactive learning content creation. Theoreti-

cal problems of culture, education, and upbringing.

Learning, 54:84–88.

Singh, N. and Singh, S. (2017). Virtual reality: A brief

survey. In 2017 International Conference on Infor-

mation Communication and Embedded Systems (ICI-

CES), pages 1–6.

AET 2020 - Symposium on Advances in Educational Technology

700

Vakaliuk, T., Kontsedailo, V., Antoniuk, D., Korotun, O.,

Mintii, I., and Pikilnyak, A. (2020). Using game sim-

ulator Software Inc in the Software Engineering edu-

cation. CEUR Workshop Proceedings, 2547:66–80.

Wang, P., Wu, P., Wang, J., Chi, H.-L., and Wang, X.

(2018). A critical review of the use of virtual real-

ity in construction engineering education and training.

International Journal of Environmental Research and

Public Health, 15(6). https://www.mdpi.com/1660-

4601/15/6/1204.

Wikipedia (2020). Panoramic photography. https://en.

wikipedia.org/wiki/Panoramic photography.

Zhao, D. and Lucas, J. (2015). Virtual reality simulation for

construction safety promotion. International Journal

of Injury Control and Safety Promotion, 22(1):57–67.

360 Photographic Panoramas as an Effective Multifunctional Aid for Teaching Technology Subjects

701