Blended Learning Quality Measurement System using Fuzzy Analytic

Hierarchy Process Method

Yeni Kustiyahningsih, Eka Mala Sari and Dwi Laras Asih

Departement of Informatics Engineering, University of Trunojoyo Madura

Keywords: Blended Learning, Synchronous, Asynchronous, Fuzzy Analytic Hierarchy Process, recommendations,

COVID-19.

Abstract: Blended learning is appropriate learning in the era of the COVID-19 pandemic. Blended learning is a learning

process that integration face-to-face and online learning. The characteristics of blended learning are learning

that combines synchronous and asynchronous learning settings correctly to achieve learning objectives. The

indicators in this study are live synchronous, virtual synchronous, independent synchronous and collaborative

asynchronous. There are many indicators in the measurement of blended learning, so a method is needed to

determine recommendations for improvement in the implementation of blended learning. The method used in

this research is Fuzzy AHP. Fuzzy method can handle data that contains uncertainty and inaccuracy. The

Fuzzy Analytic Hierarchy Process (AHP) method is used to determine the weighting of each blended learning

indicator. The purpose of this research is to build a decision support system software to determine

recommendations in implementing blended learning. Based on the research, that the indicators that most

influence the quality of blended learning are problem based learning, task collaboration and independent tasks.

The test results showed that the highest accuracy was obtained from a consistency ratio of 0.03627 with an

accuracy of 98%.

1 INTRODUCTION

Coronavirus Disease 2019 or COVID-19 has infected

millions of people worldwide and caused death. The

Coronavirus outbreak has been declared a pandemic

global by the World Health Organization (WHO).

The COVID-19 pandemic affects almost all aspects

of life, including education. In a pandemic like this,

the role and position of the educational aspect is very

crucial. Blended Learning is a learning strategy that

aims to achieve learning by combining face-to-face

learning and information technology-based learning

conducted online (Eniyati et al., 2010). Blended

learning means a integration of face to face with e-

learning that can be used by anyone (everyone),

anywhere, anytime (anytime) (Bruggeman et al.,

2019). The term blended learning means a

harmonious and ideal combination of learning or an

integraion of face-to-face and online learning

elements (Bruggeman et al., 2019).

Electronic Learning or E-Learning is an

independent learning process by utilizing information

and communication technology (ICT), or the internet

as a medium for knowledge transfer (Jeffrey et al.,

2014). The application of web-based learning (e-

learning) is one of the supports in supporting

conventional learning systems, because students and

educators do not have to meet face to face. The world

community has used e-learning a lot. the use of e-

learning in schools, training, universities and

industries, namely Cisco Systems, IBM, HP, Oracle,

and others (Kustiyahningsih et al., 2018). Previous

research e-Learning Quality Measurement based on

ISO 19796-1 with Fuzzy Analytic Network Process

Method (Kustiyahningsih et al., 2018)

The results of this study are recommendations for

e-learning improvements based on the smallest

weighting of the e-learning indicator value (Cahyani

et al., 2015). In this study, blended learning indicators

consist of Live Synchronous (Face-to-face, Problem

Based, learning and learning methods), Virtual

synchronous (online), Asynchronous standalone

(Independent Tasks) Asynchronous Collaborative

(Task Collaboration and Task Evaluation). Multi

criteria decision making technique is useful for

finding the best option from several alternatives. AHP

technique is a multi-criteria decision making

technique based on expert knowledge (Kaxancoglu et

200

Kustiyahningsih, Y., Sari, E. and Asih, D.

Blended Learning Quality Measurement System using Fuzzy Analytic Hierarchy Process Method.

DOI: 10.5220/0010306100003051

In Proceedings of the International Conference on Culture Heritage, Education, Sustainable Tourism, and Innovation Technologies (CESIT 2020), pages 200-206

ISBN: 978-989-758-501-2

al., 2018; Kustiyahningsih et al., 2017). AHP cannot

describe human thinking, so the Fuzzy AHP

technique is developed (Sevit et al., 2017; Claudia et

al., 2020). The problem of this research is the number

of indicators in the measurement of blended learning,

so a Decision Support System (DSS) is needed. The

purpose of this research is to build software DSS to

determine recommendations in implementing

blended learning. Therefore in this study using the

FAHP method to determine the indicators that most

influence the implementation of blende learning.

2 RESEARCH METHODS

2.1 Blended Learning

The blended learning method, it combines face-to-

face learning, appropriate setting and asynchronous

learning. A learning experience that is more flexible,

interactive, efficient, accessible, and varied for

teachers and students is a blended learning concept.

The adaptation of learning using technology and

traditional is also a blended learning strategy.

Appraisal evaluation is very important in determining

the success rate of blended learning. Student learning

using creative and innovative methods can provide

innovative solutions in determining learning

techniques (Jeffrey et al., 2014).

Blended learning supports more flexible,

interactive, efficient, accessible, and varied learning

for teachers and students. Blended learning approach

lies in the adaptation of existing technology-assisted

learning methods and traditional-based learning.

Assessment is a very important tool for determining

student knowledge for subjects at every level of

education. Blended learning techniques provide

instruction to deliver lectures and assess student

learning using creative and innovative methods. In

this journal studying the blended learning process, the

advantages of using blended learning techniques in

the education system. This journal also discusses

assessment methods to consider in this learning

technique. The blended learning method is very

interesting because it is more complete online and

face-to-face (Asif et al., 2012).

2.2 Characteristics of Blended

Learning

The characteristics of blended learning using a

constructive approach have two learning settings,

namely synchronous and asynchronous learning, the

follow is a picture of characteristics of blended

learning [11].

Figure 1: Characteristics and setting of blended learning

with a constructive approach.

2.3 Fuzzy Analitycal Hierarchy Process

(FAHP)

Fuzzy AHP is an analytical method developed from

traditional AHP. Although AHP is commonly used in

dealing with qualitative and quantitative criteria in

MCDM, fuzzy AHP is considered better at describing

vague decisions than traditional AHP (Tukan et al.,

2020; Ozcalici et al., 2019). Conversion from AHP to

Fuzzy AHP using Triangular Fuzzy Number (TFN)

and converting into three real numbers, namely low,

middle and upper. Fuzzy values provide strength to

factors or indicators that contain data or values that

are unclear or inaccurate (Abramovici et al., 2011;

Tsyganok et al., 2016).

Table 1: Triangular Fuzzy Number (TFN) scale.

AHP's

Intensity

Interes

Linguistic

Societies

(TFN)

Recipro

cal

Compariso

n of the

same

elements

(1, 1, 1) (1, 1, 1)

One

element is

quite

important

over the

othe

(1,3/2, 2)

(1/2,

2/3, 1)

One

element is

(2,5/2, 3)

(1/3,2/5,

1/2)

Blended Learning Quality Measurement System using Fuzzy Analytic Hierarchy Process Method

201

important

than the

othe

One

element is

important

than the

othe

(3,7/2, 4)

(1/4,2/7,

1/3)

One

element is

absolutely

important

than the

othe

(4, 9/2,

9/2)

(2/9,

2/9, 1/4)

The steps of the FAHP are as follows (Citrawati

et al., 2020; Saaty 2001):

1. Determine the pairwise comparison matrix

between each criterion can be defined as

follows:

12 13 1

21 23 2

31 32 3

123

nn n

xxx













 



With

-1

111

(x , x , x ), ( , , )

ij ij ij

lmu

ij ij

uml

ij ij ij



i,j = 1,2,...,n.

2. The geometric mean is used to determine the

weighted value of the indicator based on the

group of ratings. The calculation of the S

matrix as a geometric mean is as follows

11 12 13 1

21 22 23 2

31 32 33 3

12 3

nn n nn

ss s













 



with i,j = 1,2,...,n.

3. Calculate matrix S for criterion weight. U is the

results of the matrix criterion weight S.





































With i,j =

1,2,...,n.

4. Calculate BNP

Best Nonfuzzy Performance (BNP) method is

used to convert the fuzzy output into crisp

values. BNP can be stated as follows:

With i = 1,2,...,n

3 RESULT AND DISCUSSIONS

3.1 Blended Learning Indicator

The indicators of the quality of blended learning

examined here are live synchronous, virtual

synchronous, independent asynchronous and

collaborative asynchronous, where each indicator

consists of several processes. Live synchronous: face-

to-face, problem-based learning strategies (problem

based learning), Lectures, practices, discussions,

presentations, demonstrations, and others. Virtual

synchronous: Learning is carried out at the same time

but in the same / different room dimensions,

including: video conference, audio conference, chat.

Virtual synchronous is an extension of live

synchronous by utilizing technology to take a role in

online learning. Independent asynchronous: Learning

is carried out in different dimensions of space and

time (anytime and anywhere) through learning media

that allows students to learn independently through

printed media in the form of books, magazines,

modules. Asynchronous collaborative: Includes:

project work, mailing lists, discussion forums;

Provide opportunities for students and teachers to

discuss, observe, investigate, and analyze problems

related to material in online learning.

Table 2 presents data regarding the list of

processes for each indicator used as follows:

1/ 1/ 1/

111

x,x,x ,

nnn

lmu

ij ijk ijk ijk

kkk





















1/ 1/ 1/

11 1

() ( ) ()

,, ,

nn n

ln mn un

ij ij ij

jj j

nn n

uml

ij ij ij

ii i

ss s

 















()(u)

ul ml

ii ii

uu u

NP u





CESIT 2020 - International Conference on Culture Heritage, Education, Sustainable Tourism, and Innovation Technologies

202

Table 2: Blended learning indicator.

No Criteria Sub

Criteria

Information

1 Live

Synchro

nous

Face to

Face

(C1)

 14 Times

 10 Times

 8 Times

 5-7 Times

 0-4 Times

Problem

Based

Learnin

g (C2)

 4 Case

Studies / exercises

 3 Case

Studies / exercises

 2 Case

Studies / exercises

 1 Case

Studies / exercises

 No Case

Studies

Learnin

Methods

(C3)

 Lectures,

Practices, Discussions

and Presentations

 Lectures,

Practices, and

Discussions

 Lectures and

Practices

 Lecture

 No Learning

Methods

2 Virtual

Synchro

nous

Online

(C4)

 Video

conference, Audio

conference, Chatting

 Video

conference, Chatting

 Audio

conference, Audio

conference

 Chatting

 Nothin

3 Asynchr

onous

Standalo

Indepen

dent

Task

(C5)

 Doc, ppt, pdf,

books, modules,

journals

 Doc, ppt, pdf,

module, journal

 Doc, ppt,

journal, book

 Doc, module

 Nothing

4 Asynchr

onous

Collabor

ation

Task

Collabor

ation

(C6)

 Discussion

forums, mailing lists,

project work

 Discussion

forums, project work

 Mailing list,

project work

 Project work

 Nothin

Task

Evaluati

on (C7)

 UAS, UTS,

Post test Pretest

 UAS, UTS,

Post test

 UAS, UTS

 UAS

 Nothing

3.2 System Design

The system design is made to build a blended learning

quality measurement system including flowcharts

and use cases to be implemented. The purpose of

making this system design is so that the system will

be more focused and have a reference so that it will

be easier when implementing the programming

language.

FAHP Flowchart, The following is a flowchart of

the AHP Fuzzy Method can be seen at Figure 2.

Figure 2: Flowchart FAHP.

Based on Figure 2. The first step is to determine

the criteria and sub-criteria, then the linguistic scale

of the criteria and enter the level of importance

obtained from the expert assessors. Based on the

pairwise comparison matrix looking for

normalization, eigen value, index consistency and

consistency ratio. If consistency ratio of 10% then to

the next step and if the consistency ratio is more than

Blended Learning Quality Measurement System using Fuzzy Analytic Hierarchy Process Method

203

10% then a re-evaluation of each criterion is carried

out. The next step after CR is less than 10%,

determine is low, middle and upper fuzzy scales.

Convert to Triangular Fuzzy Number (TFN) to form

Low, Middle, Upper (L M U). Defuzzyfication

process is with Best Non Fuzzy performance (BNFP).

Enter the weight of blended learning. The priority of

the criteria matrix is multiplied by the priority of the

blended learning and the last alternative preference

results.

Use Case Diagram is an explanation of the

functionality of a system or class and how the system

interacts with the outside world. Use case diagrams

for a blended learning quality measurement system

can be seen at Figure 3.

Figure 3: Use case diagram.

Based on Figure 3, users or actors who must log

in before accessing all pages. The actor is Admin,

where the admin is a system user who has full access

rights. Admin can perform data management criteria

and blended learning which includes adding,

changing and deleting the data and being able to view

calculation results or alternative preferences.

Implementation blended learning of indicators into

FANP method. The pairwise comparison matrix can

be seen in Table 3 and Conversion of AHP to TFN

Weight can be seen in Table 4. The weighting results

of all blended e-learning indicators can be seen in

Table 5. System analysis is carried out so that the

decision support system ranking blended learning in

terms of quality can match the real situation. The

system designed in this study is a decision support

system for ranking blended learning. The initial

process is carried out in the system, namely, taking

the criteria and sub-criteria, and determining the

weight of each criterion by looking for the factors that

affect these criteria for blended learning.

The FAHP calculation process is carried out by

calculating the criteria first, where the admin will be

faced with a form to give weight in a pairwise

comparison then the system processes the input data

from the admin so that it produces the weight values

for each criterion, after getting the criteria weights,

then calculating the global weight of each criteria.

The final process is the output of the blended learning

value generated from the existing criteria, then

multiplied by the criteria weight, the result is a

ranking of blended learning which is sorted in

descending order.

Table 3: The pairwise comparison matrix.

Cri C1 C2 C3 C4 C5 C6 C7

C1 1.00 0.20 0.33 1.00

0.1

1 0.33

C2 5.00 1.00 3.00 5.00

0.3

0.2

0 3.00

C3 3.00 0.33 1.00 3.00

0.2

0.1

4 1.00

C4 1.00 0.20 0.33 1.00

0.1

1 0.33

C5 7.00 3.00 5.00 7.00

1.0

0.3

3 5.00

C6 9.00 5.00 7.00 9.00

3.0

1.0

0 7.00

C7 3.00 0.33 1.00 3.00

0.2

0.1

4 1.00

Tot

29.0

10.0

17.6

29.0

5.0

2.0

17.6

Based on Table 3. The pairwise comparison

matrix, that the comparison matrix for the same

criterion is 1, if the comparison of different criteria

will be worth according to the level of importance of

the assessor or expert in his field, for the assessment

in accordance with Table 1.

Table 4: Conversion of AHP to TFN Weight.

C1 C2 C3

L M U L M U L M U

1 1 1 0,3 0,4 0,5 0,5 0,7 1

2 2,5 3 1 1 1 1 1,5 2

1 1,5 2 0,5 0,7 1 1 1 1

1 1 1 0,3 0,4 0,5 0,5 0,7 1

3 3,5 4 1 1,5 2 2 2,5 3

4 4,5 4,5 2 2,5 3 3 3,5 4

1 1,5 2 0,5 0,67 1 1 1 1

Table 4 is a conversion table from AHP to

triangular fuzzy number (TFN). The results of this

conversion are derived from the pairwise matrix

comparison table which has a ratio consistency value

uc Use Case Admin

Admin

Masukkan Kriteria,

Blended Learning

Masukkan Bobot

Kriteria, Bobot

Blended Learning

Liha t Da ta Kriteria,

Hasil Konv ersi TFN,

LMU

Lihat Preferensi

Alternatif

«include»

CESIT 2020 - International Conference on Culture Heritage, Education, Sustainable Tourism, and Innovation Technologies

204

less than 0.1. If it meets the requirements, the matrix

will be converted into fuzzy.

Table 5: The weighting results.

Criteria Weight

C1 0.065

C2 0.149

C3 0.094

C4 0.065

C5 0.225

C6 0.307

C7 0.094

Based on Table 5, that the indicators that most

influence the quality of blended learning are problem

based learning, task collaboration and independent

tasks, because it has a higher value, namely 0.307,

0.225, 0.149.

3.3 Testing

Trial method aims to determine the alternative

preferences produced by the Fuzzy AHP method with

different consistency ratios. The trial was carried out

at three universities, namely Madura University,

Madura Islamic University and UT UPJJ

Ronggosukowati Pamekasan. The first experiment

with a consistency ratio of 0.03627, second

experiment was 1.06771, third experiment with a

consistency ratio of 0.41312 and fourth experiment

with a consistency ratio of 0.44979. This trial can be

seen in Table 5. The level of accuracy of the FAHP is

based on the Consistency Ratio (CR) value.

Table 5: Result of Accuration.

Tes

CR Value Accuration

1 0.03627 98%

2 1.06771 84%

3 0.41312 95%

4 0.44979 94%

Based on Table 3.The analysis of the results of the

above trials is that the higher the CR (Consistency

Ratio) value, the more inaccurate the results will be,

the lower the CR value, the smaller the probability of

error occur and the higher the level of accuracy.

4 CONCLUSIONS

Based on the analysis that has been carried out on the

measurement indicators of blended learning, the

indicators that most influence the quality of blended

learning are problem based learning, task

collaboration and task independence. The test results

based on the smallest CR produce the highest

accuracy, namely CR = 0.03627 with an accuracy of

98%. Further research can be developed with adjusted

indicators and fuzzy interval method.

REFERENCES

Abramovici M. Aand Jin F.A., 2011. New Method For

Monitoring Industrial Product-Service Systems Based

on BSC and AHP. In Proceedings Of The 13th

International Conference On Enterprise Information

Systems (ICEIS-2011), Pages 190-196 Scitepress DOI:

10.5220/0003483001900196 ISBN: 978-989-8425-53-

Asif K H. Noor Q, Mahaboob S S, Maresh A Abdullah, and

Vijaya Ch., 2012. “Study of Blended Learning Process

in Education Context”, I.J. Modern Education and

Computer Science, 23-29.

Bruggeman, J. Tondeur, K. Struyven, et al., Experts

speaking: Crucial teacher attributes for implementing

blended learning in higher education, The Internet and

Higher Education (2019),

https://doi.org/10.1016/j.iheduc.2020.100772.

Cahyani, A. D, Malasari, E. Kustiyahningsih., 2015. Design

an Adaptive Elearning Application Architecture based

on IEEE LTSA reference model.

Citrawati D H D, Widyawati M N and Suryono S., 2020.

The role of the analytic hierarchy process (AHP)

algorithm in health care services Journal of Physics:

Conference Series 1524 doi:10.1088/1742-

6596/1524/1/012116.

Claudia Y L, Eckert M and Earl C., 2020. A review of

fuzzy AHP methods for decision-making with

subjective judgements, Expert Systems with

Applications vol 161.

Eniyati S, and Santi R C N., 2010. Design of Decision

Support System for Lecturer Achievement Assessment

Based on Research and Community Service Jurnal

Teknologi Informasi DINAMIK vol 15 p 13.

Jeffrey, L. M., Milne, J., Suddaby. G., & Higgins, A., 2014.

Blended learning: How teachers balance the blend of

online and classroom components. Journal of

Information Technology Education: Research, 13, 121-

140

Kazancoglu Y and Ozturkoglu Y., 2018. Integrated

framework of disassembly line balancing with Green

and business objectives using a mixed MCDM Journal

of Cleaner Production 191 179-191

doi:10.1016/j.jclepro.2018.04.189.

Blended Learning Quality Measurement System using Fuzzy Analytic Hierarchy Process Method

205

Kustiyahningsih Y, Fatmawati and Suprajitno H., 2017.

Development of an interval value fuzzy number based

on MCGDM model by hybrid AHP and TOPSIS

methods, International conference on Sustainable

Information Engineering and Technology (SIET), IEEE

publisher, ISBN: 978-1-5386-2182-0 Doi

: 10.1109/SIET.2017.8304110.

Kustiyahningsih Y, Fatmawati and Suprajitno H., 2018.

MCGDM with AHP based on Adaptive interval Value

Fuzzy Telecommunication, Computing, Electronics

and Control, TELKOMNIKA 16(1) pp 314-322.

Ozcalici M Bumin M., 2019. An integrated multi-criteria

decision making model with Self-Organizing Maps for

the assessment of the performance of publicly traded

banks in Borsa Istanbul Applied Soft Computing

Journal 90.

Seyit A E, Jonas S and Zenonas T., 2017. Decision Making

in Construction Management: AHP and Expert Choice

Approach Procedia Engineering 172 pp 270 – 276.

Tsyganok V V, Kadenko S V and Andriichuk O V., 2016.

Usage Of Scales With Different Number Of Grades For

Pair Comparisoons In Decisions Support Systems

International Journal of the Analytic Hierarchy

Process vol 8 no 1 p 113.

Tukan M, Camerling B J, Afifudin M T and Hozairi., 2020.

Decision Support System for Determining Floating

Dock Locations in Maluku Islands Using AHP-TOPSIS

Journal of Physics: Conference Series 1577

doi:10.1088/1742-6596/1577/1/012001.

Saaty T L., 2001. Models, Methods, Concepts &

Applications of the Analytic Hierarchy Process, New

York: Springer Science+Business Media.

Tsyganok V V, Kadenko S V and Andriichuk O V., 2016.

Usage Of Scales With Different Number Of Grades For

Pair Comparisoons In Decisions Support Systems

International Journal of the Analytic Hierarchy Process

vol 8 no 1 p 113.

CESIT 2020 - International Conference on Culture Heritage, Education, Sustainable Tourism, and Innovation Technologies

206