User Interface Analysis to Increase User Experience on Computer

based Test Application: Case Study Smk Negeri 4 Pandeglang

Aang Munawar and Wendi Usino

Universitas Budi Luhur

Keyword: Analysis, Usability, CBT, User Experience, Jacob.

Abstract: This study aims to analyze Usability to Computer Based Test applications through the User Experience

approach in SMK Negeri 4 Pandeglang using Jacob Nielsen method using five variables namely,

Learnability, Efficiency, Memorability, Error, and Satisfaction. This study starts from the low value of

semester exam results using CBT. The population in this study are students of class XI year lesson

2015/2016 SMK Negeri 4 Pandeglang. The sampling technique of this research is a saturated sample

consists of 158 people. The results showed the level of user experience of the Computer Based Test

application including the category strongly agree the results of the questionnaire assessment of respondents

with an average value of 91.4% consists of variables Learnability, Efficiency, Memorability, Error, and

Satisfaction, students feel comfortable when the exam using CBT and able to show the actual ability to do

the work.

1. INTRODUCTION

Every corporate agency, government, and education

today can not be separated from an information

system in order to run its work activities, so that

more regular and directed with more efficient time.

Information systems and computer technology are

growing very rapidly in line with the size of the need

for information. The development of information

technology has made it easier for the world of

education in implementing the teaching and learning

process one of them is the activity to evaluate the

results of learning that usually the exam is only done

using paper, but now the test can be done practically

using a computer.

Fhadilla (2016) states that the success of an

information system is not enough determined by the

technology used required the analysis of user

experience to determine the needs of users

dynamically. The low value of the students' semester

test results using CBT below the average score, with

the development of Computer Based Test

application through usability analysis with User

Experience approach is expected to provide comfort

during the test and show the actual student ability to

the test exams.

Research on the analysis of usability to an

application previously been done related to the

analysis of User Experience Web-Based Digital

Library (I Made Agus OW 2017), and also research

entitled Analysis of the benefits of implementation

of online exam information system (Erick Andika, et

al 2017) .

2. RELATED STUDIES

2.1 Usability

Usability comes from a usable word that generally

means it can be used properly. Something can be

said to be useful if the failure to use it can be

eliminated or minimized and provide benefits and

satisfaction to the user (Wimmi and Lussy, 2016).

Understanding usability according to Nielsen is a

user experience in interacting with the application or

website until the user can operate it easily and

quickly (Nielsen, 1994), while according to I Made

Agus Oka Wijaya usability is a term that shows the

ease of humans to use a particular tool or other man-

made objects to achieve a certain purpose (I Made

Agus Oka Wijaya, Dkk, 2017).

Munawar, A. and Usino, W.

User Interface Analysis to Increase User Experience on Computer based Test Application: Case Study Smk Negeri 4 Pandeglang.

DOI: 10.5220/0008929700830086

In Proceedings of the 1st International Conference on IT, Communication and Technology for Better Life (ICT4BL 2019), pages 83-86

ISBN: 978-989-758-429-9

According to the theory of Nielsen Model, there

are five criteria in analyzing Usability include:

1. Learnability

2. Efficiency

3. Memorability

4. Error

5. Satisfaction

2.2 Computer Based Test

Computer Based Test is a computerized evaluation

application that aims to assist teachers in conducting

an evaluation, whether scoring, test execution, and

effectiveness and efficiency of its implementation

(Novrianti, 2014).

2.3 User Experience (UX)

User experience has a wider domain of User

Interface because the realm of UX starts with user

research which is then implemented into an

interface. User Interface is part of UX where UI is

the end product of UX. User experience designer is

able to generate a user interface that is easy to use by

target users. Because the UX realm looks quite large

(www.interaction-design.org). According to David

Lawrence (2016), User Experience (UX) can be

defined as any experience experienced by users

while interacting with digital devices. Any

experience or experience, such as physical

interactions, sensors, emotions, and mental. In other

words, it is the level of user satisfaction gained when

they interact with products with technology-based

contexts. User experience (user experience) is a term

user experience in perceiving ease and efficiency in

the interaction human with a computer. Includes

one's perception of practical aspects such as

usability, ease of use, and efficiency of an existing

system (Lilis Dwi Fraida, 2016).

2.4 User Interface Design

According to Tintin Chandra (2013), the goal of

UID is to design an effective interface for software

systems. Effective means ready to use, and the

results according to need.

Figure 1: User Interface Principle

Principles In Designing User Interface include:

1. User familiarity

2. Consistency

3. Minimal Surprise

4. Recoverability

5. User Guidance

6. User Diversity

3. RESEARCH METHODOLOGY

3.1 Research Methodology

Research on Usability Analysis to know User

Experience in Computer Based Test application in

SMK Negeri 4 Pandeglang is using Jacob Nielsen

method which is based on five attributes, that is:

learnability, efficiency, memorability, errors, dan

satisfaction.

Methods in application development in this

research is more to the development of User

Interface through User Experience approach.

Image 2: User Interface Principle

Population in this research is all active class XII

students who have followed the semester test

activity by using Computer Based Test in SMK

Negeri 4 Pandeglang, which amounted to 158

students. Sampling in this study with the sampling

technique saturated with the intention that the

sample participants taken actually represent the

participating population. According to Sugiyono

(2008), saturated sampling is a technique of

ICT4BL 2019 - International Conference on IT, Communication and Technology for Better Life

determining the sample when all members of the

population used as a sample. So the sample of

participants in this study was 158 people.

Library study was conducted to find and obtain

data that is theoretical and related to the research

being done. Observation technique is done by

Participant Observation. In this observation, the

researcher is directly involved with the daily

activities of the person being observed or to be used

as the source of research data. To obtain the required

data, data collection is also done through a

questionnaire or questionnaire in a closed manner

where respondents determine the answers based on a

number of alternatives provided in the questionnaire

instrument.

3.2 Research Indicators

The indicator used in this study, the researchers used

the Nielsen Usability Criteria Model, which assesses

the level of ease of user interface to use.

Table 1. Usability Indicators Nielsen Model

No Factor Usability Description

1 Learnability - Easy to

understand

- Easy to look for

specific

information

- Easy to identify

navigational

mechanism

2 Efficiency - Easy to reach

quickly

- Easy to navigate

- Easy to remember

- Easy to

reestablish

3 Memorability - Easy to remember

- Easy to

reestablish

4 Error - Few numbers of

error detected

- Easy to fix

5 Satisfaction - System pleasant

- use

- Comfort to use

4. DISCUSSION AND RESULT

Testing stage Usability analysis results to determine

whether each criteria Usability is appropriate or not,

with this test using the method of User Acceptance

Test (UAT).

Based on the results of questionnaire data from

the five criteria obtained the following results:

a. Learnability has an average score of 95%, in

which case the student states strongly agree

that the developed CBT application is easy to

learn.

b. Efficiency has an average value of 90%, with

this factor the student states strongly agree

that the CBT application has been developed

quickly in the operation of its functions.

c. Memorability has an average rating of 93%,

with this factor the student states strongly

agree that the CBT application that has been

developed is easy to remember in a certain

period of time.

d. An error has an average value of 90%, with

this factor the student states strongly agree

that the application of CBT has been

developed the occurrence of errors

minimally.

e. Satisfaction has an average value of 89%, with

this factor the student stated strongly agree

that CBT application that has been developed

convenient when used.

Based on the results of application testing

through answering questions directly by 158

students on CBT applications that have been

developed results obtained as follows:

Image 3. Rank Mid Test CBT

Information :

CBT1 (Computer Based Semester 1)

PBT1 (Paper-Based Test 1)

CBT2 (Computer-Based Test Semester 2)

PBT2 (Paper-Based Semester 2)

CBTN (Computer Based New)

5. CONCLUSION

Based on the result of Usability analysis with User

Experience approach on Computer Based Test

User Interface Analysis to Increase User Experience on Computer based Test Application: Case Study Smk Negeri 4 Pandeglang

application using the User Acceptance Test method,

it can be concluded as follows.

1. Of the 158 students of class XII SMK Negeri

4 Pandeglang are actively sampled in the

study. In this study using five variables based

on the theory of methods taken from Jacob

Nielsen such as Learnability, Efficiency,

Memorability, Error, and Satisfaction.

2. From the average percentage of five variables

is then obtained the average usability rate of

91.4% which belongs to the category Strongly

Agree. From the results of application testing

through answering activities directly by 158

students on CBT applications that have been

developed obtained.

REFERENCES

Farida, L.D, 2016, Pengukuran User Experience dengan

pendekatan Usability, Jurnal Sistem Informasi, 2302-

3805

Kusuma, W.A., Noviasari, V. dan Marthasari, G.I. 2016,

Analisis Usability dalam User Experience pada sistem

KRS-Online UMM menggunakan USE Questionnaire,

Jurnal Teknologi Informatika, Vol. 5 No. 4 2301-4156

Muhammad, F., Nugroho, R.A. dan Turioanto, D. 2016,

Analisis User Experience Untuk Tingkat Keterpilihan

Smartphone Android, Jurnal Ilmu Komputer, Vol. 04

No. 01 2406-7857

Mustaqbal, M.S., Firdaus, F.R., Rahmadi, H. 2015,

Pengujian Aplikasi Menggunakan Black Box Testing

Boundary Value Analysis, Jurnal Teknik Informatika,

Vol 1 No. 3 2407-311

Nielsen. J, 1995, '10 Usability Heuristics for User

Interface Design,' diakses pada 10 September 2017,

<https://www.nngroup.com/articls/ten-

usabilityheuristics>

Wijaya, I.M.A.O., Arthana, I.K.R. dan Pradnyana, G.A.

2017, Analisis User Experience Web Based Digital

Library Jurusan Pendidikan Teknik Informatika

Universitas Pendidikan Ganesha, Jurnal Teknik

Informatika, Vol. 6 No. 2 2252-9063



ICT4BL 2019 - International Conference on IT, Communication and Technology for Better Life