USABILITY CHALLENGES IN EMERGING MARKETS

Maryam Aziz, Chris Riley and Graham Johnson

Advanced Technology and Research,Financial Solutions Group Ltd., NCR, U.K.

Keywords: Fingerprint technology, user perceptions, usability evaluation, ATM, self-service.

Abstract: Understanding emerging cultures and adapting new services accordingly is one of the biggest challenges

faced by modern businesses today. Several user-centred approaches are employed during the life-cycle of

service adoption. These approaches mainly involve design and evaluation of a service based on specific

requirements of the target market. This paper describes an on-going multi-method case study which

involves the evaluation of a self-service ATM system with a fingerprint sensor used for identification

purposes as based in the financial sector of Pakistan. The paper is positioned to assess the validity of

traditional usability evaluation methods in the context of emerging markets. These methods include one-on-

one observations, in-depth interviews and sensor performance data analysis. The methods ensure both

objective and subjective assessment of sensor use throughout. However, several difficulties faced with

sensor evaluation such as participant recruitment, lack of participants’ response and the impact on the local

culture’s user attitude towards sensor use are discussed. The paper also presents the preliminary findings

and draws implications for both researchers and practitioners based on our experience.

1 INTRODUCTION

Emerging markets present a wealth of investment

opportunities to global industries. Despite the

efforts, global industries face several challenges to

fully exploit these opportunities. For example, it is

quite challenging to understand how traditional

western business models and strategies can be

applied successfully in emerging markets such as

China, Pakistan and India. It becomes particularly

difficult when a business strategy involves

introduction of a new product or a service in the user

domain. The success of a product or a service is

generally determined in terms of the products

usability assessed by target users. A great deal of

research on various methods and approaches to

evaluate product usability in western countries has

been carried out. These methods and approaches

have largely been applied in emerging markets with

the mindset of obtaining useful insights into product

use. However, this research questions the

appropriateness and reliability of these methods

when desired outcomes are not achieved. For

instance, Geldof (2007) reports on methodological

challenges faced during a usability study of

Information and Communication Technologies

(ICTs) by low-literate youths in Ethiopia. Such

examples of usability studies emphasise the need of

– first, understanding the requirements of the context

in which the study is being conducted and second,

adapting the research methodologies to fulfill those

predetermined requirements.

This paper presents an experience-based in-

progress case study which highlights several

challenges with usability evaluation methods

employed. The study was conducted in collaboration

with BankIslami, Karachi in Pakistan which had

recently introduced fingerprint enabled automatic

teller machines (ATMs). The purpose of introducing

fingerprint technology was to enhance security

measures and promote self-service use in the

financial sector of Pakistan. The study was

conducted in the banks branches with real

customers. Here we focus on the first phase of the

in-progress study, which assesses user perceptions of

and practical issues with fingerprint sensor usage.

The paper is structured as follow. Section 2

provides a brief background of various usability

methods and their role in product evaluation in

emerging markets. Section 3 discusses the usability

methods employed in the context of fingerprint

sensor use in the financial sector of Pakistan. The

methodological challenges faced in the study are

also discussed in this section. A brief overview of

the preliminary findings along with insights into the

160

Aziz M., Riley C. and Johnson G. (2008).

USABILITY CHALLENGES IN EMERGING MARKETS.

In Proceedings of the Tenth International Conference on Enterprise Information Systems - HCI, pages 160-165

DOI: 10.5220/0001684601600165

 SciTePress

underlying cultural factors which impacted usability

methods is presented in section 4. The paper then

derives lessons learned and implications derived

from the study for both researchers and practitioners

alike in section 5.

2 USABILITY EVALUATION

The usability of a product is seen as a key factor in

determining the product adoption. Two main streams

of research can be observed that discuss concerns

regarding the usability of products. The first stream

argues that the objective usability of a product is as

significant as the subjective usability Monk et al.

(1993). The second streams emphasises the validity

and appropriateness of methodologies employed to

evaluate the usability of products in emerging

markets Thomas and Macredie (2002).

The first stream of research highlights the

importance of adopting an approach which

incorporates both objective and subjective aspects of

usability evaluation of a product. The majority of

research in Human Computer Interaction (HCI)

tends to opt for either a subjective or an objective

approach to evaluation Barkhuus and Rode (2007),

De-Angeli et al. (2006) for example. These

approaches are often seen as mutually exclusive

Monk et al. (1993) and there seems to be a lack of

research that combines the two Barkhuus and Rode

(2007). The ISO (ISO 9241-11:1998) guidance on

usability for instance, gives three quantifiable

measures of usability which are efficiency,

effectiveness and user satisfaction. These measures

focus on the technological reliability of a product. In

order to evaluate subjective usability of a product,

user perceptions and expectations are taken into

account. Several methods such as interviews,

surveys, “think aloud”, one-on-one observations and

focus groups are but a several examples of

commonly used methods designed to assist with the

understanding of user perceptions of product

usability.

Anderson (1994) argues that research methods

are inherently biased to cultures in which they

originate. Hence these methods have been proven to

be effective in western cultures. However, the

appropriateness of such methods and the validity of

data acquired through them needs to be questioned

when they are employed in unfamiliar contexts. For

example, the verbal protocol method has proven to

be effective in assessing the usability of products in

North America. But the validity of this approach

remains unclear in cultures such as India where

voicing a negative opinion may not be the norm

Chavan. (2005).

Another study reports on the usability evaluation

of a website in North America Evers (2002). Several

methods such as verbal protocol, questionnaires and

one-on-one observations were used. It was reported

that Japanese participants were particularly

uncomfortable with the one-on-one observation

method. Similarly, a study to introduce ICTs to

youths in Ethiopia was conducted by Geldof (2007).

Several methodological difficulties with the

recruitment of participants and the impact of the

local culture on the participants answering behaviour

emerged during the study. These studies therefore

highlight the critical importance of understanding

the requirements of specific context before

employing a research method. This issue becomes of

particular importance when the subjective

assessment of product usability is in question. It is

therefore argued that subjective usability measures

are not just validation tools but they also provide an

understanding of requirements of a specific context

and user appropriation for the product Barkhuus and

Rode (2007).

We report our experiences with the methodology

we adopted to evaluate the fingerprint sensor usage

in Pakistan. The strengths and weaknesses of the

methodology along with an understanding of the

Pakistani culture are briefly discussed.

3 RESEARCH METHODOLOGY

BankIslami is an early adopter of fingerprint

technology and Pakistan is one of the first countries

that have seen fingerprint technology be

implemented in the self-service environment.

Currently, fingerprint technology is being used in 16

branches and will be launched in another 11

branches within the year. The fingerprint enabled

ATMs are installed within lobbies adjacent to the

bank’s branches. An example of such an installation

can be seen in image 1 where the fingerprint sensor

mechanism is shown.

USABILITY CHALLENGES IN EMERGING MARKETS

161

Figure 1: fingerprint sensor.

It is argued that with the emergence of mass

consumer markets, laboratory-based usability

evaluation of services becomes meaningless

(Thomas and Macredie 2002). Therefore, the

investigation was carried out at three branches in

Karachi, including the Bank’s headquarters. A

multi-method approach was adopted to conduct the

usability evaluation of the fingerprint system both in

subjective and objective terms. These methods

included field observation, semi-structured in-depth

interviews and analysis of sensor performance data.

The interviews, 34 in total, were conducted in the

native language “Urdu”. Each interview lasted from

45 minutes to an hour. All interviewees (31 males

and 3 females) were regular users of fingerprint

sensor technology in BankIslami. Their age range

was between 18 and 54. Most of the people were

contacted by phone to come to the bank branch to

participate in the study. The reason for this

arrangement was that most of the individuals were

not actual bank customers. This issue is explained in

detail in section 3.1.

The main purpose of the first two methods (field

observation and interviews) was to acquire a

subjective understanding of fingerprint sensor

usability. The subjective usability was aimed to

provide an understanding of user perceptions of

usefulness, ease of use and concerns of and practical

issues with the sensor. The sensor performance data

acquired from the bank presented the logs of actual

usage of the sensor at 10 bank branches from across

the country. The data was analysed to assess the

objective usability of the sensor in terms of false

acceptance rate, false rejection rate and number of

user attempts to verify their identity at an ATMs.

Since the aim of the paper is to discuss the

methodological difficulties faced during the study,

the focus in this section will be kept on the

execution of the first two methods.

3.1 Participant Recruitment

At the time of conducting the study, fingerprint

technology had been in use for over 11 months in

BankIslami branches across the country. It was

therefore decided to carry out the usability

evaluation of fingerprint sensors in a real world

context. Three main branches of the bank within

Karachi (including corporate headquarters) were

chosen. With the collaboration of bank staff,

arrangements were made to recruit bank customers

as participants within these branches.

A real challenge of participant recruitment

occurred when individuals arriving at branches

turned out not to be customers of the bank in

question. It was discovered that in Pakistani culture,

day-to-day financial transactions tended to be

carried out by a customers’ subordinates or

employees. People performing financial transactions

on behalf of other employers were not the customers

of the bank. These people deposited cheques,

withdrew money and paid utility bills for which they

were provided with necessary personal financial

information.

During the entire duration of the study, only

male subordinates or employees had performed

transactions on behalf of their employers. It was

interesting to discover that a very low percentage of

women who were actual bank customers managed

their own financial accounts. Therefore, going to the

bank’s branches to perform financial transactions

was not seen to be common in the female

community. This placed a real challenge on

recruiting not only male but also female customers

to participate in the study. However, upon requests

to participate in the study, customers made time and

arrived in branches on time.

3.2 User Responses

A semi-structured interview with a focus on

assessing user perceptions of fingerprint sensor ease

of use, efficiency, security and usefulness was

designed. The main purpose of conducting the

interviews in “Urdu” was to help participants

express their perceptions and concerns of fingerprint

technology freely. Despite these efforts, participants

did not seem to fully express their perceptions and

concerns regarding technology use. It is argued that

in certain cultures such as India, individuals do not

feel comfortable with voicing negative opinions

ICEIS 2008 - International Conference on Enterprise Information Systems

162

about technology use Chavan (2005). However, in

our study, users did not state their opinions positive

or negative fully. The technology was generally

perceived to be easy to use and effective.

Surprisingly, no participants were concerned about

using such technology in the sensitive financial self-

service environment. Overall, participants showed a

positive attitude towards the technology.

However, participants’ lack of responses had a

significant impact on obtaining a clearer

understanding of sensor subjective usability in user

opinions.

3.3 One-on-One Observation

In order to investigate the practical difficulties with

fingerprint sensor use, participants were observed

using the sensor during the enrolment and

verification processes. The design of this phase of

the study was based on experience from a similar

study involving the use of biometric sensors in the

UK in a controlled lab environment. Participants in

the UK not only commented on the problems with

the use of sensors, but also verbalised their

expectations about the technology design. However,

in the Pakistan study, a lack of user responses

limited the effectiveness of assessing the actual use

of sensors.

For example, during the registration phase,

participants were prompted to indicate any

difficulties with fingerprint sensor use. Several

design aspects of the sensor such as, housing, sensor

contact area, pressure, correct finger placement,

hand position and angle were assessed in this phase.

Participants were observed to face several

difficulties with sensor use. However, upon

inquiring, participants did not report any problems.

When participants failed to use the sensor correctly

they took full responsibility. Also, during the

verification process, similar user behaviour patterns

were observed. While facing difficulties with the

sensor, they believed they were not using it correctly

and tried to work around it.

3.4 Cultural Setting

During the study, it was discovered that unlike the

general trend of registering the index finger in the

west, bank customers had enrolled their thumbs. The

bank was promoting the use of biometrix technology

for registering thumb impressions. Due to the

different angle position of thumb and index fingers,

several difficulties were faced with the sensor use.

For example, users could not place the thumb

properly on the sensor. They were seen to hold the

sensor in the hand in order to place the thumb on the

sensor. They were also found to have problems with

understanding how much pressure was required for

their thumb impression to be read by the sensor.

It was discovered that in Pakistan, fingerprints in

the form of thumb impressions for identity

verification had been in use for decades. It was

reported by bank customers that due to poor literacy

rates in rural regions, thumb impressions had been

used to sign legal documents, perform financial

transactions and in acquiring national identification

documentation. Also, with the modernisation of

technology, many Pakistanis’ had used various

electronic fingerprint sensors during the course of

these transactions. For example, at the age of 18,

every person in Pakistan applied for the national

identity card, for which a thumb impression needed

to be provided. The national identity card was

mandatory when opening an account with any bank..

We believe that the strong positive attitude toward

the use of fingerprint technology found in our study

was an artifact of the extensive use of thumb

impressions in Pakistani society.

4 PRELIMINARY FINDINGS

Despite all the practical difficulties with the

fingerprint sensor, participants of the study

perceived it to be easy to use. They also expressed a

preference for fingerprint usage over personal

identification numbers (PINs). The main reason

given for this preference being that they had

difficulties remembering all of their PINs. Users

reported that they were encouraged to keep unique

PINs for all their bank accounts and most of them

had more than two bank accounts with different

banks. A few users reported that they also had kept

records of their PINs and carried them along while

going to an ATM. Users reported that on a number

of occasions, they had forgotten their PINs at an

ATM which had caused them inconvenience.

Although participants were aware of the

potential for fraud with the fingerprint system, no

particular concern regarding the use of fingerprint

enabled ATMs was reported. In addition, a demand

for the widespread introduction of the technology in

the financial sector was expressed by users. Users

also showed an interest in the use of fingerprint

technology in other technological devices such as

laptops, mobile phones, define (PDAs), lockers,

credit card payments and for over-the-counter

services.

USABILITY CHALLENGES IN EMERGING MARKETS

163

Also, due to a high level of familiarity with the

technology, individuals had become complacent

with usage. Participants tended to reference their

own behaviour when describing problems with the

fingerprint sensor. Also, despite the practical

difficulties with the sensor, users could not suggest

how the devices could be enhanced.

It was interesting to see that individuals’ shared

their accounts with family members and friends.

Employees were also trusted with confidential

information in order to perform financial

transactions. In both cases, registering a fingerprint

would restrict the sharing of identity details with

trusted family, friends and employees who provided

assistance with performing financial transactions.

Mixed responses emerged from the elderly users

toward using the fingerprint sensor at ATMs. These

responses could be divided into two groups based on

their preferences of banking services. The first group

expressed a strong desire to use fingerprint

technology as they struggled to remember all their

PINs. However, several elderly users could not

impressions. Their fingertip skin was wrinkled and

the sensor could not take a clear impression of the

fingertips. In the second group, elderly users showed

reluctance toward using the technology because of

their familiarity with using the traditional methods.

They also believed that the use of a fingerprint

would restrict their children in managing their

accounts.

5 CONCLUSIONS

Fingerprint technology was launched at BankIslami

in May 2006. The study was carried out 11 months

later to understand user acceptance of the technology

in order to further predict technology adoption. The

findings help us obtain an insight into the

perceptions of real customers and observe the actual

use of the sensor. Despite several methodological

difficulties with data collection, a positive user

attitude toward the use of fingerprint technology

emerged. The study however did highlight the need

of assessing the strength of the approach employed.

The results gained form the interviews gave rich

contextual information but were not sufficient to

identify some of the real usability problems that

were present. Interviews revealed that Pakistani

society is a trusting society where individuals share

their financial information with family, friends and

employees. This finding highlights the need of

improving the facilitation of services in the financial

sector. In this regards, the specific requirements of

individuals who open bank accounts need to be

analysed together with the requirements of

individuals who operate them and/or provide

partial/full assistance with account management.

The observation data helped the researcher’s

understand issues with fingerprint sensors generally.

However, study participants did not verbalise any

issues they had with the sensor usage. We argue that

the limited user response regarding the use of the

sensor was maybe due to their lack of reference to

compare the efficiency and effectiveness of the

sensor. Also, a number of assumptions about a

user’s familiarity and experience with fingerprint

technology were made. We also argue that by

providing users more information about the

workings of fingerprint technology and experience

with various fingerprint sensors will help them think

and verbalise their opinions. Based on these

findings, it is therefore important to investigate the

understanding of fingerprint technology for identity

verification in detail in a culture where the concept

has been ingrained for decades.

The main challenge is to then investigate how

existing methods can be shaped and new methods

introduced to evaluate usability of products in

emerging markets. For example, Chavan (2005)

suggests a usability approach ‘use the collective’ for

the Indian market, which is argued to be a

collectivist society Hofstede (2001). By using this

method, the usability of a product is evaluated in the

midst of its potential users. The effectiveness of this

method can be seen in acquiring individual opinions

of users regarding a product use in a group. The

appropriateness of this method in sensitive or private

contexts such as the financial sector still needs to be

determined. We propose a method similar to ‘use the

collective’ to gauge user opinions of a product use.

The method should involve a semi-structured

discussion session with participants who are trusted

friends and/or family members. The method is

different from the traditional focus group session in

two ways. Firstly, the session will be semi-structured

to ensure all aspects of a product evaluation are

covered. Secondly, the participants of the session

will be selected form a group on the basis of trust or

inter-relations. The group will also represent the

current or potential users of the product.

On the basis of our findings, we conclude with

the opinion that usability evaluation in (HCI)

research as a whole would benefit from practitioners

adopting diverse methods of evaluation and

embracing both sides of the subjective verses

objective debate.

ICEIS 2008 - International Conference on Enterprise Information Systems

164

Usability evaluation is an on-going process

which needs to be carried out as user perceptions

and experience evolve with a product. In future

research, we will employ a multitude of methods to

overcome the methodological difficulties, which we

faced in this study.

ACKNOWLEDGEMENTS

We thank our collaborator NCR, Karachi, Pakistan

for their support with the initiation of the study. We

are also grateful to DTI (Department of Trade and

Industry, UK) for their support of this research work

through Napier University. A special thank you goes to

BankIslami staff members for their help with data

collection which ensured the successful completion

of the study.

REFERENCES

Anderson, R.J. (1994): Representations and requirements:

The value of ethnography in system design. Human-

Computer Interaction 9 (2):151-182.

Barkhuus, l. and Rode, J. A. (2007): From Mice to Men –

24 years of Evaluation in CHI. In Proceedings of CHI

2007, San Jose, USA.

Chavan, A. L. (2005): Another Culture, another Method.

In Proceedings of the 11

International Conference on

Human-Computer Interaction, Las Vegas, Nevada,

USA.

De-Angeli, A., Sutcliffe, A. and Hartmann, J. (2006):

Interaction, usability and aesthetics: what influences

users' preferences. In Proceedings of the 6th ACM

conference on Designing Interactive systems: 271 –

280.

Geldof, M. (2007): ICT for low-literate youth in Ethiopia:

the usability challenge. In Proceedings of HCI

International, Beijing, China: 67-76.

Hofstede, G. (2001). Culture’s consequences: Comparing

values, behaviours institutions and organisations

across nations, 2

edition, Sage publications.

Monk, A., Nardi, B., Gilbert, N., Marylin, M. and

McCarthy, J. (1993): Mixing oil and water?:

Ethnography versus experimental psychology in the

study of computer-mediated communication. In

Proceedings of the INTERACT '93 and CHI '93

conference on Human factors in computing systems:

3-6.

Thomas, P. and Macredie, R. (2002): Introduction to the

new usability. ACM transactions on human computer

interaction, Vol. 9, No. 2, Pp: 69-73.

USABILITY CHALLENGES IN EMERGING MARKETS

165