John Garofalakis
, Antonia Stefani
, Vasilios Stefanis
and Michalis Xenos
Research Academic Computer Technology Institute, 26500, Patras, Greece
School of Sciences & Technology, Software Quality Research Group, Hellenic Open University
23 Sachtouri Str, Patras, Greece
University of Patras, Computer Engineering & Informatics Department, 26500 Patras, Greece
Keywords: m-commerce Systems, ISO9126, Quality attributes.
Abstract: In m-commerce applications, the design process still lacks a systematic quality control procedure. These
systems are data-intensive, user-driven, and have increasing needs for accessibility, efficiency, adaptivity,
portability and competitiveness. In this paper we study the quality of the software on which retail shopping
through the mobile web is based, i.e. the software part of B2C (business-to-customer) m-Commerce system.
A consumer’s perception quality evaluation was conducted to reveal the quality attributes that offer insights
into m-commerce systems based on the external quality characteristics of ISO9126 quality standard.
Mobile-commerce (m-commerce) systems have
been developed at a significant rate in recent years.
In particular, they offer functions and services in
order to fulfil the end-users requirements and to
provide services of high quality through a more
usable and user friendly environment.
M-commerce can be viewed as a subset of e-
commerce (Coursaris, 2002; Kwon, 2004) and refers
to transaction with monetary value that is conducted
via a mobile network (Clarke, 2001). When users
conduct e-commerce such as e-banking or purchase
products, they do not need to use a personal
computer system. Indeed, they can simply use some
mobile handheld devices such as Personal Digital
Assistants (PDA) and mobile phones. In the past,
these mobile devices or their enabling technologies
were regarded as a kind of luxury for individuals.
However, this situation has changed. The market for
mobile technologies has seen significant growth in
the past few years (Ngai, 2007). This is creating a
new opportunity for the growth of m-commerce.
Location-based services enable m-commerce to
be further leveraged so that the mobility aspect is
exploited of rather than repackaging old systems in a
new format. M-commerce systems have two major
characteristics: mobility and broad reach. Mobility
implies portability, e.g., users can conduct business
in real time via mobile devices. With m-commerce,
people can be reached at any time via a mobile
device. In this paper, m-commerce refers to
conducting commerce via wireless devices and
especially via mobile phones and examines end user
quality perspective.
The quality of m-commerce systems is an
extremely new and challenging task; especially the
quality of mobile commerce systems as it is
perceived by the end-user is an open research issue
for the software quality community. The end user
interacts with mobile system using software through
mobile devices and expects a well designed
environment where he/she can proceed a
commercial transaction.
It is with the above observation that this paper
examines the quality attributes of m-commerce
systems adopting the ISO9126 software quality
standard (ISO, 2001). Adopting and adapting ISO
9126 for specific domains is not new to the standard
itself (Losavio, 2004; Cote, 2005). The usual
approach is to enhance the hierarchical and (by
design) open scheme to include more attributes
suitable for the new domain. However, enhancing an
ISO standard is far from being an easy process.
Garofalakis J., Stefani A., Stefanis V. and Xenos M. (2007).
In Proceedings of the Second International Conference on e-Business, pages 130-136
DOI: 10.5220/0002113101300136
The quality of software concerns end-users and
developers as well. The main question posed is
“Which are the attributes that an m-commerce
system should have and how these attributes
formulate end user’s perspective about m-commerce
systems’ quality?” Another research question that
arises is that “An m-commerce system can be well
designed but it can also be characterized as a system
with high quality?” In this quality evaluation
approach the paper emphasizes at end-user
perceptive using the external quality characteristics
of ISO9126 namely Functionality, Usability,
Efficiency and Reliability.
The paper is structured as follows; in section 2
the theoretical foundations are presented, while in
section 3 m-commerce attributes are discussed. In
section 4 the evaluation process is presented, while
in section 5 the evaluation results and their quality
aspect are further discussed. Finally, in section 6
conclusions and future work are presented.
The method presented in this paper uses the external
quality characteristics of the ISO9126 quality
standard in order to model the quality attributes of
m-commerce systems. We strongly encourage
authors to use this document for the preparation of
the camera-ready. Please follow the instructions
closely in order to make the volume look as uniform
as possible. (Moore and Lopes, 1999).
2.1 The ISO9126 Quality Framework
ISO9126 is a quality standard for software product
evaluation and provides quality characteristics and
guidelines for their use (ISO, 2001). This standard,
composed of several parts, aims at defining a quality
model for software and a set of guidelines for
measuring the characteristics associated with it. This
work is based on the ISO9126 quality standard and
specifically it relies on the external quality
characteristics which are directly related to quality
as perceived by the end-users.
ISO9126 may be used as basis for m-commerce
quality evaluation but further analysis and mapping
of its characteristics is required. The main issue is
how m-commerce system’s quality can be analyzed
using this standard. In this work, we use the
following external quality characteristics of ISO
9126: Functionality, Usability, Efficiency and
Reliability. Each of the above mentioned
characteristics provide the quality framework
(actually the baseline) on which an m-commerce
system may be built, taking into account the
satisfaction of end-users requirements. The external
quality characteristics of ISO9126 are defined as
2.1.1 Functionality
Functionality refers to a set of functions and
specified properties that satisfy stated or implied
needs (ISO, 2001). The meaning of Functionality is
to provide integrative and interactive functions in
order to ensure end-user convenience. Especially for
m-commerce systems Functionality refers to the
existence of these functions and services that support
end user’s interaction via the mobile system.
2.1.2 Usability
Usability is defined as a set of attributes that bear on
the effort needed for the use of a product or service,
based on the individual assessment of such use by a
stated or implied set of users (ISO, 2001). Usability
is an important quality characteristic as all functions
of an m-commerce system are usually developed in
a way that seeks to facilitate the end-user by
simplifying end-user’s actions; this fact can however
affect negatively the system in certain cases.
2.1.3 Efficiency
Efficiency is a complex concept that entails both
conceptual challenges as well as implementation
difficulties. Efficiency is defined as the capability of
the system to provide appropriate performance,
relative to the amount of resources used, under
stated conditions (ISO, 2001). It refers to a state
where system functions are both usable and
successful, i.e. they achieve their aim, the reason for
their existence. One of the main criteria of efficiency
of an m-commerce system is the quality relating to
time and resource behaviour.
2.1.4 Reliability
Reliability is the quality characteristic that refers to a
set of attributes that bear on the capability of
software to maintain its performance level under
stated conditions for a stated period of time (ISO,
2001). Especially for m-commerce systems
reliability refers to systems tolerance on end users
Currently, m-commerce is in its early stages of
development and little is known about the factors
that influence quality’s attitudes and their value from
the end-users’ perceptions. Mobile commerce
systems include mobile financial systems, mobile
advertising, mobile inventory management, product
locating and shopping, wireless reengineering,
mobile auction, and wireless data center (Malloy,
2002). These systems aim to provide the appropriate
attributes/functions for the end user in order to
complete a purchase or to get information using
his/her mobile device. The basic advantage of m-
commerce systems is that location and time do not
constrain people from completing their transactions.
In order to model the interactions among the end
user and the m-commerce system we consider three
different steps: Presentation, Navigation and
Purchasing. Presentation describes how a product or
service is presented to the end-user. Navigation
describes the various mechanisms provided to the
end user for accessing information and services of
the m-commerce system. Purchasing refers to the
facilities provided for the commercial transaction
per se. Applying the above steps to m-commerce
requires an adjustment to the attributes that the
system presents because of its wireless
communication character. In the following
paragraphs we present the attributes of mobile
systems that constitute the end-user purchasing
process. The definition of these attributes is
basically based on the literature for mobile web
practices and especially on Mobile Web Best
Practices 1.0, W3C Proposed Recommendation
(W3C, 2006), e-commerce and m-commerce
previous experience (Stefani, 2004; Stefani, 2005;
Blekas, 2006). In this work m-commerce attributes
are categorized in the above categories providing an
overview of m-commerce systems’ quality from end
users’ point of view.
The aim of this paper is not to describe all
existing m-commerce attributes or fully present their
use but rather to offer a quality evaluation of these
attributes and to present a quality framework for m-
commerce systems.
3.1 Presentation
Presentation is supported basically by text and
images because mobile devices have limitations
such as screen size and resolution, number of
supported colors, computation power, memory size,
rate of data transfer and energy required for proper
functionality. Color usage is also important. Using
colors obviously gives a pleasant and friendly
interface, but a too colored screen confuses. All the
pages of the m-commerce system must have the
same colors so the user can feel that he/she is
navigating in the same environment. By removing
background images, background colors and text
colors the readability of the content is increased. The
use of images in Internet applications is common.
Nevertheless, using images in mobile web
applications significantly increases download and
response time and thus, usage cost.
Presentation issues are also related to issues such
as thematic consistency and the default delivery
context which intends to provide an acceptable
mobile environment for any end user from different
mobile devices. The clarity of the text presented
with meaningful, short and simple words and the
presentation of the central meaning at the first page
of each mobile device contributes attributes that a
m-commerce system should provide to the end user
for an accessible mobile environment. Additionally,
providing a descriptive title for the page allows easy
identification of the content; by keeping the title
short reduces page weight.
3.2 Navigation
The navigability of an m-commerce system is a
critical factor for its success. Navigation is an
important design element, allowing users to acquire
the information they are seeking and making that
information easier to find.
Navigation issues support m-commerce systems
quality by taking into account the quality of
components such as indexes, navigation bars, site
maps and quick links. The availability of these
components facilitates access to information and
services and enables users to locate efficiently the
information they need, while avoiding usability
bottlenecks. Additionally, navigation concerns the
facilities for accessing information and the
connectivity of the above systems.
Navigation refers at attributes that support the
navigability of m-commerce systems. These refer to
navigation bars, which according to W3C Mobile
Web Best Practices 1.0, should be placed on the top
of the page. Any other secondary navigational
element may be placed at the bottom of the page if
really needed. It is important that users should be
able to see page content once the page has been
loaded with the minimum possible scrolling. M-
commerce systems, as e-commerce systems provide
ICE-B 2007 - International Conference on e-Business
simple metaphors such as shopping cart where the
end-user may add the products that he/she intends to
buy. Mobile devices present limitations on text
inputting so an m-commerce system should be
enabled by attributes such as access keys (keyboard
short cuts), by providing defaults at any function that
the user should select an action and also by avoiding
free text with minimum text inputting. The
navigability of the mobile system is also supported
from search services which are related with device
capabilities and context presentation as well. Search
with simple text inputting in an AND/OR operator
format enables the user to find the information
needed without navigating to several mobile pages.
Search attributes can reduce the cost of mobile
browsing and prevent navigability difficulties.
Additionally, because of the limitations in
display and input mechanisms, the possible absence
of a pointing device and other constraints of mobile
devices, care should be exercised in defining the
structure and the navigation model of a Web site.
Especially the use of links should be limited aiming
at providing a balance between having a large
number of navigation links on a page and the need to
navigate multiple links to reach content.
3.3 Purchasing
Purchasing refers to all m-commerce systems
attributes that strongly support their commercial
character. In particular, it refers to attributes that
support the interaction with the m-commerce
system. These attributes are also related to the
navigability of the system but they are categorized
differently because of their significant contribution
to the purchasing process.
Localization services may enable the
presentation of the products and service because the
m-commerce system can recommend the best
selection based on end user’s positioning.
Authentication and personalization attributes support
an m-commerce system where the end user can
provide private information (ie. Credit Card
Number). Additionally notification services provide
a great advantage to m-commerce systems because
they can also be combined with localized
information. Alternative payment methods support
either a complete transaction via the m-commerce
system or otherwise combined with localized
information allow mobile users to conclude
transactions efficiently (e.g. receive a product at a
selling point closest to the user’s current location)
(Samaras, 2002).
Purchasing process success is also related to the
stability of the process via the m-commerce system
and issues like error tolerance and error recovery are
Quality evaluation of mobile applications, as long as
the end user’s requirements are concerned, gives
emphasis on the systems’ quality and especially on
ISO9126 external quality characteristics:
Functionality (F), Usability (U), Efficiency (E) and
Reliability (R). It is within this framework that the
end-users, who are also the customers of a mobile
network, will specify the relative significance for
each function that the mobile system provides to the
End-user is the best judge (when properly
questioned) of the degree to which their needs are
satisfied, and thus of the quality of the system
viewed as a service. For the presented study, three
expert quality evaluators were selected in a heuristic
evaluation method (Nielsen, 1990). Heuristic
evaluation is performed by looking at an interface
and trying to come up with an opinion about what is
good and bad about the interface. Ideally people
would conduct such evaluations according to certain
rules, such as those listed in typical guidelines
documents. The evaluators for this method are IT
experts with experience in quality evaluation and
mobile systems as well. The nature of the presented
evaluation method demands the use of expert
evaluators because of its technical character.
Figure 1: Amazon's m-commerce system from Nokia N70.
Firstly, the evaluators were asked to a complete
purchase using a mobile phone and an emulator
from their PC. For the evaluation process we have
used the Nokia N 70 mobile phone. The N70 has a
screen with resolution 176 x 208 pixels and supports
262.144 colors. The phone can also connect to 3G
networks for high rate data transfers using the Opera
Mobile 8.51 browser. In order to avoid operability
issues for the Nokia N70, help about the
functionalities of the device was provided during the
evaluation process. Additionally the emulator that
has been used was the Openwave Simulator (version
Figure 2: E-bay's m-commerce system from Nokia N70.
The three evaluators used three popular m-
commerce systems according to the Mobile Web
Google Search (www.google.com/xhtml) in order to
have the best m-commerce experience possible. The
m-commerce systems that have been selected were
Amazon.com, Ebay.com and Movietickets.com.
Each evaluator was asked to asses specific m-
commerce attributes and to evaluate each one by
assigning a value of relevance (rij). Relevance
defines the correlation among the m-commerce
systems attribute i (presented in table 1) and
software quality characteristic j ordered as they
presented in this work (i.e. j=1 for Functionality, j=2
for Usability, j=3 for Efficiency and j=4 for
Reliability) using a five-grade Liker-type scale (1).
Each evaluator selected a different value for each
quality characteristic from the Liker-type scale
assigning scale.
This provides a qualitative representation of m-
commerce systems quality and especially gives
emphasis on external quality characteristics.
Figure 3: Movietickets’ m-commerce system from Nokia
Figure 4: Payment method at movietichets.
Quality evaluation of m-commerce systems
attributes provides a quantitative representation of e-
commerce systems’ quality. Table 1 provides the
evaluation results for the m-commerce attributes
presented in section 2. It presents the values of
function relevance (r) for each attribute. These
values are the average values of all evaluators
approximated in monad.
Based on the evaluation results, the quality of m-
commerce systems can be modelled in external
quality characteristics and attributes. Providing a
value for each attribute, an ordered list for each
external quality characteristic is provided.
These values provide a first impression of end
users preferences and perquisites about m-commerce
systems’ attributes.
ICE-B 2007 - International Conference on e-Business
Table 1: Evaluation results for m-commerce attributes.
M-commerce attributes
Product’s description
3 5 3 3
Still images
3 5 3 1
4 5 3 2
3 5 4 2
4 5 3 2
3 5 4 2
Central meaning of
3 5 4 1
Text inputting
4 5 4 1
Thematic consistency
2 5 4 2
Provide defaults
3 5 4 2
Navigation mechanism 4 4 4 3
Uploading Time 3 3 5 4
Access keys 4 5 4 2
Use of Links 4 4 3 3
Help 5 5 3 3
Feedback 3 5 4 3
Undo functions 5 3 3 5
User oriented hierarchy 2 5 4 3
Redirection 5 3 4 3
navigation bar (minimum) 5 5 3 1
Scrolling 3 5 4 2
Search response time 2 4 5 4
Search results processing 3 4 5 3
Shopping cart –Metaphor 4 5 4 2
Security mechanism 3 2 4 5
Pricing Mechanism 3 4 3 3
Alternative payment
4 4 3 4
Authentication 5 2 3 5
Personalization 4 5 4 2
Localization 4 5 3 1
Transaction recourses
behavior 3 3 5 4
Notification services 3 5 3 4
Error recovery 3 3 3 5
Errors tolerance 4 3 4 4
Stability 4 3 3 5
The categorization of these attributes provides
important feedback for m-commerce systems’
assessment which is in an initial stage. By evaluating
the attributes that an m-commerce system provides
to the end user we also offer an end user perception
of quality. End user’s experience is a critical
determinate of success in mobile web applications.
If end users, who are also the customers, cannot find
what they are searching for, they will not buy it; a
site that buries key information impairs business
decision making. Poorly designed interfaces increase
user errors, which can be costly. A user-centered
evaluation approach supports all the tasks users need
to accomplish using different m-commerce systems’
attributes. The above evaluation process provides
measurement results which can be also be defined as
metrics for a quantitative representation of m-
commerce systems’ quality.
In order to evaluate m-commerce systems
features a new metric that summarizes the relevance
of each attribute is introduced. The metric Mobile
Attributes Weight (MAW) provides an evaluation
weight with respect to the four quality characteristics
and is calculated by formula 2,
where r
is the relevance for every listed m-
commerce system attribute. The value for MAW
provides a numerical value for every m-commerce
system attribute and an ordered list about end user
preference based on external quality characteristics.
The values for MAW need to be further specified,
probably with experience testing in future work and
the use of different end users’ groups.
The metric MAW actually represents attributes
importance for the end-user and as such it can be
used at the development phase in order to define end
user preferences.
The evaluation process also provided interesting
results about the quality characteristics. In an up and
down processing of the values for rij, the WF=0,24,
WU=0,30, WE=0,26, WR=0,20 values have been
defined as the normalized average values for each
quality characteristic. From these values we
conclude that m-commerce end users give great
emphasis to Usability and Efficiency issues and
somewhat lesser on Functionality and Reliability.
These values differ from e-commerce systems where
Usability and Functionality have equally great
importance (Stefani, 2004). In e-commerce systems
the end users expects different and usable
functions/services, but in m-commerce systems the
end user desires the basic functions with great
efficiency when it comes to time and resource
In this paper, we presented a quality evaluation for
selected attributes of m-commerce systems and
particularly B2C m-commerce systems. This
evaluation provides an extendable framework useful
for mobile system developers. We believe that this is
a step towards more effective measurement of m-
commerce systems’ quality. We acknowledge that
our attributes do not constitute a complete set and
may not cover every aspect of m-commerce systems.
The above evaluation results provide an initial
research for m-commerce systems’ quality.
A metric MAW has also been introduced which
measures the value of relevance for each m-
commerce system attribute. The theoretical
framework for this metric is also presented. The
validity of the presented measures should further
examine with different user groups in alternative
evaluation cases and it is included in future work. It
should be mentioned that the values presented
are not strictly defined as numerical results but
present the correlation among m-commerce systems
attributes and external quality characteristics.
Future work includes the development of
methods enabling the selection of the most
appropriate set of metrics for each evaluation
process. We also suggest the creation of an open
knowledge base of web-based m-commerce
systems’ quality factors, criteria, and metrics
according to end users’ needs.
Blekas, A., Garofalakis, J., and Stefanis, V., Use of RSS
feeds for Content Adaptation in Mobile Web
Browsing, International Cross-Disciplinary Workshop
on Web Accessibility, WWW2006, pp. 79 – 85.
Clarke, I., Emerging value propositions for m-commerce,
Journal of Business Strategies 18 (2) (2001) 133– 148.
Cote, M., Suryn, W., Laporte, C., & Martin, R. (2005) The
Evolution Path for Industrial Software Quality
Evaluation Methods Applying ISO/IEC 9126:2001
Quality Model: Example of MITRE's SQAE Method.
Software Quality Journal, 13(1), 17-30.
Coursaris, C., Hassanein, K., Understanding m-commerce,
Quarterly Journal of Electronic Commerce 3 (3)
(2002) 247– 271.
ISO/IEC 9126 (2001). Software Product Evaluation –
Quality Characteristics and Guidelines for the User.
Geneva: International Organization for
Kwon, O.B., Sadeh, N., Applying case-based reasoning
and multi-agent intelligent system to context-aware
comparative shopping, Decision Support Systems 37
(2) (2004) 199–213.
Losavio, F., Chirinos, L., Matteo, A., Levy, N., &
Ramdane, A. (2004). ISO quality standards for
measuring architectures. The Journal of Systems and
Software, 72, 209–223.
Malloy, A., Varshney, U., Snow, A., (2002). Supporting
Mobile Commerce Applications Using Dependable
Wireless Networks, Mobile Networks and
Applications, 7, 225–234,
Mobile Web Best Practices 1.0, W3C Proposed
Recommendation. http://www.w3.org/TR/mobile-bp/
Ngai, E.W.T., Gunasekaran, A., (2007). A review for
mobile commerce research and applications, Decision
Support Systems, 43, 3-15
Nielsen, J., Molich, R., (1990). Heuristic Evaluation Of
Users Interfaces, CHI90, pp. 249 - 256
Samaras, G., Mobile commerce: vision and challenges
(location and itsmanagement), Applications and the
Internet, 2002. (SAINT 2002). Proceedings. 2002
Symposium on, pp. 43-44,
Stefani, A., Stavrinoudis, D., Xenos, M., 2004.
Experimental Based Tool Calibration used for
Assessing the Quality of E-Commerce Systems. In the
IEEE International Conference on E-Business and
Telecommunication, Networks, Portugal, ICETE
2004, pp. 26-32.
Stefani, A., Stavrinoudis, D., Xenos, M., 2005.,. In depth
analysis of selected topics related to the quality
assessment of e-commerce systems. In the
International Conference on E-Business and
Telecommunication, Networks, Reading, UK, ICETE
2005, pp.122 -128..
ICE-B 2007 - International Conference on e-Business