Do We Need Specific Quality Models for Multi-Agent Systems?

Toward Using the ISO/IEC 25010 Quality Model for MAS

Toufik Marir

, Farid Mokhati

1,2

and Hassina Bouchlaghem-Seridi

Department of Mathematics and Computer Science, University of Oum El Bouaghi, Oum El Bouaghi, Algeria

LAMIS laboratory, University of Tébessa, Tébessa, Algeria

Department of Computer Science, LABGED Laboratory, University of Annaba, Annabah, Algeria

Keywords: Quality Model, Multi-Agent Systems, ISO/IEC 25010.

Abstract: The quality assurance of software is one of the most important purposes of software engineering. However,

the quality concept is undergone to subjective interpretations. Consequently, several models are proposed to

understand and evaluate the software quality. The ISO/IEC 25010 is the new international standard quality

model. Being standard increases the applicability of such model for all software products. Knowing that the

multi-agent systems are one of the most applied software paradigms; we target in this paper the applicability

of the ISO/IEC 25010 to agent-based software.

1 INTRODUCTION

The quality is one of the most required goals in

software engineering. This concept encompasses

several software characteristics which can be

sometimes conflicting (ISO/IEC, 2001). Moreover,

it is undergone to subjective interpretations. Thus,

several models are proposed to unambiguously

specify and objectively measure the software

quality. Furthermore, some standards are proposed

by the ISO (ISO/IEC, 2001; ISO/IEC, 2011).

Nowadays, the software engineering domain is

characterized by the diversity of the software

paradigms with owned specificities which require

their specific development approaches. Specifically,

each software paradigm needs its specific quality

model. Previously, the efforts were devoted to

develop quality models for specific software

paradigms (Alonso et al., 1998). However,

customizing the standard quality models to support

the specificities of each paradigm is the purpose of

the recent research works (Behkamal, Kahani and

Akbari, 2009; Lew, Olsina and Zhang, 2010).

Knowing that the multi-agent systems (MAS) are

one of the most applied software paradigms

nowadays; this work in progress targets the quality

of agent-based software. Specifically, we study in

this paper the ability to apply an international

standard quality model, called ISO/IEC 25010

(ISO/IEC, 2011), to MAS. Considering the almost

proposed studies in this field as empirical ones

(Dumke et al., 2010) is our main motivation. Despite

that Alonso et al., attempt through a set of works

(Alonso et al., 2008; Alonso et al., 2009; Alonso et

al., 2010) to develop a specific quality model for

MAS; we believe that these studies should be

preceded by the study of the ability to apply the

standard quality models to such systems. It seems

obvious that the use of standard quality models,

when it is possible, is more beneficial than the use of

specific ones.

In order to reach our goal, we start by identifying

the main features of the MAS. The identified

concepts are studied comparing to the sub-

characteristics of the ISO/IEC 25010 in order to

identify the MAS’ concepts which are not specified

in such quality model. This latter should be finally

extended to support the lacked concepts.

The remainder of this paper is organized as

follow. In section 2 we present the multi-agent

systems in order to identify the key concepts of such

paradigm. We give an overview of the ISO/IEC

25010 in section 3. Section 4 is devoted to present

the suggested extensions applied to ISO/IEC 25010

for supporting MAS. Finally, conclusion and some

future works are presented in section 5.

363

Marir T., Mokhati F. and Bouchlaghem-Seridi H..

Do We Need Speciﬁc Quality Models for Multi-Agent Systems? - Toward Using the ISO/IEC 25010 Quality Model for MAS.

DOI: 10.5220/0005097303630368

In Proceedings of the 9th International Conference on Software Engineering and Applications (ICSOFT-EA-2014), pages 363-368

ISBN: 978-989-758-036-9

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

2 MULTI-AGENT SYSTEMS

Multi-agent systems represent a well known

software paradigm which allows the modelling and

the development of complex systems. Consisted of

the intersection of several fields (such as software

engineering, distributed systems and artificial

intelligence), such paradigm is applied in various

application domains ranging from games to space

shuttles. The drawback of this variety is reflected in

the lack of a consensus about the agent definition.

We adopt in this paper the definition of

Wooldridge (2009). So, an agent is “a computer

system that is situated in some environment, and that

is capable of autonomous action in this environment

in order to meet its design objectives”. Thus, the

autonomy is the main property of an agent.

However, the intelligent agent can be characterized

by some additional properties, such as the reactivity,

the pro-activeness and the social ability

(Wooldridge, 2009). The central concepts of agents

are defined as following:

 Autonomy: the capacity of the agent to act

without the intervention of others.

Therefore, an autonomous agent has the

control over its internal state and its

behaviour.

 Situatedness: the ability of the agent to

sense and act on its environment. It seems

important to note that the environment can

either physical or software environment.

 Reactivity: the ability of the agent to

perceive and to give an adequate response

in required time.

 Pro-activeness: the ability of the agent to

exhibit goal-oriented behaviours.

 Social ability: the ability of the agent to

interact with other agents in order to

achieve its purposes.

Using the agent as the key concept of a MAS, we

can define MAS as a set of interacting agents. Figure

1 gives the meta-model of the MAS.

Several reasons encourage us to adopt this

definition of the agent. Firstly, it is one of the most

accepted in the field community. In addition, it

remains valid without interesting updates since its

first version proposed by Wooldridge and Jennings

(1995). Finally, it gives only the essential properties

of the agent, called weak notion of agency, and

allows adding other ones (such as the adaptability

and the mobility). So, adopting this weak notion

increases the applicability and the extensibility of

our work.

Figure 1: The meta-model of MAS.

In the literature, many works addressed the

quality of MAS. However, almost of them are

proposed to assess the different aspects related to the

agent-based software (Dumke et al., 2010).

Motivated by the lack of specific quality model

for MAS, Alonso et al. proposed a series of works in

order to develop a quality model for this software

paradigm (Alonso et al., 2008; Alonso et al., 2009;

Alonso et al., 2010). As a first step of this series,

they identified the main features of the MAS which

consisted of (Alonso et al., 2008): the social ability,

the autonomy, the pro-activity, the reactivity, the

mobility, the intelligence, and the adaptability. Then,

they studied separately each feature by decomposing

it in a set of attributes. In addition, the attributes can

be assessed using proposed metrics. For example,

the social ability is decomposed in communication,

cooperation and negotiation attributes (Alonso et al.,

2008). The autonomy and pro-activity have been

studied respectively in (Alonso et al., 2009) and

(Alonso et al., 2010).

Despite the importance of the approach proposed

by Alonso et al., (2008, 2009 and 2010); we believe

that it suffers from several limits. Firstly, the

proposed characteristics and their relationships are

questionable. For instance, are the mobility and the

Situated in

Environment

Autonomous Behavior

Flexible Behavior

Interactional Behavior

Proactive Behavior

Reactive Behavior

1 .. n

1.. n

0.. n

Agent

MAS

Object

ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications

364

adaptability fundamental features for any agent?

What we intend by the intelligence concept? And is

this latter totally independent from the pro-activity,

the reactivity and the adaptability? Therefore, the

proposed quality model omitted the relationships

between the cited agent specific features and the

high level software quality characteristics (such as

the reliability, the security, the maintainability, etc).

Away from all these issues, the most important

asked questions: Do we really need a specific

quality model for MAS in addition to the existing

standard ones? Specifically, is the ISO/IEC 25010

quality model unsuitable for agent-based software?

What are the possible updates that must be carry out

to this international standard quality model to

support the specificities of MAS? In fact, it is more

beneficial to use the standard quality model when

possible than the proposition of specific one for each

software paradigm. In this paper, we attempt to give

answers to the above questions.

3 ISO/IEC 25010 QUALITY

MODEL

Since 1970s where the software quality became one

of the subjects of software engineering, several

quality models are developed. Despite the usefulness

of the proposed models, their diversity made

confusion (Behkamal, Kahani and Akbari, 2009).

Consequently, the international standards are

proposed as an answer to this situation.

ISO/IEC 9126 (ISO/IEC, 2001) defined the

quality model as “the set of characteristics and the

relationships between them which provide the basis

for specifying quality requirements and evaluating

quality”. In fact, this quality model is composed of

three layers: characteristics, sub-characteristics and

metrics. In the first layer, it specified six

characteristics: the functionality, the reliability, the

usability, the efficiency, the maintainability and the

portability. Therefore, the second layer is composed

of twenty-seven sub-characteristics. Each sub-

characteristic can be evaluated using a set of metrics

proposed in the third level of this quality model.

Recently, the ISO/IEC 9126 (ISO/IEC, 2001) is

improved through the proposition of new

international standard quality model, called ISO/IEC

25010 (ISO/IEC, 2011). In fact, the ISO/IEC 25010

is proposed as a part of the SQuaRE series of

standards which considered as a new generation of

software quality models (Suryn and Abran, 2003).

These improvements consisted of adding new

characteristics, adding new sub-characteristics,

extending the scope of the quality model and

renaming some characteristics and sub-

characteristics (ISO/IEC, 2011). Consequently, the

ISO/IEC 25010 became composed of eight

characteristics: the functional suitability, the

reliability, the performance efficiency, the

operability, the security, the compatibility, the

maintainability and the transferability. Therefore,

these characteristics are decomposed on thirty-eight

sub-characteristic. The space limit of this paper

prevents us to present deeply this quality model.

Both the international standard quality models

were applied to specific software products. For

examples, the ISO/IEC 9126 standard was updated

to support the specificities of the specifications test

(Zeiss et al., 2007) and the B2B applications

(Behkamal, Kahani and Akbari, 2009). Similarly,

the ISO/IEC 25010 was applied to Web applications

(Lew, Olsina and Zhang, 2010), quality in use of

Web portals (Herrera et al., 2010) and to evaluate

the performance of cloud computing systems

(Bautista, Abran and April, 2012). Generally, the

application of the international standard quality

models passes through adding some specific

characteristics or sub-characteristics. In addition,

specific characteristics or sub-characteristics can

replace existing ones. For example, Zeiss et al.

(2007) added the reusability and replaced the

functionality by the test affectivity in order to apply

the ISO/IEC 9126 to specifications test. On the other

side, the sub-characteristics’ level has been

addressed by adding the traceability, availability,

customizability and navigability in the case of B2B

quality (Behkamal, Kahani and Akbari, 2009).

According to Behkamal, Kahani and Akbari

(2009), the variety of quality models is a source of

confusion. Thus, the development of specific quality

models for a specific software products or paradigms

is not supported when the application of the

international standard models is suitable. This point

of view is justified by the generality which

characterizes both ISO/IEC 9126 and ISO/IEC

25010 quality models. The next section is devoted to

study the applicability of the ISO/IEC 25010 to

MAS.

DoWeNeedSpecificQualityModelsforMulti-AgentSystems?-TowardUsingtheISO/IEC25010QualityModelfor

MAS

365

4 TOWARD APPLYING ISO/IEC

25010 TO MULTI-AGENT

SYSTEMS

As we mentioned above, the international standard

quality models can be applied to all software

products, taking into account the specificities of

each one. Consequently, we study in this section the

suitability of using the ISO/IEC 25010 to specify the

quality of MAS. Moreover, we attempt to propose

some extensions to support the specific features of

MAS. Compared to the ISO/IEC 9126, the choice of

ISO/IEC 25010 is unquestionable because this

quality model is a revision of the first one.

In order to reach our purpose, we apply the

following steps. First, we identify the essential

features of MAS. Then, we check the suitability of

these features according to the ISO/IEC 25010 sub-

characteristics. As a result, we select the features of

MAS which are not specified in this quality model.

Finally, we use the selected features to propose an

extension of the ISO/IEC 25010 that supports MAS.

As presented in section 2, we choose the

definition of the agent proposed by Wooldridge

(2009). Hence, the essential characteristics which

made of the agent a distinguish software paradigm

are: the autonomy, the situatedness, the reactivity,

pro-activeness and the social ability. Based on the

weak notion of agency, this list represents the

essential features of any agent. Choosing only the

essential features of agents gives two main benefits.

It gives an agreed level of consensus for our

proposed extension. On the other hand, it allows us

to extend the actual work to support some specific

kinds of agents (like mobile agent).

Compared to the features presented by Alonso et

al., (2008), we have as common ones: the autonomy,

the reactivity, the pro-activeness and the social

ability. Our list is extended only by the situatedness

that refers to the MAS environment. This latter is

considered as an essential part of any MAS

(Wooldridge, 2009; Weyns, Omicini and Odell,

2007; Beydoun et al., 2009). However, Alonso et

al., (2008) presented the mobility, the intelligence

and the adaptability as required characteristics of

agents. We think that these characteristics are not

only questionable, but they are also a source of

overlapping. The adaptability and the mobility are

additional features for only some kinds of agents

(adaptive and mobile agents) (Beydoun et al., 2009;

Wooldridge, 2009). In addition, we believe that the

intelligence is ambiguous concept which overlaps

with other features. Wooldridge (2009) made clear

that the reactivity, the pro-activeness and the social

ability are suggested capabilities of an intelligent

agent. We see also that the adaptability is an

advanced capability of the intelligence.

After selecting the main features of MAS, we

formulate them according to the ISO/IEC 25010

sub-characteristics’ definitions. This formulation

allows us to check the suitability of these features

according to the ISO/IEC 25010 sub-characteristics.

Consequently:

 The Autonomy: the degree to which an

agent has a control over its state and its

behaviour.



The Situatedness: the degree to which an

agent is able to perceive and act on its

environment.

 The Reactivity: the degree to which an

agent is able to perceive the occurred

changes and to provide timely responses to

them in order to achieve its goals.

 The Pro-activity: the degree to which an

agent is able to take the initiative in order to

satisfy its requirements.

 The Social Ability: the degree to which an

agent is able to interact with other agents to

satisfy its purposes.

The suitability of these features according to the

ISO/IEC 25010 sub-characteristics is checked in

order to identify the limits of such model to specify

the quality of MAS. By suitability relationship we

mean that the definition of the agent’s feature is

partially or completely covered by the definition of

the quality sub-characteristic. Because of the limit

size of this paper, we cannot present all the

combinations (quality sub-characteristic, agent

feature). So, here we present only the essential

combinations.

The ISO/IEC 25010 defined the confidentiality

sub-characteristic by (ISO/IEC, 2011) “the degree to

which the software product provides protection from

unauthorized disclosure of data or information,

whether accidental or deliberate”. Knowing that the

agent states are represented by a set of information

and knowledge, we can conclude that the

confidentiality sub-characteristic cover partially the

autonomy concept. In fact, protecting of these

knowledge and information from unauthorized

accesses provides to the agent the control over its

state. In addition, the accountability sub-

characteristic (“the degree to which the actions of an

entity can be traced uniquely to the entity”

(ISO/IEC, 2011)) is strongly close to the ability of

the agent to control its behaviour. So, the two

aspects of the autonomy feature (the control of agent

ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications

366

state and the control of the agent behaviour) are both

covered by the ISO/IEC 25010 sub-characteristics

(respectively by the confidentiality and the

accountability).

The social ability designates the ability of the

agent to interact with other agents. This interaction

represents the ability of the agents to operate

cooperatively in order to meet their purposes. As a

result, the interoperability (“the degree to which the

software product can be cooperatively operable with

one or more other software products” (ISO/IEC,

2011)) is an adequate concept to represent the social

ability in the ISO/IEC 25010. Moreover, the social

ability implies the co-existence of the agents in some

environment and sharing common resources with

possible conflict relationships. The co-existence as

“the degree to which the software product can co-

exist with other independent software in a common

environment sharing common resources without any

detrimental impacts” (ISO/IEC, 2011) is fully

correspondent to the latter aspect of the social

ability.

The situatedness refers to the ability of the agent

to interact with its environment. Abstracting the

environment by a set of objects that composed it, we

can consider the interoperability as a strong

candidate to cover the ability of the agent to operate

with the environment objects. However, more depth

analysis shows the drawbacks of this first opinion.

Firstly, the interoperability in the ISO/IEC 25010

refers to the ability of the software product to be

operable with one or more other software products.

Thus, the objects which composed the environment

of the agent should be, in this case, of software

nature. However, the environment of MAS can be

either of physical or software nature. In addition, the

interaction according to the above definition of the

interoperability is summarized on the “operability

with the objects”. It does not provide the possibility

of carrying out action on these objects, which is a

fundamental aspect in the situatedness feature. For

these reasons, we can say that the situatedness is not

covered in the ISO/IEC 25010.

The reactivity is the ability of the agent to

perceive changes that are occurred in its

environment and gives timely responses to them. So,

this feature outlines two basic operations: perceiving

the environment and giving timely responses. There

is no sub-characteristic in the ISO/IEC 25010

corresponding to observe the environment. The

timely responses can be ensured by the time

behaviour which is defined as “the degree to which

the software product provides appropriate response

and processing times and throughput rates when

performing its function, under stated conditions”

(ISO/IEC, 2011).

Finally, we cannot find any sub-characteristic of

ISO/IEC 25010 that covers the ability of the agent to

take the initiative in order to satisfy its requirements

(the pro-activeness).

The previous analysis gives the following results:

the autonomy and the social ability are adequately

covered by the ISO/IEC 25010 sub-characteristics

but the situatedness and the pro-activeness are not.

In addition, the reactivity is partially covered by this

quality model. Thus, the ISO/IEC 25010 cannot

express the following specificities of MAS:

 The ability to exhibit goal-oriented

behaviours.

 The ability to perceive and act on the

environment.

 The ability to perceive changes occurred in

the environment.

We formulate these features in a consistent way

to avoid the possible confusion and to be compatible

with the form of the ISO/IEC 25010 sub-

characteristics. Consequently, the following notions

are presented as the specific sub-characteristics for

MAS:

 The Proactive-ability: is the degree to

which the agent is able to take the

initiative in order to satisfy its goals.

 The Act-ability: is the degree to which an

agent is able to carry out actions on its

environment in order to satisfy its goals.

 The Perceive-ability: is the degree to

which an agent is able to perceive and

detect changes that are occurred in its

environment.

Finally, we extend the ISO/IEC 25010 quality

model by adding the above sub-characteristics.

Hence, the proactive-ability is added to the

functional suitability characteristic; the act-ability

and the perceive-ability are added to the

compatibility characteristic.

Compared to the model of Alonso et al., (2008),

this extended version of ISO/IEC 25010 specifies

the quality of MAS according to the international

standard framework. Moreover, we choose carefully

the specific features of MAS in order to avoid

confusion and to ensure a wide applicability of our

proposed extension. In addition, our proposal is open

to possible extensions to support more specific kinds

of MAS (like mobile agent).

DoWeNeedSpecificQualityModelsforMulti-AgentSystems?-TowardUsingtheISO/IEC25010QualityModelfor

MAS

367

5 CONCLUSIONS

The multi-agent systems represent an ideal software

paradigm to develop complex systems. As all

software products, their quality is an essential

requirement. In order to understand and evaluate the

software quality, several models are proposed. Some

proposed models are the subject of the

standardization by the ISO.

This paper addressed the quality of MAS. Its

main purpose is studying the ability of applying the

international standard quality model ISO/IEC 25010

to MAS. Because of the confusion which may arise

from the diversity of the specific quality models, we

think that using the ISO/IEC 25010 to MAS is more

beneficial than the development of specific quality

model for such systems.

The suitability of ISO/IEC 25010 quality model

to be applied to MAS is studied according to the

weak notion of agency. Thus, we proposed some

extensions to support the specificities of such

systems. The proposed extensions consisted of

adding the proactive-ability, the act-ability and the

perceive-ability as sub-characteristics to ISO/IEC

25010 to reflect, respectively, the pro-activeness, the

situatedness and the reactivity features. Based on the

weak notion of agent, our work is extensible to

cover other features of agents.

This work in progress is only the first step in the

study of the quality of MAS. As future works, we

plan to extend this actual version to support more

features of specific kinds of agents (such as the

cognitive agent, the adaptive agent and the mobile

agent). Furthermore, it seems useful to apply other

standards of the SQuaRE series to measure the

quality of MAS.

REFERENCES

Alonso, F., Fuertes, J. L., Montes, C., Navajo, R. J., A.,

1998. ‘Quality Model: How to Improve the Object

Oriented Software Process’, In IEEE International

Conference on Systems, Man and Cybernetics, VOL

05, Pages 4884 – 4889.

Alonso, F., Fuertes, J. L., Martinez, L., Soza, H., 2008.

‘Measuring the Social Ability of Software Agents’ In

the Proceedings of the 6th ACIS International

Conference on Software Engineering Research,

Management and Applications.

Alonso, F., Fuertes, J. L., Martinez, L., Soza, H., 2009.

‘Towards a set of Measures for Evaluating Software

Agent Autonomy’, In Proceedings of the Eighth

Mexican International Conference on Artificial

Intelligence.

Alonso, F., Fuertes, J. L., Martinez, L., Soza, H., 2010.

‘Measures for Evaluating the Software Agent Pro-

Activity’. In the Proceedings of the 25th International

Symposium on Computer and Information Sciences.

Bautista, L., Abran, A., April, A., 2012. ‘Design of a

Performance Measurement Framework for Cloud

Computing’ In Journal of Software Engineering &

Applications . Feb2012, Vol. 5 Issue 2, p69-75. 7p.

Behkamal, B., Kahani, M., Akbar, M. K., 2009.

‘Customizing ISO 9126 Quality Model for Evaluation

of B2B Applications’, In Information and Software

Technology, Elsevier.

Beydoun, G., Low, G., Henderson-Sellers, B., Mouratidis,

H., Gomez-Sanz, J., Pavon, J., Gonzalez-Perez, C.,

2009. ‘FAML: A Generic Metamodel for MAS

Development’ In Ieee Transactions On Software

Engineering, Vol. 35, No. 6.

Dumke, R., Mencke, S., Wille, C., 2010. ‘Quality

Assurance of Agent-Based and self-Managed

Systems’, CRC Press.

Herrera, M., Moraga, M. A., Caballero, I., Calero, C.,

2010. ‘Quality in Use Model for Web Portals

(QiUWeP)’ In 10th International Conference on Web

Engineering ICWE 2010 Workshops.

ISO/IEC, 2001. ‘ISO/IEC 9126-1:2001 Software

Engineering - Product Quality’.

ISO/IEC, 2011. ‘ISO/IEC 25010:2011- Systems and

Software Engineering - Systems and Software Product

Quality Requirements and Evaluation (SQuaRE) -

System and Software Quality Models”.

Lew, P., Olsina, L., Zhang, L., 2010.’ Quality, Quality in

Use, Actual Usability and User Experience as Key

Drivers for Web Application Evaluation’ In

proceeding of 10th International Conference, ICWE

2010, Web Engineering - Lecture Notes in Computer

Science Volume 6189, pp 218-232

Suryn, W., Abran, A., 2003. ‘ISO/IEC SQuaRE: The

second generation of standards for software product

quality’ In Software Engineering and Applications,

SEA’ 2003.

Weyns, D., Omicini A., Odell, J., 2007. ‘Environment as a

first class abstraction in multiagent systems’ In

Journal of Autonomous Agent and Multi-Agent

Systems 14:5–30.

Wooldridge, M., Jennings, N., 1995. ‘Intelligent Agent:

Theory and Practice’, The Knowledge Engineering

Review, Vol 10:2, (1995), 112-152.

Wooldridge M., 2009.’An Introduction to Multi Agent

Systems’, JOHN WILEY & SONS, 2

Edition.

Zeiss, B., Vega, D., Schieferdecker, I., Neukirchen, H.,

Grabowski, J., 2007. ‘Applying the ISO 9126 Quality

Model to Test Specifications - Exemplified for TTCN-

3 Test Specifications’, In Software Engineering 2007

(SE 2007). Lecture Notes in Informatics (LNI) 105,

231-242.

ICSOFT-EA2014-9thInternationalConferenceonSoftwareEngineeringandApplications

368