AN EVOLUTIONARY APPROACH FOR QUALITY MODELS
INTEGRATION
Rodrigo Santos de Espindola and Jorge Luis Nicolas Audy
Faculty of Informatics, PUCRS, Av. Ipiranga 6681, Porto Alegre – RS, Brazil
Keywords: Software quality models, Software process improvement, Integration.
Abstract: The existing quality models (as ISO/IEC 15504, CMMI, MPS.BR, ITIL, COBIT) establish different
processes and controls that must be adopted to achieve high software process reliability. Whereas it’s
possible notice similarities and overlapping areas among them, a systematic approach to integrate quality
models is not widely explored in the literature. In this work we propose an evolutionary approach to
integrate quality models. The approach defines a method that can be executed in a systematic way and has a
meta-model and a mapping table as outcome. The method is composed by two stages: the meta-model
development and the meta-model stabilization. As this is an ongoing research, this work is presenting the
application and the results from the execution of the first stage. As a result, a meta-model representing the
structure of four different quality models was developed and its applicability was verified.
1 INTRODUCTION
Several organizations, especially government, have
been demanding from software providers that they
prove high quality software development. This has
motivated the creation of several software process
quality models and standards.
Furthermore, software companies are investing
effort on software processes improvement (SPI) to
achieve several software quality certifications
demanded by different software consumers.
However, the integration of these efforts are not
trivial because several process management models,
techniques and best practices are available, like
ISO/IEC 15504, CMMI, MPS.BR, ITIL e COBIT.
Each one of them establishes different processes and
controls that must be adopted to achieve high
software process reliability and capacity.
Whereas it’s possible notice similarities and
overlapping areas among those models, techniques
and best practices, the integration among them is not
widely explored in the literature. The related work
found present the integration among specific models
by using ad-hoc methods. There is a lack of generic
approaches to support the organization on selecting
methods and integrating models.
This has motivated this research, which intends
to create an evolutionary approach to integrate
software process quality models. The proposed
approach uses meta-models to integrate software
process quality models and includes the
development of methods to do this. As future work,
a software tool to support its usage will be designed
and implemented.
The remaining of this paper is structured as
follows. Section 2 presents related works; section 3
details the proposed approach to integrate software
process quality models and presents the method
execution in order to test the proposed approach;
and, finally, section 4 addresses conclusions and
future work.
2 RELATED WORKS
Generally, software process improvement uses
models as basis. Adopting a standard software
quality model can improve quality and control costs
by more accurate estimations and planning.
There are several software quality models
adopted by the industry. The following studies are
related to the integration of quality models.
In (Pickerill, 2005) a relationship between
IDEAL (developed by SEI) and Six Sigma is
demonstrated. Using IDEAL as reference model,
Study developed by the Research Group of the PDTI 01/2008,
financed by Dell Computers of Brazil Ltd. with resources o
f
Law 8.248/91.
231
Santos de Espindola R. and Luis Nicolas Audy J. (2009).
AN EVOLUTIONARY APPROACH FOR QUALITY MODELS INTEGRATION.
In Proceedings of the 11th International Conference on Enterprise Information Systems - Information Systems Analysis and Specification, pages
231-236
DOI: 10.5220/0002008202310236
Copyright
c
 SciTePress

this works proposes the usage of both Six Sigma
implementation methods (DMADV and DMAIC) to
develop and implement process with CMMI.
(Siviy and Hallowell, 2005) has complemented
this research, evaluating the usage of Six Sigma as a
facilitator on CMMI implementation. The
conclusions demonstrated that the implantation
process and ROI verification have been accelerated.
(Rout, Tuffley and Cahill, 2001) presents a
technical report that evaluates the compatibility
between CMMI and ISO/IEC 15504-2. As a result, a
mapping table is presented and the report states that
the ISO/IEC 15504-2 significant elements are
addressed by CMMI.
A definition of a meta-model to integrate CMMI
and ISO/IEC 15504 is presented in (Lepasaar and
Mäkinen, 2002). The meta-model was applied in
both models to identify the existing structures.
On despite of high number of quality models
available in the industry, they do not cover the
integration among them. The related works do not
present a systematic approach able to incorporate
new models in an evolutionary way. Also, the
integration presented is realized in an ad-hoc way,
making difficult the integration of new models in the
same structure.
The study in (Lepasaar and Mäkinen, 2002)
distinguishes from other studies by proposing a
meta-model to support integration. As stated in
(OMG, 2005), (OMG, 2006), and (OMG, 2007),
meta-models are utilized to support integration of
processes, workflows, tools, database, and
middleware’s.
A similar idea is used in the approach proposed
in this paper. The main difference herein is the
proposal of an evolutionary approach and a method
to build a meta-model in order to integrate a chosen
set of quality models, in contrast of the ad-hoc ways
used in the related works. The goal is to integrate
new quality models to the meta-model whenever it
becomes necessary.
3 EVOLUTIONARY APPROACH
FOR QUALITY MODELS
INTEGRATION
The analysis of the main existing quality models,
such as CMMI (SEI, 2006), ISO/IEC 15504 (ISO,
1998), ISO/IEC 20000 (ISO, 2005), COBIT (IT
Governance Institute, 2005), reveals distinct
structures. Each model structure has a set of
elements. By evaluating these elements individually,
similar, or even equivalent, characteristics can be
identified in different models. This section describes
the evolutionary approach defined by the analysis,
representation, comparison, and mapping among
structural elements existing in the evaluated quality
models.
3.1 Architecture Definition
This work proposes a method to build a meta-model
with the aim of representing the structure of the
analyzed quality models. UML was adopted as
notation and OMG modelling architecture [(OMG,
2005), (OMG, 2006) and (OMG, 2007)] as
architectural base.
The Quality Models meta-model is built by a
method, detailed in the section 3.2. This method
intends to create and maintain the meta-model and a
mapping table. The mapping table registers how
every structural element from quality models is
represented in the meta-model.
We propose an evolutionary approach that allows
the gradual evolution of the meta-model. This way,
as the result of successive execution of the method
for different quality models, the meta-model will
show a continuous stability growing, being able to
represent a high number of quality models.
3.2 Method Description
The proposed method intends to create and maintain
the meta-model and the table mapping. Both meta-
model and table mapping are created and
incrementally expanded by analyzing each quality
model.
The method shall be executed every time a new
quality model is being integrated. The execution of
the method for an specific model, shall bring as
result that 1) the meta-model actualization to
incorporate the specific characteristics of the quality
model in evaluation, or 2) the quality model
mapping to the meta-model, when the meta-model is
already contemplating all characteristics.
This method has two stages of execution. The
first stage proposes the meta-model development.
Quality model structure analysis, quality model
structure modelling, meta-model mapping, and
meta-model adaptation are the activities performed
during this stage. The second stage proposes the
meta-model stabilization. Content mapping and pilot
testing are the activities performed during this stage.
In case defects are found during testing execution,
they can be fixed by performing both meta-model
mapping and meta-model adaptation activities.
ICEIS 2009 - International Conference on Enterprise Information Systems
232

Quality model structure analysis activity
goals
are: quality models analysis and structure
identification. The identification of the structural
elements and their relationships comes from the
documentation reading. Some quality models have a
graphical or textual representation of its structure in
the documentation. This representation can be very
helpful to the analysis. However, it cannot be
considered as a definitive representation of the
model and does not eliminate the need of an
analysis.
Quality model structure modelling activity
intends to formalize the quality model structure in a
diagram. A class diagram from UML is the notation
adopted. This notation allows the static
representation of all structural models and their
relationships. Only the conceptual modelling is
done. This activity can be performed in parallel with
quality model structure analysis activity. Every
structural element is represented as a class and their
relationships are represented as classes association.
Model mapping activity
uses the Mapping Table
artefact (Figure 1), where each column indicates the
quality model analyzed and the last column is
reserved to the meta-model elements. Every time the
method is executed, a new column is included. The
comparison is used to decide which elements are
equivalents.
Model 1 Model 2 Model N Metamodel
<element 1> <element x> … <element a>
…………
<element n> <element z> … <element c>
Mapping Table
Figure 1: Mapping Table.
Each element analysis leads to decide either if
there is a correspondent element in the meta-model
or if it is necessary creates a new element. This
decision is made based on the equivalency analysis
between the elements concepts, not only using their
names. When the element concept already exists in
the mapping table, the element name is included in
the same line that the element is presented in the
meta-model and in the column related to the quality
model being analyzed. When the element concept
does not exist in the mapping table, then a new line
is included and filled out in both quality model and
meta-model columns. In this case, the meta-model
adaptation activity needs to be executed also. The
element name to be given in the meta-model
depends on how representative it is comparing with
the other quality models. If a given element exists in
only one quality model, then it will receive the same
name in the meta-model column. If the element
exists in several quality models, then the meta-
model element will be named by the most popular
one.
Meta-model adaptation activity
involves the
inclusion of new elements in the meta-model and
their relationships adjustments. This activity uses the
mapping table and the class diagram as input. As
result of its execution, the meta-model is modified.
The first step is the new elements inclusion. For
each new line included in the mapping table a new
class is created in the meta-model. Besides, the
relationships between the new element and the
existing ones are created. These relationships and
their multiplicities are imported from the class
diagram. The second step consists of adjustments on
existing relationships. These adjustments are needed
when the element is not new, and its relationships
rules are different in the quality model that is being
analyzed.
Content mapping activity
intends to create an
instance of the quality model. The quality model
needs to be codified and stored according to the
meta-model. This activity is the one that demands
the higher effort in the method. The content transfer
is executed over whole quality model. As many data
is found, greater is the effort to execute this activity.
Once the mapping is done, the tool can be used to
support the quality models integration several times.
This approach does not include the tools necessary
to perform this task; neither restricts it to a specific
software solution. The elaboration of a tool is not in
the scope of this work. However, it is considered as
future work on the ongoing research.
Pilot testing activity
involves the meta-model
verification and consists on using the mapped
quality models content to help a SPI project.
3.3 Method Execution: First Stage
The first stage of the proposed method was applied
in four quality models (CMMI, ISO/IEC 15504,
ISO/IEC 20000 and COBIT) in order to test the
method and produce both the meta-model and the
mapping table. An additional analysis was done to
verify the meta-model applicability against a quality
model that was not used on its development (MR-
MPS).
The activities defined in the second stage were
not performed as they need a software tool and an
organization with a SPI project. Those activities are
characterized as future work.
AN EVOLUTIONARY APPROACH FOR QUALITY MODELS INTEGRATION
233

3.3.1 CMMI
The CMMI version 1.2 was analyzed. The CMMI is
presented in a single volume available online (SEI,
2006). Both continuous and stage representations
were analyzed.
Quality model structure analysis
: The available
documentation presents a high level structure for this
model. CMMI has processes area, maturity levels,
purpose statement, introductory notes, related
process areas, specific goals, specific practices,
typical work products, sub-practices, generic goals,
capability levels, generic practices, generic practice
elaboration, process area categories, disciplines and
discipline amplification. The introductory notes
were considered not important to models integration.
Quality model structure modelling
: The structure
was modelled based on the information obtained in
the previous activity. A conceptual class diagram
was created to formalize the structure of the model.
Meta-model mapping
: As CMMI was the first
quality model analyzed, there was no meta-model to
map the elements. In this case, the mapping was
direct. A mapping table was created with all
structural elements from CMMI.
Meta-model adaptation
: No adaptation was
needed as CMMI was the first quality model
analyzed.
3.3.2 ISO/IEC 15504
The ISO/IEC 15504 version 1998 was analyzed. The
reference model, described in the part 2
documentation was utilized (ISO, 1998).
Quality model structure analysis
: The
documentation presents a description of the
structure. This standard presents process life cycle,
process categories, processes, notes, purpose
statement, process results, sub processes, processes
attributes, capability levels and process outcomes.
Quality model structure modelling
: The structure
was modelled based on the information obtained in
the previous activity. A conceptual class diagram
was created to formalize the structure of the model.
Meta-model mapping
: The existing mapping
table was incremented during this analysis. Every
element found (ISO/IEC 15504) was compared
against the existing ones (CMMI) and the proper
mapping was done. Two new elements, called
process life cycle and sub-process, were included.
The process category (ISO/IEC 15504) was
considered equivalent to process area category
(CMMI) based on the similarity of their contents and
purpose. The processes (ISO/IEC 15504) were
considered equivalent to process area (CMMI). In
both cases the name adopted in the meta-model was
the more generic: process area and process. The
remaining elements were considered existents in the
meta-model because of their equivalency to CMMI
elements.
Meta-model adaptation
: The meta-model was
adapted to contemplate the new elements and to
adjust the associations as needed.
3.3.3 ISO/IEC 20000
The ISO/IEC 20000 version 2005 was analyzed.
Both specification and Code of Practice (ISO, 2005)
were utilized during the analysis.
Quality model structure analysis
: The structure
of ISO/IEC 20000 was defined in this work by
reading the available documentation. The identified
structure has process, process areas, objective,
requirements, notes, clarifications, good practices
and references to other processes.
Quality model structure modelling
: The structure
was modelled based on the information obtained in
the previous activity. A conceptual class diagram
was created to formalize the structure of the model.
Meta-model mapping
: New structural elements
were not found during the ISO/IEC 20000 analysis.
All elements were included in the mapping table and
mapped to the meta-model. It was not necessary
change the meta-model names, as they were already
generic and were referring the concepts found in
ISO/IEC 20000. On despite of having the process
area element in both ISO/IEC 20000 and CMMI,
they do not have the same meaning. The process
area element in ISO/IEC 20000 has a similar
concept and the same characteristics that process
areas categories in CMMI and process categories in
ISO/IEC 15504.
Meta-model adaptation
: The third version of the
meta-model presents structural characteristics
compatible with CMMI, ISO/IEC 15504 and
ISO/IEC 20000. It was not necessary create new
concepts. Only the relationships were adjusted.
3.3.4 COBIT
The COBIT version 2005 was analyzed. The quality
model is presented in a single documentation, which
has its specification (IT Governance Institute, 2005).
Quality model structure analysis
: The COBIT
documentation presents a complete description of its
structure, which is more complex than the others
already analyzed and presents a high number of
structural elements. COBIT presents domains, IT
processes, business and IT objectives, metrics,
control requirements, IT governance focus areas, IT
ICEIS 2009 - International Conference on Enterprise Information Systems
234

CMMI ISO/IEC15504 ISO20000 CobiT Meta‐model
Processlifecycle Processlifecycle
Processareacategory Processcategory ProcessArea Domain Processcategory
Processarea Process(basictype) Process Process Process
Process(componenttype) Subprocess
Referto Note Note Note
Processareapurpose Processpurpose Objective Purpose Purpose
Specificgoal Processooutcome Requirement ITGoal/BusinessGoal Goal
Specificpractice Processooutcome Bestpractice Activity Practice
Subpractice Processooutcome Subpractice
Tipicalworkproduct WorkProduct Workproduct
Discipline Discipline
Disciplineamplification Disciplineamplification
MaturityLevel MaturityLevel
Capability
level Capabilitylevel MaturityLevel Capabilitylevel
Genericgoal Processattribute Processattribute
Genericpractice Attributeoutcome Attributeoutcome
Elaboration ProcessMaturityLevel Attributeelaboration
Role Role
GovernanceFocusArea GovernanceFocusArea
Resource Resource
BusinessRequirement/
GenericControl
Requirement
Requirement
Metric Metric
ControlObjective ControlObjective
MappingTable
Figure 2: Mapping table final version.
resources, roles, responsibilities, process activity,
input and output products and maturity levels.
Figure 3: Meta-model final version.
Quality model structure modelling: The structure
was modelled based on the information obtained in
the previous activity. A conceptual class diagram
was created to formalize the structure of the model.
Meta-model mapping
: New structural elements
were found during the COBIT analysis, as roles,
governance focus area, IT resources, business
requirements, control requirements, metrics and
control objectives. All elements were included in the
mapping table (Figure 2) and mapped to the meta-
model. It was not necessary changing the meta-
model names, as they were already generic and were
referring the concepts found in COBIT. The analysis
has shown that the element maturity level found in
COBIT has the same meaning that the capability
level in the meta-model.
Meta-model adaptation
: The new elements found
were included in the meta-model. Also, the
association between work product and practices
were adjusted to incorporate the concepts of input
and output artefacts. The final version of the meta-
model (Figure 3) presents structural characteristics
compatible with CMMI, ISO/IEC 15504, ISO/IEC
20000 and COBIT.
3.3.5 Meta-model Applicability
The MR-MPS is a reference model of software
processes, which is part of MPS.BR project
(Brazilian Software Process Improvement).
According to (SOFTEX, 2008), the MPS.BR intends
to define and improve a software process
improvement and assessment model. The MR-MPS
quality model was utilized to verify the meta-model
AN EVOLUTIONARY APPROACH FOR QUALITY MODELS INTEGRATION
235

applicability. Its structure was analyzed and mapped
to the meta-model.
The MR-MPS has processes organized in
processes classes. Each process has a purpose
statement, expected results and additional
information. Additional information was not
considered a structural element as it brings
references to other standards to help on MR-MPS
interpretation and process definition. Also, the
processes have process attributes that show their
institutionalization level and have expected results.
The analysis shown that it was not necessary any
adaptation in the meta-model. All structural
elements were already represented in final version of
the meta-model (Figure 3). Also, the relationships
were compatible with MR-MPS model. The
compatibility between the model and the meta-
model demonstrates the meta-model applicability to
MR-MPS. This result is justified by the origin of
MR-MPS, which is based in both CMMI and
ISO/IEC 15504.
4 CONCLUSIONS
The organizations are investing effort to adopt and
obtain several different certifications in order to
prove their capabilities and maturity. However, the
integration of these efforts represents an extra
challenge to the organizations, especially in software
engineering. Select some solution among all the
existing ones and apply it in an integrated way is not
trivial whereas it is necessary to maximize the
results.
This paper described the initial results of an
ongoing research. An evolutionary approach for
quality models integration was created and the first
stage of its application was demonstrated in a
systematic way. As a result, a meta-model
representing the structure of four different quality
models (CMMI, ISO/IEC 15504, ISO/IEC 20000
and COBIT) was developed. As future work a tool
to support the method execution and its testing in a
real SPI project must be implemented.
From the theoretical point of view, this research
has been contributing to the software engineering on
exploring the main factors involved on integrating
the analyzed models, techniques and good practices.
It contributes to improve the existing studies and to
provide a method to integrate some of the existing
quality models. Last, it contributes to the
experimental software engineering on evaluating
possible ways to do empirical studies in software
quality area, its difficulties and easiness.
From the researchers’ point of view, this work
contributes to their professional and academicals
learning and development, by being part of a
research that is being done with methodological
rigor. Besides, it contributes to provide interaction
between industry and academy, using the academic
resources and knowledge to solve the problems
found in the industry.
REFERENCES
Software Engineering Institute, 2006. “CMMI for
Development, Version 1.2”. Carnegie Mellon
University.
International Standard Organization, 1998. “ISO/IEC TR
15504-2:1998 Information technology — Software
process assessment — Part 2: A reference model for
processes and process capability”.
International Standard Organization, 2005. “ISO 20000-
1:2005 Information technology — Service
Management — Part 1: Specification”.
International Standard Organization, 2005. “ISO 20000-
1:2005 Information technology — Service
Management — Part 2: Code of practice”.
IT Governance Institute, 2005. “CobiT 4.0”. EUA, Ilinois:
IT Governance Institute, 207pp.
SOFTEX, 2008. “MPS.BR - Melhoria de Processo do
Software Brasileiro – Guia Geral”.
http://www.softex.br/mpsbr/
Pickerill, J., 2005. “Implementing the CMMI in a Six
Sigma World”. Software Engineering Institute -
Carnegie Mellon University.
http://www.sei.cmu.edu/cmmi/adoption/pdf/pickerill.p
df
Siviy, J., M., Hallowell, D., 2005. “Bridging the Gap
Between CMMI and Six Sigma Training: An overview
and Case Study of Performance-Driven Process
analysis”. Software Engineering Institute - Carnegie
Mellon University.
http://www.sei.cmu.edu/cmmi/adoption/comparisons.h
tml
Rout, T. P., Tuffley, A., Cahill, B, 2001. “Capability
Maturity Model Integration Mapping to ISO/IEC TR
15504-2:19998”. Software Quality Institute - Griffith
University – Austrália.
http://www.sqi.gu.edu.au/cmmi/report/docs/MappingR
eport.pdf
Lepasaar, M., Mäkinen, T, 2002. “Integrating Software
Process Assessment Models using a Process Meta
Model”. IEMC '02 - IEEE International Engineering
Management Conference, Vol 1, pp 224-229.
Object Management Group, 2005. “Software Process
Engineering Metamodel”. http://www.omg.org/spem/
Object Management Group, 2006. “Meta Object Facility
(MOF) Core Specification”. http://www.omg.org/mof/
Object Management Group, 2007. “Unified Modeling
Language: Superstructure”. http://www.omg.org/uml/
ICEIS 2009 - International Conference on Enterprise Information Systems
236