TOWARDS A QUALITY EVALUATION FRAMEWORK FOR

MODEL-DRIVEN WEB ENGINEERING METHODOLOGIES

F. J. Domínguez-Mayo, M. Mejías, M. J. Escalona and A. H. Torres

University of Seville, Spain

Keywords: Model-Driven Web Engineering, Quality, Software Metrics, Methodologies, Model-Driven Engineering.

Abstract: Various development methodologies currently exist in the field of Model-Driven Web Engineering

(MDWE). Given the high number of methodologies available, it is necessary to evaluate the quality of the

existing methodologies and provide helpful information to the developers. Furthermore, proposals are

constantly appearing and the need may arise not only to evaluate the quality but also to find out how it can

be improved. This article presents the work being carried out in this field and describes tasks to define a

Quality Evaluation Framework (QuEF) to evaluate, under objective measures the quality of Model-Driven

Web Engineering methodologies.

1 INTRODUCTION

Model-Driven Engineering (MDE) is a paradigm of

software development that consists of the creation of

models closer to a particular domain rather than

concepts or a specific syntax. The domain

environment specific to MDE for web engineering is

called Model-Driven Web Engineering (MDWE).

The Object Management Group (OMG) has

developed the standard Model-Driven Architecture

(MDA) which defines an architecture platform for

proposals based on the Model-Driven paradigm

The concept of platform independence appears

frequently in MDA. Models may have the quality of

being independent from the characteristics of any

technological platform. By applying this paradigm,

the lifecycle of a software system is completely

covered, from requirements capture to its own

maintenance, through the generation of the code. In

recent years, the growing interest in the internet has

led to the generation of a high number of proposals

(W. Schwinger et al., 2008) which offer a frame of

reference for the Web environment. On the other

hand, there are a high number of approaches without

standard consensus, a lack in the use of standards,

and scarcity of both practical experience and tool

support. In the face of this situation, an important

need to assess the quality of existing methodologies

arises. In this paper, therefore, an environment for

the quality evaluation of Model-Driven Web

methodologies based on MDA is proposed.

The paper is organized into the following

sections. In Section 2 a global analysis of the

situation is presented. Section 3 presents the

problem, motivation and goal, and is intended to lay

the basis of a framework that allows evaluate the

quality of different methodological proposals. In

Section 4 concepts such as framework and MDWE

methodology are explained and the elements which

define the Quality Evaluation Framework (QuEF)

are provided. Finally, in Section 5, a set of

conclusions, contributions and possible future work

are given.

2 RELATED WORK

2.1 Surveys

There are many proposals in the area of MDWE and

numerous comparative studies (Pérez, et al, 2007),

(Escalona and Aragón, 2008), (Kroiβ and Koch,

2008). Along these lines, (Schwinger et al., 2008)

must be considered, which specifically considers

modelling concepts for their ubiquitous nature,

together with an investigation of available support

for Model-Driven Development in a comprehensive

way, using a well-defined as well as fine-grained

catalogue of more than 30 evaluation criteria.

191

J. DomÃ nguez-Mayo F., MejÃ as M., J. Escalona M. and H. Torres A.

TOWARDS A QUALITY EVALUATION FRAMEWORK FOR MODEL-DRIVEN WEB ENGINEERING METHODOLOGIES.

DOI: 10.5220/0002784601910194

In Proceedings of the 6th International Conference on Web Information Systems and Technology (WEBIST 2010), page

ISBN: 978-989-674-025-2

2.2 Quality Evaluation

In (Cachero et al., 2008), an approach is proposed to

evaluate Web quality that provides all the elements

which, according to the ISO/IEC 14598, are

essential parts of a software quality evaluation. The

idea of developing a MDE framework for evaluating

quality has been applied in various studies of

(Mohagheghi et al.), where it is stated that the

quality of models is affected by the quality of

different factors.

2.3 Software Metrics

In the literature there are numerous references to

metrics (Etien and Rolland, 2005), (Briand et al,

2006), according to which, software measurement

integration could be achieved by adopting the MDA

approach. To this end, an approach is described in

(Garcia, et al, 2007) for the management of

measurement of software processes. From the

methodological perspective, software measurement

is supported by a wide variety of proposals, the ISO

15539 and IEEE 1061-1998 standards deserving

special attention. As far as web metrics quality is

concerned, in (Calero et al., 2005) some important

metrics proposed for web information systems are

classified.

3 PROBLEMS, MOTIVATION

AND GOALS

The main goal of this research is to lay the basis of a

QuEF that facilitates the quality assessment of

different methodological proposals under some

specific criteria. Today’s modern web information

systems are called to manage a huge amount of

information which is difficult to develop and

maintain. In this sense, there is a need for the

suitable design of MDWE methodologies and

effective tools. In this way, our work concentrates

on evaluating and comparing existing proposals.

One aspect that must be considered is the use of a

MDWE methodology and its influence on the final

product quality. Nowadays, it is important in the

software industry to produce faster, cheaper software

of higher quality.

4 DEFINING A QUALITY

EVALUATION FRAMEWORK

FOR MDWE

METHODOLOGIES

4.1 The Term “Methodology”

An approach or methodology is a Model-Driven

proposal for the development of web applications. It

may provide a set of guidelines, techniques,

processes and/or tools for the structuring of

specifications, which are expressed as models. Only

web modelling approaches which are based on MDA

in the framework are considered.

4.2 The Term “Framework”

The QuEF is a basic conceptual structure composed

of a set of elements used to evaluate MDWE

methodologies. Therefore, a QuEF with a set of

elements based on existing literature is proposed

where four components for the evaluation of the

quality of MDWE methodologies can be seen:

 Approach Characteristics Template: This

component would have the responsibility of

describing the input methodology

characteristics to be evaluated.

 Thesaurus & Glossary: This component

would be responsible for improving the

standardization of the access channel and

communication between users of different

MDWE methodologies.

 Quality Model: This component is responsible

for providing the basis for specifying quality

requirements with the purpose of evaluating

quality.

 Quality Evaluation Process: This component

would have the responsibility of carrying out

the quality evaluation process.

4.3 A QuEF for MDWE Methodologies

We present the steps for defining a QuEF for

MDWE methodologies.

1. Identifying quality factors and quality attributes

2. Identifying approach characteristics and

subcharacteristics that can affect the quality

attributes.

3. Specifying how to evaluate the

subcharacteristics.

WEBIST 2010 - 6th International Conference on Web Information Systems and Technologies

192

4. Specifying association links between the

subcharacteristics and the quality attributes.

5. Defining the Quality Evaluation Process.

Concepts, tasks to be performed for each step, and

framework structuring and components which result

for each step are described. The main component for

the QuEF is the Quality Model. As shown in Figure

1, a Quality Model is a set of characteristics,

subcharacteristics and metrics, quality factors,

quality attributes and the relationships between

them, which provides the basis for specifying quality

requirements and evaluating quality. In simple terms

all the stakeholders must be well-informed of what

is expected, what the subcharacteristics to be

achieved are, which impact should be achieved on

quality attributes, what the evaluation criteria are

and how these criteria can contribute towards

achieving the goal.

Qualtiy Factor

Quality Attribute

Characteristic

Subcharacteristic Metric

Base Metric

Aggregated Metric

Derived Metric

+param

1..*

+base metric

1..*

+subcharacteristic1..*

+quality attribute

1..*

Figure 1: Quality Model metamodel.

On the other hand, in the Approach Characteristics

Template component, a template is defined with

general approach characteristics and

subcharacteristics for each characteristic based on

the Quality Model. It is used to describe an input

methodology. This template would be used as input

to the QuEF.

Therefore, for our purposes, a Quality Model

contains a minimal amount of characteristics and

subcharacteristics through which any kind of

MDWE approach can be evaluated. In order to

define a Quality Model, it contains association links

between the subcharacteristics and the quality

attributes. These association links represent the

dependencies between subcharacteristics and

quality attributes. They show quality attributes

which are affected by subcharacteristics or the areas

of the methodology that will be significantly

affected if the approach is changed. Association

links may be based on proven and real-world

experience. Furthermore, subcharacteristics have to

define quantitative or qualitative metrics which may

be used to measure each subcharacteristic.

4.3.1 Identifying Quality Factors and

Quality Attributes

Examples of quality factors are maintainability,

usability or portability. In this sense, a set of quality

factors and quality attributes based on current

literature (for example, based on ISO/IEC 9126-1)

and adapted to MDWE methodologies have to be

identified, classified and hierarchical. Therefore, an

important element for the QuEF is the Thesaurus &

Glossary component. A thesaurus is a list containing

the "terms" used to represent concepts, themes or

contents of documents in order to make a

terminological standardization to improve the access

channel and communication between users of

different MDWE methodologies.

4.3.2 Identifying Approach Characteristics

and Subcharacteristics that can Affect

the Quality Attributes

A characteristic or subcharacteristic is a feature

assigned to a product, process or technique of a

methodology, and hence is generally a set of user

needs or expectations of a methodology. The

characteristic is a higher-level concept and the

subcharacteristic is a lower-level concept.

In MDWE, models are refined progressively and

transformed into new models or code with tools.

Moreover, each methodology may define its

development process and/or techniques.

The idea is to characterize the whole MDWE

process.

4.3.3 Specifying How to Evaluate the

Subcharacteristics

For each subcharacteristic, a specification of its

evaluation is necessary. For example, the evaluation

may be via measuring quantitatively by metrics or

subjective evaluation, inspections using checklists or

interviewing the users or designers.

TOWARDS A QUALITY EVALUATION FRAMEWORK FOR MODEL-DRIVEN WEB ENGINEERING

METHODOLOGIES

193

4.3.4 Specifying Association Links between

the Subcharacteristics and the Quality

Factors

In this step, the association links between

subcharacteristics and quality attributes have to be

defined. Association links indicate which quality

attribute is affected for each subcharacteristic.

4.3.5 Defining the Quality Evaluation

Process

A new component is defined, the Quality Evaluation

Process component, which contrasts the information

from each input Approach Template Characteristic

with information from the Quality Model. The idea

is to determine which aspect needs to be improved

on MDWE methodology. it may be necessary to

establish a decision criterion. The results provide an

assessment report of the methodology and this may

be used for contrasting with the evaluation of other

MDWE methodologies.

5 CONCLUSIONS & FUTURE

WORK

The bases of a Quality Evaluation Framework

(QuEF) for MDWE methodologies are proposed in

this paper. With regards to the contributions

obtained from this research we think that the use of a

QuEF would enhance the quality of products,

processes and techniques of MDWE approaches.

Therefore the use of a QuEF may improve the

efficiency and effectiveness of MDWE

methodologies, and in turn may make their use more

widespread. This framework would help designers to

ask the right questions and solve critical issues.

The steps for defining the QuEF have to be

followed. Besides, it would be necessary to carry

out a standardization of terminology to improve the

access channel for communication in MDWE. A

software prototype would be developed to put it all

in practise.

ACKNOWLEDGEMENTS

This research has been supported by the project

QSimTest (TIN2007-67843-C06_03) and by the

RePRIS project of the Ministerio de Educación y

Ciencia (TIN2007-30391-E), Spain.

REFERENCES

L. C. Briand, W. L. Melo. and J. Wüst. “Assessing the

Applicability of Fault-Proneness Models Across

Object-Oriented Software Projects”. ISERN Report

No. ISERN-00-06, V. 2, 2006.

C. Cachero, C. Calero, Y. Marhuenda, “A Quality-Aware

Engineering Process for Web Applications”.

Handbook of research on Web Information Systems

Quality. 2008.

C. Calero, J. Ruiz, M. Piattini, “Classifying web metrics

using the web quality model”. Vol. 29, No. 3, pp. 227-

248, 2005.

A. Etien and C. Rolland, “A Process for Generating

Fitness Measures”. (CAiSE 2005), LNCS 3520, pp.

227-292. 2005.

M. J. Escalona, G. Aragón, “NDT. A Model-Driven

Approach for Web Requirements”. IEEE Transactions

on software engineering, Vol. 34, No. 3, pp. 377-390,

2008.

M. J. Escalona, N. Koch, “Requirements Engineering for

Web Applications – A comparative study”. Journal of

Web Engineering. Vol. 2, No. 3, pp. 193-212, 2004.

F. García, M. Serrano, J. Cruz-Lemus, F. Ruiz, M. Piattini,

“Managing software process measurement: A

metamodel-based approach”, Information

Sciences.Vol. 177, No. 12, pp. 2570-2586, 2007.

IEEE Std 610.12-1990. IEEE Standard Glossary of

Software Engineering Terminology,

ISO- International Organization for Standardization,

ISO/IEC 9126-1, http://www.iso.org

C. Kroiβ, N. Koch, “UWE Metamodel and Profile, User

Guide and Reference”. Technical Report 0802.

Programming and Software Engineering Unit (PST),

Institute for Informatics. Ludwig-Maximilians-

Universität München, Germany, 2008.

P. Mohagheghi, V. Dehlen, “Developing a Quality

Framework for Model-Driven Engineering”. Models

in Software Engineering: Workshops and Symposia at

MoDELS 2007, pp. 275–286, 2008.

OMG: MDA. http://www.omg.org/mda/faq_mda.htm

J. M. Pérez, F. Ruiz, M. Piattini, “Model Driven

Engineering Aplicado a Business Process

Management”, Informe Técnico. UCLM-TSI-002,

2007.

J. Ralyté, X. Lamielle, N. Arni-Bloch, M. Lèonard, “A

Framework for Supporting Management in Distributed

Information Systems Development”. 2nd In.

Conference on Research Challenges in Information

Science (RCIS 2008), pp. 381-392, Morocco, 2008.

W. Schwinger,W. Retschitzegger, A. Schauerhuber, G.

Kappel, M. Wimmer, B. Pröll, C. Cachero Castro, S.

Casteleyn, O. De Troyer, P. Fraternali, I. Garrigos, F.

Garzotto, A. Ginige, G-J. Houben, N. Koch, N.

Moreno, O. Pastor, P. Paolini, V. Pelechano Ferragud,

G. Rossi, D. Schwabe, M. Tisi, A. Vallecillo, van der

Sluijs and G. Zhang, “A survey on web modeling

approaches for ubiquitous web applications”.

International Journal of web Information Systems Vol.

4 No. 3, pp. 234-305, 2008.

WEBIST 2010 - 6th International Conference on Web Information Systems and Technologies

194