Towards a Framework for KPI Evolution

Eladio Dom

ınguez

1 a

, Beatriz P

erez

2 b

Angel L. Rubio

2 c

and Mar

ıa A. Zapata

1 d

Dpto. de Inform

atica e Ingenier

ıa de Sistemas, Univ. de Zaragoza, Zaragoza, Spain

Dpto. de Matem

aticas y Computaci

on, Univ. de La Rioja, La Rioja, Spain

Keywords:

KPIs, Performance Measurement, Evolution Framework.

Abstract:

Key Performance Indicators (KPIs) are becoming essential elements for measuring business performance. In

recent years, KPIs management has been the subject to sustained interests for researchers and practitioners

alike, deriving into a large research corpus of approaches addressing aspects in matters as varied as modelling,

maintenance or expressiveness of KPIs. In particular, since both businesses and processes have to be adapted

to ever-changing requirements, the KPIs that measure their performance must evolve accordingly. However,

based on a previous review of the literature, we found that little attention has been paid to the provision of

mechanisms to manage KPIs evolution. Our long-term research goal is to provide a fully proposal for support-

ing KPIs evolution management. In this position paper, we present the ﬁrst ideas of a conceptual framework

for addressing this issue, proposing a pattern-driven KPI evolution speciﬁcation and a KPI evolution meta-

model made up of two interconnected views. Our proposal is general enough to be applied regardless of the

speciﬁc KPIs management approach being used.

1 INTRODUCTION

Today’s data society provides an opportunity for en-

terprises and institutions to access a wealth of infor-

mation of many different types. Since continuous

business improvement is a must-have goal for any or-

ganization, proper use of available information can

lead to changes in procedures, processes and systems.

Changes can be triggered by different causes, such as

modiﬁcations of the business strategy, correction of

detected errors, legislative updates or technological

adaptations (Cognini et al., 2018).

In order to determine the aspects that must be im-

proved, it is necessary to measure the performance of

business activities. That is the reason why the deﬁ-

nition and use of Key Performance Indicators (KPIs)

within information systems has become widespread

in any business area. In view of the relevance of

KPIs, it is natural that research on this topic is in-

tense and extensive. Despite this (or precisely be-

cause of it), there does not exist a standard model for

KPIs deﬁnition and use. In (Dom

ınguez et al., 2019)

we addressed this problem, approaching our solution

https://orcid.org/0000-0002-3270-899X

https://orcid.org/0000-0001-9235-7311

https://orcid.org/0000-0002-3052-6404

https://orcid.org/0000-0002-9531-1586

by performing a literature review and deﬁning a KPI

management taxonomy, which captures, in a uniﬁed

way, the overall properties of KPIs gathered in the

literature. This taxonomy is unfolded in ﬁve dimen-

sions, each one answering a different question: What

is measured by a KPI?, What features are consid-

ered?, What is a KPI measured for?, What artifacts

are used?, and What are the characteristics of each

approach? (for details, we refer to (Dom

ınguez et al.,

2019)).

One of the ﬁndings of this KPIs taxonomy is that

very few works or approaches consider KPIs cus-

tomization and evolution management as one of their

characteristics. If, as explained, any area of busi-

ness management needs to introduce changes to adapt

business to new circumstances, KPIs that measure the

performance (of the activities) of that area should also

be modiﬁed accordingly.

Making use of the terminology that is presented

in (Reichert and Weber, 2012), in the context of

Process-Aware Information Systems we could speak

of KPI ﬂexibility to refer to any aspect related to mod-

iﬁcations that can be considered in performance in-

dicators. Reichert et al. (Reichert and Weber, 2012)

differentiate between four different variants for the

notion of process ﬂexibility: variability (implies the

existence of different process variants); looseness

Domínguez, E., Pérez, B., Rubio, Á. and Zapata, M.

Towards a Framework for KPI Evolution.

DOI: 10.5220/0009465604630469

In Proceedings of the 15th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2020), pages 463-469

ISBN: 978-989-758-421-3

463

(processes are not fully pre-speciﬁed); adaptation

(process instances are adapted to cope with emerg-

ing events); and evolution (the capacity of modify-

ing a process permanently to accommodate evolving

needs).

Although all these variants could be transferred to

the KPIs domain, we focus on the speciﬁc issue of

KPIs evolution. In view of the large number of dif-

ferent approaches to KPIs management, and that few

of them consider KPI evolution aspects (Dom

ınguez

et al., 2019), our long-term research goal is to build

on a KPI evolution framework that can be used with

whatever particular KPIs approach. This is certainly a

complex aim, as evolution can be considered on many

levels. On the one hand, and from a business per-

spective, it is necessary to determine what speciﬁc

changes in KPIs need to be considered in order to

achieve business goals. On the other hand, from the

software engineering prism, it is necessary to have an

infrastructure that allows the transfer of those changes

(ﬁxed at the business level) to an implementable con-

ceptual model. Therefore, we intend to deal with KPI

evolution in a conceptual comprehensive manner but,

at the same time, bearing in mind that the implemen-

tation must be achievable. In this position paper we

present some aspects of a framework that allows the

development of conceptual models of KPI evolution.

To achieve this goal, we rely on a notion of KPI evolu-

tion pattern and on a KPI evolution metamodel which

includes both a structural and an execution view.

This paper is organized as follows. Next, we out-

line the related work of this research. In Section 3, we

describe our approach for KPIs evolution, illustrating

it by means of two examples of application. Finally,

conclusions and further work are set out in Section 4.

2 RELATED WORK

We start by considering works that deal with KPIs

evolution, and end up with works that address evo-

lution in a general way, not linked to a speciﬁc scope.

One relevant proposal in the literature that high-

lights the need to consider evolution in performance

measurement is (Kennerley and Neely, 2002). How-

ever, this work, on the one hand, focuses on the fac-

tors that affect that evolution and, on the other hand,

raises the discussion at the level of measurement sys-

tems, and does not descend to the detail of speciﬁc

indicators. In this sense, the notion of measure redef-

inition is considered in (Sch

utz and Schreﬂ, 2014).

However, this approach is somewhat limited, as it

considers that the different elements of a measure (di-

mensions and rule of calculation) are simply overrid-

den when redeﬁnitions are included. One step further,

Diamantini et al. (Diamantini et al., 2016) consider

the effects of classic create/update/delete operations

of KPIs and, in particular, point to the need of consis-

tency checking on the set of indicators. In a slightly

different line, authors of (Friedenstab et al., 2012)

present the abstract syntax of a speciﬁcation language

of measures that includes connections with other re-

lated elements (processes and dashboards). These

connections enable the traceability of the modiﬁca-

tions that have been made. As another example, del-

ıo-Ortega et al.’s work (del-R

ıo-Ortega et al., 2013)

deﬁnes indicators called Process Performance Indi-

cators (PPIs), which are directly linked to business

processes, connecting the evolution of the former to

the changes made to the latter. The same authors

in (Estrada-Torres et al., 2016) address the question

of variability in PPIs.

The interest in considering the evolution of KPIs

does not only occur in research articles, but also

in concrete practical applications. For example, the

Health Information and Quality Authority states in its

report (HIQA, 2013) that “a plan to review the KPI at

regular intervals with a view to reﬁnement in response

to stakeholder demands or improved data availabil-

ity” is necessary. This report also asserts that, since

“health services are continually evolving”, it is fun-

damental that KPIs give answers to these changes. In

an even more general context, IBM introduces the no-

tion of KPI history (IBM Knowledge Center, 2011) in

the description of its tool Business Process Manager.

This is “an optional feature that you can use to track

and analyze KPI changes over time”.

With regard to works that address evolution in

software engineering or information systems contexts

in a general way, most of them are based on models

and metamodels (Cicchetti et al., 2012; Dom

ınguez

et al., 2011; Pons et al., 2000; Ruiz et al., 2013).

These strategies mainly focus on aspects such as

structuring issues of evolution (Pons et al., 2000),

tracing aspects (Dom

ınguez et al., 2011; Ruiz et al.,

2013), or comparing and merging of models’ version-

ing (Cicchetti et al., 2012). In the case of KPIs evolu-

tion, conceptual modeling can also be used as a de-

velopment strategy. But, as far as we know, there

is no KPI evolution metamodel in the literature. We

particularly claim that conceptual modeling in KPIs

evolution could be considered manifold. For exam-

ple, it can provide methods for gaining better under-

standing not only of the very complex relationships

between the elements directly involved in the evolu-

tion of KPIs, but also of the implications such changes

may carry (such as side effects a change in a KPI may

have on other business elements or on related KPIs).

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

464

Another aspect considered when tackling business

activities behaviour is to take advantage of the use

of patterns (Ruiz et al., 2013). Patterns are recog-

nized to be a very useful construct to capture recur-

ring concepts within a particular domain. Aimed at

being generic and reusable in a wide variety of scenar-

ios, they are speciﬁed in a language- and technology-

independent way (Russell et al., 2016). But again, we

are not aware of any pattern devoted speciﬁcally to

KPIs evolution.

3 SUPPORTING KPI EVOLUTION

Aimed at giving a proposal for supporting KPIs evo-

lution, we have slightly inspired in the framework

for business processes model evolution presented

by (Ruiz et al., 2013). Speciﬁcally, in (Ruiz et al.,

2013), authors propose a pattern-driven speciﬁcation

for models evolution by means of two artefacts: a pat-

tern deﬁnition metamodel (deﬁned based on the ideas

from (Pﬁster et al., 2011)) and an evolution meta-

model (for which they based on (Bernstein, 2003)).

Starting from these ideas, in this paper, we contribute

with a ﬁne grained support for evolution aspects ap-

plied to the particular case of KPIs deﬁnition. More

speciﬁcally, we have deﬁned a KPI evolution meta-

model which includes two complementary views (a

structural view and an execution view), which are

built upon concrete KPI evolution patterns provid-

ing a semantic description of different KPI changes.

Next, we describe in detail these two main parts of

our approach.

3.1 KPI Evolution Patterns

Our main goal for deﬁning KPI evolution patterns is

to capture commonly occurring changes in KPIs, so

that they can be applied to different KPIs manage-

ment proposals. In order to determine our pattern-

driven speciﬁcation for KPIs evolution, we have con-

sidered, not only different change pattern speciﬁca-

tion proposals (Mulyar et al., 2008; Ruiz et al., 2013;

Reichert and Weber, 2012), but also the taxonomy for

KPIs management we proposed in (Dom

ınguez et al.,

2019) which includes KPIs evolution aspects (del-

ıo-Ortega et al., 2013; Diamantini et al., 2016;

Sch

utz et al., 2016).

As for the former works (Mulyar et al., 2008;

Ruiz et al., 2013; Reichert and Weber, 2012), all con-

sider several pattern speciﬁcation elements, which are

a name, an abbreviation, a description of its func-

tionality, and illustrating examples to clarify the pat-

tern’s use. Besides these aspects, as (Mulyar et al.,

2008) does, we have considered it appropriate to in-

clude a motivation aimed at identifying the nature of

the modiﬁcations in the environment which can trig-

ger the change. As a way of example, in Table 1 we

show a KPI evolution pattern, named Update Calcu-

lation Rule (UCR), describing the change of the cal-

culation rule of a KPI. Several circumstances can mo-

tivate the need to make this type of change, from a

revision in the company’s policies to a simple correc-

tion in the deﬁnition of a KPI. For example, in Table 1

two situations exemplifying the use of this pattern are

mentioned. One of them is related to the educational

context and gathers a modiﬁcation in the evaluation

criteria of an on-line course. The change is intended

to penalise the number of wrong answers given by the

students, so that KPI ‘Session Learning Adequacy per

Training Step’ (SLATS) (Calabro et al., 2015) calcu-

lated as the sum of the ‘Correctness Score per Train-

ing Step’ (CSTS) and ‘Reload Cardinality per Train-

ing Step’ (RCTS), is changed applying a factor of 1.5

to the number of reloads (CSTS + 1.5*RCTS). The

other example is associated to sustainability and en-

vironmental standards which are changed as new reg-

ulations are established and environmental research

evolves. For instance, the G4 Sustainability Report-

ing Guidelines (Global Reporting Initiative, 2015)

were superseded by the GRI Sustainability Reporting

Standards (GRI Standards) (Global Reporting Initia-

tive, 2016). Several changes are introduced by GRI

standards, among them, as an illustrative example for

this article, we will use the fact that instead of calcu-

lating only the total weight of waste, two indicators

are introduced in GRI standards for calculating sepa-

rately hazardous and non-hazardous waste. As a con-

sequence, the calculation rule of the total waste indi-

cator is changed to be transformed into the sum of the

two new indicators (total waste = hazardous waste +

non-hazardous waste).

As for aspects speciﬁcally related to KPIs evolu-

tion, we have based on the three different character-

istics (modiﬁcation, traceability and change propa-

gation) involved in KPIs evolution established in the

taxonomy we presented in (Dom

ınguez et al., 2019).

While the ﬁrst one, modiﬁcation, is intrinsically re-

lated with the KPIs modiﬁcation itself, the other two,

traceability and propagation, could be considered or

not. Due to this optionality and based on an idea

proposed in (Reichert and Weber, 2012), we have in-

cluded in the patterns speciﬁcation a design choices

section (see Table 1). Different implementation op-

tions for the traceability and propagation aspects can

be speciﬁed in this section.

Traceability helps to maintain information about

the different business elements related with the KPIs

Towards a Framework for KPI Evolution

465

Table 1: Example of a KPI evolution pattern.

Pattern UCR: Update Calculation Rule

Description

The calculation rule of a KPI is updated

Examples

- A modification in the evaluation criteria of an on-line course, penalizing the number of wrong answers,

can imply a change in the KPI "Session Learning Adequacy per Training Step” (SLATS) (Calabro et al.,

2015) adding, for example, a factor of 1.5 to the number of reloads.

- A company supersedes the G4 Sustainability Guidelines (Global Reporting Initiative, 2015) for the GRI

Standards (Global Reporting Initiative, 2016), so that the changes introduced in the GRI Standards must

be considered in the calculation of KPIs.

Motivation

Several reasons can motivate a change in the calculation rule of a KPI, for example, a change in the goals of

the company, a change in the business processes or a change in other KPIs involved in the calculation rule.

Design

choices

Traceability:

- To store or not the information related with the change that provokes the calculation rule modification

(goal, process, KPI, …)

- The previous calculation rule is deleted or stored in order to know the evolution over time.

- The previous values of the KPI are deleted or stored together with the previous calculation rule.

Propagation:

- The relationships between KPIs must be updated according to the new calculation rule and the

calculation rules of other related KPIs must be rewritten accordingly.

- Other KPI attributes, such as, value type, target or status options, must be analyzed to decide if they

should be modified

Consistency

conditions

Properties such as identity, equivalence and consistency among the KPIs must be evaluated (Diamantini et

al., 2016). If the consistency is not preserved, then the modification will not take place.

patterns

Influenced by: Create KPI (Pattern CK), Delete KPI (Pattern DK), Update Calculation Rule (Pattern UCR)

Influences: Update Target (Pattern UT), Update Calculation Rule (Pattern UCR)

(goals, processes or the KPIs themselves) and to keep

a historical trace between the different evolution ver-

sions. For example, the relationship between Pro-

cess Performance Indicators (PPIs) and business pro-

cess elements is considered in (del-R

ıo-Ortega et al.,

2013). As for Pattern UCR, this type of relationship

can allow us to trace the processes whose changes

provoke a modiﬁcation in the calculation rule of KPIs.

Propagation, for its part, is related with the incre-

mental update mechanisms necessary for maintaining

coherence among the different involved elements of

KPIs. For example, redeﬁnition of calculated mea-

sures is proposed in (Sch

utz et al., 2016) as conse-

quence of customization processes. As another ex-

ample, a change in the calculation rule of a KPI may

involve the modiﬁcation of its target value.

Other pattern speciﬁcation aspects considered

by (Mulyar et al., 2008) are the issues related with

problems potentially encountered when using the pat-

tern, and solutions for overcoming these problems. In

the case of KPIs evolution, we consider it necessary to

specify the consistency conditions that need to be ver-

iﬁed and that can be compromised by an inadequate

change, as well as, the actions to be performed when a

change does not preserve these conditions. For exam-

ple, conditions of identity, equivalence and coherence

among calculation rules are considered in (Diaman-

tini et al., 2016) together with reasoning tools to fa-

cilitate KPIs management.

Finally, to make it explicit the relations of prop-

agation between patterns, a related patterns aspect

is speciﬁed (also included in (Reichert and Weber,

2012)), indicating the patterns inﬂuenced by the pat-

tern that is being deﬁned (inﬂuences) and the patterns

by which it is inﬂuenced (inﬂuenced by). For exam-

ple, as we show in Table 1, the Pattern UCR inﬂu-

ences the Update Target pattern (or Pattern UT) since,

as said before, a change in the calculation rule of a

KPI may involve the modiﬁcation of its target value.

Besides, the Pattern UCR is inﬂuenced by the Create

KPI pattern (or Pattern CK) and the Delete KPI pat-

tern (or Pattern DK) since the creation or deletion of

KPIs can imply the redeﬁnition of a calculation rule,

as shown with the waste indicator example. These

other patterns are described in the supplementary ma-

terial (Supplementary material, 2020). Additionally,

as presented in Table 1, the application of Pattern

UCR can itself inﬂuence the application of the same

pattern to another KPIs (thus, the pattern can be inﬂu-

enced by itself).

3.2 KPI Evolution Metamodel

As previously stated, our KPI evolution metamodel

includes two complementary views (a structural view

and an execution view), which are built upon our KPI

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

466

Pattern

name

abbreviation

description

example[*]

motivation

1..*

influences

1..*

TraceOperation

AppliedTraceOperation

0..1

AS_IS

TO_BE

AppliedConsistencyCondition

1..*

Legend

Traceability aspects

Propagation aspects

KPIInvolvedElement

Modification aspects

PropagationOperation

affects

1..*

ConsistencyCondition

0..*

influences

Consistency aspects

AppliedKPIInvolvedElement

influencedBy

Structural view

Execution view

PatternApplication

commitDate

commiter

comment

AppliedPropagationOperation

name

Figure 1: KPI evolution metamodel.

evolution patterns proposal. In Figure 1, we show

these two views and the way they are interconnected.

While the structural view deals with the structural as-

pects of the evolution as deﬁned by the KPI evolu-

tion patterns (see top of Figure 1), the execution view

concerns the application of such patterns, registering

mainly the way in which concrete KPI instances un-

dergo modiﬁcations as patterns are instantiated (see

bottom of Figure 1). Both views reﬂect the differ-

ent key elements identiﬁed by our KPI evolution pat-

terns proposal, but from different perspectives: struc-

tural deﬁnition and applications’ instances, respec-

tively. More speciﬁcally, in addition to specify KPIs

changes themselves (modiﬁcation aspects), the main

idea behind both views is to cope with the remainder

key evolution aspects as identiﬁed by our patterns’

structure (that is, not only traceability and change

propagation issues, but also consistency conditions,

and inﬂuences/inﬂuenced by patterns’ associations).

Before going on to describe each view, we would

like to remark that, trying to be as general as pos-

sible, we will explain our evolution metamodel us-

ing the terminology of the taxonomy we presented

in (Dom

ınguez et al., 2019). However, it must be

noted that the evolution metamodel can be used with

whatever external KPIs management approach (such

as the PPINOT metamodel (del-R

ıo-Ortega et al.,

2013) or the KPIOnto ontology (Diamantini et al.,

2016)). For example, in the case of the Pattern UCR

of Table 1, the KPIInvolvedElement would corre-

spond to ‘calculation rule’ according to (Dom

ınguez

et al., 2019), but it would be referred as ’measureDeﬁ-

nition’, if the PPINOT metamodel is considered, or as

‘formula’, if the KPIOnto ontology is chosen. That is,

the entities involved in the evolution of a KPI would

correspond to concrete elements of the chosen KPI

management approach (a metamodel, an ontology, a

taxonomy, etc.).

Structural View. As for the structural view (see top

of Figure 1), it includes a Pattern metaclass with

the basic deﬁnition information of each KPI evolution

pattern (such as the name, abbreviation, etc.). The en-

tities concerned by a KPI modiﬁcation are represented

by the KPIInvolvedElement metaclass. Thus, the

KPI entities involved in a KPI evolution pattern are

represented in the metamodel by the association be-

tween the Pattern and KPIInvolvedElement meta-

classes. At the same time, these entities can be ele-

ments directly or indirectly involved in the KPI evolu-

tion pattern (Ralyt

e and L

eonard, 2017). On the one

hand, the entities directly concerned by a KPI modi-

ﬁcation are those elements explicitly involved in the

Towards a Framework for KPI Evolution

467

KPI pattern (for example, the calculation rule in the

Pattern UCR). On the other hand, we refer to the enti-

ties indirectly concerned by a KPI modiﬁcation as any

other element not explicitly involved in the KPI evo-

lution pattern (for example, the target of a KPI in the

Pattern UCR).

Traceability or consistency aspects regarding ei-

ther direct or indirect entities, are represented by

the association metaclasses TraceOperation and

ConsistencyCondition, respectively, between the

Pattern and KPIInvolvedElement metaclasses (see

Figure 1). For example, if we have decided to ap-

ply the Pattern UCR so that it traces previous val-

ues of modiﬁed calculation rules, an instance of the

TraceOperation is created, linking the Pattern in-

stance corresponding to the Pattern UCR and the

KPIInvolvedElement instance with name ‘calcula-

tion rule’. This allows us to trace any evolution

aspect of a KPIInvolvedElement as consequence

of the application of a pattern. Similarly, linked

with a KPIInvolvedElement instance named ‘KPI’,

a ConsistencyCondition instance would be created

to show consistency requirements with other KPIs, as

deﬁned by the Pattern UCR. Propagation aspects are

represented by the PropagationOperation meta-

class which registers the side effects of the evolu-

tion pattern on other related entities. This fact is

represented in Figure 1 by the influencedBy and

influences association roles.

Execution View. Regarding the execution view

(see bottom of Figure 1), as described previously,

it represents aspects regarding the application of

evolution patterns as established in the structural

view. In particular, this view is linked with the pre-

vious one by means of the relationship between the

Pattern metaclass and the PatternApplication

metaclass. Each time a pattern is applied, a

PatternApplication instance is created which is

linked with the corresponding Pattern instance.

The PatternApplication metaclass includes

concrete attributes (such as the commitDate, with

the moment in which the pattern’s application takes

place, the committer, and any remarkable com-

ment she/he considers necessary to make). More

speciﬁcally, as authors suggest in (Cicchetti et al.,

2012), we advocate to represent not only “who”

performs the change (in contrast to (Ruiz et al.,

2013) which does not consider change’s responsibili-

ties), but also “when” such changes are performed.

This view mainly deﬁnes entities representing

application instances of the ones deﬁned in the

structural view (AppliedPropagationOperation,

AppliedTraceOperation and AppliedConsis-

tencyCondition, together with the AppliedKPI-

InvolvedElement). In particular, as an evolution

pattern is applied, this view registers concrete

evolution traces from a source element version

(AS IS) to another target version (TO BE) of

such an element (both instances of the external

AppliedKPIInvolvedElement).

As a way of example, we use the SLATS KPI (Cal-

abro et al., 2015), included in Table 1 to exemplify

our patterns approach. As we have said before, lets

suppose that we want to change the calculation rule

of this KPI, currently given by ‘Correctness Score per

Training Step’ (CSTS) plus ‘Reload Cardinality per

Training Step’ (RCTS), and we proceed to apply the

Pattern UCR. Before executing this pattern, the corre-

sponding consistency conditions are checked to make

sure that the change will not contradict any previously

deﬁned KPI. If the result is afﬁrmative, an instance

of the AppliedConsistencyCondition is registered

per any checked KPI. Thus, the pattern’s application

takes place, evolving the KPI’s calculation rule from

the current calculation rule instance (AS IS), given

by CSTS + RCTS, to the evolved calculation rule in-

stance (TO BE) represented by CSTS + 1.5*RCTS. As

for propagation aspects, they are present in the waste

indicator example where, in particular, the redeﬁni-

tion of the calculation rule of the waste indicator is

inﬂuenced by the creation of the two new indicators

(‘hazardous waste’ and ‘non-hazardous waste’). In

this case, the propagation task would result in two

instances of the AppliedPropagationOperation

metaclass, each one linked with an instance of the

PatternApplication metaclass corresponding to

an application of the Pattern CK (inﬂuences role),

and on the other hand, with the instance of the

PatternApplication metaclass related to the appli-

cation of the Pattern UCR (inﬂuencedBy role).

4 CONCLUSIONS AND FUTURE

WORK

This paper presents our research proposal to deﬁne

and develop mechanisms for supporting KPIs evolu-

tion management. In particular, we propose to sup-

port KPIs evolution by means of a KPI evolution pat-

tern notion and a KPI evolution metamodel, made up

of two interconnected views. The goal of the long-

term research we are developing is to provide a gen-

eral framework for dealing with KPIs evolution.

In order to develop this general framework, sev-

eral lines for further research must be addressed.

Firstly, the conceptual proposal presented in this pa-

per must be applied to real scenarios analyzing its

advantages and detecting improvements that may be

ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering

468

incorporated. In addition, the integration of our

proposal within other KPIs management approaches

(such as the PPINOT metamodel (del-R

ıo-Ortega

et al., 2013) or the KPIOnto ontology (Diamantini

et al., 2016)) must be studied, determining the way

in which the elements of our proposal are embedded

properly in such proposals. Finally, mechanisms for

giving automatic support to our overall approach must

be proposed, based on the possibility of improving

tools already developed for the management of KPIs.

ACKNOWLEDGEMENTS

This research was funded by the Spanish Ministry

of Economy and Competitiveness, project number

EDU2016-79838-P.

REFERENCES

Bernstein, P. A. (2003). Applying Model Management to

Classical Meta Data Problems. In Proc. of the 1st

Biennial Conference on Innovative Data Systems Re-

search (CIDR), pages 209–220.

Calabro, A., Lonetti, F., and Marchetti, E. (2015). KPI eval-

uation of the business process execution through event

monitoring activity. In Proc. of the International Con-

ference on Enterprise Systems (ES), pages 169–176.

Cicchetti, A., Ciccozzi, F., and Leveque, T. (2012). A so-

lution for concurrent versioning of metamodels and

models. J. object technol., 11:1:1–32.

Cognini, R., Corradini, F., Gnesi, S., Polini, A., and Re,

B. (2018). Business process ﬂexibility-a systematic

literature review with a software systems perspective.

Inform. Syst. Front., 20(2):343–371.

del-R

ıo-Ortega, A., Resinas, M., Cabanillas, C., and Cort

es,

A. R. (2013). On the deﬁnition and design-time anal-

ysis of process performance indicators. Inf. Syst.,

38(4):470–490.

Diamantini, C., Potena, D., and Storti, E. (2016). SemPI:

A semantic framework for the collaborative construc-

tion and maintenance of a shared dictionary of perfor-

mance indicators. Future Gener. Comp. Sy., 54:352 –

365.

Dom

ınguez, E., Lloret, J., P

erez, B., Rodr

ıguez, A., Rubio,

A. L., and Zapata, M. A. (2011). Evolution of XML

schemas and documents from stereotyped UML class

models: A traceable approach. Inf. Softw. Technol.,

53(1):34–50.

Dom

ınguez, E., P

erez, B., Rubio,

A. L., and Zapata, M. A.

(2019). A taxonomy for key performance indicators

management. Comput. Stand. Inter., 64:24–40.

Estrada-Torres, B., del R

ıo-Ortega, A., Resinas, M., and

Ruiz-Cort

es, A. (2016). Identifying variability in pro-

cess performance indicators. In Proc. of the Business

Process Management (BPM) Forum, pages 91–107.

Friedenstab, J.-P., Janiesch, C., Matzner, M., and Muller,

O. (2012). Extending BPMN for Business Activity

Monitoring. In Proc. of the 45th Hawaii International

Conference on System Sciences (HICSS), pages 4158–

4167.

Global Reporting Initiative (2015). G4 Sustain-

ability Reporting Guidelines. Online at

https://www2.globalreporting.org/standards/g4/

Pages/default.aspx. Accessed on January 2020.

Global Reporting Initiative (2016). GRI Sus-

tainability Reporting Standards. Online at

https://www.globalreporting.org. Accessed on

January 2020.

HIQA (2013). Guidance on developing key performance

indicators (KPIs) and minimum data sets to moni-

tor healthcare quality version 1.1. Technical report,

Health Information and Quality Authority.

IBM Knowledge Center (2011). KPI history. Online

at https://www.ibm.com/support/knowledgecen

ter/SSFTDH 7.5.0/com.ibm.wbpm.mon.widget.

doc/topics/kpihist ov.html. Accessed on January

2020.

Kennerley, M. and Neely, A. (2002). A framework of

the factors affecting the evolution of performance

measurement systems. Int. J. Oper. Prod. Man.,

22(11):1222–1245.

Mulyar, N., van der Aalst, W. M., and Russell, N. (2008).

Process ﬂexibility patterns. Technische Universiteit

Eindhoven.

Pﬁster, F., Chapurlat, V., Huchard, M., and Nebut, C.

(2011). A design pattern meta model for systems engi-

neering. In Proc. of the 18th World Congress of the In-

ternational Federation of Automatic Control (IFAC).

Pons, C., Baum, G., and Kutsche, R.-D. (2000). Model

evolution and system evolution. J. Ccomput. Sci. Tech-

Ch., 1(3):18.

Ralyt

e, J. and L

eonard, M. (2017). Evolution Models for

Information Systems Evolution Steering. In Proc. of

the 10th IFIP Working Conference on The Practice of

Enterprise Modeling (PoEM), pages 221–235.

Reichert, M. and Weber, B. (2012). Enabling Flexibility

in Process-Aware Information Systems - Challenges,

Methods, Technologies. Springer.

Ruiz, M., Espa

na, S., Pastor, O., and Gonz

alez, A.

(2013). Supporting organisational evolution by means

of model-driven reengineering frameworks. In Proc.

of the 7th International Conference on Research Chal-

lenges in Information Science (RCIS), pages 1–10.

Russell, N., van der Aalst, W. M., and ter Hofstede, A. H.

(2016). Workﬂow patterns: the deﬁnitive guide. MIT

Press.

Sch

utz, C. and Schreﬂ, M. (2014). Customization of

Domain-Speciﬁc Reference Models for Data Ware-

houses. In Proc. of the IEEE 18th International

Enterprise Distributed Object Computing Conference

(EDOC), pages 61–70.

Sch

utz, C. G., Neumayr, B., Schreﬂ, M., and Neub

ock,

T. (2016). Reference Modeling for Data Analy-

sis: The BIRD Approach. Int. J. Coop. Inf. Syst.,

25(02):1650006:1–1650006:46.

Supplementary material (2020). KPI evolution pat-

terns. Online at https://zenodo.org/record/3628668.

Accessed on January 2020.

Towards a Framework for KPI Evolution

469