Process Mining and Performance Business Rules

Ella Roubtsova

and Yoeri Berk

Open University of the Netherlands, Department of Information Science, Heerlen, The Netherlands

Keywords:

Process Mining, Performance Business Rules, Interview, Methodology, Application Process of an Employ-

ment Agency.

Abstract:

Process mining uses event logs to build a business process model, which potentially can be used to better

understand the business process performance. However, the understanding of performance almost always

demands deﬁnitions in form of performance business rules within the business process. This means that

process mining should be accompanied with another research method to retrieve the performance business

rules of the studied business process. This paper presents an approach showing how the process mining

techniques shape the structure of the semi-structured interviews for retrieving performance business rules to

be checked on the logged business process. The proposed approach is illustrated with a case study of process

mining and performance assessment of an application process in an employment agency.

1 INTRODUCTION

Some modern research methods based on data

analysis are not completely self-sufﬁcient when

they should serve the needs of business (Williams,

2016),(Dabrowski et al., 2017). One of such busi-

ness needs is the performance analysis of business

processes. Ghasemi and Amyot (2020) in their review

of process mining research papers has found that “re-

search about performance indicators measuring goals

associated with process mining is sparse”(Ghasemi

and Amyot, 2020).

Our research has been inspired by the need for

performance analysis at an employment agency. An

application process at an employment agency deter-

mines a candidate-pool that is proposed by the agency

to the client-company, looking for employees. The

correspondence of the candidates to the predeﬁned

qualities and the speed of delivery of the candidate-

pool inﬂuence the added value, reputation of the em-

ployment agency and the next requests of clients. The

move to online job hunting has increased the reach

of job postings and thus the number of applications.

Each application has own process case, because its

applicant may forget to send a necessary document

that creates a process loop, or may be rejected at dif-

ferent stages. With this high number of applications

and the large variety of process cases, the analyzing

process performance is challenging.

https://orcid.org/0000-0002-4067-3088

Process mining is a technique that is aimed to

extract process-related knowledge (e.g., process mo-

dels) from event logs and exploit it for further ana-

lysis (Cheng and Kumar, 2015). Initial application

of the process mining tool Disco (DISCO, 2022) to a

log of agency (Berk, 2021) reveals, for example, that

for one client, 1, 472 applications resulted in 224 dif-

ferent process cases. Inspired with this process at an

employment agency we have investigated the follow-

ing research question: What research method should

accompany process mining for performance analysis

of a business process with many process cases?

Section 2 presents the results of our search of

an answer in literature. Section 3 describes a struc-

ture of a semi-structured interview for gaining per-

formance business rules. The interview has been de-

signed based on ﬁlters chosen in process mining tools.

Section 4 discusses a case study applying a semi-

structured interview with the designed structure, re-

trieving performance business rules and using them

for process mining and performance assessment. Sec-

tion 5 presents conclusions and future work.

2 PERFORMANCE AND

PROCESS MINING

The deﬁnition of performance found in literature have

two parts.

Roubtsova, E. and Berk, Y.

Process Mining and Performance Business Rules.

DOI: 10.5220/0011056300003176

In Proceedings of the 17th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2022), pages 387-394

ISBN: 978-989-758-568-5; ISSN: 2184-4895

387

• First, performance is often deﬁned as the achieve-

ment of quantiﬁed objectives. It is often called

effectiveness, i.e.“delivering desired outputs, and

even outcomes” (Ghalem et al., 2016, page 7).

• The second deﬁnition of performance includes not

only a matter of what people achieve, but also

how they are achieving it; how they are using

the available knowledge and resources; how they

are choosing the shortest(cheapest) process path

of the business process. The second deﬁnition is

called efﬁciency (Ghalem et al., 2016, page 7).

In this work we follow (Ghalem et al., 2016, page 9),

who “believe that performance is the combination of

these two terms (effectiveness and efﬁciency), how-

ever, and depending on the context in which the per-

formance might be used, other elements can be added

to deﬁne the term, such as relevance, economy, etc.”

If we deﬁne an event as a data representation

of an occurrence of interest in the real-world busi-

ness domain (McNeile and Simons, 2006); if we

deﬁne a business process as a structure of ordered

events (Roubtsova and Wiersma, 2018), then

• the effectiveness can be seen as reachability of de-

sired events in the process;

• the efﬁciency, can be seen as desired sub-

sequences of events or orders of events.

A log of events corresponds to the deﬁnition of a pro-

cess as a a structure of ordered events and the two

performance elements can be related to a log. “A log

is a textual ﬁle containing information about events

recorded in the order of happening.” (Roubtsova and

Wiersma, 2018). Indeed, a log stores the ordered

events using the time-stamps. A log describes each

event using a log record that contains case designation

(process instance), activity(event) label, time stamp,

and also resource executing event or used for execu-

tion and data elements recorded with the event (e.g.,

the size of an order, the reason of withdrawal. etc.).

On the other hand, a log is the input for process

mining techniques. The question is whether there is a

process mining type that can sufﬁciently serve to as-

sessment of effectiveness and efﬁciency of the mined

business process.

The three main types of process mining are pro-

cess discovery, conformance checking, and process

model enhancement (Lamghari et al., 2019).

In process discovery, no prior process-related in-

formation is used, no quantiﬁed objectives are given.

The business process discovered by mining the event

log is analyzed to see some degree of effectiveness:

What outputs are delivered? It is difﬁcult to analyze

the efﬁciency of the mined process. The analysis may

show that some cases of the process have more steps,

but it remains an observation as the analyst does not

know what is efﬁcient for the mined process.

In conformance checking, an existing process

model is compared with an event log of the same pro-

cess. The results of conformance checking present

some descriptive elements of efﬁciency and effective-

ness: what events are often skipped, what resources

are scars. If the existing process model is deﬁned

by or transformed to business rules that deﬁne de-

sired outputs, desired sequences of process steps, the

conformance checking can be used for analysis of

effectiveness and efﬁciency. The practical problem

is that, the desired process is often presented as a

“happy ﬂow”, i.e. an ideal process that does not

cover exceptions and the business rules are not do-

cumented (Berk, 2021). More over, to analyze efﬁ-

ciency, the desired time and resource restrictions have

to be added to existing process.

In process model enhancement, there is a cur-

rent (a-priori) process model (as a goal) and a pro-

cess mining of an event log. The process mining is

used to ﬁnd possibilities to improve the mined busi-

ness process. Process model enhancement aims to

change or extend the a-priori model (van der Aalst,

2016). Additional requirements are used to extend the

model. They may introduce new outputs. i.e. change

model effectiveness. They may introduce new possi-

ble events, time and resource restrictions, i.e. change

the process efﬁciency. Any model change should have

its own goal that can be expressed in terms of perfor-

mance. This means that the process model enhance-

ment type of process mining is related to process per-

formance. As we see, this process mining type needs

some methods that provide the goal, requirements and

business rules for process model enhancement.

The concept of an organizational “performance”

can be seen as the degree to which the organization

meets its objectives (Hong, 2016). The objectives

consist of three elements: targets (outcomes) to be

reached, ways to get targets (outcomes) and process

time (van den Ingh, 2016). In order to measure these

elements of performance, business rules and perfor-

mance indicators are usually deﬁned. Business rules

are often related to the execution order of tasks in

cases, to the involvement of a role in cases and pro-

cesses. The performance indicators are related to the

business value (costs, time) achieved in cases of the

analyzed process.

For example, the Performance Business Rule

“CV Received comes before Candidate Proposed’’ is

about the execution order of the process. ‘‘CV Re-

ceived” and “Candidate Proposed” are two events on

the business process. Another example, the Perfor-

mance Business Rule “Every handling of an applica-

ENASE 2022 - 17th International Conference on Evaluation of Novel Approaches to Software Engineering

388

tion should take less than 5 days” speciﬁes the desired

time of the process “Handling of an application”. The

time of a speciﬁc case of the process “Handling of an

application” can exceed 5 days and violate this per-

formance business rule.

There are goal-driven methods for applying pro-

cess mining to performance. The process mining

project methodology PM

(van Eck, M. L., Lu, X.,

Leemans, S. J. J., and van der Aalst, W. M. P., 2015)

uses the ﬁrst iteration of process mining to obtain ge-

neral insights into the process. After that the goal and

business rules are formulated. The PM

(van Eck, M.

L., Lu, X., Leemans, S. J. J., and van der Aalst, W. M.

P., 2015) methodology provides a solution for plan-

ning and management of process mining. However,

the methodology does not explain how business rules

are retrieved from stakeholders and how the process

mining ﬁlters and techniques are selected.

Therefore, we propose to extend the methodology

with semi-structured interviews of process stakehold-

ers. We propose a reusable structure of such an in-

terview inﬂuenced by ﬁlters of process mining tools

that can be mapped on business rules. The interview

is aimed to collect the performance business rules that

can be directly reformulated into process mining ﬁl-

ters.

3 COLLECTING PERFORMANCE

BUSINESS RULES

Let a log structure be given. For example Case, Ac-

tivity, Time Stamp, Resources. A log entry is a record

of the log structure ﬁlled with values. Let a set of log

entries be given. This set of log entries presents a set

of cases of a process (workﬂow).

The ﬁlters of the process mining tools support ﬁl-

tering the cases with true (or false) values of speciﬁed

temporal and boolean expressions. The ﬁlters can be

classiﬁed as

• Followers ﬁlters : SELECT CASES such that a

temporal relation (an order) on activities(events)

is speciﬁed with operators ALWAYS, NEXT, IN-

FUTURE, ALWAYS UNTIL, EXITS UNTIL.

• End-Activity ﬁlters: SELECT CASES that end

with speciﬁed events or entries with speciﬁed at-

tributes.

• Times Frame and Attribute ﬁlters: SELECT EN-

TRIES with speciﬁed time frame and/or attribute

value.

A workﬂow is a process that starts and ends with the

speciﬁed start and end activities. A log may contain entries

of many workﬂows (Berk, 2021).

In order to use Process Mining for assessment of busi-

ness process performance, we need to identify busi-

ness rules that correspond to ﬁlters of process mining

tools. We propose a method that combines Process

Mining and semi-structured interviews of the reusable

structure. The interview structure and, further, the in-

terview questions have been designed around the ﬁl-

ters provided by process mining tools.

3.1 Method Combining Interviews and

Process Mining

1. Introduction. “We investigate the use of tools that

analyze logs. We do not try to inﬂuence the busi-

ness process of your company.”

Look at the process identiﬁed in the log by the

tool.”

(a) Do you recognize the names of activi-

ties(events)?

(b) Rethink last process cases that you handled.

2. Visualization of the business process using the ac-

tivities named by the interviewee.

3. Performance Business Rules.

(a) What is the goal of a process (a set of cases)?

(b) What is the goal of each process case?

achieved?

(d) Are there examples where the internal goal of

company deviates from the external goal of the

process of clients?

(e) Are there events(activities) that should follow

one another? What is the relation of these

events(activities)?

(f) Are there mandatory events(activities) in each

case? Why are they mandatory?

(g) When can the case be ended? What is the de-

sired outcome of a case?

(h) How is deﬁned who executes a process or a

case?

(i) Are there cost-beneﬁt considerations in the

choice of an event(activity)?

(j) Are there any obligations about time duration

of a case of a business process?

(k) Is the time between events (activities) decisive

for the success of the process? If yes, between

which events(activities)?

4. Analysis of answers of interviewees to identify

business rules and performance indicators.

5. Process mining for each identiﬁed business rule

and performance indicator.

Process Mining and Performance Business Rules

389

Figure 1: Business process of a recruiting agency.

ENASE 2022 - 17th International Conference on Evaluation of Novel Approaches to Software Engineering

390

4 CASE STUDY: EMPLOYMENT

AGENCY

The scope of the log extraction was limited to all re-

quests received by an Employment Agency and ﬁ-

nished during the period from 01-10-2020 to 31-12-

2020. This period was selected to rule out seasonal

market inﬂuences. The extracted log can be found

in (Berk and Roubtsova, 2021).

4.1 Introduction of Interviews

The interviews have been conducted with four re-

cruiters. On the basis of the answers, a business pro-

cess has been depicted in BPML (Figure 1). The ac-

tivities mentioned by interviewees have been used for

the process description.

4.2 Visualization of the Business Process

Figure 1 shows that a client-organization, that hires,

submits a request for a candidate (Request Candidate)

that is logged in the requests. The recruiters use this

request to track demand and handle applications.

A request is reviewed (Review Request), than it

is registered and sent to one of the hunting methods:

Open Vacancy, Promotions leaﬂets and Newspapers,

Database search, Use of machine learning tools. Posi-

tions are open and Applications are accepted until the

customer withdraws, the customer ﬁlls the position, a

competitor ﬁlls the position, or, in the best-case sce-

nario, the agency ﬁlls the position. The request is then

closed and the process ends (Berk, 2021).

If an Application has been accepted, it is screened

on required parts (CV, Motivation). If application is

complete, its applicant goes through an Internal In-

take by the recruiter, Client Proposal, and External

Intake by the hiring client. There may be no match if

the recruiter or client ﬁnds the candidate not suitable,

depending on where in the procedure the candidate

is. In addition, in all of these procedures, the candi-

date may lose interest in the function or the request

may become closed due to reasons outlined in the re-

quest process. In the best-case scenario, the candidate

passes through all activities and has an opportunity to

start within the function he/she desired. If a candidate

is hired, the Request Candidate Process is closed.

The Application Process (Figure 1) can be di-

vided into the main process of handling a request of

the client and the sub-process for applications, where

each request can have a number of applications. The

processes have to be analyzed together as the Re-

quest Process has data needed for the Application

Process. Therefore, the Request Process activities

(Open, Close, Connect Vacancy) have been added to

each application log. Thus, applications within one

request have the activity “Open Request” with the

same time stamp. This makes it impossible to count

the number of Open Request activities, but makes it

possible to compare the time stamps within the re-

quests with the time stamps of the applications. The

data transformations have been done within Alteryx-

tool (Alteryx, 2021) and resulted in 326,198 records

with 1,420 requests and 48,307 applications. The

ﬁelds in the dataset are presented in Figure 2.

4.3 Performance Business Rules

In order to identify business rules in answers of inter-

viewees, the analyst should know the activities of the

logged business process. Analyzing the answers of

the interview questions, the analyst identiﬁes pairs of

following activities in one case and uses them to ﬁlter

cases.

For example,

Question: What is the goal of the process?

Answer: “You want that the person you propose to be

accepted.”

Business rule: A business rule identiﬁes a case

that has event [Proposed] and “in future” event [Ac-

cepted]. If a case contains [Proposed] and gets “in

future” event [Rejected], this case is ﬁltered as a case

violating the business rule. The answers of intervie-

wees and their analysis are available in (Berk, 2021).

For each business rule, a new process mining pro-

cess is started. Each business rule aims at a speciﬁc

problem, e.g., the relationship between vacancies and

exit reasons. Because of the space limitation, we

present process mining for four business rules of 10

identiﬁed business rules.

Business Rule 1: A vacancy should attract candi-

dates who meet the demands requested by the client.

The agency uses vacancies to attract and select can-

didates and ﬁll in the open positions for the client.

Each application of a candidate is screened; this takes

time. The application and the job vacancy description

are used to make an assumption whether or not the

applied candidate is suitable for the job. If this as-

sumption is incorrect, it could be caused by the qual-

ity of the job vacancy description. Not every rejection

in the screening stage is based on requirements for

the function. There are other exit reasons: hard re-

quirements, availability, accessibility etc. These exist

reasons should be speciﬁed by business rules.

The business rule 1 combines

• Trace: [Vacancy opened,New,Rejected] and

Process Mining and Performance Business Rules

391

Figure 2: Fields of the log of the analyzed employment agency.

• Source: Rejected reason =

“hard requirements”, “availability” or

“accessibility”.

The process map for all 1015 applications in the

Unit A of the agency was made in Disco (DISCO,

2022)(Figure 3). This map shows 100% of the activ-

ities and 25% of the paths. 372 (marked in Figure 3)

go directly from New to Rejected; 275 of these appli-

cations were identiﬁed by this ﬁlter.

The summary results (Table 1) show that for the

entire agency, approximately 1 of 4 applications are

from candidates who do not meet the stated require-

ments. The conclusion is: “the requirements should

be made clearer by the job vacancy description.”

Business Rule 2: Select cases with withdrawal rea-

sons that should be clear from the job vacancy de-

scription: “commuting distance”, “different function

expectations”, “hours at customer”, “salary”, “travel

expenses”, “want no shifts”.

The amount of cases with these withdrawal rea-

sons is 1%. This shows that the job vacancy descrip-

tion is clear about these mentioned points.

Business Rule 6: A recruiter should react within 24

hours after a candidate’s application.

The 24-hour reaction time on an application is a

standard by the agency to ensure that candidates are

not waiting in the process and are quickly open for

new opportunities. In addition, candidates could ﬁnd

a job via other agencies; thus, the faster the agency re-

sponds, the less opportunity there is for other agencies

to get in contact with the candidates. When a candi-

date applies for a job, the application is automatically

added to the request, which is registered as “New” in

the log. The candidate then goes to screening, which

results in either “Reject Candidate” or “Plan Intake”

in the log, which is shown in Figure 1.

The choice between “Reject Candidate” and “Plan

Intake” is not made mandatory in the system; there-

fore, in this business rule, we look at any activity that

occurs after “New”, which results in this business rule

that identiﬁes processes that do not meet the 24-hour

reaction time.

The business rule combines the trace that starts

with New and follows by any other event (activity)

∗:

Trace: [New,*] Time between events New and the

next following event is: > 24 hours.

Based on the data for the entire agency (Table 1),

the reaction time is above 24 hours for approximately

1 of 3 applications. Thus, for these cases, the stan-

dard of reaction has not being met. The validity of the

standard of reaction should be discussed in agency.

Business Rule 7: The agency works to prevent the

cases when each “Proposed” candidate is “Rejected”

by the client.

The cases with a fragment

Trace[Proposed, Rejected]

violate the business rule 7.

47% of the candidates proposed by the agency are

rejected by the client. This is a warning result that

agency should discuss and understand.

ENASE 2022 - 17th International Conference on Evaluation of Novel Approaches to Software Engineering

392

Table 1: Percentage of cases violating Business Rules.

N Business Rule % of Agency

1 Each vacancy should attract candidates who meet Total cases 27%

the requested demands from the client

2 Each vacancy fully described Total cases 1%

the employment conditions

3 Each vacancy is directly opened after Total cases 15%

receiving a request

4 Each vacancy is closed in time to get Total cases 44%

a chance to every applied candidate

5 The number of cases with intake is limited Cases with intake 38%

6 The recruiter reacts within 24 hours after Total cases 33%

receiving of each application.

7 Each candidate proposed to the client Cases with Proposed 47%

meets requested requirements

8 Proposed candidate had an internal intake Cases with Proposed 49%

9 Candidate turns up at his appointment Total cases 0%

10 Proposed candidate ﬁts within the team of the client Cases with Intake s 10%

Figure 3: Disco Map for Business Rule 1.

Process Mining and Performance Business Rules

393

4.4 Summary of the Application of the

Proposed Method

The semi-structured interviews have retrieved 10

business rules that have been mapped to process min-

ing ﬁlters. In order to answer the interview questions,

the interviewees should have knowledge of the de-

tailed steps of the process. Thus, most useful answers

have come from the operational employees. Some an-

swers have not mentioned the activities of the busi-

ness process.

Table 1 presents the results of Process Min-

ing for analysis of violation of 10 business rules.

Disco (DISCO, 2022) ﬁlters based on the business

rules have been used to separate wanted and unwanted

process cases. The percentage of wanted cases to all

cases gives an information about performance of the

process. The agency has identiﬁed the process frag-

ments for which improvement can be made.

Process discovery has been used to observe how

the performance business rules affect the process

map. Each of the unwanted processes can also be ana-

lyzed separately to identify reasons for deviation from

the expected process.

5 CONCLUSIONS

The results reported in this paper provide an evidence

that process mining directs business to well under-

stood and deﬁned performance analysis, to formula-

tion of goals and business rules.

We have proposed a method that uses ﬁlters avail-

able in process mining tools to shape the semi-

structured interviews. The interviews are aimed to

identify the performance business rules for these

types of ﬁlters. Filters are related to the business

process performance and are used to understand and

formalize the desired outcomes and desired process

sequences, their time and resources restrictions. As

we identify performance business rules, our research

combines elements of goal-oriented methods with

process mining techniques.

In this paper the method has been used for per-

formance analysis of a business process at an em-

ployment agency. In near future, several projects

will reuse the combination of process mining and

semi-structured interviews for identiﬁcation of per-

formance business rules in different business do-

mains. The aim of this replication experiment is to

generalize the types of performance business rules ap-

plicable for process mining.

REFERENCES

Alteryx (2021). Simply Powerful. Extraordinarily Easy.

https://www.alteryx.com/.

Berk, Y. (2021). A process mining method for

analysing performance-based business rules of the

application process at an employment agency,

https://research.ou.nl/ws/portalﬁles/portal/41056781.

Berk, Y. and Roubtsova, E. (2021). Data from employment

recruiting agency. https://doi.org/10.17026/dans-zjv-

87yz.

Cheng, H.-J. and Kumar, A. (2015). Process mining on

noisy logs—can log sanitization help to improve per-

formance? Decision Support Systems, 79:138–149.

Dabrowski, J., Kifetew, F. M., Mu

nante, D., Letier, E.,

Siena, A., and Susi, A. (2017). Discovering require-

ments through goal-driven process mining. In 25th

International Requirements Engineering Conference

Workshops (REW), pages 199–203. IEEE.

DISCO (2022). Flexicon DISCO, http://ﬂuxicon.com/

disco/.

Ghalem,

A., Chaﬁk, O., Chroqui, R., and El Alami, S.

(2016). Performance: A concept to deﬁne. La per-

formance: Un concept

a deﬁner.

Ghasemi, M. and Amyot, D. (2020). From event logs

to goals: a systematic literature review of goal-

oriented process mining. Requirements Engineering,

25(1):67–93.

Hong, T. T. B. (2016). Process Mining-driven Performance

Analysis in Manufacturing Process: Cost and Quality

Perspective, http://unist.dcollection.net.

Lamghari, Z., Radgui, M., Saidi, R., and Rahmani, M. D.

(2019). Passage challenges from data-intensive sys-

tem to knowledge-intensive system related to process

mining ﬁeld. In ArabWIC 6th Annual International

Conference Research Track, pages 1–6.

McNeile, A. and Simons, N. (2006). Protocol mod-

elling: a modelling approach that supports reusable

behavioural abstractions. Software & Systems Model-

ing, 5(1):91–107.

Roubtsova, E. and Wiersma, N. (2018). Involvement

of business roles in auditing with process mining.

In International Conference on Evaluation of Novel

Approaches to Software Engineering, pages 24–44.

Springer.

van den Ingh, L. (2016). Evaluating business pro-

cess performance based on process mining,

https://pure.tue.nl/ws/portalﬁles/portal/46934802/

846648-1.pdf.

van der Aalst, W. (2016). Data science in action. In Process

mining, pages 3–23. Springer.

van Eck, M. L., Lu, X., Leemans, S. J. J., and van der

Aalst, W. M. P. (2015). PM2: a process mining project

methodology. In Lecture Notes in Computer Science,

volume 9097, pages 297–313. Springer.

Williams, S. (2016). Business intelligence strategy and big

data analytics: a general management perspective.

Morgan Kaufmann.

ENASE 2022 - 17th International Conference on Evaluation of Novel Approaches to Software Engineering

394