Multicriteria Decision Method for Project Ranking Considering
Guilherme Augusto Barucke Marcondes
National Institute of Telecommunications, Inatel, Av. Joao de Camargo, 510, Santa Rita do Sapucai, Brazil
Project Selection, Multicriteria Decision Methods, Uncertainty, ELECTRE II.
Frequently, decision makers face the challenge of selecting projects to be executed. Resources are not enough
for funding all of them. In this challenge, they need the support of a tool or method, due to several criteria to be
considered simultaneously. Multicriteria decision methods can provide a good support. However, as inherent
to all estimation in projects, uncertainty must be addressed. This works proposes a method for incorporating
uncertainty in project selection using ELECTRE II method and Monte Carlo simulation.
In general, companies face a challenge when need to
decide about projects to be executed. Available re-
source isn’t enough for executing all of them simulta-
neously Dutra et al. (2014); Agapito et al. (2019). It
leads decision makers to select a subset of projects to
be included in company’s portfolio, among the can-
didates Abbassi et al. (2014). For selection, a rank-
ing prioritizing projects is helpful, considering those
which are more aligned to the strategies and market-
ing demand, aiming to execute the best set of projects
Perez and Gomez (2014).
The correct selection of projects is essential for
companies, avoiding waste of resources Urli and Ter-
rien (2010). Applying formal project selection meth-
ods increases the chances of success Dutra et al.
When selecting projects, decision maker needs
to compare several criteria. Multicriteria Decision
Methods (MCDM) help it, allowing elaborate a rank-
ing of options, enumerating from the best to the worst
Wallenius et al. (2008). MCDM applications have
grown in academic work publications Sadi-Nezhad
Preference Ranking Organization Method for En-
richment Evaluation II (PROMETHEE II), VIseKri-
terijumska Optimizacija I Kompromisno Resenje
(VIKOR), Technique for Order of Preference by Sim-
ilarity to Ideal Solution (TOPSIS) and Elimination Et
Choix Traduisant la R
e II (ELECTRE II) are ex-
amples of MCDM Brans and Vincke (1985); Opri-
covic (2012); Hwang (1981); Roy and Bertier (1973).
All of them can offer, at the end, a ranking from
the best to the worst options Martins and Marcondes
For using MCDM, specialists must evaluate the
criteria, estimating objective values for each project.
It includes uncertainty in selection, once it is inherent
in the estimation process, and a normal and inevitable
phenomenon in projects Bohle et al. (2015). There-
fore, uncertainty should be considered when applying
the MCDM.
Three-point estimation method allows to incorpo-
rate the variation caused in the values due to uncer-
tainty, given that, instead of estimating by a single
value, three are used: most likely, optimistic and pes-
simistic PMI (2017). These three values can be used
to define a triangular probability distribution for pa-
rameters under analysis, and dealing with uncertainty
in evaluations Stein and Keblis (2009).
Monte Carlo simulation is a tool for considering
uncertainty in evaluations PMI (2017); Marcondes
et al. (2017). Each simulation round draws a random
value for the parameters, based on the defined proba-
bility distribution. After several rounds, you can ob-
serve the variation in the results.
This work proposes a way of considering un-
certainty in ranking projects by MCDM. For each
project, three-point estimation is proceeded, triangu-
lar distributions are set and simulation done. The
proposal is exemplified by applying ELECTRE II
method over a set of eleven real software development
projects (the same procedure could be applied to any
Marcondes, G.
Multicriteria Decision Method for Project Ranking Considering Uncertainty.
DOI: 10.5220/0010183601230128
In Proceedings of the 10th International Conference on Operations Research and Enterprise Systems (ICORES 2021), pages 123-128
ISBN: 978-989-758-485-5
2021 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
other MCDM listed above).
The remaining of this paper is organized as fol-
low: Section 2 presents the principles of multicriteria
decision methods, detailing ELECTRE II; the impor-
tance of uncertainty in project selection problems is
presented in Section 3; Section 4 proposes a method
for selecting projects considering uncertainty; which
is exemplified by a real problem in Section 5; Section
6 concludes the work.
Single criterion decisions are very intuitive, once one
must choose the alternative with higher preference
score. However, when decision depends on more
than one criterion, the choice must consider, for in-
stance, weights and conflicts among criteria. It de-
mands more sophisticated methods Tzeng and Huang
For project selection, it is important to use MCDM
to find an appropriate assessment, once it often is in-
volved with multiple criteria. These methods allow
to rank different alternatives subject to qualitative cri-
teria. Its application has been rising in the past few
years Sadi-Nezhad (2017).
There are a lot of MCDM in the literature. Four of
these methods are: Preference Ranking Organization
Method for Enrichment Evaluation II (PROMETHEE
II), VIseKriterijumska Optimizacija I Kompromisno
Resenje (VIKOR), Technique for Order of Preference
by Similarity to Ideal Solution (TOPSIS) and ELimi-
nation Et Choix Traduisant la R
II). PROMETHEE II can be used when a complete
classification is necessary in the presence of a finite
set of alternatives. VIKOR solves decision prob-
lems with criteria of the same priority, identifying
the alternative closest to the ideal (allows the def-
inition of rankings). TOPSIS evaluates the perfor-
mance of alternatives with several comparison crite-
ria. The closer to the ideal solution and far from the
non-ideal, the better the alternative Martins and Mar-
condes (2020). ELECTRE II allows the calculation
of the agreement and disagreement indices for each
alternative, allowing, with these values, the construc-
tion of a preference classification Tzeng and Huang
For project selection, MCDM must follow the
steps (adapted from Opricovic and Tzeng (2004)):
Establishing evaluation criteria that relate to
Evaluating projects in terms of criteria;
Applying an MCDM;
Accepting one alternative (or some of them) as se-
An MCDM that is able to produce a final ranking is
useful. This ranking can indicate a prioritizing list of
projects, allowing the selection of the first one, two,
three, four, and so on, options, depending on the num-
ber of projects to be executed. Any method presented
in this section allows decision makers to achieve this
requirement. For this work, the example presented in
Section 5 uses ELECTRE II.
2.1 Elimination Et Choix Traduisant la
There is four ELECTRE (in English Elimination and
Choice Translating Reality) methods (I, II, III and
IV). They have different applications Opricovic and
Tzeng (2006):
ELECTRE I for selection, but without a ranking;
ELECTRE II, III and IV for ranking problems;
ELECTRE II and III when it is possible and desir-
able to quantify the relative importance;
ELECTRE III incorporates the fuzzy nature of de-
cision making;
ELECTRE IV when quantification is not possible.
The ELECTRE II method was chosen due to the pos-
sibility of ranking the alternatives and because, in
project selection problems, it is possible quantifying
relative importance. It is an approach for multicri-
teria decision, based on the outranking relation. It
works with the concepts of concordance and discor-
dance. For each alternative (projects, in this paper),
these two indexes are calculated, considering all cri-
teria Opricovic and Tzeng (2006); Tzeng and Huang
For identifying how much alternative a is, at least,
as good as alternative b, one must calculate concor-
dance index C(a, b). Discordance index D(a, b) is a
measure of how much strictly preferable alternative
b is in comparison to alternative a Tzeng and Huang
Some quantities are needed for calculating these
indexes Tzeng and Huang (2011).
(a, b) =
| g
(a) > g
(a, b) =
| g
(a) = g
(a, b) =
| g
(a) < g
ICORES 2021 - 10th International Conference on Operations Research and Enterprise Systems
(a, b) =
(a, b) =
(a, b) =
i represents the i
selection criterion (i = 1, ..., n);
( j) indicates the preference value of the i
lection criterion for alternative j ( j = 1, ..., J);
is the weight of the i
selection criterion;
[0, 1] indicates if, for the i
selection crite-
rion, alternative b is strictly preferable in compar-
ison to alternative a, or the opposite, respectively.
The concordance index C(a, b) of alternative a with
respect to alternative b is Tzeng and Huang (2011):
C(a, b) =
(a, b) +W
(a, b)
(a, b) +W
(a, b) +W
(a, b)
The discordance index D(a, b) of alternative a
with respect to alternative b is Tzeng and Huang
D(a, b) =
(a) g
is the highest preference value for the i
tion criterion;
is the lowest preference value for the i
tion criterion.
For ranking the alternatives, decision maker must
compare the lists of concordance index, in descend-
ing order, discordance index, in ascending order.
Alternatively, a final ELECTRE II index could be
calculated by:
e = C(a, b) D(a, b) (9)
The ranking is constructed ordering e in descending
order, from the highest (best option) to the lowest
(worst option) values.
For applying MCDM in projects, it is necessary that
decision maker evaluates and estimates the values of
criteria for each project. Uncertainty is an inherent
effect of estimating and forecasting Marcondes et al.
When defining the value of any parameter, the de-
cision maker (or project specialist) chooses the one
that best represents his evaluation. However, this es-
timation may not be accurate. Or even, if more than
one person evaluates and estimates such parameters,
the estimated values may differ.
For instance, if three specialist in the projects de-
fine their values for some selection criteria, instead of
a single value for each one, they are likely to have
a range of values. It represents the uncertainty in
parameter definition, impacting in final decision of
For minimizing this impact, instead of working
with a single value, three-point estimation can be use-
ful. It is done by estimating the values most likely,
optimistic and pessimistic. The most likely value is
the estimation of the parameter best understanding by
evaluator (probably, this the value estimated if a sin-
gle point estimation is proceeded). Optimistic and
pessimistic values must reflect the best and worst sce-
narios, respectively PMI (2017).
These three values can be used to construct a tri-
angular probability distribution PMI (2017), as pre-
sented in Figure 1:
parameter a is equal to pessimistic estimation;
parameter b is equal to optimistic estimation;
parameter c is equal to most likely estimation.
Probability Density Function
Random Variable
a bc
Figure 1: Triangular distribution based on three point esti-
The method proposed in this work is for supporting
decision makers in project selection. Its final objec-
tive is a definition of a ranked list of projects, from
the best option to the worst, for supporting decision
of which ones to execute. It is built based on ELEC-
TRE II method, considering uncertainty.
For each project, three parameters are estimated
for each criterion. Triangular distributions are set, as
Multicriteria Decision Method for Project Ranking Considering Uncertainty
described in Section 3. Also the weights of criteria
must be established.
Monte Carlo simulation allows incorporating un-
certainty in evaluation. For each round (m rounds),
project parameters are randomly chosen based on the
triangular distributions. The values are normalized in
a common scale, for avoiding distortion due to dif-
ferent value ranges. Then ELECTRE II evaluation is
executed. At the end of rounds, there are m sets of e
indexes calculated, and a final e index set is achieved
defining the mean of them.
The algorithm for ranking projects, with a
stochastic approach to address uncertainty in the def-
inition of parameters and using Monte Carlo simula-
tion is the following:
% m is the number of Monte Carlo rounds.
Define m
Read pessimistic_project_estimation
Read most_likely_project_estimation
Read optimistic_project_estimation
Read criteria_weigths
Repeat m times
Define ramdomly project_parameters
Normalize values
Calculate e_index
Store e_index
Calculate mean of m sets of e_index values
Define final ranking
For exemplify, the method proposed in Section 4
was applied in a set of eleven real software develop-
ment projects, from a software R&D service provider.
Three specialists on software projects and market
from the company estimated the values for four cri-
C1 - Return/risk rate (weight - 0,4): a ratio be-
tween the estimated return and the associated risk
(from 1 - the lowest to 10 - the highest);
C2 - Competitiveness improvement (weight -
0,3): the capacity of project for improving com-
pany competitiveness (from 1 - the lowest to 10 -
the highest);
C3 - Market potential (weight - 0,2): the capac-
ity of project for improving market share or mar-
ket insertion (from 1 - the lowest to 10 - the high-
C4 - Degree of innovation (weight - 0,1): how
innovative the project is (from 1 - the lowest to 10
- the highest).
The objective was to select three projects for exe-
cution. They should be chosen in order to be more
aligned with the company’s strategies and met the de-
fined criteria in the best way. Preparing a ranking of
projects (based on the criteria and using ELECTRE
II method), the three best ranked ones should be exe-
The first step was the definition, by the specialists
consulted, of the values of project estimations. For
each of these, they defined the most likely, pessimistic
and optimistic values, of the criteria defined for se-
lection. Table 1 presents most likely values, which
is generally used when single estimation is applied
in selection. Tables 2 and 3 present pessimistic and
optimistic values, respectively, to allow applying the
selection considering the uncertainty.
Table 1: Projects Characteristics - Most Likely.
Project C1 C2 C3 C4
A 10 3 2 3
B 8 5 8 7
C 2 6 5 4
D 1 2 9 10
E 5 9 10 6
F 6 3 2 2
G 7 7 7 9
H 3 5 3 3
I 8 1 6 7
J 9 9 2 4
K 3 8 1 5
Before simulation using the proposed method, one
was made using the single estimation (values pre-
sented in Table 1) as a reference for comparing the
results. In this case, the projects that would be se-
lected are J, E and G. They were the best ranked by
ELECTRE II e index, as can be seen in graphic of
Figure 2, presenting the best option on the left, and
ICORES 2021 - 10th International Conference on Operations Research and Enterprise Systems
Table 2: Projects Characteristics - Pessimistic.
Project C1 C2 C3 C4
A 8 2 1 2
B 7 4 7 6
C 1 4 3 3
D 1 1 7 9
E 3 6 8 5
F 5 2 1 1
G 5 5 5 8
H 2 4 2 2
I 7 1 4 5
J 6 8 1 3
K 2 7 1 3
Table 3: Projects Characteristics - Optimistic.
Project C1 C2 C3 C4
A 10 4 4 5
B 9 6 9 8
C 4 7 6 5
D 2 3 10 10
E 6 10 10 7
F 7 6 3 4
G 9 8 9 10
H 4 6 5 4
I 9 3 7 8
J 10 10 3 5
K 5 10 2 7
the worst on the right.
Figure 2: Projects Ranking - Without Uncertainty.
Applying the selection with uncertainty, as pre-
sented in Section 4 (Monte Carlo simulation with
10,000 rounds), the ranking of projects changed to the
one shown in the Figure 3, indicating projects G, J and
B for execution.
Comparing these two results (without and with
uncertainty), some differences are identified. The first
four positions of ranking changed from J, E, G and B
Figure 3: Projects Ranking - With Uncertainty.
to G, J, B and E. And the best ranked project changed
from J to G. Another difference was the inversion ob-
served in 8
and 9
ranking positions (from C / F to
F / C). The best three options, in simulation with un-
certainty, were projects G, J and B (comparing to re-
sult without uncertainty, project B was included, and
project E excluded). These changes showed the im-
portance of considering uncertainty in project selec-
Finally, considering the uncertainty in selection,
the chosen projects were G, J and B. They were the
best ones in ELECTRE II e index ranking.
Project selection is a challenge in portfolio manage-
ment in companies. Generally, there is no enough re-
sources for funding all listed projects. Due to this,
decision makers need to choose those which will be
The selection is often not direct, as it depends on
several criteria, which must be evaluated simultane-
ously. Some of them can also be in conflict with each
Multicriteria decision methods can help decision
makers in selection. They are a good solution for
handling decisions that involve multiple criteria, as:
However, these methods based the decision on
values estimated for the criteria. As an estimation,
these values can bring uncertainty for selection.
The work presented in this paper proposes a
method to incorporate uncertainty in decision, sup-
porting decision makers. Instead of estimation with a
single value, it is done using three: most likely, pes-
simistic and optimistic. Based on them, one proceeds
a Mote Carlo simulation, defining the parameters ran-
domly in each round (in this case, applying triangular
Multicriteria Decision Method for Project Ranking Considering Uncertainty
In the numerical example presented, the target was
choosing three of eleven projects. The results indi-
cated an important change. Without uncertainty, the
selected projects would be J, E and G. However, the
ELECTRE II e index ranking changed when select-
ing with uncertainty. After Monte Carlo simulation
(10,000 rounds), projects indicated for execution were
G, J and B. The best option changed from J to G. And
the project E, presented in the list when no uncertainty
was considered, was excluded of the final list, includ-
ing project B. It highlighted the importance of con-
sidering uncertainty in selection, due to its impact on
final results.
For future works, some issues must be considered
in selection:
Apply a fuzzy approach to address uncertainty
rather than Monte Carlo simulation;
Constraints as developers and equipment avail-
Evaluate projects with more than one MCDM;
Time needed for each project execution.
Abbassi, M., Ashrafi, M., and Tashnizi, E. S. (2014). Se-
lecting balanced portfolios of R&D projects with in-
terdependencies: A cross-entropy based methodology.
Technovation, 34(1):54–63.
Agapito, A. O., Vianna, M. F. D., Moratori, P. B., Vianna,
D. S., Meza, E. B. M., and Matias, I. O. (2019). Using
multicriteria analysis and fuzzy logic for project port-
folio management. Brazilian Journal of Operations &
Production Management, 16(2):347–357.
Bohle, F., Heidling, E., and Schoper, Y. (2015). A new
orientation to deal with uncertainty in projects. Inter-
national Journal of Project Management, 34(7):1384–
Brans, J. P. and Vincke, P. (1985). A preference ranking
organisation method: (the promethee method for mul-
tiple criteria decision-making). Management Science,
Dutra, C. C., Ribeiro, J. L. D., and de Carvalho, M. M.
(2014). An economic-probabilistic model for project
selection and prioritization. International Journal of
Project Management, 32(6):1042–1055.
Hwang, C.L.; Yoon, K. (1981). Multiple Attribute Decision
Making: Methods and Applications. Springer-Verlag,
Nova York, EUA.
Marcondes, G. A. B., Leme, R. C., Leme, M. S., and
da Silva, C. E. S. (2017). Using mean-Gini and
stochastic dominance to choose project portfolios with
parameter uncertainty. The Engineering Economist,
Martins, D. T. and Marcondes, G. A. B. (2020). Project
portfolio selection using multi-criteria decision meth-
ods. In IEEE International Conference on Technology
and Entrepreneurship ICTE.
Opricovic, S. (2012). Multi-criteria optimization of civil en-
gineering systems (in Serbian, Visekriterijumska opti-
mizacija sistema u gradjevinarstvu). PhD thesis, Fac-
ulty of Civil Engineering, Belgrade.
Opricovic, S. and Tzeng, G.-H. (2004). Compromise so-
lution by mcdm methods: A comparative analysis of
vikor and topsis. European Journal of Operational
Research, 156(2):445–455.
Opricovic, S. and Tzeng, G.-H. (2006). Extended vikor
method in comparison with outranking methods. Eu-
ropean Journal of Operational Research, 178(2):514–
Perez, F. and Gomez, T. (2014). Multiobjective project port-
folio selection with fuzzy constraints. Annals of Op-
erations Research, 236:1–23.
PMI (2017). A Guide to the Project Management Body of
Knowledge. Project Management Institute, Atlanta,
EUA, 6 edition.
Roy, B. and Bertier, P. (1973). La methode ELECTRE II:
Une application au media-planning. Operational Re-
search, page 291–302.
Sadi-Nezhad, S. (2017). A state-of-art survey on project
selection using mcdm techniques. Journal of Project
Management, 2(1):1–10.
Stein, W. E. and Keblis, M. F. (2009). A new method to
simulate the triangular distribution. Mathematical and
Computer Modelling, 49(5-6):1143–1147.
Tzeng, G. H. and Huang, J. J. (2011). Multiple attribute de-
cision making: methods and applications. Chapman
and Hall/CRC.
Urli, B. and Terrien, F. (2010). Project portfolio selec-
tion model, a realistic approach. INTERNATIONAL
Wallenius, J., Dyer, J. S., Fishburn, P. C., Steuer, R. E.,
Zionts, S., and Deb, K. (2008). Multiple criteria deci-
sion making, multiattribute utility theory: Recent ac-
complishments and what lies ahead. Management Sci-
ence, 54(7):1336–1349.
ICORES 2021 - 10th International Conference on Operations Research and Enterprise Systems