Parameter Sensitivity Patterns in the Plant Propagation Algorithm
Marleen de Jonge
a
and Daan van den Berg
b
Informatics Institute, University of Amsterdam, The Netherlands
Keywords:
Plant Propagation Algorithm, Evolutionary Algorithms, Parameterization, Parameter Sensitivity, Parameter
Robustness, Offspring Number, Population Size, Metaheuristics, Function Optimization.
Abstract:
The parameter sensitivity of the plant propagation algorithm’s performance, defined as the maximum impact
on attained objective values, is studied on function optimization. The number of offspring and population size
are varied across a large range of values and tested on multidimensional benchmark test functions. As the
dimensionality of a function increases, the parametric sensitivity shows one of three distinct different patterns:
sublinear increase, superlinear increase or an up-down phase transition. We conjecture that the difference in
algorithmic behaviour might be due to the intrinsic mathematical properties of the functions themselves.
1 INTRODUCTION
Hard optimization problems come in a broad bou-
quet of both theoretical and real-world applica-
tions. Continuous problems such as molecular dock-
ing, scientific imaging techniques and process flow-
sheet optimization in chemical plants (Meier et al.,
2010)(Chambolle and Pock, 2016)(Salhi and Fraga,
2011) are countered by discrete exemplars such
as vehicle routing, protein design and computa-
tional creativity (Kara et al., 2007)(Allouche et al.,
2014)(Paauw and Van den Berg, 2019). To date,
techniques to obtain good solutions for such prob-
lems can roughly be divided into two categories: ex-
act methods and heuristic methods(Puchinger and
Raidl, 2005)
1
. Exact methods exhaustively search
the complete or partially pruned state space, thereby
guaranteeing to find an optimal solution, but run-
times tend to increase exponentially (or worse)
with problem instance size, prohibiting practical de-
ployment on any real-world problem. By sacri-
ficing the guarantee of optimality, heuristic meth-
ods produce solutions which are good (enough),
and reachable within reasonable time budgets
(Puchinger and Raidl, 2005). For this reason, the de-
velopment (meta)heuristic algorithms has flourished
as an approach for a multitude of optimization tasks.
a
https://orcid.org/0000-0003-4911-2647
b
https://orcid.org/0000-0001-5060-3342
1
For some spun out examples, see (Sleegers et al., 2020)
and (van den Berg et al., 2016).
All is not rosegardens however. As the field has
witnessed a blossoming proliferation of new methods,
the often creative metaphorical references (to nature)
obfuscate real progress in the field and prevent trans-
parency, replicability and scientific rigor (S
¨
orensen,
2015). Furthermore, the community has been com-
plaining for ages that the wildgrowth of new algo-
rithms is seldomly parameterically tested, and the per-
formance across different problems is poorly under-
stood (S
¨
orensen, 2015)(Eiben and Smit, 2011). As
new research on heuristic methods seems primarily
focused on achieving slight improvements on a sin-
gle problem instance, in-depth understanding of al-
gorithmic behaviour and performance dependencies
obtain much less attention. Kenneth S
¨
orensen has in
fact devoted a whole paper to it, rightfully criticizing
this development as it “threatens to come at the ex-
pense of scientific rigor in the field of metaheuristics”
(S
¨
orensen, 2015). The authors of this study generally
agree to these viewpoints.
Extensive understanding of parameter settings and
their contribution to algorithmic performance can be
of great importance, both in terms of its best found ob-
jective value (or fitness) and in terms of its robustness
(Smit and Eiben, 2009). Unfortunately, the vast num-
ber of possible parameter configurations, the com-
plexity of their interaction and the diversity of mea-
suring methods make the process of finding optimal
parameter settings often harder than disentangling a
wall of brambles.
One relatively successful algorithm lacking a pa-
rameterization study is the plant propagation algo-
92
de Jonge, M. and van den Berg, D.
Parameter Sensitivity Patterns in the Plant Propagation Algorithm.
DOI: 10.5220/0010134300920099
In Proceedings of the 12th International Joint Conference on Computational Intelligence (IJCCI 2020), pages 92-99
ISBN: 978-989-758-475-6
Copyright
c
2020 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
Figure 1: Nine multidimensional benchmark test functions used in this study. All are continuous and dimensionally scalable,
but vary in properties such as modality, range, domain, and elementary mathematical constituents. Shown here are the 2D
versions.
rithm (PPA), introduced by Abdellah Salhi and Eric
S. Fraga (Salhi and Fraga, 2011). Even though it
is metaphorically linked to nature, its paradigm is
rooted in simple operations, and its parametric canopy
is quite modest. It has been demonstrated to work
well on a variety of continuous benchmark test func-
tions as well as on some discrete problems such as the
Traveling Salesman Problem, University Timetabling
and even explicitly tracking down NP-complete prob-
lem instances (Sulaiman et al., 2014)(Salhi and Fraga,
2011) (Selamo
˘
glu and Salhi, 2016)(Vrielink and
van den Berg, 2019) (Geleijn et al., 2019)(Sleegers
and van den Berg, 2020). Additionally, the avail-
ability of open source python repositories lower the
threshold for implementation, experimentation, test-
ing, and replication(De Jonge, 2020).
In most experimentation on PPA, its parame-
ters are arbitrarily set, and little is known about
their individual or interdependent impact on perfor-
mance. Following a preliminary investigation on two-
dimensional functions(de Jonge and van den Berg,
2020), we will address multidimensional functions,
which behave substantially different. We investigate
the interplay of two interdependent parameters of the
algorithm, the population size (popSize) and the max-
imum number of offspring per individual (n
max
). We
explore 400 parameter settings of PPA on nine care-
fully chosen n-dimensional benchmark test functions.
Adopting a practical approach advocated by Selmar
Smit and Gusz Eiben, we choose parameter values
that optimize the algorithm’s performance while si-
multaneously mapping how its performance depends
on its parameter settings, referring to the latter as ‘ro-
bustness’ or ‘sensitivity’. The setup will yield some
puzzling results, and raise questions about the inter-
play between the algorithm and the problem instance
it is trying to solve
2
. Luckily, we also get some
straight up answers.
The rest of the paper is organized as follows: the
following section will provide an explanation on PPA,
whereas Section 3 will introduce the benchmark test
2
An elaborate discussion on exactly this topic took place
in the GECCO-workshop “Good Benchmarking Practices
for Evolutionary Computation”, which took place (online)
at GECCO2020, and also supplied a public paper (Bartz-
Beielstein et al., 2020).
Parameter Sensitivity Patterns in the Plant Propagation Algorithm
93
functions used in this study. Thereafter, Section 4
offers a detailed description of the experiment after
which the results and conclusions will be presented
in Section 5. We finalize the paper by discussing the
choices we made along the way, and the shortcomings
our approach might have in Section 6.
2 PLANT PROPAGATION
ALGORITHM
In its seminal form, PPA is an implementation on con-
tinuous optimization problems (Algorithm 1) (Salhi
and Fraga, 2011), and the enveloping paper also in-
cluded some runs on a suite of benchmark test func-
tions, all of which have been included in this study
(Figure 1, Table 1). The only difference is that for
this investigation, all global minima are normalized
to zero by applying the necessary ‘vertical’ shifts.
Algorithm 1: PPA pseudo code.
Require: objective f(x), x R
1: Generate population P = {p
1
, p
2
, ..., p
popSize
}
2: g 1
3: for g 1 to g
max
do
4: compute N
i
= f (p
i
), p
i
P
5: sort P in descending order of N
i
6: create new population φ
7: for each p
i
P do {take best popSize only}
8: r
i
set of offspring where both the size
of the set and the mutability for each new individ-
ual is functionally related to the fitness N
i
9: φ φ r
i
merge population with off-
spring; death occurs by selection of popSize best
individuals
10: end forP φ new population
11: end for
12: return P, population of solutions
In this population based algorithm, an individual
is a vector of D points, in which D is the dimension-
ality of the benchmark function at hand, and its ob-
jective value is calculated from the dimensionalized
benchmark function on the vector. After randomly
initializing popSize individuals, calculating objective
values and normalizing them to [0, 1] (Eq. 1), the
hyperbolic tangent function maps the normalized fit-
ness N
i
nonlinearly to (0,1) (Eq. 2). This asymptotic
function serves to prevent offspring to be generated at
the exact same location as the parent, and “provide[s]
a means of emphasising further better solutions over
those which are not as good” (Salhi and Fraga, 2011).
Then, each generation, individuals within the popula-
tion with higher mapped fitness N
i
will generate more
offspring, whereas individuals with the lower N
i
will
generate fewer offspring (Eq. 3). But whereas the
number of offspring n
i
is proportional to N
i
, the muta-
bility m
i
of an individual’s offspring is inversely pro-
portional to the fitness (Eq. 4).
z(x
i
) =
f (x
max
) f (x
i
)
f (x
max
) f (x
min
)
(1)
N
i
=
1
2
(tanh(4z(x
i
) 2) + 1) (2)
n
i
= [n
max
N
i
r] (3)
m
i
= 2(1 N
i
)(r 0.5) (4)
In these equations, r is random number [0, 1), drawn
anew every time it is used, and m
i
is applied to every
dimension of the benchmark function at hand. The
newly created offspring is then added tot the popula-
tion, which is sorted to fitness, and the popSize best
individuals are selected for the next generation. Note
that thereby, PPA always retains its best individual,
and adopts an elitist approach – its best fitness cannot
decrease during a run.
3 BENCHMARK FUNCTIONS
Benchmarking is a relatively standardized way of
measuring performance and behaviour of algorithms
(Jamil and Yang, 2013). Literature on test functions
however, is quite diverse and sometimes poorly de-
fined, causing multiple function definitions and spec-
ifications to circulate. For this reason, we explicit
list the used formulae (Table 1) and supply pub-
licly accessible source code (De Jonge, 2020). The
suite was compiled by Wouter Vrielink and forms
a union of functions earlier used on the fireworks
algorithm or the plant propagation algorithm (Tan
and Zhu, 2010)(Salhi and Fraga, 2011)(Vrielink and
van den Berg, 2019). We use the identical multi-
dimensional test functions from Vrielink’s suite to
enable direct comparison between all forementioned
studies. All code is written in Python version 3.6.3
and can be found in a publicly accessible online
repository(De Jonge, 2020).
ECTA 2020 - 12th International Conference on Evolutionary Computation Theory and Applications
94
Table 1: The entire test suite of multidimensional benchmark functions used in this study. Test have been performed on six
different dimensionalities with 400 parameter settings on each dimensionality.
Name Function Bounds
Tablet
f (x) = 10000x
2
1
+
D
i=2
x
2
i
x [100, 100]
D
Ellipse
f (x) =
D
i=1
10000
(
i1
D1
)
x
2
i
x [100, 100]
D
Cigar
f (x) = x
2
1
+
D
i=2
10000x
2
i
x [100, 100]
D
Sphere
f (x) =
D
i=1
x
2
i
x [100, 100]
D
Rosenbrock
f (x) =
D1
i=1
(100(x
i+1
x
2
i
)
2
+ (x
i
1)
2
) x [5, 10]
D
Griewank
f (x) = 1 +
D
i=1
x
2
i
4000
D
i=1
cos
x
i
i
x [600, 600]
D
Schwefel
f (x) = 418.9829D
D
i=1
(x
i
·sin(|
x
i
|)) x [500, 500]
D
Rastrigin
f (x) = 10D +
D
i=1
(x
2
i
10cos(2πx
i
)) x [5.12,5.12]
D
Ackley
f (x) = 20 ·exp(0.2 ·
q
1
D
D
i=1
x
2
i
) exp(
1
D
D
i=1
cos(2πx
i
)) + 20 + e x [100, 100]
D
4 EXPERIMENT
The experimental setup is as massive as it is
straightforward; we perform optimization runs with
PPA on nine different n-dimensional benchmark
functions (Table 1) in six dimensionalities D
{2, 5, 10, 20, 50, 100} resulting in 54 heatmaps (see
Figure 2). Every heatmap is comprised of 400 cells
for 400 different parameter settings 1 popSize 40,
1 n
max
10, and every cell contains ten PPA-runs
of 10,000 function evaluations, adding up to a grand
total of roughly 2.16 billion function evaluations for
the whole experiment. Due to parallelization efforts
however, the runtime for the experiment could be lim-
ited to approximately 34 hours on 10 nodes with 12 to
48 cores each, SURFSara’s LISA cluster computer at
Amsterdam Science Park.
After completing ten runs for one parameter set-
ting, the median was taken to a heatmap’s cell (e.g.
Rosenbrock’s six heatmaps in Figure 2). Generally
speaking, but surely in Rosenbrocks case, the average
error increases with the dimensionality of the bench-
mark function, resulting in ever darker heatmaps. But
this is to be expected; for all benchmark test functions
used in this study, the dimensionality is reflected in
the sum over its dimensions (
D
i=1
), whose terms often
cannot be negative within the domain. Therefore, the
randomly chosen initial values are much higher on the
50-dimensional function than on the same function in
10 dimensions.
So how to account for this dimensional upscal-
ing when quantifying the parametric influence on the
attained objective value? Considering these obser-
vations, the degree to which PPA is susceptible to
changes in parameter values can be seen as the max-
imal difference its various parameterizations might
have within the 10,000 evaluation range of this exper-
iment. We therefore choose to define the parameter
sensitivity as the interval between the best and worst
heatmap cell relative to the range in the attained ob-
jective value:
Max( f
popSize,n
max
(x)) Min( f
popSize,n
max
(x))
µ
benchmark
(5)
Finally, for µ
benchmark
, we take the mean of 10,000
randomly sampled points (see Figure 3). This value is
alone sufficient for these functions, as the global min-
imum of any of these in any dimensionality is zero.
5 RESULTS & CONCLUSION
From a simple and straightforward experimental setup
come some surprising results. For all functions, per-
formance of the plant propagation algorithm deterio-
rated with dimensionality, comparable to earlier re-
sults (Vrielink and van den Berg, 2019). The ex-
ception is the Ackley function, on which the found
minimum is near 20 for all D > 2, but this too is
known from literature (although the dimensionality
Parameter Sensitivity Patterns in the Plant Propagation Algorithm
95
Figure 2: Performance of the plant propagation algorithm with 400 different parameter settings on the Rosenbrock function.
Every cell contains the median value of ten runs of 10,000 evaluations. As the dimensionality increases, so does the mean
error (µ = 0.0005 to µ = 112011) and the variance thereof (σ = 0.0002 to σ = 21744). The absolute difference between the
best and worst parameter settings increases faster than its mean error, signifying an increase of parameter sensitivity in D.
bound might be slightly more accurate in (Vrielink
and van den Berg, 2019)). The parameter settings, in
all cases except the last three, did not make a differ-
ence of more than a magnitude.
A second, more interesting pattern, can be ob-
served by assessing the absolute parameter sensitivi-
ties in respect to the dimensionalities. For two bench-
mark test functions, the Tablet and the Ellipse, the
parameter sensitivity apparently shows a sublinear
increase with the dimensionality, which could pos-
sibly be characterized by a polynomial of O(n
c
) with
0 < c < 1. For four others, the Cigar, Sphere, Rosen-
brock and Griewank functions, the parameter sensi-
tivity displays a superlinear increase in dimension-
ality, which might be best characterized by a polyno-
mial of O(n
c
) with c > 1. In the three last functions,
Schwefel, Rastrigin and Ackley, the parameter sen-
sitivity shows something that looks like a up-down
phase transition. A dramatic initial increase in sen-
sitivity reaches an early summit somewhere between
5 and 10 dimensions, after which a rapidly collapse
follows, in one case even to near zero.
But a third, perhaps even more intriguing pat-
tern, can be discerned from ordering the nine sensi-
tivity patterns (Figure 4). With some imagination, one
can see the dimensional parameter sensitivity patterns
undergoing a structural phase transition themselves,
from sublinear, to superlinear, to up-down. When
looking closely, the up-down phase transition pattern
might already be materializing in the Griewank func-
tion, perhaps even in the Rosenbrock. Going back to
Table 1, one could easily wonder whether it is a coin-
cidence that the ‘simple quadratics’ only show sub-
linear and superlinear sensitivity patterns (the table
is sorted in the same order as Figure 4). The sec-
ond category, ’quadratic plus trigonometric’ shows
only superlinear or up-down phase transition patterns.
The third category, ’trigonometric plus exponential’
only has an up-down phase transition pattern, and it
is hard not to conjecture that these functional defini-
tions themselves shelter the structural properties that
determine the parameter sensitivities of PPA, and per-
haps other (population based) heuristic algorithms.
An inquiry into what property constitutes which pat-
tern could be extremely valuable. And computation-
ally expensive, not to mention.
ECTA 2020 - 12th International Conference on Evolutionary Computation Theory and Applications
96
Figure 3: Parameter sensitivity is defined as the difference between the absolute best parameterization and absolute worst
parameterization relative to the average initial condition, which, for most benchmark functions, increases with the dimen-
sionality of the function. PPAs senstivity is 0.16 on the Schwefel 10D function, but 0.12 on the Schwefel 50D function, a
counterintuitive result which can be seen in more detail in Figure 4.
Figure 4: Parameter sensitivity patterns for PPA on benchmark test functions comes in three types: sublinear increase, su-
perlinear increase and up-down phase transition. A pattern could be to some degree related to a function’s mathematical
properties, such as being quadratic-only, quadratic-trigonometric, or exponential.
Parameter Sensitivity Patterns in the Plant Propagation Algorithm
97
6 DISCUSSION
From a simple and straightforward experimental
came some not-so-very-simple-and-straightforward
answers. Along the way, we made some choices that
are well open for discussion. First of all, we stud-
ied just two interdependent PPA-parameters and there
is absolutely no reason to presume that these results
are stable under replacement of the tanh-function, dif-
ferent mutability rates, or a different population se-
lection procedure. Second, the choice of benchmark
test functions could matter. Does a ‘combination of
Schwefel and Rastrigin’ show an intermittent stabil-
ity pattern? Third, the measure of sensitivity could be
different, based on standard deviations, or based on
the number of evaluations required to reach a certain
target value v, as suggested in (Eiben and Smit, 2011).
It could also be seen as a measure of uncertainty rel-
ative to a run’s performance, which is in turn relative
to the budget of function evaluations (set to 10,000
in this study). Fourth, considering the difference in
sensitivity patterns for continuous benchmark func-
tions, bifurcated study into more real-life examples
such as the optimization of chemical plant parame-
terizations might provide useful insights. First, the
parameter sensitivity could be assessed, similar to the
study presented here. Second, an algebraic compari-
son between it parameterization-fitness projection and
the known benchmark functions could be made to fur-
ther our knowledge of the relation between parameter
sensitivity and the properties of the continuity it is try-
ing to minimize. Last but not least, the best-to-worst
order of individual heatmap cells could be studied; is
there a pattern to be found, even if just modest? All in
all, many possible roads lead into the future, and we
should make efforts to progress pedestrianally
3
.
ACKNOWLEDGEMENTS
Reitze Jansen (UvA), meticulous and accurate as ever,
was kind enough to pull some mistakes from this pa-
per. Thanks Reitze.
REFERENCES
Allouche, D., Andr
´
e, I., Barbe, S., Davies, J., de Givry, S.,
Katsirelos, G., O’Sullivan, B., Prestwich, S., Schiex,
3
Abdellian saying from ‘The Book of Strawberries’;
“Pedestrianally” (adj., adv.): calmly, considerately, unor-
ganizedly, chaotically, improvisingly. Wisely taking things
one step at the time; seeing where the road leads as events
unfold.
T., and Traor
´
e, S. (2014). Computational protein de-
sign as an optimization problem. Artificial Intelli-
gence, 212:59–79.
Bartz-Beielstein, T., Doerr, C., Bossek, J., Chandrasekaran,
S., Eftimov, T., Fischbach, A., Kerschke, P., Lopez-
Ibanez, M., Malan, K. M., Moore, J. H., Naujoks, B.,
Orzechowski, P., Volz, V., Wagner, M., and Weise, T.
(2020). Benchmarking in optimization: Best practice
and open issues.
Chambolle, A. and Pock, T. (2016). An introduction to
continuous optimization for imaging. Acta Numerica,
25:161–319.
De Jonge, M. (2020). Source code of algorithm used in
the experiment. https://github.com/marleentheyoung/
BScKI Thesis PPA.
de Jonge, M. and van den Berg, D. (2020). Plant propa-
gation parameterization: Offspring & population size.
Evo* 2020, page 19.
Eiben, A. E. and Smit, S. K. (2011). Parameter tuning for
configuring and analyzing evolutionary algorithms.
Swarm and Evolutionary Computation, 1(1):19–31.
Geleijn, R., van der Meer, M., van der Post, Q., and van den
Berg, D. (2019). The plant propagation algorithm on
timetables: First results. EVO* 2019, page 2.
Jamil, M. and Yang, X.-S. (2013). A literature survey
of benchmark functions for global optimization prob-
lems. arXiv preprint arXiv:1308.4008.
Kara, I., Kara, B. Y., and Yetis, M. K. (2007). Energy mini-
mizing vehicle routing problem. In International Con-
ference on Combinatorial Optimization and Applica-
tions, pages 62–71. Springer.
Meier, R., Pippel, M., Brandt, F., Sippl, W., and Baldauf, C.
(2010). Paradocks: a framework for molecular dock-
ing with population-based metaheuristics. Journal of
chemical information and modeling, 50(5):879–889.
Paauw, M. and Van den Berg, D. (2019). Paintings,
polygons and plant propagation. In International
Conference on Computational Intelligence in Music,
Sound, Art and Design (Part of EvoStar), pages 84–
97. Springer.
Puchinger, J. and Raidl, G. R. (2005). Combining meta-
heuristics and exact algorithms in combinatorial opti-
mization: A survey and classification. In International
work-conference on the interplay between natural and
artificial computation, pages 41–53. Springer.
Salhi, A. and Fraga, E. S. (2011). Nature-inspired optimi-
sation approaches and the new plant propagation algo-
rithm.
Selamo
˘
glu, B.
˙
I. and Salhi, A. (2016). The plant propaga-
tion algorithm for discrete optimisation: The case of
the travelling salesman problem. In Nature-inspired
computation in engineering, pages 43–61. Springer.
Sleegers, J. and van den Berg, D. (2020). Plant propagation
& hard hamiltonian graphs. Evo* 2020, page 10.
Smit, S. K. and Eiben, A. E. (2009). Comparing parameter
tuning methods for evolutionary algorithms. In 2009
IEEE congress on evolutionary computation, pages
399–406. IEEE.
ECTA 2020 - 12th International Conference on Evolutionary Computation Theory and Applications
98
S
¨
orensen, K. (2015). Metaheuristics—the metaphor ex-
posed. International Transactions in Operational Re-
search, 22(1):3–18.
Sulaiman, M., Salhi, A., Selamoglu, B. I., and Kirikchi,
O. B. (2014). A plant propagation algorithm for con-
strained engineering optimisation problems. Mathe-
matical problems in engineering, 2014.
Tan, Y. and Zhu, Y. (2010). Fireworks algorithm for opti-
mization. In International conference in swarm intel-
ligence, pages 355–364. Springer.
van den Berg, D., Braam, F., Moes, M., Suilen, E., and Bhu-
lai, S. (2016). Almost squares in almost squares: Solv-
ing the final instance. DATA ANALYTICS 2016, page
81.
Vrielink, W. and van den Berg, D. (2019). Fireworks algo-
rithm versus plant propagation algorithm.
Parameter Sensitivity Patterns in the Plant Propagation Algorithm
99