loading
Documents

Research.Publish.Connect.

Paper

Authors: Tome Eftimov 1 ; Peter Korošec 2 and Barbara Koroušić Seljak 3

Affiliations: 1 Jožef Stefan Institute and Jožef Stefan International Postgraduate School, Slovenia ; 2 Jožef Stefan Institute, Faculty of Mathematics and Natural Science and Information Technologies, Slovenia ; 3 Jožef Stefan Institute, Slovenia

ISBN: 978-989-758-274-5

Keyword(s): Statistical Comparison, Single Objective Functions, Deep Statistics, Stochastic Optimization Algorithms.

Abstract: Deep Statistical Comparison (DSC) is a recently proposed approach for the statistical comparison of meta-heuristic stochastic algorithms for single-objective optimization. The main contribution of the DSC is a ranking scheme, which is based on the whole distribution, instead of using only one statistic, such as average or median, which are commonly used. Contrary to common approach, the DSC gives more robust statistical results, which are not affected by outliers or misleading ranking scheme. The DSC ranking scheme uses a statistical test for comparing distributions in order to rank the algorithms. DSC was tested using the two-sample Kolmogorov-Smirnov (KS) test. However, distributions can be compared using different criteria, statistical tests. In this paper, we analyze the behavior of the DSC using two different criteria, the two-sample Kolmogorov-Smirnov (KS) test and the Anderson-Darling (AD) test. Experimental results from benchmark tests consisting of single-objective problems, show that both criteria behave similarly. However, when algorithms are compared on a single problem, it is better to use the AD test because it is more powerful and can better detect differences than the KS test when the distributions vary in shift only, in scale only, in symmetry only, or have the same mean and standard deviation but differ on the tail ends only. This influence is not emphasized when the approach is used for multiple-problem analysis. (More)

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 18.206.16.123

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Eftimov, T.; Korošec, P. and Koroušić Seljak, B. (2017). The Behavior of Deep Statistical Comparison Approach for Different Criteria of Comparing Distributions.In Proceedings of the 9th International Joint Conference on Computational Intelligence - Volume 1: IJCCI, ISBN 978-989-758-274-5, pages 73-82. DOI: 10.5220/0006499900730082

@conference{ijcci17,
author={Tome Eftimov. and Peter Korošec. and Barbara Koroušić Seljak.},
title={The Behavior of Deep Statistical Comparison Approach for Different Criteria of Comparing Distributions},
booktitle={Proceedings of the 9th International Joint Conference on Computational Intelligence - Volume 1: IJCCI,},
year={2017},
pages={73-82},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006499900730082},
isbn={978-989-758-274-5},
}

TY - CONF

JO - Proceedings of the 9th International Joint Conference on Computational Intelligence - Volume 1: IJCCI,
TI - The Behavior of Deep Statistical Comparison Approach for Different Criteria of Comparing Distributions
SN - 978-989-758-274-5
AU - Eftimov, T.
AU - Korošec, P.
AU - Koroušić Seljak, B.
PY - 2017
SP - 73
EP - 82
DO - 10.5220/0006499900730082

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.