loading
Papers Papers/2020

Research.Publish.Connect.

Paper

Authors: Mina Basirat 1 and Peter Roth 2 ; 1

Affiliations: 1 Institute of Computer Graphics and Vision, Graz University of Technology, Austria ; 2 Data Science in Earth Observation, Technical University of Munich, Germany

ISBN: 978-989-758-488-6

ISSN: 2184-4321

Keyword(s): Visual Categorization, Activation Functions, Particle Swarm Optimization.

Abstract: Recently, it has been shown that properly parametrized Leaky ReLU (LReLU) as an activation function yields significantly better results for a variety of image classification tasks. However, such methods are not feasible in practice. Either the only parameter (i.e., the slope of the negative part) needs to be set manually (L*ReLU), or the approach is vulnerable due to the gradient-based optimization and, thus, highly dependent on a proper initialization (PReLU). In this paper, we would like to exploit the benefits of piecewise linear functions, avoiding these problems. To this end, we propose a fully automatic approach to estimate the slope parameter for LReLU from the data. We realize this via Stochastic Optimization, namely Particle Swarm Optimization (PSO): S*ReLU. In this way, we can show that, compared to widely-used activation functions (including PReLU), better results can be obtained on seven different benchmark datasets. Moreover, the results even match those of L*ReLU, where the optimal parameter is estimated in a brute-force manner. In this way, our fully-automatic approach allows for drastically reducing the computational effort. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 18.204.48.69

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Basirat, M. and Roth, P. (2021). S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, ISBN 978-989-758-488-6; ISSN 2184-4321, pages 645-652. DOI: 10.5220/0010338506450652

@conference{visapp21,
author={Mina Basirat. and Peter Roth.},
title={S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP,},
year={2021},
pages={645-652},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010338506450652},
isbn={978-989-758-488-6},
issn={2184-4321},
}

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP,
TI - S*ReLU: Learning Piecewise Linear Activation Functions via Particle Swarm Optimization
SN - 978-989-758-488-6
IS - 2184-4321
AU - Basirat, M.
AU - Roth, P.
PY - 2021
SP - 645
EP - 652
DO - 10.5220/0010338506450652

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.
0123movie.net