loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Ricard Durall 1 ; 2 ; 3 ; Avraam Chatzimichailidis 4 ; 1 ; 2 ; Peter Labus 1 ; 2 and Janis Keuper 2 ; 5

Affiliations: 1 Fraunhofer Center Machine Learning, Germany ; 2 Fraunhofer ITWM, Germany ; 3 IWR, University of Heidelberg, Germany ; 4 Chair for Scientific Computing, TU Kaiserslautern, Germany ; 5 Institute for Machine Learning and Analytics, Offenburg University, Germany

Keyword(s): Generative Adversarial Network, Second-order Optimization, Mode Collapse, Stability, Eigenvalues.

Abstract: Generative adversarial networks (GANs) provide state-of-the-art results in image generation. However, despite being so powerful, they still remain very challenging to train. This is in particular caused by their highly non-convex optimization space leading to a number of instabilities. Among them, mode collapse stands out as one of the most daunting ones. This undesirable event occurs when the model can only fit a few modes of the data distribution, while ignoring the majority of them. In this work, we combat mode collapse using second-order gradient information. To do so, we analyse the loss surface through its Hessian eigenvalues, and show that mode collapse is related to the convergence towards sharp minima. In particular, we observe how the eigenvalues of the G are directly correlated with the occurrence of mode collapse. Finally, motivated by these findings, we design a new optimization algorithm called nudged-Adam (NuGAN) that uses spectral information to overcome mode collapse , leading to empirically more stable convergence properties. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 44.193.77.196

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Durall, R.; Chatzimichailidis, A.; Labus, P. and Keuper, J. (2021). Combating Mode Collapse in GAN Training: An Empirical Analysis using Hessian Eigenvalues. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP; ISBN 978-989-758-488-6; ISSN 2184-4321, SciTePress, pages 211-218. DOI: 10.5220/0010167902110218

@conference{visapp21,
author={Ricard Durall. and Avraam Chatzimichailidis. and Peter Labus. and Janis Keuper.},
title={Combating Mode Collapse in GAN Training: An Empirical Analysis using Hessian Eigenvalues},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP},
year={2021},
pages={211-218},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010167902110218},
isbn={978-989-758-488-6},
issn={2184-4321},
}

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP
TI - Combating Mode Collapse in GAN Training: An Empirical Analysis using Hessian Eigenvalues
SN - 978-989-758-488-6
IS - 2184-4321
AU - Durall, R.
AU - Chatzimichailidis, A.
AU - Labus, P.
AU - Keuper, J.
PY - 2021
SP - 211
EP - 218
DO - 10.5220/0010167902110218
PB - SciTePress