loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Martin Becker ; Jens Lippel and Thomas Zielke

Affiliation: University of Applied Sciences Düsseldorf, Münsterstr. 156, 40476, Düsseldorf and Germany

Keyword(s): Deep Neural Networks, Learning Process Visualization, Machine Learning, Numerical Optimization, Gradient Descent Methods.

Related Ontology Subjects/Areas/Topics: Abstract Data Visualization ; Computer Vision, Visualization and Computer Graphics ; General Data Visualization ; Information and Scientific Visualization ; Visual Data Analysis and Knowledge Discovery ; Visual Representation and Interaction

Abstract: We present an approach to visualizing gradient descent methods and discuss its application in the context of deep neural network (DNN) training. The result is a novel type of training error curve (a) that allows for an exploration of each individual gradient descent iteration at line search level; (b) that reflects how a DNN’s training error varies along each of the descent directions considered; (c) that is consistent with the traditional training error versus training iteration view commonly used to monitor a DNN’s training progress. We show how these three levels of detail can be easily realized as the three stages of Shneiderman’s Visual Information Seeking Mantra. This suggests the design and development of a new interactive visualization tool for the exploration of DNN learning processes. We present an example that showcases a conceivable interactive workflow when working with such a tool. Moreover, we give a first impression of a possible DNN hyperparameter analysis.

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 18.208.203.36

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Becker, M.; Lippel, J. and Zielke, T. (2019). Gradient Descent Analysis: On Visualizing the Training of Deep Neural Networks. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - IVAPP; ISBN 978-989-758-354-4; ISSN 2184-4321, SciTePress, pages 338-345. DOI: 10.5220/0007583403380345

@conference{ivapp19,
author={Martin Becker. and Jens Lippel. and Thomas Zielke.},
title={Gradient Descent Analysis: On Visualizing the Training of Deep Neural Networks},
booktitle={Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - IVAPP},
year={2019},
pages={338-345},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007583403380345},
isbn={978-989-758-354-4},
issn={2184-4321},
}

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - IVAPP
TI - Gradient Descent Analysis: On Visualizing the Training of Deep Neural Networks
SN - 978-989-758-354-4
IS - 2184-4321
AU - Becker, M.
AU - Lippel, J.
AU - Zielke, T.
PY - 2019
SP - 338
EP - 345
DO - 10.5220/0007583403380345
PB - SciTePress