Evaluating Learning Potential with Internal States in Deep Neural Networks

Shogo Takasaki, Shuichi Enokida

2024

Abstract

Deploying deep learning models on small-scale computing devices necessitates considering computational resources. However, reducing the model size to accommodate these resources often results in a trade-off with accuracy. The iterative process of training and validating to optimize model size and accuracy can be inefficient. A potential solution to this dilemma is the extrapolation of learning curves, which evaluates a model’s potential based on initial learning curves. As a result, it is possible to efficiently search for a network that achieves a balance between accuracy and model size. Nonetheless, we posit that a more effective approach to analyzing the latent potential of training models is to focus on the internal state, rather than merely relying on the validation scores. In this vein, we propose a module dedicated to scrutinizing the network’s internal state, with the goal of automating the optimization of both accuracy and network size. Specifically, this paper delves into analyzing the latent potential of the network by leveraging the internal state of the Long Short-Term Memory (LSTM) in a traffic accident prediction network.

Paper Citation

in Harvard Style

Takasaki S. and Enokida S. (2024). Evaluating Learning Potential with Internal States in Deep Neural Networks. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 317-324. DOI: 10.5220/0012298500003660

in Bibtex Style

@conference{visapp24,
author={Shogo Takasaki and Shuichi Enokida},
title={Evaluating Learning Potential with Internal States in Deep Neural Networks},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2024},
pages={317-324},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012298500003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Evaluating Learning Potential with Internal States in Deep Neural Networks
SN - 978-989-758-679-8
AU - Takasaki S.
AU - Enokida S.
PY - 2024
SP - 317
EP - 324
DO - 10.5220/0012298500003660
PB - SciTePress