Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?

Daniel Lehmann, Marc Ebner

Abstract

Before a neural network can be trained the network weights have to be initialized somehow. If a model is trained from scratch, current approaches for weight initialization are based on random values. In this work we examine another approach to initialize the weights of convolutional neural network models for image classification. Our approach relies on presetting the weights of convolutional layers based on information given in the training images. To initialize the weights of convolutional layers we use small patches extracted from the training images to preset the filters of the convolutional layers. Experiments conducted on the MNIST, CIFAR-10 and CIFAR-100 dataset show that using image patches for the network initialization performs similar to state-of-the-art initialization approaches. The advantage is that our approach is more robust with respect to the learning rate. When a suboptimal value for the learning rate is used for training, our approach performs slightly better than current approaches. As a result, information given in the training images seems to be useful for network initialization resulting in a more robust training process.

Download


Paper Citation


in Harvard Style

Lehmann D. and Ebner M. (2021). Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?.In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, ISBN 978-989-758-488-6, pages 346-353. DOI: 10.5220/0010206603460353


in Bibtex Style

@conference{visapp21,
author={Daniel Lehmann and Marc Ebner},
title={Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP,},
year={2021},
pages={346-353},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010206603460353},
isbn={978-989-758-488-6},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP,
TI - Are Image Patches Beneficial for Initializing Convolutional Neural Network Models?
SN - 978-989-758-488-6
AU - Lehmann D.
AU - Ebner M.
PY - 2021
SP - 346
EP - 353
DO - 10.5220/0010206603460353