Authors:
Csanád Sándor
1
;
2
;
Szabolcs Pável
1
;
2
and
Lehel Csató
1
Affiliations:
1
Faculty of Mathematics and Informatics, Babeş-Bolyai University, Kogălniceanu 1, Cluj-Napoca, Romania
;
2
Robert Bosch SRL, Someşului 14, Cluj-Napoca, Romania
Keyword(s):
Neural Network Pruning, Filter Pruning, Structured Pruning, Neural Network Acceleration.
Abstract:
Neural network pruning is an effective way to reduce memory- and time requirements in most deep neural network architectures. Recently developed pruning techniques can remove individual neurons or entire filters from convolutional neural networks, making these “slim” architectures more robust and more resource- efficient. In this paper, we present a simple yet effective method that assigns probabilities to the network units – to filters in convolutional layers and to neurons in fully connected layers – and prunes them based on these values. The probabilities are learned by maximizing the expected value of a score function – calculated from the accuracy – that ranks the network when different units are tuned off. Gradients of the probabilities are estimated using Monte Carlo gradient estimation. We conduct experiments on the CIFAR-10 dataset with a small VGG-like architecture as well as on the lightweight version of the ResNet architecture. The results show that our pruning method has
comparable results with different state-of-the-art algorithms in terms of parameter and floating point operation reduction. In case of the ResNet-110 architecture, our pruning method removes 72.53% of the floating point operations and 68.89% of the parameters, that marginally surpasses the result of existing pruning methods.
(More)