Unsupervised Feature Learning using Self-organizing Maps

Marco Vanetti, Ignazio Gallo, Angelo Nodari

2013

Abstract

In recent years a great amount of research has focused on algorithms that learn features from unlabeled data. In this work we propose a model based on the Self-Organizing Map (SOM) neural network to learn features useful for the problem of automatic natural images classification. In particular we use the SOM model to learn single-layer features from the extremely challenging CIFAR-10 dataset, containing 60.000 tiny labeled natural images, and subsequently use these features with a pyramidal histogram encoding to train a linear SVM classifier. Despite the large number of images, the proposed feature learning method requires only few minutes on an entry-level system, however we show that a supervised classifier trained with learned features provides significantly better results than using raw pixels values or other handcrafted features designed specifically for image classification. Moreover, exploiting the topological property of the SOM neural network, it is possible to reduce the number of features and speed up the supervised training process combining topologically close neurons, without repeating the feature learning process.

References

  1. Bengio, Y., Lamblin, P., Popovici, D., and Larochelle, H. (2007). Greedy Layer-Wise Training of Deep Networks. In Neural Information Processing Systems, pages 153-160.
  2. Bosch, A., Zisserman, A., and Muoz, X. (2007a). Image Classification using Random Forests and Ferns. In International Conference on Computer Vision, pages 1- 8.
  3. Bosch, A., Zisserman, A., and Muoz, X. (2007b). Representing shape with a spatial pyramid kernel. In Conference on Image and Video Retrieval, pages 401-408.
  4. Coates, A., Lee, H., and Ng, A. Y. (2011). An analysis of single-layer networks in unsupervised feature learning. In AISTATS.
  5. Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine Learning, 20:273-297.
  6. Csurka, G., Dance, C. R., Fan, L., Willamowski, J., and Bray, C. (2004). Visual Categorization with Bags of Keypoints. In European Conference on Computer Vision.
  7. Gersho, A. and Gray, R. M. (1992). Vector quantization and signal compression.
  8. Haykin, S. (1999). Neural networks a comprehensive foundation.
  9. Hinton, G. E., Osindero, S., and Teh, Y.-W. (2006). A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 18:1527-1554.
  10. Hinton, G. E. and Salakhutdinov, R. R. (2006). Reducing the Dimensionality of Data with Neural Networks. Science, 313:504-507.
  11. Hyvarinen, A. and Oja, E. (2000). Independent component analysis: algorithms and applications. Neural Networks, 13:411-430.
  12. Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. (2009). What is the best multi-stage architecture for object recognition? In International Conference on Computer Vision, pages 2146-2153.
  13. Kohonen, T. (1990). The self-organizing map. Proceedings of the IEEE, 78:1464-1480.
  14. Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical report.
  15. Lazebnik, S., Schmid, C., and Ponce, J. (2006). Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. In Computer Vision and Pattern Recognition, volume 2, pages 2169- 2178.
  16. Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., Dean, J., and Ng, A. (2012). Building highlevel features using large scale unsupervised learning. In International Conference on Machine Learning.
  17. Lee, H., Battle, A., Raina, R., and Ng, A. Y. (2006). Efficient sparse coding algorithms. In Neural Information Processing Systems, pages 801-808.
  18. Olshausen, B. A. and Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381:607-609.
  19. Raina, R., Madhavan, A., and Ng, A. Y. (2009). Large-scale deep unsupervised learning using graphics processors. In International Conference on Machine Learning, pages 110-880.
  20. Vanetti, M., Gallo, I., and Nodari, A. (2012). Unsupervised self-organizing texture descriptor. In Images: Fundamentals, Methods and Applications (CompIMAGE2012).
Download


Paper Citation


in Harvard Style

Vanetti M., Gallo I. and Nodari A. (2013). Unsupervised Feature Learning using Self-organizing Maps . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2013) ISBN 978-989-8565-47-1, pages 596-601. DOI: 10.5220/0004210305960601


in Bibtex Style

@conference{visapp13,
author={Marco Vanetti and Ignazio Gallo and Angelo Nodari},
title={Unsupervised Feature Learning using Self-organizing Maps},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2013)},
year={2013},
pages={596-601},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004210305960601},
isbn={978-989-8565-47-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2013)
TI - Unsupervised Feature Learning using Self-organizing Maps
SN - 978-989-8565-47-1
AU - Vanetti M.
AU - Gallo I.
AU - Nodari A.
PY - 2013
SP - 596
EP - 601
DO - 10.5220/0004210305960601