On the Capacity of Hopfield Neural Networks as EDAs for Solving Combinatorial Optimisation Problems

Kevin Swingler

2012

Abstract

Multi-modal optimisation problems are characterised by the presence of either local sub-optimal points or a number of equally optimal points. These local optima can be considered as point attractors for hill climbing search algorithms. It is desirable to be able to model them either to avoid mistaking a local optimum for a global one or to allow the discovery of multiple equally optimal solutions. Hopfield neural networks are capable of modelling a number of patterns as point attractors which are learned from known patterns. This paper shows how a Hopfield network can model a number of point attractors based on non-optimal samples from an objective function. The resulting network is shown to be able to model and generate a number of local optimal solutions up to a certain capacity. This capacity, and a method for extending it is studied.

References

  1. De Jong, K. (2006). Evolutionary computation : a unified approach. MIT Press, Cambridge, Mass.
  2. Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley Professional, 1 edition.
  3. Hertz, J., Krogh, A., and Palmer, R. G. (1991). Introduction to the Theory of Neural Computation. AddisonWesley, New York.
  4. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences USA, 79(8):2554-2558.
  5. Hopfield, J. J. and Tank, D. W. (1985). Neural computation of decisions in optimization problems. Biological Cybernetics, 52:141-152.
  6. Jin, Y. (2005). A comprehensive survey of fitness approximation in evolutionary computation. Soft Computing - A Fusion of Foundations, Methodologies and Applications, 9:3-12.
  7. Kennedy, J. (2001). Swarm intelligence. Morgan Kaufmann Publishers, San Francisco.
  8. Kubota, T. (2007). A higher order associative memory with mcculloch-pitts neurons and plastic synapses. In Neural Networks, 2007. IJCNN 2007. International Joint Conference on, pages 1982 -1989.
  9. McEliece, R., Posner, E., Rodemich, E., and Venkatesh, S. (1987). The capacity of the hopfield associative memory. Information Theory, IEEE Transactions on, 33(4):461 - 482.
  10. Mühlenbein, H. and Paaß, G. (1996). From recombination of genes to the estimation of distributions i. binary parameters. In Voigt, H.-M., Ebeling, W., Rechenberg, I., and Schwefel, H.-P., editors, Parallel Problem Solving from Nature PPSN IV, volume 1141 of Lecture Notes in Computer Science, pages 178-187. Springer Berlin / Heidelberg.
  11. Shakya, S., McCall, J., Brownlee, A., and Owusu, G. (2012). Deum - distribution estimation using markov networks. In Shakya, S. and Santana, R., editors, Markov Networks in Evolutionary Computation, volume 14 of Adaptation, Learning, and Optimization, pages 55-71. Springer Berlin Heidelberg.
  12. Storkey, A. J. and Valabregue, R. (1999). The basins of attraction of a new hopfield learning rule. Neural Netw., 12(6):869-876.
Download


Paper Citation


in Harvard Style

Swingler K. (2012). On the Capacity of Hopfield Neural Networks as EDAs for Solving Combinatorial Optimisation Problems . In Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: ECTA, (IJCCI 2012) ISBN 978-989-8565-33-4, pages 152-157. DOI: 10.5220/0004113901520157


in Bibtex Style

@conference{ecta12,
author={Kevin Swingler},
title={On the Capacity of Hopfield Neural Networks as EDAs for Solving Combinatorial Optimisation Problems},
booktitle={Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: ECTA, (IJCCI 2012)},
year={2012},
pages={152-157},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004113901520157},
isbn={978-989-8565-33-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: ECTA, (IJCCI 2012)
TI - On the Capacity of Hopfield Neural Networks as EDAs for Solving Combinatorial Optimisation Problems
SN - 978-989-8565-33-4
AU - Swingler K.
PY - 2012
SP - 152
EP - 157
DO - 10.5220/0004113901520157