A Simple Node Ordering Method for the K2 Algorithm based on the Factor Analysis

Vahid Rezaei Tabar


In this paper, we use the Factor Analysis (FA) to determine the node ordering as an input for K2 algorithm in the task of learning Bayesian network structure. For this purpose, we use the communality concept in factor analysis. Communality indicates the proportion of each variable's variance that can be explained by the retained factors. This method is much easier than ordering-based approaches which do explore the ordering space. Because it depends only on the correlation matrix. As well, experimental results over benchmark networks ‘Alarm’ and ‘Hailfinder’ show that our new method has higher accuracy and better degree of data matching.


  1. Abramson, B., Brown, J., Edwards, W., Murphy, A., & Winkler, R. L. (1996). Hailfinder: A Bayesian system for forecasting severe weather. International Journal of Forecasting, 12(1), 57-71.
  2. Beinlich, I. A., Suermondt, H. J., Chavez, R. M., & Cooper, G. F. (1989).The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks (pp. 247-256). Springer Berlin Heidelberg.
  3. Chen, X. W., Anantha, G., & Lin, X. (2008). Improving Bayesian network structure learning with mutual information-based node ordering in the K2 algorithm. IEEE Transactions on Knowledge and Data Engineering, 20(5), 628-640.
  4. Chow, C., & Liu, C. (1968). Approximating discrete probability distributions with dependence trees. IEEE transactions on Information Theory, 14(3), 462-467.
  5. Cooper, G. F., & Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine learning, 9(4), 309-347.
  6. Friedman, N., & Goldszmidt, M. (1998). Learning Bayesian networks with local structure. In Learning in graphical models (pp. 421-459). Springer Netherlands.
  7. Friedman, N., & Koller, D. (2000). Being Bayesian about network structure. In Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence (pp. 201-210). Morgan Kaufmann Publishers Inc.
  8. Geiger, D., Verma, T., & Pearl, J. (1990). Identifying independence in Bayesian networks. Networks, 20(5), 507-534.
  9. Ghahramani, Z. (1998). Learning dynamic Bayesian networks. In Adaptive processing of sequences and data structures (pp. 168-197). Springer Berlin Heidelberg.
  10. Grossman, D., & Domingos, P. (2004, July). Learning Bayesian network classifiers by maximizing conditional likelihood. In Proceedings of the twentyfirst international conference on Machine learning (p. 46). ACM.
  11. Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational research methods, 7(2), 191-205.
  12. Heckerman, D. (1998). A tutorial on learning with Bayesian networks. In Learning in graphical models (pp. 301-354). Springer Netherlands.
  13. Hruschka, E. R., & Ebecken, N. F. (2007). Towards efficient variables ordering for Bayesian networks classifier. Data & Knowledge Engineering,63(2), 258- 269.
  14. Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179- 185.
  15. Hsu, W. H., Guo, H., Perry, B. B., & Stilson, J. A. (2002, July). A Permutation Genetic Algorithm For Variable Ordering In Learning Bayesian Networks From Data. In GECCO (Vol. 2, pp. 383-390).
  16. Jensen, F. V. (1996). An introduction to Bayesian networks (Vol. 210). London: UCL press.
  17. Johnson, R. A., & Wichern, D. W. (2002). Applied multivariate statistical analysis (Vol. 5, No. 8). Upper Saddle River, NJ: Prentice hall.
  18. Kaiser, Henry F. (1992). "On Cliff's formula, the KaiserGuttman rule, and the number of factors." Perceptual and motor skills 74.2: 595-598.
  19. Kim, J. O., & Mueller, C. W. (1978). Factor analysis: Statistical methods and practical issues (Vol. 14). Sage.
  20. Lamma, E., Riguzzi, F., & Storari, S. (2005). Improving the K2 algorithm using association rule parameters. Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU04), 1667-1674.
  21. Larranaga, P., Kuijpers, C. M., Murga, R. H., & Yurramendi, Y. (1996). Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE transactions on systems, man, and cybernetics-part A: systems and humans, 26(4), 487-493.
  22. Leray, P., & Francois, O. (2004). BNT structure learning package: Documentation and experiments. Laboratoire PSI, Universitè et INSA de Rouen, Tech. Rep.
  23. Madigan, D., York, J., & Allard, D. (1995). Bayesian graphical models for discrete data. International Statistical Review/Revue Internationale de Statistique, 215-232.
  24. Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. San Francisco, CA: Morgan Kaufmann.
  25. Perrier, E., Imoto, S., & Miyano, S. (2008). Finding optimal Bayesian network given a superstructure. Journal of Machine Learning Research,9(Oct), 2251-2286.
  26. Powers, D. M. (2011). Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation, Journal of machine Learning technologies, 2(1), 37-63.
  27. Ruiz, C. (2005). Illustration of the K2 algorithm for learning Bayes net structures. Department of Computer Science, WPI.
  28. Romero, T., Larrañaga, P., & Sierra, B. (2004). Learning Bayesian networks in the space of orderings with estimation of distribution algorithms. International Journal of Pattern Recognition and Artificial Intelligence, 18(04), 607-625.
  29. Spirtes, P., Glymour, C., & Scheines, R. (2000). Causation, Prediction, and Search. Adaptive Computation and Machine Learning Series. The MIT Press, 49, 77-78.

Paper Citation

in Harvard Style

Rezaei Tabar V. (2017). A Simple Node Ordering Method for the K2 Algorithm based on the Factor Analysis . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 273-280. DOI: 10.5220/0006095702730280

in Bibtex Style

author={Vahid Rezaei Tabar},
title={A Simple Node Ordering Method for the K2 Algorithm based on the Factor Analysis},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},

in EndNote Style

JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - A Simple Node Ordering Method for the K2 Algorithm based on the Factor Analysis
SN - 978-989-758-222-6
AU - Rezaei Tabar V.
PY - 2017
SP - 273
EP - 280
DO - 10.5220/0006095702730280