EVALUATING PREDICTION STRATEGIES IN AN ENHANCED META-LEARNING FRAMEWORK

Silviu Cacoveanu, Camelia Lemnaru, Rodica Potolea

2010

Abstract

Finding the best learning strategy for a new domain/problem can prove to be an expensive and time-consuming process even for the experienced analysts. This paper presents several enhancements to a meta-learning framework we have previously designed and implemented. Its main goal is to automatically identify the most reliable learning schemes for a particular problem, based on the knowledge acquired about existing data sets, while minimizing the work done by the user but still offering flexibility. The main enhancements proposed here refer to the addition of several classifier performance metrics, including two original metrics, for widening the evaluation criteria, the addition of several new benchmark data sets for improving the outcome of the neighbor estimation step, and the integration of complex prediction strategies. Systematic evaluations have been performed to validate the new context of the framework. The analysis of the results revealed new research perspectives in the meta-learning area.

References

  1. Aha D. W., 1992, Generalizing from Case Studies: A Case Study, Proceedings of the Ninth International Conference on Machine Learning, pp. 1 -10
  2. Bensusan H., Giraud-Carrier C., Kennedy C. J., 2000, A Higher-Order Approach to Meta-Learning, proceedings of the ECML-2000 Workshop on MetaLearning: Building Automatic Advice Strategies for Model Selection and Method Combination, pp. 33-42
  3. Brazdil P. B., Soares C., Da Costa J. P., 2003, Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results, Machine Learning 50, pp. 251-277
  4. Cacoveanu S., Vidrighin C., Potolea R., 2009, Evolutional Meta-Learning Framework for Automatic Classifier Selection, proceedings of the 5th International Conference on Intelligent Computer Communication and Processing, Cluj-Napoca, pp. 27-30
  5. Caruana R., Niculescu-Mizil A., 2004, Data Mining in Metric Space: An Empirical Analysis of Supervised Learning Performance Criteria, Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 69-78
  6. Giraud-Carrier C., Bensusan H., 2000, Discovering Task Neighbourhoods Through Landmark Learning Performances, Proceedings of the Fourth European Conference of Principles and Practice of Knowledge Discovery in Databases, pp. 325-330
  7. Japkowicz N., Stephen S., 2002, The Class Imbalance Problem: A Systematic Study, Intelligent Data Analysis, Volume 6, Number 5, pp. 429-450
  8. Kalousis A., 2002, Algorithm Selection via Meta_Learning, PhD Thesis, Faculte des sciences de l'Universite de Geneve
  9. Linder C., Studer R., 1999, Support for Algorithm Selection with a CBR Approach, Proceedings of the 16th International Conference on Machine Learning, pp. 418-423
  10. Michie D., Spiegelhalter D. J., Taylor C. C., 1994, Machine Learning. Neural and Statistical Classification, Ellis Horwood Series in Artificial Intelligence
  11. Niculescu-Mizil A., et al, 2009, Winning the KDD Cup Orange Challenge with Ensemble Selection, JMLR Workshop and Conference Proceedings 7
  12. Rendel L., Seshu R., Tcheng D., 1987, Layered concept learning and dynamically variable bias management, 10th Internatinal Joint Conf. on AI, pp. 308-314
  13. Schaffner C., 1993, Selecting a classification method by cross validation, Machine Learning 13, pp. 135-143
  14. Sleeman D., Rissakis M., Craw S., Graner N., Sharma S., 1995, Consultant-2: Pre and post-processing of machine learning applications, International Journal of Human Computer Studies, pp. 43-63
  15. UCI Machine Learning Data Repository, http://archive.ics.uci.edu/ml/, last accessed Jan. 2010
  16. Vilalta R., Giraud-Carrier C., Brazdil P., Soares C., 2004, Using Meta-Learning to Support Data Mining, Iternational Journal of Computer Science & Applications, pp. 31-45
  17. Witten I. H., Frank E., 2005, Data Mining: Practical Machine Learning Tools and Techniques, 2nd edition, Morgan Kaufmann Publishers, Elsevier Inc.
Download


Paper Citation


in Harvard Style

Cacoveanu S., Lemnaru C. and Potolea R. (2010). EVALUATING PREDICTION STRATEGIES IN AN ENHANCED META-LEARNING FRAMEWORK . In Proceedings of the 12th International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-8425-05-8, pages 148-156. DOI: 10.5220/0002975401480156


in Bibtex Style

@conference{iceis10,
author={Silviu Cacoveanu and Camelia Lemnaru and Rodica Potolea},
title={EVALUATING PREDICTION STRATEGIES IN AN ENHANCED META-LEARNING FRAMEWORK},
booktitle={Proceedings of the 12th International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2010},
pages={148-156},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002975401480156},
isbn={978-989-8425-05-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - EVALUATING PREDICTION STRATEGIES IN AN ENHANCED META-LEARNING FRAMEWORK
SN - 978-989-8425-05-8
AU - Cacoveanu S.
AU - Lemnaru C.
AU - Potolea R.
PY - 2010
SP - 148
EP - 156
DO - 10.5220/0002975401480156