HOW TO LEARN A LEARNING SYSTEM - Automatic Decomposition of a Multiclass Task with Probability Estimates

Cristina Garcia Cifuentes, Marc Sturzel

2010

Abstract

Multiclass classification is the core issue of many pattern recognition tasks. In some applications, not only the predicted class is important but also the confidence associated to the decision. This paper presents a complete framework for multiclass classification that recovers probability estimates for each class. It focuses on the automatic configuration of the system so that no user-provided tuning is needed. No assumption about the nature of data or the number of classes is done either, resulting in a generic system. A suitable decomposition of the original multiclass problem into several biclass problems is automatically learnt from data. State-of-the-art biclass classifiers are optimized and their reliabilities are assessed and considered in the combination of the biclass predictions. Quantitative evaluations on different datasets show that the automatic decomposition and the reliability assessment of our system improve the classification rate compared to other schemes, as well as it provides probability estimates of each class. Besides, it simplifies considerably the user effort to use the framework in a specific problem, since it adapts automatically.

References

  1. Allwein, E.L., Schapire, R.E., Singer, Y., 2000. Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research, 1:113-141.
  2. Asuncion, A. Newman, D.J., 2007. UCI Machine Learning Repository. Irvine, CA: University of California, [http://www.ics.uci.edu/mlearn/MLRepository.html] Chawala N. V., Bowyer K. W., Hall L. O., Kegelmeyer W. P., 2002. SMOTE: Synthetic Minority Oversampling Technique. Journal of Artifical Intelligence Research 16, 321-357.
  3. Dekel, O., Singer, Y., 2003. Multiclass learning by probabilistic embeddings. In Advances in Neural Information Processing Systems 15, 945-952.
  4. Dietterich, T. G., Bakiri, G., 1995. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2:263-286.
  5. Guruswami, V., Sahai, A., 1999. Multiclass learning, boosting, and error-correcting codes. In COLT 7899: Proceedings of the twelfth annual conference on Computational learning theory, 145-155. ACM Press/ Hastie, T., Tibshirani, R., 1998. Classification by pairwise coupling. In The Annals of Statistics, 26(2):451-471.
  6. Kong, E. G., Dietterich, T. G., 1997. Probability estimation using error-correcting output coding. In International Conference on Artifical Intelligence and Soft Computing.
  7. Maskell, S., 2008. A Bayesian approach to fusing uncertain, imprecise and conflicting information. Information Fusion, 9(2):259-277.
  8. Piat, E., Meizel, D., 1997. A probabilistic framework for believes fusion. Traitement du Signal, 14(5):485-498.
  9. Sebag, M., Azé J., Lucas N., 2003. ROC-based Evolutionary Learning: Application to Medical Data Mining. Artificial Evolution'03, Springer 384-396.
  10. Zadrozny, B., 2001. Reducing multiclass to binary by coupling probability estimates. In Advances in Neural Information Processing Systems 14, 1041-1048.
  11. Zadrozny, B., Elkan, C., 2002. Transforming classifier scores into accurate multiclass probability estimates. In KDD 7802: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 694-699.
  12. Zou, H., Zhu, J., Hastie, T., 2005. The margin vector, admissible loss and multi-class margin-based classifiers. Technical report, University of Minnesota.
Download


Paper Citation


in Harvard Style

Garcia Cifuentes C. and Sturzel M. (2010). HOW TO LEARN A LEARNING SYSTEM - Automatic Decomposition of a Multiclass Task with Probability Estimates . In Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-674-021-4, pages 589-594. DOI: 10.5220/0002725005890594


in Bibtex Style

@conference{icaart10,
author={Cristina Garcia Cifuentes and Marc Sturzel},
title={HOW TO LEARN A LEARNING SYSTEM - Automatic Decomposition of a Multiclass Task with Probability Estimates},
booktitle={Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2010},
pages={589-594},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002725005890594},
isbn={978-989-674-021-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - HOW TO LEARN A LEARNING SYSTEM - Automatic Decomposition of a Multiclass Task with Probability Estimates
SN - 978-989-674-021-4
AU - Garcia Cifuentes C.
AU - Sturzel M.
PY - 2010
SP - 589
EP - 594
DO - 10.5220/0002725005890594