MULTI-CLASS FROM BINARY - Divide to conquer

Anderson Rocha, Siome Goldenstein



Several researchers have proposed effective approaches for binary classification in the last years. We can easily extend some of those techniques to multi-class. Notwithstanding, some other powerful classifiers (e.g., SVMs) are hard to extend to multi-class. In such cases, the usual approach is to reduce the multi-class problem complexity into simpler binary classification problems (divide-and-conquer). In this paper, we address the multi-class problem by introducing the concept of affine relations among binary classifiers (dichotomies), and present a principled way to find groups of high correlated base learners. Finally, we devise a strategy to reduce the number of required dichotomies in the overall multi-class process.


  1. Allwein, E., Shapire, R., and Singer, Y. (2000). Reducing multi-class to binary: A unifying approach for margin classifiers. JMLR, 1(1):113-141.
  2. Anand, R., Mehrotra, K., Mohan, C., and Ranka, S. (1995). Efficient classification for multi-class problems using modular neural networks. TNN, 6(1):117-124.
  3. Athisos, V., Stefan, A., Yuan, Q., and Sclaroff, S. (2007). Classmap: Efficient multiclass recognition via embeddings. In ICCV.
  4. Bishop, C. (2006). Pattern Recognition and Machine Learning. Springer, 1 edition.
  5. Clark, P. and Boswell, R. (1991). Rule induction with CN2: Some improvements. In EWSL, pages 151-163.
  6. Crammer, K. and Singer, Y. (2002). On the learnability and design of output codes for multi-class problems. JMLR, 47(2-3):201-233.
  7. Dietterich, T. and Bakiri, G. (1996). Solving multi-class problems via ECOC. JAIR, 2(1):263-286.
  8. Klautau, A., Jevtic, N., and Orlitsky, A. (2004). On nearestneighbor ECOC with application to all-pairs multiclass support vector machines. JMLR, 4(1):1-15.
  9. Moreira, M. and Mayoraz, E. (1998). Improved pairwise coupling classification with correcting classifiers. In ECML.
  10. Narasimhamrthy, A. (2005). Theoretical bounds of majority voting performance for a binary classification problem. TPAMI, 27(12):1988-1995.
  11. Passerini, A., Pontil, M., and Frasconi, P. (2004). New results on error correcting output codes of kernel machines. TNN, 15(1):45-54.
  12. Pedrajas, N. and Boyer, D. (2006). Improving multi-class pattern recognition by the combination of two strategies. TPAMI, 28(6):1001-1006.
  13. Platt, J., Christiani, N., and Taylor, J. (1999). Large margin dags for multi-class classification. In NIPS, pages 547-553.
  14. Pujol, O., Radeva, P., and Vitria, J. (2006). Discriminant ECOC: A heuristic method for application dependent design of ECOC. TPAMI, 28(6):1007-1012.
  15. Stehling, R., Nascimento, M., and Falca˜o, A. (2002). A compact and efficient image retrieval approach based on border/interior pixel classification. In CIKM, pages 102-109.
  16. Takenouchi, T. and Ishii, S. (2007). Multi-class classification as a decoding problem. In FOCI, pages 470-475.
  17. Windeatt, T. and Ghaderi, R. (2003). Coding and decoding strategies for multi-class learning problems. Information Fusion, 4(1):11-21.
  18. Young, C., Yen, C., Pao, Y., and Nagurka, M. (2006). Oneclass-at-time removal sequence planning method for multi-class problems. TNN, 17(6):1544-1549.

Paper Citation

in Harvard Style

Rocha A. and Goldenstein S. (2009). MULTI-CLASS FROM BINARY - Divide to conquer . In Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009) ISBN 978-989-8111-69-2, pages 323-330. DOI: 10.5220/0001777803230330

in Bibtex Style

author={Anderson Rocha and Siome Goldenstein},
title={MULTI-CLASS FROM BINARY - Divide to conquer},
booktitle={Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)},

in EndNote Style

JO - Proceedings of the Fourth International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2009)
TI - MULTI-CLASS FROM BINARY - Divide to conquer
SN - 978-989-8111-69-2
AU - Rocha A.
AU - Goldenstein S.
PY - 2009
SP - 323
EP - 330
DO - 10.5220/0001777803230330