ON THE VC-DIMENSION OF UNIVARIATE DECISION TREES

Olcay Taner Yildiz

2012

Abstract

In this paper, we give and prove lower bounds of the VC-dimension of the univariate decision tree hypothesis class. The VC-dimension of the univariate decision tree depends on the VC-dimension values of its subtrees and the number of inputs. In our previous work (Aslan et al., 2009), we proposed a search algorithm that calculates the VC-dimension of univariate decision trees exhaustively. Using the experimental results of that work, we show that our VC-dimension bounds are tight. To verify that the VC-dimension bounds are useful, we also use them to get VC-generalization bounds for complexity control using SRM in decision trees, i.e., pruning. Our simulation results shows that SRM-pruning using the VC-dimension bounds finds trees that are more accurate as those pruned using cross-validation.

References

  1. Aslan, O., Yildiz, O. T., and Alpaydin, E. (2009). Calculating the vc-dimension of decision trees. In Proceedings of the 24th International Symposium on Computer and Information Sciences, pages 193-198.
  2. Bishop, C. M. (1995). Neural Networks for Pattern Recognition. Oxford University Press.
  3. Blake, C. and Merz, C. (2000). UCI repository of machine learning databases.
  4. Cherkassky, V. and Mulier, F. (1998). Learning From Data. John Wiley and Sons.
  5. Mansour, Y. (1997). Pessimistic decision tree pruning based on tree size. In Proceedings of the 14th international conference on Machine learning.
  6. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1:81-106.
  7. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann, San Meteo, CA.
  8. Simon, H. U. (1991). The vapnik-chervonenkis dimension of decision trees with bounded rank. Information Processing Letters, 39(3):137-141.
  9. Vapnik, V. (1995). The Nature of Statistical Learning Theory. Springer Verlag, New York.
  10. Yildiz, O. T. and Alpaydin, E. (2001). Omnivariate decision trees. IEEE Transactions on Neural Networks, 12(6):1539-1546.
Download


Paper Citation


in Harvard Style

Taner Yildiz O. (2012). ON THE VC-DIMENSION OF UNIVARIATE DECISION TREES . In Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-8425-98-0, pages 205-210. DOI: 10.5220/0003777202050210


in Bibtex Style

@conference{icpram12,
author={Olcay Taner Yildiz},
title={ON THE VC-DIMENSION OF UNIVARIATE DECISION TREES},
booktitle={Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2012},
pages={205-210},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003777202050210},
isbn={978-989-8425-98-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - ON THE VC-DIMENSION OF UNIVARIATE DECISION TREES
SN - 978-989-8425-98-0
AU - Taner Yildiz O.
PY - 2012
SP - 205
EP - 210
DO - 10.5220/0003777202050210