A Comparative Analysis of Ensemble and Non-Ensemble Machine Learning Algorithms

Yifei Wang

2024

Abstract

The development of machine learning has led to the design of various algorithms to effectively address complex problems. Among these, both ensemble and non-ensemble methods have attracted significant attention due to their unique advantages and applications. This paper compares the performance of ensemble and non-ensemble machine learning algorithms in terms of accuracy, efficiency, and stability, using two classification datasets. This work evaluates six algorithms: three non-ensemble methods, which include support vector classification, decision tree, and k-nearest neighbors; and three ensemble methods, which include random forest, gradient boosting, and voting. The performance is validated on two tasks: heart attack prediction and mushroom classification. The results indicate that ensemble algorithms, particularly random forest, and gradient boosting, generally achieve higher accuracy and greater stability compared to the non-ensemble decision tree algorithm. However, despite the slight accuracy improvement, ensemble methods tend to be much slower during both the training and prediction phases. Support vector classification is efficient on smaller datasets but exhibits slower performance on larger ones. Additionally, the performance of voting algorithms is highly dependent on the selection of base models. These findings highlight the trade-offs between accuracy, efficiency, and stability when choosing appropriate machine learning algorithms for specific tasks.

Paper Citation

in Harvard Style

Wang Y. (2024). A Comparative Analysis of Ensemble and Non-Ensemble Machine Learning Algorithms. In Proceedings of the 1st International Conference on Modern Logistics and Supply Chain Management - Volume 1: MLSCM; ISBN 978-989-758-738-2, SciTePress, pages 382-386. DOI: 10.5220/0013332500004558

in Bibtex Style

@conference{mlscm24,
author={Yifei Wang},
title={A Comparative Analysis of Ensemble and Non-Ensemble Machine Learning Algorithms},
booktitle={Proceedings of the 1st International Conference on Modern Logistics and Supply Chain Management - Volume 1: MLSCM},
year={2024},
pages={382-386},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013332500004558},
isbn={978-989-758-738-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Modern Logistics and Supply Chain Management - Volume 1: MLSCM
TI - A Comparative Analysis of Ensemble and Non-Ensemble Machine Learning Algorithms
SN - 978-989-758-738-2
AU - Wang Y.
PY - 2024
SP - 382
EP - 386
DO - 10.5220/0013332500004558
PB - SciTePress