Heterogeneous AdaBoost with stochastic algorithm selection

Abstract

We propose an ensemble learning algorithm based on AdaBoost and employing heterogeneous algorithms with a stochastic process for algorithm selection. Diversity is an important factor in ensemble learning and AdaBoost creates diversity by manipulating training data sets. However, we observe that AdaBoost generates training data sets of low diversity in later iterations. Some researchers suggest the employment of heterogeneous algorithms in ensemble learning to achieve better diversity. Following the idea, we extend AdaBoost and propose an algorithm that employs different base learning algorithms in different iterations. The most distinguishing feature of our algorithm is that it selects algorithms using a stochastic process where their earlier performance is considered. The results from experiments on several data sets show the utility of our algorithm: It could outperform AdaBoost on 22 to 33, depending on settings, out of 40 data sets considered in experiments. © 2017 ACM.