##### Personal tools
You are here: Home Robust Ensemble Learning
##### Document Actions

G Rätsch, B Schölkopf, A J Smola, S Mika, T Onoda, and K Müller (2000)

# Robust Ensemble Learning

In: Proc. of the NIPS*Workshop on Large Margin Classifiers: Advances in Large Margin Classifiers, ed. by A. J. Smola, P. L. Bartlett, B. Schölkopf and D. Schuurmans, pp. 207-219, MIT Press, Cambridge, MA.

AdaBoost and other ensemble methods have successfully been applied to a number of classification tasks, seemingly defying problems of overfitting. AdaBoost performs gradient descent in an error function with respect to the margin, asymptotically concentrating on the patterns which are hardest to learn. For noisy problems, however, this can be disadvantageous. Indeed, theoretical analysis has shown that the margin distribution, as opposed to just the minimal margin, plays a crucial role in understanding this phenomenon. Loosely speaking, some outliers should be tolerated if this has the benefit of substantially increasing the margin on the remaining points. We propose new boosting algorithms which, similar to $\nu$-Support-Vector Classification, allows us for the possibility of a pre-specified fraction of points to lie in the margin area or even on the wrong side of the decision boundary. Unlike other regularized boosting algorithms, this gives a nicely interpretable way of controlling the trade-off between minimizing the training error and capacity.
similarly appeared in the Journal of the Japanese Society of AI, 2001