Penalized Maximum Likelihood Boosting with Predictive Measures, 41-67
Ruey S. Tsay and Tomohiro Ando
In this paper we use the penalized maximum likelihood and information
criteria to propose a new boosting algorithm for various statistical models,
including linear regression, generalized linear, and multi-class classification
models. In contrast to previous studies, where the empirical goodness-of-fit
measures were often used for model updating, information criteria, as a
predictive measure of a model, are employed to select a model in each iteration
of the proposed algorithms. In addition, the proposed algorithms select the
smoothing parameter in each iteration whereas previous methods fixed the
parameter for all iterations.
We show that the penalized maximum likelihood L2 boosting is consistent for high-dimensional linear models under the conditions that (a) the true underlying regression function is sparse and (b) the number of predictor variables is allowed to grow exponentially. We then demonstrate the proposed boosting algorithms using both simulated and real data. Comparison with some existing methods shows that the proposed boosting algorithms work well.