Linear or smooth? Enhanced model choice in boosting via deselection of base-learners

• Published in: Statistical modelling 2023-10, Vol.23 (5-6), p.441-455
• Main Authors: Mayr, Andreas, Wistuba, Tobias, Speller, Jan, Gude, Francisco, Hofner, Benjamin
• Format: Article
• Language: English
• ISSN: 1471-082X; 1477-0342
• DOI: 10.1177/1471082X231170045

The specification of a particular type of effect (e.g., linear or non-linear) of a covariate in a regression model can be either based on graphical assessment, subject matter knowledge or also on data-driven model choice procedures. For the latter variant, we present a boosting approach that is available for a huge number of different model classes. Boosting is an indirect regularization technique that leads to variable selection and can easily incorporate also non-linear or smooth effects. Furthermore, the algorithm can be adapted in a way to automatically select whether to model a continuous variable with a smooth or a linear effect. We enhance this model choice procedure by trying to compensate the inherent bias towards the more complex effect by incorporating a pragmatic and simple deselection technique that was originally implemented for enhanced variable selection. We illustrate our approach in the analysis of T3 thyroid hormone levels from a larger Galician cohort and investigate its performance in a simulation study.