New quantile based ridge M-estimator for linear regression models with multicollinearity and outliers

• Published in: Communications in statistics. Simulation and computation 2023-04, Vol.52 (4), p.1417-1434
• Main Authors: Suhail, Muhammad, Chand, Sohail, Aslam, Muhammad
• Format: Article
• Language: English
• Subjects: Errors; Estimators; Least squares; M-estimator; Mean squared error; Multicollinearity; Normal distribution; Outliers; Parameters; Regression analysis; Regression models; Ridge parameter; Ridge regression; Signal to noise ratio; Statistical analysis; Variance
• ISSN: 0361-0918; 1532-4141
• DOI: 10.1080/03610918.2021.1884715

The ordinary least squares and ridge regression estimators in a multiple linear regression model with multicollinearity and y-direction outliers lead to unfavorable results. In order to mitigate such situation, the available literature provides few ridge M-estimators to get precise estimates. The ridge parameter, k, plays a vital role in a bias-variance tradeoff for these estimators. However, for high signal-to-noise ratio and multicollinearity with y-direction outliers, the available methods may not perform well in terms of their mean squared error. In this article, we propose a new quantile based ridge M-estimator. The new estimator gives an automated choice of quantile probability of ridge parameter according to the level of noise and multicollinearity. Based on a simulation study, the new estimator outperforms the ordinary least square estimator, ridge estimator, and other considered ridge M-estimators especially for high multicollinearity, significant error variance, and y-direction outliers. Besides normal distribution, new estimator also performs well for heavy-tailed error distribution. Finally, two real-life examples are used to illustrate the application of the proposed estimator.