A Two-Step Feature Selection Procedure to Handle High-Dimensional Data in Regression Problems

A wide range of fields are interested in high-dimensional data, including finance, tomography, genetics, etc. Feature selection facilitates the handling of this type of data in Machine Learning. This improves prediction performance by separating relevant variables from irrelevant ones. In this paper, we propose a two-step feature selection method for regression. An initial feature space is reduced by feature screening. To remove extra noise and irrelevant variables, the resulting reduced dataset is fitted to an elastic net model. The proposed method improves elastic net models' prediction results. This is demonstrated in this paper through simulations and real datasets. We demonstrate our method's application to gene co-expression data analysis using the Eyedata dataset.