Another Look at High-Dimensional Regression in Principal Components Space and The Blessing of Dimensionality

Benefits of exploiting sparsity in the space of principal components have been well documented both empirically and theoretically (Lang and Zou, 2020; Silin and Fan, 2022). In this paper, we further reveal another unexpected advantage of exploiting sparsity in the space of principal components when the data are contaminated by measurement errors. Assuming the coefficient vector resides in an lq ball ( 0≤q≤1), we show that an l1 penalized principal components regression has a prediction performance on error-contaminated data that reaches the minimax-optimal rate obtained from clean data. Moreover, our theory does not require any knowledge of the covariance matrix of measurement errors. Our theory also reveals an interesting blessing-of-dimensionality phenomenon: the impact of measurement errors on prediction performance diminishes as the number of covariates increases. This is fundamentally different from the sparse measurement-error regression in the original input variables space where the negative impact of measurement errors only increases with the number of covariates.