A nonlinear mixed-effects functional regression model based on variable selection

The mixed-effects functional regression (MFR) model offers a valuable tool for analyzing dynamic data with individual-specific variations. However, challenges arise in scenarios with nonlinear relationships and variable selection among covariates. To address this, we propose a novel extension to the MFR model. Our approach incorporates nonlinear components using bivariate splines, enabling a robust framework for complex relationship modeling. For variable selection in high-dimensional regression, we employ a group-minimax concave penalty (MCP) that treats parameters from the same spline basis as a group, ensuring accurate and unbiased selection. This methodology allows for the estimation of fixed-effects and random-effects in a two-step procedure. We contribute a flexible and comprehensive framework that accommodates nonlinear covariates and a MCP variable selection approach. Empirical validations and theoretical justifications support the effectiveness of our proposed methodology. In summary, our approach provides a versatile and efficient tool for modeling functional responses in the presence of nonlinear relationships and mixed effects.