Model Selection via Misspecified Cramér-Rao Bound Minimization

In many applications of estimation theory, the true data model is unknown, and a set of parameterized models are used to approximate it. This problem is encountered in learning systems, where the assumed model parameters are estimated using training data. One of the challenges in these problems is choosing the architecture used for the approximated model. Complex and high-order models with limited training data size may lead to overfitting, while simple and low-order models may lead to model misspecification. In this paper, we propose to use the misspecified Cramér-Rao bound (MCRB) as a criterion for model selection. The MCRB takes into account modeling errors due to both overfitting and model misspecification. The performance of the proposed approach is evaluated via simulations for model order selection in a linear regression problem. The proposed method outperforms the minimum description length and the Akaike information criterion.