Data-driven recovery of incomplete geotechnical dataset using low-rank matrix completion

Real geotechnical data from a typical site might be characterized as MUSIC-X (i.e., multivariate, uncertain, unique, sparse, incomplete, and potentially corrupted, with X denoting spatial/temporal variability). One of the key challenges in developing site-specific statistical models for multiple geotechnical properties (i.e., multivariate) is missing (or incomplete) values from different tests at various depths/locations. This raises a critical question in geotechnical site investigations: how to recover the missing values in real geotechnical datasets from available measurements by leveraging the underlying structure of geotechnical datasets? Since different geotechnical properties are not only correlated among different properties, but also auto-correlated across different depths, this suggests that a simple underlying structure with only a limited number of important features/patterns might exist for multivariate geotechnical datasets. Leveraging on this observation, this study proposes a novel, data-driven method for predicting missing values by low-rank matrix completion. The proposed method exploits the auto- and cross-correlation structures of different test data. Missing values are then recovered using a singular value thresholding algorithm, and a k-fold cross-validation strategy is employed to determine the level of measurement noise. The method is illustrated and validated using a real geotechnical dataset. The results indicate that the proposed method can provide reliable predictions.