Supervised Learning

Abstract Supervised learning is an important area in machine learning. In practice, many problems can be solved using supervised learning techniques to deal with the corresponding covariate–response data. One general goal is to find a model that predicts the response from the covariates well. We focus on supervised learning methods that can be formulated into the optimization of “loss + penalty.” In particular, the loss term keeps the fidelity of the resulting model to the data, while the penalty term penalizing the complexity can prevent the fitted model from overfitting. In this article, we review some commonly used supervised learning methods under this framework. We mainly focus on the statistical models and computational algorithms.