Robust Regression

Abstract Unusual observations can have a large effect on least squares fits of regression coefficients. The most common defenses against this problem are residual plots and the diagnostic statistics based on residuals and leverages. These tools are most effective when the dataset is moderate in size and the analyst is in close touch with, or is, the person who collected the data. However, regression diagnostics become very time consuming to use in large data sets. In this situation it may be preferable to use more resistant estimators that are relatively unaffected by even large changes to a small proportion of the data. In general, resistance must be ‘purchased’ at the cost of some loss of efficiency when the standard model assumptions hold, but these assumptions are seldom tenable in large data sets anyway. Regression diagnostics and robust regression are regarded by some as conflicting solutions to the problem of the sensitivity of standard least squares regression parameter estimates to unusual, possibly erroneous, observations. However, the two approaches can be seen as complementary. Residuals from robust fits can highlight interesting groups of observations not obvious from standard residual plots; leverage diagnostics can supplement the fits of some convenient, but only partially robust, regression methods.