On deletion diagnostic statistic in regression

Abstract: The change in the least squares estimator (LSE) of a vector of regression coefficients due to a case deletion is often used for investigating the influence of an observation on the LSE. A normalization of the change in the LSE using the Moore-Penrose inverse of the covariance matrix of the change in the LSE is derived. This normalization turns out to be a square of the internally studentized residual. It is shown that the numerator term of Cook's distance does not in general have a chi-squared distribution except for a single case. An elaborate explanation about the inappropriateness of the choice of a scaling matrix defining Cook's distance is given. By reflecting a distributional property of the change in the LSE due to a case deletion, a new diagnostic measure that is a scalar is suggested. Three numerical examples are given for illustration.