4.3.1 Regularized Discriminant Analysis

The regularized covariance matrices have the form (4.13): \[ \hat{\Sigma}_k(\alpha)=\alpha \hat{\Sigma}_k + (1 - \alpha)\hat{\Sigma} \]

Here \(\alpha \in [0, 1]\) allows a continuum of models between LDA and QDA.

Similar modifications allow \(\hat{\Sigma}\) itself to be shrunk toward the scalar covariance (4.14), \[ \hat{\Sigma}(\gamma)=\gamma\hat{\Sigma}+(1-\gamma)\hat{\sigma}^2\mathbf{I} \]

for \(\gamma \in [0, 1]\). Replacing \(\hat{\Sigma}\) in (4.13) by \(\hat{\Sigma}(\gamma)\) leads to a more general family of covariances \(\hat{\Sigma}(\alpha, \gamma)\).