A closer look at the bias–variance trade-off in multivariate calibration
✍ Scribed by Nicolaas (KLAAS) M. Faber
- Publisher
- John Wiley and Sons
- Year
- 1999
- Tongue
- English
- Weight
- 54 KB
- Volume
- 13
- Category
- Article
- ISSN
- 0886-9383
No coin nor oath required. For personal study only.
✦ Synopsis
Principal component regression (PCR) and partial least squares (PLS) avoid the high variance often associated with ordinary least squares (OLS) results by allowing a small bias in the model. This paper presents a closer look at this bias-variance trade-off by discussing three practical aspects: (1) variance increases relatively slowly with increasing model complexity; (2) bias may be zero for the optimum model; (3) variance does not necessarily increase with increasing model complexity. While the first aspect is well known, the last two aspects are not. The second aspect implies that so-called biased regression methods do not necessarily yield biased predictions, while the third aspect, which is only encountered with non-linear estimation methods such as PLS, even contradicts the concept of bias-variance trade-off. The possibility of having both variance and bias decreasing with increasing PLS model complexity is illustrated using a near-infrared data set published by Fearn.