The structure of noise in a dataset and, in particular, whether it is homoscedastic or heteroscedastic, can significantly affect the properties of multivariate calibration models. This is particularly true when the data are subjected to a nonlinear transformation prior to the formation of the model. The problems of mathematical modelling in the frequency domain in the presence of heteroscedastic noise are demonstrated using simple, illustrative, synthesised datasets and partial least squares regression. The heteroscedasticity spreads signal-dependent information throughout the spectrum of the signal, removing the localisation seen with band-limited signals with homoscedastic noise. Heteroscedasticity significantly reduces the scope for efficient variable selection to allow modelling on a reduced variable set, with consequences for the production of sparse models which generalise well according to the parsimony principle. However, significant modelling can take place purely on the noise components even when the frequency range of the signal has been completely excluded. Optimal denoising schemes will beneficially take into account the noise structure of a dataset.