In this work we demonstrate that explicit modeling of correlations between spectral parameters in speech recognition improves speech models both in terms of their descriptive power (higher likelihoods) and classification accuracy.
Most large-vocabulary speech recognition systems are based on some form of hidden Markov models (HMMs) modeling sub-word speech segments. Most of the time speech segments are represented using short term spectra. In this work we employ three-state left-toright phone models and LPC cepstral parameters including their first and second order time differentials. We investigate the importance of modeling correlations between cepstral parameters for high accuracy phone recognition.
Several different types of distributions for each HMM state are compared. The simplest uses a single multivariate Gaussian distribution with a full covariance matrix. The next uses a weighted mixture of multivariate Gaussian distributions with diagonal covariances. It uses implicit rather than explicit modeling of parameter correlations. The most elaborate model employs a mixture of Gaussian distributions, just like the previous model, but in addition it uses a parameter space rotation which is specific to a given state in an HMM. It thus explicitly models parameter correlations in exactly the same way as the simplest model which uses a single distribution per state.
The highest phone accuracy on the DARPA Resource Management task Feb 89 test set is obtained using the most elaborate model, with mixtures and space rotation (-82 \cdot 4 %) phone accuracy. The next best result was achieved using single distributions, which also explicitly model parameter correlations, with (80.8 %) phone accuracy. The worst result was obtained using distributions which only implicitly model parameter correlations, achieving (78.7 %) phone accuracy. These results clearly demonstrate the importance of explicitly modeling parameter correlations for improving speech recognition performance.