✦ LIBER ✦

Improved language modelling through better language model evaluation measures

✍ Scribed by Philip Clarkson; Tony Robinson

Publisher: Elsevier Science
Year: 2001
Tongue: English
Weight: 172 KB
Volume: 15
Category: Article
ISSN: 0885-2308
DOI: 10.1006/csla.2000.0156

No coin nor oath required. For personal study only.

✦ Synopsis

This paper explores the interaction between a language model's perplexity and its effect on the word error rate of a speech recognition system. Much recent research has indicated that these two measures are not as well correlated as was once thought, and many examples exist of models which have a much lower perplexity than the equivalent N -gram model, yet lead to no improvement in recognition accuracy. This paper investigates the reasons for this apparent discrepancy. Perplexity's calculation is based solely on the probabilities of words contained within the test text; it disregards the probabilities of alternative words which will be competing with the correct word within the decoder. It is shown that by considering the probabilities of the alternative words it is possible to derive measures of language model quality which are better correlated with word error rate than perplexity is. Furthermore, optimizing language model parameters with respect to these new measures leads to a significant reduction in the word error rate.

📜 SIMILAR VOLUMES

Evaluating goal models within the goal-o

Evaluating goal models within the goal-oriented requirement language

✍ Daniel Amyot; Sepideh Ghanavati; Jennifer Horkoff; Gunter Mussbacher; Liam Peyto 📂 Article 📅 2010 🏛 John Wiley and Sons 🌐 English ⚖ 949 KB

In this article, we introduce the application of rigorous analysis procedures to goal models to provide several beneÞts beyond the initial act of modeling. Such analysis can allow modelers to assess the satisfaction of goals, facilitate evaluation of high-level design alternatives, help analysts dec

Improved Estimation in Measurement Error

Improved Estimation in Measurement Error Models Through Stein Rule Procedure

✍ Shalabh 📂 Article 📅 1998 🏛 Elsevier Science 🌐 English ⚖ 203 KB

This paper examines the role of Stein estimation in a linear ultrastructural form of the measurement errors model. It is demonstrated that the application of Stein rule estimation to the matrix of true values of regressors leads to the overcoming of the inconsistency of the least squares procedure a

An alternative scheme for perplexity est

An alternative scheme for perplexity estimation and its assessment for the evaluation of language models

✍ Frédéric Bimbot; Marc El-Bèze; Stéphane Igounet; Michèle Jardino; Kamel Smaili; 📂 Article 📅 2001 🏛 Elsevier Science 🌐 English ⚖ 194 KB

Language models are usually evaluated on test texts using the perplexity derived from the model likelihood function computed on these texts (test set perplexity). In order to use this measure in the framework of a comparative evaluation campaign, we have developed an alternative scheme for estimatin

CORRIGENDUM: Volume 67, Number 1 (1998),

CORRIGENDUM: Volume 67, Number 1 (1998), in the article “Improved Estimation in Measurement Error Models Through Stein Rule Procedure,” by Shalabh, pp. 35–48

📂 Article 📅 2000 🏛 Elsevier Science 🌐 English ⚖ 24 KB

where B and C are nonstochastic matrices of the appropriate order in each case. When additionally B is symmetric, However, in the article, only the first result was utilized, with no serious consequences, as the matrix B is symmetric. The other results in (4) are correct.