An alternative scheme for perplexity est
✍
Frédéric Bimbot; Marc El-Bèze; Stéphane Igounet; Michèle Jardino; Kamel Smaili;
📂
Article
📅
2001
🏛
Elsevier Science
🌐
English
⚖ 194 KB
Language models are usually evaluated on test texts using the perplexity derived from the model likelihood function computed on these texts (test set perplexity). In order to use this measure in the framework of a comparative evaluation campaign, we have developed an alternative scheme for estimatin