On topic identification and dialogue move recognition

✍ Scribed by Philip N. Garner

• Book ID: 102565949
• Publisher: Elsevier Science
• Year: 1997
• Tongue: English
• Weight: 366 KB
• Volume: 11
• Category: Article
• ISSN: 0885-2308
• DOI: 10.1006/csla.1997.0032

✦ Synopsis

Dialogue move recognition is cited as being representative of a class of problem which may be of concern in data driven natural language processing. The dialogue move recognition problem is formulated as a keyword-based topic identification problem, and is shown to be sensitive to the issue of unknown vocabulary. A model based on the multiple Poisson distribution is shown to alleviate the unknown vocabulary issue, subject to the assumption that the occurrence of keywords represents a small fraction of the data. A keyword selection strategy is derived to ensure this assumption is valid. It is shown that a modified version of Zipf's law provides a suitable prior probability distribution for keywords, and that its inclusion increases classification performance.