Speaker Verification by Human Listeners: Experiments Comparing Human and Machine Performance Using the NIST 1998 Speaker Evaluation Data
✍ Scribed by Astrid Schmidt-Nielsen; Thomas H. Crystal
- Book ID
- 102569471
- Publisher
- Elsevier Science
- Year
- 2000
- Tongue
- English
- Weight
- 360 KB
- Volume
- 10
- Category
- Article
- ISSN
- 1051-2004
No coin nor oath required. For personal study only.
✦ Synopsis
The speaker verification performance of human listeners was compared to that of computer algorithms/systems. Listening protocols were developed to emulate as closely as possible the 1998 algorithm evaluation run by the U.S. National Institute of Standards and Technology (NIST), while taking into account human memory limitations. A subset of the target speakers and test samples from the same telephone conversation data was used. Ways of combining listener data to arrive at a group decision were explored, and the group mean worked well. The human results were very competitive with the best computer algorithms in the same handset condition. For same number testing, with 3-s samples, listener panels and the best algorithm had the same equal-error rate (EER) of 8%. Listeners were better than typical algorithms. For different number testing, EERs increased but humans had a 40% lower equal-error rate. Human performance in general seemed relatively robust to degradation.