SmartKom: Foundations of Multimodal Dialogue Systems

✍ Scribed by Wolfgang Wahlster (auth.), Prof. Dr. Wolfgang Wahlster (eds.)

• Publisher: Springer-Verlag Berlin Heidelberg
• Year: 2006
• Tongue: English
• Leaves: 638
• Series: Cognitive Technologies
• Edition: 1
• Category: Library

✦ Synopsis

Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster is the Director and CEO of the German Research Center for Artificial Intelligence (DFKI GmbH) and a Professor of Computer Science at the Universität des Saarlandes, Saarbrücken. In 2000, he was coopted as a Professor of Computational Linguistics at the same university. In addition, he is the Head of the Intelligent User Interfaces Lab at DFKI.

He was the Scientific Director of the Verbmobil consortium on spontaneous speech translation (1993-2000), the SmartKom consortium on multimodal dialog systems (1999-2003), and the SmartWeb consortium on mobile multimodal access to semantic web services (2004-2008).

He has authored more than 150 technical papers and 6 books on language technology and intelligent user interfaces, and edited many books, among them the Springer titles "Verbmobil: Foundations of Speech-to-Speech Translation", "SmartKom: Foundations of Multimodal Dialogue Systems", and "SemProM: Foundations of Semantic Product Memories for the Internet of Things". His research includes multimodal and perceptive user interfaces, user modeling, embodied conversational agents, smart navigation systems, semantic web services, and resource-adaptive cognitive technologies.

He was awarded the Deutscher Zukunftspreis (German Future Award) in 2001 and he received the Federal Cross of Merit, First Class, the Officer's Cross of the Order of Merit of the Federal Republic of Germany in 2006.

He is the Chief Academic Advisor for Information and Communication Technologies (ICT) in the Research Union of the German Government, he is a member or chair of many key international scientific advisory and governance boards, including the National Institute of Informatics (NII) of Japan, the jury for the European ICT Prize of the European Commission, the Singapore Agency for Science, Technology and Research, the SAP Next Business & Technology Advisory Board, the Center for Advanced Security Research in Darmstadt, the Intel Visual Computing Institute, the Fondazione Bruno Kessler, and the Future Internet Public-Private Partnership of the European Commission.

He is a member of many academies, including the German Academy of Sciences and Literature, Mainz, the Royal Swedish Academy of Sciences, Stockholm, the German Academy of Natural Scientists Leopoldina, Acatech, the Council for Engineering Sciences of the Union of German Academies of Science and Humanities, and the Berlin-Brandenburg Academy of Sciences and Humanities.

He has chaired and given invited talks at the key international conferences in computational linguistics and artificial intelligence, he is a board member of the key AI journals, and he is a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI), the European Coordinating Committee for Artificial Intelligence (ECCAI), and the Gesellschaft für Informatik (GI).

✦ Table of Contents

Front Matter....Pages 1-1
Dialogue Systems Go Multimodal: The SmartKom Experience....Pages 3-27
Facts and Figures About the SmartKom Project....Pages 29-39
An Exemplary Interaction with SmartKom....Pages 41-52
Front Matter....Pages 53-53
The SmartKom Architecture: A Framework for Multimodal Dialogue Systems....Pages 55-70
Modeling Domain Knowledge: Know-How and Know-What....Pages 71-84
Speech Recognition....Pages 85-107
Class-Based Language Model Adaptation....Pages 109-121
The Dynamic Lexicon....Pages 123-138
The Prosody Module....Pages 139-152
The Sense of Vision: Gestures and Real Objects....Pages 153-165
The Facial Expression Module....Pages 167-180
Multiple Biometrics....Pages 181-193
Natural Language Understanding....Pages 195-207
The Gesture Interpretation Module....Pages 209-219
Front Matter....Pages 221-221
Modality Fusion....Pages 223-235
Discourse Modeling....Pages 237-253
Overlay: The Basic Operation for Discourse Processing....Pages 255-267
In Context: Integrating Domain- and Situation-Specific Knowledge....Pages 269-284
Intention Recognition....Pages 285-299
Plan-Based Dialogue Management for Multiple Cooperating Applications....Pages 301-316
Front Matter....Pages 221-221
Emotion Analysis and Emotion-Handling Subdialogues....Pages 317-332
Problematic, Indirect, Affective, and Other Nonstandard Input Processing....Pages 333-346
Front Matter....Pages 347-347
Realizing Complex User Wishes with a Function Planning Module....Pages 349-362
Intelligent Integration of External Data and Services into SmartKom....Pages 363-378
Multimodal Fission and Media Design....Pages 379-400
Natural Language Generation with Fully Specified Templates....Pages 401-410
Multimodal Speech Synthesis....Pages 411-435
Front Matter....Pages 437-437
Building Multimodal Dialogue Applications: System Integration in SmartKom....Pages 439-452
SmartKom-English: From Robust Recognition to Felicitous Interaction....Pages 453-470
SmartKom-Public....Pages 471-492
SmartKom-Home: The Interface to Home Entertainment....Pages 493-503
SmartKom-Mobile: Intelligent Interaction with a Mobile System....Pages 505-522
SmartKom-Mobile Car: User Interaction with Mobile Services in a Car Environment....Pages 523-537
Front Matter....Pages 539-539
Wizard-of-Oz Recordings....Pages 541-570
Annotation of Multimodal Data....Pages 571-596
Multimodal Emogram, Data Collection and Presentation....Pages 597-602
Empirical Studies for Intuitive Interaction....Pages 603-615
Evaluation of Multimodal Dialogue Systems....Pages 617-643

✦ Subjects

Artificial Intelligence (incl. Robotics); User Interfaces and Human Computer Interaction; Simulation and Modeling; Computer Imaging, Vision, Pattern Recognition and Graphics; Computer Graphics; Image Processing and Computer Vision