Multimodal User Interfaces: From Signals to Interaction

✍ Scribed by Dimitrios Tzovaras (eds.)

• Publisher: Springer-Verlag Berlin Heidelberg
• Year: 2008
• Tongue: English
• Leaves: 320
• Series: Signals and Commmunication Technologies
• Edition: 1
• Category: Library

✦ Synopsis

Where multimodal signal processing meets multimodal human computer interaction...

Signal Processing (SP) and Human-Computer Interaction (HCI), two very popular scientific areas, are brought together to provide solutions to a series of great challenges in the field of multimodal interfaces. Focus is on multimodal interfaces that respond efficiently to speech, gestures, vision, haptics and direct brain connections taking advantage of recent developments in both HCI and SP.

The book presents a common theoretical framework for fusion and fission of multimodal information using the most advanced signal processing algorithms constrained by HCI rules, described in detail and integrated in the context of a common distributed software platform for easy and efficient development and usability assessment of multimodal tools. The book also demonstrates the use of the multimodal framework for the development of a set of applications in the areas of edutainment, interfaces for disabled people and interfaces for medical applications.

✦ Table of Contents

Front Matter....Pages i-x
Introduction....Pages 1-4
Multimodality Theory....Pages 5-29
Information-Theoretic Framework for Multimodal Signal Processing....Pages 31-59
Multimodality for Plastic User Interfaces: Models, Methods, and Principles....Pages 61-84
Face and Speech Interaction....Pages 85-117
Recognition of Emotional States in Natural Human-Computer Interaction....Pages 119-153
Two SIMILAR Different Speech and Gestures Multimodal Interfaces....Pages 155-184
Multimodal User Interfaces in Ubiquitous Environments....Pages 185-199
Software Engineering for Multimodal Interactive Systems....Pages 201-218
Gestural Interfaces for Hearing-Impaired Communication....Pages 219-250
Modality Replacement Framework for Applications for the Disabled....Pages 251-270
A medical Component-based Framework for Image Guided Surgery....Pages 271-280
Multimodal Interfaces for Laparoscopic Training....Pages 281-290
Back Matter....Pages 291-315

✦ Subjects

Signal, Image and Speech Processing; User Interfaces and Human Computer Interaction; Image Processing and Computer Vision; Communications Engineering, Networks