Dialogues with Social Robots: Enablements, Analyses, and Evaluation (Lecture Notes in Electrical Engineering, 427)

✍ Scribed by Kristiina Jokinen (editor), Graham Wilcock (editor)

• Publisher: Springer
• Year: 2016
• Tongue: English
• Leaves: 477
• Category: Library

✦ Synopsis

This book explores novel aspects of social robotics, spoken dialogue systems, human-robot interaction, spoken language understanding, multimodal communication, and system evaluation. It offers a variety of perspectives on and solutions to the most important questions about advanced techniques for social robots and chat systems.

Chapters by leading researchers address key research and development topics in the field of spoken dialogue systems, focusing in particular on three special themes: dialogue state tracking, evaluation of human-robot dialogue in social robotics, and socio-cognitive language processing.

The book offers a valuable resource for researchers and practitioners in both academia and industry whose work involves advanced interaction technology and who are seeking an up-to-date overview of the key topics. It also provides supplementary educational material for courses on state-of-the-art dialogue system technologies, social robotics, and related research fields.

✦ Table of Contents

Preface
Organization of IWSDS 2016
Contents
Part I The Northernmost Spoken Dialogue Workshop
DigiSami and Digital Natives: Interaction Technology for the North Sami Language
1 Introduction
2 DigiSami and North Sami Language Resources
2.1 The DigiSami Project
2.2 The Sami Languages
2.3 The North Sami Language
2.4 Existing North Sami Language Resources
2.5 Existing North Sami Speech Technology
3 The DigiSami Corpus
3.1 Preliminary Analysis: Engagement and Interaction
3.2 Preliminary Analysis: Influence of Majority Language
3.3 Preliminary Analysis: Adjectives in Spoken Language
4 Towards SamiTalk: A Sami-Speaking Robot Application
5 Conclusions and Future Work
References
Part II Methods and Techniques for Spoken Dialogue Systems
A Comparative Study of Text Preprocessing Techniques for Natural Language Call Routing
1 Introduction
2 Corpus Description
3 Term Weighting Methods
4 Dimensionality Reduction Methods
5 Classification Algorithms
6 Numerical Experiments
7 Conclusions
References
Compact and Interpretable Dialogue State Representation with Genetic Sparse Distributed Memory
1 Introduction
2 Background
2.1 The Reinforcement Learning Framework
2.2 The Sparse Distributed Memory Model
2.3 Genetic Sparse Distributed Memory for Classification
3 Genetic Sparse Distributed Memory for Reinforcement Learning (GSDMRL)
3.1 Building the Set of Prototypes
3.2 Q-Function Parametrisation
3.3 Re-engineering the Prototypes
4 Experiments with the NASTIA Dialogue System
4.1 Comparing GSDMRL to a Grid-Based Representation
4.2 Scalability and Interpretability of GSDMRL
4.3 Results
5 Conclusion
References
Incremental Human-Machine Dialogue Simulation
1 Introduction
2 Simulated Environment
2.1 Service
2.2 User Simulator
2.3 ASR Output Simulator
2.4 Scheduler
3 Illustration
3.1 Turn-Taking Strategy Example
3.2 Evaluation
4 Conclusion and Future Work
References
Active Learning for Example-Based Dialog Systems
1 Introduction
2 An Active Learning Framework for Example-Based Dialog Managers
2.1 Example-Based Dialog Managers and Their Evaluation
2.2 Active Learning Framework
3 Input Selection Strategies
4 Experiments
4.1 Experimental Setup
4.2 Experimental Results and Additional Analysis
5 Conclusion
References
Question Selection Based on Expected Utility to Acquire Information Through Dialogue
1 Introduction
2 Appropriate Questions for Acquiring Information
3 Question Selection Based on Expected Utility
3.1 Utility for Each Question Type
3.2 Probability Representing Content of Questions
3.3 Calculating Expected Utility
4 Empirical Setting of Parameters
4.1 Setting Utility Values
4.2 Integration of Two Confidence Measures to Obtain Probability
5 Subjective Evaluation for Utility Values
6 Conclusion
References
Separating Representation, Reasoning, and Implementation for Interaction Management: Lessons from Automated Planning
1 Introduction
2 A Survey of Interaction Management Toolkits
2.1 TrindiKit/DIPPER
2.2 Ravenclaw
2.3 COLLAGEN/DISCO
2.4 OpenDial
2.5 IrisTK
2.6 InproTK
2.7 Pamini
2.8 Summary
3 The Separation of Representation and Reasoning in Automated Planning
4 Plan-Based Interaction Management in a Robot Bartender Domain
5 Summary and Future Work
References
SimpleDS: A Simple Deep Reinforcement Learning Dialogue System
1 Introduction
2 Deep Reinforcement Learning for Dialogue Control
3 The SimpleDS Dialogue System
4 Summary
References
Breakdown Detector for Chat-Oriented Dialogue
1 Introduction
2 Proposed Method
2.1 Dialogue Data Analysis and Proposed Taxonomy
2.2 Breakdown Detection by Combining Different Error Detectors
3 Experiments
3.1 Baseline Classifier
3.2 Aggregated Classifiers by Level
4 Breakdown Detector
5 Related Work
6 Conclusion
References
User Involvement in Collaborative Decision-Making Dialog Systems
1 Introduction
2 Mixed-Initiative Planning
3 Related Work
4 User Study About User Involvement Strategies
4.1 Used Questionnaires
4.2 Hypotheses
4.3 Results
4.4 Discussion
5 Conclusion
References
Part III Socio-Cognitive Language Processing
Natural Language Dialog System Considering Speaker's Emotion Calculated from Acoustic Features
1 Introduction
2 Overview of the Proposed Method
3 Expansion of AIML-based Response Generating Rule
4 Emotion Estimation Method Based on Acoustic Features
5 Experiment
6 Conclusion
References
Salient Cross-Lingual Acoustic and Prosodic Features for English and German Emotion Recognition
1 Introduction
2 Significant Related Work
3 Corpora
4 Feature Selection with MOGA
5 Evaluation and Results
6 Salient Features for Cross-Lingual Emotion Recognition
7 Conclusion and Future Work
References
Entropy-Driven Dialog for Topic Classification: Detecting and Tackling Uncertainty
1 Introduction
2 Overview: The Dialog System
3 The Suggestion Generation Module
3.1 The Information Graph
3.2 The Suggestion Generation Algorithm
3.3 Answer Evaluation
4 The Task and Feature Selection
5 Experimental Evaluation
5.1 Classifier Evaluation
5.2 Recovering from Uncertainty
6 Conclusions and Future Work
References
Evaluation of Question-Answering System About Conversational Agent's Personality
1 Introduction
2 Person DataBase: PDB
3 Experiments
3.1 Objective Evaluation: Estimation Accuracy of Question Categories
3.2 Subjective Evaluation 1: Response Appropriateness
3.3 Subjective Evaluation 2: Online-Chat Experiments
4 Conclusion
References
Fisher Kernels on Phase-Based Features for Speech Emotion Recognition
1 Introduction
2 Methods
2.1 Modified Group Delay Feature
2.2 Fisher Kernels
3 Experiments
3.1 Geneva Whispered Emotion Corpus
3.2 Experimental Setup
3.3 Results
4 Conclusions
References
Part IV Towards Multilingual, Multimodal, Open Domain Spoken Dialogue Systems
Internationalisation and Localisation of Spoken Dialogue Systems
1 Introduction
2 Previous Work and Methods
2.1 Software Internationalisation and Localisation
2.2 Dialogue System Localisation
3 WikiTalk
4 WikiTalk Internationalisation
4.1 String Extraction, Variables and Language Selection
4.2 Wikipedia Content Processing
4.3 Finnish WikiTalk
4.4 Japanese WikiTalk
5 Limitations on Internationalisation
5.1 Grammatical Features in Text Interfaces
5.2 Issues in Multimodal Interfaces
6 Conclusions and Future Work
References
A Multi-lingual Evaluation of the vAssist Spoken Dialog System. Comparing Disco and RavenClaw
1 Introduction
2 Related Work
3 Main Goals and Contributions
4 System Description
4.1 Speech Recognition
4.2 Natural Language Generation and Text-to-Speech
4.3 Semantic Parsing
4.4 Semantic Unification and Resolution
4.5 Dialog Act Mapping
4.6 Dialog Management Based on Disco
4.7 Dialog Management Based on RavenClaw
5 Task and Experimental Scenarios
6 Experimental Evaluation
6.1 System Performance
6.2 Task Easiness and Usability
6.3 Speech Assessment
6.4 Disco-LFF and RavenClaw DM Comparison
7 Conclusion
References
Multimodal HALEF: An Open-Source Modular Web-Based Multimodal Dialog Framework
1 Introduction and Related Work
2 Foundational Frameworks
3 Framework Integration
4 Supporting Modules
4.1 Database
4.2 Participant Web-Survey
4.3 STAR Portal
4.4 Visual Processing Service
5 Example Application: Job Interview
5.1 Preliminary Data Collection
6 Conclusions and Future Work
References
Data Collection and Synchronisation: Towards a Multiperspective Multimodal Dialogue System with Metacognitive Abilities
1 Introduction
2 Related Corpora
2.1 IFA Dialogue Corpus
2.2 AMI Meeting Corpus
2.3 Youth Parliament Corpus
2.4 MLA-14 Data
2.5 Limitations
3 The METALOGUE Data Collection Process
3.1 Recording Settings
3.2 Wizard of Oz Software
3.3 First Sitting for Recording
3.4 Second Sitting for Recording
3.5 Data Synchronisation
3.6 Speaker Characteristics
4 Conclusion and Future Work
References
HELPR: A Framework to Break the Barrier Across Domains in Spoken Dialog Systems
1 Introduction
2 Data Collection
2.1 Smartphone Data Collection and Annotation
2.2 Interactive Dialog Task
3 Methodology
3.1 Models for Intention Understanding
3.2 Conveying Intention Understanding
4 Study
4.1 Intention Interpretation and Realization
4.2 Intention Representation in Natural Language
5 Conclusion and Future Work
References
Towards an Open-Domain Social Dialog System
1 Introduction
2 Background
3 System Description
3.1 Response Combination
3.2 Fact Response
3.3 Inquiring Questions
3.4 POMDP Goal-Oriented Dialog
4 Results
5 Demo Setup
6 Conclusion
References
Part V Evaluation of Human--Robot Dialogue in Social Robotics
Is Spoken Language All-or-Nothing? Implications for Future Speech-Based Human-Machine Interaction
1 Introduction
2 The Nature of the Problem
2.1 The ``Habitability Gap''
2.2 Half a Language?
3 What Is Language?
4 The Way Forward?
5 Final Remarks
References
Toward a Context-Based Approach to Assess Engagement in Human-Robot Social Interaction
1 Introduction
2 Related Work
2.1 Engagement in HRI
2.2 Communication Accommodation Theory (CAT)
3 CAT and Engagement for HRI
3.1 Contributions of CAT
3.2 Multi-level Expectations
3.3 Influencing Factors
4 Towards a Context-Based Approach to Quantify Engagement
5 Conclusion and Future Work
References
Extrinsic Versus Intrinsic Evaluation of Natural Language Generation for Spoken Dialogue Systems and Social Robotics
1 Introduction
2 Background Literature
3 Natural Language Generation Module
4 The End-to-End Spoken Dialogue System
5 Evaluating the NLG component in an End-to-End System
5.1 The Tasks
6 Results
7 Discussion and Future Work
References
Engagement in Dialogue with Social Robots
1 Introduction
2 Engagement
3 Research into Interactive Speech Synthesis
4 Natural Human-Machine Conversational Interaction
5 Findings and Observations
6 Conclusions
References
Lend a Hand to Service Robots: Overcoming System Limitations by Asking Humans
1 Introduction
2 Related Work
3 Wizard of Oz Platform
3.1 Hardware
3.2 Software
4 First User Study
5 Conclusion and Future Work
References
Dialogue with Robots to Support Symbiotic Autonomy
1 Introduction
2 Related Work
3 A Pragmatic Approach for Dialogue Modeling
3.1 Background and Motivation
3.2 A Framework for Flexible Pragmatic Task-Based Dialogues
4 Use Cases
4.1 Dialogue for Human Augmented Mapping
5 Conclusion
References
Towards SamiTalk: A Sami-Speaking Robot Linked to Sami Wikipedia
1 Introduction
2 WikiTalk and SamiTalk
3 Speech Technology for SamiTalk
3.1 Speech Synthesis
3.2 Speech Recognition
4 Example Interaction
5 Conclusions and Future Work
References
Part VI Dialogue Quality Assessment
Utterance Selection Using Discourse Relation Filter for Chat-oriented Dialogue Systems
1 Introduction
2 Related Work
3 Utterance Selection with Discourse Relations
3.1 Utterance Ranking Function
3.2 Discourse Relation Identification
3.3 Utterance Filtering Using Discourse Relations
4 Experiment
4.1 Dataset
4.2 Comparative Methods
4.3 Experimental Results
5 Conclusion
References
Analysis of Temporal Features for Interaction Quality Estimation
1 Introduction
2 Significant Related Work
3 Temporal Feature Analysis
3.1 The Interaction Quality Paradigm
3.2 Manual Analysis and Feature Set Extension
3.3 Analysis of Parameter Levels
3.4 Analysis of Window Size
4 Discussion
5 Conclusion and Future Work
References
Recurrent Neural Network Interaction Quality Estimation
1 Introduction
2 Significant Related Work
2.1 User Satisfaction
2.2 The Interaction Quality Paradigm and the LEGO Corpus
3 Recurrent Neural Networks for IQ Estimation
4 Experiments and Results
4.1 Evaluation Metrics
4.2 Experimental Setup
4.3 Results
5 Conclusion
References
An Evaluation Method for System Response in Chat-Oriented Dialogue System
1 Introduction
2 Related Works
3 Framework of Response Generator
4 Evaluation Method for System Response
4.1 Evaluation Indices for Continuous and Long-Term Dialogue
4.2 Evaluation Indices Estimator
5 Experimental Results
6 Conclusion
References
The Negotiation Dialogue Game
1 Introduction
2 The Negotiation Dialogue Game
3 The Inter-User Policy Experiment
3.1 User Profiles
3.2 Reinforcement Learning Implementation
3.3 Experiment Results
4 Towards Real Users Profiling
5 Conclusion
References
Estimation of User's Willingness to Talk About the Topic: Analysis of Interviews Between Humans
1 Introduction
2 Related Work
3 Experimental Dialog Data
3.1 Experimental Setup
3.2 Results of Recording
4 Extraction and Analysis of Multi-modal Information
5 Discrimination of Willingness to Talk About the Topic
6 Conclusion
References
Recognising Conversational Speech: What an Incremental ASR Should Do for a Dialogue System and How to Get There
1 Introduction
2 The Challenge of Interactive, Conversational Speech
3 Desiderata for Incremental ASR for Interactive SDSs
3.1 Incrementality and Timing Information
3.2 Suitability for Disfluency
4 Evaluation Metrics
4.1 Utterance-Level Accuracy and Disfluency Suitability
4.2 Timing
4.3 Diachronic Evolution
5 Evaluation Domain: Pentomino Puzzle Playing Dialogue
6 Evaluation of Three ASR Systems: Google, Sphinx-4, and Kaldi
6.1 Experiment 1: Off-the-shelf Models for a Dialogue Domain
6.2 Experiment 2: Training Models on In-Domain Data
6.3 Results
7 Conclusions
References
Part VII Dialogue State Tracking Challenge 4
The Fourth Dialog State Tracking Challenge
1 Introduction
2 Challenge Overview
2.1 Problem Statement
2.2 Challenge Design
2.3 Data
3 Main Task: Evaluation
3.1 Evaluation Metrics
3.2 Baseline Tracker
4 Main Task: Results
5 Main Task: Ensemble Learning
6 Pilot Tasks
6.1 Evaluation Metrics
6.2 Web-Based Evaluation
6.3 Results
7 Conclusions
References
Convolutional Neural Networks for Multi-topic Dialog State Tracking
1 Introduction
2 Data Characteristics and Problem Description
3 Related Work
4 Convolutional Neural Network Model
4.1 Model Definition
4.2 Multi-topic Model
5 Experiment Setup
5.1 Feature Representation
5.2 Hyperparameters and Training
5.3 Semi-supervised Learning
6 Results and Discussion
6.1 Multi-topic Model
6.2 Semi-supervised Learning
7 Conclusion
References
The MSIIP System for Dialog State Tracking Challenge 4
1 Introduction
2 Task Description
3 Probabilistic Framework of the Turn-Taking Algorithm
3.1 Probabilistic Enhanced Frame Structure
3.2 State Update Model and Output Model
4 Slot Value Classification Parser
4.1 Training and Parsing Algorithms of the SVC Model
4.2 An Iterative Alignment Method for SVC
5 Slot-Based Score Averaging Ensemble
6 Experiment
7 Conclusion
References
Robust Dialog State Tracking for Large Ontologies
1 Introduction
2 The DSTC 4 Dataset
3 Method
3.1 Fuzzy Matching Baseline
3.2 Machine-Learning-Based Trackers
3.3 Elaborate Rule-Based Tracker
3.4 Hybrid Tracker
4 Results
5 Conclusion and Future Work
References
40 Erratum to: Dialogues with Social Robots
Erratum to: K. Jokinen and G. Wilcock (eds.), Dialogues with Social Robots, Lecture Notes in Electrical Engineering 427, DOI 10.1007/978-981-10-2585-3
Author Index