Multi-Modal User Interactions in Controlled Environments (Multimedia Systems and Applications, 34)

✍ Scribed by Chaabane Djeraba, Adel Lablack, Yassine Benabbas

• Publisher: Springer
• Year: 2010
• Tongue: English
• Leaves: 226
• Category: Library

✦ Synopsis

Multi-Modal User Interactions in Controlled Environments investigates the capture and analysis of user’s multimodal behavior (mainly eye gaze, eye fixation, eye blink and body movements) within a real controlled environment (controlled-supermarket, personal environment) in order to adapt the response of the computer/environment to the user. Such data is captured using non-intrusive sensors (for example, cameras in the stands of a supermarket) installed in the environment. This multi-modal video based behavioral data will be analyzed to infer user intentions while assisting users in their day-to-day tasks by adapting the system’s response to their requirements seamlessly. This book also focuses on the presentation of information to the user. Multi-Modal User Interactions in Controlled Environments is designed for professionals in industry, including professionals in the domains of security and interactive web television. This book is also suitable for graduate-level students in computer science and electrical engineering.

✦ Table of Contents

Foreword
Preface
Acknowledgements
Contents
Chapter 1 ntroduction
1.1 Introduction
1.2 Objective
1.3 Practical Applications
1.4 Research Challenges
1.4.1 Event Detection
1.4.2 Flow Estimation
1.4.3 Gaze Estimation
1.4.4 Role of the Context
1.4.5 Societal Issues
1.5 Technical Contribution
1.6 How the Book is Organized
Chapter 2 Abnormal Event Detection
2.1 Introduction
2.2 Related Work
2.3 Proposed Approach
2.3.1 Low-Level Features
2.3.1.1 Motion Heat Map
2.3.1.2 Points of Interest
2.3.1.3 Tracking Points of Interest
2.3.1.4
2.3.2 Intermediate-Level Features
2.3.2.1 Motion Area Ratio
2.3.2.2 Direction Variance
2.3.2.3 Motion Magnitude Variance
2.3.2.4 Direction Histogram
2.3.2.5 Direction Map
2.3.2.6 Difference of Direction Map
2.3.3 Other Intermediate-Level Features
2.3.3.1 Motion Continuity Factor
2.3.3.2 Motion Description Factor
2.3.3.3 Motion Trajectory of the Blob
2.3.4 High-Level Features
2.3.4.1 Detecting Collapsing Events
2.3.4.2 Detecting Opposing Flow Event
2.4 Group Detection and Tracking
2.4.1 Detection and Tracking of PoIs
2.4.2 Direction and Magnitude Models
2.4.3 Block Clustering
2.4.4 Group Tracking
2.5 Detecting Multiple Flows and Events in a Crowd Scene
2.5.1 Multiple Flow Detection
2.5.2 Event Recognition
2.5.3 Running and Walking Events
2.5.4 Crowd Convergence and Divergence Events
2.5.5 Results
2.6 Method Adaptation to Context
2.6.1 Overview
2.6.2 Context Factors
2.6.3 Method Extensions
2.6.4 Experiments
2.6.4.1 Some examples
2.6.4.2 Data Set
2.6.4.3 Methodology
2.6.5 Results
2.7 Conclusion
Chapter 3
Flow Estimation
3.1 Introduction
3.2 Related Works
3.2.1 Methods based on Motion Detection and Analysis
3.2.1.1 Approach Proposed by Xu and al. [149]
3.2.1.2 Approach Proposed by Zhang and Chen [153]
3.2.2 Methods Based on Contour Analysis
3.2.2.1 Approach Proposed by Bozzoli and al. [19]
3.2.3 Template-Based Methods
3.2.3.1 Approach Proposed by Sidla and al. [120]
3.2.4 Stereovision-Based methods
3.2.4.1 Approach Proposed by Terada and al. [130]
3.2.5 Spatio-Temporal Methods
3.2.5.1 Approach Proposed by Albiol and al. [1]
3.2.6 Commercial Applications
3.2.6.1 Cognimatics
3.2.6.2 Infodev
3.2.6.3 Eurecam Sarl
3.2.7 Contribution
3.3 Approach Steps
3.3.1 Blob Detection
3.3.2 Count Estimation
3.4 Experiments and Results
3.5 Conclusion
Chapter 4
Estimation of Visual Gaze
4.1 Human Vision System
4.2 History of Gaze Tracking
4.3 Gaze Tracking Techniques
4.3.1 Intrusive Systems
4.3.1.1 Electro-Oculography
4.3.1.2 Contact Lenses with Magnetic Coils
4.3.1.3 Localization of the Limb
4.3.1.4 Analysis of Eye Images
4.3.2 Non-Intrusive Systems
4.4 Applications
4.4.1 Interaction During Meetings
4.4.2 Driver Monitoring
4.4.3 Virtual Reality
4.4.4 Human Computer Interaction
4.4.5 Extraction of Saliency Maps in Images
4.4.6 Store Marketing
4.5 Contribution of Head Pose in Visual Gaze
4.5.1 Database
4.5.2 Calculating the Contribution of Head Pose
4.5.3 Prediction of the Target
4.6 Estimating Gaze Direction Based on Eye Localization Only
4.7 Head Pose Estimation
4.7.1 State of the Art
4.7.1.1 Definition
4.7.1.2 Human Capacity for Estimating Head Orientation
4.7.1.3 Problems Encountered when Estimating Head Poses
4.7.2 Image Datasets
4.7.2.1 Building Image Datasets
4.7.2.2 Utilized Image Database
4.7.3 Estimation of Head Pose Based on Global Appearance
4.7.3.1 Utilized Image Dataset
4.7.3.2 Feature Selection
4.7.3.3 Experimental Results
4.7.4 Cylindrical Model for Head Tracking
4.8 Conclusion
Chapter 5
Visual Field Projection and Region of Interest Analysis
5.1 Visual Field Estimation
5.1.1 Physiological Data
5.1.2 Visual Field Estimation and Fixation Point for Frontal Pose
5.1.3 Visual Field Adaptation to Head Orientation
5.1.3.1 Matrix Approach
5.2 Visual Field Projection
5.2.1 Point Projection
5.2.2 Perception Volume Projection
5.3 Visual Field Display and Projection on an Image
5.4 Region-of-Interest Extraction
5.4.1 Representation of Gaze Information
5.4.2 Gaze Point Correction
5.4.3 Calculation of Tilt and Pan Angles Corresponding to a Gaze Point
5.5 Metrics for Gaze Analysis
5.5.1 Construction of a System Measuring Media Relevance
5.5.1.1 Raw Data Collection
5.5.1.2 Identification of Fixations
5.5.2 Metrics Related to Fixation Distribution
5.5.3 Experiment
5.5.3.1 Images
5.5.3.2 Videos
5.5.4 Discussions
5.6 Conclusion
Chapter 6
Conclusion
6.1 Challenge
6.2 Perspectives
References
Appendix A Societal Recommendations
A.1 Societal Recommendations
A.1.1 Public Awareness
A.1.1.1 Observation and Surveillance Technology Awareness
A.1.1.2 Legal Awareness
A.1.1.3 Recommendation 1
A.1.2 Public Policy of Research and Development
A.1.2.1 EU R&D Awareness
A.1.2.2 Recommendation 2
A.1.2.3 Democratic Discussion
A.1.2.4 Recommendation 3
A.1.2.5 R&D Program Evaluation
A.1.2.6 Recommendation 4
A.1.2.7 Management and Human Science Researchers’ Role
A.1.2.8 Recommendation 5
A.1.3 Democratic Requirement for OST Regulation
A.1.3.1 Reinforcement of Public Authorities’ Assets with Regard to the Protection of Privacy and Data Protection
A.1.3.2 Recommendation 6
A.1.3.3 Intelligibility of the OST Systems
A.1.3.4 Accessibility to the OST Systems
A.1.3.5 Recommendation 7
A.1.3.6 Legitimacy of the OST’s Finalities
A.1.3.7 Recommendation 8
A.1.3.8 Recommendation 9
A.1.3.9 Privatization of Public Issues
A.1.3.10 Recommendation 10
A.2 Legal Recommendations
A.2.1 Data Protection and Privacy Issues
A.2.1.1 Recommendation 11
A.2.1.2 Recommendation 12
A.2.1.3 Recommendation 13
A.2.1.4 Recommendation 14
A.2.1.5 Recommendation 15
A.2.1.6 Recommendation 16
A.2.1.7 Recommendation 17
A.2.1.8 Recommendation 18
A.2.1.9 Recommendation 19
A.2.1.10 Recommendation 20
A.2.1.11 Recommendation 21
A.2.2 Beyond Data Protection
A.2.2.1 Data Protection and Fundamental Liberties
A.2.2.2 Recommendation 22
A.2.2.3 Recommendation 23
A.2.2.4 Recommendation 24
A.2.2.5 Specific Provisions about Terminals and Infrastructures
A.2.2.6 Recommendation 25
A.2.2.7 Recommendation 26
A.2.2.8 Recommendation 27
A.2.2.9 Recommendation 28
A.2.2.10 Recommendation 29
A.2.2.11 Recommendation 30
A.2.2.12 Towards a Regulation of Profiling Activities
A.2.2.13 Recommendation 31
A.2.2.14 Recommendation 32
A.2.2.15 Recommendation 33
A.2.2.16 Recommendation 34
A.2.2.17 Recommendation 35
A.2.3 Consumer Protection
A.2.3.1 Recommendation 36
A.2.3.2 Recommendation 37
A.2.3.3 Recommendation 38
A.2.3.4 Recommendation 39
Glossary
Index