Computer Analysis of Human Behavior

✍ Scribed by Albert Ali Salah (editor), Theo Gevers (editor)

• Publisher: Springer
• Year: 2011
• Tongue: English
• Leaves: 412
• Category: Library

✦ Synopsis

This book provides a broad survey of advanced pattern recognition techniques for human behavior analysis. Clearly structured, the book begins with concise coverage of the major concepts, before introducing the most frequently used techniques and algorithms in detail, and then discussing examples of real applications. Features: contains contributions from an international selection of experts in the field; presents a thorough introduction to the fundamental topics of human behavior analysis; investigates methods for activity recognition, including gait and posture analysis, hand gesture analysis, and semantics of human behavior in image sequences; provides an accessible psychological treatise on social signals for the analysis of social behaviors; discusses voice and speech analysis, combined audiovisual cues, and social interactions and group dynamics; examines applications in different research fields; each chapter concludes with review questions, a summary of the topics covered, and a glossary.

✦ Table of Contents

Computer Analysis of Human Behavior
Preface
Overview and Goals
Target Audience
Organization of the Book
Contents
Contributors
Acronyms
Part I: Tools of the Trade
Chapter 1: Bayesian Methods for the Analysis of Human Behaviour
1.1 Bayesian Methods
1.2 A Note on Bayesian Networks
1.3 Approximating the Marginal Distribution
1.3.1 Sampling
1.3.2 Variational Approximations
1.4 Non-parametric Methods
1.4.1 The Dirichlet Process
1.4.2 The Gaussian Process
1.4.2.1 Gaussian Process as Bayesian Linear Regression
1.4.2.2 Kernel Functions
1.4.2.3 Classification
1.5 Human Behaviour Modelling
1.6 Summary
1.7 Questions
1.8 Glossary
References
Chapter 2: Introduction to Sequence Analysis for Human Behavior Understanding
2.1 Introduction
2.2 Graphical Models
2.2.1 Graph Theory
2.2.2 Conditional Independence
2.3 Bayesian Networks
2.3.1 Factorization
2.3.2 The d-Separation Criterion
2.3.3 Hidden Markov Models
2.4 Conditional Random Fields
2.4.1 Factorization and Conditional Independence
2.4.2 Linear Chain Conditional Random Fields
2.5 Training and Inference
2.5.1 Message Passing
2.5.1.1 Inference
2.5.1.2 Training
2.6 Summary
2.7 Questions
2.8 Glossary
References
Chapter 3: Detecting and Tracking Action Content
3.1 Introduction
3.2 Representations
3.2.1 Spatial Representations
3.2.2 Spatio-Temporal Representations
3.3 Descriptors
3.3.1 Non-appearance Based Descriptors
3.3.2 Appearance Based Descriptors
3.4 Finding Action Content
3.4.1 Point Detection
3.4.2 Body Parts Detection
3.4.3 Detection of Silhouette
3.5 Tracking
3.5.1 Tracking by Association
3.5.2 Flow Estimation
3.5.3 Kernel Tracking
3.6 Summary
3.7 Questions
3.8 Glossary
References
Chapter 4: Computational Visual Attention
4.1 What Is Attention? And Do We Need Attentive Machines?
4.2 Human Visual Attention
4.2.1 The Human Visual System
4.2.1.1 Eye, Retina, and Ganglion Cells
4.2.1.2 From the Optic Chiasm to V1
4.2.1.3 Beyond V1: the Extrastriate Cortex and the Visual Pathways
4.2.1.4 Neurobiological Correlates of Visual Attention
4.2.2 Psychological Concepts of Attention
4.2.3 Important Psychological Attention Models
4.3 Computational Attention Systems
4.3.1 General Structure
4.3.2 Bottom-up Saliency
4.3.2.1 Intensity Channel
4.3.2.2 Color Channel
4.3.2.3 Orientation Channel
4.3.2.4 Motion Channel
4.3.2.5 The Uniqueness Weight
4.3.2.6 Normalization
4.3.2.7 The Conspicuity Maps
4.3.2.8 The Saliency Map and Focus Selection
4.3.3 Visual Search with Top-down Cues
4.3.3.1 Learning Mode
4.3.3.2 Several Training Images
4.3.3.3 Search Mode
4.3.3.4 Bottom-up and Top-down Cues Compete for Attention
4.4 Evaluation of Computational Attention Systems
4.5 Applications in Computer Vision and Robotics
4.6 Open Source Code, Databases, and Further Reading
4.6.1 Open Source Code
4.6.2 Databases
4.6.3 Further Reading
4.7 Summary
4.8 Questions
4.9 Glossary
References
Part II: Analysis of Activities
Chapter 5: Methods and Technologies for Gait Analysis
5.1 Introduction
5.2 Motion Measurements
5.2.1 Marker-Based Motion Capture
5.2.1.1 Calibration of Anatomical Landmarks
5.2.1.2 Protocols
5.2.1.3 Errors
5.2.2 Markerless Motion Capture
5.2.2.1 Cardboard Models
5.2.2.2 Tracking
5.2.2.3 Matching
5.2.2.4 3-D Model-Based Approaches
5.2.3 Inertial Measurements
5.2.3.1 Accelerometers
5.2.3.2 Gyroscopes
5.2.3.3 Magnetometers
5.3 Force Platforms and Electromyography
5.4 Summary
5.5 Questions
5.6 Glossary
References
Chapter 6: Hand Gesture Analysis
6.1 Hand Gestures in Human Communication
6.1.1 Taxonomy of Hand Gestures
6.1.2 Hand Gestures Accompanying Speech
6.1.3 Hand Gestures in Hearing Impaired Communication
6.1.4 Hand Gestures in Human-Computer Interaction
6.2 Hand Gesture Recognition Framework
6.3 Hand Pose Estimation
6.3.1 Modeling Hand Shape
6.3.2 Hand Shape Classification
6.4 Hand Gesture Recognition
6.4.1 Modeling Hand Gestures with Graphical Models
6.4.2 HMM and CRF Variants Used in Gesture Recognition
6.4.2.1 Hidden Conditional Random Fields
6.4.2.2 Latent Dynamic Conditional Random Fields
6.4.2.3 Input Output Hidden Markov Models
6.4.2.4 Comparison of Graphical Models
6.4.3 Continuous Gesture Recognition
6.4.3.1 HMM Based Methods
6.4.3.2 CRF Based Methods
6.4.3.3 IOHMM Based Methods
6.5 Applications
6.5.1 SignTutor: An Interactive System for Sign Language Tutoring
6.5.1.1 System and Modules
6.5.1.2 Evaluation
6.5.2 STARS: Sign Tracking and Recognition System Using IOHMMs
6.5.2.1 System and Modules
6.5.2.2 Evaluation
6.6 Summary
6.7 Questions
6.8 Glossary
References
Chapter 7: Semantics of Human Behavior in Image Sequences
7.1 Introduction
7.2 State of the Art
7.2.1 Bottom-up Approaches in Behavior Understanding
7.2.2 Top-down Modeling of Behavior Understanding
7.2.3 Taxonomy of Human Events
7.2.4 Human Event Datasets and Benchmarks
7.3 Methodology
7.3.1 Architecture
7.3.2 Activity Recognition
7.3.2.1 Entity Level
7.3.2.2 Action Level: Pose Estimation
7.3.2.3 Activity Level: Spatial Interaction
7.3.2.4 Classification
7.3.2.5 Experiments and Results
7.3.3 Behavior Modeling
7.4 Summary
7.5 Questions
7.6 Glossary
References
Part III: Social and Affective Behaviors
Chapter 8: Social Signals: A Psychological Perspective
8.1 Introduction
8.2 The Dawn of Social Signals. From Computer Science to Social Psychology and Back
8.3 Social Cognition. How Others Are Represented in Our Mind and Our Brain
8.4 A Goal and Belief Model of Mind and Social Action
8.5 Social Signals. A Definition
8.5.1 Social and Non-social, Informative and Communicative Signals
8.5.2 Communicative Signals and Their Communicative Goals
8.5.3 Direct and Indirect Social Signals
8.6 Modalities of Social Signals
8.6.1 Verbal and Vocal Features
8.6.2 Gestures
8.6.3 Head movements
8.6.4 Gaze
8.6.4.1 Gaze and Social Attention
8.6.4.2 Lexicon and Parameters of Gaze
8.6.5 Posture
8.6.6 Proxemics and Touch
8.7 Social Facts and Their Signals
8.7.1 Social Interaction
8.7.2 Social Attitudes
8.7.2.1 Persuasion
8.7.2.2 Signals of Agreement
8.7.3 Social Relationships
8.7.3.1 Dominance and Its Signals
8.7.3.2 Blatant and Subtle Dominance Strategies
8.7.4 Social Emotions
8.7.4.1 Shame and the Multimodal Discourse of Blush
8.7.4.2 Pride
8.7.4.3 Enthusiasm
8.8 Summary
8.9 Questions
8.10 Glossary
References
Chapter 9: Voice and Speech Analysis in Search of States and Traits
9.1 Vocal Behaviour Analysis-an Introduction
9.1.1 A Short Motivation
9.1.2 From Affection to Zest
9.1.3 Principle
9.2 Voice'-the Acoustic Analysis 9.2.1 Chunking 9.2.2 Acoustic Feature Extraction 9.2.3 Feature Selection 9.2.4 Classification and Regression 9.2.5 Parameter Tuning 9.3Speech'-the (Non-)linguistic Analysis
9.3.1 Analysis of Linguistic Content
9.3.2 Analysis of Non-linguistic Content
9.3.3 (Non-)linguistic Feature Extraction
9.3.4 Classification and Regression
9.4 Data, Benchmarks, and Application Examples
9.4.1 Frequently Encountered Data-Sets and Their Benchmarks
9.4.2 Human-to-Human Conversation Analysis
9.4.3 Human-to-Agent Conversation Analysis
9.5 Summary
9.6 Questions
9.7 Glossary
References
Chapter 10: Continuous Analysis of Affect from Voice and Face
10.1 Introduction
10.2 Affect in Dimensional Space
10.3 Affect Dimensions and Signals
10.3.1 Affect Dimensions
10.3.2 Visual Signals
10.3.3 Audio Signals
10.3.4 Bio Signals
10.4 Overview of the Current Technology
10.4.1 Data Acquisition and Annotation
10.4.2 Automatic Dimensional Affect Prediction and Recognition
10.4.3 Challenges and Prospects
10.4.4 Applications
10.5 A Representative System: Continuous Analysis of Affect from Voice and Face
10.5.1 Dataset
10.5.2 Data Pre-processing and Segmentation
10.5.2.1 Annotation Pre-processing
Interpolation
Binning
Normalization
Statistics and Metrics
10.5.2.2 Automatic Segmentation
Detecting and Matching Crossovers
Segmentation Driven by Matched Crossovers
10.5.3 Feature Extraction
10.5.4 Dimensional Affect Prediction
10.5.4.1 Bidirectional Long Short-Term Memory Neural Networks
10.5.4.2 Single-Cue Prediction
10.5.4.3 Model-Level Fusion
10.5.4.4 Output-Associative Fusion
10.5.5 Experiments and Analysis
10.5.5.1 Experimental Setup
10.5.5.2 Results and Analysis
10.6 Concluding Remarks
10.7 Summary
10.8 Questions
10.9 Glossary
References
Chapter 11: Analysis of Group Conversations: Modeling Social Verticality
11.1 Introduction
11.2 Social Verticality in Human Interaction and Nonverbal Behavior
11.3 Automatic Analysis of Social Verticality from Nonverbal Features
11.3.1 Processing
11.3.1.1 Audio Processing
11.3.1.2 Video Processing
11.3.2 Feature Extraction
11.3.2.1 Audio Nonverbal Features
11.3.2.2 Visual Nonverbal Features
11.3.2.3 Other Sensors
11.3.3 Inference
11.3.3.1 Rule-Based Approaches
11.3.3.2 Unsupervised Approaches
11.3.3.3 Supervised Approaches
11.3.3.4 Temporal Modeling
11.4 Case Studies
11.4.1 Dominance Estimation
11.4.1.1 Task Definition and Data
11.4.1.2 Features and Model
11.4.1.3 Experiments and Results
11.4.2 Identifying Emergent Leaders
11.4.2.1 Task Definition and Data
11.4.2.2 Features and Model
11.4.2.3 Experiments and Results
Correlation Between the Questionnaires and the Nonverbal Features
Automatic Inference
11.4.3 Recognizing Functional Roles
11.4.3.1 Task Definition and Data
11.4.3.2 Features and Model
11.4.3.3 Experiments and Results
11.4.4 Discovering Leadership Styles in Group Conversations
11.4.4.1 Task Definition and Data
11.4.4.2 Features and Model
11.4.4.3 Experiments and Results
11.5 Summary
11.6 Questions
11.7 Glossary
References
Part IV: Selected Applications
Chapter 12: Activity Monitoring Systems in Health Care
12.1 Introduction
12.2 Sensing Systems
12.2.1 Body Worn Sensors
12.2.2 Wireless Sensor Networks
12.2.3 Visual Sensors
12.3 Monitoring of Simple Actions
12.3.1 Fall Detection
12.3.2 Wandering and Elopement
12.3.3 Prevention and Therapy
12.4 Recognition of More Complex Activities
12.5 Visualization, Coaching and Communication
12.6 Acceptance and Impact
12.7 Summary
12.8 Questions
12.9 Glossary
References
Chapter 13: Behavioral, Cognitive and Virtual Biometrics
13.1 Introduction to Behavioral Biometrics
13.2 Description of Behavioral Biometrics
13.2.1 Avatar Representation
13.2.2 Biometric Sketch
13.2.3 Blinking
13.2.4 Calling Behavior
13.2.5 Car Driving Style
13.2.6 Center of Gravity
13.2.7 Command Line Lexicon
13.2.8 Credit Card Use
13.2.9 Dynamic Facial Features
13.2.10 Email Behavior
13.2.11 Finger Pressure
13.2.12 Floor Pressure
13.2.13 Gaze/Eye Tracking
13.2.14 Gait/Stride
13.2.15 Game Strategy
13.2.16 Handgrip
13.2.17 Haptic
13.2.18 Human Shadows
13.2.19 Keystroke Dynamics
13.2.20 Lip Movement
13.2.21 Mouse Dynamics
13.2.22 Motion of Fingers
13.2.23 Painting Style
13.2.24 Programming Style
13.2.25 Signature/Handwriting
13.2.26 Short Term Memory
13.2.27 Soft Behavioral Biometrics
13.2.28 Tapping
13.2.29 Text Authorship
13.2.30 Visual Scan/Search and Detection
13.2.31 Voice/Speech/Singing
13.3 Biological Signals as a Behavioral Biometrics
13.3.1 Behavioral Passwords
13.3.1.1 Text-Based Behavioral Passwords
13.3.1.2 Graphics-Based Behavioral Passwords
13.3.2 Comparison and Analysis
13.4 Privacy Concerns
13.5 Summary
13.6 Questions
13.7 Glossary
References
Chapter 14: Human Behavior Analysis in Ambient Gaming and Playful Interaction
14.1 Introduction
14.2 History of Games
14.3 The Gamer Put into Action
14.3.1 More Advanced Interaction: Audiovisual Based Input
14.3.2 Other (Physiological) Sensors and Wearables
14.4 Game Experience and Human Behavior Analysis
14.5 Summary
14.6 Glossary
References
Index