Music Emotion Recognition (Multimedia Computing, Communication and Intelligence)

✍ Scribed by Yi-Hsuan Yang, Homer H. Chen

• Publisher: CRC Press
• Year: 2011
• Tongue: English
• Leaves: 251
• Series: Multimedia Computing, Communication and Intelligence
• Edition: 1
• Category: Library

✦ Synopsis

Providing a complete review of existing work in music emotion developed in psychology and engineering, Music Emotion Recognition explains how to account for the subjective nature of emotion perception in the development of automatic music emotion recognition (MER) systems. Among the first publications dedicated to automatic MER, it begins with a comprehensive introduction to the essential aspects of MER—including background, key techniques, and applications. This ground-breaking reference examines emotion from a dimensional perspective. It defines emotions in music as points in a 2D plane in terms of two of the most fundamental emotion dimensions according to psychologists—valence and arousal. The authors present a computational framework that generalizes emotion recognition from the categorical domain to real-valued 2D space. They also: Introduce novel emotion-based music retrieval and organization methods Describe a ranking-base emotion annotation and model training method Present methods that integrate information extracted from lyrics, chord sequence, and genre metadata for improved accuracy Consider an emotion-based music retrieval system that is particularly useful for mobile devices The book details techniques for addressing the issues related to: the ambiguity and granularity of emotion description, heavy cognitive load of emotion annotation, subjectivity of emotion perception, and the semantic gap between low-level audio signal and high-level emotion perception. Complete with more than 360 useful references, 12 example MATLAB® codes, and a listing of key abbreviations and acronyms, this cutting-edge guide supplies the technical understanding and tools needed to develop your own automatic MER system based on the automatic recognition model.

✦ Table of Contents

Contents......Page 6
Preface......Page 12
Abbreviations......Page 14
1.1 Importance of Music Emotion Recognition......Page 17
1.2 Recognizing the Perceived Emotion of Music......Page 20
1.3.1 Ambiguity and Granularity of Emotion Description......Page 22
1.3.2 Heavy Cognitive Load of Emotion Annotation......Page 23
1.3.3 Subjectivity of Emotional Perception......Page 24
1.3.4 Semantic Gap between Low-Level Audio Signal and High-Level Human Perception......Page 25
1.4 Summary......Page 28
2.1 Emotion Description......Page 31
2.1.1 Categorical Approach......Page 32
2.1.2 Dimensional Approach......Page 34
2.1.3 Music Emotion Variation Detection......Page 36
2.2 Emotion Recognition......Page 37
2.2.1 Categorical Approach......Page 38
2.2.2 Dimensional Approach......Page 45
2.2.3 Music Emotion Variation Detection......Page 47
2.3 Summary......Page 48
3. Music Features......Page 51
3.1 Energy Features......Page 52
3.2 Rhythm Features......Page 53
3.3 Temporal Features......Page 58
3.4 Spectrum Features......Page 60
3.5 Harmony Features......Page 67
3.6 Summary......Page 70
4.1 Adopting the Dimensional Conceptualization of Emotion......Page 71
4.2.1 Weighted Sum of Component Functions......Page 73
4.2.3 System Identification Approach (System ID)......Page 74
4.3.1 Regression Theory......Page 75
4.3.3 Regression Algorithms......Page 76
4.4 System Overview......Page 78
4.5.1 Data Collection......Page 79
4.5.2 Feature Extraction......Page 81
4.5.4 Regressor Training......Page 83
4.6.1 Consistency Evaluation of the Ground Truth......Page 84
4.6.2 Data Transformation......Page 86
4.6.3 Feature Selection......Page 87
4.6.4 Accuracy of Emotion Recognition......Page 90
4.6.5 Performance Evaluation for Music Emotion Variation Detection......Page 93
4.6.6 Performance Evaluation for Emotion Classification......Page 94
4.7 Summary......Page 95
5.1 Motivation......Page 97
5.2 Ranking-Based Emotion Annotation......Page 98
5.3 Computational Model for Ranking Music by Emotion......Page 100
5.3.2 Ranking Algorithms......Page 101
5.5 Implementation......Page 106
5.5.1 Data Collection......Page 108
5.5.2 Feature Extraction......Page 111
5.6 Performance Evaluation......Page 112
5.6.1 Cognitive Load of Annotation......Page 113
5.6.2 Accuracy of Emotion Recognition......Page 114
5.7 Discussion......Page 120
5.8 Summary......Page 121
6.1 Motivation......Page 123
6.2.1 Fuzzy k-NN Classifier......Page 124
6.2.2 Fuzzy Nearest-Mean Classifier......Page 125
6.3 System Overview......Page 128
6.4.2 Feature Extraction and Feature Selection......Page 129
6.5.2 Music Emotion Variation Detection......Page 130
6.6 Summary......Page 133
7.1 Motivation......Page 135
7.2 Personalized MER......Page 137
7.3 Groupwise MER......Page 138
7.4.1 Data Collection......Page 140
7.4.2 Personal Information Collection......Page 142
7.4.3 Feature Extraction......Page 143
7.5.1 Performance of the General Method......Page 144
7.5.2 Performance of GWMER......Page 146
7.6 Summary......Page 150
8.1 Problem Formulation......Page 151
8.2 Bag-of-Users Model......Page 152
8.3 Residual Modeling and Two-Layer Personalization Scheme......Page 153
8.4 Performance Evaluation......Page 155
8.5 Summary......Page 159
9.1 Motivation......Page 161
9.2 Problem Formulation......Page 162
9.3.1 Ground Truth Collection......Page 164
9.3.2 Regressor Training......Page 166
9.3.3 Regressor Fusion......Page 169
9.3.4 Output of Emotion Distribution......Page 172
9.4.2 Feature Extraction......Page 173
9.5.1 Comparison of Different Regression Algorithms......Page 177
9.5.2 Comparison of Different Distribution Modeling Methods......Page 178
9.5.3 Comparison of Different Feature Representations......Page 181
9.5.4 Evaluation of Regressor Fusion......Page 182
9.6 Discussion......Page 183
9.7 Summary......Page 188
10.1 Motivation......Page 189
10.2 Lyrics Feature Extraction......Page 190
10.2.1 Uni-Gram......Page 191
10.2.2 Probabilistic Latent Semantic Analysis (PLSA)......Page 192
10.2.3 Bi-Gram......Page 193
10.3 Multimodal MER System......Page 195
10.4.1 Comparison of Multimodal Fusion Methods......Page 197
10.4.2 Performance of PLSA Model......Page 199
10.5 Summary......Page 200
11.1 Chord Recognition......Page 202
11.1.2 Hidden Markov Model and N-Gram Model......Page 203
11.1.3 Chord Decoding......Page 205
11.2 Chord Features......Page 206
11.2.2 Chord Histogram......Page 207
11.4.1 Evaluation of Chord Recognition System......Page 208
11.4.2 Accuracy of Emotion Classification......Page 209
11.5 Summary......Page 211
12.1 Motivation......Page 213
12.2 Two-Layer Music Emotion Classification......Page 214
12.3.1 Data Collection......Page 215
12.3.2 Analysis of the Correlation between Genre and Emotion......Page 216
12.3.3 Evaluation of the Two-Layer Emotion Classification Scheme......Page 219
12.4 Summary......Page 221
13.1 Emotion-Based Music Retrieval......Page 223
13.3 Retrieval Methods......Page 224
13.3.4 Query by Lyrics and Emotion (QBLE)......Page 225
13.4 Implementation......Page 226
13.5 Summary......Page 228
14.1 Exploiting Vocal Timbre for MER......Page 229
14.2 Emotion Distribution Prediction Based on Rankings......Page 230
14.4 Situational Factors of Emotion Perception......Page 231
14.6 Music Retrieval and Organization in 3D Emotion Space......Page 232
References......Page 235