Patterns Identification and Data Mining in Weather and Climate (Springer Atmospheric Sciences)

✍ Scribed by Abdelwaheb Hannachi

• Publisher: Springer
• Year: 2021
• Tongue: English
• Leaves: 607
• Category: Library

✦ Synopsis

Advances in computer power and observing systems has led to the generation and accumulation of large scale weather & climate data begging for exploration and analysis. Pattern Identification and Data Mining in Weather and Climate presents, from different perspectives, most available, novel and conventional, approaches used to analyze multivariate time series in climate science to identify patterns of variability, teleconnections, and reduce dimensionality. The book discusses different methods to identify patterns of spatiotemporal fields. The book also presents machine learning with a particular focus on the main methods used in climate science. Applications to atmospheric and oceanographic data are also presented and discussed in most chapters. To help guide students and beginners in the field of weather & climate data analysis, basic Matlab skeleton codes are given is some chapters, complemented with a list of software links toward the end of the text. A number oftechnical appendices are also provided, making the text particularly suitable for didactic purposes.

The topic of EOFs and associated pattern identification in space-time data sets has gone through an extraordinary fast development, both in terms of new insights and the breadth of applications. We welcome this text by Abdel Hannachi who not only has a deep insight in the field but has himself made several contributions to new developments in the last 15 years.

- Huug van den Dool, Climate Prediction Center, NCEP, College Park, MD, U.S.A.

Now that weather and climate science is producing ever larger and richer data sets, the topic of pattern extraction and interpretation has become an essential part. This book provides an up to date overview of the latest techniques and developments in this area.

- Maarten Ambaum, Department of Meteorology, University of Reading, U.K.

This nicely and expertly written book covers a lot of ground, ranging from classical linear pattern identification techniques to more modern machine learning, illustrated with examples from weather & climate science. It will be very valuable both as a tutorial for graduate and postgraduate students and as a reference text for researchers and practitioners in the field.

- Frank Kwasniok, College of Engineering, University of Exeter, U.K.

✦ Table of Contents

Preface
Pattern Identification and Data Mining in Weather and Climate
Acknowledgements
Contents
1 Introduction
1.1 Complexity of the Climate System
1.2 Data Exploration, Data Mining and Feature Extraction
1.3 Major Concern in Climate Data Analysis
1.3.1 Characteristics of High-Dimensional SpaceGeometry
Volume Paradox of Hyperspheres
Two Further Paradoxical Examples
1.3.2 Curse of Dimensionality and Empty SpacePhenomena
1.3.3 Dimension Reduction and Latent Variable Models
1.3.4 Some Problems and Remedies in Dimension Reduction
Known Difficulties
Some Remedies
1.4 Examples of the Most Familiar Techniques
2 General Setting and Basic Terminology
2.1 Introduction
2.2 Simple Visualisation Techniques
2.3 Data Processing and Smoothing
2.3.1 Preliminary Checking
2.3.2 Smoothing
Moving Average
Exponential Smoothing
Spline Smoothing
Kernel Smoothing
2.3.3 Simple Descriptive Statistics
2.4 Data Set-Up
2.5 Basic Notation/Terminology
2.5.1 Centring
2.5.2 Covariance Matrix
2.5.3 Scaling
2.5.4 Sphering
2.5.5 Singular Value Decomposition
2.6 Stationary Time Series, Filtering and Spectra
2.6.1 Univariate Case
2.6.2 Multivariate Case
3 Empirical Orthogonal Functions
3.1 Introduction
3.2 Eigenvalue Problems in Meteorology: Historical Perspective
3.2.1 The Quest for Climate Patterns: Teleconnections
3.2.2 Eigenvalue Problems in Meteorology
3.3 Computing Principal Components
3.3.1 Basis of Principal Component Analysis
3.3.2 Karhunen–Loéve Expansion
3.3.3 Derivation of PCs/EOFs
3.3.4 Computing EOFs and PCs
Singular Value Decomposition and Similar Algorithms
Basic Iterative Approaches
3.4 Sampling, Properties and Interpretation of EOFs
3.4.1 Sampling Variability and Uncertainty
Uncertainty Based on Asymptotic Approximation
Probabilistic PCA
Monte-Carlo Resampling Methods
Bootstrap Application to EOFs of Atmospheric Fields
3.4.2 Independent and Effective Sample Sizes
Serial Correlation
Time Varying Fields
3.4.3 Dimension Reduction
3.4.4 Properties and Interpretation
3.5 Covariance Versus Correlation
3.6 Scaling Problems in EOFs
3.7 EOFs for Multivariate Normal Data
3.8 Other Procedures for Obtaining EOFs
3.9 Other Related Methods
3.9.1 Teleconnectivity
3.9.2 Regression Matrix
3.9.3 Empirical Orthogonal Teleconnection
3.9.4 Climate Network-Based Methods
4 Rotated and Simplified EOFs
4.1 Introduction
4.2 Rotation of EOFs
4.2.1 Background on Rotation
4.2.2 Derivation of REOFs
4.2.3 Computing REOFs
Rotation or Simplicity Criteria
Computation of REOFs
4.3 Simplified EOFs: SCoTLASS
4.3.1 Background
4.3.2 LASSO-Based Simplified EOFs
4.3.3 Computing the Simplified EOFs
5 Complex/Hilbert EOFs
5.1 Background
5.2 Conventional Complex EOFs
5.2.1 Pairs of Scalar Fields
5.2.2 Single Field
5.3 Frequency Domain EOFs
5.3.1 Background
5.3.2 Derivation of FDEOFs
5.4 Complex Hilbert EOFs
5.4.1 Hilbert Transform: Continuous Signals
5.4.2 Hilbert Transform: Discrete Signals
5.4.3 Application to Time Series
5.4.4 Complex Hilbert EOFs
Complexified Field
Computational Aspects
5.5 Rotation of HEOFs
6 Principal Oscillation Patterns and Their Extension
6.1 Introduction
6.2 POP Derivation and Estimation
6.2.1 Spatial Patterns
6.2.2 Time Coefficients
6.2.3 Example
6.3 Relation to Continuous POPs
6.3.1 Basic Relationships
6.3.2 Finite Time POPs
6.4 Cyclo-Stationary POPs
6.5 Other Extensions/Interpretations of POPs
6.5.1 POPs and Normal Modes
6.5.2 Complex POPs
6.5.3 Hilbert Oscillation Patterns
6.5.4 Dynamic Mode Decomposition
6.6 High-Order POPs
6.7 Principal Interaction Patterns
7 Extended EOFs and SSA
7.1 Introduction
7.2 Dynamical Reconstruction and SSA
7.2.1 Background
7.2.2 Dynamical Reconstruction and SSA
7.3 Examples
7.3.1 White Noise
7.3.2 Red Noise
7.4 SSA and Periodic Signals
7.5 Extended EOFs or Multivariate SSA
7.5.1 Background
7.5.2 Definition and Computation of EEOFs
7.5.3 Data Filtering and Oscillation Reconstruction
7.6 Potential Interpretation Pitfalls
7.7 Alternatives to SSA and EEOFs
7.7.1 Recurrence Networks
7.7.2 Data-Adaptive Harmonic Decomposition
8 Persistent, Predictive and Interpolated Patterns
8.1 Introduction
8.2 Background on Persistence and Prediction of Stationary Time Series
8.2.1 Decorrelation Time
8.2.2 The Prediction Problem and Kolmogorov Formula
8.3 Optimal Persistence and Average Predictability
8.3.1 Derivation of Optimally Persistent Patterns
8.3.2 Estimation from Finite Samples
8.3.3 Average Predictability Patterns
8.4 Predictive Patterns
8.4.1 Introduction
8.4.2 Optimally Predictable Patterns
8.4.3 Computational Aspects
8.5 Optimally Interpolated Patterns
8.5.1 Background
8.5.2 Interpolation and Pattern Derivation
8.5.3 Numerical Aspects
8.5.4 Application
8.6 Forecastable Component Analysis
9 Principal Coordinates or Multidimensional Scaling
9.1 Introduction
9.2 Dissimilarity Measures
9.3 Metric Multidimensional Scaling
9.3.1 The Problem of Classical Scaling
9.3.2 Principal Coordinate Analysis
Classical Scaling in Presence of Errors
9.3.3 Case of Non-Euclidean Dissimilarity Matrix
9.4 Non-metric Scaling
9.5 Further Extensions
9.5.1 Replicated and Weighted MDS
9.5.2 Nonlinear Structure
9.5.3 Application to the Asian Monsoon
9.5.4 Scaling and the Matrix Nearness Problem
10 Factor Analysis
10.1 Introduction
10.2 The Factor Model
10.2.1 Background
10.2.2 Model Definition and Terminology
10.2.3 Model Identification
Case of Autocorrelated Factors
10.2.4 Non-unicity of Loadings
10.3 Parameter Estimation
10.3.1 Maximum Likelihood Estimates
10.3.2 Expectation Maximisation Algorithm
Necessary Optimality Condition
Expectation Maximisation Algorithm
Model Assessment
10.4 Factor Rotation
10.4.1 Oblique and Orthogonal Rotations
10.4.2 Examples of Rotation Criteria
• Quartimax
• Quartimin
• Oblimin
• Oblimax
• Entropy
10.5 Exploratory FA and Application to SLP Anomalies
10.5.1 Factor Analysis as a Matrix Decomposition Problem
10.5.2 A Factor Rotation
10.6 Basic Difference Between EOF and Factor Analyses
10.6.1 Comparison Based on the Standard Factor Model
10.6.2 Comparison Based on the Exploratory Factor Analysis Model
11 Projection Pursuit
11.1 Introduction
11.2 Definition and Purpose of Projection Pursuit
11.2.1 What Is Projection Pursuit?
11.2.2 Why Projection Pursuit?
11.3 Entropy and Structure of Random Variables
11.3.1 Shannon Entropy
11.3.2 Differential Entropy
11.4 Types of Projection Indexes
11.4.1 Quality of a Projection Index
11.4.2 Various PP Indexes
Friedman and Tukey's Index
Jones and Sibson's Index
Entropy/Information Index
Moments-Based Indexes
Friedman and Related L2 Norm-Based Indices
Chi-Square Index
Clustering Index
11.4.3 Practical Implementation
11.5 PP Regression and Density Estimation
11.5.1 PP Regression
11.5.2 PP Density Estimation
11.6 Skewness Modes and Climate Application of PP
12 Independent Component Analysis
12.1 Introduction
12.2 Background and Definition
12.2.1 Blind Deconvolution
12.2.2 Blind Source Separation
12.2.3 Definition of ICA
12.3 Independence and Non-normality
12.3.1 Statistical Independence
12.3.2 Non-normality
12.4 Information-Theoretic Measures
12.4.1 Entropy
12.4.2 Kullback–Leibler Divergence
Properties of the K–L Divergence
12.4.3 Mutual Information
12.4.4 Negentropy
12.4.5 Useful Approximations
12.5 Independent Component Estimation
12.5.1 Choice of Objective Function for ICA
Negentropy
Non-normality
Information-Theoretic Approach
Likelihood Maximisation Approach
Information Maximisation Approach
A Non-parametric Approach
Other Methods
12.5.2 Numerical Implementation
Sphering/Whitening
Optimisation Algorithms
12.6 ICA via EOF Rotation and Weather and Climate Application
12.6.1 The Standard Two-Way Problem
12.6.2 Extension to the Three-Way Data
12.7 ICA Generalisation: Independent Subspace Analysis
13 Kernel EOFs
13.1 Background
13.2 Kernel EOFs
13.2.1 Formulation of Kernel EOFs
13.2.2 Practical Details of Kernel EOF Computation
13.2.3 Illustration with Concentric Clusters
13.3 Relation to Other Approaches
13.3.1 Spectral Clustering
13.3.2 Modularity Clustering
13.4 Pre-images in Kernel PCA
13.5 Application to An Atmospheric Model and Reanalyses
13.5.1 Application to a Simplified Atmospheric Model
13.5.2 Application to Reanalyses
13.6 Other Extensions of Kernel EOFs
13.6.1 Extended Kernel EOFs
Direct Formulation
Alternative Formulations
13.6.2 Kernel POPs
14 Functional and Regularised EOFs
14.1 Functional EOFs
14.2 Functional PCs and Discrete Sampling
14.3 An Example of Functional PCs from Oceanography
14.4 Regularised EOFs
14.4.1 General Setting
14.4.2 Case of Spatial Fields
The Example of the RBF Solution
14.5 Numerical Solution of the Full Regularised EOF Problem
14.6 Application of Regularised EOFs to SLP Anomalies
15 Methods for Coupled Patterns
15.1 Introduction
15.2 Canonical Correlation Analysis
15.2.1 Background
Definition
15.2.2 Formulation of CCA
15.2.3 Computational Aspect
Estimation Using Sample Covariance Matrix
Estimation Using EOFs
15.2.4 Regularised CCA
15.2.5 Use of Correlation Matrices
15.3 Canonical Covariance Analysis
15.4 Redundancy Analysis
15.4.1 Redundancy Index
15.4.2 Redundancy Analysis
15.5 Application: Optimal Lag Between Two Fields and Other Extensions
15.5.1 Application of CCA
15.5.2 Application of Redundancy
15.6 Principal Predictors
15.7 Extension: Functional Smooth CCA
15.7.1 Introduction
15.7.2 Functional Non-smooth CCA and Indeterminacy
15.7.3 Smooth CCA/MCA
Canonical Correlation
Application
Maximum Covariance
15.7.4 Application of SMCA to Space–Time Fields
15.8 Some Points on Coupled Patterns and Multivariate Regression
16 Further Topics
16.1 Introduction
16.2 EOFs and Random Projection
16.3 Cyclo-stationary EOFs
16.3.1 Background
16.3.2 Theory of Cyclo-stationary EOFs
16.3.3 Application of CSEOFs
16.4 Trend EOFs
16.4.1 Motivation
16.4.2 Trend EOFs
16.4.3 Application of Trend EOFs
Illustration with a Simple Example
Application to Reanalysis Data
16.5 Common EOF Analysis
16.5.1 Background
16.5.2 Formulation of Common EOFs
16.6 Continuum Power CCA
16.6.1 Background
16.6.2 Continuum Power CCA
16.6.3 Determination of the Degree Parameter
16.7 Kernel MCA
16.7.1 Background
16.7.2 Kernel MCA
16.7.3 An Alternative Way
16.8 Kernel CCA and Its Regularisation
16.8.1 Primal and Dual CCA Formulation
16.8.2 Regularised KCCA
16.8.3 Some Computational Issues
16.9 Archetypal Analysis
16.9.1 Background
16.9.2 Derivation of Archetypes
16.9.3 Numerical Solution of Archetypes
Alternating Algorithm
Riemannian Manifold-Based Optimisation
16.9.4 Archetypes and Simplex Visualisation
16.9.5 An Application of AA to Climate
16.10 Other Nonlinear PC Methods
16.10.1 Principal Nonlinear Dynamical Modes
16.10.2 Nonlinear PCs via Neural Networks
17 Machine Learning
17.1 Background
17.2 Neural Networks
17.2.1 Background and Rationale
17.2.2 General Structure of Neural Networks
17.2.3 Examples of Architectures
17.2.4 Learning Procedure in NNs
17.2.5 Costfunctions for Multiclass Classification
17.3 Self-organising Maps
17.3.1 Background
17.3.2 SOM Algorithm
Mapping Identification and Kohonen Network
Training of SOM
Summary of SOM Algorithm
17.4 Random Forest
17.4.1 Decision Trees
What Are They?
Classification and Regression Trees
17.4.2 Random Forest: Definition and Algorithm
17.4.3 Out-of-Bag Data, Generalisation Error and Tuning
Out-of-Bag Data and Generalisation Error
Parameter Selection
17.5 Application
17.5.1 Neural Network Application
NN Nonlinear PCs
Application to Weather Forecasting
17.5.2 SOM Application
17.5.3 Random Forest Application
A Smoothing Techniques
A.1 Smoothing Splines
A.1.1 More on Smoothing Splines
A.1.2 Choice of the Smoothing Parameter
A.2 Radial Basis Functions
A.2.1 Exact Interpolation
Examples of Radial Basis Functions
Example: Thin-Plate Spline
A.2.2 RBF and Noisy Data
A.2.3 Relation to PDEs and Other Techniques
A.3 Kernel Smoother
B Introduction to Probability and Random Variables
B.1 Background
B.2 Sets Theory and Probability
B.2.1 Elements of Sets Theory
Sets and Subsets
Operations on Subsets
B.2.2 Definition of Probability
Link to Sets Theory
Definition/Axioms of Probability
Properties of Probability
B.3 Random Variables and Probability Distributions
B.3.1 Discrete Probability Distributions
B.3.2 Continuous Probability Distributions
Moments of a Random Variable
Cumulants
B.3.3 Joint Probability Distributions
B.3.4 Expectation and Covariance Matrix of Random Vectors
B.3.5 Conditional Distributions
B.4 Examples of Probability Distributions
B.4.1 Discrete Case
Bernoulli Distribution
Binomial Distribution
Negative Binomial Distribution
Poisson Distribution
B.4.2 Continuous Distributions
The Uniform Distribution
The Normal Distribution
The Exponential Distribution
The Gamma Distribution
The Chi-Square Distribution
The Student Distribution
The Fisher–Snedecor Distribution
The Multivariate Normal Distribution
The Wishart Distribution
B.5 Stationary Processes
C Stationary Time Series Analysis
C.1 Autocorrelation Structure: One-Dimensional Case
C.1.1 Autocovariance/Correlation Function
Properties of the Autocovariance Function
C.1.2 Time Series Models
Some Basic Notations
ARMA Models
C.2 Power Spectrum
C.3 The Multivariate Case
C.3.1 Autocovariance Structure
C.3.2 Cross-Spectrum
C.4 Autocorrelation Structure in the Sample Space
C.4.1 Autocovariance/Autocorrelation Estimates
C.4.2 The Periodogram
Raw Periodogram
Periodogram Smoothing
D Matrix Algebra and Matrix Function
D.1 Background
D.1.1 Matrices and Linear Operators
Matrices
Matrices and Linear Operators
D.1.2 Operation on Matrices
Transpose
Product
Diagonal
Trace
Determinant
Matrix Inversion
Symmetry, Orthogonality and Normality
Direct Product
Positivity
Eigenvalues/Eigenvectors
Some Properties of Square Matrices
Singular Value Decomposition (SVD)
Theorem of Sums of Products
Theorem of Partitioned Matrices
D.2 Most Useful Matrix Transformations
D.3 Matrix Derivative
D.3.1 Vector Derivative
D.3.2 Matrix Derivative
D.3.3 Examples
Case of Independent Elements
Case of Symmetric Matrices
D.4 Application
D.4.1 MLE of the Parameters of a MultinormalDistribution
D.4.2 Estimation of the Factor Model Parameters
D.4.3 Application to Results from PCA
D.5 Common Algorithms for Linear Systems and Eigenvalue Problems
D.5.1 Direct Methods
D.5.2 Iterative Methods
Case of Eigenvalue Problems
Case of Linear Systems
E Optimisation Algorithms
E.1 Background
E.2 Single Variable
E.2.1 Direct Search
E.2.2 Derivative Methods
E.3 Direct Multivariate Search
E.3.1 Downhill Simplex Method
E.3.2 Conjugate Direction/Powell's Method
E.3.3 Simulated Annealing
E.4 Multivariate Gradient-Based Methods
E.4.1 Steepest Descent
E.4.2 Newton–Raphson Method
E.4.3 Conjugate Gradient
E.4.4 Quasi-Newton Method
E.4.5 Ordinary Differential Equations-Based Methods
E.5 Constrained Minimisation
E.5.1 Background
E.5.2 Approaches for Constrained Minimisation
Lagrangian Method
Penalty Function
Gradient Projection
Other Gradient-Related Methods
F Hilbert Space
F.1 Linear Vector and Metric Spaces
F.1.1 Linear Vector Space
F.1.2 Metric Space
F.2 Norm and Inner Products
F.2.1 Norm
F.2.2 Inner Product
F.2.3 Consequences
F.2.4 Properties
F.3 Hilbert Space
F.3.1 Completeness
F.3.2 Hilbert Space
Examples of Hilbert Space
F.3.3 Application to Prediction
The Univariate Case
The Multivariate Case
G Systems of Linear Ordinary Differential Equations
G.1 Case of a Constant Matrix A
G.1.1 Homogeneous System
G.1.2 Non-homogeneous System
G.2 Case of a Time-Dependent Matrix A
G.2.1 General Case
G.2.2 Particular Case of Periodic Matrix A: Floquet Theory
H Links for Software Resource Material
References
Index