Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains

✍ Scribed by Stephen A. Billings

• Publisher: Wiley
• Year: 2013
• Tongue: English
• Leaves: 607
• Edition: 1
• Category: Library

✦ Synopsis

Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains describes a comprehensive framework for the identification and analysis of nonlinear dynamic systems in the time, frequency, and spatio-temporal domains. This book is written with an emphasis on making the algorithms accessible so that they can be applied and used in practice.

Includes coverage of:

• The NARMAX (nonlinear autoregressive moving average with exogenous inputs) model
• The orthogonal least squares algorithm that allows models to be built term by term where the error reduction ratio reveals the percentage contribution of each model term
• Statistical and qualitative model validation methods that can be applied to any model class
• Generalised frequency response functions which provide significant insight into nonlinear behaviours
• A completely new class of filters that can move, split, spread, and focus energy
• The response spectrum map and the study of sub harmonic and severely nonlinear systems
• Algorithms that can track rapid time variation in both linear and nonlinear systems
• The important class of spatio-temporal systems that evolve over both space and time
• Many case study examples from modelling space weather, through identification of a model of the visual processing system of fruit flies, to tracking causality in EEG data are all included
  to demonstrate how easily the methods can be applied in practice and to show the insight that the algorithms reveal even for complex systems

NARMAX algorithms provide a fundamentally different approach to nonlinear system identification and signal processing for nonlinear systems. NARMAX methods provide models that are transparent, which can easily be analysed, and which can be used to solve real problems.

This book is intended for graduates, postgraduates and researchers in the sciences and engineering, and also for users from other fields who have collected data and who wish to identify models to help to understand the dynamics of their systems.

✦ Table of Contents

Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Tempora Domains
Copyright
Contents
Preface
1 Introduction
1.1 Introduction to System Identification
1.1.1 System Models and Simulation
1.1.2 Systems and Signals
1.1.3 System Identification
1.2 Linear System Identification
1.3 Nonlinear System Identification
1.4 NARMAX Methods
1.5 The NARMAX Philosophy
1.6 What is System Identification For?
1.7 Frequency Response of Nonlinear Systems
1.8 Continuous-Time, Severely Nonlinear, and Time-Varying Models and Systems
1.9 Spatio-temporal Systems
1.10 Using Nonlinear System Identification in Practice and Case Study Examples
References
2 Models for Linear and Nonlinear Systems
2.1 Introduction
2.2 Linear Models
2.2.1 Autoregressive Moving Average with Exogenous Input Model
2.2.1.1 FIR Model
2.2.1.2 AR Model
2.2.1.3 MA Model
2.2.1.4 ARMA Model
2.2.1.5 ARX Model
2.2.1.6 ARMAX Model
2.2.1.7 Box–Jenkins Model
2.2.2 Parameter Estimation for Linear Models
2.2.2.1 ARX Model Parameter Estimation – The Least Squares Algorithm
2.2.2.2 ARMAX Model Parameter Estimation – The Extended Least Squares Algorithm
2.3 Piecewise Linear Models
2.3.1 Spatial Piecewise Linear Models
2.3.1.1 Operating Regions
2.3.1.2 Parameter Estimation
2.3.1.3 Simulation Example
2.3.2 Models with Signal-Dependent Parameters
2.3.2.1 Decomposition of Signal-Dependent Models
2.3.2.2 Parameter Estimation of Signal-Dependent Models
2.3.2.3 Simulation Example
2.3.3 Remarks on Piecewise Linear Models
2.4 Volterra Series Models
2.5 Block-Structured Models
2.5.1 Parallel Cascade Models
2.5.2 Feedback Block-Structured Models
2.6 NARMAX Models
2.6.1 Polynomial NARMAX Model
2.6.2 Rational NARMAX Model
2.6.2.1 Integral Model
2.6.2.2 Recursive Model
2.6.2.3 Output-affine Model
2.6.3 The Extended Model Set Representation
2.7 Generalised Additive Models
2.8 Neural Networks
2.8.1 Multi-layer Networks
2.8.2 Single-Layer Networks
2.8.2.1 Activation Functions
2.8.2.2 Radial Basis Function Networks
2.9 Wavelet Models
2.9.1 Dynamic Wavelet Models
2.9.1.1 Random Noise
2.9.1.2 Coloured Noise
2.10 State-Space Models
2.11 Extensions to the MIMO Case
2.12 Noise Modelling
2.12.1 Noise-Free
2.12.2 Additive Random Noise
2.12.3 Additive Coloured Noise
2.12.4 General Noise
2.13 Spatio-temporal Models
References
3 Model Structure Detection and Parameter Estimation
3.1 Introduction
3.2 The Orthogonal Least Squares Estimator and the Error Reduction Ratio
3.2.1 Linear-in-the-Parameters Representation
3.2.2 The Matrix Form of the Linear-in-the-Parameters Representation
3.2.3 The Basic OLS Estimator
3.2.4 The Matrix Formulation of the OLS Estimator
3.2.5 The Error Reduction Ratio
3.2.6 An Illustrative Example of the Basic OLS Estimator
3.3 The Forward Regression OLS Algorithm
3.3.1 Forward Regression with OLS
3.3.1.1 The FROLS Algorithm
3.3.1.2 Variants of the FROLS Algorithm
3.3.2 An Illustrative Example of Forward Regression with OLS
3.3.3 The OLS Estimation Engine and Identification Procedure
3.4 Term and Variable Selection
3.5 OLS and Sum of Error Reduction Ratios
3.5.1 Sum of Error Reduction Ratios
3.5.2 The Variance of the s -Step-Ahead Prediction Error
3.5.3 The Final Prediction Error
3.5.4 The Variable Selection Algorithm
3.6 Noise Model Identification
3.6.1 The Noise Model
3.6.2 A Simulation Example with Noise Modelling
3.7 An Example of Variable and Term Selection for a Real Data Set
3.8 ERR is Not Affected by Noise
3.9 Common Structured Models to Accommodate Different Parameters
3.10 Model Parameters as a Function of Another Variable
3.10.1 System Internal and External Parameters
3.10.2 Parameter-Dependent Model Structure
3.10.3 Modelling Auxetic Foams – An Example of External Parameter-Dependent Model Identification
3.11 OLS and Model Reduction
3.12 Recursive Versions of OLS
References
4 Feature Selection and Ranking
4.1 Introduction
4.2 Feature Selection and Feature Extraction
4.3 Principal Components Analysis
4.4 A Forward Orthogonal Search Algorithm
4.4.1 The Basic Idea of the FOS-MOD Algorithm
4.4.2 Feature Detection and Ranking
4.4.3 Monitoring the Search Procedure
4.4.4 Illustrative Examples
4.5 A Basis Ranking Algorithm Based on PCA
4.5.1 Principal Component-Derived Multiple Regression
4.5.2 PCA-Based MFROLS Algorithms
4.5.3 An Illustrative Example
References
5 Model Validation
5.1 Introduction
5.2 Detection of Nonlinearity
5.3 Estimation and Test Data Sets
5.4 Model Predictions
5.4.1 One-Step-Ahead Prediction
5.4.2 Model Predicted Output
5.5 Statistical Validation
5.5.1 Correlation Tests for Input–Output Models
5.5.2 Correlation Tests for Time Series Models
5.5.3 Correlation Tests for MIMO Models
5.5.4 Output-Based Tests
5.6 Term Clustering
5.7 Qualitative Validation of Nonlinear Dynamic Models
5.7.1 Poincaré Sections
5.7.2 Bifurcation Diagrams
5.7.3 Cell Maps
5.7.4 Qualitative Validation in Nonlinear System Identification
5.7.4.1 Poincaré Maps for Model Validation
5.7.4.2 Bifurcation Diagrams for Model Validation
5.7.4.3 Poincaré Maps and Bifurcation Diagrams for Model Validation of Chaotic Systems
References
6 The Identification and Analysis of Nonlinear Systems in the Frequency Domain
6.1 Introduction
6.2 Generalised Frequency Response Functions
6.2.1 The Volterra Series Representation of Nonlinear Systems
6.2.1.1 The Volterra Series
6.2.1.2 Volterra Series Models of Continuous- and Discrete-Time Nonlinear Systems
6.2.2 Generalised Frequency Response Functions
6.2.3 The Relationship Between GFRFs and Output Response of Nonlinear Systems
6.2.3.1 The System Time Domain Output Response Representation Using GFRFs
6.2.3.2 The Relationship Between GFRFs and the System Frequency Domain Output Response
6.2.4 Interpretation of the Composition of the Output Frequency Response of Nonlinear Systems
6.2.5 Estimation and Computation of GFRFs
6.2.5.1 Multi-dimensional Spectral Estimation Approaches
6.2.5.2 Frequency-Domain Volterra System Identification Approaches
6.2.5.3 Parametric Model-Based Approach
6.2.6 The Analysis of Nonlinear Systems Using GFRFs
6.2.6.1 Summary of the Parametric Method of Estimating GFRFs
6.2.6.2 Case Study of a Real System
6.3 Output Frequencies of Nonlinear Systems
6.3.1 Output Frequencies of Nonlinear Systems under Multi-tone Inputs
6.3.2 Output Frequencies of Nonlinear Systems for General Inputs
6.4 Nonlinear Output Frequency Response Functions
6.4.1 Definition and Properties of NOFRFs
6.4.2 Evaluation of NOFRFs
6.4.3 Damage Detection Using NARMAX Modelling and NOFRF-Based Analysis
6.4.3.1 Basic Idea
6.4.3.2 Damage Detection Procedure
6.4.3.3 An Experimental Case Study
6.5 Output Frequency Response Function of Nonlinear Systems
6.5.1 Definition of the OFRF
6.5.2 Determination of the OFRF
6.5.2.1 Determining the OFRF Structure
6.5.2.2 Determining the OFRF `Coefficients’
6.5.3 Application of the OFRF to Analysis of Nonlinear Damping for Vibration Control
References
7 Design of Nonlinear Systems in the Frequency Domain – Energy Transfer Filters and Nonlinear Damping
7.1 Introduction
7.2 Energy Transfer Filters
7.2.1 The Time and Frequency Domain Representation of the NARX Model with Input Nonlinearity
7.2.2 Energy Transfer Filter Designs
7.2.2.1 The Problem Description
7.2.2.2 ETF Design for a Specified Input
7.2.2.3 ETF Designs Using Orthogonal Least Squares
7.2.2.4 ETF Design for Several Specified Inputs
7.3 Energy Focus Filters
7.3.1 Output Frequencies of Nonlinear Systems with Input Signal Energy Located in Two Separate Frequency Intervals
7.3.2 The Energy Focus Filter Design Procedure and an Example
7.4 OFRF -Based Approach for the Design of Nonlinear Systems in the Frequency Domain
7.4.1 OFRF -Based Design of Nonlinear Systems in the Frequency Domain
7.4.1.1 General Procedure for the OFRF -Based Design of Nonlinear Systems in the Frequency Domain
7.4.2 Design of Nonlinear Damping in the Frequency Domain for Vibration Isolation: An Experimental Study
7.4.2.1 Experimental Setup
7.4.2.2 Modelling the Experimental Vibration Isolation System
7.4.2.3 The OFRF -Based Design for Nonlinear Damping
References
8 Neural Networks for Nonlinear System Identification
8.1 Introduction
8.2 The Multi-layered Perceptron
8.3 Radial Basis Function Networks
8.3.1 Training Schemes for RBF Networks
8.3.2 Fixed Kernel Centres with a Single Width
8.3.3 Limitation of RBF Networks with a Single Kernel Width
8.3.4 Fixed Kernel Centres and Multiple Kernel Widths
8.4 Wavelet Networks
8.4.1 Wavelet Decompositions
8.4.2 Wavelet Networks
8.4.3 Limitations of Fixed Grid Wavelet Networks
8.4.4 A New Class of Wavelet Networks
8.4.4.1 The Structure of the New Wavelet Networks
8.4.4.2 Determining the Number of Candidate Wavelet Terms
8.4.4.3 Determining Significant Wavelet Terms
8.4.4.4 A Procedure to Construct a Wavelet Network
8.5 Multi-resolution Wavelet Models and Networks
8.5.1 Multi-resolution Wavelet Decompositions
8.5.2 Multi-resolution Wavelet Models and Networks
8.5.3 An Illustrative Example
References
9 Severely Nonlinear Systems
9.1 Introduction
9.2 Wavelet NARMAX Models
9.2.1 Nonlinear System Identification Using Wavelet Multi-resolution NARMAX Models
9.2.2 A Strategy for Identifying Nonlinear Systems
9.3.1.1 Fading Memory Requirement
9.3.1.2 Complex Nonlinear Phenomena
9.3 Systems that Exhibit Sub-harmonics and Chaos
9.3.1 Limitations of the Volterra Series Representation
9.3.2 Time Domain Analysis
9.4 The Response Spectrum Map
9.4.1 Introduction
9.4.2 Examples of the Response Spectrum Map
9.5 A Modelling Framework for Sub-harmonic and Severely Nonlinear Systems
9.5.1 Input Signal Decomposition
9.5.2 MISO NARX Modelling in the Time Domain
9.5.2.1 A Simulation Example
9.6 Frequency Response Functions for Sub-harmonic Systems
9.6.1 MISO Frequency Domain Volterra Representation
9.6.2 Generating the GFRFs from the MISO model
9.7 Analysis of Sub-harmonic Systems and the Cascade to Chaos
9.7.1 Frequency Domain Response Synthesis
9.7.2 An Example of Frequency Domain Analysis for Sub-harmonic Systems
References
10 Identification of Continuous-Time Nonlinear Models
10.1 Introduction
10.2 The Kernel Invariance Method
10.2.1 Definitions
10.2.2 Reconstructing the Linear Model Terms
10.2.3 Reconstructing the Quadratic Model Terms
10.2.4 Model Structure Determination
10.3 Using the GFRFs to Reconstruct Nonlinear Integro-differential Equation Models Without Differentiation
10.3.1 Introduction
10.3.2 Reconstructing the Linear Model Terms
10.3.3 Reconstructing the Quadratic Model Terms
10.3.4 Reconstructing the Higher-Order Model Terms
10.3.5 A Real Application
References
11 Time-Varying and Nonlinear System Identification
11.1 Introduction
11.2 Adaptive Parameter Estimation Algorithms
11.2.1 The Kalman Filter Algorithm
11.2.2 The RLS and LMS Algorithms
11.2.3 Some Practical Considerations for the KF, RLS, and LMS Algorithms
11.3 Tracking Rapid Parameter Variations Using Wavelets
11.3.1 A General Form of TV-ARX Model Using Wavelets
11.3.2 A Multi-wavelet Approach for Time-Varying Parameter Estimation
11.4 Time-Dependent Spectral Characterisation
11.4.1 The Definition of a Time-Dependent Spectral Function
11.5 Nonlinear Time-Varying Model Estimation
11.6 Mapping and Tracking in the Frequency Domain
11.6.1 Time-Varying Frequency Response Functions
11.6.2 First- and Second-Order TV-GFRFs
11.7 A Sliding Window Approach
References
12 Identification of Cellular Automata and N -State Models of Spatio-temporal Systems
12.1 Introduction
12.2 Cellular Automata
12.2.1 History of Cellular Automata
12.2.2 Discrete Lattice
12.2.3 Neighbourhood
12.2.4 Transition Rules
12.2.4.1 Truth Table
12.2.4.2 Boolean Function
12.2.4.3 Totalistic Rule
12.2.4.4 Probabilistic Rule
12.2.4.5 Polynomial Model
12.2.5 Simulation Examples of Cellular Automata
12.3 Identification of Cellular Automata
12.3.1 Introduction and Review
12.3.2 Polynomial Representation
12.3.3 Neighbourhood Detection and Rule Identification
12.3.3.1 Background
12.3.3.2 Neighbourhood Detection Based on the CA-OLS Algorithm
12.3.3.3 Neighbourhood Detection Based on Mutual Information
12.3.3.4 Rule Identification Based on a Coarse-to-Fine Approach
12.4 N -State Systems
12.4.1 Introduction to Excitable Media Systems
12.4.2 Simulation of Excitable Media
12.4.2.1 The Greenberg-Hasting Model
12.4.2.2 Hodgepodge Machine Model
12.4.3 Identification of Excitable Media Using a CA Model
12.4.3.1 Neighbourhood Detection
12.4.3.2 Rule Identification
12.4.4 General N -State Systems
12.4.4.1 Introduction
12.4.4.2 Identification of n-State Spatio-temporal Systems
References
13 Identification of Coupled Map Lattice and Partial Differential Equations of Spatio-temporal Systems
13.1 Introduction
13.2 Spatio-temporal Patterns and Continuous-State Models
13.2.1 Stem Cell Colonies
13.2.2 The Belousov–Zhabotinsky Reaction
13.2.3 Oxygenation in Brain
13.2.4 Growth Patterns
13.2.5 A Simulated Example Showing Spatio-temporal Chaos from CML Models
13.3 Identification of Coupled Map Lattice Models
13.3.1 Deterministic CML Models
13.3.1.1 Deterministic CML State-Space Models
13.3.1.2 Input–Output Representation of CMLs
13.3.1.3 Polynomial Representation
13.3.1.4 B-Spline Wavelet Representation
13.3.2 The Identification of Stochastic CML Models
13.4 Identification of Partial Differential Equation Models
13.4.1 Model Structure
13.4.2 Time Discretisation
13.4.3 Nonlinear Function Approximation
13.4.3.1 Approximation of the Nonlinear Function
13.4.3.2 Finite Difference Schemes for Spatial Derivatives
13.4.3.3 Dealing with the Boundary
13.5 Nonlinear Frequency Response Functions for Spatio-temporal Systems
13.5.1 A One-Dimensional Example
13.5.2 Higher-Order Frequency Response Functions
References
14 Case Studies
14.1 Introduction
14.2 Practical System Identification
14.3 Characterisation of Robot Behaviour
14.3.1 Door Traversal
14.3.2 Route Learning
14.4 System Identification for Space Weather and the Magnetosphere
14.5 Detecting and Tracking Iceberg Calving in Greenland
14.5.1 Causality Detection
14.5.2 Results
14.6 Detecting and Tracking Time-Varying Causality for EEG Data
14.6.1 Data Acquisition
14.6.2 Causality Detection
14.6.3 Detecting Linearity and Nonlinearity
14.7 The Identification and Analysis of Fly Photoreceptors
14.7.1 Identification of the Fly Photoreceptor
14.7.2 Model-Based System Analysis in the Time and Frequency Domain
14.8 Real-Time Diffuse Optical Tomography Using RBF Reduced-Order Models of the Propagation of Light for Monitoring Brain Haemodynamics
14.8.1 Diffuse Optical Imaging
14.8.1.1 The Forward Problem
14.8.1.2 Image Reconstruction
14.8.2 In-vivo Real-Time 3-D Brain Imaging Using Reduced-Order Forward Models
14.8.2.1 Tomographic Reconstruction Algorithm Using Reduced-Order Forward Models
14.8.2.2 Experiment Description
14.8.2.3 Incorporating the Anatomical and Functional a priori Information
14.8.2.4 Image Reconstruction Algorithm
14.8.2.5 Results
14.9 Identification of Hysteresis Effects in Metal Rubber Damping Devices
14.9.1 Dynamic Modelling of Metal Rubber Damping Devices
14.9.2 Model Identification of a Metal Rubber Specimen
14.10 Identification of the Belousov–Zhabotinsky Reaction
14.10.1 Data Acquisition
14.10.2 Model Identification
14.10.2.1 Chemical Oscillation Frequency
14.10.2.2 Propagation Speed
14.10.2.3 Model Validation
14.11 Dynamic Modelling of Synthetic Bioparts
14.11.1 The Biopart and the Experiments
14.11.2 NARMAX Model of the Synthetic Biopart
14.12 Forecasting High Tides in the Venice Lagoon
14.12.1 Time Series Forecasting Problem
14.12.2 Water-Level Modelling and High-Tide Forecasting
14.12.2.1 The Data
14.12.2.2 The Model
14.12.2.3 Prediction Results
References
Index
Supplemental Images