Tensor Computation for Data Analysis

✍ Scribed by Yipeng Liu, Jiani Liu, Zhen Long, Ce Zhu

• Publisher: Springer
• Year: 2021
• Tongue: English
• Leaves: 347
• Edition: 1st ed. 2022
• Category: Library

✦ Synopsis

Tensor is a natural representation for multi-dimensional data, and tensor computation can avoid possible multi-linear data structure loss in classical matrix computation-based data analysis.

 

This book is intended to provide non-specialists an overall understanding of tensor computation and its applications in data analysis, and benefits researchers, engineers, and students with theoretical, computational, technical and experimental details. It presents a systematic and up-to-date overview of tensor decompositions from the engineer's point of view, and comprehensive coverage of tensor computation based data analysis techniques. In addition, some practical examples in machine learning, signal processing, data mining, computer vision, remote sensing, and biomedical engineering are also presented for easy understanding and implementation. These data analysis techniques may be further applied in other applications on neuroscience, communication, psychometrics, chemometrics, biometrics, quantum physics, quantum chemistry, etc.

 

The discussion begins with basic coverage of notations, preliminary operations in tensor computations, main tensor decompositions and their properties. Based on them, a series of tensor-based data analysis techniques are presented as the tensor extensions of their classical matrix counterparts, including tensor dictionary learning, low rank tensor recovery, tensor completion, coupled tensor analysis, robust principal tensor component analysis, tensor regression, logistical tensor regression, support tensor machine, multilinear discriminate analysis, tensor subspace clustering, tensor-based deep learning, tensor graphical model and tensor sketch. The discussion also includes a number of typical applications with experimental results, such as image reconstruction, image enhancement, data fusion, signal recovery, recommendation system, knowledge graph acquisition, traffic flow prediction, link prediction, environmental prediction, weather forecasting, background extraction, human pose estimation, cognitive state classification from fMRI, infrared small target detection, heterogeneous information networks clustering, multi-view image clustering, and deep neural network compression.

✦ Table of Contents

Preface
Contents
Acronyms
Symbols
1 Tensor Computation
1.1 Notations
1.1.1 Special Examples
1.2 Basic Matrix Computation
1.3 Tensor Graphical Representation
1.4 Tensor Unfoldings
1.4.1 Mode-n Unfolding
1.4.2 Mode-n1n2 Unfolding
1.4.3 n-Unfolding
1.4.4 l-Shifting n-Unfolding
1.5 Tensor Products
1.5.1 Tensor Inner Product
1.5.2 Mode-n Product
1.5.3 Tensor Contraction
1.5.4 t-Product
1.5.5 3-D Convolution
1.6 Summary
References
2 Tensor Decomposition
2.1 Introduction
2.2 Canonical Polyadic Decomposition
2.2.1 Tensor Rank
2.2.2 CPD Computation
2.2.3 Uniqueness
2.3 Tucker Decomposition
2.3.1 The n-Rank
2.3.2 Computation and Optimization Model
2.3.3 Uniqueness
2.4 Block Term Decomposition
2.4.1 The Computation of BTD
2.4.2 Uniqueness
2.5 Tensor Singular Value Decomposition
2.6 Tensor Networks
2.6.1 Hierarchical Tucker Decomposition
2.6.1.1 Computation of Hierarchical Tucker Decomposition
2.6.1.2 Generalization of Hierarchical Tucker Decomposition
2.6.2 Tensor Train Decomposition
2.6.2.1 Computation of Tensor Train Decomposition
2.6.2.2 Generations of Tensor Train Decomposition
2.7 Hybrid Decomposition
2.7.1 Hierarchical Low-Rank Tensor Ring Decomposition
2.8 Scalable Tensor Decomposition
2.8.1 Scalable Sparse Tensor Decomposition
2.8.2 Strategy on Dense Tensor Decomposition
2.8.2.1 Scalable Distributed Tensor Decomposition
2.8.2.2 Randomized Tensor Decomposition
2.9 Summary and Future Work
References
3 Tensor Dictionary Learning
3.1 Matrix Dictionary Learning
3.1.1 Sparse and Cosparse Representation
3.1.2 Dictionary Learning Methods
3.1.2.1 The MOD Method
3.1.2.2 K-Singular Value Decomposition (K-SVD)
3.1.2.3 Analysis K-Singular Value Decomposition (Analysis K-SVD)
3.2 Tensor Dictionary Learning Based on DifferentDecompositions
3.2.1 Tucker Decomposition-Based Approaches
3.2.1.1 Classical Tucker-Based Tensor Dictionary Learning
3.2.1.2 Tucker-Based Tensor Dictionary Learning with Normalized Constraints
3.2.1.3 Tucker Based on Tensor Dictionary Learning with Orthogonal Constraints
3.2.2 CP Decomposition-Based Approaches
3.2.3 T-Linear-Based Approaches
3.3 Analysis Tensor Dictionary Learning
3.3.1 Analysis Tensor Dictionary Learning Based on Mode-n Product
3.3.1.1 Noiseless Analysis Tensor Dictionary Learning
3.3.1.2 Analysis Tensor Dictionary Learning with Noisy Observations
3.3.2 Tensor Convolutional Analysis Dictionary Learning Model
3.4 Online Tensor Dictionary Learning
3.5 Applications
3.5.1 Image Denoising
3.5.2 Fusing Hyperspectral and Multispectral Images
3.6 Summary
References
4 Low-Rank Tensor Recovery
4.1 Introduction
4.2 Matrix Completion
4.2.1 Rank Minimization Model
4.2.2 Low-Rank Matrix Completion Model
4.3 Tensor Completion
4.3.1 Rank Minimization Model
4.3.2 Low-Rank Tensor Factorization Model
4.4 Applications
4.4.1 Visual Data Recovery
4.4.2 Recommendation System
4.4.3 Knowledge Graph Completion
4.4.4 Traffic Flow Prediction
4.5 Summary
References
5 Coupled Tensor for Data Analysis
5.1 Introduction
5.2 Coupled Tensor Component Analysis Methods
5.2.1 Coupled Matrix and Tensor Factorization Model
5.2.2 Coupled Tensor Factorization Model
5.2.3 Generalized Coupled Tensor Factorization Model
5.3 Applications
5.3.1 HSI-MSI Fusion
5.3.2 Link Prediction in Heterogeneous Data
5.3.3 Visual Data Recovery
5.4 Summary
References
6 Robust Principal Tensor Component Analysis
6.1 Principal Component Analysis: From Matrix to Tensor
6.2 RPTCA Methods Based on Different Decompositions
6.2.1 t-SVD-Based RPTCA
6.2.1.1 Classical RPTCA Model and Algorithm
6.2.1.2 RPTCA with Different Sparse Constraints
6.2.1.3 Improved RPTCA with Different Scales
6.2.1.4 Improved RPTCA with Respect to Rotation Invariance
6.2.1.5 Improved RPTCA with Low-Rank Core Matrix
6.2.1.6 Improved TNN with Frequency Component Analysis
6.2.1.7 Nonconvex RPTCA
6.2.1.8 RPTCA with Transformed Domain
6.2.2 Higher-Order t-SVD-Based RPTCA
6.2.3 RTPCA Based on Other Tensor Decompositions
6.3 Online RPTCA
6.4 RTPCA with Missing Entries
6.5 Applications
6.5.1 Illumination Normalization for Face Images
6.5.2 Image Denoising
6.5.3 Background Extraction
6.5.4 Video Rain Streaks Removal
6.5.5 Infrared Small Target Detection
6.6 Summary
References
7 Tensor Regression
7.1 High-Dimensional Data-Related Regression Tasks
7.2 Tensor Regression Framework
7.3 Linear Tensor Regression Models
7.3.1 Simple Tensor Linear Regression
7.3.1.1 Rank Minimization Method
7.3.1.2 Alternating Least Squares Method
7.3.1.3 Greedy Low-Rank Learning
7.3.1.4 Projected Gradient Descent Method
7.3.2 Generalized Tensor Linear Regression
7.3.3 Penalized Tensor Regression
7.3.3.1 Tensor Ridge Regression
7.3.3.2 Tensor Sparse Regression
7.3.4 Bayesian Approaches for Tensor Linear Regression
7.3.5 Projection-Based Tensor Regression
7.3.5.1 Partial Least Squares Regression
7.3.5.2 Tensor Partial Least Squares Regression
7.4 Nonlinear Tensor Regression
7.4.1 Kernel-Based Learning
7.4.2 Gaussian Process Regression
7.4.3 Tensor Additive Models
7.4.4 Random Forest-Based Tensor Regression
7.5 Applications
7.5.1 Vector-on-Tensor Regression
7.5.2 Tensor-on-Vector Regression
7.5.3 Tensor-on-Tensor Regression
7.6 Summary
References
8 Statistical Tensor Classification
8.1 Introduction
8.2 Tensor Classification Basics
8.3 Logistic Tensor Regression
8.3.1 Logistic Regression
8.3.2 Logistic Tensor Regression
8.4 Support Tensor Machine
8.4.1 Support Vector Machine
8.4.2 Support Tensor Machine
8.4.3 Higher-Rank Support Tensor Machine
8.4.4 Support Tucker Machine
8.5 Tensor Fisher Discriminant Analysis
8.5.1 Fisher Discriminant Analysis
8.5.2 Tensor Fisher Discriminant Analysis
8.6 Applications
8.6.1 Handwriting Digit Recognition
8.6.2 Biomedical Classification from fMRI Images
8.7 Summary
References
9 Tensor Subspace Cluster
9.1 Background
9.2 Tensor Subspace Cluster Based on K-Means
9.2.1 Matrix-Based K-Means Clustering
9.2.2 CP Decomposition-Based Clustering
9.2.3 Tucker Decomposition-Based Clustering
9.3 Tensor Subspace Cluster Based on Self-Representation
9.3.1 Matrix Self-Representation-Based Clustering
9.3.2 Self-Representation in Tucker Decomposition Form
9.3.3 Self-Representation in t-SVD Form
9.4 Applications
9.4.1 Heterogeneous Information Networks Clustering
9.4.2 Biomedical Signals Clustering
9.4.3 Multi-View Subspace Clustering
9.5 Summary
References
10 Tensor Decomposition in Deep Networks
10.1 Introduction to Deep Learning
10.1.1 Convolutional Neural Network
10.1.2 Recurrent Neural Network
10.2 Network Compression by Low-Rank Tensor Approximation
10.2.1 Network Structure Design
10.2.2 t-Product-Based DNNs
10.2.3 Tensor Contraction Layer
10.2.4 Deep Tensorized Neural Networks Based on Mode-n Product
10.2.5 Parameter Optimization
10.3 Understanding Deep Learning with Tensor Decomposition
10.4 Applications
10.4.1 MNIST Dataset Classification
10.4.2 CIFAR-10 Dataset Classification
10.5 Summary
References
11 Deep Networks for Tensor Approximation
11.1 Introduction
11.2 Classical Deep Neural Networks
11.3 Deep Unrolling
11.4 Deep Plug-and-Play
11.5 Applications
11.5.1 Classical Neural Networks for Tensor RankEstimation
11.5.1.1 Tensor Rank Learning Architecture
11.5.1.2 Tensor Rank Network with Pre-decomposition Architecture
11.5.1.3 Experimental Analysis
11.5.2 Deep Unrolling Models for Video Snapshot Compressive Imaging
11.5.2.1 Tensor FISTA-Net
11.5.2.2 Experimental Results
11.5.3 Deep PnP for Tensor Completion
11.5.3.1 Deep Plug-and-Play Prior for Low-Rank Tensor Completion
11.5.3.2 Experimental Results
11.6 Summary
References
12 Tensor-Based Gaussian Graphical Model
12.1 Background
12.2 Vector-Variate-Based Gaussian Graphical Model
12.3 Matrix-Variate-Based Gaussian Graphical Model
12.4 Tensor-Based Graphical Model
12.5 Applications
12.5.1 Environmental Prediction
12.5.2 Mice Aging Study
12.6 Summary
References
13 Tensor Sketch
13.1 Introduction
13.2 Count Sketch
13.3 Tensor Sketch
13.4 Higher-Order Count Sketch
13.5 Applications
13.5.1 Tensor Sketch for Tensor Decompositions
13.5.1.1 Tensor Sketch for CP Decomposition
13.5.1.2 Tensor Sketch for Tucker Decomposition
13.5.2 Tensor Sketch for Kronecker Product Regression
13.5.3 Tensor Sketch for Network Approximations
13.5.3.1 Tensor Sketch in Convolutional and Fully Connected Layers
13.5.3.2 Higher-Order Count Sketch for Tensor Regression Network
13.6 Conclusion
References
A Tensor Software
References
Index