Linear Algebra for Data Science

✍ Scribed by Moshe Haviv

• Publisher: World Scientific Publishing Company
• Year: 2023
• Tongue: English
• Leaves: 256
• Category: Library

✦ Synopsis

This book serves as an introduction to linear algebra for undergraduate students in data science, statistics, computer science, economics, and engineering. The book presents all the essentials in rigorous (proof-based) manner, describes the intuition behind the results, while discussing some applications to data science along the way.The book comes with two parts, one on vectors, the other on matrices. The former consists of four chapters: vector algebra, linear independence and linear subspaces, orthonormal bases and the Gram-Schmidt process, linear functions. The latter comes with eight chapters: matrices and matrix operations, invertible matrices and matrix inversion, projections and regression, determinants, eigensystems and diagonalizability, symmetric matrices, singular value decomposition, and stochastic matrices. The book ends with the solution of exercises which appear throughout its twelve chapters.

✦ Table of Contents

Contents
Preface
Part 1. Vectors
1. Vector Algebra
1.1 Definition of Vectors
1.2 Vector Operations
1.3 Norm of a Vector and Orthogonality
1.4 Projecting One Vector on Another
2. Linear Independence and Linear Subspaces
2.1 Linear Independence
2.2 Linear Subspaces
2.3 Bases and Dimension
2.4 Projections on Subspaces
2.4.1 The complementary orthogonal subspace
2.5 Simple Linear Regression
3. Orthonormal Bases and the Gram–Schmidt Process
3.1 Orthonormal Bases
3.2 The Gram–Schmidt Process
3.3 Optimization Under Equality Constraints
4. Linear Functions
4.1 Definition
4.2 A Linear Function and Its Linear Subspaces
Part 2. Matrices
5. Matrices and Matrix Operations
5.1 Basic Concepts
5.2 Matrix by Vector Multiplication
5.3 The Rank of a Matrix
5.4 Linear Equations and Homogeneous Equations
5.5 Matrix by Matrix Multiplication
5.6 The QR Factorization
5.7 Row and Column Operations
5.8 Echelon Matrices and the Rank of a Matrix
6. Invertible Matrices and the Inverse Matrix
6.1 Left-Inverses
6.2 Right-Inverses
6.3 Invertible Matrices
6.4 Solving a Set of Linear Equations
6.5 Inverting Matrices
6.6 Inverting a Matrix by Row Operations
6.7 Change of Bases and Similar Matrices
6.8 Special Inverses
6.8.1 Block matrices
6.8.2 Rank-one updates
7. The Pseudo-Inverse Matrix, Projections and Regression
7.1 Least Squares Solutions
7.2 Simple Linear Regression
7.3 Multiple Linear Regression
8. Determinants
8.1 Permutations
8.2 The Determinant
8.3 Determinants and Row Operations
8.4 Minor Matrices and the Determinant
8.5 The Adjoint Matrix
8.6 Cramer’s Method for Solving Linear Equations
8.6.1 Fitting a polynomial
9. Eigensystems and Diagonalizability
9.1 The Characteristic Polynomial
9.2 Right and Left Eigensystems
9.3 Algebraic and Geometric Multiplicities of Eigenvalues
9.4 Similar Matrices and Their Eigensystems
9.5 Bases with Eigenvectors and Diagonalizable Matrices
9.6 The Spectral Representation
10. Symmetric Matrices
10.1 Eigensystems and Symmetric Matrices
10.2 Positive Matrices
10.2.1 Two criteria for the positiveness of matrices
10.3 Covariance Matrices
10.4 Computing Eigensystems: The Power Method and Deflation
10.5 Cholesky Factorization
10.6 Principal Components Analysis (PCA)
11. Singular Value Decomposition
11.1 Introduction and Some Preliminaries
11.2 Singular Value Decomposition
12. Stochastic Matrices
12.1 Introduction and the Case of Positive Transition Probabilities
12.2 The Limit Probabilities
12.2.1 Computing the limit probabilities
12.3 The Deviation Matrix
12.4 Generalizations
12.4.1 The irreducible and a-periodic case
12.4.2 The unichain and a-periodic case
12.4.3 The general a-periodic case
12.4.4 The periodic and irreducible case
13. Solutions to Exercises
13.1 Chapter 1
13.2 Chapter 2
13.3 Chapter 3
13.4 Chapter 4
13.5 Chapter 5
13.6 Chapter 6
13.7 Chapter 7
13.8 Chapter 8
13.9 Chapter 9
13.10 Chapter 10
13.11 Chapter 12
Bibliography
Index