Exercises in Applied Mathematics: With a View toward Information Theory, Machine Learning, Wavelets, and Statistical Physics

✍ Scribed by Daniel Alpay

• Publisher: Birkhäuser
• Year: 2024
• Tongue: English
• Series: Chapman Mathematical Notes
• Edition: 1
• Category: Library

✦ Synopsis

This text presents a collection of mathematical exercises with the aim of guiding readers to study topics in statistical physics, equilibrium thermodynamics, information theory, and their various connections. It explores essential tools from linear algebra, elementary functional analysis, and probability theory in detail and demonstrates their applications in topics such as entropy, machine learning, error-correcting codes, and quantum channels. The theory of communication and signal theory are also in the background, and many exercises have been chosen from the theory of wavelets and machine learning. Exercises are selected from a number of different domains, both theoretical and more applied. Notes and other remarks provide motivation for the exercises, and hints and full solutions are given for many. For senior undergraduate and beginning graduate students majoring in mathematics, physics, or engineering, this text will serve as a valuable guide as theymove on to more advanced work.

✦ Table of Contents

Contents
1 Prologue
1.1 A Guide Throughout the Book
1.2 Introduction
1.3 The Source Partition Theorem: Entropy
1.4 Another Motivation for H
1.5 Shannon's Theorem for the Binary Symmetric Channel (BSC)
1.6 Repetition Code and Hamming Code
1.7 Maximum Entropy and a First Connection with Statistical Physics
1.8 The Case of Continuous Signals
1.9 Some Questions Related to Statistical Physics
1.10 Some Questions and Tools in Quantum Mechanics
1.11 Some Questions and Tools in Quantum Information Theory
1.12 Machine Learning
1.13 Solutions of the Exercises
1.14 Some Specific Exercises and Questions
Part I Algebra
2 Complements in Linear Algebra
2.1 Complex Numbers
2.2 Beyond Complex Numbers
2.3 Vector Spaces and Linear Transformations
2.4 Hermitian Forms and Positive Operators
2.5 The Perceptron Convergence Theorem
2.6 The Scalar Case
2.7 Solutions of the Exercises
3 Positive Semi-Definite Matrices
3.1 Matrices
3.2 Schur Complement
3.3 Newton's Formulas
3.4 Hermitian Matrices
3.5 Positive Semi-Definite Matrices
3.6 Tensor Products and the Tensor Algebra
3.7 Completely Positive Maps
3.8 Maximum Entropy Analysis and the Yule-Walker Equation
3.9 Solutions of the Exercises
4 Algebra and Error-Correcting Codes
4.1 Sets, Functions, and Groups
4.2 Rings, Ideals, and Fields
4.3 Finite Fields
4.4 Splitting Field
4.5 Error-Correcting Codes: Generalities
4.6 Linear Block Codes
4.7 A Code Which Corrects Two Errors
4.8 Cyclic Codes
4.9 Solutions of the Exercises
Part II Analysis
5 Complements in Real and Complex Analysis
5.1 Warm-Up Questions and Exercises
5.2 Some Known Facts: Bolzano, Rolle, and Cauchy et les autres
5.3 Convex and Concave Functions
5.4 Stirling's Formula
5.5 Ordinary Differential Equations
5.6 Liouville's Theorem
5.7 Multivariable Calculus and Differentials
5.8 Optimization
5.9 Analytic Functions
5.10 Solution of the Exercises
6 Complements in Functional Analysis
6.1 Topologies and Metric Spaces
6.2 Hilbert Spaces
6.3 The Lebesgue Space bold upper L 2 left parenthesis double struck upper R comma d t right parenthesisL2(R,dt)
6.4 Operators in Banach and Hilbert Space
6.5 The Fourier Transform and Distributions
6.6 Legendre and Zak
6.7 Positive Definite Functions and Kernels
6.8 Positive Operators
6.9 Reproducing Kernel Hilbert Spaces
6.10 Bochner's Theorem
6.11 The Fock Space
6.12 Hermitian Forms and Reproducing Kernel Krein Spaces
6.13 Solutions of the Exercises
Part III Probability and Applications
7 Probability Theory
7.1 Review on Finite Probability Spaces
7.2 Random Variables in Finite Probability Spaces
7.3 Conditional Probabilities
7.4 Probability Densities
7.5 Sigma-Algebras and General Probability Spaces
7.6 Markov Chains
7.7 Second-Order Stationary Processes: Discrete Case
7.8 Loève's Theorem
7.9 Solutions of the Exercises
8 Entropy: Discrete Case
8.1 Proof of the Source Partition Theorem
8.2 Properties of the Entropy
8.3 Other Measures of Entropy
8.4 Capacity
8.5 Metrics on the Set of Random Variables and Entropy
8.6 Boltzmann-Gibbs Distribution
8.7 Solutions of the Exercises
9 Thermodynamics
9.1 Thermodynamics
9.2 Fluids and the State Equation
9.3 The Ideal Gas
9.4 Nonideal Gas
9.5 Solutions of the Exercises
References
Index
Name Index

✦ Subjects

Information Theory; Machine Learning; Statistical Physics; Wavelets; Signal analysis