Markov Chains and Invariant Probabilities (Progress in Mathematics, 211)

✍ Scribed by Onésimo Hernández-Lerma, Jean B. Lasserre

• Publisher: Birkhäuser
• Year: 2003
• Tongue: English
• Leaves: 223
• Edition: 2003
• Category: Library

✦ Synopsis

This book is about discrete-time, time-homogeneous, Markov chains (Mes) and their ergodic behavior. To this end, most of the material is in fact about stable Mes, by which we mean Mes that admit an invariant probability measure. To state this more precisely and give an overview of the questions we shall be dealing with, we will first introduce some notation and terminology. Let (X,B) be a measurable space, and consider a X-valued Markov chain ~. = {~k' k = 0, 1, ... } with transition probability function (t.pJ.) P(x, B), i.e., P(x, B) := Prob (~k+1 E B I ~k = x) for each x E X, B E B, and k = 0,1, .... The Me ~. is said to be stable if there exists a probability measure (p.m.) /.l on B such that () VB EB. /.l(B) = Ix /.l(dx) P(x, B) If () holds then /.l is called an invariant p.m. for the Me ~. (or the t.p.f. P).

✦ Table of Contents

MARKOV CHAINS AND INVARIANT PROBABILITIES
Title Page
Copyright Page
Contents
Acknowledgements
Preface
Abbreviations
List of Symbols
Chapter 1. Preliminaries
1.1 Introduction
1.2 Measures and Functions
1.2.1 Measures and Signed Measures
1.2.2 Function Spaces
1.3 Weak Topologies
1.4 Convergence of Measures
1.4.1 Setwise Convergence
1.4.2 Convergence in the Total Variation Norm
1.4.3 Convergence of Measures in a Metric Space
1.5 Complements
1.5.1 A Uniform Principle of Weak Convergence
1.5.2 Fatou's Lemma, Monotone and Lebesgue Dominated Convergence for Measures
1.6 Notes
Part I: Markov Chains and Ergodicity
Chapter 2. Markov Chains and Ergodic Theorems
2.1 Introduction
2.2 Basic Notation and Definitions
2.2.1 Examples
2.2.2 Invariant Probability Measures
2.3 Ergodic Theorems
2.3.1 The Chacon–Ornstein Theorem
2.3.2 Ergodic Theorems for Markov Chains
2.3.3 A "Dual" Ergodic Theorem
2.4 The Ergodicity Property
2.5 Pathwise Results
2.6 Notes
Chapter 3. Countable Markov Chains
3.1 Introduction
3.2 Classification of States and Class Properties
3.2.1 Communication
3.2.2 Essential and Inessential States
3.2.3 Absorbing Sets and Irreducibility
3.2.4 Recurrence and Transience
3.3 Limit Theorems
3.4 Notes
Chapter 4. Harris Markov Chains
4.1 Introduction
4.2 Basic Definitions and Properties
4.3 Characterization of Harris Recurrence via Occupation Measures
4.3.1 Positive Harris Recurrent Markov Chains
4.3.2 Aperiodic Positive Harris Recurrent Markov Chains
4.3.3 Geometric Ergodicity
4.3.4 Discussion
4.4 Sufficient Conditions for P.H.R.
4.5 Harris and Doeblin Decompositions
4.6 Notes
Chapter 5. Markov Chains in Metric Spaces
5.1 Introduction
5.2 The Limit in Ergodic Theorems
5.2.1 The Limiting Transition Probability Function
5.2.2 Extension to Some Nonmetric Spaces
5.3 Yosida's Ergodic Decomposition
5.3.1 Additive-Noise Systems
5.4 Pathwise Results
5.5 Proofs
5.5.1 Proof of Lemma 5.2.3
5.5.2 Proof of Lemma 5.2.4
5.5.3 Proof of Theorem 5.2.2
5.5.4 Proof of Lemma 5.3.2
5.5.5 Proof of Theorem 5.4.1
5.6 Notes
Chapter 6. Classification of Markov Chains via Occupation Measures
6.1 Introduction
6.2 A Classification
6.2.1 Examples
6.3 On the Birkhoff Individual Ergodic Theorem
6.3.1 Finitely-Additive Invariant Measures
6.3.2 Discussion
6.4 Notes
Part II: Further Ergodicity Properties
Chapter 7. Feller Markov Chains
7.1 Introduction
7.2 Weak- and Strong-Feller Markov Chains
7.2.1 The Feller Property
7.2.2 Sufficient Condition for Existence of an Invariant Probability Measure
7.2.3 Strong-Feller Chains
7.3 Quasi Feller Chains
7.4 Notes
Chapter 8. The Poisson Equation
8.1 Introduction
8.2 The Poisson Equation
8.3 Canonical Pairs
8.4 The Cesàro-Averages Approach
8.5 The Abelian Approach
8.6 Notes
Chapter 9. Strong and Uniform Ergodicity
9.1 Introduction
9.2 Strong and Uniform Ergodicity
9.2.1 Notation
9.2.2 Ergodicity
9.2.3 Strong Stability
9.2.4 The Link with the Poisson Equation
9.3 Weak and Weak Uniform Ergodicity
9.3.1 Weak Ergodicity
9.3.2 Weak Uniform Ergodicity
9.4 Notes
Part III: Existence and Approximation of Invariant Probability Measures
Chapter 10. Existence and Uniqueness of Invariant Probability Measures
10.1 Introduction and Statement of the Problems
10.2 Notation and Definitions
10.3 Existence Results
10.3.1 Problem P 1
10.3.2 Problem P 2
10.3.3 Problem P* 3
10.4 Markov Chains in Locally Compact Separable Metric Spaces
10.5 Other Existence Results in Locally Compact Separable Metric Spaces
10.5.1 Necessary and Sufficient Conditions
10.5.2 Lyapunov Sufficient Conditions
10.6 Technical Preliminaries
10.6.1 On Finitely Additive Measures
10.6.2 A Generalized Farkas Lemma
10.7 Proofs
10.7.1 Proof of Theorem 10.3.1
10.7.2 Proof of Theorem 10.4.3
10.7.3 Proof of Theorem 10.5.1
10.8 Notes
Chapter 11. Existence and Uniqueness of Fixed Points for Markov Operators
11.1 Introduction and Statement of the Problems
11.2 Notation and Definitions
11.3 Existence Results

11.3.2 Problem P 1
11.3.3 Problem P 2
11.3.4 Problem P 3
11.3.5 Example
11.4 Proofs
11.4.1 Proof of Theorem 11.3.1
11.4.2 Proof of Theorem 11.3.2
11.4.3 Proof of Theorem 11.3.3
11.4.4 Proof of Theorem 11.3.5
11.5 Notes
Chapter 12. Approximation Procedures for Invariant Probability Measures
12.1 Introduction
12.2 Statement of the Problem and Preliminaries
12.2.1 Constraint-Aggregation
12.2.2 Inner Approximation in M(X)
12.3 An Approximation Scheme
12.3.1 Aggregation
12.3.2 Aggregation-Relaxation-Inner Approximation
12.4 A Moment Approach for a Special Class of Markov Chains
12.4.1 Upper and Lower Bounds
12.4.2 An Approximation Scheme
12.4.3 Sharp Upper and Lower Bounds
12.5 Notes
Bibliography
Index
Back Cover