Continuous Time Markov Processes

✍ Scribed by Thomas M. Liggett

• Publisher: American Mathematical Society
• Year: 2010
• Tongue: English
• Leaves: 289
• Series: Graduate Studies in Mathematics 113
• Category: Library

✦ Synopsis

Markov processes are among the most important stochastic processes for both theory and applications. This book develops the general theory of these processes and applies this theory to various special examples. The initial chapter is devoted to the most important classical example--one-dimensional Brownian motion. This, together with a chapter on continuous time Markov chains, provides the motivation for the general setup based on semigroups and generators. Chapters on stochastic calculus and probabilistic potential theory give an introduction to some of the key areas of application of Brownian motion and its relatives. A chapter on interacting particle systems treats a more recently developed class of Markov processes that have as their origin problems in physics and biology.

This is a textbook for a graduate course that can follow one that covers basic probabilistic limit theorems and discrete time processes.

Readership: Graduate students and research mathematicians interested in probability.

✦ Table of Contents

Preface

Chapter 1 One-Dimensional Brownian Motion
1.1. Some motivation
1.2. The multivariate Gaussian distribution
1.3. Processes with stationary independent increments
1.4. Definition of Brownian motion
1.5. The construction
1.6. Path properties
1.6.1. Nonsmoothness.
1.6.2. Quadratic variation.
1.6.3. Other properties
1.7. The Markov property
1.8. The strong Markov property and applications
1.8.1. Stopping times
1.8.2. The strong Markov property
1.8.3. Applications
1.9. Continuous time martingales and applications
1.9.1. Martingales
1.9.2. Mart ingales derived from Brownian motion
1.9.3. Applications.
1.10. The Skorokhod embedding
1.11. Donsker's theorem and applications

Chapter 2 Continuous Time Markov Chains
2.1. The basic setup
2.2. Some examples
2.3. From Markov chain to infinitesimal description
2.4. Blackwell's example
2.5. From infinitesimal description to Markov chain
2.5.1. The backward equation
2.5.2. The probabilistic construction
2.5.3. The forward equation
2.6. Stationary measures, recurrence, and transience
2.6.1. Stationary and reversible measures
2.6.2. Recurrence and transience
2.6.3. Convergence
2.7. More examples
2.7.1. Birth and death chain.
2.7.2. Branching chain
2.7.3. Independent particle systems.
2.7.4. Zero range particle systems.

Chapter 3 Feller Processes
3.1. The basic setup
3.1.1. The process.
3.1.2. The semigroup
3.1.3. Levy processes
3.1.4. The generator
3.2. From Feller process to infinitesimal description
3.3. From infinitesimal description to Feller process
3.4. A few tools
3.4.1. Construction of generators
3.4.2. Construction of martingales.
3.4.3. Stationary distributions
3.4.4. Duality
3.4.5. Superpositions of processes
3.4.6. The Feynman-Kac formula.
3.5. Applications to Brownian motion and its relatives
3.5.1. Brownian motion with speed change.
3.5.2. Brownian motion with special boundary behavior
3.5.3. Some examples in higher dimensions.
3.5.4. Construction of diffusions.

Chapter 4 Interacting Particle Systems
4.1. Some motivation
4.2. Spin systems
4.2.1. Construction of spin systems.
4.2.2. Ergodicity of spin systems
4.2.3. Coupling of spin systems; attractiveness
4.2.4. Applications of coupling and monotonicity.
4.2.5. Correlation inequalities.
4.3. The voter model
4.3.1. Voter model duality
4.3.2. The recurrent case.
4.3.3. The transient case.
4.4. The contact process
4.4.1. The graphical representation, additivity
4.4.2. Survival and extinction; critical values.
4.4.3. The contact process on a homogeneous tree
4.4.4. The contact process on the integers
4.4.5. The contact process on Z^d.
4.5. Exclusion processes
4.5.1. Existence
4.5.2. Product form stationary distributions.
4.5.3. Symmetric exclusion processes stationary distributions
4.5.4. Symmetric exclusion processes - distributional properties.
4.5.5. Translation invariant exclusion processes

Chapter 5 Stochastic Integration
5.1. Some motivation
5.2. The Ito integral
5.2.1. The variance process
5.2.2. Construction of the integral.
5.3. Ito's formula and applications
5.3.1. Ito's formula for single martingales
5.3.2. Ito's formula for several semimartingales
5.3.3. Applications of Ito's formula
5.4. Brownian local time
5.5. Connections to Feller processes on R^1
5.5.1. Some examples
5.5.2. Solving the stochastic differential equation
5.5.3. The Feller connection.

Chapter 6 Multi-Dimensional Brownian Motion and the Dirichiet Problem
6.1. Harmonic functions and the Dirichlet problem
6.2. Brownian motion on R"
6.3. Back to the Dirichlet problem
6.4. The Poisson equation

Appendix
A.1. Commonly used notation
A.2. Some measure theory
A.3. Some analysis
A.4. The Poisson distribution
A.5. Random series and laws of large numbers
A.6. The central limit theorem and related topics
A.6.1. Weak convergence.
A.6.2. Characteristic functions.
A.6.3. The central limit problem
A.6.4. The moment problem.
A.7. Discrete time martingales
A.8. Discrete time Markov chains
A.9. The renewal theorem
A.10. Harmonic functions for discrete time Markov chains
A.11. Subadditive functions

Bibliography

Index