A First Course in Probability and Markov Chains

✍ Scribed by Modica, Giuseppe

• Publisher: Wiley
• Year: 2013
• Tongue: English
• Leaves: 348
• Category: Library

✦ Synopsis

Provides an introduction to basic structures of probability with a view towards applications in information technology

"A First Course in Probability and Markov Chains" presents an introduction to the basic elements in probability and focuses on two main areas. The first part explores notions and structures in probability, including combinatorics, probability measures, probability distributions, conditional probability, inclusion-exclusion formulas, random variables, dispersion indexes, independent random variables as well as weak and strong laws of large numbers and central limit theorem. In the second part of the book, focus is given to Discrete Time Discrete Markov Chains which is addressed together with an introduction to Poisson processes and Continuous Time Discrete Markov Chains. This book also looks at making use of measure theory notations that unify all the presentation, in particular avoiding the separate treatment of continuous and discrete distributions.

"A First Course in Probability and Markov Chains: "Presents the basic elements of probability.Explores elementary probability with combinatorics, uniform probability, the inclusion-exclusion principle, independence and convergence of random variables. Features applications of Law of Large Numbers. Introduces Bernoulli and Poisson processes as well as discrete and continuous time Markov Chains with discrete states. Includes illustrations and examples throughout, along with solutions to problems featured in this book.

The authors present a unified and comprehensive overview of probability and Markov Chains aimed at educating engineers working with probability and statistics as well as advanced undergraduate students in sciences and engineering with a basic background in mathematical analysis and linear algebra.

✦ Table of Contents

A First Course in Probability and Markov Chains......Page 3
Contents......Page 7
Preface......Page 13
1.1.1 Pascal triangle......Page 15
1.1.2 Some properties of binomial coefficients......Page 16
1.1.3 Generalized binomial coefficients and binomial series......Page 17
1.1.4 Inversion formulas......Page 18
1.1.5 Exercises......Page 20
1.2.2 Permutations......Page 22
1.2.3 Multisets......Page 24
1.2.4 Lists and functions......Page 25
1.2.6 Monotone increasing functions......Page 26
1.2.7 Monotone nondecreasing functions......Page 27
1.2.8 Surjective functions......Page 28
1.3.1 Ordered drawings......Page 30
1.3.3 Multiplicative property of drawings......Page 31
1.3.4 Exercises......Page 32
1.4.1 Collocations of pairwise different objects......Page 33
1.4.2 Collocations of identical objects......Page 36
1.4.3 Multiplicative property......Page 37
1.4.5 Exercises......Page 38
Chapter 2 Probability measures......Page 41
2.1 Elementary probability......Page 42
2.1.1 Exercises......Page 43
2.2 Basic facts......Page 47
2.2.1 Events......Page 48
2.2.2 Probability measures......Page 50
2.2.3 Continuity of measures......Page 51
2.2.4 Integral with respect to a measure......Page 53
2.2.5 Probabilities on finite and denumerable sets......Page 54
2.2.6 Probabilities on denumerable sets......Page 56
2.2.7 Probabilities on uncountable sets......Page 58
2.2.8 Exercises......Page 60
2.3.1 Definition......Page 65
2.3.2 Bayes formula......Page 66
2.3.3 Exercises......Page 68
2.4 Inclusion-exclusion principle......Page 74
2.4.1 Exercises......Page 77
3.1 Random variables......Page 82
3.1.1 Definitions......Page 83
3.1.2 Expected value......Page 89
3.1.3 Functions of random variables......Page 91
3.1.4 Cavalieri formula......Page 94
3.1.6 Markov and Chebyshev inequalities......Page 96
3.1.7 Variational characterization of the median and of the expected value......Page 97
3.1.8 Exercises......Page 98
3.2.2 Binomial distribution......Page 105
3.2.3 Hypergeometric distribution......Page 107
3.2.4 Negative binomial distribution......Page 108
3.2.5 Poisson distribution......Page 109
3.2.6 Geometric distribution......Page 112
3.2.7 Exercises......Page 115
3.3.1 Uniform distribution......Page 116
3.3.2 Normal distribution......Page 118
3.3.3 Exponential distribution......Page 120
3.3.4 Gamma distributions......Page 122
3.3.5 Failure rate......Page 124
3.3.6 Exercises......Page 125
4.1 Joint distribution......Page 127
4.1.1 Joint and marginal distributions......Page 128
4.1.2 Exercises......Page 131
4.2.1 Random variables with finite expected value and variance......Page 134
4.2.3 Exercises......Page 137
4.3.1 Independent events......Page 138
4.3.2 Independent random variables......Page 141
4.3.3 Independence of many random variables......Page 142
4.3.4 Sum of independent random variables......Page 144
4.3.5 Exercises......Page 145
4.4.1 Weak law of large numbers......Page 154
4.4.2 Borel-Cantelli lemma......Page 156
4.4.3 Convergences of random variables......Page 157
4.4.4 Strong law of large numbers......Page 160
4.4.5 A few applications of the law of large numbers......Page 166
4.4.6 Central limit theorem......Page 173
4.4.7 Exercises......Page 177
5.1 Stochastic matrices......Page 182
5.1.1 Definitions......Page 183
5.1.2 Oriented graphs......Page 184
5.1.3 Exercises......Page 186
5.2.1 Stochastic processes......Page 187
5.2.4 Markov chains......Page 188
5.2.5 Canonical Markov chains......Page 192
5.2.6 Exercises......Page 195
5.3.1 Steps for a first visit......Page 201
5.3.2 Probability of (at least) r visits......Page 203
5.3.3 Recurrent and transient states......Page 205
5.3.4 Mean first passage time......Page 207
5.3.5 Hitting time and hitting probabilities......Page 209
5.3.6 Exercises......Page 212
5.4.1 Canonical representation......Page 215
5.4.2 States classification......Page 217
5.4.3 Exercises......Page 219
5.5.1 Iterated maps......Page 220
5.5.2 Existence of fixed points......Page 223
5.5.3 Regular stochastic matrices......Page 224
5.5.4 Characteristic parameters......Page 232
5.5.5 Exercises......Page 234
5.6.1 Number of steps between consecutive visits......Page 236
5.6.2 Ergodic theorem......Page 238
5.6.3 Powers of irreducible stochastic matrices......Page 240
5.6.4 Markov chain Monte Carlo......Page 242
5.7.1 Periodicity......Page 247
5.7.2 Renewal theorem......Page 248
5.7.3 Exercises......Page 253
6.1 Poisson process......Page 255
6.2.1 Definitions......Page 260
6.2.2 Continuous semigroups of stochastic matrices......Page 262
6.2.3 Examples of right-continuous Markov chains......Page 270
6.2.4 Holding times......Page 273
A.1 Basic properties......Page 275
A.2 Product of series......Page 277
A.3 Banach space valued power series......Page 278
A.3.2 Exercises......Page 281
B.1.1 Basic properties......Page 284
B.1.2 Construction of measures......Page 286
B.2 Measurable functions and integration......Page 293
B.2.1 Measurable functions......Page 294
B.2.2 The integral......Page 297
B.2.3 Properties of the integral......Page 298
B.2.4 Cavalieri formula......Page 300
B.2.6 Null sets and the integral......Page 301
B.2.7 Push forward of a measure......Page 303
B.2.8 Exercises......Page 304
B.3.1 Product measures......Page 308
B.3.2 Reduction formulas......Page 310
B.3.3 Exercises......Page 311
B.4.1 Almost everywhere convergence......Page 312
B.4.2 Strong convergence......Page 314
B.4.3 Fatou lemma......Page 315
B.4.4 Dominated convergence theorem......Page 316
B.4.6 Differentiation of the integral......Page 319
B.4.7 Weak convergence of measures......Page 322
B.4.8 Exercises......Page 326
C.1.1 Uniqueness......Page 327
C.1.2 Existence......Page 329
C.2.1 Similarity methods......Page 331
C.2.2 Putzer method......Page 333
C.3 Continuous semigroups......Page 335
References......Page 338
Index......Page 341