cover......Page 1
Half title......Page 2
Title......Page 4
Copyright......Page 5
CONTENTS......Page 6
PREFACE......Page 12
1.1 What Is Probability?......Page 16
1.2 Experiments, Outcomes, and Events......Page 18
1.3 Venn Diagrams......Page 19
1.4 Random Variables......Page 20
1.5 Basic Probability Rules......Page 21
1.6 Probability Formalized......Page 24
1.7 Simple Theorems......Page 26
1.8 Compound Experiments......Page 30
1.9 Independence......Page 31
1.10 Example: Can S CommunicateWith D?......Page 32
1.10.1 List All Outcomes......Page 33
1.10.2 Probability of a Union......Page 34
1.10.3 Probability of the Complement......Page 35
1.11.1 A Big Table......Page 36
1.11.2 Break Into Pieces......Page 37
1.12 Computational Procedures......Page 38
Summary......Page 39
Problems......Page 40
2.1 Definitions of Conditional Probability......Page 44
2.2 Law of Total Probability and BayesTheorem......Page 47
2.3 Example: Urn Models......Page 49
2.4 Example: A Binary Channel......Page 51
2.5 Example: Drug Testing......Page 53
2.6 Example: A Diamond Network......Page 55
Summary......Page 56
Problems......Page 57
3.1 Basics of Counting......Page 62
3.2 Notes on Computation......Page 67
3.3 Combinations and the Binomial Coefficients......Page 68
3.4 The Binomial Theorem......Page 69
3.5 Multinomial Coefficient andTheorem......Page 70
3.6 The Birthday Paradox and Message Authentication......Page 72
3.7 Hypergeometric Probabilities and Card Games......Page 76
Summary......Page 81
Problems......Page 82
4.1 Probability Mass Functions......Page 90
4.2 Cumulative Distribution Functions......Page 92
4.3 Expected Values......Page 93
4.4 Moment Generating Functions......Page 98
4.5 Several Important Discrete PMFs......Page 100
4.5.1 Uniform PMF......Page 101
4.5.2 Geometric PMF......Page 102
4.5.3 The Poisson Distribution......Page 105
4.6 Gambling and Financial Decision Making......Page 107
Summary......Page 110
Problems......Page 111
5.1 Multiple Random Variables and PMFs......Page 116
5.2 Independence......Page 119
5.3.1 Expected Values for Two Random Variables......Page 120
5.3.2 Moments for Two Random Variables......Page 121
5.4 Example: Two Discrete Random Variables......Page 123
5.4.2 Independence......Page 124
5.4.4 TransformationsWith One Output......Page 125
5.4.5 TransformationsWith Several Outputs......Page 127
5.5 Sums of Independent Random Variables......Page 128
5.6 Sample Probabilities, Mean, and Variance......Page 132
5.7 Histograms......Page 134
5.8 Entropy and Data Compression......Page 135
5.8.1 Entropy and InformationTheory......Page 136
5.8.2 Variable Length Coding......Page 138
5.8.3 Encoding Binary Sequences......Page 142
5.8.4 Maximum Entropy......Page 143
Summary......Page 146
Problems......Page 147
6.1 Basics of the Binomial Distribution......Page 152
6.2 Computing Binomial Probabilities......Page 156
6.3 Moments of the Binomial Distribution......Page 157
6.4 Sums of Independent Binomial Random Variables......Page 159
6.5.1 Connections Between Binomial and Hypergeometric Probabilities......Page 161
6.5.2 Multinomial Probabilities......Page 162
6.5.3 The Negative Binomial Distribution......Page 163
6.5.4 The Poisson Distribution......Page 164
6.6 Binomial and Multinomial Estimation......Page 166
6.7 Alohanet......Page 167
6.8 Error Control Codes......Page 169
6.8.1 Repetition-by-Three Code......Page 170
6.8.2 General Linear Block Codes......Page 172
Summary......Page 175
Problems......Page 177
7.1 Basic Properties......Page 182
7.2 Example Calculations for One Random Variable......Page 186
7.3.1 The Uniform Distribution......Page 189
7.3.2 The Exponential Distribution......Page 191
7.4 Conditional Probabilities......Page 194
7.5 Discrete PMFs and Delta Functions......Page 197
7.6 Quantization......Page 199
Summary......Page 202
Problems......Page 204
8.1 Joint Densities and Distribution Functions......Page 207
8.3 Independence......Page 209
8.4 Conditional Probabilities for Multiple Random Variables......Page 210
8.5 Extended Example: Two Continuous Random Variables......Page 213
8.6 Sums of Independent Random Variables......Page 217
8.7 Random Sums......Page 220
8.8 General Transformations and the Jacobian......Page 222
8.10 Comparison of Discrete and Continuous Distributions......Page 229
Summary......Page 230
Problems......Page 231
9.1 The Gaussian Distribution and Density......Page 236
9.2 Quantile Function......Page 242
9.3 Moments of the Gaussian Distribution......Page 243
9.4 The Central LimitTheorem......Page 245
9.5 Related Distributions......Page 250
9.5.2 The Rayleigh Distribution......Page 251
9.5.3 The Chi-Squared and F Distributions......Page 253
9.6.1 Independent Gaussian Random Variables......Page 255
9.6.2 Transformation to Polar Coordinates......Page 256
9.6.3 Two Correlated Gaussian Random Variables......Page 258
9.7.1 Background......Page 261
9.7.2 Discrete Time Model......Page 262
9.7.3 Monte Carlo Exercise......Page 268
9.7.4 QAM Recap......Page 273
Summary......Page 274
Problems......Page 275
10.1 A Simple Election Poll......Page 280
10.2 Estimating the Mean and Variance......Page 284
10.3 Recursive Calculation of the Sample Mean......Page 286
10.4 ExponentialWeighting......Page 288
10.5 Order Statistics and Robust Estimates......Page 289
10.6 Estimating the Distribution Function......Page 291
10.7 PMF and Density Estimates......Page 293
10.8 Confidence Intervals......Page 295
10.9 Significance Tests and p-Values......Page 297
10.10 Introduction to EstimationTheory......Page 300
10.11 Minimum Mean Squared Error Estimation......Page 304
10.12 Bayesian Estimation......Page 306
Problems......Page 310
11.1 Gaussian Random Vectors......Page 313
11.2 Linear Operations on Gaussian Random Vectors......Page 318
11.3 Linear Regression......Page 319
11.3.1 Linear Regression in Detail......Page 320
11.3.2 Statistics of the Linear Regression Estimates......Page 324
11.3.3 Computational Issues......Page 326
11.3.4 Linear Regression Examples......Page 328
11.3.5 Extensions of Linear Regression......Page 332
Summary......Page 334
Problems......Page 335
12.1 Hypothesis Testing: Basic Principles......Page 339
12.2 Example: Radar Detection......Page 341
12.3 Hypothesis Tests and Likelihood Ratios......Page 346
12.4 MAP Tests......Page 350
Summary......Page 351
Problems......Page 352
13.1 Introduction to Random Signals......Page 355
13.2 A Simple Random Process......Page 356
13.3 Fourier Transforms......Page 357
13.4 WSS Random Processes......Page 361
13.5 WSS Signals and Linear Filters......Page 365
13.6.1 Probabilistic Properties of Noise......Page 367
13.6.2 Spectral Properties of Noise......Page 368
13.7 Example: Amplitude Modulation......Page 369
13.9 The Sampling Theorem forWSS Random Processes......Page 372
13.9.1 Discussion......Page 373
13.9.2 Example: Figure 13.4......Page 374
13.9.3 Proof of the Random Sampling Theorem......Page 376
Summary......Page 377
Problems......Page 379
14.1 The Lightbulb Process......Page 381
14.2 The Poisson Process......Page 383
14.3 Markov Chains......Page 387
14.4.1 The Optimal Filter and Example......Page 396
14.4.2 QR Method Applied to the Kalman Filter......Page 399
Summary......Page 401
Problems......Page 403
A.1 Matlab......Page 406
A.2 Python......Page 408
A.3 R......Page 410
B ACRONYMS......Page 414
C.1 Tables of Gaussian Probabilities......Page 416
D BIBLIOGRAPHY......Page 418
INDEX......Page 420