Understanding Computational Bayesian Statistics (Wiley Series in Computational Statistics)

✍ Scribed by William M. Bolstad

• Publisher: Wiley
• Year: 2009
• Tongue: English
• Leaves: 334
• Series: Wiley Series in Computational Statistics
• Edition: 1
• Category: Library

✦ Synopsis

A hands-on introduction to computational statistics from a Bayesian point of viewProviding a solid grounding in statistics while uniquely covering the topics from a Bayesian perspective, Understanding Computational Bayesian Statistics successfully guides readers through this new, cutting-edge approach. With its hands-on treatment of the topic, the book shows how samples can be drawn from the posterior distribution when the formula giving its shape is all that is known, and how Bayesian inferences can be based on these samples from the posterior. These ideas are illustrated on common statistical models, including the multiple linear regression model, the hierarchical mean model, the logistic regression model, and the proportional hazards model.The book begins with an outline of the similarities and differences between Bayesian and the likelihood approaches to statistics. Subsequent chapters present key techniques for using computer software to draw Monte Carlo samples from the incompletely known posterior distribution and performing the Bayesian inference calculated from these samples. Topics of coverage include:Direct ways to draw a random sample from the posterior by reshaping a random sample drawn from an easily sampled starting distributionThe distributions from the one-dimensional exponential familyMarkov chains and their long-run behaviorThe Metropolis-Hastings algorithmGibbs sampling algorithm and methods for speeding up convergenceMarkov chain Monte Carlo samplingUsing numerous graphs and diagrams, the author emphasizes a step-by-step approach to computational Bayesian statistics. At each step, important aspects of application are detailed, such as how to choose a prior for logistic regression model, the Poisson regression model, and the proportional hazards model. A related Web site houses R functions and Minitab macros for Bayesian analysis and Monte Carlo simulations, and detailed appendices in the book guide readers through the use of these software packages.Understanding Computational Bayesian Statistics is an excellent book for courses on computational statistics at the upper-level undergraduate and graduate levels. It is also a valuable reference for researchers and practitioners who use computer programs to conduct statistical analyses of data and solve problems in their everyday work.

✦ Table of Contents

Understanding Computational Bayesian Statistics......Page 5
Contents......Page 9
Preface......Page 13
1.1 The Frequentist Approach to Statistics......Page 17
1.2 The Bayesian Approach to Statistics......Page 19
1.3 Comparing Likelihood and Bayesian Approaches to Statistics......Page 22
1.4 Computational Bayesian Statistics......Page 35
1.5 Purpose and Organization of This Book......Page 36
2 Monte Carlo Sampling from the Posterior......Page 41
2.1 Acceptance-Rejection-Sampling......Page 43
2.2 Sampling-Importance-Resampling......Page 49
2.3 Adaptive-Rejection-Sampling from a Log-Concave Distribution......Page 51
2.4 Why Direct Methods Are Inefficient for High-Dimension Parameter Space......Page 58
3.1 Bayesian Inference from the Numerical Posterior......Page 63
3.2 Bayesian Inference from Posterior Random Sample......Page 70
4.1 One-Dimensional Exponential Family of Densities......Page 77
4.2 Distributions for Count Data......Page 78
4.3 Distributions for Waiting Times......Page 85
4.4 Normally Distributed Observations with Known Variance......Page 91
4.5 Normally Distributed Observations with Known Mean......Page 94
4.6 Normally Distributed Observations with Unknown Mean and Variance......Page 96
4.7 Multivariate Normal Observations with Known Covariance Matrix......Page 101
4.8 Observations from Normal Linear Regression Model......Page 103
Appendix: Proof of Poisson Process Theorem......Page 113
5 Markov Chains......Page 117
5.1 Stochastic Processes......Page 118
5.2 Markov Chains......Page 119
5.3 Time-Invariant Markov Chains with Finite State Space......Page 120
5.4 Classification of States of a Markov Chain......Page 125
5.5 Sampling from a Markov Chain......Page 130
5.6 Time-Reversible Markov Chains and Detailed Balance......Page 133
5.7 Markov Chains with Continuous State Space......Page 136
6 Markov Chain Monte Carlo Sampling from Posterior......Page 143
6.1 Metropolis-Hastings Algorithm for a Single Parameter......Page 146
6.2 Metropolis-Hastings Algorithm for Multiple Parameters......Page 153
6.3 Blockwise Metropolis-Hastings Algorithm......Page 160
6.4 Gibbs Sampling......Page 165
6.5 Summary......Page 166
7 Statistical Inference from a Markov Chain Monte Carlo Sample......Page 175
7.1 Mixing Properties of the Chain......Page 176
7.2 Finding a Heavy-Tailed Matched Curvature Candidate Density......Page 178
7.3 Obtaining An Approximate Random Sample For Inference......Page 184
Appendix: Procedure for Finding the Matched Curvature Candidate Density for a Multivariate Parameter......Page 192
8 Logistic Regression......Page 195
8.1 Logistic Regression Model......Page 196
8.2 Computational Bayesian Approach to the Logistic Regression Model......Page 200
8.3 Modelling with the Multiple Logistic Regression Model......Page 208
9 Poisson Regression and Proportional Hazards Model......Page 219
9.1 Poisson Regression Model......Page 220
9.2 Computational Approach to Poisson Regression Model......Page 223
9.3 The Proportional Hazards Model......Page 230
9.4 Computational Bayesian Approach to Proportional Hazards Model......Page 234
10 Gibbs Sampling and Hierarchical Models......Page 251
10.1 Gibbs Sampling Procedure......Page 252
10.2 The Gibbs Sampler for the Normal Distribution......Page 253
10.3 Hierarchical Models and Gibbs Sampling......Page 258
10.4 Modelling Related Populations with Hierarchical Models......Page 260
Appendix: Proof That Improper Jeffrey's Prior Distribution for the Hypervariance Can Lead to an Improper Posterior......Page 277
11 Going Forward with Markov Chain Monte Carlo......Page 281
A Using the Included Minitab Macros......Page 287
B Using the Included R Functions......Page 305
References......Page 323
Topic Index......Page 329