Regression: Models, Methods and Applications

✍ Scribed by Fahrmeir, L., Kneib, Th., Lang, S., Marx, B.

• Publisher: Springer
• Year: 2013
• Tongue: English
• Leaves: 713
• Category: Library

✦ Synopsis

Applied and unified introduction into parametric, non- and semiparametric regression that closes the gap between theory and application
Written in textbook style suitable for students, the material is close to current research on advanced regression analysis
Availability of (user-friendly) software is a major criterion for the methods selected and presented
Many examples and applications from diverse fields illustrate models and methods
Most of the data sets are available via http://www.regressionbook.org/

The aim of this book is an applied and unified introduction into parametric, non- and semiparametric regression that closes the gap between theory and application. The most important models and methods in regression are presented on a solid formal basis, and their appropriate application is shown through many real data examples and case studies. Availability of (user-friendly) software has been a major criterion for the methods selected and presented. Thus, the book primarily targets an audience that includes students, teachers and practitioners in social, economic, and life sciences, as well as students and teachers in statistics programs, and mathematicians and computer scientists with interests in statistical modeling and data analysis. It is written on an intermediate mathematical level and assumes only knowledge of basic probability, calculus, and statistics. The most important definitions and statements are concisely summarized in boxes. Two appendices describe required matrix algebra, as well as elements of probability calculus and statistical inference.

Content Level » Graduate

Keywords » generalized linear models - linear regression - mixed models - semiparametric regression - spatial regression

Related subjects » Business, Economics & Finance - Econometrics / Statistics - Public Health - Statistical Theory and Methods - Systems Biology and Bioinformatics

✦ Table of Contents

Cover......Page 1
S Title......Page 2
Regression: Models, Methods and Applications......Page 4
LCCN: 2013934096......Page 5
Preface......Page 6
Table of Contents......Page 10
1 Introduction......Page 16
1.1 Examples of Applications......Page 19
1.2.1 Univariate Distributions of the Variables......Page 26
1.2.2 Graphical Association Analysis......Page 28
1.3 Notational Remarks......Page 34
2.1 Introduction......Page 36
2.2.1 Simple Linear Regression Model......Page 37
2.2.2 Multiple Linear Regression......Page 41
2.3 Regression with Binary Response Variables: The Logit Model......Page 48
2.4 Mixed Models......Page 53
2.5 Simple Nonparametric Regression......Page 59
2.6 Additive Models......Page 64
2.7 Generalized Additive Models......Page 67
2.8 Geoadditive Regression......Page 70
2.9 Beyond Mean Regression......Page 76
2.9.1 Regression Models for Location, Scale, and Shape......Page 77
2.9.2 Quantile Regression......Page 81
2.10.3 Poisson Regression (Chap.5)......Page 83
2.10.5 Linear Mixed Models (LMMs, Chap.7)......Page 84
2.10.7 Generalized Additive (Mixed) Models (GA(M)Ms, Chap.9)......Page 85
2.10.9 Quantile Regression (Chap.10)......Page 86
3.1 Model Definition......Page 88
3.1.1 Model Parameters, Estimation, and Residuals......Page 92
3.1.2 Discussion of Model Assumptions......Page 93
3.1.3 Modeling the Effects of Covariates......Page 101
3.2.1 Estimation of Regression Coefficients......Page 119
3.2.2 Estimation of the Error Variance......Page 123
3.2.3 Properties of the Estimators......Page 125
3.3 Hypothesis Testing and Confidence Intervals......Page 140
3.3.1 Exact F-Test......Page 143
3.3.2 Confidence Regions and Prediction Intervals......Page 151
3.4 Model Choice and Variable Selection......Page 154
3.4.1 Bias, Variance and Prediction Quality......Page 157
3.4.2 Model Choice Criteria......Page 161
3.4.3 Practical Use of Model Choice Criteria......Page 165
3.4.4 Model Diagnosis......Page 170
3.5.2 Proofs......Page 183
4.1.1 Model Definition......Page 192
4.1.2 Weighted Least Squares......Page 193
4.1.3 Heteroscedastic Errors......Page 197
4.1.4 Autocorrelated Errors......Page 206
4.2 Regularization Techniques......Page 216
4.2.1 Statistical Regularization......Page 217
4.2.2 Ridge Regression......Page 218
4.2.3 Least Absolute Shrinkage and Selection Operator......Page 223
4.2.4 Geometric Properties of Regularized Estimates......Page 226
4.2.5 Partial Regularization......Page 231
4.3.1 Basic Principles......Page 232
4.3.2 Componentwise Boosting......Page 233
4.3.3 Generic Componentwise Boosting......Page 237
4.4 Bayesian Linear Models......Page 240
4.4.1 Standard Conjugate Analysis......Page 242
4.4.2 Regularization Priors......Page 252
4.4.3 Classical Bayesian Model Choice (and Beyond)......Page 258
4.4.4 Spike and Slab Priors......Page 268
4.5.1 Bibliographic Notes......Page 272
4.5.2 Proofs......Page 273
5 Generalized Linear Models......Page 284
5.1.1 Binary Regression Models......Page 285
5.1.2 Maximum Likelihood Estimation......Page 294
5.1.3 Testing Linear Hypotheses......Page 300
5.1.4 Criteria for Model Fit and Model Choice......Page 302
5.1.5 Estimation of the Overdispersion Parameter......Page 307
5.2.1 Models for Count Data......Page 308
5.2.2 Estimation and Testing: Likelihood Inference......Page 310
5.2.4 Estimation of the Overdispersion Parameter......Page 312
5.3 Models for Nonnegative Continuous Response Variables......Page 313
5.4.1 General Model Definition......Page 316
5.4.2 Likelihood Inference......Page 321
5.5 Quasi-likelihood Models......Page 324
5.6 Bayesian Generalized Linear Models......Page 326
5.6.1 Posterior Mode Estimation......Page 328
5.6.2 Fully Bayesian Inference via MCMC Simulation Techniques......Page 329
5.6.3 MCMC-Based Inference Using Data Augmentation......Page 331
5.7 Boosting Generalized Linear Models......Page 334
5.8.1 Bibliographic Notes......Page 335
5.8.2 Proofs......Page 336
6.1 Introduction......Page 340
6.2 Models for Unordered Categories......Page 344
6.3.1 The Cumulative Model......Page 349
6.3.2 The Sequential Model......Page 352
6.4 Estimation and Testing: Likelihood Inference......Page 358
6.5 Bibliographic Notes......Page 362
7 Mixed Models......Page 364
7.1.1 Random Intercept Models......Page 365
7.1.2 Random Coefficient or Slope Models......Page 372
7.1.3 General Model Definition and Matrix Notation......Page 376
7.1.4 Conditional and Marginal Formulation......Page 380
7.1.5 Stochastic Covariates......Page 381
7.2 General Linear Mixed Models......Page 383
7.3.1 Known Variance–Covariance Parameters......Page 386
7.3.2 Unknown Variance–Covariance Parameters......Page 387
7.3.3 Variability of Fixed and Random Effects Estimators......Page 393
7.3.4 Testing Hypotheses......Page 395
7.4 Bayesian Linear Mixed Models......Page 398
7.4.1 Estimation for Known Covariance Structure......Page 399
7.4.2 Estimation for Unknown Covariance Structure......Page 400
7.5.1 GLMMs for Longitudinal and Clustered Data......Page 404
7.5.2 Conditional and Marginal Models......Page 407
7.6 Likelihood and Bayesian Inference in GLMMs......Page 409
7.6.1 Penalized Likelihood and Empirical Bayes Estimation......Page 410
7.6.2 Fully Bayesian Inference Using MCMC......Page 412
7.7.1 General Guidelines and Recommendations......Page 416
7.7.2 Case Study on Sales of Orange Juice......Page 418
7.8.1 Bibliographic Notes......Page 424
7.8.2 Proofs......Page 425
8 Nonparametric Regression......Page 428
8.1.1 Polynomial Splines......Page 430
8.1.2 Penalized Splines (P-Splines)......Page 446
8.1.3 General Penalization Approaches......Page 461
8.1.4 Smoothing Splines......Page 463
8.1.5 Random Walks......Page 467
8.1.6 Kriging......Page 468
8.1.7 Local Smoothing Procedures......Page 475
8.1.8 General Scatter Plot Smoothing......Page 483
8.1.9 Choosing the Smoothing Parameter......Page 493
8.1.10 Adaptive Smoothing Approaches......Page 505
8.2 Bivariate and Spatial Smoothing......Page 515
8.2.1 Tensor Product P-Splines......Page 518
8.2.2 Radial Basis Functions and Thin Plate Splines......Page 527
8.2.3 Kriging: Spatial Smoothing with Continuous Location Variables......Page 530
8.2.4 Markov Random Fields......Page 536
8.2.5 Summary of Roughness Penalty Approaches......Page 542
8.2.6 Local and Adaptive Smoothing......Page 544
8.3 Higher-Dimensional Smoothing......Page 545
8.4 Bibliographic Notes......Page 546
9 Structured Additive Regression......Page 550
9.1 Additive Models......Page 551
9.2 Geoadditive Regression......Page 555
9.3 Models with Interactions......Page 558
9.3.1 Models with Varying Coefficient Terms......Page 559
9.3.2 Interactions Between Two Continuous Covariates......Page 562
9.4 Models with Random Effects......Page 564
9.5 Structured Additive Regression......Page 568
9.6.1 Penalized Least Squares or Likelihood Estimation......Page 576
9.6.2 Inference Based on Mixed Model Representation......Page 581
9.6.3 Bayesian Inference Based on MCMC......Page 583
9.7 Boosting STAR Models......Page 588
9.8.1 General Guidelines......Page 591
9.8.2 Descriptive Analysis......Page 595
9.8.3 Modeling Variants......Page 598
9.8.4 Estimation Results and Model Evaluation......Page 599
9.8.5 Automatic Function Selection......Page 604
9.9 Bibliographic Notes......Page 609
10 Quantile Regression......Page 612
10.1 Quantiles......Page 614
10.2.1 Classical Quantile Regression......Page 616
10.2.2 Bayesian Quantile Regression......Page 624
10.3 Additive Quantile Regression......Page 627
10.4.1 Bibliographic Notes......Page 631
10.4.2 Proofs......Page 633
A.1 Definition and Elementary Matrix Operations......Page 636
A.2 Rank of a Matrix......Page 641
A.3 Block Matrices and the Matrix Inversion Lemma......Page 643
A.4 Determinant and Trace of a Matrix......Page 644
A.6 Eigenvalues and Eigenvectors......Page 646
A.7 Quadratic Forms......Page 648
A.8 Differentiation of Matrix Functions......Page 650
B.1 Some Univariate Distributions......Page 654
B.2 Random Vectors......Page 660
B.3.1 Definition and Properties......Page 663
B.3.2 The Singular Multivariate Normal Distribution......Page 665
B.3.4 Multivariate t-Distribution......Page 666
B.3.5 Normal-Inverse Gamma Distribution......Page 667
B.4.1 Maximum Likelihood Estimation......Page 668
B.4.2 Numerical Computation of the MLE......Page 675
B.4.4 Likelihood-Based Tests of Linear Hypotheses......Page 677
B.4.5 Model Choice......Page 679
B.5.1 Basic Concepts of Bayesian Inference......Page 680
B.5.2 Point and Interval Estimation......Page 684
B.5.3 MCMC Methods......Page 685
B.5.4 Model Selection......Page 691
B.5.5 Model Averaging......Page 694
Bibliography......Page 696
Index......Page 706