Semiparametric Regression

✍ Scribed by David Ruppert, M. P. Wand, R. J. Carroll

• Publisher: Cambridge University Press
• Year: 2003
• Tongue: English
• Leaves: 404
• Series: Cambridge Series in Statistical and Probabilistic Mathematics
• Edition: 1
• Category: Library

✦ Synopsis

Science abounds with problems where the data are noisy and the answer is not a straight line. Semiparametric regression analysis helps make sense of such data in application areas that include engineering, finance, medicine and public health. The book is geared towards researchers and professionals with little background in regression as well as statistically oriented scientists (biostatisticians, econometricians, quantitative social scientists, and epidemiologists) with knowledge of regression and the desire to begin using more flexible semiparametric models.

✦ Table of Contents

Half-title......Page 3
Series-title......Page 4
Title......Page 5
Copyright......Page 6
Dedication......Page 7
Contents......Page 9
Preface......Page 15
Acknowledgments......Page 16
Guide to Notation......Page 17
1 Introduction......Page 19
1.1 Assessing the Carcinogenicity of Phenolphthalein......Page 21
1.2 Salinity and Fishing in North Carolina......Page 22
1.3 Management of a Retirement Fund......Page 23
1.5 Term Structure of Interest Rates......Page 25
1.6 Air Pollution and Mortality in Milan: The Harvesting Effect......Page 29
2.2 Linear Regression Models......Page 33
2.2.1 General Linear Model......Page 37
2.3 Regression Diagnostics......Page 38
2.3.1 Influence Diagnostics......Page 42
2.3.2 Autocorrelation......Page 44
2.4.1 Confidence and Prediction Intervals......Page 46
2.4.2 Inference about the Regression Coefficients......Page 48
2.4.3 t-Statistics and p-Values......Page 49
2.4.5 Inference about New Observations......Page 50
2.4.6 Estimation of Sigma......Page 51
2.4.7 Extra Sums of Squares and Hypothesis Testing......Page 52
2.5 Parametric Additive Models......Page 54
2.5.1 Displaying Additive Fits......Page 55
2. 5.1.1 Vertical Alignment......Page 56
2.5.1.2 Variability Bands......Page 57
2.5.1.3 Partial Residuals......Page 58
2.5.1.4 Derivative Plots......Page 59
2.5.2 Degrees of Freedom......Page 60
2.6 Model Selection......Page 62
2.7 Polynomial Regression Models......Page 64
2.8 Nonlinear Regression......Page 66
2.9 Transformations in Regression......Page 69
2.9.1 LIDAR Monitoring of Air Pollutants......Page 72
2.11 Summary of Formulas......Page 73
3.1 Introduction......Page 75
3.2 Preliminary Ideas......Page 76
3.3 Practical Implementation......Page 80
3.4 Automatic Knot Selection......Page 82
3.5 Penalized Spline Regression......Page 83
3.6 Quadratic Spline Bases......Page 85
3.7 Other Spline Models and Bases......Page 87
3.7.1 B-Splines......Page 88
3.7.2 Natural Cubic Splines......Page 89
3.7.3 Radial Basis Functions......Page 90
3.8 Other Penalties......Page 92
3.9 General Definition of a Penalized Spline......Page 93
3.11 Error of a Smoother......Page 94
3.12 Rank of a Smoother......Page 96
3.13 Degrees of Freedom of a Smoother......Page 98
3.14 Residual Degrees of Freedom......Page 100
3.15.1 Local Polynomial Fitting......Page 102
3.15.2 Series-Based Smoothers......Page 104
3.16 Choosing a Scatterplot Smoother......Page 105
3.17 Bibliographic Notes......Page 106
3.18 Summary of Formulas......Page 107
4.2 Mixed Models......Page 109
4.2.1 Degrees-of-Freedom Interpretation......Page 111
4.3 Prediction......Page 113
4.3.1 Best Linear Prediction (BLP)......Page 114
4.3.2 Application to Pig Weight Example......Page 115
4.5.1 Estimation of Fixed Effects......Page 116
4.5.3 Best Linear Unbiased Prediction (BLUP)......Page 117
4.5.4 Estimation of Covariance Matrices......Page 118
4.6 Estimated BLUP (EBLUP)......Page 119
4.7 Standard Error Estimation......Page 120
4.7.1 Summary of Fit to Pig Weights......Page 121
4.8.1 Normal Theory Tests......Page 122
4.8.2 Likelihood Ratio Tests......Page 123
4.8.3 Restricted Likelihood Ratio Tests......Page 125
4.9 Penalized Splines as BLUPs......Page 126
4.11 Summary of Formulas......Page 128
5.1 Introduction......Page 130
5.2 The Likelihood Approach......Page 131
5.3.1 Cross-Validation (CV)......Page 132
5.3.2 Generalized Cross-Validation (GCV)......Page 134
5.3.2.1 Age and Income Data......Page 135
5.3.3 Mallows’s C Criterion......Page 136
5.3.3.2 Relationship between GCV and C......Page 137
5.4 Caveats of Automatic Parameter Selection......Page 138
5.5.1 Varying the Number of Knots......Page 141
5.5.2 Varying the Degree of the Regression Spline......Page 142
5.5.3 Default Choices for Knot Locations......Page 143
5.6.2 Full-Search Algorithm......Page 145
5.6.3 A Simulation Study......Page 146
5.6.4 Fossil Data......Page 147
5.8 Summary of Formulas......Page 149
6.2 Variability Bands......Page 151
6.3 Confidence and Prediction Intervals......Page 153
6.4 Inference for Penalized Splines......Page 155
6.5 Simultaneous Confidence Bands......Page 160
6.6 Testing the Adequacy of Parametric Models......Page 163
6.6.1 Restricted Likelihood Ratio Tests......Page 164
6.6.2 F-Test Approach......Page 165
6.7 Testing for No Effect......Page 167
6.7.1 F-Test for No Effect......Page 168
6.8 Inference Using First Derivatives......Page 169
6.8.1 Derivative Estimation via Penalized Splines......Page 171
6.8.3 LIDAR Data......Page 172
6.9 Testing for Existence of a Feature......Page 174
6.10 Bibliographical Notes......Page 176
6.11 Summary of Formulas......Page 177
7.2 Beyond Scatterplot Smoothing......Page 179
7.3 Semiparametric Binary Offset Model......Page 180
7.5 General Parametric Component......Page 182
7.6 Inference......Page 185
7.7 Bibliographical Notes......Page 186
8.1 Introduction......Page 188
8.2 Fitting an Additive Model......Page 189
8.3 Degrees of Freedom......Page 192
8.4 Smoothing Parameter Selection......Page 194
8.4.1 Upper Cape Cod Birthweight Data......Page 197
8.5 Hypothesis Testing......Page 199
8.5.2 F-tests......Page 200
8.6 Model Selection......Page 201
8.6.3 MCMC Model Selection Algorithms......Page 202
8.7 Bibliographical Notes......Page 203
9.2 Additive Mixed Models......Page 204
9.2.1 Additive Model Extension......Page 206
9.2.2 Serially Correlated Errors......Page 208
9.3 Subject-Specific Curves......Page 209
9.4 Bibliographical Notes......Page 210
10.2 Binary Response Data......Page 212
10.3 Logistic Regression......Page 213
10.4 Other Generalized Linear Models......Page 215
10.4.1 Poisson Regression and Overdispersion......Page 216
10.4.3 The Gamma GLM: A Model with a Constant Coefficient of Variation......Page 217
10.5 Iteratively Reweighted Least Squares......Page 218
10.7 Overdispersion and Variance Functions: Pseudolikelihood......Page 219
10.8 Generalized Linear Mixed Models......Page 221
10.8.1 Estimation of Model Parameters......Page 222
10.8.2 Penalized Quasilikelihood (PQL)......Page 223
10.8.5 Fitting via Expectation Maximization......Page 224
10.8.6 Bayesian Fitting via Markov Chain Monte Carlo......Page 225
10.8.8 Standard Error Estimation......Page 226
10.9 Deviance......Page 227
10.10.1 Fitting a Logistic Regression......Page 228
10.10.2 Standard Error Estimation in Logistic Regression......Page 229
10.10.4 Derivation of PQL......Page 230
10.11 Bibliographical Notes......Page 231
11.1 Introduction......Page 232
11.2 Generalized Scatterplot Smoothing......Page 233
11.2.1 Application to Wage–Union Data......Page 234
11.3 Generalized Additive Mixed Models......Page 235
11.4 Degrees-of-Freedom Approximations......Page 237
11.6 Hypothesis Testing......Page 238
11.8 Density Estimation......Page 239
11.9 Bibliographical Notes......Page 240
12.1 Introduction......Page 241
12.2 Binary-by-Continuous Interaction Models......Page 242
12.3.1 Modularity of Spline Models......Page 244
12.3.2 Example: Ragweed Pollen Revisited......Page 245
12.3.3 Discrete-by-Continuous Interactions......Page 247
12.3.4 Interactions in Additive Models......Page 248
12.3.5 Generalized Additive Models with Interactions......Page 249
12.3.6 Pollen Data......Page 250
12.4 Varying Coefficient Models......Page 252
12.5 Continuous-by-Continuous Interactions......Page 253
12.6 Bibliographical Notes......Page 255
13.1 Introduction......Page 256
13.2 Choice of Bivariate Basis Functions......Page 258
13.3 Kriging......Page 260
13.3.1 The Kriging Algorithm......Page 265
13.4 General Radial Smoothing......Page 266
13.4.2 Positive Definitization......Page 269
13.4.4 Low-Rank Radial Smoothers......Page 270
13.4.5 Higher-Dimensional Radial Smoothers......Page 271
13.4.6 Choice of Knots......Page 273
13.5 Default Automatic Bivariate Smoother......Page 274
13.6 Geoadditive Models......Page 276
13.9 Appendix: Equivalence of BLUP using Z and Z......Page 277
13.10 Bibliographical Notes......Page 278
14.1 Introduction......Page 279
14.2 Formulation......Page 281
14.3 Application to the LIDAR Data......Page 282
14.4 Quasilikelihood and Variance Functions......Page 284
14.5 Bibliographical Notes......Page 285
15.1 Introduction......Page 286
15.2 Formulation......Page 287
15.3 The Expectation Maximization (EM) Algorithm......Page 288
15.5 Sensitivity Analysis Example......Page 291
15.6 Bibliographical Notes......Page 293
16.1 Introduction......Page 294
16.2.1 Markov Chain Monte Carlo......Page 295
16.2.2 Credible Sets......Page 296
16.3 Scatterplot Smoothing......Page 297
16.3.1 Application to LIDAR Data......Page 300
16.4 Linear Mixed Models......Page 303
16.5 Generalized Linear Mixed Models......Page 306
16.5.1 Probit Mixed Models......Page 307
16.5.1.1 Union and Wages Data Revisited......Page 308
16.6 Rao–Blackwellization......Page 309
16.7 Bibliographical Notes......Page 310
17.1 Introduction......Page 311
17.2 A Local Penalty Method......Page 312
17.3 Completely Automatic Algorithm......Page 313
17.4 Bayesian Inference......Page 314
17.5.1 Effects of the Tuning Parameters......Page 316
17.5.2 The Automatic Algorithms......Page 317
17.5.3 Bayesian Inference......Page 320
17.6 LIDAR Example......Page 322
17.7.2 Simulations of an Additive Model......Page 323
17.8 Bibliographical Notes......Page 325
18.3 Salinity and Fishing in North Carolina......Page 326
18.4 Management of a Retirement Fund......Page 331
18.5 Biomonitoring of Airborne Mercury......Page 332
18.6 Term Structure of Interest Rates......Page 333
18.7 Air Pollution and Mortality in Milan: The Harvesting Effect......Page 337
19.2 Minimalist Statistics......Page 338
19.3.3 Nonquadratic Penalties......Page 339
19.3.5 Missing Data......Page 340
19.3.9 Diagnostics......Page 341
19.3.11 Constrained Smoothing......Page 342
19.4 Future Research......Page 343
A.2.2 Eigenvalues and Eigenvectors......Page 344
A.2.5 Elementwise Function Notation......Page 345
A.2.7 Triangular Matrices......Page 346
A.2.11 Matrix Square Root......Page 347
A.2.12 Derivative Vector and Hessian Matrix......Page 348
A.3.1 Vectors and Vector Spaces......Page 349
A.3.4 Bases......Page 350
A.4.2 Covariance Matrix of a Random Vector......Page 351
A.4.6 Multivariate Normal Distribution......Page 352
A.6 Bibliographical Notes......Page 353
B.1.1 Demmler–Reinsch Orthogonalization......Page 354
B.1.1.1 Justification of Algorithm A.1......Page 356
B.1.1.2 S-PLUS Implementation of Algorithm A.1......Page 357
B.1.1.3 MATLAB Implementation of Algorithm A.1......Page 358
B.1.2 QR Decomposition......Page 368
B.2 Computation of Covariance Matrix Estimators......Page 369
B.3.1.1 S-PLUS Functions......Page 371
B.3.2 S-PLUS Mixed Model Functions......Page 372
B.3.3 SAS Mixed Model Procedures......Page 374
B.3.4.1 Logistic Semiparametric Models......Page 375
B.3.4.5 Other Models......Page 378
Bibliography......Page 379
Author Index......Page 393
Notation Index......Page 398
Example Index......Page 399
Subject Index......Page 400