Statistical Foundations, Reasoning and Inference: For Science and Data Science (Springer Series in Statistics)

✍ Scribed by Göran Kauermann, Helmut Küchenhoff, Christian Heumann

• Publisher: Springer
• Year: 2021
• Tongue: English
• Leaves: 361
• Edition: 1st ed. 2021
• Category: Library

✦ Synopsis

This textbook provides a comprehensive introduction to statistical principles, concepts and methods that are essential in modern statistics and data science. The topics covered include likelihood-based inference, Bayesian statistics, regression, statistical tests and the quantification of uncertainty. Moreover, the book addresses statistical ideas that are useful in modern data analytics, including bootstrapping, modeling of multivariate distributions, missing data analysis, causality as well as principles of experimental design. The textbook includes sufficient material for a two-semester course and is intended for master’s students in data science, statistics and computer science with a rudimentary grasp of probability theory. It will also be useful for data science practitioners who want to strengthen their statistics skills.

✦ Table of Contents

Preface
Contents
1 Introduction
1.1 General Ideas
1.2 Databases, Samples and Biases
2 Background in Probability
2.1 Random Variables and Probability Models
2.1.1 Definitions of Probability
2.1.2 Independence, Conditional Probability, BayesTheorem
2.1.3 Random Variables
2.1.4 Common Distributions
2.1.5 Exponential Family Distributions
2.1.6 Random Vectors and Multivariate Distributions
2.2 Limit Theorems
2.3 Kullback–Leibler Divergence
2.4 Exercises
3 Parametric Statistical Models
3.1 Likelihood and Bayes
3.2 Parameter Estimation
3.2.1 Bayes Estimation
3.2.2 Maximum Likelihood Estimation
3.2.3 Method of Moments
3.2.4 Loss Function Approach
3.2.5 Kullback–Leibler Loss
3.3 Sufficiency and Consistency, Efficiency
3.3.1 Sufficiency
3.3.2 Consistency
3.3.3 Cramer-Rao Bound
3.4 Interval Estimates
3.4.1 Confidence Intervals
3.4.2 Credibility Interval
3.5 Exercises
4 Maximum Likelihood Inference
4.1 Score Function and Fisher Information
4.2 Asymptotic Normality
4.3 Numerical Calculation of ML Estimate
4.4 Likelihood-Ratio
4.5 Exercises
5 Bayesian Statistics
5.1 Bayesian Principles
5.2 Selecting a Prior Distribution
5.2.1 Jeffrey's Prior
5.2.2 Empirical Bayes
5.2.3 Hierarchical Prior
5.3 Integration Methods for the Posterior
5.3.1 Numerical Integration
5.3.2 Laplace Approximation
5.3.3 Monte Carlo Approximation
5.4 Markov Chain Monte Carlo (MCMC)
5.5 Variational Bayes
5.6 Exercises
6 Statistical Decisions
6.1 The Idea of Testing
6.2 Classical Tests
6.2.1 t-Test
6.2.2 Wald Test
6.2.3 Score Test
6.2.4 Likelihood-Ratio Test
6.3 Power of a Test and Neyman–Pearson Test
6.4 Goodness-of-Fit Tests
6.4.1 Chi-Squared Goodness-of-Fit Test
6.4.2 Kolmogorov–Smirnov Test
6.5 Tests on Independence
6.5.1 Chi-Squared Test of Independence
6.5.2 Fisher's Exact Test
6.5.3 Correlation-Based Tests
6.6 p-Value, Confidence Intervals and Test
6.6.1 The p-Value
6.6.2 Confidence Intervals and Tests
6.7 Bayes Factor
6.8 Multiple Testing
6.9 Significance and Relevance
6.9.1 Significance in Large Samples
6.9.2 Receiver Operating Characteristics
6.10 Exercises
7 Regression
7.1 Linear Model
7.1.1 Simple Linear Model
7.1.2 Multiple Linear Model
7.1.3 Bayesian Inference in the Linear Model
7.2 Weighted Regression
7.3 Quantile Regression
7.4 Nonparametric Smooth Models
7.5 Generalised Linear Models
7.6 Case Study in Generalised Additive Models
7.7 Exercises
8 Bootstrapping
8.1 Nonparametric Bootstrap
8.1.1 Motivation
8.1.2 Empirical Distribution Function and the Plug-In Principle
8.1.3 Bootstrap Estimate of a Standard Error
8.1.4 Bootstrap Estimate of a Bias
8.2 Parametric Bootstrap
8.3 Bootstrap in Regression Models
8.4 Theory and Extension of Bootstrapping
8.4.1 Theory of the Bootstrap
8.4.2 Extensions of the Bootstrap
8.4.3 Subsampling
8.5 Bootstrapping the Prediction Error
8.5.1 Prediction Error
8.5.2 Cross Validation Estimate of the Prediction Error
8.5.3 Bootstrapping the Prediction Error
8.6 Bootstrap Confidence Intervals and Hypothesis Testing
8.6.1 Bootstrap Confidence Intervals
8.6.2 Testing
8.7 Sampling from Data
8.8 Exercises
9 Model Selection and Model Averaging
9.1 Akaike Information Criterion
9.1.1 Maximum Likelihood in Misspecified Models
9.1.2 Derivation of AIC
9.1.3 AIC for Model Comparison
9.1.4 Extensions and Modifications
Bias-Corrected AIC
The Bayesian Information Criterion
Deviance Information Criterion
Cross Validation
9.2 AIC/BIC Model Averaging
9.3 Inference After Model Selection
9.4 Model Selection with Lasso
9.5 The Bayesian Model Selection
9.6 Exercises
10 Multivariate and Extreme Value Distributions
10.1 Multivariate Normal Distribution
10.1.1 Parameterisation
10.1.2 Graphical Models
10.1.3 Principal Component Analysis
10.2 Copulas
10.2.1 Copula Construction
10.2.2 Common Copula Models
Gaussian and Elliptical Copulas
Archimedean Copula
Pair Copula
10.2.3 Tail Dependence
10.3 Statistics of Extremes
10.4 Exercises
11 Missing and Deficient Data
11.1 Missing Data
11.1.1 Missing Data Mechanisms
11.1.2 EM Algorithm
11.1.3 Multiple Imputation
11.1.4 Censored Observations
11.1.5 Omitting Variables (Simpson's Paradox)
11.2 Biased Data
11.3 Quality Versus Quantity
11.4 Measurement and Measurement Error
11.4.1 Theory of Measurement
11.4.2 Effect of Measurement Error in Regression
11.4.3 Correction for Measurement Error in LinearRegression
11.4.4 General Strategies for Measurement Error Correction
11.5 Exercises
12 Experiments and Causality
12.1 Design of Experiments
12.1.1 Experiment Versus Observational Data
12.1.2 ANOVA
12.1.3 Block Designs
12.1.4 More Complex Designs
12.2 Instrumental Variables
12.3 Propensity Score Matching
12.4 Directed Acyclic Graphs (DAGs)
12.5 Exercises
References
Index