Data Analysis for Scientists and Engineers

✍ Scribed by Edward L. Robinson

• Publisher: Princeton University Press
• Year: 2017
• Tongue: English
• Leaves: 408
• Edition: Core Textbook
• Category: Library

✦ Synopsis

Data Analysis for Scientists and Engineers is a modern, graduate-level text on data analysis techniques for physical science and engineering students as well as working scientists and engineers. Edward Robinson emphasizes the principles behind various techniques so that practitioners can adapt them to their own problems, or develop new techniques when necessary.

Robinson divides the book into three sections. The first section covers basic concepts in probability and includes a chapter on Monte Carlo methods with an extended discussion of Markov chain Monte Carlo sampling. The second section introduces statistics and then develops tools for fitting models to data, comparing and contrasting techniques from both frequentist and Bayesian perspectives. The final section is devoted to methods for analyzing sequences of data, such as correlation functions, periodograms, and image reconstruction. While it goes beyond elementary statistics, the text is self-contained and accessible to readers from a wide variety of backgrounds. Specialized mathematical topics are included in an appendix.

Based on a graduate course on data analysis that the author has taught for many years, and couched in the looser, workaday language of scientists and engineers who wrestle directly with data, this book is ideal for courses on data analysis and a valuable resource for students, instructors, and practitioners in the physical sciences and engineering.

• In-depth discussion of data analysis for scientists and engineers
• Coverage of both frequentist and Bayesian approaches to data analysis
• Extensive look at analysis techniques for time-series data and images
• Detailed exploration of linear and nonlinear modeling of data
• Emphasis on error analysis
• Instructor’s manual (available only to professors)

✦ Table of Contents

Contents
Preface
1. Probability
1.1 The Laws of Probability
1.2 Probability Distributions
1.2.1 Discrete and Continuous Probability Distributions
1.2.2 Cumulative Probability Distribution Function
1.2.3 Change of Variables
1.3 Characterizations of Probability Distributions
1.3.1 Medians, Modes, and Full Width at Half Maximum
1.3.2 Moments, Means, and Variances
1.3.3 Moment Generating Function and the Characteristic Function
1.4 Multivariate Probability Distributions
1.4.1 Distributions with Two Independent Variables
1.4.2 Covariance
1.4.3 Distributions with Many Independent Variables
2.
Some Useful Probability Distribution Functions
2.1 Combinations and Permutations
2.2 Binomial Distribution
2.3 Poisson Distribution
2.4 Gaussian or Normal Distribution
2.4.1 Derivation of the Gaussian Distribution—Central Limit Theorem
2.4.2 Summary and Comments on the Central Limit Theorem
2.4.3 Mean, Moments, and Variance of the Gaussian Distribution
2.5 Multivariate Gaussian Distribution
2.6 χ^2 Distribution
2.6.1 Derivation of the χ^2 Distribution
2.6.2 Mean, Mode, and Variance of the X^2 Distribution
2.6.3 X^2 Distribution in the Limit of Large n
2.6.4 Reduced X^2
2.6.5 X^2 for Correlated Variables
2.7 Beta Distribution
3.
Random Numbers and Monte Carlo Methods
3.1 Introduction
3.2 Nonuniform Random Deviates
3.2.1 Inverse Cumulative Distribution Function Method
3.2.2 Multidimensional Deviates
3.2.3 Box-Müller Method for Generating Gaussian Deviates
3.2.4 Acceptance-Rejection Algorithm
3.2.5 Ratio of Uniforms Method
3.2.6 Generating Random Deviates from More Complicated Probability Distributions
3.3 Monte Carlo Integration
3.4 Markov Chains
3.4.1 Stationary, Finite Markov Chains
3.4.2 Invariant Probability Distributions
3.4.3 Continuous Parameter and Multiparameter Markov Chains
3.5 Markov Chain Monte Carlo Sampling
3.5.1 Examples of Markov Chain Monte Carlo Calculations
3.5.2 Metropolis-Hastings Algorithm
3.5.3 Gibbs Sampler
4. Elementary Frequentist Statistics
4.1 Introduction to Frequentist Statistics
4.2 Means and Variances for Unweighted Data
4.3 Data with Uncorrelated Measurement Errors
4.4 Data with Correlated Measurement Errors
4.5 Variance of the Variance and Student’s t Distribution
4.5.1 Variance of the Variance
4.5.2 Student’s t Distribution
4.5.3 Summary
4.6 Principal Component Analysis
4.6.1 Correlation Coefficient
4.6.2 Principal Component Analysis
4.7 Kolmogorov-Smirnov Test
4.7.1 One-Sample K-S Test
4.7.2 Two-Sample K-S Test
5. Linear Least Squares Estimation
5.1 Introduction
5.2 Likelihood Statistics
5.2.1 Likelihood Function
5.2.2 Maximum Likelihood Principle
5.2.3 Relation to Least Squares and X^2 Minimization
5.3 Fits of Polynomials to Data
5.3.1 Straight Line Fits
5.3.2 Fits with Polynomials of Arbitrary Degree
5.3.3 Variances, Covariances, and Biases
5.3.4 Monte Carlo Error Analysis
5.4 Need for Covariances and Propagation of Errors
5.4.1 Need for Covariances
5.4.2 Propagation of Errors
5.4.3 Monte Carlo Error Propagation
5.5 General Linear Least Squares
5.5.1 Linear Least Squares with Nonpolynomial Functions
5.5.2 Fits with Correlations among the Measurement Errors
5.5.3 X^2 Test for Goodness of Fit
5.6 Fits with More Than One Dependent Variable
6.
Nonlinear Least Squares Estimation
6.1 Introduction
6.2 Linearization of Nonlinear Fits
6.2.1 Data with Uncorrelated Measurement Errors
6.2.2 Data with Correlated Measurement Errors
6.2.3 Practical Considerations
6.3 Other Methods for Minimizing S
6.3.1 Grid Mapping
6.3.2 Method of Steepest Descent, Newton’s Method, and Marquardt’s Method
6.3.3 Simplex Optimization
6.3.4 Simulated Annealing
6.4 Error Estimation
6.4.1 Inversion of the Hessian Matrix
6.4.2 Direct Calculation of the Covariance Matrix
6.4.3 Summary and the Estimated Covariance Matrix
6.5 Confidence Limits
6.6 Fits with Errors in Both the Dependent and Independent Variables
6.6.1 Data with Uncorrelated Errors
6.6.2 Data with Correlated Errors
7.
Bayesian Statistics
7.1 Introduction to Bayesian Statistics
7.2 Single-Parameter Estimation: Means, Modes, and Variances
7.2.1 Introduction
7.2.2 Gaussian Priors and Likelihood Functions
7.2.3 Binomial and Beta Distributions
7.2.4 Poisson Distribution and Uniform Priors
7.2.5 More about the Prior Probability Distribution
7.3 Multiparameter Estimation
7.3.1 Formal Description of the Problem
7.3.2 Laplace Approximation
7.3.3 Gaussian Likelihoods and Priors: Connection to Least Squares
7.3.4 Difficult Posterior Distributions: Markov Chain Monte Carlo Sampling
7.3.5 Credible Intervals
7.4 Hypothesis Testing
7.5 Discussion
7.5.1 Prior Probability Distribution
7.5.2 Likelihood Function
7.5.3 Posterior Distribution Function
7.5.4 Meaning of Probability
7.5.5 Thoughts
8.
Introduction to Fourier Analysis
8.1 Introduction
8.2 Complete Sets of Orthonormal Functions
8.3 Fourier Series
8.4 Fourier Transform
8.4.1 Fourier Transform Pairs
8.4.2 Summary of Useful Fourier Transform Pairs
8.5 Discrete Fourier Transform
8.5.1 Derivation from the Continuous Fourier Transform
8.5.2 Derivation from the Orthogonality Relations for Discretely Sampled Sine and Cosine Functions
8.5.3 Parseval’s Theorem and the Power Spectrum
8.6 Convolution and the Convolution Theorem
8.6.1 Convolution
8.6.2 Convolution Theorem
9.
Analysis of Sequences: Power Spectra and Periodograms
9.1 Introduction
9.2 Continuous Sequences: Data Windows, Spectral Windows, and Aliasing
9.2.1 Data Windows and Spectral Windows
9.2.2 Aliasing
9.2.3 Arbitrary Data Windows
9.3 Discrete Sequences
9.3.1 The Need to Oversample Fm
9.3.2 Nyquist Frequency
9.3.3 Integration Sampling
9.4 Effects of Noise
9.4.1 Deterministic and Stochastic Processes
9.4.2 Power Spectrum of White Noise
9.4.3 Deterministic Signals in the Presence of Noise
9.4.4 Nonwhite, Non-Gaussian Noise
9.5 Sequences with Uneven Spacing
9.5.1 Least Squares Periodogram
9.5.2 Lomb-Scargle Periodogram
9.5.3 Generalized Lomb-Scargle Periodogram
9.6 Signals with Variable Periods:The O−C Diagram
10. Analysis of Sequences: Convolution and Covariance
10.1 Convolution Revisited
10.1.1 Impulse Response Function
10.1.2 Frequency Response Function
10.2 Deconvolution and Data Reconstruction
10.2.1 Effect of Noise on Deconvolution
10.2.2 Wiener Deconvolution
10.2.3 Richardson-Lucy Algorithm
10.3 Autocovariance Functions
10.3.1 Basic Properties of Autocovariance Functions
10.3.2 Relation to the Power Spectrum
10.3.3 Application to Stochastic Processes
10.4 Cross-Covariance Functions
10.4.1 Basic Properties of Cross-Covariance Functions
10.4.2 Relation to X^2 and to the Cross Spectrum
10.4.3 Detection of Pulsed Signals in Noise
Appendices
A.
Some Useful Definite Integrals
B.
Method of Lagrange Multipliers
C.
Additional Properties of the Gaussian Probability Distribution
D.
The n-Dimensional Sphere
E. Review of Linear Algebra and Matrices
F.
Limitof [1+f (x)/n]^n for Large n
G.
Green’s Function Solutions for Impulse Response Functions
H.
Second-Order Autoregressive Process
Bibliography
Index