Preface
The Conceptual Orientation of This Book, Its Purpose, and the Intended Audience
Organizational Overview
New to This Edition
General Changes
Chapter-Specific Changes
Author’s Acknowledgments
Publisher’s Acknowledgments
About the Author
Chapter 1 • Psychometrics and the Importance of Psychological Measurement
Why Psychological Testing Matters to You
Observable Behavior and Unobservable Psychological Attributes
Psychological Tests: Definition and Types
What Is a Psychological Test?
Types of Tests
What Is Psychometrics?
Psychometrics
A Brief History of Psychometrics
Challenges to Measurement in Psychology
The Importance of Individual Differences
But Psychometrics Goes Well Beyond “Differential” Psychology
Suggested Readings
PART I • BASIC CONCEPTS IN MEASUREMENT
Chapter 2 • Scaling
Fundamental Issues With Numbers
The Property of Identity
The Property of Order
The Property of Quantity
The Number 0
Units of Measurement
Additivity and Counting
Additivity
Counts: When Do They Qualify as Measurement?
Four Scales of Measurement
Nominal Scales
Ordinal Scales
Interval Scales
Ratio Scales
Scales of Measurement: Practical Implications
Additional Issues Regarding Scales of Measurement
Technical Appendix: R Syntax
Summary
Suggested Readings
Chapter 3 • Differences, Consistency, and the Meaning of Test Scores
The Nature of Variability
Importance of Individual Differences
Variability and Distributions of Scores
Central Tendency
Variability
Distribution Shapes and Normal Distributions
Quantifying the Association or Consistency Between Distributions
Interpreting the Association Between Two Variables
Scatterplots: Visually Representing the Association Between Two Variables
Covariance
Correlation
Variance and Covariance for “Composite Variables”
Binary Items
Interpreting Test Scores
Needed: An Interpretive Frame of Reference
z Scores (Standard Scores)
Converted Standard Scores (Standardized Scores)
Percentile Ranks
Normalized Scores
Test Norms
Representativeness of the Reference Sample
Technical Appendix: R Syntax
Summary
Suggested Readings
Chapter 4 • Test Dimensionality and Factor Analysis
Test Dimensionality
Three Dimensionality Questions: What They Are and Why They Matter
Unidimensional Tests
Multidimensional Tests With Correlated Dimensions (Tests With Higher-Order Factors)
Multidimensional Tests With Uncorrelated Dimensions
The Psychological Meaning of Test Dimensions
Factor Analysis: Examining the Dimensionality of a Test
The Logic and Purpose of Exploratory Factor Analysis: A Conceptual Overview
Conducting and Interpreting an Exploratory Factor Analysis
A Deeper Perspective on Factors, Factor Loadings, and Rotation
Factor Analysis of Binary Items
A Quick Look at Confirmatory Factor Analysis
Technical Appendix: R Syntax
Summary
Suggested Readings
PART II • RELIABILITY
Chapter 5 • Reliability: Conceptual Basis
Overview of Reliability and Classical Test Theory
Observed Scores, True Scores, and Measurement Error
Variances in Observed Scores, True Scores, and Error Scores
Four Ways to Think of Reliability
Reliability as the Ratio of True Score Variance to Observed Score Variance
Reliability as Lack of Error Variance
Reliability as the (Squared) Correlation Between Observed Scores and True Scores
Reliability as the Lack of (Squared) Correlation Between Observed Scores and Error Scores
Reliability and the Standard Error of Measurement
From Theory to Practice: Measurement Models and Their Implications for Estimating Reliability
Overview of Key Assumptions
Parallel Tests
Tau-Equivalent and Essentially Tau-Equivalent Tests
Congeneric Tests
Tests With Correlated Errors
Summary
Domain Sampling Theory
Summary
Suggested Readings
Chapter 6 • Empirical Estimates of Reliability
Alternate Forms Method of Estimating Reliability
Test–Retest Method of Estimating Reliability
Internal Consistency Method of Estimating Reliability
Split-Half Estimates of Reliability
“Raw” Coefficient Alpha
“Standardized” Coefficient Alpha
Raw Alpha for Binary Items: KR20
Omega
On the Assumptions Underlying Alpha and Omega, the Relative Applicability of Those Indices, and Their Limitations
Internal Consistency Versus Dimensionality
Factors Affecting the Reliability of Test Scores
Sample Heterogeneity and Reliability Generalization
Reliability of Difference Scores
Defining Difference Scores
Estimating the Reliability of Difference Scores
Factors Affecting the Reliability of Difference Scores
The Problem of Unequal Variability
Difference Scores: Summary and Caution
Technical Appendix: R Syntax
Summary
Suggested Readings
Note
Chapter 7 • The Importance of Reliability
Applied Behavioral Practice: Evaluation of an Individual’s Test Score
Point Estimates of True Scores
Confidence Intervals
Debate and Alternatives
Summary
Behavioral Research
Reliability, True Associations, and Observed Associations
Measurement Error (Low Reliability) Attenuates the Observed Associations Between Measures
Reliability, Effect Sizes, and Statistical Significance
Implications for Conducting and Interpreting Behavioral Research
Summary
Test Construction and Refinement
Item Discrimination and Other Information Regarding Internal Consistency
Item Difficulty (Mean) and Item Variance
Technical Appendix: R Syntax
Summary
Suggested Readings
PART III • VALIDITY
Chapter 8 • Validity: Conceptual Basis
What Is Validity?
The Importance of Validity
Validity Evidence: Test Content
Expert Rating Evidence
Threats to Content Validity
Content Validity Versus Face Validity
Validity Evidence: Internal Structure of the Test
Factor-Analytic Evidence
Validity Evidence: Response Processes
Direct Evidence
Indirect Evidence
Validity Evidence: Associations With Other Variables
Convergent Evidence
Discriminant Evidence
Criterion, Concurrent, and Predictive Evidence
Validity Evidence: Consequences of Testing
Evidence of Intended Effects
Evidence Regarding Unintended Differential Impact on Groups
Evidence Regarding Unintended Systemic Effects
Other Perspectives on Validity
Contrasting Reliability and Validity
Summary
Suggested Readings
Chapter 9 • Estimating and Evaluating Convergent and Discriminant Validity Evidence
A Construct’s Nomological Network
Methods for Evaluating Convergent and Discriminant Validity
Focused Associations
Sets of Correlations
Multitrait–Multimethod Matrices
Quantifying Construct Validity
Factors Affecting a Validity Coefficient
Associations Between Constructs
Random Measurement Error and Reliability
Restricted Range
Skew and Relative Proportions
Method Variance
Time
Predictions of Single Events
Interpreting a Validity Coefficient
Squared Correlations and “Variance Explained”
Estimating Practical Effects: Binomial Effect Size Display, Taylor-Russell Tables, Utility Analysis, and Sensitivity/Specificity
Guidelines or Norms for a Field
Statistical Significance
Technical Appendix: R Syntax
Summary
Suggested Readings
Notes
PART IV • THREATS TO PSYCHOMETRIC QUALITY
Chapter 10 • Response Biases
Types of Response Biases
Acquiescence Bias (“Yea-Saying and Nay-Saying”)
Extreme and Moderate Responding
Social Desirability (“Faking Good”)
Malingering (“Faking Bad”)
Careless or Random Responding
Guessing
Methods for Coping With Response Biases
Minimizing the Existence of Bias by Managing the Testing Context
Minimizing the Existence of Bias by Managing Test Content
Minimizing the Effects of Bias by Managing Test Content or Scoring
Managing Test Content to Detect Bias and Intervene
Using Specialized Tests to Detect Bias and Intervene
Response Biases, Response Sets, and Response Styles
Summary
Suggested Readings
Chapter 11 • Test Bias
Why Worry About Test Score Bias?
Detecting Construct Bias: Internal Evaluation of a Test
Reliability
Rank Order
Item Discrimination Index
Factor Analysis
Differential Item Functioning Analyses
Summary
Detecting Predictive Bias: External Evaluation of a Test
Basics of Regression Analysis
One Size Fits All: The Common Regression Equation
Intercept Bias
Slope Bias
Intercept and Slope Bias
Criterion Score Bias
The Effect of Reliability
Other Statistical Procedures
Test Fairness
Example: Is the SAT Biased in Terms of Race or Socioeconomic Status?
Race/Ethnicity
Socioeconomic Status
Technical Appendix: R Syntax
Summary
Suggested Readings
Notes
PART V • ADVANCED PSYCHOMETRIC APPROACHES
Chapter 12 • Confirmatory Factor Analysis
On the Use of EFA and CFA
The Frequency and Roles of EFA and CFA
Using CFA to Evaluate Measurement Models
The Process of CFA for Analysis of a Scale’s Internal Structure
Overview of CFA and an Example
Preliminary Steps
Step 1: Specification of the Measurement Model
Step 2: Computations
Step 3: Interpreting and Reporting Output
Step 4: Model Modification and Reanalysis (If Necessary)
Comparing Models
Summary
CFA and Reliability
Evaluating Types of Classical Test Theory Measurement Models
Estimating Reliability (Omega Index)
CFA and Validity
CFA and Measurement Invariance
The Meaning of Measurement Invariance
Levels of Invariance: Meaning and Detection
Technical Appendix: R Syntax
Summary
Suggested Readings
Chapter 13 • Generalizability Theory
Multiple Facets of Measurement
Generalizability, Universes, and Variance Components
G Studies and D Studies
Conducting and Interpreting Generalizability Theory Analysis: A One-Facet Design
Phase 1: G Study
Phase 2: D Study
Conducting and Interpreting Generalizability Theory Analysis: A Two-Facet Design
Phase 1: G Study
Phase 2: D Study
Other Measurement Designs
Number of Facets
Random Versus Fixed Facets
Crossed Versus Nested Designs
Relative Versus Absolute Decisions
A Practical, Consistency-Oriented Interpretation of Variance Components
Systematic Variance Components Reflect “Consistent Variance”
Residual/Error Variance Component Reflects Inconsistent Variance
Generalizability Coefficients as the Proportion of Variance That Is Consistent
More Complex Designs
Technical Appendix: R Syntax
Summary
Suggested Readings
Notes
Chapter 14 • Item Response Theory and Rasch Models
Factors Affecting Responses to Test Items
Respondent Trait Level as a Determinant of Item Responses
Item Difficulty as a Determinant of Item Responses
Item Discrimination as a Determinant of Item Responses
Guessing
IRT Measurement Models
One-Parameter Logistic Model (or Rasch Model)
Two-Parameter Logistic Model
Three-Parameter Logistic Model
Graded Response Model
Obtaining Parameter Estimates: A 1PL Example
Model Fit
Item and Test Information
Item Characteristic Curves
Item Information and Test Information
Applications of IRT
Test Development and Improvement
Differential Item Functioning
Person Fit
Computerized Adaptive Testing
Technical Appendix: R Syntax
Summary
Suggested Readings
Glossary
References
Index