Title Page......Page 4
Copyright Page......Page 5
Contents......Page 6
List of Illustrations......Page 12
Preface......Page 16
Acknowledgments......Page 18
Brief History of LSA......Page 20
Key Issues Related to LSAs......Page 23
Conclusion......Page 26
References......Page 27
PART I: ASSESSMENT DESIGN, DEVELOPMENT, AND DELIVERY......Page 30
Introduction......Page 32
Traditional Procedures for Large-Scale Test Item Development......Page 34
Examples of Testing Programs With "Cognition in Mind"......Page 39
Conclusion......Page 43
References......Page 44
3 Role and Design of Background Questionnaires in Large-Scale Assessments......Page 46
An Example: Who Is Failing Ontario's Literacy Test and Why?......Page 47
What Information Do We Need?......Page 48
Prioritizing the Information Needed......Page 53
Writing the Questions......Page 54
Testing the Questions......Page 57
Resources......Page 59
References......Page 60
4 Student Motivation in Large-Scale Assessments......Page 62
Theories of Motivation......Page 63
Self-Report Measures of Motivation......Page 64
Student Test-Taking Behaviors as Measures of Motivation......Page 65
The Impact of Low Motivation on Large-Scale Assessment Results......Page 67
Minimizing the Impact of Low Motivation on Large-Scale Assessment Results......Page 68
Multiple Perspectives......Page 69
Agenda for Future Research......Page 77
Conclusion......Page 78
References......Page 79
Introduction......Page 81
A Brief History of CBT......Page 82
CBT Venues and Platforms......Page 83
Item Types and Interfaces......Page 86
Test Designs for Delivering CBT......Page 90
Conclusion......Page 99
Notes......Page 100
References......Page 101
PART II: ASSESSING DIVERSE POPULATIONS......Page 104
Introduction......Page 106
Population Misspecification and the Definition of English Language Learners......Page 109
Measurement Error, Sampling, and the Testing of English Language Learners......Page 111
Measurement Error and Overgeneralization in Test Adaptation for English Language Learners......Page 115
Probabilistic Approaches for Improving Assessment Systems for English Language Learners......Page 119
Conclusion......Page 123
References......Page 124
Introduction......Page 129
Review of Research on Comparability of Multiple Language Versions of Tests......Page 131
Current Practices Within Canadian Jurisdictions......Page 132
Challenges and Solutions to Ensuring Score Comparability Within Jurisdictions......Page 135
References......Page 140
Introduction......Page 144
Accommodation Appropriateness......Page 146
Research on the Effectiveness of Accommodations......Page 148
Determining Accommodation Appropriateness to Examine Effectiveness......Page 150
Practical Implications......Page 151
Upgrading Practice......Page 155
Conclusion......Page 156
References......Page 157
PART III: SCORING, SCORE REPORTING, AND USE OF SCORES......Page 160
Introduction......Page 162
Overview of Provincial LSAs Administered in British Columbia......Page 163
Scoring in LSAs of Educational Achievement......Page 165
Diverse Models Used to Score the Foundation Skills Assessment......Page 169
Conclusion......Page 170
References......Page 171
Introduction......Page 173
Performance Level Descriptors......Page 175
Traditional Standard-Setting Methods......Page 178
The Big Picture......Page 185
Expected Future of Standard Setting......Page 186
Issues of Reliability and Validity for Standard Setting......Page 190
Conclusion......Page 191
References......Page 192
Introduction......Page 194
A Brief Overview of Online Reporting Resources......Page 195
Guidelines for Static, Results-Oriented Online Documents......Page 198
Guidelines for Interactive Results-Oriented Tools......Page 199
Guidelines for Programmatic/Informational Web Pages......Page 200
Guidelines for Test- and Score-Reporting Website Home Pages......Page 201
Developing Score-Reporting Resources: The Hambleton and Zenisky Model......Page 202
Conclusion......Page 203
References......Page 204
Introduction......Page 205
Value-Added Models......Page 207
How Well Do VAMs Estimate the Effects of Teachers or Schools on Student Achievement?......Page 209
Value-Added Inferences and Shoe Leather......Page 212
The Stability of Value-Added Indicators......Page 216
Implicit Assumptions About Large-Scale Assessments......Page 217
Conclusion......Page 219
References......Page 220
PART IV: PSYCHOMETRIC MODELING AND STATISTICAL ANALYSIS......Page 224
13 Complex Person and Item Sampling: Implications for Analysis and Inference......Page 226
Sampling and Statistical Inference......Page 227
Types of Sampling Designs for Students and Items......Page 228
Motivating Example......Page 230
Impact of Sampling Design on Estimation and Inference......Page 233
Accommodating Complex Sampling Designs of Students in Analyses......Page 237
Accommodating Complex Sampling Designs of Items in Analyses......Page 245
Conclusion......Page 247
References......Page 249
Introduction......Page 257
Formulation of Logistic Models......Page 258
Multidimensional Models of Person-Characteristic Curves......Page 265
Estimation of Person Parameters......Page 267
Examples of Application......Page 268
Example of the Application to the PISA Data (2003)......Page 270
Conclusion......Page 273
References......Page 274
Introduction......Page 279
Definition, Classification, and Mechanisms......Page 280
Prevention......Page 281
Diagnosis......Page 283
Treatment......Page 284
Conclusion......Page 293
References......Page 294
Introduction......Page 295
Measurement Issues......Page 296
Statistical Analysis Issues......Page 299
Conclusion......Page 306
References......Page 307
Index......Page 310