Advanced Methods in Automatic Item Generation: Theoretical Foundations and Practical Applications

✍ Scribed by Mark J. Gierl, Hollis Lai, Vasily Tanygin

• Publisher: Routledge
• Year: 2021
• Tongue: English
• Leaves: 246
• Category: Library

✦ Synopsis

Advanced Methods in Automatic Item Generation is an up-to-date survey of the growing research on automatic item generation (AIG) in today’s technology-enhanced educational measurement sector. As test administration procedures increasingly integrate digital media and Internet use, assessment stakeholders—from graduate students to scholars to industry professionals—have numerous opportunities to study and create different types of tests and test items. This comprehensive analysis offers thorough coverage of the theoretical foundations and concepts that define AIG, as well as the practical considerations required to produce and apply large numbers of useful test items.

✦ Table of Contents

Cover
Half Title
Title Page
Copyright Page
Table of Contents
Preface
A Word of Thanks
Chapter 1: Introduction: The Changing Context of Educational Testing
The Problem of Scaling Item Development
Automatic Item Generation: An Augmented Intelligence Approach to Item Development
Benefits of Using AIG for Item Development
Purpose of This Book
References
Section 1:
Basic Concepts
Required for
Generating
Constructed- and
Selected-Response
Items
Chapter 2: Cognitive Model Development: Cognitive Models and Item Generation
Benefits of Using Cognitive Models For AIG
Developing Cognitive Models for AIG
A Word of Caution When Creating Cognitive Models
Two Types of Cognitive Models for AIG
Logical Structures Cognitive Model
Key Features Cognitive Model
References
Chapter 3: Item Model Development: Template-Based AIG Using Item Modelling
Layers in Item Models
Item Generation With 1-Layer Models
n-Layer Item Models
Item Generation With n-Layer Models
Two Important Insights Involving Cognitive and Item Modelling in AIG
Non-template AIG: A Review of the State of the Art
Is It Preferable to Use a Template for AIG?
Note
References
Chapter 4: Item Generation: Approaches for Generating Test Items
The Importance of Constraint Coding
Logical Constraint Coding Using Bitmasking
Demonstration of Item Assembly Using the Logical Constraints Approach
Logical Structures Cognitive Model
Key Features Cognitive Model
Item Assembly Using the Logical Constraints Approach
References
Chapter 5: Distractor Generation: The Importance of the Selected-Response Item in Educational Testing
The Contribution of Distractors in the Selected-Response Item Format
Traditional Approach for Writing Distractors
Methods for Distractor Generation
Distractor Generation With Rationales
Distractor Pool Method With Random Selection
Systematic Distractor Generation
Note
References
Chapter 6: Putting It All Together to Generate Test Items: Overview
Mathematics Example Using the Logical Structures Model
Cognitive Model Development
Item Model Development
Item Generation Using Constraint Coding
Systematic Distractor Generation
A Sample of Generated Math Items
Medical Example Using Key Features
Cognitive Model Development
Item Model Development
Item Generation Using Constraint Coding
Systematic Distractor Generation
A Sample of Generated Medical Items
Chapter 7: Methods for Validating Generated Items: A Focus on Model-Level Outcomes
Substantive Methods for Evaluating AIG Models
Cognitive and Item Model Review Using a Validation Table
Distractor Model Review Using a Validation Table
Substantive Model Review Using a Rating Scale
Substantive Methods for Evaluating AIG Items
AIG versus Traditional Item Review: Item Quality
AIG versus Traditional Item Review: Predictive Accuracy
Statistical Methods for Evaluating AIG Items
Statistical Analyses of the Correct Option
Statistical Analyses of the Incorrect Options
Cosine Similarity Index (CSI)
The Key to Validating Generated Items
References
Section 2:
Advanced Topics in
AIG
Chapter 8: Content Coding: Challenges Inherent to Managing Generated Items in a Bank
Managing Generated Items With Metadata
Content Coding for Item Generation
Assembling Content Codes in Item Generation
Content Coding Examples
Logical Structures Mathematics Model
Key Features Medical Model
References
Chapter 9: Generating Alternative Item Types Using Auxiliary Information: Expanding the Expression of Generated Items
Generating Items With Symbols
Generating Items With Images
Generating Items With Shapes
Challenges With Generating Items Using Auxiliary Information
References
Chapter 10: Rationale Generation: Creating Rationales as Part of the Generation Process
Methods for Generating Rationales
Correct Option
Correct Option With Rationale
Correct Option With Distractor Rationale
A Cautionary Note on Generating Solutions and Rationales
Benefits and Drawbacks of Rationale Generation
References
Chapter 11: Multilingual Item Generation: Beyond Monolingual Item Development
Challenges With Writing Items in Different Languages
Methods for Generating Multilingual Test Items
Language-Dependent Item Modelling
Successive-Language Item Modelling
Simultaneous-Language Item Modelling
Example of Multilingual Item Generation
Validation of Generated Multilingual Test Items
References
Chapter 12: Conclusions and Future Directions
Is AIG an Art or Science?
Is It “Automatic” or “Automated” Item Generation?
How Do We Define the Word “Item” in AIG?
How Do You Generate Items?
What Is an Item Model?
How Do You Ensure That the Generated Items Are Diverse?
How Should Generated Items Be Scored?
How Do You Organize Large Numbers of Generated Items?
What Does the Future Hold for Item Development?
References
Author Index
Subject Index