Cover
Half Title
Series Information
Title Page
Copyright Page
Table of Contents
Contributors
Acknowledgements
Introduction: Theory, Research, Reflection, and Action: What Is this Book About?
Setting the Stage: Let the Play Begin
Intended Readership: Actors and Audiences
Overall Organization: Acts, Scenes, and Stage Directions
Part I – Building Foundations for Transdisciplinary Dialogue: New Directions in Assessment Research
Part II – Transdisciplinary Research (TR) in Practice: Building Connections
Part III – Transdisciplinarity in Practice: Moving the Research Agenda Forward
The Curtain Rises: Understanding Resistance to Transdisciplinarity and the Challenges Ahead
Notes
Part I Building Foundations for Transdisciplinary Dialogue: New Directions in Language Assessment Research
1 The Problem of Context in Language Assessment: Validity, Social Theories, and a Transdisciplinary Research (TR) Agenda
Context, Orientations, and Communities
Orientations
Social Theories and Cognitive Theories
Assessment
Communities
What’s in a Name?
Differing Perspectives On Context
Background
Question One: What Is the Problem With Context in Assessment Validation Practices?
Question Two: What Is the Potential of a Transdisciplinary Research (TR) Agenda in Addressing the Problem of Context in Assessment Validation?
Defining Transdisciplinarity
Transdisciplinarity in Practice
Question Three: What Role Do Disciplines Play in Constraining and Limiting Innovation?
Moving the Research Agenda Forward
Notes
2 Validity as an Evolving Rhetorical Art: Context and Consequence in Assessment
The Evolution of Validity Theory: Validation as a Social Practice and Rhetorical Art
Past Foundations: From Shibboleth Tests to Correlation Coefficients
Innovative Recognitions: From Construct Validity (Cronbach & Meehl, 1955) to Contextualism (Cf. Cronbach, 1988)
The Game-Changer: Messick’s (1989) Unified View of Validity (Philosophical Conceits and the Progressive Matrix)
Key Features of Messick’s (1989) Definition of Validity
Revolution and Evolution: Validation in Turbulent Times
The Methodological Turn: Expanding Philosophical Breadth (1990–2010)
Current Trends in Validation (2011–2021)
Disagreements Over Consequences and Context: Scope and Focus in Validation Research
So, What Is the Way Forward?
Renewed Arguments for Disciplinarity: Addressing the “Regrettable Consequences” (Cizek, 2020, P. Xiv) of Messick’s Writings
Advancing a Transdisciplinary Agenda: Fulfilling the Promise of Messick’s Theory
Concluding Comments
Notes
3 Unpacking the Conundrum of Context in Language Assessment: Why Do Social Theories Matter?
Why Do Theories Matter?
How Have Dualisms Contributed to the Problem of Context in Language Assessment?
What Can We Learn From Changing Conceptions of Language and Context?
What Have Social Theories Contributed to Conceptions of Language?
What General Persuasions Do Social Theories Share?
What About Cognitive Persuasions and Their Contributions?
Selected Social Theories: Prominence and Usefulness in Considerations of Context
A Selective Account of Some Antecedent Social Theorists and Theories
Bakhtin and Language as Communication
Vygotsky and the Sociocultural Turn in Psychology
The Social Construction of Reality: Schutz, Berger, and Luckmann
A Selective Review of Some Socio-Theoretical Perspectives
Rhetorical Genre Studies
Situated Learning
Communities of Practice
Cultural-Historical Activity Theory (CHAT)
Distributed Cognition (Hutchins, 1995a, 1995b)
English as a Lingua Franca (ELF)
Theories of Intercultural Communication
Theory of Interactional Competence
What Evidence Is There of the Contributions of Social Theories to Language-Centred Communities?
What Are the Additive Benefits of Social Theories in Assessment Research?
Conclusion
Notes
4 The Contributions of Language Assessment Research: Evolving Considerations of Context, Constructs, and Validity
Why and How Has the Challenging Disciplinary Position of Language Assessment Impeded Progress in Addressing the Complex Issue of Context?
What Role Have Social Theories Played in the Assessment Research Literature? A Meta-Review of Four Prominent Assessment-Focused Research Journals (2004–2020)
Inclusion/exclusion Criteria and the Meta-Review Process
An Overview of the Key Findings of the Meta-Review
Confronting the Problem of Context in Language Assessment: What Is the Construct? What Is the Role of Context?
Contributions of the Language Assessment Community
Examples of Research Aligned With Assessment-Centred, Individualist, Cognitive Perspectives
Fairness
Bias
Localization
Examples of Research Aligned With Language-Centred, Socio-Theoretical Perspectives: the Social Thread
Fairness and Bias
The Discursive Construction of Testing, Tests, and Test Takers
Washback and Impact
Critical Language Testing
The Role of Social Theories: Alternative Considerations of the Consequences of Test Interpretation and Use
Social Theories in Teaching and Learning Languages: Classroom-Based Assessment
Socially Informed Considerations of Justice, Power, and Policy in Assessment
The Social Thread in Language Assessment: Requirements for Transdisciplinary Practices
Recognizing and Addressing Applicability Gaps
Recognizing and Developing Theoretical Understanding
Looking Ahead to Part II
Notes
Part II Transdisciplinary Research (TR) in Practice: Building Connections
5 Clarifying the Testing of Aural/oral Proficiency in an Aviation Workplace Context: Social Theories and Transdisciplinary Partnerships in Test Development and Validation
An Overview of this Chapter’s Organization and Purpose By Section: Some ‘heads-Up’ Advice
Background On LSP Test Development
Construct and Context in Language for Specific Purposes (LSP) Testing
LSP Test Development: a Transdisciplinary Approach Informed By Policy, Selected Social Theories, Empirical Research, and Stakeholders’ Engagement
What Is the Context?
What Is the Construct (In the Context of ICAO Policy, Standards, and Requirements)? The Role of Mandate and Purpose in LSP Test Development
What Is the Construct (In the Context of Radiotelephony Communication)? The Affordances of Alternative Social Theories and Empirical Research in LSP Test Development
Distributed Cognition
English as a Lingua Franca (ELF)
Intercultural Communication and Intercultural Awareness (ICA)
Interactional Competence (IC)
Building Theoretical Models
Construct Specification
The Contributions of Stakeholders in Transdisciplinary Test Development Spaces: Validation
Matrix Validation
Conclusion
Notes
6 Validation of a Rating Scale in a Post-Admission Diagnostic Assessment: A Rhetorical Genre Studies Perspective
Introduction
Background
Rhetorical Genre Studies
Background On the Development of a Diagnostic Assessment
Unpacking a Diagnostic Assessment Writing Task Through the RGS Lens
Defining and Reinforcing Key RGS Notions: the Diagnostic Writing Task and Analytic Rating Scale
An Example of RGS-Informed, Evidence-Driven Validation of a Diagnostic Rating Scale
Summary of Analysis
Diagnosis
Summary of Analysis
Diagnosis
Summary of Analysis
Diagnosis
From RGS Analysis to the Rating Scale, Rater Training, and Pedagogical Support
Conclusion and Implications
Note
7 Social Theories and Transdisciplinarity: Reflections On the Learning Potential of Three Technologically Mediated Learning Spaces
Putting Space in Context: Defining Terms and Situating the Discussion
Viewing Learning Spaces Through Lenses Afforded By Theory and Empirical Research
Social Theories of Learning: a Focus On Affordances
What Are the Affordances of Online Contexts for Assessment?
Transdisciplinarity: a Requirement for Teaching in Online Learning Spaces
Example 1: 3DVLE
Transdisciplinary Partnerships
Detailed Description of the 3DVLE
Affordances
Reconsidered Teaching Practice
Reconsidered Assessment Practice
Example 2: EPortfolio
Transdisciplinary Partnerships
Detailed Description of the EPortfolio Learning Space
Affordances
Reconsidered Teaching Practice
Reconsidered Assessment Practice
Example 3: Open-Source Content Creator (H5P)
Transdisciplinary Partnerships
Detailed Description of the H5P Tool
Affordances
Reconsidered Teaching Practice
Reconsidered Assessment Practice
Online Learning Spaces in Changing Teaching and Learning Contexts: a Pandemic Postscript
A Teacher’s Reflection: Perils and Possibilities of Online Teaching and Learning Contexts in a Pandemic World (Narrated By Peggy Hartwick)
Hartwick’s Reflection On “How Do I Redesign and Best Accommodate My Learners and Support Learning in a Fully Online Learning Course During a Pandemic?”
Afterthoughts: What Other Social Theories Might Have Offered
Acknowledgements
Notes
Part III Transdisciplinarity in Practice: Moving the Research Agenda Forward
8 Language Assessment in the Wild: Storying Our Transdisciplinary Experiences
Taking a Closer Look: Transdisciplinary Research (TR) Practices-In-Process
Naming as Rhetorical Action: Reconsidering Community
Beneficial Interactions of Social Theories and Transdisciplinary Practices-In-Process
Hutchins’ (1995a, 1995b) Theory of Distributed Cognition
Contested Views of Wenger’s (1998) Communities of Practice (CoPs)
Wenger’s (1998) Dimensions of a Community of Practice (CoP)
Understanding Transdisciplinary Practice-As-Process Through Narratives of Assessment Design, Development, and Implementation
Three Narratives of Transdisciplinary Assessment Experiences
Context One. What Is the Construct? Law, Policy, and Power in Transdisciplinary Assessment (Narrated By Janna Fox)
Postscript
Context Two. Shifting Partnerships in a Transdisciplinary Project (Narrated By Janna Fox)
Context Three. Transdisciplinarity at Its Best (Narrated By Janna Fox and Natasha Artemeva)
Project Initiation
Ongoing Project Development, 2011–2019
What Can We Learn About Transdisciplinarity From These Three Narratives of Experience?
Toward a Socially Informed, Transdisciplinary Dialogue On Context and Validity
Notes
References
Index