Reconciling (seemingly) discrepant findings: Implications for practice and future research

✍ Scribed by Nicholas A. Bowman; Serge Herzog

• Publisher: John Wiley and Sons
• Year: 2011
• Weight: 50 KB
• Volume: 2011
• Category: Article
• ISSN: 0271-0579
• DOI: 10.1002/ir.393

✦ Synopsis

Decades of research in survey methodology and psychology have yielded important insights about how to create effective and valid survey instruments. As Porter (in press) has argued convincingly, college student surveys often fall well short of these standards by placing unrealistic demands on students' memory and by assuming that students readily understand complex, vague terminology. Given strong reliance on college student surveys for research and assessment purposes (Gonyea, 2005), surprisingly few studies directly examined the validity of student self-report data until very recently. This volume of New Directions for Institutional Research provides insights into these issues through seven empirical studies of the validity and limitations of college student self-report data, with a primary focus on self-reported gains. Most of this research demonstrates limitations and concerns with prevalent interpretations of self-report data, but some findings lend support to such use in certain circumstances.

Here, we review some of the key fi ndings from these chapters. In many edited books and monographs, this sort of summary addresses the chapters in the exact order in which they appear. In this particular volume, we feel it is preferable to synthesize these fi ndings and contributions into a framework that illustrates the consistency among seemingly inconsistent results. For instance, Chapter Three (Pike) in some ways critiques Chapter Two (Herzog), and perhaps Chapter One (Bowman). However, we believe that all of the results are "correct" in an important sense of the This concluding chapter synthesizes the empirical chapters and suggests implications for practice and future research.