Improved Test–retest Reliability Criteria as Applied to Probability Assessments
✍ Scribed by SHAWN P. CURLEY; JAMES I. GOLDEN
- Publisher
- John Wiley and Sons
- Year
- 1996
- Tongue
- English
- Weight
- 390 KB
- Volume
- 9
- Category
- Article
- ISSN
- 0894-3257
No coin nor oath required. For personal study only.
✦ Synopsis
Test-retest reliability is a common indicator of response consistency. It is argued that using regression coefficients for detecting systematic response error is less appropriate than testing for shifts in the mean (median) and variance. This procedure is exemplified using probability response data. For this data, shifts in centrality were found to be about 2.5 times more likely than shifts in variability. The shifts in centrality did not favor any particular direction; however, variability tended to decrease over time in early sessions.
KEY WORDS reliability; probability assessment
Reliability refers to response consistency. Reliability is a necessary condition for validity, i.e. for a measure to be meaningful. It is for this reason that the reliability of probabilities as a measure for individuals' degrees of belief has been of interest (Whitcomb et al., 1993). Although this methodological note is specifically addressed at the reliability of probability measures, the content applies more widely to other reliability situations.