Inferring accuracy for judges and items: Choice of unit of analysis reverses the conclusions
✍ Scribed by Yaacov Schul; Ilan Yaniv
- Publisher
- John Wiley and Sons
- Year
- 1997
- Tongue
- English
- Weight
- 169 KB
- Volume
- 10
- Category
- Article
- ISSN
- 0894-3257
No coin nor oath required. For personal study only.
✦ Synopsis
This study explores the relationship between the precision and the accuracy of forecasts using either judge or item as the unit of analysis. Participants in ®ve experiments answered general-knowledge questions by indicating intervals that were likely to include the correct answer. Results indicate that the precision of an interval estimate is not a straightforward cue to the likelihood that such an interval includes the truth (hit rate). Whereas judges who state more precise estimates (i.e. who provide narrower interval estimates) have lower hit rates, questions for which the average judgment is more precise have higher hit rates. Thus, the relation between precision and accuracy depends on whether one `slices' the data by judge or by question. We oer an explanation for this seemingly paradoxical eect and implement it as a computer simulation to demonstrate its validity.