Using dwell time as an implicit measure of usefulness in different task types

✍ Scribed by Chang Liu; Jingjing Liu; Nicholas Belkin; Michael Cole; Jacek Gwizdka

• Publisher: Wiley (John Wiley & Sons)
• Year: 2011
• Tongue: English
• Weight: 1002 KB
• Volume: 48
• Category: Article
• ISSN: 0044-7870
• DOI: 10.1002/meet.2011.14504801200

✦ Synopsis

In this study, we attempted to use dwell time on content pages as a predictor of document usefulness, and evaluated the prediction performance in different types of tasks. A user study was conducted to address this research problem. A total of 32 participants conducted searches associated with 4 different tasks, varying along several task type dimensions. In the study, participants were asked to save documents that were useful for the tasks during the searches. Participants' information-seeking activities were recorded. Using the logged dataset, we used the recursive partitioning method to identify the best threshold (cutoff point) of dwell time to predict which pages would be useful, i.e., be saved by the users. We examined the differences in the threshold and compared the prediction performance for each type of task. Our results indicate that the threshold should be developed according to the type of task users are working on. While the prediction performance using dwell time as the only indicator of document usefulness is acceptable for Factual tasks, it is not good for Intellectual tasks. Our results have implications on IR system design.

Keywords

Dwell

time, usefulness prediction, task type, personalization 1 Although they are different in application scope as described by Cole et al. (2010), both terms are IR system evaluation criteria. They are used interchangeably in this paper, especially in literature review, according to how the cited work used them.