Estimating the Query Difficulty for Information Retrieval (Synthesis Lectures on Information Concepts, Retrieval, and Services Ser.)

✍ Scribed by David Carmel, Elad Yom-Tov

• Year: 2010
• Tongue: English
• Leaves: 90
• Edition: 1
• Category: Library

✦ Synopsis

Many information retrieval (IR) systems suffer from a radical variance in performance when responding to users' queries. Even for systems that succeed very well on average, the quality of results returned for some of the queries is poor. Thus, it is desirable that IR systems will be able to identify "difficult" queries so they can be handled properly. Understanding why some queries are inherently more difficult than others is essential for IR, and a good answer to this important question will help search engines to reduce the variance in performance, hence better servicing their customer needs. Estimating the query difficulty is an attempt to quantify the quality of search results retrieved for a query from a given collection of documents. This book discusses the reasons that cause search engines to fail for some of the queries, and then reviews recent approaches for estimating query difficulty in the IR field. It then describes a common methodology for evaluating the prediction quality of those estimators, and experiments with some of the predictors applied by various IR methods over several TREC benchmarks. Finally, it discusses potential applications that can utilize query difficulty estimators by handling each query individually and selectively, based upon its estimated difficulty. Table of Contents: Introduction - The Robustness Problem of Information Retrieval / Basic Concepts / Query Performance Prediction Methods / Pre-Retrieval Prediction Methods / Post-Retrieval Prediction Methods / Combining Predictors / A General Model for Query Difficulty / Applications of Query Difficulty Estimation / Summary and Conclusions

✦ Table of Contents

Acknowledgments......Page 12
Introduction - The Robustness Problem of Information Retrieval......Page 14
Reasons for retrieval failures - the RIA workshop......Page 16
Instability in retrieval - the TREC's Robust tracks......Page 17
Estimating the query difficulty......Page 19
Summary......Page 20
The retrieval task......Page 22
Linear correlation......Page 23
Prediction robustness......Page 25
Summary......Page 26
Query Performance Prediction Methods......Page 28
Linguistic approaches......Page 30
Definitions......Page 32
Specificity......Page 33
Coherency......Page 34
Evaluating pre-retrieval methods......Page 35
Summary......Page 37
Post-Retrieval Prediction Methods......Page 38
Definition......Page 39
Examples......Page 40
Other Clarity measures......Page 41
Query perturbation......Page 42
Document perturbation......Page 43
Cohesion......Page 44
Score distribution analysis......Page 45
Evaluating post-retrieval methods......Page 48
Summary......Page 49
Combining pre-retrieval predictors......Page 52
Combining post-retrieval predictors......Page 53
Combining predictors based on statistical decision theory......Page 54
Evaluating the UEF framework......Page 55
Results......Page 56
Combining predictors in the UEF model......Page 57
Summary......Page 58
Geometrical illustration......Page 60
General model......Page 61
Validating the general model......Page 63
The relationship between aspect coverage and query difficulty......Page 64
Validating the relationship between aspect coverage and query difficulty......Page 65
Summary......Page 66
Feedback: To the user and to the system......Page 68
Federation and metasearch......Page 69
Content enhancement using missing content analysis......Page 71
Selective query expansion......Page 73
Adaptive use of pseudo relevance feedback......Page 74
Other uses of query difficulty prediction......Page 75
Summary......Page 76
Summary......Page 78
What next?......Page 80
Concluding remarks......Page 81
Bibliography......Page 82
Authors' Biographies......Page 90