We propose a novel method for automatically adjusting in documents, parameters of the similarity metric, reparameters in ranked-output text retrieval systems to trieval thresholds, etc. One goal in the design of the reimprove retrieval performance. A ranked-output text retrieval system is to set and adjust these parameters to tune trieval system implements a ranking function which orthe system for better retrieval performance. For example, ders documents, placing documents estimated to be in ranked-output retrieval systems, the system implements more relevant to the user's query before less relevant ones. The system adjusts its parameters to maximize a ranking function which orders documents based on their the match between the system's document ordering and estimated relevance to the user's query. Adjusting system a target ordering. The target ordering is typically given parameters typically results in alternative orderings made by user feedback on a set of sample queries, but is more by the system. The goal, then, is to adjust the parameters generally any document preference relation. We demonstrate the utility of the approach by using it to estimate so that, as much as possible over a range of possible a similarity measure (scoring the relevance of docuqueries, the ordering of the documents given by the sysments to queries) in a vector space model of information tem corresponds well to the actual ordering of the docuretrieval. Experimental results using several collections ment's relevance to the user's need. This can be an exindicate that the approach automatically finds a similartremely difficult task, both because of the large number of ity measure which performs equivalently to or better than all ''classic'' similarity measures studied.