are typically generated manually by human beings, they We present a two-pass image retrieval system in which re-provide compact, important [8], though sometimes biased trieval techniques for text and image documents are combined and incomplete, descriptions of the visual content. Such in a novel approach. In the first pass, the text-based initial text-based systems can therefore leverage on matured text query is matched against the text captions of the images in the retrieval techniques to produce satisfactory results. database to obtain the initial retrieved set. In the second pass, Nevertheless, it has been reported [9] that users querying text and image features obtained from this initial retrieved set an image collection tend to be much more specific in their are used to expand the initial query. Additional images from requests and information needs, than when querying a the database are then retrieved based on the expanded query. text database. Moreover, most text annotations tend to The image features that we have used are color histograms, DC coefficients from the discrete cosine transform, and two be short. These imply that a simple term-by-term match texture features: multiresolution simultaneous autoregressive between query terms and annotations terms may not be model and local binary pattern. These are low-level statistical effective in some applications. Several solutions such as image features that can be easily computed. Extensive experisemantic distances [4] and concept spaces [7] have been ments have been performed on 1019 color pictures of mixed proposed to tackle this problem. variety with captions, relevance judgments and queries supplied In contrast, image retrieval systems that are based purely by a national archives agency. Objective precision-recall results on image features are still in the preliminary stages of have been obtained with various combinations of text and image development. As image processing and computer vision features. The results show that the image features do not perform well when used on their own. However, when image featechniques that provide domain independent recognition tures are used in query expansion, they increase the average of image content are still in their infancy, these systems precision more significantly than text annotations. Moreover, rely mainly on low-level image features. These include these findings are valid at all precision levels and are not sensicolor, composition, texture, structure [10-12], shape [13], tive to the image feature acquisition parameters. Β© 1998 Acamotion parameters [14], Gabor wavelets [15], and DCT demic Press coefficients [16].
Being poor descriptors of perceptual information and semantic similarity, these low-level features have exhibited