IRE Retrieval effectiveness chapter Cornelis J. van Rijsbergen Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. References 43 potential performance of different models, and (3) to study the effects of %.`rnaII changes to any given model. Of course comparing upper bounds can be iiiisleading for the following reasons. It may well be that upper bound A exceeds upper bound B significantly; however, it may not only be that it is hot possible to design a reasonable estimation rule for A: it could be that the model for A is so complicated that we can not get sufficient data to estimate (S parameters. 3.6 Conclusions l[OCRerr]i this chapter I have tried to show the considerable interplay that exists I)ctween probabilistic definitions of effectiveness and certain models for ILtrieval. In the past researchers, including myself, have tried to argue the icI[OCRerr]itive merits of different measures of effectiveness independent of how these might influence the design of retrieval systems. I now think that the host important consequence of defining a measure of retrieval effectiveness iii 1 particular way is the ability it gives us to make theoretical statements ihout certain models. These statements can then be tested empirically .ig[OCRerr]iinst stimulated or real users. Rcferences ROBERTSON, S. F. The probability ranking principle in IR. Journol of I)o(uoi('Oto!1()n 33. 394 t()4 (1977) SWETS, J. A. Information retrieval systems. Sc&iuc, .\T y, 141. 245 250 (1963) VAN RIJSBERGEN, C. J. In/ormotion Retriciol, 2nd edn, Butterworths, London (1979) I hARPER, D. J. Relcionee Feedho[OCRerr]'k in DO('unhnl Retriei[OCRerr]o/ Sv[OCRerr]teni[OCRerr]. Ph.D. The[OCRerr]is, Computer I iboratory, University of Cambridge (1980) ROBERTSON, S. F. and SPARCK JONES, K. Relevance weighting of search terms, Journol of the `Imerican Soe&tj' A)r JnA)rniation Science 27.129[OCRerr] 146 (1976) (`OOPER, w. S. The suboptimality of retrieval ranking based on the probability of usefulness (private communication to S. F. Robertson) (1977) VAN RJJSBERGEN, C. J. Retrieval Effectiveness. In: Progress in (`ommunicotion Sciences. (r.d. M. J. Voigt and G. J. Hanneman), Vol.1, pp.91 118, Ablex Publishing Corporation. Ne[OCRerr] York (1979) S SPARCK JONES, K. Performance averaging for recall and precision, Journolof InA)rnioti(s 2,95 105(1978) `) IDE, F. Releconce FCe(/ho('k in on Autoniotic I)o([OCRerr]unient Retriei[OCRerr]ol Sisteni, Master's Thesis, Report ISR- 15. Department of Computer Science. ([OCRerr]orneII Luiversity (1969)