IRE Ineffable concepts in information retrieval chapter Nicholas J. Belkin Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. 56 Ineffable concepts in information retrieval 4.6 Conclusion This chapter has not tried to provide a list or grab-bag of techniques whicli could be used to investigate the `ineffable' concepts of information retrieval1 Rather, the aim has been to indicate the ways in which these concepts ar[OCRerr] important to the information retrieval situation, some reasons for theji peculiar intractability, and some reasons for why they need to be considered in the testing of information retrieval systems. Perhaps the discussion and examples have indicated some special problems that these concepts pose: this chapter has been written in the hope that these problems will be taken seriously into account in the design and conduct of future informatioix retrieval system testing. There are a number of specific conclusions which I think can be drawn from this discussion of ineffable concepts in information retrieval. The first is that studies of these variables per se are very much needed, in order to provide a sounder basis for evaluation measures of information retrieval systems, as well as to provide sounder design principles for information retrieval systems. It also seems that evaluation of information retrieval systems must now begin to take more explicit account of the nature of these concepts, especially the user-related group, as problems of evaluation of online systems become more acute. And finally, in order to become explanatory and predictive, rather than merely descriptive, information retrieval system evaluation should change its emphasis (as, indeed, it is already beginning to do). Testing or evaluation has largely been on the basis of describing and comparing the results of system performance, without a great deal of emphasis on the theory underlying the systems. But if information retrieval systems design is to progress in a meaningful sense, then we need theories which allow us to explain why one system works better than another. Except in what seem to be relatively minor ways, this cannot be achieved without taking account of at least the variables discussed in this chapter. To that extent, it seems to me that the future of information retrieval system testing and design lies necessarily in the investigation of these concepts and their application. 4.7 Acknowledgement Much of this work was accomplished while I was on a visiting appointment at the School of Library and Information Science, University of Western Ontario. I would like to thank the School and my colleagues and students there for their help (which they may not recognize). References 1. BELKIN, N.J. and ROBERTSON, S. E. Information science and the phenomenon of information, Journal of the American Society for Information Science 27, 197-204 (1976) 2. ROBERTSON, S. E. Indexing theory and retrieval effectiveness, Drexel Library Quarterly 14, 40-56 (1979) 3. PAISLEY, w. j. and PARKER, E. B. Information retrieval as a receiver-controlled communication system. In: Educationfor Information Science (Fd. L. P. Heilprin, B. E. Markuson and F. L. Goodman), pp.23-31, Macmillan, London (1965) I I I Li II I I t I, i I I--- i i -`I 41 i'-[OCRerr]i