IRE The pragmatics of information retrieval experimentation chapter Jean M. Tague Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. 102 The pragmatics of information retrieval experimentation 9. LESK, M. E. and SALTON, G. Relevance assessments and retrieval system evaluation, Information Storage and Retrieval 4, 343-359 (1968) 10. REES, A. M. and SCHULTZ, D. G. A Field Experimental Approach to Relevance Assessments in Relation to Document Searching, 2 Vols, Centre for Documentation and Communication Research, Case Western Reserve University (1967) 11. MILLER, G. A. The Psychology of Communication, Basic Books, New York (1967) 12. GILBERT, H. and SPARCK JONES, K. Statistical Bases of Relevance Assessment for the `Jdeal' Information Retrieval Test Collection, British Library Research and Development Report 5481, Computer Laboratory, University of Cambridge (1979) 13. COOPER, M. D. Input-output relationships in online bibliographic searching, Journal of the American Society for Information Science 28, 153-156(1977) 14. SPARCK JONES, K. and VAN RIJSBERGEN, C. J. Information retrieval test collections, Journal of Documentation 32, 59-75 (1976) 15. TAGUE, J. and FARRADANE, 1. Estimation and reliability of retrieval effectiveness measures, Information Processing and Management 14, 1-16 (1978) 16. HARTIGAN, J. A. Clustering Algorithms, Wiley, New York (1975) 17. SALTON, G. Dynamic Library and Information Processing, Prentice-Hall, Englewood Cliffs, N.J. (1975) 18. WINER, B. J. Statistical Principles in Experimental Design, 2nd edn, McGraw-Hill, New York (1971) 19. FISHER, R. A. and YATES, F. Statistical Tables for Biological, Agricultural, and Medical Research, Oliver and Boyd, Edinburgh (1953) 20. KERLINGER, F. N. Foundations of Behavioral Research, Holt, Rinehart and Winston, New York (1964) 21. 5PARCK JONES, K. Performance averaging for recall and precision, Journal of Informatics 2, 95-105 (1978) 22. MENDENHALL, W. w. et al. Statistics: a Tool for the Social Sciences, Duxbury Press, North Scituate, Mass. (1974) 23. WINKLER, R. L. and HAYES, W. L. Statistics: Probability, Infrrence, and Decision-Making, 2nd edn, Rolt, Rinehart and Winston, New York (1975) 24. KENDALL, M. G. and STUART, A. The Advanced Theory of Statistics, Vols 1-3, Hafner, New York (1961-1966) 25. NOETHER, G.E. Elements of Nonparametric Statistics, Wiley, New York (1967) 26. GILCIrRIST, W. Statistical Forecasting, Wiley, New York (1976) 27. BOX, G.E. P. and JENKINS, G. W. Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco (1976) 28. MOORE, P. W. Introduction to the Use of Computer Packagesfor Statistkal Analysis, Prentice- hall, Englewood Cliffs, N.J. (1978) I