IRE The Smart environment for retrieval system evaluation-advantages and problem areas chapter Gerard Salton Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. 328 The Smart environment for retrieval System evaluation ence, inner product similarity function, etc.), one may expect that the system operations are representative of a much wider area outside the formal limits. After more than ten years of experimentation with Smart, the main problem is not apparently the fact that Smart may have been `doing the wrong kind of chemistry', but rather the fact that laboratory-type chemistry is not quite the same as production work in a chemical factory; which does not imply of course that laboratory work is useless or unnecessary. References LUHN, H. P. The automatic derivation of information retrieval encodements for machine- readable texts. In: Information Retrieval and Machine Translation, (Ed. A. Kent), Part 2, Interscience, New York (1961) 2. LUHN, H. P. A statistical approach to mechanized encoding and searching of literary information, IBM Journal of Research and Development 1, 309-317 (1957) 3. SWANSON, D. R. Searching natural language text by computer, Science, Ne[OCRerr] York 132, 1099- 1104(1960) 4. MONTGOMERY, C. and SWANSON, D. R. Machine-like indexing by people, American Documentation 13, 359 366 (1962) 5. SALTON, G. Manipulation of trees in information retrieval, Communications of the ACM 5, 103-114(1962) 6. SALTON, G. Dj'namic Inji)rmation and Librar[OCRerr]' Processing, Chapter 6, Prentice-Hall, Englewood Cliffs, N.J. (1975) 7 ATHERTON, P. A Proposed Standard Description for Reporting Evaluation Tests of Retrieval St'.[OCRerr]ten's, presented at Seventh Institute on Information Storage and Retrieval, American University, Washington, D.C. (1965) [OCRerr]. SPARCK JONES, K. and BATES, R. 6. Report on a Design Studyfor the Ideal Information Retrieval Test Collcction, British Library Research and Development Report 5428, Computer Laboratory, University of Cambridge (1977) 9. FAIRTHORNE, R. A. Basic parameters of retrieval tests, Proceedings of the American Documentotion Institute 1, 343 346, American Documentation Institute, Washington, D.C. (1964) 10. SALTON, 6. The evaluation of automatic retrieval procedures selected test results using the Smart system, American Documentation 16, 209-222 (1965) II. SALTON, 6. and LESK, M. E. The Smart automatic document retrieval system an illustration, (`()moiuni(.ations ofthe ACM 8, 391-398 (1965) 12. SALTON, 6. Designing automatic information systems: results obtained with the Smart programs, Social Science In/i)rmation 6, 11 1-117 (1967) 13. CLEVERt'oN, C. W. and KEEN, E. M. Factors Determining the Performance of Indexing Systems, lol. 2 --Test Results, Aslib Crantield Research Project, College of Aeronautics, Cranfield, England (1966) 14. BOURRELLY, L. and CROURAQUI, v. Le S.i'steme Documentaire Satin 1, Vol. 1, Description Ge'ne'rah' et Manuel d'Utilisotion, Centre National de Ia Recherche Scientifique, Paris (1974) IS. SALTON, 6. and LESK, M. E. Computer evaluation of indexing and text processing, Journal of tIIeACMI5,8 36(l96[OCRerr]) 16. IDE, E. and SALTON, 6. User controlled file organization and search strategies, Proceedings of thc ASIS Notional (`onierenee, Vol.6, pp.183-191, Greenwood Publishing Corp., Westport, Connecticut (1969) I 7 SALTON, 6. Search Strategy and the Optimization of Retrieval Eftectiveness, Mechanized In/n rotation Storage, Retriei'al and Dissemination, (Ed. K. Samuelson), pp.73-107, North- Holland. Amsterdam (1968) I [OCRerr]. SALTON, 6. Experiments in multilingual information retrieval, Information Processing Letters 2,6-11(1973) 19 SALTON, 6. Associative document retrieval techniques using bibliographic information, Jour,uil o/ tie' ACM 10, 440[OCRerr]457 (1963) 20 SALTON, 6 Recent studies in automatic text analysis and document retrieval, Journal ofthe ACM 20,258-278(1973)