IRE An experiment: search strategy variations in SDI profiles chapter Lynn Evans Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. 286 An experiment: search strategy variations in SDI profiles profile started too complicated and having to read a detailed User Manual, including grappling with the intricacies of nested boolean logic, was a positive discouragement. At that time INSPEC profiles were almost exclusively of the boolean type. The general assumption was that full advantage of the machine facility should be taken and profiles compiled with complex logic to match the user's statement of information requirements. However if the initial user-system interaction was to be eased then simpler search strategies was one area where this might be achieved. There was some evidence of a non-quantitative nature which suggested that, even in profiles incorporating sophisticated logic, many of the relevant matches obtained were derived from compara- tively simple parts of the logic statement. In addition to doubts about the need for sophisticated logic, work at Cranfield in the field of precision engineering4 showed that the case for any boolean logic was not proved and that a straightforward co-ordination of all the profile terms might be equally or even more efficient. Quite separately it was also felt that the issue of simpler search strategies was to become important in the development of online retrospective search systems. For these to be operated directly by the people requiring the information rather than by intermediary information scientists and librarians then the man-machine interaction needed to be much simpler than was currently the case. In the event, some 5 years later, online searching of bibliographic databases is still very much the prerogative of intermediaries and is likely to remain so until the development of truly interactive retrieval software. This then was the rationale leading to the particular experiment described and analysed in this paper, viz. search strategy variations in SDI profiles5. The overall aim was to develop the most cost-effective search strategy by studying the cost, retrieval performance, and ease of use of a number of different search strategy types. In the general framework of information retrieval experiment this work was most like a laboratory test but perhaps lacking the rigid control necessary for a `true' laboratory experiment. Although carried out in the environment of an operational system it was in no way an investigation of the operational system. It is probably most accurately categorized as a developmental project, i.e. one pursued with as much experimental rigour as possible but not at the expense of losing touch with `real-world' conditions. 14.2 Experiment Very broadly a conventional evaluation in information retrieval requires: (1) a collection of documents with various attributes (titles, abstracts, assigned index headings, etc.); (2) a set of queries whose subject area is covered by the document collection; and (3) knowledge of which documents in the collection are relevant/non-relevant to particular queries. In addition, in this experi- ment, the important considerations were the search strategies, profile compilation procedures, and search software.