IRE Evaluation within the enviornment of an operating information service chapter F. Wilfrid Lancaster Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. Some problems of evaluation applied to operating Systems 125 participate in the network. Information centres participating in an international programme, such as INIS or AGRIS, may be particularly prone to compromise. They may have to accept a thesaurus that is not ideal I;[)r their own needs and may even be required to work in English as a carrier language when they would much prefer to use Spanish or Portuguese. Another problem in the real world is the sheer inertia of large organizations or those that have been operating for many years. A librarian may recognize the desirability of converting from one classification scheme to another but the 3 million volumes in the collection already are, to say the least, a discouragement to change. Likewise for changes in a large card catalogue. A change in cataloguing policy or practice is not only difficult to implement, but it will be many years before, applied to the catalogue as a whole, the change will have any significant effect. Another barrier is simply that of the cost of change. Change may be inexpensive in the experimental system but the cost of certain changes may be prohibitive in an operating service. The experimenter in information retrieval is in an unusually fortunate position in that he/she will frequently have control of all the factors that affect the performance of the experimental system. Not so in the real world. It is rare for an information centre to have control over all the factors affecting the service it provides to its users. Consider, for example, an industrial library in the United States that is conducting online literature searches for its users (Figure 6.2) through some online service centre such as Lockheed. It can be seen from the diagram that the industrial information centre has direct control of only a few of the factors that determine the effectiveness of the services it provides. It has control over the way in which it interacts with its own users. To a very large extent it has the ability to influence the quality of interaction through a redesigned search request form, through improvements in the training of the information staff[OCRerr]for example, in the conduct of request interviews-or through insistence on a greater level of user involvement in the search process itself[OCRerr]for example, asking the user to be present at the terminal at the time the search is conducted. But the information centre does not have absolute control over even this stage of the information retrieval process, because it does not have complete control over its own users. The staff of the information centre can go only so far in leading and helping users. If a particular individual, for one reason or another, is completely unable to verbalize his information need, there may be very little the information specialist can do to help him. Similarly, the local centre may be said to have control over its own search strategies. It does, of course, but this control is not absolute either. The characteristics of a search strategy are very largely determined by the characteristics of the software used to search online. The individual searcher must operate within the constraints of this software. For example, the extent to which he can use truncation in searching is obviously completely controlled by the truncation capabilities of the query language. Obviously, too, the search strategy must make use of the vocabulary of the database, and it must adapt to the indexing policies in use in the database, and these important factors affecting performance are under the control of only the database producer. In fact, then, a typical information centre has control over only a few of the factors affecting the quality of the information services it provides. This