IRE Opportunities for testing with online systems chapter Elizabeth D. Barraclough Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. Types of tests 133 requested. Similarly, word adjacency can be utilized as well as other context indicators. Index search The purpose of this part of the system is to provide for the user a count of the number of references which satisfy his formulated request. The system also creates a record of the citation numbers for subsequent retrieval from the database. The user has no control over this part of the system, but he may, of course, return to the formulation stage if he is not satisfied with the number of references retrieved. Database search and print The function here is merely to extract references from the database and print them in a form requested by the user. Most systems offer a wide range of alternative styles of printing from the brief reference giving only sufficient bibliographic details to identify the citation to a full reference giving all the data, including abstract or index terms, held for the citation. The user may select the style, whether to print locally on his terminal or elsewhere on a fast printer and how many retrieved references to print. Re has no control over the selection of the retrieved citations to be printed, generally the systems provide the most recent references first, but nothing is guaranteed. The user at any stage can return to amend his search in the light of the information supplied by the system. It is this interaction which should make online systems much more powerful than the original batch systems. Most systems provide access to more than one database, with the system's functions used in the same way on every database available to the system. However, it is not possible to ignore the database being used when assessing the facilities provided by the system functions. For example, the method by which index terms were assigned can make a vast difference to the efficacy of the systems. The user can thus apply his search to more than one database in the system and may retrieve the same reference from several databases as there is often considerable overlap between databases. What is potentially worrying is the retrieval of a reference from only one database when it is known to be covered by another database, the different indexing or abstracting methods being generally responsible for this. 7.8 Types of tests Search effectiveness This type of test, as Lancaster has pointed out in the previous chapter, is a macroevaluation; the system is being treated as a `black box' and the evaluation is concerned only with the results the system produces not with the methods used to reach them. The criteria for evaluation of effectiveness depend upon the type of search being attempted. Possible user needs were previously identified, each of which would need a different set of parameters in the evaluation.