IRE Laboratory tests of manual systems chapter E. Michael Keen Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. Controlling searching in experiments 145 50 13x x16 ii[OCRerr][OCRerr];[OCRerr]x/15 12 18 x 19 () 2C I- C 0 ic U 0 20 40 60 80 100 120 140 160 180 200 4) Index IQnguage specJticity Qutput of srngle term search 4) 40 4) I 3C F Figure 8.4[OCRerr] Cranfield 2 data re-presented from Figure 16-J6 in Keen and Digger9. x single terms; 0 simple concepts; I] controlled terms variables of many values and combinations if correlations with performance could be established, so that systems having intermediate values could just be read off a suitable characteristics/performance plot. 8.4 Controlling searching In experiments The kinds of experimental control needed to measure the variables in manual testing are illustrated best by the stage of searching. This stage encounters most of the problems posed at other stages and provides the severest test of an experiment. Affer considering the general methods of setting up the search stage of an experiment, six topics will be sampled to give the flavour of the problems and some of the current solutions. Conducting experiments In an experimental situation there is the need to decide what constitutes a search and who is to perform it. In a laboratory one cannot have a real user conducting an unconstrained actual search at the time an information need arises. So the usual practice is either to obtain previously used search requests, or manufacture realistic ones, and have people other than the requesters do the searches. The searches themselves have often to be conducted in a closely prescribed manner, adopting particular procedures, recording the results in some way, and timing the process. What can never take place is a discussion of the meaning of the search request with its originator, and it is the inviolate written request that has to constitute the search query. Thus there can be no warrant to stray from the stated request, no radical alteration of it, if only because associated document relevance