IRE Simulation, and simulation experiments chapter Michael D. Heine Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. 180 Simulation, and simulation experiments features of just one concept, both justifying our provisional use of the one term and encouraging us to relate it to information systems. First, the notion of `optimization' provides one general feature. One does not vary either the representatives of the components of a system or the sequence of operations in a system, capriciously. One does so in order to identify a system that is `best' according to some criterion. In that a notion of social utility is so implied, simulation could be seen as differing from the classical sciences- which are simply predictive systems of thought-and positioned rather more closely to technology. Yet it differs from the latter too in that [OCRerr]udgement' is always a component of simulation in practice a second general feature. This consists, at the least, in the specification of a criterion or of criteria by which optimization is to be judged; and will also consist in the delimitation of the `system' to be studied and the identification of the system's essential components. If simulation could both delimit an area for analysis as well as be the analysis, this would involve attributing an `intelligence' to it. Obviously it is the presence of human judgement together with some rationale for analysing phenomena that embodies this intelligence. (It would be `an error of judgement' to omit to identify a variable that sensitively affected another variable by which optimality was to be judged.) This raises intriguing questions where the subject of the simulation is human decision-making, for example `relevance-judging' or `query-forming' in information retrieval. Here one is (in simulating same) making judgements about judgements. And since a formulation of a query (say) is in part a judgement about how an indexer will have chosen index terms, i.e. again a judgement about a judgement, we have a third tier of judgement. It seems that a recursive definition of simulation is needed to clarify things here, but to the writer's knowledge this has not yet been attempted. A third general feature of simulation is the admission, into the system description, of the notions of `input' and `output'. These entities, usually seen as separated in time, may be real artefacts (e.g. documents, money) or information (a message on a VDU) or decisions (`Get file XYZ'), to choose information retrieval instances. Lastly, and really in continuation of the first feature above, we note that one's interest in simulating systems is usually prompted by the possibility of intervention in the system: or even total control of it. Without an interventionist ethos we would be back in the realm of classical theoretical science. In the face of this complexity, partly of the making of those who have written on simulation, the temptation is to wield an Occam's razor in the form of just one clear prescriptive definition of `simulation'. In the writer's view however this would be a mistake in the particular context in which information retrieval experimentation finds itself: a context which is commonly denoted `information science' but which might better be called `the philosophy of information'. For the ambiguities of simulation in this context are indicative of the ambiguities that we need to overcome if information science is to develop. The characterization of `information', `variable', `representation', `system' as part of the human cognitive apparatus, and the attaching of abstract meanings to concepts such as `observable' or `objective viewpoint' or `explaining power' or `language' are just the kinds of things we need to do in order to develop information science. So that taking the broadest definition of information science we could say, more rhetorically, I