IRE Laboratory tests: automatic systems chapter Robert N. Oddy Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. Model building 171 this is a less common activity, the terms `theory' and `model' are used almost interchangeably in the information retrieval field. A proper discussion of the differences between the usages of these words in science would be extensive and out of place here54' 55. I should like to use the term `model' to refer to a particular type of theory: one that attempts explanation of a phenomenon by describing aprocess and exploring its effects. This gives us an approach to the problem of relating the outputs to the inputs of our systems which is distinct from mathematical argument. Recently, mathematical theories have been used to suggest retrieval algorithms16:I would not call such an algorithm a emodel[OCRerr] while it is still entirely derivative. One might ask why we should wish to build models without a sound mathematical theory. In information retrieval, we are trying to reproduce automatically a cognitive act: the decision as to whether a document, or `item' of information, is relevant. It is notoriously difficult to formulate mathematical theories which account for, and enable us to handle the `)ariability in human behaviour56' [OCRerr] According to Farrell50, many psychologists believe that outputs cannot be related to inputs in any simple (mathematical) way, and that internal states must be taken into account. This is best done by modelling. If we are looking for retrieval systems which are more responsive and adaptable to the individual user we could do worse than model the behaviour of good human information providers subject experts, librarians and information scientists. Following a `systems' approach, one can formulate a high-level structural model of their behaviour, illustrated by a block diagram showing the relationships between components, for instance, and experi- mentally determine whether the human behaviour fits the model (see, for example, Olney58, Ingwersen and Kaae59, Bivins60 and, for methodology, the chapter by Keen (8)). Alternatively, one can attempt to build a computer program with the hypothetical structure, run it, and observe its behaviour. In the latter approach, the modeller must elaborate the meaning of all his high- level components in a very precise way. Before making more general points, I should like to illustrate the process, briefly, from my own experience, by explaining how certain essential parts of a particular program, called Thomas61' 62, were written. The program is an interactive system which provides a browsing facility for the user. He is not required to formulate a query, and as the dialogue progresses, his reactions to the indexed references are used by the program to build a picture of his area of concern. At the highest level, a `cognitive' model of dialogue, essentially like Hollnagel's63, is assumed. Each participant has his own image of the world, which includes an image of the other's world-image: let me call this included image a `meta-image'. Communication becomes more effective as these meta-images more accurately portray the current concerns of the participants. To implement this model in a man-machine dialogue, the computer program must have a fund of knowledge about the world, and a means of representing its image of the man. It must be able to improve that meta-image as the dialogue progresses, that is, in response to the man's utterances, and in displaying information to him it must aim not only to give him relevant references, but also to help him form his image of the program's world-image. (We are beginning to see some guidelines for a program design.) Thomas' world-image is a graph in which the nodes represent documents, subject terms and authors, and the arcs associations between them, derived