IRE The Cranfield tests chapter Karen Sparck Jones Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. Cranfield 1 259 one, focusing on indexing languages, and seeking to identify and control other system factors bearing on the study variable. The indexing systems chosen for test were `(a) The Universal Decimal Classification. (b) Alphabetical subject catalogue. (c) A facetted classification scheme. (d) The Uniterm system of coordinate indexing.' (p.8) According to the Report, `the basis for this selection was that the schemes differed as fundamentally as is possible and represented the principal types of retrieval systems which have any significance in the present state of the art.' (p.8) The UDC was chosen as the most widely used system of the `enumerative' type, illustrating tree-of-knowledge classification and the use of a decimal notation. Facet classification specifically lacks these three features. The alphabetical index is deliberately anti-classificatory and relatively uncon- trolled in language, while Uniterms are equally uncontrolled, but allow all permutations and combinations of individual terms to define subjects. The facet and Uniterm languages both represented novel approaches to indexing, the former being particularly carefully prepared. With respect to the other main test variables, the document subject area, aerodynamics, was determined by convenience, while the set of documents used was chosen ad hoc, to allow variability in detailed document topics, types, and sources. The indexers were chosen to have various types of experience and familiarity with the systems being tested. (It may be noted that `the project imposed a severe mental strain on the indexers' (p.2).) Equipment, i.e. the physical form of indexes, was not studied as a test variable. The size of the document set for the indexes was prima[OCRerr]ily based on the wish to ensure that retrieval was not too obvious, without being wasteful; more specifically, with 60 permutations of the three major variables, obtained as explained below, and taking 100 documents as a convenient number for each permutation, there were 3 `subprogrammes' of 6000 documents for the total of 18 000. In fact, since half the documents were in a specialized subject area, the collection was deemed representative of a much larger set. Overall, `the assumption on which the investigation [was] based [was] that the only valid way to measure the efficiency of any system of indexing is by basing measurements on economic costs and in this there are always three matters to be considered, these being: (a) The cost of indexing. (b) The cost of preparing the physical index. (c) The cost of searching.' (p.18) So on the indexing side, since equipment questions were deliberately excluded, the major variables being investigated were the system, the indexer, and the indexing time, i.e. 4 systems x 3 indexers x 5 times = 60 permutations. The procedures adopted in the indexing are described in detail, in relation