CRANV2 Methods for presentation of results chapter Cyril Cleverdon Michael Keen Cranfield An investigation supported by a grant to Aslib by the National Science Foundation. Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. - 33 - majority of cases, the most straightforward and most meaningful method is the Recall/Precision curve. These are, in general, the measures used in this report, although to illustrate certain points various other measures and methods of presentation are used. A detailed account of the Cranfield work on performance measures has been presented in a thesis by M. Keen, but the following is a resum6 of the more important points which led to the decisions; other matters relevant to the presentation of results in this volume are also considered. In tests of experimental systems, it is essential that measures should be used that accurately reflect the changes in the particular component being tested, which primarily, in this particular test, was a range of index language devices. In addition, there is the strong desirability, if not the absolute necessity, that it should be possible to make direct comparison between different sets of test results. Measures of retrieval performance may be used in experimental tests of information retrieval systems when the following requirements are met: - 1. 2. A document collection of known size to be used in the test; A set of questions, together with decisions as to exactly which documents are relevant to each question; 3. A set of results of searches made in the test; these usually give the numbers of documents retrieved in the searches, divided into the relevant and non-relevant documents. The successive dichotomies of the total collection have been displayed by B.C, Vickery (Ref. [OCRerr], page 174) by the following table:- TOTAL COLLECTION RELEVANT NON-RELEVANT NOT RETRIEVED RETRIEVED NOT RETRIEVED (c) (a) (b) (d) The more usual way to present the categories is in the form of a 2 x 2 contingency table as shown in Fig. 3:2. The notation given in this figure will be used throughout the remainder of this report.