CRANV2 Methods for presentation of results chapter Cyril Cleverdon Michael Keen Cranfield An investigation supported by a grant to Aslib by the National Science Foundation. Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. - 31 CHAPTER 3 Methods for Presentation of Results Lord Kelvin is often credited with remarking, [OCRerr]When you can measure what you are speaking about and express it in numbers you know something about it, but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind'. The problem of validity - how closely do the figures relate to the 'thing we are talking about' must be separated from the problem of reliability - how accurate are the figures themselves. L.T. Wilkins: Social Deviance, page 147 In Cranfield I, the results of the main project did no more than record what is now generally known as the Recall Ratio, which was 100R calculated on the basis of [OCRerr] where R equals the number of relevant documents retrieved and C equals the total number of documents in the collection which are relevant to the questions. In the subsequent test of the Western Reserve University Index, (Ref. 2) measurement was carried to the stage where, by making relevance assessments of all the retrieved documents, it was also possible to calculate what was originally called the Relevance Ratio, but which is now generally known 100R as the Precision Ratio, namely [OCRerr] , where L equals the total number of documents retrieved in the series of searches. In the course of this evaluation of the W.R.U. Index, the effect of varying the exhaustivity of indexing was measured, and allowed the production of the first - and, incidentally, so far the only - performance curve from the Cranfield project. It is reproduced in Fig. 3.1P and showed two interesting characteristics. The first was the inverse relationship between recall and precision, which has been considered at some length in Volume 1 of this report. The second point was that, when documents of lower relevance, were accepted, there was at any given level of indexing exhaustivity, a lower recall ratio but an improved precision ratio. It was tentatively suggested that this latter point was connected with a variation in the average number of relevant documents for each question, and that, for any.given situation, it would be necessary to state also what was to be later termed the 1000C Generality Number$, expressing it as [OCRerr] , where N equals the total number of documents in the collection. Sin the earlier volume of this report, this was called Generality Ratio.