IRS13 Evaluation Parameters chapter E. M. Keen Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. 11-50 viewpoint result in a precision-recall graph, since an equation to adjust precision for generality is given in (2], namely, Adjusted Precision Ratio = RlxG (R x G) + F (1000 - G) 1 1 where R = Recall ratio at a given cut-off point 1 F Fallout ratio at a given cut-off point 1 G = Generality number (1000 x total relevant) to which collection size it is desired to alter the results. Thus, in Fig. 28(c), the ADI recall and fallout ratios are recorded as R1 and F for a series of cut-off points and G is set to 2316, in order to 1 adjust the generality of ADI to fit the generality of Cran-l. The adjusted precision versus recall curve is given in Fig. 28(c). It should be noted that the precision for ADI does not now represent a user-oriented evaluation, but has been artificially adjusted to give a system oriented evaluation. A series of tables appears in (2] in which the fallout values for ranges of recall and precision values have been computed, for a range of generality numbers, primarily to permit quick calculation of adjusted precision ratios. Some comparisons involving changes in generality are given in section I and Appendix A, and further comparisons using the Cran-l and larger Cran-2 collections will require performance measures of this type. It should be emphasized, however, that the ordinary precision-recall curve still gives a vali[OCRerr] and useful user-oriented result, and it is in experimental test com- parisons only that the two viewpoints for evaluation (Fig. 2) give different and complementary results. The normalized evaluation measures appear to reflect a system-oriented result since the equations both contain "N", the total number of documents in the collection. For example, the normalized