IRS13 Evaluation Parameters chapter E. M. Keen Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. 11-57 non-relevant compared with the specific requests. But also at each cutoff, the recall ratios favor the specific requests, and both fallout and precision ratios favor the general requests. The recall merit is explained by noting that the smaller number of total relevant in the specific requests means that each one found "counts'1 for more in recall, since one relevant found in the specific requests increases recall by .016, and one relevant found in the general requests increases recall by .008. The fallout and precision merit is clearly affected by the higher concentration of relevant documents that is found in the general requests. It is not clear that fallout is free from the effects of generality in this sense, and therefore it is not certain that the fallout versus recall plot truly reflects system effectiveness when a gen- erality change of this type is encountered. Also, since recall is here af- fected by the difference in request generality, it is not certain that re- call accurately reflects user satisfaction, although it obviously does measure what the user examines. This last difficulty arises because it is not really clear just how the positions should be weighed when specific and general requests are compared. Six cases for comparisQn are given in Fig. 31: if some rational hand ranking of the merit of these six requests is not possible, then no satisfactory per- formance measur[OCRerr] to compare specific and general requests can be derived. One obvious solution is to recognize formally what has often been stated, namely, that users1 needs differ considerably, and the two ends of the spectrum may be represented by the high recall need and the high precision need. For example, if the high precision need is defined to mean that the best precision should be obtained in the process of finding just two relevant documents only, then the cases A, D, and F, in Fig. 31 are superior to B, E, and C. Also, if the